Collaborative surgical system with automated pre-configured robotic arm configurations

Abstract

Described herein is a collaborative robotic system that can be used to assist in surgical procedures, including laparoscopic surgery. The collaborative robotic system allows surgeons to use commercially available surgical tools while providing the benefits associated with surgical robotics. Advantageously, surgical tools can be seamlessly coupled to the robotic arm using a disposable coupler, while the reusable portion of the robotic arm remains in a sterile drape. Furthermore, the collaborative robotic system can operate in multiple modes, improving usability and safety, while allowing the surgeon to position instruments directly with the instrument handle and then maintain the desired position of the instrument using the robotic arm.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is a continuation of U.S. patent application Ser. No. 18 / 540,710 filed December 14, 2023, U.S. patent application Ser. No. 18 / 535,991 filed December 11, 2023, U.S. patent application Ser. No. 18 / 480,360 filed October 3, 2023, U.S. patent application Ser. No. 18 / 331,073 filed June 7, 2023 (now U.S. Patent No. 11,839,442), U.S. patent application Ser. No. 18 / 331,070 filed June 7, 2023 (now U.S. Patent No. 11,832,910), U.S. patent application Ser. No. 18 / 331,064 filed June 7, 2023 (now U.S. Patent No. 11,832,910), and U.S. patent application Ser. No. 18 / 331,064 filed June 7, 2023 (now U.S. Patent No. 11,832,910). No. 18 / 331,060 (now U.S. Pat. No. 11,819,302), filed June 7, 2023; U.S. Patent Application No. 18 / 331,054 (now U.S. Pat. No. 11,844,583), filed June 7, 2023; U.S. Provisional Patent Application No. 63 / 495,527, filed April 11, 2023; U.S. Provisional Patent Application No. 63 / 479,142, filed January 9, 2023; and European Patent Application No. 23305026.9, filed January 9, 2023, the contents of each of which are incorporated herein by reference in their entirety.

[0002] The present technology relates to collaborative robotic systems, such as those designed to couple to a clinician-selected surgical instrument and enable movement of the robotic arm via movement at the handle of the surgical instrument, with improved features for setup and automatic intraoperative movement. [Background technology]

[0003] Managing visibility and access during laparoscopic procedures is a challenge. Surgical assistant frameworks are inherently imperfect because the assistant is required to anticipate and understand the surgeon's perspective without standing where the surgeon stands, and similarly anticipate and adjust the degree to which the surgeon desires tissue of interest to be exposed throughout the procedure. For example, during a laparoscopic procedure, one assistant may be required to hold a retractor device and expose tissue for the surgeon, while another assistant may be required to hold a laparoscopic device and provide the surgeon with a view of the surgical space within the patient during the procedure, one of whom may be required to hold their respective tools in an impractical position, for example, between the surgeon's arms while the surgeon actively operates additional surgical instruments.

[0004] Various attempts have been made at solving this problem. For example, a rail-mounted orthopedic retractor, which is a purely mechanical device mounted to the patient bed / table, can be used to hold laparoscopic devices in place during laparoscopic procedures, and another rail-mounted orthopedic retractor can be used to hold retractor devices in place during laparoscopic procedures. However, rail-mounted orthopedic retractors require extensive manual interaction to unlock, reposition, and lock the tools in place.

[0005] Complex robotic-assisted systems, such as the Da Vinci Surgical System (marketed by Intuitive Surgical, Sunnyvale, California), are being used by surgeons to enhance laparoscopic surgical procedures by allowing them to remotely perform the procedure from a surgeon console remote from the patient console where the surgeon holds the surgical instruments. Such complex robotic-assisted systems are very expensive and have a very large footprint, occupying a lot of space in the operating room. Furthermore, such robotic-assisted systems typically require unique, system-specific surgical instruments to be compatible with the system; therefore, surgeons may not be able to use the standard, off-the-shelf surgical instruments they are accustomed to. Therefore, surgeons are required to learn how to perform an entirely different laparoscopic procedure.

[0006] In light of the aforementioned shortcomings of previously known systems and methods, there is a need for a system that provides a surgeon with the ability to seamlessly position and manipulate various surgical instruments as needed, thus avoiding the workflow limitations inherent in both human and mechanical solutions. Summary of the Invention [Means for solving the problem]

[0007] The present disclosure overcomes the shortcomings of previously known systems and methods by providing a co-manipulated surgical system for assisting in laparoscopic surgery performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween. The co-manipulated surgical system may include a robotic arm having a proximal end, a plurality of links removably coupled to the surgical instrument, a distal end, and a plurality of joints between the proximal and distal ends. The co-manipulated surgical system may further include a controller operably coupled to the robotic arm. The controller may be programmed to cause the robotic arm to automatically switch between a passive mode in which, in response to determining that movement of the robotic arm due to movement at the surgical instrument handle is less than a predetermined amount for at least a predetermined dwell time, the controller can be programmed to cause the robotic arm to maintain a stationary position in a passive mode, and a collaborative manipulation mode in which, in response to determining that a force applied to the robotic arm due to a force applied to the surgical instrument handle exceeds a predetermined threshold, the controller can be programmed to allow the robotic arm to be freely movable using the surgical instrument to perform a laparoscopic surgical procedure in response to movement at the surgical instrument handle in the collaborative manipulation mode, wherein the controller can be programmed to apply a first impedance to the robotic arm and account for the weight of the surgical instrument and the robotic arm in the collaborative manipulation mode. The controller may be further programmed to cause the robotic arm to automatically switch to a haptic mode in response to determining that at least a portion of the robotic arm is outside a predefined haptic barrier, and in the haptic mode, the controller may be programmed to apply a second impedance to the robotic arm that is greater than the first impedance, thereby causing movement of the robotic arm in the haptic mode to be more tenacious in response to movement in the handle of the surgical instrument than in the collaborative manipulation mode.

[0008] According to one aspect of the present disclosure, a cooperatively manipulated surgical system is provided for assisting in laparoscopic surgery performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween. The cooperatively manipulated surgical system may include a robotic arm having a proximal end, a distal end configured to be removably coupled to the surgical instrument, a plurality of links and a plurality of joints between the proximal and distal ends, and a controller operably coupled to the robotic arm and configured to enable the robotic arm to be freely movable in response to movements in the handle of the surgical instrument to perform the laparoscopic surgery. The controller is programmed to: cause the robotic arm to maintain a stationary position in a passive mode in response to determining that movement of the robotic arm due to movement at the handle of the surgical instrument is less than a predetermined amount for at least a predetermined dwell time; and, when the surgical instrument comprises a laparoscope having a field of view, identify a target surgical instrument within the field of view of the laparoscope based on image data from the laparoscope; and switch the robotic arm to an instrument centering mode in which the robotic arm moves the laparoscope to maintain the target surgical instrument within the field of view of the laparoscope.

[0009] The controller may be configured to automatically switch the robotic arm to a collaborative manipulation mode in response to determining that a force applied to the robotic arm due to a force applied to the surgical instrument handle exceeds a predetermined threshold. Thus, the controller may be configured to apply an impedance to the robotic arm in the collaborative manipulation mode, allowing the robotic arm to be freely movable in response to movements at the surgical instrument handle while accounting for the weight of the surgical instrument and the robotic arm. Additionally, the controller may be configured to identify a target surgical instrument within the laparoscopic field of view by detecting a predefined gesture pattern by the target surgical instrument within the laparoscopic field of view. The predefined gesture pattern may comprise positioning the target surgical instrument within a central portion of the laparoscopic field of view and maintaining the position of the target surgical instrument within the central portion for at least a predetermined holding period. In some embodiments, the controller may be configured to identify a target surgical instrument within the laparoscopic field of view based on user input identifying the target surgical instrument. Additionally, the controller may be configured to identify the target surgical instrument within the laparoscope's field of view based on a direction in which the target surgical instrument enters the laparoscope's field of view, the direction matching a predefined direction associated with a known handedness of a user manipulating the target surgical instrument. For example, the known handedness of a user may be stored in a user profile associated with the user, and the controller may be configured to execute the user profile. In some embodiments, the controller may be configured to identify the target surgical instrument within the laparoscope's field of view by detecting a motion pattern of the target surgical instrument's trajectory within the laparoscope's field of view.

[0010] Further, the controller may be configured to distinguish the target surgical instrument from one or more other surgical instruments within the field of view of the laparoscope. In an instrument centering mode, the controller may cause the robotic arm to move the laparoscope and maintain the target surgical instrument within a predefined boundary area within the field of view of the laparoscope, such that the robotic arm does not move the laparoscope unless the target surgical instrument moves outside the predefined boundary area. In some embodiments, the controller may be configured to determine whether the motion pattern of the target surgical instrument is small motion, such that in the instrument centering mode, the controller may cause the robotic arm to move the laparoscope and not maintain the target surgical instrument within the predefined boundary area if the motion pattern of the target surgical instrument is small motion. For example, the controller may be configured to detect the speed or acceleration of the tracked surgical instrument based on the image data, such that the controller may determine that the motion pattern of the target surgical instrument is small motion if the tracked surgical instrument moves outside the predefined boundary area at a speed or acceleration that exceeds a predetermined speed or acceleration threshold. Additionally or alternatively, the controller may determine that the motion pattern of the target surgical instrument is small motion if the tracked surgical instrument moves outside the predefined boundary area but remains within the laparoscope's field of view for less than a predetermined period of time before returning within the predefined boundary area.

[0011] Further, in the instrument centering mode, the controller may execute a trajectory generation algorithm to cause the robotic arm to generate a trajectory from the current position of the laparoscope to a desired position of the laparoscope, and cause the robotic arm to move the laparoscope by moving the laparoscope along the trajectory to maintain the target surgical instrument within the field of view of the laparoscope. Thus, the controller may be configured to apply an impedance to the robotic arm in the collaborative manipulation mode, and, in response to determining that a force applied to the robotic arm due to a force applied to the laparoscope exceeds a predetermined threshold while taking into account the weights of the laparoscope and the robotic arm, to: allow the robotic arm to be freely movable in the collaborative manipulation mode; record a trajectory of the freely moving robotic arm when the movement of the robotic arm deviates from the generated trajectory; and update the trajectory generation algorithm based on the recorded trajectory. The generated trajectory may include moving the robotic arm along a longitudinal axis of the laparoscope to maintain the target surgical instrument within the field of view of the laparoscope and within a predetermined resolution threshold. Additionally, the generated trajectory may include moving the robotic arm along at least one of the longitudinal axis of the laparoscope or an axis perpendicular to the longitudinal axis of the laparoscope to maintain the target surgical instrument within the field of view of the laparoscope.

[0012] The trajectory can be generated by measuring a current position of the distal end of the robotic arm, determining an entry point of the laparoscope into the patient, and calculating a distance needed to move the distal end of the robotic arm from its current position to a second position that moves the distal end of the laparoscope from its current position to a desired position based on the entry point and a known length between the distal end of the robotic arm and the distal end of the laparoscope. The controller can move the laparoscope along the trajectory by calculating a force needed to move the distal end of the robotic arm the distance from its current position to the second position, and applying torques to at least some of the joints of the robotic arm based on the calculated force to move the distal end of the robotic arm the distance from its current position to the second position, thereby moving the distal end of the laparoscope from its current position to the desired position. Additionally, the controller may be configured to detect an offset angle between the laparoscope camera head and the laparoscope, and to calibrate a trajectory to correct for the offset angle such that movement of the laparoscope along the calibrated trajectory maintains the target surgical instrument within the laparoscope's field of view. For example, the controller may be configured to detect the offset angle by moving the robotic arm along a predetermined trajectory in a known direction within the robotic arm coordinate frame, measuring actual movement of a static object within the laparoscope's field of view in response to movement of the robotic arm along the predetermined trajectory, and comparing the actual movement of the static object to an expected movement of the static object associated with the predetermined trajectory.

[0013] The controller may be further configured to cause the robotic arm to switch to an instrument centering mode in response to a user input. In some embodiments, the system may further include an actuator operably coupled to the controller and disposed on a link of the plurality of links of the robotic arm, the actuator configured to receive a user input and be actuated to transmit one or more signals indicative of the user input to the controller. Additionally, the controller may be configured to determine a phase of the laparoscopic surgical procedure, estimate a target surgical instrument based on the phase of the laparoscopic surgical procedure, and identify the target surgical instrument within the field of view of the laparoscope based on the estimation and image data from the laparoscope. Furthermore, the controller may be configured to determine a phase of the laparoscopic surgical procedure and automatically switch to the instrument centering mode in response to the phase of the laparoscopic surgical procedure. Thus, the controller may be configured to identify one or more anatomical structures within the field of view of the laparoscope based on image data from the laparoscope, determine a phase of the laparoscopic surgery based on the identified one or more anatomical structures, and cause the robotic arm to move the laparoscope in an instrument centering mode to maintain the identified one or more anatomical structures within the field of view of the laparoscope. Additionally, the controller may be configured to generate an overlay indicating the target surgical instrument and cause the overlay to be displayed over the image data from the laparoscope via the graphical user interface.

[0014] The controller may be configured to cause the robotic arm to move the laparoscope along a predetermined trajectory, compare an actual trajectory of image data from the laparoscope during movement along the predetermined trajectory with an expected trajectory of image data associated with the predetermined trajectory, and determine the angle of the distal tip of the laparoscope. For example, the predetermined trajectory may include a circular pattern in a single plane. Furthermore, the controller may be configured to identify a target surgical instrument within the field of view of the laparoscope based on the image data from the laparoscope using a machine learning algorithm executed in the controller. For example, the machine learning algorithm may be trained on a database of image data annotated with associated surgical instruments. Thus, the machine learning algorithm may be configured to evaluate pixels of the image data from the laparoscope, indicate whether the pixel corresponds to a target surgical instrument, and identify the target surgical instrument. The controller may be configured to identify the target surgical instrument within the field of view of the laparoscope in real time. In an instrument centering mode, the controller may be configured to cause the robotic arm to move the laparoscope and track a target surgical instrument manually held by a user, e.g., a surgeon. In some embodiments, the system may include a second robotic arm configured to be removably coupled to a target surgical instrument manually held by the surgeon.

[0015] The controller may be further configured to determine, based on the image data, a size of the tracked surgical instrument relative to the field of view of the laparoscope, and to cause the robotic arm, in an instrument centering mode, to move the laparoscope at a speed or acceleration based on the size of the tracked surgical instrument relative to the field of view of the laparoscope. Additionally, the controller may be configured to determine a length of the laparoscope within the patient's body, and to cause the robotic arm, in an instrument centering mode, to move the laparoscope at a speed or acceleration based on the length of the laparoscope within the patient's body. For example, the controller may be configured to determine the length of the laparoscope within the patient's body based on a known length of the laparoscope and a position of the distal end of the robotic arm relative to a trocar placed on the patient's body through which the laparoscope is inserted.

[0016] According to another aspect of the present disclosure, a method for assisting in laparoscopic surgery is provided. The method may include providing a robotic arm having a proximal end, a distal end configured to be removably coupled to a laparoscope, a plurality of links, and a plurality of joints between the proximal and distal ends, enabling, via a controller operably coupled to the robotic arm, the robotic arm to be freely movable in response to movement at a handle of the laparoscope to perform laparoscopic surgery, automatically, via the controller, causing the robotic arm to maintain a stationary position in a passive mode in response to determining that movement of the robotic arm due to movement at the handle of the laparoscope is less than a predetermined amount for at least a predetermined dwell time, identifying, via the controller, a target surgical instrument within a field of view of the laparoscope based on image data from the laparoscope, switching, via the controller, the robotic arm to an instrument centering mode, and automatically, via the controller, causing the robotic arm to move the laparoscope and maintain the target surgical instrument within the field of view of the laparoscope while in the instrument centering mode. For example, identifying the target surgical instrument within the laparoscope's field of view may include detecting, via the controller, a predefined gesture pattern by the target surgical instrument within the laparoscope's field of view, the predefined gesture pattern comprising positioning the target surgical instrument within a central portion of the laparoscope's field of view and maintaining the position of the target surgical instrument within the central portion for at least a predetermined holding period.

[0017] According to another aspect of the present disclosure, a collaborative surgical system for assisting in surgical procedures, such as laparoscopic surgery, performed using a surgical instrument is provided. The collaborative surgical system may include a robotic arm having a plurality of links, a plurality of joints, a proximal end operably coupled to a base of the robotic arm, and a distal region having a distal end configured to be removably coupled to a surgical instrument, and a platform coupled to the base of the robotic arm. The platform may include a stage assembly configured to independently move the base of the robotic arm in at least two degrees of freedom relative to the platform. Thus, in a user-guided setup mode, application of a force at the distal region of the robotic arm in a first direction can cause the stage assembly to move the base of the robotic arm in a first degree of freedom relative to the platform.

[0018] For example, in the user-guided setup mode, the stage assembly may be configured to move the base of the robot arm in a first degree of freedom when a force applied to the distal region of the robot arm in a first direction exceeds a predetermined force threshold. Furthermore, in the user-guided setup mode, the stage assembly may be configured to stop movement of the base of the robot arm in the first degree of freedom when a force applied to the distal region of the robot arm in the first direction falls below a predetermined release threshold. Furthermore, in the user-guided setup mode, the stage assembly may be configured to stop movement of the base of the robot arm in the first degree of freedom upon application of an opposing force at the robot arm in a second direction opposite the first direction. Additionally, in the user-guided setup mode, application of a force at the distal region of the robot arm in a second direction may cause the stage assembly to move the base of the robot arm in a second of the at least two degrees of freedom relative to the platform. The system may further include an actuator configured to be actuated to switch the system to the user-guided setup mode. In some embodiments, the system remains in the user-guided setup mode only while the actuator is actuated. The actuator may be disposed on a collar rotatably coupled to a link of the plurality of links such that actuation of the actuator enables rotation of the collar in a first direction, causing a distal link of the plurality of links adjacent to a set joint of the plurality of joints to rotate in a corresponding first direction relative to a proximal link of the plurality of links adjacent to the set joint, and enables rotation of the collar in a second direction, causing a distal link adjacent to the set joint to rotate in a corresponding second direction relative to the proximal link adjacent to the set joint.

[0019] The system may further include a graphical user interface operably coupled to the stage assembly. The graphical user interface may be configured to display an actuator configured to be actuated to cause the stage assembly to move the base of the robotic arm in at least one of at least two degrees of freedom relative to the platform. For example, the actuator may include a slidable cursor configured to be moved relative to a neutral center point of the cursor pad such that movement of the slidable cursor in a direction relative to a neutral center point in the cursor pad can cause the stage assembly to move the base of the robotic arm in the corresponding direction relative to the platform. The stage assembly may be configured to move the base of the robotic arm in the corresponding direction relative to the platform at a speed that correlates with the distance of the slidable cursor from the neutral center point. Additionally, the graphical user interface may be configured to display one or more indicators that indicate the configuration of the robotic arm relative to the platform in real time in response to actuation of the actuator. Furthermore, in the collaborative manipulation mode, the robotic arm may be enabled to be freely movable in response to movement in a handle of a surgical instrument for performing a laparoscopic surgical procedure.

[0020] The system may further include a plurality of motors disposed within the base, the plurality of motors operably coupled to at least some of the plurality of joints, and a controller operably coupled to the plurality of motors. The controller may be programmed to measure currents in the plurality of motors, the measured currents indicative of forces applied to a distal region of the robot arm, and to cause the stage assembly to move the base of the robot arm in at least one of the at least two degrees of freedom based on the measured currents in a user-guided setup mode. The controller may further be operably coupled to a set joint of the plurality of joints of the robot arm such that the controller may be programmed to determine whether one or more objects are within a predetermined proximity threshold of the robot arm, and to automatically rotate a distal link of the plurality of links adjacent to the set joint relative to a proximal link of the plurality of links adjacent to the set joint to avoid collision with the one or more objects when the stage assembly moves the base of the robot arm in at least one of the at least two degrees of freedom relative to the platform in the user-guided setup mode.

[0021] The system may further include one or more depth sensors configured to detect one or more objects adjacent to the robotic arm and generate one or more signals indicative of the proximity of the one or more objects to the robotic arm. Thus, the controller may be configured to determine whether the one or more objects are within a predetermined proximity threshold of the robotic arm based on the one or more signals. For example, the one or more depth sensors may include one or more proximity sensors disposed within the base of the robotic arm, the one or more proximity sensors including at least one of an electromagnetic, capacitive, ultrasonic, or infrared proximity sensor. Additionally or alternatively, the one or more depth sensors may include one or more depth cameras. Thus, the controller may be configured to stop movement of the base of the robotic arm through the stage assembly when the one or more objects are within a predetermined proximity threshold. The collaborative surgical system may not be remotely controlled via user input received at a remote surgeon console.

[0022] According to another aspect of the present disclosure, a method for assisting in laparoscopic surgery is provided using a robotic arm including a plurality of links, a plurality of joints, a proximal end operably coupled to a base of the robotic arm, and a distal region having a distal end configured to be removably coupled to a surgical instrument. The method may include switching the system to a user-guided setup mode via a controller operably coupled to a stage assembly operably coupled to the base of the robotic arm, and causing, via the controller, the stage assembly in the user-guided setup mode to move the base of the robotic arm in a first degree of freedom of at least two degrees of freedom relative to a platform coupled to the stage assembly upon application of a force at the distal region of the robotic arm in a first direction. For example, causing the stage assembly to move the base of the robotic arm in the first degree of freedom may include causing, via the controller, the stage assembly in the user-guided setup mode to move the base of the robotic arm in the first degree of freedom when a force applied to the distal region of the robotic arm in the first direction exceeds a predetermined force threshold.

[0023] The method may further include causing, via the controller, the stage assembly in the user-guided setup mode to stop movement of the base of the robot arm in the first degree of freedom when a force applied to the distal region of the robot arm in a first direction falls below a predetermined release threshold. Additionally, the method may include causing, via the controller, the stage assembly in the user-guided setup mode to stop movement of the base of the robot arm in the first degree of freedom upon application of a counter force on the robot arm in a second direction opposite the first direction. Furthermore, the method may include causing, via the controller, the stage assembly in the user-guided setup mode to move the base of the robot arm in a second of the at least two degrees of freedom relative to the platform upon application of a force on the distal region of the robot arm in the second direction. Furthermore, switching the system to the user-guided setup mode may include switching the system to the user-guided setup mode in response to actuation of an actuator operably coupled to the controller, such that the system may remain in the user-guided setup mode only while the actuator is actuated.

[0024] The method may further include causing, via the controller, the stage assembly in a user-guided setup mode to move the base of the robot arm in at least one of the at least two degrees of freedom relative to the platform in response to actuation of the actuators displayed on a graphical user interface operably coupled to the controller. Thus, the method may further include causing, via the controller, the graphical user interface in the user-guided setup mode to display one or more indicators indicative of a configuration of the robot arm relative to the platform in real time in response to actuation of the actuators. The method may further include determining, via the controller, in the user-guided setup mode, whether one or more objects are within a predetermined proximity threshold of the robot arm, and, via the controller, stopping, via the stage assembly, movement of the base of the robot arm if the one or more objects are within the predetermined proximity threshold to avoid collision with the one or more objects when the stage assembly moves the base of the robot arm in at least one of the at least two degrees of freedom relative to the platform. Further, the method may include switching the system, via the controller, to a collaborative operation mode, and enabling, via the controller, in the collaborative operation mode, the robotic arm to be freely movable in response to movement at the handle of a surgical instrument to perform the laparoscopic surgery.

[0025] According to another aspect of the present disclosure, a collaborative surgical system for assisting in surgical procedures, such as laparoscopic surgery, performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween is provided. The collaborative surgical system may include a robotic arm including a plurality of links, a plurality of joints including one or more motorized joints, a setting joint, and one or more passive joints, a proximal end operably coupled to a base of the robotic arm, and a distal region having a distal end configured to be removably coupled to a surgical instrument, and a plurality of motors operably coupled to the one or more motorized joints and the setting joint. Additionally, the system may include an actuator operably coupled to the setting joint and configured to be actuated in response to actuation of the actuator to cause rotation of a distal link of the plurality of links adjacent to the setting joint relative to a proximal link of the plurality of links adjacent to the setting joint from a first setting configuration to a second setting configuration. Thus, when the actuator is in a non-actuated state, the robotic arm can be enabled to be freely movable in response to movements at the handle of a surgical instrument to perform a surgical procedure via one or more motorized joints and one or more passive joints, while the distal link adjacent the setting joint and the proximal link adjacent the setting joint remain in the second setting configuration.

[0026] The actuator may include a collar rotatably coupled to a link of the plurality of links, the collar configured to be rotated in a first direction relative to a link of the plurality of links to cause rotation of a distal link adjacent the set joint in a corresponding first direction relative to a proximal link adjacent the set joint and to be rotated in a second direction relative to a link of the plurality of links to cause rotation of the distal link adjacent the set joint in a corresponding second direction relative to the proximal link adjacent the set joint. Further, the collar may include a set mode actuator configured to be actuated in response to rotation of the collar to enable rotation of the distal link adjacent the set joint in the corresponding first and second directions relative to the proximal link adjacent the set joint. The set mode actuator may be configured to be actuated in a plurality of actuation patterns, each actuation pattern of the plurality of actuation patterns being associated with a unique user input configured to initiate a predetermined function of the cooperatively manipulated surgical system. The collar may be spring-loaded such that upon release of the collar at any position, the collar is configured to return to a neutral position relative to the link of the plurality of links.

[0027] The system may further include a graphical user interface operably coupled to the set joint such that the actuator may be configured to be displayed on the graphical user interface. For example, the actuator may include a slidable cursor configured to be moved relative to a neutral center point such that movement of the slidable cursor in a first direction relative to the neutral center point causes rotation of a distal link adjacent to the set joint in the first direction relative to a proximal link adjacent to the set joint, and movement of the slidable cursor in a second direction relative to the neutral center point causes rotation of the distal link adjacent to the set joint in the second direction relative to the proximal link adjacent to the set joint. In some embodiments, the distal link adjacent to the set joint may be configured to rotate in a corresponding direction relative to the proximal link adjacent to the set joint at a rate that correlates with the distance of the slidable cursor from the neutral center point. Additionally, the graphical user interface may be configured to display an indicator that indicates the configuration of the distal link adjacent to the set joint relative to the proximal link adjacent to the set joint in real time in response to actuation of the actuator. Additionally, the graphical user interface may be configured to display a graphical representation of multiple configurations of a distal link adjacent to the set joint relative to a proximal link adjacent to the set joint such that the position of an indicator relative to the graphical representation of the multiple configurations may indicate the configuration of the distal link adjacent to the set joint relative to the proximal link adjacent to the set joint in real time in response to actuation of the actuator.

[0028] The system may further include a controller operably coupled to the robotic arm, the controller being programmed to, during the operation phase, allow the robotic arm to move freely in response to movement of a handle of a surgical instrument for performing laparoscopic surgery. The controller may be configured to switch from the operation phase to the setting phase upon activation of the set mode actuator, such that activation of the actuator causes rotation of the distal link adjacent the set joint relative to the proximal link adjacent the set joint only when the set mode actuator is in an actuated state. When the actuator is in an actuated state, application of a force at the distal region of the robotic arm in a first direction can cause rotation of the distal link adjacent the set joint in the first direction relative to the proximal link adjacent the set joint, and application of a force at the distal region of the robotic arm in a second direction can cause rotation of the distal link adjacent the set joint in the second direction relative to the proximal link adjacent the set joint. Furthermore, when the actuator is in an unactuated state, the set joint can be configured such that the distal and proximal links adjacent the set joint are fixed relative to each other in the second setting configuration. Additionally, all of the motors of the plurality of motors operably coupled to one or more motorized joints may be located within the base of the robot arm. Further, a shoulder link of the plurality of links may include a distal shoulder link rotatably coupled to a proximal shoulder link via a set joint, and a motor of the plurality of motors operably coupled to the set joint may not be backdrivable. For example, a motor of the plurality of motors operably coupled to a set joint may be located on a shoulder link adjacent to the set joint.

[0029] The system may further include a platform operably coupled to the base of the robot arm, the platform comprising a stage assembly configured to independently move the base of the robot arm horizontally and vertically relative to the platform. Thus, in the user-guided setup mode, application of a force at a distal region of the robot arm in a first direction may cause the stage assembly to move the base of the robot arm horizontally relative to the platform, and application of a force at the distal region of the robot arm in a second direction may cause the stage assembly to move the base of the robot arm vertically relative to the platform. The system may further include a setup mode actuator configured to be actuated to switch the system to the user-guided setup mode such that the system may remain in the user-guided setup mode only while the setup mode actuator is actuated. In some embodiments, the actuator may include a collar rotatably coupled to a link of the plurality of links such that the setup mode actuator may be disposed on the collar. Thus, actuation of the set mode actuator may enable rotation of the collar in a first direction, causing rotation of the distal link adjacent the set joint relative to the proximal link adjacent the set joint in the corresponding first direction, and enable rotation of the collar in a second direction, causing rotation of the distal link adjacent the set joint relative to the proximal link adjacent the set joint in the corresponding second direction. The collaborative surgical system may not be remotely operated via user input received at a remote surgeon console.

[0030] According to another aspect of the present disclosure, a method of assisting in laparoscopic surgery using a robotic arm is provided, the robotic arm including a plurality of links, a plurality of joints including one or more motorized joints, a setting joint, and one or more passive joints, a proximal end operably coupled to a base of the robotic arm, and a distal region having a distal end configured to be removably coupled to a surgical instrument. The method may include actuating an actuator operably coupled to a motor operably coupled to the setting joint, causing rotation of a distal link of the plurality of links adjacent to the setting joint relative to a proximal link of the plurality of links adjacent to the setting joint from a first setting configuration to a second setting configuration in response to the actuation of the actuator, and moving the robotic arm via the one or more motorized joints and the one or more passive joints in response to movement of a handle of a surgical instrument to perform the laparoscopic surgery, while the distal link adjacent to the setting joint and the proximal link adjacent to the setting joint remain in the second setting configuration when the actuator is in an unactuated state. For example, actuating the actuator to cause rotation of the distal link adjacent the set joint relative to the proximal link adjacent the set joint may include rotating a collar rotatably coupled to a link of the plurality of links in a first direction to cause rotation of the distal link adjacent the set joint relative to the proximal link adjacent the set joint in the corresponding first direction, and rotating the collar in a second direction to cause rotation of the distal link adjacent the set joint relative to the proximal link adjacent the set joint in the corresponding second direction. Furthermore, actuating the actuator to cause rotation of the distal link adjacent the set joint relative to the proximal link adjacent the set joint may further include actuating a set mode actuator disposed on the collar to enable rotation of the distal link adjacent the set joint relative to the proximal link adjacent the set joint in the corresponding first and second directions in response to the rotation of the collar.

[0031] Additionally, actuating an actuator to cause rotation of a link distal to the set joint relative to a link proximal to the set joint may include actuating an actuator displayed on the graphical user interface. For example, actuating an actuator displayed on the graphical user interface may include moving a slidable cursor relative to a neutral center point such that moving the slidable cursor in a first direction relative to the neutral center point causes rotation of the distal link adjacent to the set joint in the first direction relative to the proximal link adjacent to the set joint, and moving the slidable cursor in a second direction relative to the neutral center point causes rotation of the distal link adjacent to the set joint in the second direction relative to the proximal link adjacent to the set joint. Thus, the method may further include displaying, via the graphical user interface, an indicator that indicates, in real time, the configuration of the distal link adjacent to the set joint relative to the proximal link adjacent to the set joint in response to actuation of the actuator.

[0032] Additionally, the method may include displaying, via the graphical user interface, a graphical representation of a plurality of configurations of a distal link adjacent to the set joint relative to a proximal link adjacent to the set joint, such that a position of an indicator relative to the graphical representation of the plurality of configurations indicates a configuration of the distal link adjacent to the set joint relative to the proximal link adjacent to the set joint in real time in response to actuation of the actuator. Further, actuating the actuator to cause rotation of the distal link of the set joint relative to the proximal link of the set joint may include, when the actuator is in an actuated state, applying a force to a distal region of the robot arm in a direction to cause rotation of the distal link adjacent to the set joint relative to the proximal link adjacent to the set joint in a corresponding direction. The method may further include, in a user-guided set mode, applying a force to a distal region of the robot arm in a direction to a stage assembly operably coupled to the base of the robot arm and moving the base of the robot arm in the corresponding direction relative to a platform coupled to the stage assembly.

[0033] According to another aspect of the present disclosure, a collaborative surgical system is provided for providing adaptive gravity compensation to a robotic arm including a plurality of links, a plurality of joints, and a distal end configured to be removably coupled to a surgical instrument. The collaborative surgical system may include at least one processor configured to: apply initial gravity compensation to the robotic arm to compensate for gravity of the surgical instrument based on estimated instrument parameters associated with the surgical instrument; calculate a holding force required to maintain the distal end of the robotic arm in a rest position in a passive mode during application of the initial gravity compensation; and determine calibrated instrument parameters for the surgical instrument based on the holding force, the calibrated instrument parameters being selected to adjust the holding force required to maintain the distal end of the robotic arm in a rest position in the passive mode during application of adjusted gravity compensation to the robotic arm based on the calibrated instrument parameters.

[0034] The at least one processor may be further configured to apply torque to one or more motorized joints of the plurality of joints of the robotic arm, apply initial gravity compensation to the robotic arm, and compensate for gravity of the surgical instrument. The estimated instrument parameters and calibrated instrument parameters may comprise at least one of a mass or center of mass associated with the surgical instrument. Additionally, the at least one processor may be configured to load a calibration file associated with known parameters of the surgical instrument, such that the calibration file may comprise the estimated instrument parameters. For example, the known parameters may comprise a diameter of the elongated shaft of the surgical instrument. Further, the at least one processor may be configured to determine the known parameters upon coupling of the surgical instrument to the distal end of the robotic arm via a coupler body removably coupled to the surgical instrument and the distal end of the robotic arm. In some embodiments, the at least one processor may be configured to determine the known parameters based on the coupler body. The system may further include an optical sensor configured to collect depth data, such that the at least one processor may be configured to determine the known parameters based on the depth data. Additionally or alternatively, the system may include a user interface operably coupled to the at least one processor, whereby the at least one processor is configured to determine the known parameters via user input received by the user interface.

[0035] The calibrated instrument parameters can be selected to adjust the holding force within a predetermined range associated with the known parameters of the surgical instrument based on the calibrated instrument parameters during application of the adjusted gravity compensation. Furthermore, when the distal end of the robotic arm is not subjected to any external forces other than gravity on the robot arm and surgical instrument in a rest position, the calibrated instrument parameters can be selected to adjust the holding force to zero or near zero during application of the adjusted gravity compensation based on the calibrated instrument parameters. In addition, when the distal end of the robotic arm is subjected to one or more external forces in addition to gravity on the robot arm and surgical instrument in a rest position, the calibrated instrument parameters can be selected to adjust the holding force within a predetermined range associated with the known parameters of the surgical instrument.

[0036] The at least one processor may be further configured to calculate adjusted gravity compensation for the surgical instrument based on the calibrated instrument parameters and apply the adjusted gravity compensation to the robotic arm to compensate for the gravity of the surgical instrument. For example, the at least one processor may be configured to apply torque to one or more motorized joints of the robotic arm to apply the adjusted gravity compensation to the robotic arm to compensate for the gravity of the surgical instrument. Furthermore, the at least one processor may be configured to automatically switch the robotic arm to a collaborative manipulation mode in response to determining that a force applied to the robotic arm due to a force applied to the handle of the surgical instrument exceeds a predetermined force threshold. Additionally, the at least one processor may be configured to apply adjusted gravity compensation to the robotic arm in the collaborative manipulation mode to compensate for the gravity of the surgical instrument while allowing the robotic arm to be freely movable in the collaborative manipulation mode in response to movement at the handle of the surgical instrument.

[0037] The at least one processor may be further configured to calculate an adjusted holding force upon application of the adjusted gravity compensation and maintain the distal end of the robotic arm at a stationary position in the passive mode. Thus, the at least one processor may be configured to establish a baseline holding force based on the adjusted holding force after a predetermined period upon initiation of the passive mode, and to apply a predetermined constant breakaway force threshold to the robotic arm based on the baseline holding force, such that if the holding force exceeds a predetermined constant breakaway force threshold, the at least one processor may not maintain the distal end of the robotic arm at a stationary position. Additionally, the at least one processor may be configured to apply a predetermined high breakaway force threshold for a predetermined period of time, such that if the holding force exceeds a predetermined high breakaway force threshold for the predetermined period of time, the at least one processor may not maintain the distal end of the robotic arm at a stationary position. Furthermore, the at least one processor may be configured to automatically switch the robotic arm to the passive mode in response to determining that movement of the robotic arm due to movement at the handle of the surgical instrument is less than a predetermined amount for at least a predetermined dwell time. The at least one processor may be further configured to record the calibrated instrument parameters in a calibration file associated with the surgical instrument.

[0038] According to another aspect of the present disclosure, a method is provided for assisting in laparoscopic surgery using a robotic arm including a proximal end, a distal end configured to be removably coupled to a surgical instrument, a plurality of links, and a plurality of joints between the proximal and distal ends. The method may include applying initial gravity compensation to the robotic arm based on estimated instrument parameters when the surgical instrument is coupled to the distal end of the robotic arm associated with the surgical instrument to compensate for gravity of the surgical instrument, calculating, via the controller, a holding force required to maintain the distal end of the robotic arm at a rest position in a passive mode during application of the initial gravity compensation, and determining, via the controller, calibrated instrument parameters for the surgical instrument based on the holding force, the calibrated instrument parameters being selected to adjust the holding force required to maintain the distal end of the robotic arm at a rest position in the passive mode during application of the adjusted gravity compensation to the robotic arm based on the calibrated instrument parameters. The estimated and calibrated instrument parameters may comprise at least one of a mass or a center of mass associated with the surgical instrument.

[0039] The method may further include loading, via the controller, a calibration file associated with known parameters of the surgical instrument, such that the calibration file may comprise estimated instrument parameters. For example, the known parameters may comprise a diameter of the elongate shaft of the surgical instrument. In addition, the method may include coupling the surgical instrument to a distal end of the robotic arm via a coupler body removably coupled to the surgical instrument, and determining, via the controller, the known parameters based on the coupler body. In addition, the method may include determining, via the controller, the known parameters via user input received by a user interface operably coupled to the controller. Furthermore, determining the calibrated instrument parameters based on the holding force may include determining a calibrated instrument parameter selected to adjust the holding force within a predetermined range associated with the known parameters of the surgical instrument upon application of the adjusted gravity compensation. The method may further include calculating, via the controller, an adjusted gravity compensation for the surgical instrument based on the calibrated instrument parameters, and applying, via the controller, a torque to one or more motorized joints of a plurality of joints of the robotic arm to apply the adjusted gravity compensation to the robotic arm to compensate for gravity of the surgical instrument.

[0040] Additionally, the method may include automatically switching, via the controller, to a collaborative manipulation mode in response to determining that a force applied to the robotic arm due to a force applied to the surgical instrument handle exceeds a predetermined force threshold, and applying, via the controller, adjusted gravity compensation to the robotic arm in the collaborative manipulation mode to allow the robotic arm to be freely movable in response to movement at the surgical instrument handle while compensating for the gravity of the surgical instrument. The method may further include calculating, via the controller, an adjusted holding force upon application of the adjusted gravity compensation to maintain the distal end of the robotic arm at a stationary position in the passive mode, establishing, via the controller, a baseline holding force after a predetermined period at initiation of the passive mode based on the adjusted holding force, and applying, via the controller, a predetermined constant breakaway force threshold to the robotic arm based on the baseline holding force, wherein the controller does not maintain the distal end of the robotic arm at a stationary position if the holding force exceeds the predetermined constant breakaway force threshold.

[0041] According to another aspect of the present disclosure, a collaborative surgical system for operating a robotic arm including a plurality of links, a plurality of joints, and a distal end configured to be removably coupled to a surgical instrument is provided. The collaborative surgical system may include at least one processor configured to cause the robotic arm to switch to a passive mode in response to determining that movement of the robotic arm due to movement at a handle of a surgical instrument is less than a predetermined amount for at least a predetermined dwell time, the at least one processor configured to: cause the robotic arm to maintain a stationary position in the passive mode; apply gravity compensation to the robotic arm to compensate for the gravity of the surgical instrument; calculate a holding force required to maintain the distal end of the robotic arm at a stationary position in the passive mode during application of the gravity compensation; establish a baseline holding force based on the holding force; and apply a break-away force threshold to the robotic arm based on the baseline holding force, the break-away force threshold being a predetermined amount of force that needs to be applied to the robotic arm to cause the robotic arm to exit the passive mode. The magnitude of the break-away force threshold may be equal in all directions relative to the baseline holding force. For example, the total amount of force applied to the robot arm in a direction that is required to bring the robot arm out of passive mode may be the sum of the baseline holding force in the direction and the breakaway force threshold.

[0042] The holding force required to maintain the distal end of the robotic arm at a stationary position can be continuously calculated in the passive mode. Thus, the at least one processor can be configured to determine in the passive mode that the surgical instrument is in contact with one or more anatomical structures if the holding force gradually increases over time. Additionally, the at least one processor can be configured to calculate the holding force required to maintain the distal end of the robotic arm at a stationary position in the passive mode when one or more external forces are applied to the surgical instrument by one or more anatomical structures having unknown masses. Furthermore, the at least one processor can be configured to determine the force applied to the distal end of the robotic arm and required to move the distal end of the robotic arm from a current position to a stationary position, and to calculate the holding force required to maintain the distal end of the robotic arm at a stationary position in the passive mode. The at least one processor may be further configured to cause the robotic arm to automatically switch to the collaborative manipulation mode in response to determining that the holding force required to maintain the distal end of the robotic arm in a stationary position exceeds a breakaway force threshold, such that the at least one processor may be configured to apply gravity compensation to the robotic arm in the collaborative manipulation mode to compensate for the gravity of the surgical instrument while allowing the robotic arm to be freely movable in response to movements at the handle of the surgical instrument in the collaborative manipulation mode.

[0043] The at least one processor may be configured to sense a force applied to the distal end of the robotic arm and calculate a holding force needed to maintain the distal end of the robotic arm at a stationary position in the passive mode. For example, the at least one processor may be configured to measure currents in multiple motors operably coupled to at least some of the multiple joints and sense the force applied to the distal end of the robotic arm. Further, the at least one processor may be configured to apply torques to at least some of the multiple joints of the robotic arm and apply gravity compensation to the robotic arm to compensate for the gravity of the surgical instrument.

[0044] Additionally, the at least one processor may be configured to establish a baseline holding force after a predetermined period of time upon initiation of the passive mode. Thus, the at least one processor may be configured to apply a high breakaway force threshold to the robot arm for a predetermined period of time, the high breakaway force threshold exceeding the breakaway force threshold, such that the at least one processor may be configured to cause the robot arm to exit the passive mode if the holding force required to maintain the distal end of the robot arm in a stationary position exceeds the high breakaway force threshold for a predetermined period of time. For example, the high breakaway force threshold may be selected to prevent inadvertent disengagement of the robot arm from the passive mode in response to an inadvertent force applied to the distal end of the robot arm for a predetermined period of time. The at least one processor may be further configured to apply an initial breakaway force threshold to the robot arm for a predetermined period of time, such that the at least one processor may be configured to cause the robot arm to exit the passive mode if a holding force required to maintain the distal end of the robot arm in a stationary position exceeds a high breakaway force threshold for a predetermined period of time, and to apply a high breakaway force threshold for a predetermined period of time if the force applied to the distal end of the robot arm exceeds the initial breakaway force threshold for a predetermined period of time, the high breakaway force threshold exceeding the breakaway force threshold.

[0045] Further, the at least one processor may be configured to apply a default breakaway force threshold to the robotic arm upon initiation of the passive mode if the holding force fluctuates such that a baseline holding force cannot be established based on the calculated holding force after a predetermined period of time, such that the at least one processor may be configured to cause the robotic arm to exit the passive mode if the holding force required to maintain the distal end of the robotic arm in a stationary position exceeds the default breakaway force threshold. For example, the at least one processor may be configured to select the default breakaway force threshold from among a default high breakaway force threshold and a default low breakaway force threshold based on user input via a graphical user interface operably coupled to the at least one processor. Additionally, the at least one processor may be configured to adjust at least one of the default high breakaway force threshold or the default low breakaway force threshold based on user input via the graphical user interface.

[0046] The at least one processor may be further configured to monitor the position of the distal end of the robot arm for a predetermined period of time, such that the at least one processor may be configured to cause the robot arm to exit the passive mode if a rate of change of the position of the distal end of the robot arm exceeds a predetermined position threshold for the predetermined period of time. Additionally or alternatively, the at least one processor may be configured to calculate a first force applied to the distal end of the robot arm at a first time during the predetermined period of time, apply a first break-away force threshold to the robot arm based on the first force at a second time after the first time during the predetermined period of time, and calculate a second force applied to the distal end of the robot arm at the second time, such that the at least one processor may be configured to cause the robot arm to exit the passive mode if a second force applied to the distal end of the robot arm at a second time exceeds a first break-away force threshold. Additionally, the at least one processor may be configured to apply a second detachment force threshold to the robotic arm based on the second force at a third time after the second time during the predetermined period of time, the first and second detachment force thresholds having a magnitude greater than the first and second forces, respectively, by a certain predetermined force amount. Additionally or alternatively, the at least one processor may be configured to apply the first detachment force threshold to the robotic arm for a first predetermined period of time and the second detachment force threshold to the robotic arm for the remainder of the predetermined period of time, the second detachment force threshold exceeding the first detachment force threshold and the detachment force threshold, such that the at least one processor may be configured to cause the robotic arm to exit the passive mode if a holding force required to maintain the distal end of the robotic arm in a stationary position exceeds the first detachment force threshold for the first predetermined period of time or exceeds the second detachment force threshold for the remainder of the predetermined period of time.

[0047] The at least one processor may be further configured to apply gravity compensation to the robotic arm based on estimated instrument parameters associated with the surgical instrument to compensate for gravity of the surgical instrument, such that a baseline holding force may be established based on the holding force needed to maintain the distal end of the robotic arm in a stationary position in the passive mode upon application of the adjusted gravity compensation to the robotic arm, determine calibrated instrument parameters for the surgical instrument based on the holding force, and apply adjusted gravity compensation to the robotic arm based on the calibrated instrument parameters. Further, the calibrated instrument parameters may be selected such that during application of the adjusted gravity compensation, the holding force is adjusted within a predetermined range associated with the known parameters of the surgical instrument.

[0048] According to another aspect of the present disclosure, a method is provided for assisting in laparoscopic surgery using a robotic arm including a proximal end, a distal end configured to be removably coupled to a surgical instrument, a plurality of links, and a plurality of joints between the proximal and distal ends. The method may include: causing the robot arm to switch to a passive mode, via a controller operably coupled to the robot arm, in response to determining that movement of the robot arm due to movement at the handle of the surgical instrument is less than a predetermined amount for at least a predetermined dwell time, the controller causing the robot arm to maintain a stationary position in the passive mode; applying, via the controller, gravity compensation to the robot arm to compensate for the gravity of the surgical instrument; calculating, via the controller, a holding force needed to maintain a distal end of the robot arm in a stationary position in the passive mode during application of the gravity compensation; establishing, via the controller, a baseline holding force based on the holding force; and applying, via the controller, a break-away force threshold to the robot arm based on the baseline holding force, the break-away force threshold being a predetermined amount of force that needs to be applied to the robot arm to cause it to exit the passive mode. The magnitude of the break-away force threshold may be equal in all directions relative to the baseline holding force, and the total amount of force applied to the robot arm and needed in a direction to cause it to exit the passive mode may be the sum of the baseline holding force in the direction and the break-away force threshold.

[0049] Calculating the holding force required to maintain the distal end of the robot arm at a rest position in the passive mode may include continuously calculating, via the controller, the holding force required to maintain the distal end of the robot arm at a rest position in the passive mode. Further, the method may include determining, via the controller, that the surgical instrument is in contact with one or more anatomical structures when the holding force gradually increases over time in the passive mode. Additionally, calculating the holding force required to maintain the distal end of the robot arm at a rest position in the passive mode may include calculating, via the controller, the holding force required to maintain the distal end of the robot arm at a rest position in the passive mode when one or more external forces are applied to the surgical instrument by one or more anatomical structures having unknown mass. Additionally or alternatively, calculating the holding force required to maintain the distal end of the robot arm at a rest position in the passive mode may include determining, via the controller, the force required to be applied to the distal end of the robot arm to move the distal end of the robot arm from a current position to a rest position. The method may further include causing the robotic arm, via the controller, to automatically switch to the collaborative manipulation mode in response to determining that the holding force required to maintain the distal end of the robotic arm in a stationary position exceeds a breakaway force threshold, such that while in the collaborative manipulation mode the robotic arm may be enabled to be freely movable in response to movements at the handle of the surgical instrument, gravity compensation is applied to the robotic arm to compensate for the gravity of the surgical instrument in the collaborative manipulation mode.

[0050] Further, calculating a holding force required to maintain the distal end of the robotic arm at a stationary position in the passive mode may include sensing, via the controller, a force applied to the distal end of the robotic arm. For example, sensing a force applied to the distal end of the robotic arm may include measuring, via the controller, currents of a plurality of motors operably coupled to at least some of the plurality of joints. Additionally, applying gravity compensation to the robotic arm to compensate for the gravity of the surgical instrument may include applying, via the controller, torques to at least some of the plurality of joints of the robotic arm. Establishing a baseline holding force based on the holding force may include establishing, via the controller, a baseline holding force upon initiation of the passive mode and after a predetermined period of time. Thus, the method may further include applying, via the controller, a high breakaway force threshold to the robotic arm for a predetermined period of time, the high breakaway force threshold exceeding the breakaway force threshold, and causing, via the controller, the robotic arm to exit the passive mode if the holding force required to maintain the distal end of the robotic arm at a stationary position exceeds the high breakaway force threshold for the predetermined period of time. For example, a high disengagement force threshold may be selected to prevent inadvertent disengagement of the robot arm from the passive mode in response to an inadvertent force applied to the distal end of the robot arm for a predetermined period of time.

[0051] The method may further include applying, via the controller, an initial breakaway force threshold to the robotic arm for a predetermined period of time, applying, via the controller, a high breakaway force threshold for a predetermined period of time if a force applied to the distal end of the robotic arm exceeds the initial breakaway force threshold for the predetermined period of time, the high breakaway force threshold being greater than the breakaway force threshold, and causing, via the controller, the robotic arm to exit the passive mode if a holding force required to maintain the distal end of the robotic arm in a stationary position exceeds the high breakaway force threshold for the predetermined period of time. The method may further include applying, via the controller, a default breakaway force threshold to the robotic arm upon initiation of the passive mode if, after the predetermined period of time, the holding force varies such that a baseline holding force cannot be established based on the calculated holding force, and causing, via the controller, the robotic arm to exit the passive mode if the holding force required to maintain the distal end of the robotic arm in a stationary position exceeds the default breakaway force threshold. Additionally, the method may include selecting, via the controller, a default breakaway force threshold from among a default high breakaway force threshold and a default low breakaway force threshold based on user input via a graphical user interface operably coupled to the controller. The method may further include adjusting, via the controller, at least one of the default high breakaway force threshold or the default low breakaway force threshold based on user input via the graphical user interface.

[0052] Further, applying gravity compensation to the robotic arm to compensate for the gravity of the surgical instrument may include applying gravity compensation to the robotic arm to compensate for the gravity of the surgical instrument, via the controller, based on estimated instrument parameters associated with the surgical instrument. Thus, the method may further include determining, via the controller, calibrated instrument parameters for the surgical instrument based on the holding force, and applying, via the controller, adjusted gravity compensation to the robotic arm based on the calibrated instrument parameters, just as establishing a baseline holding force based on the holding force may include establishing, via the controller, a baseline holding force based on a holding force required to maintain the distal end of the robotic arm in a stationary position in a passive mode upon application of the adjusted gravity compensation to the robotic arm. For example, the calibrated instrument parameters may be selected such that during application of the adjusted gravity compensation, the holding force is adjusted within a predetermined range associated with known parameters of the surgical instrument.

[0053] According to another aspect of the present disclosure, another cooperative surgical system for assisting in laparoscopic surgery performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween is provided. The system may include a robotic arm having a proximal end, a distal end configured to be removably coupled to a surgical instrument, a plurality of links, and a plurality of joints, and a controller operably coupled to the robotic arm and configured to enable the robotic arm to be freely movable in response to movements at the handle of the surgical instrument to perform a surgical procedure using the surgical instrument. The controller may be programmed to identify a type of surgical instrument coupled to the distal end of the robotic arm, apply a first impedance to the robotic arm, taking into account the weight of the surgical instrument and the robotic arm, and apply a second impedance to the robotic arm based on the type of surgical instrument, adjusting the stiffness at the distal end of the robotic arm, thereby guiding movements of the surgical instrument by a user during a predetermined phase of the surgical procedure.

[0054] For example, the identified type of surgical instrument may include a suturing device, and the predetermined phase of the surgical procedure may include a suturing phase, such that the second impedance may be sufficient to provide greater tenacity control of the suturing device during the suturing phase of the surgical procedure. Additionally or alternatively, the identified type of surgical instrument may include a stapling device, and the predetermined phase of the surgical procedure may include a stapling phase, such that the second impedance may be sufficient to provide firm grounding and facilitate force application of the stapling device during the stapling phase of the surgical procedure. The controller may be further configured to identify the predetermined phase of the surgical procedure based on the type of surgical instrument. Furthermore, the type of surgical instrument may be selected from a list comprising at least one of a wrist instrument, a stapling device, a dissection device, a suturing device, a retraction device, a tissue removal device, or a clip applier device. The controller can be configured to apply a second impedance to the robotic arm and adjust the stiffness at the distal end of the robotic arm based on the type of surgical instrument, thereby guiding the user's movement of the surgical instrument during certain phases of the surgical procedure without actively causing movement of the robotic arm.

[0055] According to another aspect of the present disclosure, a computer-implemented system for providing image registration to a robotic arm including a plurality of links, a plurality of joints, and a distal end configured to be removably coupled to a laparoscope having a rotatable camera sensor module is provided. The system may include at least one processor configured to retrieve a plurality of images from the laparoscope during movement of a field of view of the laparoscope, calculate via computer vision techniques a motion of individual pixels between successive images of the plurality of images, where the motion of the individual pixels indicates image motion, calculate an average of the motion of the individual pixels in the x and y directions of the plurality of images to obtain an image motion direction, and calculate an angular offset between the camera sensor module and the distal end of the robotic arm based on the image motion direction.

[0056] The at least one processor may be further configured to synchronize image movement with movement of the distal end of the robotic arm associated with movement of the laparoscope field of view, and to compare the image movement direction with movement of the distal end of the robotic arm to calculate an angular offset between the camera sensor module and the distal end of the robotic arm. Furthermore, the at least one processor may be configured to cause the robotic arm to move the laparoscope along a predetermined trajectory in the foreground mode, such that image movement may be synchronized with movement of the distal end of the robotic arm associated with movement of the laparoscope along the predetermined trajectory. Additionally or alternatively, in the background mode, image movement may be synchronized with movement of the distal end of the robotic arm in response to movement of the laparoscope field of view by a user. The at least one processor may be configured to retrieve data indicative of movement of the distal end of the robotic arm via one or more sensors operably coupled to at least some of the joints of the robotic arm.

[0057] Further, the at least one processor may be configured to verify the image motion direction. For example, the at least one processor may be configured to calculate a vector norm of the image motion direction to determine the magnitude of the image motion and compare the magnitude of the image motion with a magnitude threshold. Thus, the image motion direction may be verified if the magnitude of the image motion exceeds the magnitude threshold. Additionally or alternatively, the at least one processor may be configured to calculate a percentage of image pixels that moved between successive images based on the movement of individual pixels and compare the percentage with a percentage threshold. Thus, the image motion direction may be verified if the percentage exceeds the percentage threshold. The at least one processor may be configured to determine whether the image motion is due to at least one of movement of the laparoscope field of view or local movement of one or more tools or tissues within the multiple images based on the comparison of the percentage with the percentage threshold. Additionally or alternatively, at least one processor may be configured to calculate the relative angle between each motion of the individual pixels and the image motion direction, determine whether each motion of the individual pixels matches the image motion direction, and compare the percentage of individual pixel motions that match the image motion direction to a match threshold. Thus, the image motion direction may be verified if the percentage exceeds the match threshold.

[0058] The at least one processor may be further configured to cause the robotic arm to move the laparoscope along a predetermined axial trajectory, compare the image motion direction to a directional threshold, and determine whether the laparoscope has a flat tip or an angled tip based on comparing the image motion direction to the directional threshold. In some embodiments, movement of the laparoscope's field of view may be due to zooming of the camera sensor module, such that the at least one processor may be configured to compare the image motion direction to a directional threshold and determine whether the laparoscope has a flat tip or an angled tip based on comparing the image motion direction to the directional threshold. The controller may be further configured to automatically switch the robotic arm to a collaborative manipulation mode in response to determining that a force applied to the robotic arm due to a force applied to the laparoscope exceeds a predetermined threshold. Thus, the controller may be configured to apply an impedance to the robotic arm in the collaborative manipulation mode, allowing the robotic arm to be freely movable in response to movement in the laparoscope while taking into account the weight of the laparoscope and the robotic arm.

[0059] According to another aspect of the present disclosure, a system for robotic surgery is provided. The system may include a robotic arm having a proximal end operably coupled to a base of the robotic arm, a distal end, a plurality of links, and a plurality of joints between the proximal and distal ends, the robotic arm being configured to be positioned adjacent to a bed for supporting a patient during a surgical procedure. The system may further include a platform coupled to the base of the robotic arm, the platform including a stage assembly configured to independently move the base of the robotic arm in at least two degrees of freedom relative to the platform. Additionally, the system may include a graphical user interface including a plurality of predetermined, selectable preset configurations, and a controller operably coupled to the robotic arm. The controller may be programmed to, upon selection of a first preset configuration of the plurality of predetermined selectable preset configurations during the surgical setup phase, automatically position the robotic arm in a first configuration associated with the first preset configuration, and to, upon selection of a second preset configuration of the plurality of predetermined selectable preset configurations during the surgical setup phase, automatically position the robotic arm in a second configuration associated with the second preset configuration, wherein the first configuration associated with the first preset configuration is different from the second configuration associated with the second preset configuration.

[0060] The first preset configuration or the second preset configuration may include a storage mode such that, upon selection of the storage mode during the surgical setup phase, the controller automatically positions the robot arm via the multiple links and joints in a retracted storage configuration and rotates the robot arm about the base so that the robot arm is within the footprint of the platform. Additionally, the first preset configuration or the second preset configuration may include a compact mode such that, upon selection of the compact mode during the surgical setup phase, the controller automatically positions the robot arm via the multiple links and joints in a semi-retracted configuration such that the robot arm extends away from the platform to facilitate transport of the system. The semi-retracted configuration may not be retracted as far as the retracted storage configuration of the storage mode. Furthermore, the first preset configuration or the second preset configuration may include a drape mode such that, upon selection of the drape mode during a surgical setup phase, the controller may automatically position the robot arm in an extended configuration via the multiple links and joints so that the robot arm extends away from the platform and facilitates draping of the robot arm. For example, in the extended configuration, a distal link of the multiple links of the robot arm may be substantially parallel to the ground. Additionally, in the drape mode, the controller may be configured to apply a temporary local virtual haptic boundary to a distal region of the robot arm so that, in the drape mode, movement of the robot arm is more tenacious, thereby stabilizing the robot arm.

[0061] Additionally, the first preset configuration or the second preset configuration may comprise a custom preset configuration associated with a custom robotic arm configuration pre-stored in the controller's memory such that upon selection of the custom preset configuration during a surgical setup phase, the controller may automatically position the robotic arm in the custom robotic arm configuration. For example, the custom preset configuration associated with the custom robotic arm configuration may be configured to be stored in a user-specific profile, and the controller may be further configured to display the custom preset configuration on a graphical user interface upon selection of the user-specific profile. Furthermore, the custom robotic arm configuration may be selected to facilitate a given surgical procedure, such as a cholecystectomy, sleeve gastrectomy, hiatal hernia repair, Nissen-Fandomization, inguinal hernia repair (TEP), right, left, and / or total colectomy, gastric bypass, sigmoid resection, umbilical hernia repair, or incisional hernia repair.

[0062] For example, the predetermined surgical procedure can be a cholecystectomy, such that, upon selection of a custom preset configuration, the controller can be configured to rotate a shoulder link of the plurality of links of the robotic arm in a leftward direction relative to the platform and to cause the stage assembly to move a base of the robotic arm downward in a first of the at least two degrees of freedom and outward in a second of the at least two degrees of freedom. In some embodiments, the system can further include a second robotic arm, such that, upon selection of a custom preset configuration, the controller can be configured to cause the stage assembly of the second robotic arm to move a base of the second robotic arm upward in a first of the at least two degrees of freedom and inward in a second of the at least two degrees of freedom.

[0063] A controller operably coupled to the surgical procedure and the robotic arm. The controller can be programmed to automatically, during the surgical setup phase, position the robotic arm relative to the bed at a first position specific to the first surgical procedure in response to selection of a first surgical procedure of the plurality of predetermined selectable surgical procedures, and to automatically, during the surgical setup phase, position the robotic arm relative to the bed at a second position specific to the second surgical procedure in response to selection of a second surgical procedure of the plurality of predetermined selectable surgical procedures. For example, the first position specific to the first surgical procedure can be different from the second position specific to the second surgical procedure.

[0064] Further, the controller may be configured to rotate a shoulder link of the plurality of links of the robotic arm in a leftward direction relative to the platform when a cholecystectomy is selected, and to cause the stage assembly to move a base of the robotic arm in a downward direction in a first degree of freedom of the at least two degrees of freedom and in an outward direction in a second degree of freedom of the at least two degrees of freedom. Additionally, the system may include a second robotic arm such that the controller may be configured to cause the stage assembly of the second robotic arm to move a base of the second robotic arm in an upward direction in a first degree of freedom of the at least two degrees of freedom and in an inward direction in a second degree of freedom of the at least two degrees of freedom when a cholecystectomy is selected. At least one of the first or second surgical procedures may include cholecystectomy, sleeve gastrectomy, hiatal hernia repair, Nissen-Fandom plication, inguinal hernia repair (TEP), right, left, and / or total colectomy, gastric bypass, sigmoid colectomy, umbilical hernia repair, or incisional hernia repair.

[0065] According to another aspect of the present disclosure, a cooperative surgical system for assisting in surgical procedures, e.g., laparoscopic surgery, performed using surgical instruments is provided. The system may include multiple robotic arms, each robotic arm including multiple links, multiple joints, a proximal region operably coupled to a base, and a distal region configured to be removably coupled to a surgical instrument. The system may further include a platform coupled to the base of each robotic arm, wherein in a surgical mode, each of the robotic arms may be configured to extend away from the platform toward a surgical site, and a controller configured to, in activation of a storage mode, transition each of the robotic arms to a retracted storage configuration via the respective multiple links and joints of the robotic arms and rotate each of the robotic arms about its respective base such that in the retracted storage configuration, each of the robotic arms is within the footprint of the platform. For example, in the retracted storage configuration, each of the robotic arms may be entirely within the footprint of the platform.

[0066] When operating in the storage mode, the controller may rotate a first robotic arm of the plurality of robotic arms about its base in a first direction and a second robotic arm of the plurality of robotic arms about its base in a second direction, the second direction being opposite to the first direction. Alternatively, the second direction may be the same as the first direction. In the retracted storage configuration, the first robotic arm of the plurality of robotic arms may have the same configuration and orientation as the second robotic arm of the plurality of robotic arms relative to the platform. Additionally, in the retracted storage configuration, each of the robot arms may extend in a direction toward the rear of the platform. When operating in the storage mode, the controller may move at least one robotic arm of the plurality of robotic arms to a position via the plurality of links and joints of each of the at least one robotic arm such that a collision between the at least one robotic arm and the platform is avoided when the at least one robotic arm rotates about its respective base.

[0067] The platform may include a stage assembly configured to independently move the base of each robotic arm in at least two degrees of freedom relative to the platform. Thus, during operation in the storage mode, the controller may cause the stage assembly to move the base of each robotic arm in an inward direction along a first of the at least two degrees of freedom so that, in the retracted storage configuration, each of the robotic arms is within the footprint of the platform upon rotation of each of the robotic arms about its respective base. Additionally, during operation in the storage mode, the controller may cause the stage assembly to move the base of each robotic arm downward along a second of the at least two degrees of freedom so that the robot arms have a minimum height relative to the platform for transport and / or storage of the collaboratively manipulated surgical system in the retracted storage configuration. Furthermore, during operation in the storage mode, the controller may cause the stage assembly to move the base of each robotic arm downward along a second of the at least two degrees of freedom so that the robot arms have a minimum height relative to the platform for transport and / or storage of the collaboratively manipulated surgical system in the retracted storage configuration. Additionally, when operating in the storage mode, the controller may cause the stage assembly to move a base of the at least one robot arm in an outward direction along a first of the at least two degrees of freedom and / or an upward direction along a second of the at least two degrees of freedom so as to avoid collision between the at least one robot arm and the platform upon rotation of at least one robot arm of the multiple robot arms about the base of the at least one robot arm. In some embodiments, when operating in the storage mode, the controller may cause the stage assembly to rotate the bases of each robot arm toward each other along a horizontal plane relative to the platform such that each of the robot arms is within the footprint of the platform upon rotation of each of the robot arms about the base of the robot arm in the retracted stored configuration.

[0068] The system may further include a graphical user interface operably coupled to the controller. The graphical user interface may be configured to display a retraction mode actuator configured to be actuated by a user to activate the retraction mode. Furthermore, during operation of the compact mode, the controller may cause each of the robot arms to transition to a semi-retracted configuration and rotate about its respective base via its respective multiple links and joints so that each of the robot arms extends away from the platform and facilitates transport of the cooperatively manipulated surgical system, the semi-retracted configuration being less retracted than the retracted storage configuration of the retraction mode. Additionally, during operation of the drape mode, the controller may cause each of the robot arms to transition to an extended configuration and rotate about its respective base via its respective multiple links and joints so that each of the robot arms extends away from the platform and facilitates draping of the robot arms. In the extended configuration, a distal link of the multiple links of each robot arm may be substantially parallel to the ground.

[0069] According to another aspect of the present disclosure, a method for assisting in laparoscopic surgery is provided. The method may include providing a plurality of robotic arms, each robotic arm including a plurality of links, a plurality of joints, a proximal region operably coupled to a base, and a distal region configured to be removably coupled to a surgical instrument, each base coupled to a platform such that in a surgical mode, each of the robotic arms may be configured to extend away from the platform toward a surgical site; automatically, upon activation of a storage mode, transitioning each of the robotic arms to a retracted, stored configuration via the respective plurality of links and joints of the robotic arms; and rotating each of the robotic arms about its respective base such that in the retracted, stored configuration, each of the robotic arms is within a footprint of the platform. Automatically transitioning each of the robotic arms to the retracted, stored configuration may include automatically rotating a first robotic arm of the plurality of robotic arms about its base in a first direction and automatically rotating a second robotic arm of the plurality of robotic arms about its base in a second direction, the second direction being opposite to the first direction. Alternatively, the second direction may be the same as the first direction. Additionally, automatically transitioning each of the robotic arms to the retracted, stored configuration may include automatically moving at least one robotic arm of the plurality of robotic arms to a position via a plurality of links and joints of each of the at least one robotic arms such that a collision between the at least one robotic arm and the platform is avoided upon rotation of the at least one robotic arm about the respective base of the at least one robotic arm.

[0070] The platform may include a stage assembly configured to independently move the base of each robot arm in at least two degrees of freedom relative to the platform. Thus, automatically transitioning each of the robot arms to the retracted storage configuration may include automatically causing the stage assembly to move the base of each robot arm in an inward direction along a first of the at least two degrees of freedom such that, in the retracted storage configuration, each of the robot arms is within the footprint of the platform upon rotation of each of the robot arms about its respective base. Automatically transitioning each of the robot arms to the retracted storage configuration may include automatically causing the stage assembly to move the base of each robot arm downward along a second of the at least two degrees of freedom such that the robot arms have a minimum height relative to the platform for transport and / or storage in the retracted storage configuration. Additionally, automatically transitioning each of the robot arms to the retracted stored configuration may include automatically causing the stage assembly to move a base of the at least one robot arm in an outward direction along a first of the at least two degrees of freedom and / or an upward direction along a second of the at least two degrees of freedom to avoid collision between the at least one robot arm and the platform upon rotation of the at least one robot arm of the plurality of robot arms about a respective base of the at least one robot arm.

[0071] The method may further include displaying, via the graphical user interface, a storage mode actuator configured to be actuated by a user to activate the storage mode. Furthermore, the method may include automatically, upon activation of the compact mode, transitioning each of the robot arms via their respective multiple links and joints to a semi-retracted configuration and rotating each of the robot arms about their respective bases to extend away from the platform and facilitate transport of the platform, the semi-retracted configuration being less retracted than the retracted storage configuration of the storage mode. Additionally, the method may automatically, upon activation of the drape mode, transitioning each of the robot arms via their respective multiple links and joints to an extended configuration and rotating each of the robot arms about their respective bases to extend away from the platform and facilitate draping of the robot arms. In the extended configuration, a distal link of the multiple links of each robot arm may be substantially parallel to the ground. The method may further include draping the multiple robot arms with a sterile drape in the extended configuration.

[0072] According to another aspect of the present disclosure, a collaborative surgical system for assisting in surgical procedures, e.g., laparoscopic surgery, performed using a surgical instrument is provided. The collaborative surgical system may include a robotic arm including a proximal region having a base, a plurality of links, an elbow joint, a shoulder joint coupled to the base, and a distal region having a distal end configured to be removably coupled to a surgical instrument; and a platform coupled to the base of the robotic arm, the platform configured to be movable on a floor. Thus, in a drape mode, a link of the plurality of links between the elbow joint and the shoulder joint may be configured to extend away from the base at a first angle, and a link of the plurality of links distal to the elbow joint may be configured to extend away from the base at a second angle to be more aligned with the floor and thereby facilitate draping of the robotic arm. For example, the first angle may be selected such that, in the drape mode, the link between the elbow joint and the shoulder joint is configured to be substantially vertical. Additionally, the second angle can be selected such that in the drape mode, the distal link of the elbow joint is configured to be substantially horizontal. Additionally, or alternatively, the second angle can be selected such that in the drape mode, the distal link of the elbow joint is configured to be substantially parallel to the floor.

[0073] The platform may include a plurality of wheels configured to enable mobility of the platform on the floor. Additionally, the robotic arm may not be remotely controlled via user input received at the remote surgeon console. The collaborative surgical system may further include a controller operably coupled to the robotic arm. The controller may be configured to automatically extend a link between the elbow joint and the shoulder joint away from the base at a first angle and extend a distal link of the elbow joint away from the base at a second angle when the drape mode is activated. For example, when the drape mode is activated, the controller may automatically rotate the robotic arm about the base so that the link between the elbow joint and the shoulder joint extends away from the platform at a first angle and the distal link of the elbow joint extends away from the platform at a second angle. The controller may further be configured to apply a virtual haptic boundary around the robotic arm when the robotic arm is within the virtual haptic boundary, such that the controller applies a high level of impedance to the robotic arm such that movement of the robotic arm is more compliant within the virtual haptic boundary. Additionally or alternatively, the controller may be further configured to apply a temporary local virtual haptic boundary to a distal region of the robot arm in the drape mode so that movement of the robot arm is more tenacious in the drape mode, thereby stabilizing the robot arm.

[0074] The collaborative surgical system may further include a coupler configured to be removably coupled to the distal end of the robotic arm after application of a sterile drape over the robotic arm, the coupler being configured to be removably coupled to the surgical instrument, e.g., to removably couple the surgical instrument to the distal end of the robotic arm. Thus, the controller may be further configured to remove a temporary local virtual force boundary at the distal region of the robotic arm upon detecting that the coupler is removably coupled to the distal end of the robotic arm. Furthermore, the platform may be configured such that, when activated in the drape mode, the controller automatically causes the platform to move the base of the robotic arm in at least one of at least two degrees of freedom relative to the floor to align the distal link of the elbow joint with a user's height and facilitate draping of the robotic arm by the user. Thus, the controller may be further configured to store data indicative of the user's height in a user-specific profile and to execute the user-specific profile in response to user input received at a user interface operably coupled to the controller. Additionally or alternatively, the collaborative surgical system may further include an optical sensor operably coupled to the controller and configured to collect depth data, such that the controller may be configured to determine the user's height based on the depth data collected by the optical sensor.

[0075] In some embodiments, the collaborative surgical system may further include a second robotic arm having a second base coupled to the platform such that, in the drape mode, the links of the second robotic arm may be parallel to the corresponding links of the robotic arm. The collaborative surgical system may further include a sterile drape having a first portion sized and shaped to cover at least the robotic arm and a second portion sized and shaped to cover at least the second robotic arm. For example, each of the first and second portions of the sterile drape may include a sealing end portion configured to contact the distal end of the respective robotic arm such that the robotic arm is fully enclosed within the sterile drape. Furthermore, the proximal end of the sterile drape may include an opening sized and shaped to receive at least a portion of the platform. The proximal end of the sterile drape may include an elastic band along the opening configured to facilitate wrapping of the sterile drape over at least a portion of the platform. Additionally, the sterile drape may include one or more straps configured to secure the first and second portions of the sterile drape to the respective robotic arms. Furthermore, the sterile drape may include one or more rigid guides configured to be grasped by a user and to guide the sterile drape over the robotic arms. The cooperatively manipulated surgical system may further include first and second couplers configured to be removably coupled to the distal ends of the respective robotic arms after application of the sterile drape over the robotic arms, each of the first and second couplers configured to be removably coupled to a surgical instrument. Thus, the controller may be further configured to apply a temporary local virtual haptic boundary to the distal region of the robotic arm in the drape mode so that movement of the robotic arm is more tenacious, thereby stabilizing the robotic arm in the drape mode, and to remove the temporary local virtual haptic boundary at the distal region of the robotic arm upon detecting that both the first and second couplers are removably coupled to the distal ends of the robotic arms.

[0076] According to another aspect of the present disclosure, a method for assisting in laparoscopic surgery is provided. The method may include providing a robotic arm including a proximal region having a base coupled to a platform configured to be movable on a floor, a plurality of links, an elbow joint, a shoulder joint coupled to the base, and a distal region having a distal end configured to be removably coupled to a surgical instrument; and transitioning the robotic arm to a drape mode, wherein a link of the plurality of links between the elbow joint and the shoulder joint extends away from the base at a first angle and a link of the plurality of links distal to the elbow joint extends away from the base at a second angle to be more aligned with the floor and thereby facilitate draping of the robotic arm. For example, when the distal link of the elbow joint extends away from the base at the second angle, the distal link of the elbow joint may be substantially horizontal.

[0077] Transitioning the robot arm to the drape mode may include automatically, via a controller operably coupled to the robot arm, extending a link between the elbow joint and the shoulder joint away from the base at a first angle and extending a distal link of the elbow joint away from the base at a second angle when the drape mode is activated. Additionally, transitioning the robot arm to the drape mode may further include automatically, via the controller, rotating the robot arm about the base when the drape mode is activated so that the link between the elbow joint and the shoulder joint extends away from the platform at the first angle and the distal link of the elbow joint extends away from the platform at a second angle. The method may further include applying, via the controller, a temporary local virtual haptic boundary to a distal region of the robot arm in the drape mode so that movement of the robot arm is more tenacious, thereby stabilizing the robot arm in the drape mode. Additionally, the method may include applying a sterile drape over the robotic arm, removably coupling a coupler to the distal end of the robotic arm such that the sterile drape is disposed between the sterile drape and the distal end of the robotic arm, and removably coupling a surgical instrument to the coupler. Further, the method may include providing a second robotic arm having a second base coupled to the platform, and transitioning the second robotic arm to a drape mode, wherein a plurality of links of the second robotic arm are parallel to a corresponding plurality of links of the robotic arm in the drape mode. [Brief explanation of the drawings]

[0078] [Figure 1] 1A and 1B illustrate a conventional laparoscopic procedure performed by a surgeon and one or more assistants.

[0079] [Figure 2A] 2A and 2B illustrate an exemplary cooperative surgical system constructed in accordance with the principles of the present disclosure. [Figure 2B] 2A and 2B illustrate an exemplary cooperative surgical system constructed in accordance with the principles of the present disclosure.

[0080] [Figure 2C] FIG. 2C illustrates movement of an exemplary stage assembly of the platform of FIGS. 2A and 2B in accordance with the principles of the present disclosure.

[0081] [Figure 2D] FIG. 2D illustrates a proximity sensor in the base of the robotic arm of the system of FIGS. 2A and 2B.

[0082] [Figure 3] FIG. 3 illustrates an exemplary robotic arm of the system of FIGS. 2A and 2B constructed in accordance with the principles of the present disclosure.

[0083] [Figure 4] FIG. 4 illustrates an exemplary robotic arm of the system of FIGS. 2A and 2B constructed in accordance with the principles of the present disclosure.

[0084] [Figure 5] 5A and 5B illustrate an exemplary surgical instrument coupling mechanism at the distal end of the robotic arm of FIG. 3 constructed in accordance with the principles of the present disclosure.

[0085] [Figure 6] FIG. 6 illustrates an exemplary coupler interface of the surgical instrument coupling mechanism of FIGS. 5A and 5B.

[0086] [Figure 7-1] 7A-7C illustrate an exemplary coupler body of the surgical instrument coupling mechanism of FIGS. 5A and 5B.

[0087] [Figure 7-2] 7D-7H illustrate an alternative exemplary surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure. [Figure 7-3] 7D-7H illustrate an alternative exemplary surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure.

[0088] [Figure 8A] FIG. 8A is a cross-sectional view of the coupler body of FIG. 7A when the coupler body is removed from the coupler interface.

[0089] [Figure 8B] 8B is a cross-sectional view of the surgical instrument coupling mechanism of FIG. 5A when the coupler body is coupled to the coupler interface.

[0090] [Figure 8C] FIG. 8C is a cross-sectional view of the surgical instrument coupling mechanism of FIG. 5A when a surgical instrument is coupled to the coupler body.

[0091] [Figure 9] FIG. 9 is a cross-sectional view of another exemplary surgical instrument coupling mechanism when a surgical instrument constructed in accordance with the principles of the present disclosure is coupled to the coupler body.

[0092] [Figure 10A] FIG. 10A illustrates the robotic arm in a sterile-drape ready configuration.

[0093] [Figure 10B] FIG. 10B illustrates an exemplary sterile drape having first and second drape portions constructed in accordance with the principles of the present disclosure.

[0094] [Figure 10C] FIG. 10C illustrates exemplary method steps for draping a robotic arm.

[0095] [Figure 10D] FIG. 10D illustrates the robotic arm covered in a sterile drape.

[0096] [Figure 10E] FIG. 10E illustrates the robotic arm being draped within a sterile drape using straps.

[0097] [Figure 11] 11A and 11B illustrate the rotation of a shoulder link of a robotic arm in accordance with the principles of the present disclosure.

[0098] [Figure 12] FIG. 12A illustrates the field of view of an optical scanner during a laparoscopic surgical procedure, and FIG. 12B illustrates a depth map of the field of view of the optical scanner of FIG. 12A.

[0099] [Figure 13-1] 13A-13D illustrate the setup of a cooperative surgical system according to the principles of the present disclosure. [Figure 13-2] 13A-13D illustrate the setup of a cooperative surgical system according to the principles of the present disclosure.

[0100] [Figure 14] FIG. 14 illustrates some exemplary components that may be included within a collaboratively manipulated robotic platform according to the principles of the present disclosure.

[0101] [Figure 15] FIG. 15 illustrates an exemplary virtual overlay of a graphical user interface of a collaborative surgical system.

[0102] [Figure 16] FIG. 16 is a table of example values ​​associated with several configurations of the passive mode of a robotic arm in accordance with the principles of the present disclosure.

[0103] [Figure 17A] FIG. 17A is a flow chart illustrating training of a cooperatively manipulated surgical system to identify and track surgical instruments for instrument centering in accordance with the principles of the present disclosure.

[0104] [Figure 17B] FIG. 17B is a flow chart illustrating robotic arm trajectory generation for instrument centering in accordance with the principles of the present disclosure.

[0105] [Figure 18] FIG. 18 is a flow chart illustrating the operation of a cooperative surgical system according to the principles of the present disclosure.

[0106] [Figure 19] FIG. 19 is a flow chart illustrating surgical instrument calibration for a cooperative surgical system in accordance with the principles of the present disclosure.

[0107] [Figure 20] FIG. 20 is a flow chart illustrating the operation of a robotic arm in accordance with the principles of the present disclosure.

[0108] [Figure 21] FIG. 21 is a flow chart illustrating instrument centering in accordance with the principles of the present disclosure.

[0109] [Figure 22] FIG. 22 illustrates an exemplary tracking overlay of a graphical user interface of a collaborative surgical system.

[0110] [Figure 23A] 23A and 23B are free body diagrams illustrating the forces applied to a surgical instrument coupled to a robotic arm during a laparoscopic surgical procedure. [Figure 23B] 23A and 23B are free body diagrams illustrating the forces applied to a surgical instrument coupled to a robotic arm during a laparoscopic surgical procedure.

[0111] [Figure 23C] 23C-23E are free body diagrams illustrating the movement of a surgical instrument coupled to a robotic arm along a trajectory for instrument centering. [Figure 23D] 23C-23E are free body diagrams illustrating the movement of a surgical instrument coupled to a robotic arm along a trajectory for instrument centering. [Figure 23E] 23C-23E are free body diagrams illustrating the movement of a surgical instrument coupled to a robotic arm along a trajectory for instrument centering.

[0112] [Figure 24] FIG. 24 illustrates an exemplary adaptive gravity compensation process for dynamically adjusting gravity compensation in accordance with the principles of the present disclosure.

[0113] [Figure 25] 25A and 25B illustrate adaptive gravity compensation as applied to a robotic arm coupled to a surgical instrument that is subjected to external forces in addition to gravity.

[0114] [Figure 26A] 26A-26C illustrate retention force over time for establishing a withdrawal force threshold based on retention force according to the principles of the present disclosure. [Figure 26B] 26A-26C illustrate retention force over time for establishing a withdrawal force threshold based on retention force according to the principles of the present disclosure. [Figure 26C] 26A-26C illustrate retention force over time for establishing a withdrawal force threshold based on retention force according to the principles of the present disclosure.

[0115] [Figure 27] FIG. 27A illustrates the separation force threshold independent of the holding force, and FIG. 27B illustrates the separation force threshold based on the holding force.

[0116] [Figure 28] FIG. 28A illustrates a conventional laparoscopic device, and FIG. 28B illustrates a laparoscopic device coupled to the distal end of a robotic arm.

[0117] [Figure 29]FIG. 29 is a flowchart illustrating an exemplary framework for detecting and determining angular offset between a laparoscopic camera sensor module and the distal end of a robotic arm according to the principles of the present disclosure.

[0118] [Figure 30A] FIG. 30A illustrates the determination of image motion direction via computer vision techniques, and FIG. 30B illustrates the movement of a laparoscopic video feed along the image motion direction. [Figure 30B] FIG. 30A illustrates the determination of image motion direction via computer vision techniques, and FIG. 30B illustrates the movement of a laparoscopic video feed along the image motion direction.

[0119] [Figure 31] FIG. 31 illustrates an exemplary pivoting motion for calculating the angular offset between the laparoscopic camera sensor module and the distal end of the robotic arm.

[0120] [Figure 32] FIG. 32 illustrates an exemplary overview of some features and capabilities of a cooperative surgical system according to the principles of the present disclosure.

[0121] [Figure 33] FIG. 33 illustrates a virtual map of a collaborative surgical system in an operating room.

[0122] [Figure 34] FIG. 34 is a schematic overview of some of the electrical components and connectivity of a cooperatively manipulated surgical system according to the principles of the present disclosure.

[0123] [Figure 35] FIG. 35 is a flowchart illustrating an exemplary process for obtaining and processing data from an imaging device and an exemplary application of the data in accordance with the principles of the present disclosure.

[0124] [Figure 36]FIG. 36 is a simplified overview of data flow for a cooperative surgical system in accordance with the principles of the present disclosure.

[0125] [Figure 37] FIG. 37 is another schematic overview of data flow for a cooperative surgical system according to the principles of the present disclosure.

[0126] [Figure 38] FIG. 38 is a schematic overview of data flow within a network of collaborative surgical systems according to the principles of the present disclosure.

[0127] [Figure 39A] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39B] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39C] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39D] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39E] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39F] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39G] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39H] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39I]39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39J] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39K] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39L] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39M] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39N] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39O] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39P] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39Q] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 39R] 39A-39R illustrate an exemplary graphical user interface of a collaborative surgical system.

[0128] [Figure 40A] 40A-40C illustrate an exemplary graphical user interface of a collaborative surgical system displaying a virtual map according to the principles of the present disclosure. [Figure 40B] 40A-40C illustrate an exemplary graphical user interface of a collaborative surgical system displaying a virtual map according to the principles of the present disclosure. [Figure 40C]40A-40C illustrate an exemplary graphical user interface of a collaborative surgical system displaying a virtual map according to the principles of the present disclosure.

[0129] [Figure 41] FIG. 41 illustrates the degrees of freedom of movement of the shoulder portion and stage of a cooperative surgical system for a preset configuration of the platform and robotic arm according to the principles of the present disclosure.

[0130] [Figure 42] FIG. 42 illustrates the co-operation surgical system in an exemplary pre-set cholecystectomy configuration.

[0131] [Figure 43-1] 43A-43D illustrate a cooperative surgical system in an exemplary preset storage configuration. [Figure 43-2] 43A-43D illustrate a cooperative surgical system in an exemplary preset storage configuration.

[0132] [Figure 44] FIG. 44 illustrates movement of the shoulder and stage of the cooperatively manipulated surgical system for the preset stored configuration of FIGS. 43A-43D.

[0133] [Figure 45-1] 45A-45D illustrate a cooperative surgical system in another exemplary preset storage configuration. [Figure 45-2] 45A-45D illustrate a cooperative surgical system in another exemplary preset storage configuration.

[0134] [Figure 46A] 46A and 46B illustrate an alternative exemplary collaborative surgical system platform having a rotatable stage assembly constructed in accordance with the principles of the present disclosure. [Figure 46B]46A and 46B illustrate an alternative exemplary collaborative surgical system platform having a rotatable stage assembly constructed in accordance with the principles of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0135] Disclosed herein are cooperatively operated surgical robotic systems and methods of use for assisting an operator, e.g., a surgeon, in performing a surgical procedure, e.g., a laparoscopic procedure. Currently, laparoscopic procedures typically require a surgeon and one or more assistants. For example, as shown in FIG. 1A , during a laparoscopic procedure, assistant A1 may be required to hold retractor device 12 and expose tissue for surgeon S, while another assistant A2 may be required to hold laparoscopic device 10 during the procedure and provide surgeon S with a view of the surgical space within the patient via a display (not shown). As shown in FIG. 1A , assistant A2 may be required to hold laparoscopic device 10 in an impractical position, e.g., between the arms of surgeon S, while the surgeon actively operates additional surgical instruments, e.g., surgical instruments 14 and 16. As further shown in FIG. 1A, surgeon S may need to let go of surgical instruments 16 to guide / reposition laparoscopic device 10 held by assistant A2 to achieve the field of view desired by the surgeon.

[0136] As shown in FIG. 1B, rail-mounted orthopedic retractors 18 may be used to hold one or more surgical instruments in place during a laparoscopic procedure in an attempt to free the surgeon's and / or assistant's hands for other tasks and for stability. As shown in FIG. 1B, a first rail-mounted orthopedic retractor 18a may include a retractor end 20a to engage and hold in place a laparoscopic device 10 upon activation of a locking portion 22a. For example, the locking portion 22a may be disengaged so that the retractor 18a may be manually positioned at a desired location relative to the patient and reengaged to lock the retractor 18a, and thus the laparoscopic device 10, in the desired position. As shown in FIG. 1B, a second rail-mounted orthopedic retractor 18b having a retractor end 20b may also be used to engage and hold in place another surgical instrument during a procedure upon activation of a locking portion 22b. Thus, retractors 18a and 18b require extensive manual interaction with locks 22a and 22b, and with retractors 18a and 18b themselves, to reposition and lock the respective tools in place.

[0137] The collaborative surgical robotic systems described herein provide superior control and stability so that surgeons and / or assistants can seamlessly position a variety of off-the-shelf surgical instruments as needed, thus avoiding workflow limitations inherent in both human and mechanical solutions. For example, the robotic arms of the collaborative surgical robotic system can provide surgical assistance by holding a first surgical instrument, e.g., a laparoscope, via a first robotic arm and a second surgical instrument, e.g., a retractor, via a second robotic arm steadily throughout the procedure, providing optimal views of the surgical site and reducing variability in the forces applied by the surgical instruments to the body wall at the trocar point. As will be understood by one skilled in the art, the robotic arms of the collaborative surgical robotic systems described herein can preferably hold any surgical instrument having an elongated instrument shaft used for surgical procedures such as laparoscopic procedures, including, for example, endoscopes / laparoscopes, retractors, graspers, surgical scissors, needle holders, clamps, suturing instruments, cautery tools, staplers, clip appliers, hooks, etc.

[0138] A collaborative surgical robotic system also allows the surgeon to easily manipulate both tools when necessary, providing superior control, stability, and overall safety throughout the procedure. Any implementation of the system described herein allows the surgeon to collaboratively manipulate instruments directly at the patient bedside, while remaining sterile. For example, the system may include two robotic arms that the surgeon can use to hold both a laparoscope and a retractor. During the surgical procedure, the system can seamlessly reposition either instrument to provide optimal visualization and exposure of the surgical field. Both instruments may be directly coupled to the system's robotic arms, and the system may constantly monitor and record the positions of the two instruments and / or the two robotic arms throughout the procedure. Additionally, the system may record information such as the location of surgical entry ports, the position and movement of the surgeon's hands, the position and orientation of surgical instruments, the position and orientation of surgical instruments attached to the robotic arm including whether they are attached to a robotic arm, the patient position, and the orientation and height of the patient table, sensor readings related to the forces applied at the proximal and distal ends of surgical instruments attached to the robotic arm, the forces required to hold each instrument in place, endoscopic video streams, algorithm parameters, and operating room 3D streams captured using optical scanning devices.

[0139] Such data may be used to develop a database of historical data, which in some implementations may be used to develop algorithms used to control one or more aspects of the system's operation. Additionally, such data may be used to control one or more aspects of the system's operation / one or more algorithms of the system during a procedure. For example, data may be used to assess the level of fatigue of a user of the system, as described in U.S. Patent No. 11,504,197, the entire contents of which are incorporated herein by reference.

[0140] As an operator manipulates the robotic arm of a collaborative surgical robotic system by applying movements to a surgical instrument coupled to the robotic arm, the system may automatically transition the robotic arm between various operating modes upon determination of a predefined condition. For example, the system may transition the robotic arm to the passive mode in response to determining that movement of the robotic arm due to movement at the surgical instrument handle is less than a predetermined amount for at least a predetermined dwell time, such that in the passive mode the robotic arm maintains a stationary position and prevents damage to equipment and / or injury to the patient. Additionally, the system may transition the robotic arm to the collaborative mode in response to determining that a force applied at the robotic arm due to a force applied at the surgical instrument handle exceeds a predetermined threshold, such that in the collaborative mode a first impedance is applied to the robotic arm to account for the weight of the surgical instrument and the robotic arm. Additionally, the system may transition the robotic arm to the haptic mode in response to determining that at least a portion of the robotic arm is outside a predefined haptic barrier such that in the haptic mode a second impedance greater than the first impedance is applied to the robotic arm, thereby making movement of the robotic arm more tenacious in the haptic mode than in the collaborative manipulation mode in response to movement at the handle of the surgical instrument. The system may further transition the robotic arm to the robot-assisted mode in response to detecting various conditions that warrant automated movement of the robotic arm, for example, to guide a surgical instrument attached thereto along a planned trajectory or to avoid collision with another object or person in the surgical space.For example, in an instrument centering mode of the robotically assisted mode, a robotic arm coupled to a laparoscope can automatically move the laparoscope along a planned trajectory, track an identified surgical instrument, maintain the instrument within the laparoscope's field of view, and provide assisted instrument centering. As described in more detail below, the system can further transition the robotic arm into one or more setup modes for manual and / or automatic reconfiguration of the robotic arm to an optimized position for a given surgical procedure.

[0141] 2A and 2B, a collaborative surgical robotic system 100 is provided. As shown in FIGURES 2A and 2B, system 100 may include a platform 200, e.g., a surgical cart, sized and shaped to support one or more robotic arms, e.g., robotic arm 300a and robotic arm 300b (collectively referred to herein as robotic arms 300), each having a surgical instrument coupler interface, e.g., coupler interface 400a and coupler interface 400b, for removably coupling to a surgical instrument, and system 100 may include a computing system operably coupled to platform 200 and robotic arm 300. As shown in FIG. 2A, system 100 may further include one or more optical scanners, e.g., optical scanner 202a and optical scanner 202b, for capturing depth data, and a graphical user interface display 210, which displays operational information and receives user input.

[0142] 2A and 2B, platform 200 may include a stage assembly, e.g., one or more stages, coupled to base portions of one or more robotic arms (e.g., base portion 302a of robotic arm 300a and base portion 302b of robotic arm 300b) to provide movement (e.g., at least horizontally and vertically relative to platform 200) for the respective robotic arms. Each stage may include a vertical extension, e.g., vertical extension 206a and vertical extension 206b, for moving robotic arm 300a and robotic arm 300b, respectively, independently vertically relative to platform 200, and a horizontal extension, e.g., horizontal extension 208a and horizontal extension 208b, for moving robotic arm 300a and robotic arm 300b, respectively, independently horizontally relative to platform 200, thereby allowing operator flexibility in positioning robotic arm 300 relative to the patient. Thus, platform 200 can move each of robotic arms 300a and 300b independently in any direction, including a first or vertical direction toward and away from the floor (e.g., along the z-axis), and / or a second or horizontal direction toward and away from the patient (e.g., along the x-axis) as shown in FIG. 2C, and / or a third or horizontal direction along the length of the patient (e.g., along the y-axis).

[0143] In some embodiments, platform 200 may simultaneously move robot arms 300a and 300b in the same direction and / or may further cause rotational movement of robot arms 300a and 300b. For example, a stage may be rotatable relative to platform 200, thereby causing robot arms 300a and 300b to rotate toward or away from each other, e.g., along a horizontal plane defined by the x-axis and y-axis. The centers of rotation of each stage may be spaced apart from each other or, alternatively, may be coaxial with each other. Thus, in some embodiments, platform 200 may have a smaller footprint in the retracted mode such that robot arms 300a, 300b may be entirely within the footprint of platform 200, and the stage may rotate and extend base portions 302a, 302b of robot arms 300a, 300b away from each other along a horizontal plane to achieve the spaced-apart configuration generally provided by a wider footprint platform, as described in more detail below with respect to Figures 46A and 46B.

[0144] Referring again to FIG. 2A , platform 200 may include a plurality of wheels 204, e.g., caster wheels, to provide mobility for platform 200, and thus robotic arm 300, within the operating room. Each of wheels 204 may include a braking mechanism that may be activated to prevent movement of platform 200 via wheels 204. Preferably, wheels 204 may be manually activated by an operator to mechanically engage / disengage the respective braking mechanism. For example, as shown in FIG. 2B , platform 200 may include a locking pedal 211 a configured to be activated (e.g., stepped on) by a user to engage the braking mechanism, and an unlocking pedal 211 b configured to be activated (e.g., stepped on) by a user to disengage the braking mechanism. The locking pedal 211a and the unlocking pedal 211b may be configured such that movement of the locking pedal 211a in a first direction causes movement of the unlocking pedal 211b in a second direction opposite the first direction, and vice versa.

[0145] Additionally or alternatively, wheels 104 may be powered such that they can be actuated to electrically engage / disengage their respective braking mechanisms. Once preparation for surgery is complete, platform 200 may be moved to a desired position at the side of the patient bed and locked in place via wheels 204, and the vertical and horizontal positions of robotic arms 300a and 300b may be adjusted to the optimal position for the procedure relative to the patient via vertical extensions 206a, 206b and horizontal extensions 208a, 208b in response to user input received via graphical user interface display 210 and / or via user-guided stage controls as described in more detail below. As described in more detail below, platform 200 may automatically move robotic arms 300a and 300b in response to, for example, detection of potential collisions with other objects and / or people in the operating room and / or user input applied via the robotic arms during the laparoscopic procedure and / or during robotic arm setup.

[0146] Additionally, system 100 may include multiple depth sensors, e.g., proximity sensors 212, disposed on platform 100. Proximity sensors 212 may be, for example, depth cameras, stereoscopic RGB cameras, LiDAR devices, and / or electromagnetic, capacitive, ultrasonic, or infrared proximity sensors, etc. For example, a first set of proximity sensors 212 may be positioned on robot arm 300a, e.g., on the lower portion of base portion 302a, and a second set of proximity sensors 212 may be positioned on robot arm 300b, e.g., on the lower portion of base portion 302b, thereby improving detection of objects approaching the vicinity of robot arms 300a, 300b. For example, as shown in FIG. 2D , each base portion 302 may include a pair of proximity sensors, e.g., a front proximity sensor 212a for detecting and determining the proximity of objects in front of and around base portion 302 and a bottom proximity sensor 212b for detecting and determining the proximity of objects below and around base portion 302. As will be understood by one skilled in the art, each base portion may have fewer or more than two proximity sensors. In some embodiments, proximity sensor 212 may be active only when the system is unlocked during movement of a stage of platform 200, as described in further detail below. Alternatively, proximity sensor 212 may be active only during movement of platform 200, for example, when the braking mechanism of wheels 204 is disengaged.

[0147] Because the base portions 302a, 302b are generally lower than the more distal components of the robotic arms 300a, 300b, they may be more susceptible to collisions, for example, with a patient bed, as the stage of the platform 200 moves the robotic arms 300a, 300b horizontally and vertically relative to the platform 200. Thus, the system may generate an alert, for example, via the indicator 334, when a proximity sensor detects that the proximity between the robotic arm and one or more objects in the operating room is below a predetermined distance threshold, as described in further detail below. For example, the indicator 334 may illuminate in a predetermined color and / or pattern, e.g., blinking, to indicate proximity with one or more objects, and the frequency of the blinking may increase as the proximity becomes closer. Additionally, the system may cause the stage assembly of the platform 200 to stop movement of the robotic arm relative to the platform 200 when the proximity between the robotic arm and one or more objects in the operating room is below a predetermined distance threshold. Additionally, the system may further display, for example, via the GUI 210, an indication that the object is within a predetermined proximity of the robot arm, as determined by the front proximity sensor 212a and / or the bottom proximity sensor 212b.

[0148] Surgical robotic system 100 is configured for cooperative manipulation, whereby system 100 can assist a user or operator, e.g., a surgeon and / or a surgical assistant, by allowing the user to freely move robotic arm 300 a and / or robotic arm 300 b due to manipulation of one or more surgical instruments coupled to the robotic arm in response to forces applied by the user to the surgical instruments. Thus, system 100 can be configured such that it is not remotely controlled, whereby robotic arm 300 moves directly in response to the operator's movements of the surgical instruments coupled to robotic arm 300, while compensating for the mass of the surgical instruments and respective robotic arms and providing local impedance along the robotic arms, thereby increasing the precision of the operator's movements or actions as they manipulate the surgical instruments.

[0149] System 100 may be particularly useful in laparoscopic and / or other surgical procedures utilizing elongated instruments that may be inserted into a patient's body, for example, via a cannula, to enable surgical intervention. As will be understood by those skilled in the art, system 100 may be used for any desired or appropriate surgical procedure. Furthermore, system 100 may be used in conjunction with or in cooperation with video surveillance provided by one or more cameras and / or one or more endoscopes so that an operator of system 100 may view and monitor the use of instruments coupled to robotic arms 300a, 300b via respective coupler interfaces 400a, 400b. For example, robotic arm 300a may be removably coupled to and manipulate an endoscope, while robotic arm 300b may be removably coupled to and manipulate a surgical instrument.

[0150] As shown in FIG. 2A, system 100 may further include one or more optical scanners: e.g., optical scanners 202a, 202b (collectively referred to herein as optical scanners 202); a LiDAR scanner or other suitable optical scanning device (e.g., an RGBD camera or sensor, an RGB camera with machine learning, a time-of-flight depth camera, structured light, multiple projection cameras, stereoscopic cameras, ultrasonic sensors, laser scanners, other types of coordinate measuring area scanners, or any combination of the foregoing, etc.) to provide a video stream of the surgical scene, e.g., via streaming, for monitoring and analysis. For example, the LiDAR camera / scanner may be capable of recording both color (RGB) and depth (D) of the surgical field and may include, for example, an Intel RealSense LiDAR Camera L515 or Intel RealSense Depth Camera D435i (commercially available from Intel, Santa Clara, California) or other LiDAR or depth camera having similar or suitable specifications, including, but not limited to, any of the following specifications: (i) range: 25 cm to 500 cm; depth accuracy: 5 mm or approximately 5 mm; depth field of view: 70 x 55 or approximately 70 x 55 (degrees); depth output resolution: 1,024 x 768 pixels or approximately 1,024 x 768 pixels; depth / RGB frame rate: 30 frames per second; RGB frame resolution: 1,920 x 1,080; and / or RGB field of view: 70 x 43 degrees or approximately 70 x 43 degrees. The LiDAR scanner or optical scanner may further include both 1 / 4-20 UNC thread or 2×M3 thread mounting points.

[0151] The optical scanner 202 and any other electronics, wiring, or other components of the system may be supported via the platform 200, such that the optical scanner 202 is mounted in a fixed location relative to other objects in the surgical space, and the position and orientation of the optical scanner 202 may be known or determined relative to the global coordinate system of the system, and therefore the robotic arm. This allows all data streams to be converted to a single coordinate system for development purposes. Furthermore, telemetry data captured by the optical scanner 202 showing, for example, the movement of the surgeon's hands, other body parts, the patient bed, cutouts in a sterile drape over the patient on the surgical bed, skin exposed through cutouts in the sterile drape, trocars, surgical instruments, and other components of the system may be recorded to provide a rich and detailed data set describing the precise movements and forces applied by the surgeon throughout the procedure.

[0152] As shown in FIG. 2A, a first optical scanner, e.g., optical scanner 202a, may be supported on an upper portion of platform 200, e.g., via lighthouse 203, and may be adjusted, e.g., up / down, in / out, right / left, to adjust the field of view of optical scanner 202a and enable optical scanner 202a to obtain an optimal view or position relative to other components of the system, e.g., robotic arms 300a, 300b, surgical instruments attached to the robotic arms, the surgeon, and / or a surgical assistant. For example, the optical scanner 202a may collect depth data indicative of, e.g., the height of the surgical bed, the angle of the surgical bed (cranial to caudal and medial to lateral), the plane of the surgical bed, the head end of the surgical bed, the position and orientation of the surgical bed, the location of one or more trocar ports, the movement of a surgical instrument coupled to the distal end of the robotic arm, the movement of a handheld surgical instrument not coupled to the robotic arm, e.g., held by a user, the attachment and detachment of a surgical instrument to the distal end of the robotic arm, etc.

[0153] As shown in FIG. 2A , lighthouse 203 may include an indicator 334, e.g., an LED ring, disposed thereon to display a visual alert, as described in further detail below. Additionally, system 100 may include one or more robotic arm markers, e.g., markers 205, configured to indicate which robotic arm (e.g., robotic arms 300 a, 300 b) is in operation / active. For example, markers 205 may include a visual representation associated with each robotic arm, e.g., a Roman numeral I associated with robotic arm 300 a and a Roman numeral II associated with robotic arm 300 b, each of which may illuminate to indicate that the respective robotic arm is in operation, e.g., being moved by an operator and / or the system. Markers 205 may be disposed on the front side of lighthouse 203 as shown in FIG. 2A , and may further be disposed on the rear side of lighthouse 203 and / or on power button panel 207, as shown in FIG. 2B , so that markers 205 are visible to a user standing behind platform 200. As shown in FIG. 2A, the lighthouse 203 may further include a drape hook 209 sized and shaped to support a sterile drape, for example, below the marker 205 on the front side of the lighthouse 203, as described in more detail below with respect to FIG. 10F.

[0154] 2A , a second optical scanner, e.g., optical scanner 202b, may be supported on a lower portion of platform 200, allowing optical scanner 202b to provide the system with a more complete view of the operating room that may not be captured by first optical scanner 202a, e.g., objects on the operating room floor, such as the patient table and electrical cables. For example, optical scanner 202a may collect depth data indicating, e.g., the distance / proximity between system 100 and the surgical bed, the relative angle between system 100 and the surgical bed, the nearest feature on the surgical bed and its edges, the head and caudal ends of the surgical bed, one or more objects / persons between system 100 and the surgical bed, one or more objects / persons on the other side of the surgical bed, etc. As will be understood by one skilled in the art, three or more optical scanners may be used to further improve the field of view of the system.

[0155] Data acquired by the optical scanner can be used to optimize procedures performed by the system, including, for example, automatic servoing (i.e., movement) of one or more portions of the robotic arm 300. By tracking a surgeon's tendency to keep a tool within a particular region of interest and / or the surgeon's tendency to avoid moving a tool into a particular region of interest, the system can optimize the automatic servo control algorithms to provide more stability within the particular region of interest. Additionally, data acquired can be used to optimize procedures performed by the system, including, for example, automatic recentering of the field of view of the system's optical scanning device. For example, if the system detects that the surgeon has moved outside of the field of view or predicts that the surgeon may move outside of the field of view, the system can automatically adjust the optical scanning device, e.g., a robotic arm supporting a laparoscope, to track the desired location on the image as the surgeon performs the desired procedure, as described in further detail below. This behavior can be surgeon-specific and require understanding of a particular surgeon's preferences for the operating region of interest. Additionally or alternatively, this behavior can be procedure-specific. Thus, the system can control the robotic arm according to the specific operational requirements and / or preferences of a particular surgeon. Additionally, if the system detects that the robotic arm is in an extended position for a period of time that exceeds a predetermined threshold, the system can move the robotic arm in a manner that facilitates extension of the robotic arm to a stage coupled to a base portion of the robotic arm, thereby providing additional range for extension of the robotic arm by the user.

[0156] Referring now to FIG. 3 , a surgical support arm is provided. As described above, system 100 may include multiple robotic arms, e.g., robotic arm 300a and robotic arm 300b. However, because each robotic arm may be identically constructed, only a single robotic arm, e.g., robotic arm 300, will be described collectively with reference to FIG. 3 as robotic arm 300 for brevity. Aspects of the robotic arm described herein may utilize structures from U.S. Pat. No. 10,118,289 to Louveau, U.S. Pat. No. 11,504,197 to Noonan, U.S. Pat. No. 11,622,826 to Basafa, U.S. Pat. No. 11,812,938 to Wu, and WO 2023 / 203491 to Gayet (the entire contents of each of which are incorporated herein by reference). Robotic arm 300 may include multiple arm segments / links extending from a base portion and multiple articulation joints. For example, the robotic arm 300 may include a base portion, a shoulder portion, an elbow portion, and a wrist portion, thereby mimicking the kinematics of a human arm. As shown in FIG. 3 , the robotic arm 300 may include a base, which includes a base portion 302 rotatably coupled to a shoulder portion 304 at a base joint 303. For example, the shoulder portion 304 may be on the base portion 302 and may be rotated relative to the base portion 302 about axis Q1 at the base joint 303. In some embodiments, the robotic arm 300 may be swapped, replaced, or coupled to a base in any desired arrangement.

[0157] The robotic arm 300 may include a shoulder link 305, which further includes a proximal shoulder link 306 rotatably coupled to a distal shoulder link 308. A proximal end of the proximal shoulder link 306 may be rotatably coupled to the base shoulder portion 304 at a shoulder joint 318 such that the proximal shoulder link 306 may be rotated relative to the shoulder portion 304 about an axis Q2 at the shoulder joint 318. As shown in FIG. 3 , the axis Q2 may be perpendicular to the axis Q1. A distal end of the proximal shoulder link 306 may be rotatably coupled to the proximal end of the distal shoulder link 308 at a joint 320 such that the distal shoulder link 308 may be rotated relative to the proximal shoulder link 306 about an axis Q3 at the joint 320. As shown in FIG. 3 , the axis Q3 may be parallel to the longitudinal axis of the shoulder link 305.

[0158] Additionally, the robotic arm 300 may include an actuator 330, such as a collar, lever, button, or switch, operably coupled to a motor operably coupled to the distal shoulder link 308 and / or the proximal shoulder link 306 at joint 320, such that the distal shoulder link 308 may be rotated only relative to the proximal shoulder link 306 upon actuation of the actuator 330. The actuator 330 may be configured to allow dual actuation, for example, a first actuation causes the distal shoulder link 308 to rotate relative to the shoulder link 306 in a first direction, and a second actuation causes the distal shoulder link 308 to rotate in a second direction opposite the first direction. For example, as shown in FIG. 3 , the actuator 330 may be a collar rotatably coupled to a link of the robotic arm 300, such as the elbow link 310 described below, such that rotation of the collar 330 in a first direction about the longitudinal axis of the link 310 causes the distal shoulder link 308 to rotate relative to the proximal shoulder link 306 in a corresponding first direction, and rotation of the collar 330 in a second direction opposite the first direction about the longitudinal axis of the link 310 causes the distal shoulder link 308 to rotate relative to the proximal shoulder link 306 in a corresponding second direction opposite the first direction.

[0159] 3, the collar 330 may include a set mode actuator 336, e.g., a button, disposed thereon, and the system may require the set mode actuator 336 to be actuated to enable rotation of the collar 330 to cause a corresponding rotation of the distal shoulder link 308 relative to the proximal shoulder link 306. For example, a user may be required to actuate the set actuator 336 to switch the system into a user-guided set mode and maintain the set actuator 336 in an actuated state while the collar 330 is rotated to cause a corresponding rotation of the distal shoulder link 308 relative to the proximal shoulder link 306. The set actuator 336 may include redundant circuitry whereby both circuits may need to be closed upon actuation of the set actuator 336 (e.g., by pressing and actuating the set actuator 336). In addition to actuating the setting actuator 336 to enable rotation of the collar 330 and cause rotation of the distal shoulder link 308 relative to the proximal shoulder link 306, in some embodiments, actuation of the setting actuator 336 in a predefined manner / pattern can serve as an input to the system to enable additional functionality. For example, rapidly actuating the setting actuator 336, e.g., twice, in a predefined pattern can initiate a predetermined function of the system, e.g., an instrument centering mode. Furthermore, actuating the setting actuator 336 in another, different predefined pattern can initiate another predetermined function of the system, e.g., recording video data captured by one or more optical sensors of the system, and actuating the setting actuator 336 in another, different predefined pattern can stop recording video.

[0160] Additionally, the collar 330 may be spring-loaded such that upon release of the collar 330 at any position, the collar 330 returns to a neutral position relative to the link 310, thereby preventing the distal shoulder link 308 from rotating relative to the proximal shoulder link 306. Alternatively, or additionally, instead of actuating the collar 330 to cause rotation of the distal shoulder link 308 relative to the proximal shoulder link 306, in some embodiments, upon actuation of the setting actuator 336, application of a force (e.g., left / right force) to the distal end of the robotic arm may cause the distal shoulder link 308 to rotate in a corresponding direction relative to the proximal shoulder link 306. The distal shoulder link 308 may continue to rotate relative to the proximal shoulder link 306 until the applied force is released and / or a counteracting force in the opposite direction is applied to the distal end of the robotic arm and / or until maximum rotation is reached.

[0161] Axis Q3 may therefore be a “setup” axis such that the distal shoulder link 308 can be rotated and fixed relative to the proximal shoulder link 306 during a setup phase prior to an operation phase in which the robotic arm 300 is used in a surgical procedure, as described in further detail with respect to FIGS. 11A and 11B . Additionally, the system may switch between the operation and setup phases during a surgical procedure to allow reconfiguration of the robotic arm via the setup joint, as needed. When the setup actuator 336 is in an unactuated state, the setup joint 320 prevents relative movement between the distal shoulder link 308 and the proximal shoulder link 306 such that the distal shoulder link 308 is fixed relative to the proximal shoulder link 306. Upon actuation of the setup actuator 336 and the actuator 330, the distal shoulder link 308 can be automatically rotated relative to the proximal shoulder link 306 until the actuator 330 is released. Alternatively, the actuator 330 may be operably coupled to the distal shoulder link 308 and / or the proximal shoulder link 306 such that, upon actuation of the actuator 330, the distal shoulder link 308 may be manually rotated in predefined increments relative to the proximal shoulder link 306.

[0162] The robotic arm 300 may further include an elbow link 310. A proximal end of the elbow link 310 may be rotatably coupled to a distal end of the distal shoulder link 308 at an elbow joint 322 such that the elbow link 310 may be rotated relative to the distal shoulder link 308 about axis Q4 at the elbow joint 322. The robotic arm 300 may further include a wrist portion 311 which may include a proximal wrist link 312 rotatably coupled to a distal end of the elbow link 310 at a wrist joint 324, a middle wrist link 314 rotatably coupled to the proximal wrist link 312 at a joint 326, and a distal wrist link 316 coupled to / extending from the middle wrist link 314 at a joint 328, which may be rotatably coupled to a surgical instrument coupler interface 400 (not shown), as further shown in FIGS. 5A and 5B . Thus, the wrist portion 311 can be rotated relative to the elbow link 310 about axis Q5 at wrist joint 324, the central wrist portion 314 can be rotated relative to the proximal wrist link 312 about axis Q6 at joint 326, and the surgical instrument coupler interface 400 can be rotated relative to the distal wrist link 316, and therefore the central wrist link 314, about axis Q7 at joint 328.

[0163] 3 , the robotic arm 300 may include an actuator 332, e.g., a lever, button, or switch, operably coupled to the elbow link 310 and / or the proximal wrist link 312 at joint 324, such that the proximal wrist link 312 can only be rotated relative to the elbow link 310 upon actuation of the actuator 332. Thus, axis Q5 may be a “set” axis such that the proximal wrist link 312 can be rotated and fixed relative to the elbow link 310 during a set phase upon actuation of the actuator 332, e.g., prior to an operation phase in which the robotic arm 300 is used in a surgical procedure. When the actuator 332 is in an unactuated state, the set joint 324 prevents relative movement between the proximal wrist link 312 and the elbow link 310 such that the proximal wrist link 312 is fixed relative to the elbow link 310. In some preferred embodiments, upon actuation of the actuator 332, the proximal wrist link 312 may be manually rotated relative to the elbow link 310 in predefined increments, thereby eliminating the need for additional motors and / or electronics in the distal region of the robotic arm 300. Alternatively, upon actuation of the actuator 332, the proximal wrist link 312 may be automatically rotated relative to the elbow link 310 until the actuator 332 is released, for example, via a motor operably coupled to the proximal wrist link 312 and / or elbow link 310 at joint 324.

[0164] 3, the robotic arm 300 may include multiple motors, such as motors M1, M2, and M3, which may all be located within the base of the robotic arm 300, and motor M4, which may preferably be located adjacent to joint 320. Alternatively, motor M4 may also be located within the base of the robotic arm 300. Each of motors M1, M2, and M3 may be operatively coupled to a respective motorized joint of the robotic arm 300, such as base joint 303, shoulder joint 318, and elbow joint 322, thereby applying a local impedance at the respective joint. For example, motors M1, M2, and M3 may create an impedance / torque at either base joint 303, shoulder joint 318, or elbow joint 322, respectively, thereby effectively applying an impedance at the distal end of the robotic arm, such as at the point of attachment with a surgical instrument, which may improve not only the sensations experienced by the operator during manipulation of the surgical instrument, but also the operator's performance during the surgical procedure. For example, impedance may be applied to the distal end of the robotic arm 300, and thus the surgical instrument coupled thereto, to provide a sense of stickiness, stiffness, and / or inertia to the operator manipulating the surgical instrument. Additionally, the applied impedance may simulate tissue density or stiffness, communicate surgical boundaries to the operator, and be used to guide the surgical instrument along a desired path or otherwise. In some embodiments, motors may actuate the respective joints, thereby causing movement of the robotic arm 300 about the respective joints. Thus, each of axes Q1, Q2, and Q4 may be a "motor" axis, such that motors M1, M2, and M3 may apply impedance / torque to the base joint 303, shoulder joint 318, and elbow joint 322, respectively, to prevent or actuate rotation about the respective axis. As described in further detail below, motors M1, M2, and M3 may be controlled by a processor of the collaborative robotic platform.By using three motor axes, some implementations of the robotic arm 300 can apply force / torque in three directions at the distal end of the robotic arm 300, thereby moving a surgical instrument coupled to the distal end of the robotic arm 300 in three degrees of freedom.

[0165] Motor M4 is operably coupled to set joint 320 and can apply a torque to joint 320 to actuate rotation of distal shoulder link 308 relative to proximal shoulder link 306 about axis Q3. Unlike other motorized joints described herein, e.g., base joint 303, shoulder joint 318, and elbow joint 322, motorized joint 320 is preferably not “backdrivable,” in that a user cannot actuate motorized joint 320 when the system is in a collaborative operation mode, e.g., via movement of a surgical instrument coupled to the robotic arm. Instead, as described above, actuation of motorized joint 320 can be via one or more actuators, e.g., actuator 330 and / or actuators displayed on GUI 210, which can be actuated to automatically cause rotation of distal shoulder link 308 relative to proximal shoulder link 306.

[0166] Each of axis Q6 and axis Q7 may be a “passive” axis whereby the middle wrist link 314 may be rotated relative to the proximal wrist link 312 at a passive joint 326 without any applied impedance from the system 100, and the surgical instrument coupler interface 400 may be rotated relative to the distal wrist link 316 at a passive joint 328 without any applied impedance from the system 100. The distal end of the distal wrist link 316 may be removably coupled to a surgical instrument and coupler interface 400, as described in further detail below, for example, rotatably coupled to a surgical instrument coupler interface for removably coupling with a surgical instrument via a coupler body 500 as shown in FIGS. 5A and 5B. Alternatively, the wrist portion 311 may include a passive ball joint at the point of attachment with the surgical instrument, as described in U.S. Pat. No. 10,582,977, the entire disclosure of which is incorporated herein by reference.

[0167] 3, the robotic arm 300 may further include multiple encoders, e.g., encoders E1-E7, disposed on at least some of the joints of the robotic arm 300. For example, encoder E1 for measuring angulation between the base portion 302 and the shoulder portion 304 may be disposed on or adjacent to base joint 303 in the base, encoder E2 for measuring angulation between the shoulder portion 304 and the proximal shoulder link 306 may be disposed on or adjacent to shoulder joint 318 in the base, encoder E3 for measuring angular rotation between the proximal shoulder link 306 and the distal shoulder link 308 may be disposed on or adjacent to joint 320, and encoder E4 for measuring angulation between the distal shoulder link 308 and elbow link 310 may be disposed on or adjacent to joint 320, and encoder E5 for measuring angulation between the distal shoulder link 308 and elbow link 310 may be disposed on or adjacent to elbow joint 322. 322, an encoder E5 for measuring angular rotation between the elbow link 310 and the proximal wrist link 312 may be located on or adjacent to the wrist joint 324, an encoder E6 for measuring angulation between the proximal wrist link 312 and the middle wrist link 314 may be located on or adjacent to the joint 326, and an encoder E7 for measuring angulation between the distal wrist link 316 and the surgical instrument coupler interface may be located on or adjacent to the joint 328. Alternatively, encoder E4 may be located on or adjacent to the elbow joint 322. The encoders may be absolute encoders or other position / angle sensors configured to generate data for accurately determining the position and / or angulation of the corresponding link at each joint and / or the precise position of a surgical instrument coupled to the distal end of the robotic arm 300. Thus, the exact position of each link, joint, and distal end of the robot 300 can be determined based on measurements obtained from multiple encoders.Preferably, redundant encoders are placed at each location along the robot arm 300 where an encoder is installed, as described in more detail below, to provide more accurate position data and to detect fault conditions.

[0168] Prior to attachment to a surgical instrument, the robotic arm 300 may be manually manipulated by a user, for example, to position the robotic arm 300 at a desired position for coupling with a surgical instrument. For example, a user may manually manipulate the robotic arm 300 via the wrist portion 311, the actuator 330, and / or the actuator 332. Upon actuation of the actuator 330, the user may automatically rotate the distal shoulder link 308, and upon actuation of the actuator 332, the user may manually manipulate the proximal wrist portion 312. Additionally, the robotic arm 300 may be further manually moved by application of forces directly onto other links and / or joints of the robotic arm 300.

[0169] In some embodiments, in the user-guided setup mode, in response to a force applied by the user to a distal region of the robotic arm 300 (e.g., wrist portion 311, wrist joint 324, elbow link 310, anywhere distal to Q4, such as a surgical instrument) exceeding a predetermined force threshold, for example, or in a predetermined pattern in a given direction (e.g., in / out and / or up / down), the processor of the collaborative robotic platform may move the robotic arm 300 in the same / corresponding direction, e.g., via the vertical extensions 206 a, 206 b and horizontal extensions 208 a, 208 b of the platform 200 coupled to the base portion 302 of the robotic arm 300, until the system detects that the force applied by the user to the robotic arm 300 falls below the predetermined threshold (e.g., when the user releases the robotic arm 300). Due to the lever arm effect, forces applied to the robot arm 300 further from Q4 exceed forces applied closer to Q4, and therefore may be preferable to moving the stage of the platform 200 during the user-guided setup mode. In some embodiments, the system may, upon application of a force at the distal region of the robot arm, cause movement of the base of the robot arm via the stage assembly at a speed corresponding to the amount of force applied to the distal region of the robot arm in the user-guided setup mode. Thus, the speed of movement of the stage assembly may be controlled by adjusting the amount of force applied to the distal region of the robot arm in the user-guided setup mode, and further, the speed of movement of the stage assembly may slow as the stage assembly reaches or near its maximum extension range.

[0170] As described above, in some embodiments, the processor of the collaborative robotic platform may also rotate the distal shoulder link relative to the proximal shoulder link in response to a force applied by a user to a distal region of the robotic arm 300 exceeding a predetermined force threshold or in a predetermined pattern, given direction, e.g., left / right. In some embodiments, the system may stop movement of the robotic arm 300 in the same direction as the user-applied force when the user applies a counterforce to the robotic arm 300, e.g., in a direction opposite to the direction of movement of the robotic arm 300, to facilitate setting up the robotic arm 300 relative to the patient. This feature may be activated / deactivated via user actuation (e.g., by activating the actuator 336 on the collar 330, by voice command, etc.). In a preferred embodiment, the system switches to a user-guided setting mode only when the actuator 336 is in an activated state (e.g., being actively pressed by the user). Thus, a user can actuate actuator 336 with one hand while actuator 336 is actuated, while simultaneously applying a force to the distal region of the robot arm with the other hand to cause movement of a stage of platform 200.

[0171] For example, during operation of the user-guided setup mode, a user may apply a force to the wrist portion 311 in a first direction that exceeds a predetermined force threshold, e.g., by applying a pulling or pushing force, which causes the stage of the platform 200 to move the robot arm 300 in that same direction until the user stops moving the wrist portion 311, e.g., by releasing the robot arm 300 or by applying a counter force to the robot arm 300, and / or the maximum extension of the stage assembly is reached, thereby causing the system to stop moving the stage of the platform 200. For example, a subsequent pushing force may counter the initial pulling force, and a subsequent pulling force may counter the initial pushing force. Furthermore, the stage of the platform 200 may stop moving the robot arm 300 when the force applied to the distal region of the robot arm 300 falls below a predetermined release threshold, which may include releasing the robot arm 300. Thus, forces applied to the distal region of the robotic arm, e.g., wrist portion 311, wrist joint 324, elbow link 310, etc., can serve as input for generated movement of the robotic arm in a particular direction via the stage coupled thereto. Such automated movement of the stage of platform 200 in response to forces applied by a user to the distal end of robotic arm 300 may be limited to when the system is in a predefined operating mode (e.g., a user-guided setup mode), which may be input during setup and / or during a surgical procedure, e.g., upon actuation of actuator 336, GUI 210, and / or via voice control.

[0172] Similarly, when a user applies an opposing force exceeding a predetermined threshold in a predefined direction different from the direction causing horizontal (x-axis) and vertical (z-axis) movement of the stage of platform 200, the system can automatically actuate motorized joint 320 to cause rotation of distal shoulder link 308 relative to proximal shoulder link 306 to facilitate movement of robot arm 300 in the predefined direction. For example, similar to how the system can move robot arm 300 in response to a user moving a distal region of robot arm 300 on a stage of platform 200, e.g., forward / backward along the x-axis or up / down along the z-axis, as described above, the system can cause motorized joint 320 to rotate distal shoulder link 308 relative to proximal shoulder link 306 to move robot arm 300 along the y-axis in response to a user moving the distal end of robot arm 300 along the y-axis. Thus, the system may deactivate the motorized joint 320 when the force applied by the user to the distal region of the robotic arm 300 falls below a predetermined threshold. M4 may be controlled by the processor of the collaborative robotic platform.

[0173] When attached to a surgical instrument, the robotic arm 300 can still be manually manipulated by a user by directly exerting forces, such as one or more linear forces and / or one or more torques, on the robotic arm 300. However, during a laparoscopic procedure, the operator preferably manipulates the robotic arm 300 solely through the handle of the surgical instrument, which applies forces / torques to the distal end of the robotic arm 300 and thus to the links and joints of the robotic arm 300. When the operator applies forces to a surgical instrument attached to the robotic arm 300, thereby causing movement of the surgical instrument, the robotic arm 300 will move in response to the movement of the surgical instrument, providing the operator with the ability to freely move the surgical instrument relative to the patient. As described in further detail below, the robotic arm 300 may apply impedance to account for the weight of the surgical instrument and the robotic arm 300 itself, e.g., gravity compensation, as the operator moves the surgical instrument, thereby making it easier for the operator to move the instrument despite gravity and / or inertial forces exerted on the robotic arm and / or surgical instrument. As will be understood by one skilled in the art, the robotic arm 300 may include fewer or more articulating joints and a corresponding number of motors and encoders / sensors than those shown in FIG.

[0174] Additionally, each of the robotic arms 300 may further include an indicator 334 for visually indicating, in real time, an operational mode associated with the respective robotic arm. For example, the indicator 334 may be located on at least one of the shoulder portion 304, shoulder link 305, elbow link 310, platform 200, lighthouse 203, display 210, etc. As shown in FIG. 4 , the robotic arm 300a may include an indicator 334a on the shoulder portion 304a, e.g., adjacent shoulder joint 318a, on the shoulder link 305a, e.g., adjacent shoulder joint 318a, and on the elbow link 310a, e.g., adjacent elbow joint 322a, and the robotic arm 300b may include an indicator 334b ​​on the shoulder portion 304b, e.g., adjacent shoulder joint 318b, on the shoulder link 305b, e.g., adjacent shoulder joint 318b, and on the elbow link 310b, e.g., adjacent elbow joint 322b. In some embodiments, the status of the system conveyed by the indicator on the lighthouse 203 may be different from the status of the system / robot arm conveyed by indicators elsewhere on the system, e.g., on the shoulder portion, base portion, elbow link, etc. For example, the indicator on the lighthouse 203 may be programmed to illuminate a predetermined amount of color that is less than the predetermined amount illuminated by other indicators in the system, thereby conveying a predetermined status of the overall system, while the other indicators in the system may be illuminated in different colors and convey specific status of the system and robot arm, e.g., when a coupler is mounted on the robot arm, the current operating mode of the robot arm, etc.

[0175] Additionally, indicators 334, 334a, 334b ​​may include lights, e.g., LED lights, that may illuminate in a variety of different colors and in different patterns, e.g., solid or flashing. For example, each operational mode of system 100 may be associated with a uniquely colored light, such as red, yellow, blue, green, purple, white, amber, etc., as described, for example, in U.S. Pat. No. 11,504,197, the contents of which are incorporated herein by reference. Thus, indicators 334, 334a, 334b ​​may indicate a transition from one operational mode to another. Additionally or alternatively, a transition from one operational mode to another may be indicated to a user via tactile feedback, e.g., vibrations delivered to the distal end of the robotic arm and, therefore, to a surgical instrument coupled thereto. For example, the distal end of the robotic arm may vibrate upon release by the user and / or when the robotic arm transitions from collaborative manipulation mode to static mode after the system identifies a hold as part of the instrument detection phase of the instrument centering mode described below, to assure the user that the robotic arm is in static / passive mode and will remain in place. Additionally or alternatively, an audible alert may be emitted to indicate to the user when the robotic arm transitions from one operating mode to another.

[0176] 5A and 5B, close-up views of the coupling mechanism of coupler interface 400 and coupler body 500 are provided. The coupling mechanism may be constructed as described in U.S. Patent No. 11,812,938. For example, the coupling mechanism may include coupler interface 400 at the distal end of the distal-most link (illustratively, link 316) of the robotic arm and coupler body 500, which may be configured to be removably coupled to a surgical instrument and coupler interface 400 such that a sterile drape may be placed between coupler interface 400 and coupler body 500. Thus, coupler body 500 may be disposable or, alternatively, capable of being sterilized between surgical procedures. Additionally, the coupling mechanism may be operably coupled to one or more sensors, as described in further detail below, that not only detect when the coupler body 500 is coupled to the coupler interface 400, when a surgical instrument is coupled to the coupler body 500, but also the type / size / product name of the surgical instrument coupled to the coupler body 500.

[0177] 6 illustrates a coupler interface 400 at the distal end of a link 316 of a robotic arm. As shown in FIG. 6, the coupler interface 400 may include a protrusion 404 extending from a flat portion 402. The flat portion 402 may have an outer diameter that matches the outer diameter of the link 316. The protrusion 404 may have a non-circular profile that corresponds to the geometry of a groove 505 in the coupler body 500, as described in further detail below. Additionally, the protrusion 404 may include one or more locking portions 406 disposed on an outer surface of a sidewall of the protrusion 404. For example, locking portions 406 may be recesses / grooves extending along the outer surface of protrusion 404 that are sized and shaped to engage locking arms 506 of coupler body 500 to secure coupler body 500 to coupler interface 400 and to secure a sterile drape between coupler body 500 and coupler interface 400, as described in further detail below. Preferably, protrusion 404 includes at least one pair of locking portions 406 so that coupler body 500 can be securely coupled to coupler interface 400 in two orientations.

[0178] Further, coupler interface 400 may include an extended portion configured to be inserted into link 316. Coupler interface 400 may be rotatably coupled to the distal end of distal wrist link 316 using any suitable fastener or connector, such as a magnet, a screw, a pin, a clamp, a weld, an adhesive, a rivet, and / or any other suitable fastener or any combination of the foregoing. Additionally, as described in U.S. Pat. No. 11,812,938, coupler interface 400 may include a repulsive magnet disposed within protrusion 404. The repulsive magnet is configured to facilitate determining when coupler body 500 is coupled to coupler interface 400 and a surgical instrument is not coupled to coupler body 500, and / or to facilitate coupling of a surgical instrument to coupler body 500, by applying a magnetic force to a magnet slidably disposed within coupler body 500, for example, by moving the magnet to a position within coupler body 500 with a maximum distance from coupler interface 400, as described in further detail below. Additionally, as described above, the robotic arm 300 may include one or more encoders E7, which may be located, for example, on or adjacent to the joint 328 in the link 316, to measure the angulation between the distal wrist link 316 and the surgical instrument coupler interface 400. For example, the encoder E7 may include two or more encoders positioned circumferentially around the extended portion of the coupler interface 400.

[0179] 7A-7C, a coupler body 500 is provided. The coupler body 500 can be configured to be removably coupled to a surgical instrument having a predefined shaft diameter, e.g., a 10 mm surgical instrument. The coupler body 500 is preferably designed to lock onto the distal end of a robotic arm with a sterile drape therebetween so that the robotic arm remains covered and sterile throughout the procedure. Additionally, the coupler body 500 also has a separate portion for locking onto a surgical instrument (e.g., a commercially available laparoscopic instrument), allowing a clinician to perform surgery using the robotic arm as described herein. As shown in FIGS. 7A-7C, the coupler body 500 can include a coupler interface connecting portion 504 and a surgical instrument connecting portion 502. As shown in FIG. 7C, the coupler interface connecting portion 504 can include a groove 505 extending inward from the bottom surface of the coupler body 500. Groove 505 may have a geometric shape that corresponds to the profile shape of protrusion 404 of coupler interface 400, such that protrusion 404 may be received by groove 505 while limiting rotational movement between coupler body 500 and coupler interface 400. A sterile drape may be positioned between protrusion 404 and groove 505 when protrusion 404 is disposed within groove 505. Preferably, the profile of protrusion 404 and the corresponding geometric shape of groove 505 are symmetrical such that protrusion 404 may be received by groove 505 in at least two orientations. For example, in some embodiments, the profiles of protrusion 404 and groove 505 may comprise a diamond, rectangular, or oval shape. Furthermore, the profile of protrusion 404 and the corresponding geometric shape of groove 505 may guide a user in mating coupler body 500 and coupler interface 400.

[0180] Additionally, the coupler interface connecting portion 504 may include a pair of locking arms 506 configured to facilitate locking between the coupler body 500 and the coupler interface 400 when the protrusion 404 is disposed within the groove 505. Each of the locking arms 506 may include a handle portion 510 sized and shaped to be actuated by a user's finger and a connecting portion 508 sized and shaped to engage with the locking portion 406 of the protrusion 404. For example, the connecting portion 508 may have a tapered profile to securely engage with the locking portion 406. The locking arms 506 may be pivotally coupled to the coupler interface connecting portion 504 such that the locking arms 506 may be transitionable between an unlocked state and a locked state. Furthermore, the locking arms 506 may be pivotally coupled to the coupler interface connecting portion 504 via a spring, e.g., a torsion spring, an extension spring, a compression spring, or the like, such that the locking arms 506 are biased toward the locked state. Thus, the handle 510 can be actuated to transition the locking arm 506 from a locked state to an unlocked state.

[0181] Thus, prior to coupling coupler body 500 to coupler interface 400, a sterile drape may be positioned between coupler body 500 and coupler interface 400 such that the sterile drape may be draped over robotic arm 300, as described above. Additionally, elastic bands of the sterile drape may be hooked onto hooks located on light house 203 to secure the drape over light house 203. The sterile drape may be marked and secured, for example, with peel-off indicators, to facilitate efficient application of the drape. A user may then apply force to handle portions 510 of locking arms 506, for example, pinching handle portions 510 toward each other to provide clearance for protrusions 404 to be received within grooves 505, thereby moving connecting portions 508 away from each other, toward an unlocked state, and out of grooves 505. When the locking arms 506 are in their unlocked state, the coupler body 500 may be coupled to the coupler interface 400 such that the protrusions 404 are disposed within the grooves 505. Once the protrusions 404 are disposed within the grooves 505, the user may release the handle portions 510, causing the locking arms 506 to move back toward their locked state and the connecting portions 508 to engage with the locking portions 406 of the protrusions 406. Thus, the engagement of the connecting portions 508 and the locking portions 406 due to the corresponding geometric shapes of the connecting portions 508 and the locking portions 406 may prevent movement between the coupler body 500 and the coupler interface 400, thereby securely coupling the coupler body 500 to the coupler interface 400.

[0182] As shown in Figures 7A-7C, the surgical instrument interface portion 502 can include an extending opening 516 sized and shaped to receive a surgical instrument shaft. For example, the opening 516 can be sized and shaped to receive a 10 mm surgical instrument shaft. The opening 516 can be defined by a channel extending downwardly from the upper surface of the surgical instrument interface portion 502, whereby a surgical instrument can be inserted through the channel and into the opening 516. As shown in Figures 7A and 7B, the upper surface of the surgical instrument interface portion 502 can include a tapered portion 514 that angles downwardly toward the opening 516, thereby defining a channel within the opening 516. Thus, tapered portion 514 ensures that the shaft of a surgical instrument is properly inserted into opening 516 in one of two orientations by rotating coupler body 500 (and thus distal wrist link 316) to align with the longitudinal axis of the surgical instrument in one of the two orientations. For example, tapered portion 514 may facilitate “self-alignment” of the distal end of robotic arm 300 by, for example, causing coupler interface 400 coupled to coupler body 500 to automatically rotate relative to distal wrist link 316 about axis Q7 at passive joint 328 such that the instrument shaft is guided along tapered portion 514, thereby aligning the longitudinal axis of opening 516 with the longitudinal axis of the surgical instrument. Thus, the user does not need to align the instrument shaft with the opening 516, but rather rotates the opening 516 through rotation of the coupler body 500 and surgical instrument coupler interface 400 relative to the distal wrist link 316 so that the opening 516 is aligned with the longitudinal axis of the instrument shaft.

[0183] Additionally, surgical instrument interface portion 502 may include a clamp 518 pivotally coupled to surgical instrument interface portion 502 about axis 512, where clamp 518 may be transitionable between an unlocked state and a locked state. Further, clamp 518 may be pivotally coupled to surgical instrument interface portion 502 via a torsion spring such that clamp 518 is biased toward the locked state. Clamp 518 may include a locking portion 520 configured to secure a surgical instrument within opening 516 when clamp 518 is in its locked state. For example, a lower surface of locking portion 520 may define an upper surface of opening 516 when clamp 518 is in its locked state, thereby preventing upward movement of a surgical instrument when a surgical instrument is positioned within opening 516 and clamp 518 is in its locked state.

[0184] The upper surface of locking portion 520 may be tapered to, in conjunction with tapered portion 514, facilitate guidance of a surgical instrument into opening 516. Thus, the tapered angle of locking portion 520 may be alone sufficient to allow a surgical instrument to be inserted into opening 516, whereby insertion of a surgical instrument toward opening 516 applies a force against the tapered upper surface of locking portion 520, thereby causing clamp 518 to rotate about axis 512 from a locked state to an unlocked state, allowing the surgical instrument to be received by opening 516. Clamp 518 may further include a handle 522 sized and shaped to be actuated by a user's finger to transition clamp 518 from the locked state to the unlocked state. For example, handle 522 may be actuated to transition clamp 518 to the unlocked state for insertion of a surgical instrument into opening 516 and / or removal of a surgical instrument from opening 516.

[0185] Moreover, coupler body 500 may further include a switch 524 pivotally coupled to surgical instrument interface portion 502 and configured to facilitate securement of a surgical instrument within opening 516. For example, switch 524 may include one or more surgical instrument engagement portions 526, each having a geometry corresponding to an outer diameter of a shaft of a surgical instrument to be inserted within opening 516. In addition, switch 524 may include a handle portion 528 sized and shaped to be actuated by a user's finger to transition switch 524 between an unlocked state and a locked state in which surgical instrument engagement portion 526 engages a surgical instrument shaft within opening 516 and applies a frictional force to the surgical instrument shaft.

[0186] Additionally, in its locked state, surgical instrument engaging portion 526 further defines opening 516. Surgical instrument engaging portion 526 may have a coefficient of friction such that when a surgical instrument is disposed within opening 516 and switch 524 is in its locked state, surgical instrument engaging portion 526 applies a frictional force to the surgical instrument that prevents longitudinal movement of the surgical instrument relative to coupler body 500 while allowing rotational movement of the surgical instrument within opening 516. For example, the frictional force applied to shaft 10 a by surgical instrument engaging portion 526 facilitates securement of shaft 10 a within coupler body 500 such that longitudinal movement of surgical instrument 10 is prevented unless the longitudinal force applied to surgical instrument 10 at least exceeds the frictional force applied to shaft 10 a by surgical instrument engaging portion 526, while minimizing the rotational force required to overcome the frictional force and rotate shaft 10 a within opening 516. Thus, when a surgical instrument is placed within opening 516, switch 524 can be actuated to its unlocked state to allow a user to readjust / move the surgical instrument vertically relative to coupler body 500 within opening 516, and can return to its locked state to prevent vertical movement of the surgical instrument relative to coupler body 500. Preferably, both switch 524 and clamp 518 must be in their unlocked states to allow removal of the surgical instrument from coupler body 500.

[0187] Alternatively, the coupler interface and coupler body may be constructed as described in U.S. Patent No. 11,812,938, as shown in Figures 7D-7H. For example, coupler interface 600 may be coupled to or otherwise integrated with link 316, and connecting portion 650 may be coupled to a coupler body, e.g., coupler body 500 or coupler body 900, to removably couple the coupler body to coupler interface 600. As shown in Figure 7D, coupler interface 600 may include a protrusion 604 extending from flat portion 602. Flat portion 602 may have an outer diameter that matches the outer diameter of link 316. Additionally, coupler interface 600 may include an extended portion 608 extending from flat portion 602 and configured to be inserted into link 316. Like protrusion 404, protrusion 604 may have a non-circular profile that corresponds to the geometry of groove 652 in connecting portion 650 of the coupler body, as described in further detail below. For example, as shown in FIG. 7D , protrusion 604 may have a diamond-shaped profile. Thus, when protrusion 604 is positioned within groove 652 in connecting portion 650, rotational movement between coupler interface 600 and connecting portion 650 is prevented.

[0188] Additionally, protrusion 604 may include one or more locking portions 606 disposed on the outer surface of the sidewall of protrusion 604. For example, locking portion 606 may be a recess / groove extending along the outer surface of protrusion 604 and sized and shaped to engage with locking arms 660 of connecting portion 650 to secure the coupler body to coupler interface 600 and to secure a sterile drape between connecting portion 650 and coupler interface 600, as described in further detail below. Preferably, protrusion 604 includes a pair of locking portions 606. For example, as shown in FIGS. 7D and 7F , the pair of locking portions 606 may be disposed on opposite vertices of the diamond-shaped profile of protrusion 604. Thus, connecting portion 650 may be fixedly coupled to coupler interface 600 in two orientations.

[0189] As shown in FIG. 7D , coupler interface 600 may include one or more additional protrusions 610, e.g., “mating dots,” disposed on flat portion 602. For example, coupler interface 600 may preferably include multiple protrusions 610 uniformly spaced along flat portion 602, e.g., adjacent the outer edge of flat portion 602. Protrusions 610 may have a geometric shape that corresponds to the geometric shape of one or more additional grooves 654 of connecting portion 650, as shown in FIG. 7E . For example, protrusions 610 may have a hemispherical shape, and grooves 654 may have a corresponding hemispherical shape. As shown in FIG. 7E , grooves 654 may be disposed along connecting portion 650 such that grooves 654 are aligned with protrusions 610 such that protrusions 610 may be disposed within grooves 654 when connecting portion 650 is coupled to coupler interface 600, as shown in FIG. 7F . Thus, when protrusion 602 is disposed within groove 652 of connecting portion 650 and protrusion 610 is disposed within groove 654, rotational movement between coupler interface 600 and connecting portion 650 is prevented. As will be understood by one of ordinary skill in the art, coupler interface 600 and connecting portion 650 may include more or fewer protrusions 610 and grooves 654, respectively, than those shown in Figures 7D and 7E. Additionally, other coupler interfaces and coupler bodies described herein, e.g., coupler interface 400 and coupler bodies 500, 900, may include similar additional protrusions and grooves to provide additional stabilization when the coupler interface is coupled to the coupler body.

[0190] As shown in FIG. 7G , connecting portion 650 may include a pair of locking arms 660, which may be constructed similarly to locking arms 506 of connecting portion 504, for releasably securing connecting portion 650 to coupler interface 600. For example, locking arm 660 may include a handle portion 664 sized and shaped to be actuated by a user's finger and a connecting portion 662 sized and shaped to engage locking portion 606 of protrusion 604. Thus, locking arm 660 may transition between an unlocked state in which locking arm 660 is disengaged from protrusion 604, as shown in FIG. 7G , and a locked state in which connecting portion 662 of locking arm 660 is engaged with locking portion 606 of protrusion 604 such that locking arm 660 is biased toward the locked state, as shown in FIG. 7H .

[0191] 8A is a cross-sectional view of coupler body 500 when coupler body 500 is not coupled to coupler interface 400, FIG. 8B is a cross-sectional view of coupler body 500 when coupler body 500 is coupled to coupler interface 400, and FIG. 8C is a cross-sectional view of coupler body 500 when coupler body 500 is coupled to coupler interface 400 and a surgical instrument is coupled to coupler body 500. As shown in FIG. 8A, coupler body 500 may further include a holder 530 disposed within surgical instrument connecting portion 502. Holder 530 is configured to be slidably disposed within surgical instrument connecting portion 502, for example, toward or away from coupler interface connecting portion 504. Further, holder 530 is configured to hold a magnet 540. For example, holder 530 may include one or more cradles 534 extending between a contact surface, e.g., friction pad 532, and a magnet harness 538 configured to hold magnets 540. Each cradle 534 of holder 530 may include a channel 536 extending therein in a direction from magnet harness 538 toward friction pad 532. Channel 536 may be sized and shaped to slidably receive a longitudinally extending rod therethrough, such that the longitudinally extending rod extends along axis 512 between channels 536. Clamp 518 may be pivotally coupled to the longitudinally extending rod such that clamp 518 may rotate about axis 512, as described above. The shaft 512 may be fixed relative to the surgical instrument connection portion 502 such that the holder 530 may be moved toward / away from the coupler interface connection portion 504 via movement of the channel 536 along the longitudinally extending rod.

[0192] 8A , the upper surface of friction pad 532 defines the lower surface of opening 516. The upper surface of friction pad 532 may have a curved profile that may match the curvature of a surgical instrument. Friction pad 532 may have a coefficient of friction such that when a surgical instrument is placed within opening 516 and switch 524 is in its locked state, friction pad 532 applies a frictional force to the surgical instrument that prevents longitudinal movement of the surgical instrument relative to coupler body 500 while allowing rotational movement of the surgical instrument within opening 516. As will be understood by one skilled in the art, friction pad 532 may be formed from a single or multiple pieces configured to contact a surgical instrument within opening 516, or alternatively, may be wrapped around or otherwise integrated with the upper surface of holder 530. When switch 524 is moved to its unlocked state, the frictional force of friction pad 532 may not be sufficient to prevent longitudinal movement of the surgical instrument relative to coupler body 500 .

[0193] The magnet 540 may have a magnetic force such that, when the coupler body 500 is coupled to the coupler interface 400, the magnet 540 induces a magnetic field that can be detected, for example, by one or more magnetic field sensors disposed within the link 316 and / or the coupler interface 400. Thus, the strength of the induced magnetic field will be proportional to the distance between the magnet 540 and the coupler interface 400, such that the magnetic field detected by the magnetic field sensor may indicate the position of the magnet 540, and therefore the holder 530, within the coupler body 500. Similarly, when no magnetic field is induced via the magnet 540, the magnetic field sensor may detect that the coupler body 500 is not coupled to the coupler interface 400. Furthermore, the repulsive magnet of the coupler interface 400 may have a magnetic force such that, when the coupler body 500 is coupled to the coupler interface 400, the repulsive magnet applies a magnetic force to the magnet 540, thereby moving the magnet 540, and therefore the holder 530, away from the coupler interface connecting portion 504. The position of the holder 530 relative to the coupler body 500 can indicate whether a surgical instrument is coupled to the coupler body 500 when the coupler body 500 is coupled to the coupler interface 400. For example, as shown in FIG. 8A , if there are no repelling magnets of the coupler interface 400 within the vicinity of the magnet 540, no magnetic force will be applied to the magnet 540 and will not cause displacement of the holder 530, e.g., toward the opening 516. Thus, the holder 530 can be in a neutral position due to gravity, e.g., toward the coupler interface connecting portion 504.

[0194] 8B , when coupler body 500 is coupled to coupler interface 400 and a surgical instrument is not coupled to coupler body 500, repelling magnets may apply a magnetic force to magnet 540, thereby moving magnet 540, and therefore holder 530, toward opening 516 and away from coupler interface 400 within channel 503, e.g., to a position within coupler body 500 with a maximum distance from coupler interface 400. Thus, when coupler body 500 is coupled to coupler interface 400, friction pad 532 may be closer to locking portion 520 of clamp 518, thereby reducing the size of opening 516. Furthermore, when coupler body 500 is coupled to coupler interface 400 and no instrument is coupled to coupler body 500, the magnetic field induced by magnet 540 when magnet 540 is at a position in channel 503 farthest from coupler interface 400 in response to the magnetic force of the repelling magnets may provide a clean signal that may be detected by a magnetic field sensor, indicating that coupler body 500 is coupled to coupler interface 400 without a surgical instrument attached thereto. Thus, the system may determine that coupler body 500 is coupled to coupler interface 400 without a surgical instrument coupled thereto, for example, based on the strength of the magnetic field induced by magnet 540 when magnet 540 is at a maximum distance from coupler interface 400 within coupler body 500.

[0195] 8C , when shaft 10 a of surgical instrument 10 is inserted into opening 516, shaft 10 a applies a downward force against friction pad 532, causing holder 530 to move downward within channel 503 and increase the size of opening 516 until shaft 10 a is fully disposed within opening 516 and clamp 518 is allowed to transition back to its locked state so that shaft 10 a is positioned between the lower surface of locking portion 520 and friction pad 532. Upon release of surgical instrument 10 by the user, friction pad 532 applies an upward force to shaft 10 a due to the magnetic force of the repulsive magnet applied to magnet 540 so that shaft 10 a is pinned between the lower surface of locking portion 520 and friction pad 532. Thus, in response to the magnetic force of the repulsive magnet when coupler body 500 is coupled to coupler interface 400, the magnetic field induced by magnet 540 when it is in position within channel 503 and the force applied by shaft 10a via friction pad 532 to holder 530, and therefore magnet 540, can be detected by the magnetic field sensor, which can indicate that coupler body 500 is coupled to coupler interface 400 and that surgical instrument 10 is coupled to coupler body 500. Thus, based on the strength of the magnetic field induced by magnet 540, the system can determine that coupler body 500 is coupled to coupler interface 400 and that surgical instrument 10 is coupled to coupler body 500.

[0196] Furthermore, the position of magnet 540 within channel 503 depends on the diameter size of the surgical instrument placed within opening 516 when coupler body 500 is coupled to coupler interface 400, whereby the induced magnetic field will vary based on the surgical instrument shaft size placed within opening 516. Thus, the system can identify the precise size of the surgical instrument shaft based on the strength of the magnetic field induced by magnet 540 as detected by the magnetic field sensor. Based on the identified type of surgical instrument coupled to coupler body 500, the system can load a calibration file associated with the identified surgical instrument, as described above. Furthermore, based on the identified product name of the surgical instrument, provided that each specific product name has a distinguishable shaft diameter size, the system can determine whether the attached surgical instrument is approved for use with the system.

[0197] 9 , another coupler body configured to be removably coupled to a surgical instrument having a predefined shaft diameter is provided. Coupler body 900 may be constructed similarly to coupler body 500. For example, surgical instrument connecting portion 902, channel 903, coupler interface connecting portion 904, groove 905, locking arm 906, shaft 912, tapered portion 914, opening 916, clamp 918, switch 924, and holder 930 of coupler body 900 correspond to surgical instrument connecting portion 502, channel 503, coupler interface connecting portion 504, groove 505, locking arm 506, shaft 512, tapered portion 514, opening 516, clamp 518, switch 524, and holder 530 of coupler body 500, respectively. Coupler body 900 differs from coupler body 500 in that coupler body 900 may be configured to be removably coupled to a smaller diameter surgical instrument, for example, a 5 mm surgical instrument, such as surgical instrument 12 described above. Coupler body 500 and coupler body 900 may include visual indicators, for example, color and / or size markings, to easily inform a user of the respective coupler body size.

[0198] 10A, the robotic arm 300 can be positioned in a surgical drape-ready configuration (e.g., a preset "drape mode"). For example, upon activation of the drape mode, e.g., via GUI 210, the system automatically extends the robotic arms 300a, 300b such that the wrist portions 311a, 311b, elbow links 310a, 310b, and shoulder links 305a, 305b extend away from their respective proximal shoulder portions 304a, 304b, as shown in FIG. 10A, allowing a surgical / sterile drape to be draped over each component of the robotic arms 300a, 300b, as shown in FIG. 10D. Additionally, the system may automatically cause shoulder portion 304a to rotate relative to base portion 302a of robot arm 300a and / or shoulder portion 304b to rotate relative to base portion 302b of robot arm 300b as robot arms 300a, 300b extend away from platform 200. This configuration allows for efficient and accessible draping of each robotic arm with a single surgical / sterile drape, e.g., sterile drape 800, having a first drape portion 801a sized and shaped to drape robot arm 300a and a second drape portion 801b sized and shaped to drape robot arm 300b, as described in more detail below with respect to FIG. Preferably, the robotic arms 300a, 300b extend away from the platform 200 so that the shoulder links 305a, 305b extend at an angle away from the shoulder portions 304a, 340b, and the wrist portions 311a, 311b and elbow links 310a, 310b are substantially parallel to each other and to the ground, as shown in FIG. 10A.Alternatively, robot arm 300a and robot arm 300b can be angled away from each other in the draped mode, for example, by rotating shoulder portion 304a relative to base portion 302a of robot arm 300a and / or rotating shoulder portion 304b relative to base portion 302b of robot arm 300b, so that wrist portion 311a, elbow link 310a, and shoulder link 305a extend away from wrist portion 311b, elbow link 310b, and shoulder link 305b. This configuration can be advantageous when robot arms 300a, 300b are draped individually.

[0199] In some embodiments, when operating in drape mode, the system may automatically move the base portions 302 a, 302 b to the stage of the platform 200, for example, vertically relative to the ground, thereby aligning the wrist portions 311 a, 311 b and elbow links 310 a, 310 b with the user's height and facilitating the application of a sterile drape over the robotic arms 300 a, 300 b. The user's height may be automatically determined by the system based on depth data collected by one or more optical scanners. Additionally or alternatively, the user's height and / or preferred height of the user's wrist portions 311 a, 311 b and elbow links 310 a, 310 b for draping in drape mode may be stored in the system's memory in a user-specific profile, which may be uploaded for execution by the system in response to user input in the GUI 210. The system 100 may store a predetermined drape-ready configuration, for example, in a user-specific profile, so that during user setup, when the system is operated in "drape mode," the system 100 may automatically move the robot arm 300 to the predetermined drape-ready configuration.

[0200] Furthermore, in the extended position in the drape mode, the robot arm may be within a virtual haptic boundary, whereby the robot arm is in haptic mode and a high level of impedance is applied to the robot arm, thereby making movement of the robot arm more viscous and making it easier for the operator to drape the robot arm, while providing movement thereto as needed. Additionally, if the robot arm is not within a predefined virtual haptic boundary around the workspace in the drape-ready configuration, the system may apply a temporary local virtual haptic boundary to at least the distal end of the robot arm, as described in more detail below. The system may remove the temporary local virtual haptic boundary upon detecting that each of the coupler bodies is coupled to the distal end of the robot arm. For example, the system may remove the temporary local virtual haptic boundary from both robot arms upon determining that both coupler bodies are coupled to both robot arms. Alternatively, the system may remove the temporary local virtual haptic boundary from each robot arm upon determining that each of the coupler bodies is coupled to a robot arm.

[0201] As shown in Figure 10B, a single sterile drape 800 is provided having a first drape portion 801a sized and shaped to drape robotic arm 300a and a second drape portion 801b sized and shaped to drape robotic arm 300b. The sterile drape 800 can be used to drape at least the front side of both robotic arms 300a, 300b and platform 200, as shown in Figure 10D. As shown in Figure 10B, the sterile drape 800 can be fully closed at its end portions, e.g., the distal portions of the first and second drape portions 801a, 801b that contact the respective coupler interfaces of the robotic arms. Alternatively, in some embodiments, the sterile drape 800 may have openings (which may optionally have sterile seals or interfaces) in its distal portion through which portions of the robotic arm 300, coupler interface 400, coupler body 500, and / or surgical instruments may pass. A drape that has sealed end portions without any openings and is sealed along its length may provide a better sterile barrier for the system 100. Thus, all of the robotic arms 300a, 300b may be located inside and / or completely enclosed within the sterile drape 800, except for an opening at the proximal end of the sterile drape 800, e.g., near the base of the robotic arm 300. In some embodiments, the coupler body 500 and coupler interface 400 may have electrical connectors for creating an electronic connection between the robotic arm 300 and the surgical instruments. Thus, electrical signals may be transmitted through the sterile drape 800. The surgical instrument and coupler body may alternatively be passive or non-electronic so that electrical wires do not need to pass through the sterile drape 800 .

[0202] As shown in FIG. 10B, sterile drape 800 may include one or more straps / bands, such as bands 802a and 802b, configured to secure drape portions 801a and 801b to robotic arms 300a and 300b, respectively, as shown in FIG. 10E. Bands 802a and 802b may be integrated with sterile drape 800 and affixed to themselves once positioned at a target location relative to robotic arms 300a and 300b, as shown in FIG. 10E. Alternatively, bands 802a and 802b may be made from an elastic material, such that they can be simply stretched and threaded over robotic arms 300a and 300b until positioned at a target location. As will be understood by one of ordinary skill in the art, fewer or more than two bands may be used to secure sterile drape 800 over each of the robotic arms. For example, a first band may be used to secure drape portion 801a at elbow link 310a of robot arm 300a, a second band may be used to secure drape portion 801a at shoulder link 305a of robot arm 300a, a third band may be used to secure drape portion 801b at elbow link 310b of robot arm 300b, and / or a fourth band may be used to secure drape portion 801b at shoulder link 305b of robot arm 300b.

[0203] As shown in FIG. 10B, the sterile drape 800 may include one or more rigid guides, e.g., guides 804a, 804b, integrated with the sterile drape 800, which may be grasped by a user to guide the sterile drape 800 over each robotic arm. For example, guides 804a, 804b may be formed from cardboard, plastic, metal, or another rigid material. As will be understood by those skilled in the art, the sterile drape 800 may include more than two guides, and / or two or more guides may be positioned on the sterile drape 800 in locations other than those shown in FIG. 10B. Additionally, the proximal edge of the sterile drape 800 may include an elastic band 806 to facilitate wrapping the sterile drape 800 over the stage of the platform 200 and engaging with the drape hooks 209 of the lighthouse 203, as described in further detail below with respect to FIG. 10C. Additionally, as shown in FIG. 10B, the sterile drape 800 may be marked and / or further secured with one or more peel-off indicators, e.g., indicators 808a, 808b, which have directional markings to facilitate efficient application of the drape over the robotic arm.

[0204] FIG. 10C illustrates exemplary method steps for draping robotic arms 300a, 300b with sterile drape 800. Initially, sterile drape 800 may be laid flat so that drape portions 801a, 801b are side-by-side. The user may then tear perforated straps, if applicable, that may be provided to keep sterile drape 800 compact, ensuring that sterile drape 800 is free / unconstrained. The user may then tent the opening of sterile drape 800, for example, through elastic band 806, and grasp the top portions of guides 804a, 804b while elastic band 806 remains over the user's wrist. The user may refer to indicators 808a, 808b to ensure that right guide 804b is grasped by the user's right hand and left guide 804a is grasped by the user's left hand. While standing in front of robotic arms 300a, 300b, the user can verify that drape portion 801a is in front of robotic arm 300a and drape portion 801b is in front of robotic arm 300b, and, if applicable, hang guide 804a on elbow link 310a of robotic arm 300a and guide 804b on elbow link 310b of robotic arm 300b. The user should ensure that drape portions 801a, 801b are securely positioned over wrist joints 324a, 324b, respectively, before releasing sterile drape 800.

[0205] Next, standing outside the robotic arm, e.g., robotic arm 300b, a user can pull drape portion 801b over elbow link 310b and shoulder link 305b via guide 804b until guide 804b is positioned at least partially over shoulder portion 304b. Preferably, the user holds guide 804b from inside the folded sterile drape 800 and pulls drape portion 801b over robotic arm 300b while maintaining sterile conditions. Then, standing outside the other robotic arm, e.g., robotic arm 300a, a user can hold guide 804a from inside the folded sterile drape 800 and pull drape portion 801a over elbow link 310a and shoulder link 305a via guide 804a while maintaining sterile conditions. Guides 804a, 804b may be located on base portions 302a, 302b of robotic arms 300a, 300b once sterile drape 800 is fully draped over the robotic arms. The user may then wrap the proximal portion of sterile drape 800 around base portions 302a, 302b and the stage of platform 200 via elastic band 806 and hook elastic band 806 onto hook 209 of light house 203. As shown in FIG. 10C , drape hook 209 may protrude outward from the surface of light house 203 and may further include a grooved portion, thereby forming a hook shape for easily receiving the proximal edge of sterile drape 800. Thus, elastic bands 806 can be hooked onto drape hooks 209 of light house 203 of platform 200, for example, below markers 205 on the front side of light house 203, as shown in Figure 10D. As explained above, bands 802a, 802b can be used to secure drape portions 801a, 801b to robotic arms 300a, 300b in such a manner that there is sufficient slack in sterile drape 800 so that robotic arms 300a, 300b can move sterile drape 800 without tearing or disengaging bands 802a, 802b, as shown in Figure 10E.

[0206] 11A and 11B, rotation of the distal shoulder link 308 relative to the proximal shoulder link 306 of the shoulder link 305 is provided. As explained above, motor axis Q3 may be a “setup” axis, whereby the distal shoulder link 308 may be automatically rotated relative to the proximal shoulder link 306 upon actuation of actuator 330 during a set-up phase of the robotic arm 300, for example, prior to operation of the robotic arm 300 in a surgical procedure. Optionally, prior to rotation of the distal shoulder link 308 relative to the proximal shoulder link 306, the shoulder portion 304 may first be rotated relative to the base portion 302 to a desired position, thereby causing rotation of all links distal to the proximal shoulder link 306 coupled to the shoulder portion 304 to rotate relative to the base portion 302 and providing sufficient space for rotation of the robotic arm 300 about joint 320. Additionally, the wrist portion 311 may be extended at least partially away from the base portion 302 to avoid collision with any components of the robotic arm 300 upon rotation of the robotic arm 300 about the joint 320.

[0207] As described above, motor M4 must be actuated, for example, via actuator 330, to automatically rotate distal shoulder link 308 relative to proximal shoulder link 306 at joint 320. As shown in FIG. 11A , motor M4 may be operably coupled to a motion transmission mechanism (e.g., worm gear 323 via gear 321) coupled to distal shoulder link 308, whereby actuation of motor M4 causes rotation of distal shoulder link 308 relative to proximal shoulder link 306 via engagement between gear 321 and worm gear 323. FIG. 11B illustrates robotic arm 300 in a desired location for a particular laparoscopic procedure upon rotation of distal shoulder link 308 relative to proximal shoulder link 306.

[0208] As described in further detail below, the system 100 can store one or more predetermined robotic arm configurations, including predetermined degrees of rotation of the distal shoulder link 308 relative to the proximal shoulder link 306, for one or more known surgical procedures, such that during setup, upon activation of the system in a "ready to operate mode," the system 100 can automatically move the robotic arm 300 to the predetermined robotic arm configuration. Additionally, as the robotic arm is being moved, either manually by a user or automatically, based on depth data obtained from one or more optical scanners during setup, the system can detect when either the stage of the platform 200 or the robotic arm approaches a predetermined distance threshold relative to an object in the operating room, such as a surgical bed. Thus, the system can automatically reconfigure the robotic arm to avoid a collision with the object, for example, by automatically actuating the motorized joint 320 to rotate the distal shoulder link 308 relative to the proximal shoulder link 306. Similarly, the system 100 can automatically reconfigure the robotic arm to avoid collisions with objects in the operating room during a surgical procedure by automatically actuating the motorized joints 320.

[0209] 12A and 12B illustrate example data produced by the optical scanner 202. For example, FIG. 12A illustrates image data captured by the optical scanner 202, and FIG. 12B illustrates a depth map of at least some objects in the surgical space generated from the data captured by the optical scanner 202. Specifically, the optical scanner 202 may create a depth map, e.g., a point cloud, in which the value of each pixel is associated with a distance from the optical scanner 202. For example, the difference between pixels for a first object (such as a first surgical instrument) and a second object (e.g., a trocar) would allow the system to calculate the distance between the surgical instrument and the trocar. Furthermore, the difference between pixels for the first object (such as a first surgical instrument) at a first time point and the first object at a second time point would allow the system to calculate whether the first object moved, the trajectory of the movement, the speed of the movement, and / or other parameters associated with the changed position of the first object.

[0210] For example, the system may measure and record any of the following within the coordinate space of the system: the movement of handheld surgical instruments (attached to a robotic arm or separate from it) operated by the surgeon; the presence or absence of other surgical staff (e.g., instrument nurses, circulating nurses, anesthesiologists, etc.); the height and angular orientation of the surgical table; the patient position and volume on the surgical table; the presence or absence of drapes on the patient; the presence or absence of trocar ports, and if present, their location and orientation; gestures made by the surgical staff, tasks being performed by the surgical staff, and interactions between the surgical staff and the system; surgical instrument identification; attachment or detachment "actions" of surgical instruments and the system; position and orientation tracking of specific features of the surgical instrument relative to the system (e.g., camera head, couplers, fiducial markers, etc.); measurements of motion profiles or specific features within the scene that allow aspects of the surgery to be identified; the position, orientation, identification, and / or movement of any other instruments, features, and / or components of the system or used by the surgical team.

[0211] The system may combine the measurements and / or other data described above with any other telemetry data from the system and / or video data from the laparoscope to provide a comprehensive data set that can be used to improve the overall usability, functionality, and safety of the collaborative robotic-assisted surgical system described herein. For example, as the system is set up to begin a procedure, the optical scanner 202 may detect the height and orientation of the surgical table. This information may enable the system to automatically configure the degrees of freedom of the platform 200 supporting the robotic arm 300 to a desired or correct position relative to the surgical table. Specifically, the optical scanner 202 may be used to ensure that the height of the platform 200 is optimally positioned to ensure that the robotic arm 300 overlaps with the intended surgical workspace. Additionally, as described above, the system may automatically reconfigure not only the degrees of freedom of the platform 200 but also the placement of the robotic arm 300 in response to movement of the surgical table, and thus the trocar, to maintain the relative position between the distal end of the robotic arm and the trocar.

[0212] Additionally, the optical scanner 202 may identify the specific surgeon performing the procedure, so that the system can use the surgeon's identification to load into the system a system profile associated with the particular surgeon. The system profile may include information related to the surgeon's operating parameters and / or preferences, the surgeon's patient list with parameters for each patient, desired or required algorithm sensitivities for the surgeon, positioning degrees of freedom for the support platform, etc. Examples of algorithm sensitivities that may be surgeon-specific include adapting / adjusting the force required to transition from a passive mode to a collaborative manipulation mode (e.g., low to high force), adapting / adjusting the stickiness felt by the surgeon when collaboratively manipulating the robotic arm (e.g., low to high stickiness), preferred surgical instrument trajectories when performing a specific laparoscopic procedure, etc. Furthermore, surgeon preferences may include preferred placement of the robotic arm 300 with respect to a specific surgical instrument, e.g., the positioning of the links and joints of the robotic arm 300 relative to the patient; for example, preferred placement may differ between a laparoscope and a retractor.

[0213] Based on the data captured by the optical scanner 202, the system may generate virtual models of pieces of capital equipment and / or other objects in the operating room that are within the robotic arm's range of motion in the same coordinate space as the robotic arm and its coupled surgical instruments, for example, so that the virtual models can be stored and monitored to detect potential collisions. Additionally, the system may track the position and orientation of objects within the virtual models as they move relative to one another, so that the system may alert a user if the proximity of either the virtual model or the objects (i.e., the spacing between them) falls below a predefined threshold, e.g., within 50 mm, 75 mm, 30 mm, or less to 100 mm, or more. The system may use this information to recommend repositioning of the platform 200 and / or other components of the system, the surgical table, and / or the patient, and / or prevent the robotic arm from switching to a cooperative operation mode as a result of a force applied to the robotic arm due to a collision with a staff member, even if the force exceeds the robotic arm's predetermined force threshold. Additionally, the system may stop or block (e.g., prevent) further movement of the robotic arm (e.g., freeze the robotic arm) if the proximity of either the virtual model or an object, e.g., the robotic arm, reaches or falls below a predefined threshold relative to another object in the surgical space.

[0214] Additionally, based on the data captured by the optical scanner 202, the system can track the motion of handheld surgical instruments that are not coupled to a robotic arm and are controlled directly and independently by the surgeon. For example, the optical scanner 202 can track clearly defined features on the instrument, fiducial markers attached to the instrument or the surgeon's gloves (e.g., sterile gloves), the coupling between the robotic arm and the instrument, the distal tip of the instrument, and / or any other defined location on the instrument. The following are examples of uses and purposes of motion data: (i) closing the control loop between the handheld instrument and the robotic arm holding the camera, thus allowing the surgeon to servo-control (i.e., move) the camera by "pointing" at the handheld instrument; (ii) tracking information that can be used independently or in combination with other data streams to identify aspects of the surgical procedure; (iii) identifying the surgeon's handedness; (iv) monitoring metrics associated with the surgeon's experience; (v) identifying which tools the surgeon is using and when to change them for other tools; and / or (vi) tracking the number, location, and orientation of the patient's skin surface and trocar ports. This data and information can also be used and calculated by the system as part of a collaborative control framework. As will be understood by those skilled in the art, the location / movement of a surgical instrument coupled to the system's robotic arm will be known by the system based on known robot telemetry and the current kinematics of the robotic arm without the need for data captured by the optical scanner 202.

[0215] Based on the data captured by the optical scanner 202, the system may further track: the instruments being used at each port; how often instruments are swapped between ports; if the system is holding an instrument in place while the patient or surgical table is being moved (in which case the system may adjust for the movement by changing the robotic arm's operating mode to passive mode and repositioning the robotic arm 300 and / or platform 200), manually held instruments versus ports with instruments coupled to a robotic arm to monitor and determine if additional trocar ports are being added; and / or other conditions or parameters of the operating room or system. Knowledge of the location and orientation of the skin surface and trocar ports relative to the robotic arm may facilitate the implementation of a "virtual boundary," as described in more detail below.

[0216] 13A-13D, a setup for a collaborative surgical system is provided. As shown in FIG. 13A, platform 200 can be moved by a user, for example, via wheels 204, to a desired position relative to a patient table PT, while robotic arms 300a, 300b are in their respective stored configurations. As platform 200 is moved toward the patient, the scene can be directly observed by one or more optical scanners 202 and one or more proximity sensors 212. From the depth map observed and generated by the optical scanner 202 and the proximity data observed and generated by the proximity sensor 212, important features can be identified, such as, for example, the height and / or location of the patient table PT, the surface of the patient's abdomen, the surgeon's position and other characteristics, including the surgeon's height, and the trocar ports and the bases of the robotic arms 300 a, 300 b, e.g., base portions 302 a, 302 b and shoulder portions 304 a, 304 b, the robotic arms 300 a, 300 b, and / or one or more surgical instruments coupled thereto, and the distances between the platform 200 and the robotic arms 300 a, 300 b and other objects in the room, such as the patient table PT. Identification of such important features can be performed using standard computer vision techniques, such as template matching, feature tracking, edge detection, etc.

[0217] Once each feature is registered, its position and orientation can be assigned a local coordinate system and transformed to the system's global coordinate system using a standard transformation matrix. Once all features are transformed into a single global coordinate system, optimization algorithms, such as least squares and gradient descent, can be used to identify the most appropriate vertical and horizontal positions of the robotic arms 300a, 300b, and the positions can be adjusted to maximize the robotic arm's workspace relative to the insertion point on the patient via the platform 200. The optimal workspace can depend on the surgical procedure to be performed and / or the surgeon's preferred position. Additionally, the system can generate and display a virtual map, e.g., via a GUI 210 that graphically depicts identified features within the operating room based on depth and proximity data, to guide the user when moving the platform 200, as described in more detail below with respect to FIG. 33.

[0218] Referring again to FIGURE 13B, when platform 200 is in its desired position relative to patient table PT, wheels 204 are locked and robotic arms 300a, 300b can be extended away from their respective retracted configurations. The vertical position of the robotic arms relative to platform 200 can be adjusted to the desired position, as shown in FIGURE 13C, and the horizontal position of the robotic arms relative to platform 200 can be adjusted to the desired position, as shown in FIGURE 13D. The desired position of the robotic arms can be stored as a "ready to operate" configuration, which can be specific to the procedure being performed as well as the surgeon's preferences. Thus, when the platform 200 is in a desired position relative to the patient table PT, as shown in FIG. 13A, the system can be actuated, for example, via GUI 210 or voice control, to automatically move the platform 200 and robotic arms 300a, 300b toward a "ready for operation" configuration relative to the patient table PT based on the depth and proximity data observed and generated by the optical sensors 202 and proximity sensors 212, while avoiding collisions between the platform 200 and robotic arms 300a, 300b and other objects in the room.

[0219] 14, components that may be included within a collaboratively manipulated robotic platform 1400 are described. The platform 1400 may include one or more processors 1402, communications circuitry 1404, a power supply 1406, a user interface 1408, and / or memory 1410. One or more electrical components and / or circuits may perform some or all of the roles of the various components described herein. Although described separately, it should be understood that the electrical components need not be separate structural elements. For example, the platform 1400 and communications circuitry 1404 may be embodied within a single chip. Additionally, while the platform 1400 is described as having memory 1410, the memory chip may be provided separately.

[0220] Platform 1400 may include memory and / or be coupled via one or more buses to read and write information from and to memory. Memory 1410 may include a processor cache, including a multi-level hierarchical cache, where different levels have different capacities and access speeds. Memory may also include random access memory (RAM), other volatile storage devices, or nonvolatile storage devices. Memory 1410 may be RAM, ROM, flash, other volatile or nonvolatile storage devices, or other known memory, or some combination thereof, and preferably includes storage, in which data may be selectively stored. For example, storage devices may include, for example, hard drives, optical disks, flash memory, and Zip drives. Programmable instructions may be stored on memory 1410 to execute algorithms, for example, to calculate a desired force to be applied along robotic arm 300 and / or a surgical instrument coupled thereto, apply impedances at each joint of robotic arm 300, and effect the desired force.

[0221] Platform 1400 may incorporate processor 1402, which may be comprised of one or more processors, such as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any suitable combination thereof designed to perform the functions described herein. Platform 1400 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Platform 1400 may execute an operating system (e.g., operating system 1446), such as Windows, Mac OS, QNX, Unix, or Solaris 5.10, in conjunction with firmware / software stored in memory. Platform 1400 executes software applications stored in memory. For example, the software may be a program in any suitable programming language known to those skilled in the art, including, for example, C++, PHP, or Java.

[0222] The communications circuitry 1404 may include circuitry that allows the platform 1400 to communicate with image capture devices, such as optical scanners and / or endoscopes. The communications circuitry 1404 may be configured for wired and / or wireless communications networks, such as via the Internet, telephone networks, Bluetooth networks, and / or Wifi networks, using techniques known in the art. The communications circuitry 1404 may be communications chips known in the art, such as Bluetooth chips and / or Wifi chips. The communications circuitry 1404 allows the platform 1400 to transfer information, such as force measurements on the body wall at the trocar insertion point, locally and / or to a remote location, such as a server.

[0223] The power supply 1406 may provide alternating current or direct current. The power supply 1406 may be a port to allow the platform 1400 to be plugged into a conventional wall socket, for example, via a cord with an AC / DC power converter and / or a USB port, to power the components within the platform 1400. The power supply 1406 may be operably coupled to an emergency switch such that, upon activation of the emergency switch, power supplied to components within the platform 1400, including braking mechanisms disposed on at least some of the joints of the robotic arm 300, is stopped. For example, the braking mechanisms may require power to be turned off such that, without power supplied to the braking mechanisms, the braking mechanisms act to prevent movement of the robotic arm 300 without power. In a direct current embodiment, the power supply may include a suitable battery, such as a replaceable or rechargeable battery, and the device may include circuitry for charging the rechargeable battery and a detachable power cord. For example, the battery may be an uninterruptible power supply (UPS) that may be charged when the system is plugged in, and the UPS may be operably coupled to certain computing components of the system, e.g., processor 1402, so that the battery may automatically power the computing components when the system is temporarily unplugged from an electrical power source, e.g., during a multi-quadrant procedure to move the system to another side of a patient table. In some embodiments, the braking mechanisms of the wheels 204 of the platform 200 may also be operably coupled to the battery so that they may be engaged / disengaged while the system is unplugged and moved around the operating room.

[0224] The user interface 1408 may be used to receive input from a user and / or provide output to a user. For example, the user interface 1408 may include a touchscreen, a display, switches, dials, lights, etc. Thus, the user interface 1408 may display information such as a selected surgical instrument identification and force measurements observed during operation of the robotic arm 300. Additionally, the user interface 1408 may receive user input including: a predetermined amount of travel or a predetermined dwell time at the handle of a surgical instrument to cause the robotic arm to automatically switch to a passive mode; a predetermined threshold of force to be applied at the handle of a surgical instrument to cause the robotic arm to automatically switch to a collaborative manipulation mode; a predefined position of a force sense barrier; an identification of the surgical instrument coupled to the distal end of the robotic arm; the vertical height of the robotic arm; the horizontal position of the robotic arm; etc., so that the platform 1400 may adjust the information / parameters accordingly. In some embodiments, the user interface 1408 is not present on the platform 1400 but instead is provided on a remote, external computing device communicatively connected to the platform 1400 via the communications network 1404 .

[0225] The memory 1410, which is an example of a non-transitory computer-readable medium, may be used to store an operating system (OS) 1446, a surgical instrument identification module 1412, a surgical instrument calibration module 1414, an encoder interface module 1416, a robotic arm position determination module 1418, a trocar position detection module 1420, a force detection module 1422, an impedance calculation module 1424, a motor interface module 1426, an optical scanner interface module 1428, a gesture detection module 1430, a passive mode determination module 1432, a collaborative manipulation mode determination module 1434, a haptic mode determination module 1436, a robot-assisted mode determination module 1438, a trajectory generation module 1440, an obstacle detection module 1442, and an indicator interface module 1444. The modules are provided in the form of computer-executable instructions / algorithms that may be executed by the processor 1402 to perform various operations according to the present disclosure.

[0226] For example, during a procedure, the system may continuously invoke the algorithms described herein based on data collected by the system. That data may be collected and / or recorded using any of the components and methods disclosed herein (including, for example, from sensors / encoders within the robot, from an optical scanning device communicating with other components of the robotic system, and / or from manual input by the system operator). Thus, the system's algorithms, data, and configurations may enable a user to coordinate the robotic arm with minimal shock and impact from the weight of the robotic arm and / or surgical instruments coupled thereto, gravity, and other forces that conventional robotic arms cannot compensate for. Some of the parameters of the algorithms described herein may control aspects of the system's behavior, including, for example, the robustness of detected features, sensitivity to false positives, robot control gain, the number of features to track, the dead-zone radius, etc.

[0227] The surgical instrument identification module 1412 may be executed by the processor 1402 to identify the surgical instrument coupled to each of the robotic arms and load the appropriate calibration file into the controller system. For example, a calibration file for each surgical instrument may be stored in a database accessible by the surgical instrument identification module 1412 and may include information associated with the surgical instrument, such as instrument type, product name, weight, center of mass, length, instrument shaft diameter, etc. Thus, once the appropriate calibration file is loaded and the associated surgical instrument is coupled to the robotic arm 300, the system will automatically account for the mass of the surgical instrument, for example, to compensate for the force of gravity on the surgical instrument when it is attached to the robotic arm 300, based on the data in the calibration file, so that the robotic arm 300 can hold the surgical instrument in place after it is coupled to the robotic arm and the operator releases the surgical instrument. For example, the surgical instrument identification module 1412 may identify a surgical instrument based on user input via the user interface 1408, e.g., an operator may select a surgical instrument from a database of surgical instruments stored in the memory 1410.

[0228] Further, in some embodiments, the system may be configured so that only pre-approved and certified surgical instruments, for example, of a certain product name, are authorized for use with the system. The list of approved instruments may be stored in a database in memory 1410 and / or uploaded from a remote database, for example, a cloud database. The surgical instrument identification module 1412 may determine that a surgical instrument is authorized for use with the system, for example, through user input by a user via user interface 1408 indicating that the surgical instrument is among the list of pre-approved instruments, a calibration file loaded for the surgical instrument automatically when the surgical instrument is attached to the robotic arm or manually loaded by the user, and / or real-time surgical instrument identification by the system. For example, the surgical instrument identification module 1412 may identify the product name of the surgical instrument based on image data observed and generated via the optical scanner 202 and / or laparoscope before and during the procedure. Specifically, the surgical instrument identification module 1412 may identify distinctive features of the surgical instrument, such as a manufacturer's logo, handle design, instrument packaging, etc., from the image data and determine the type / product name of the instrument. For example, many surgical instruments include an identifying marker, such as a trademark logo, etched or otherwise marked on or near the distal tip of the instrument; therefore, the product name of the surgical instrument may be identified via a laparoscope video feed received by the optical scanner interface module 1428 with the distal tip of the surgical instrument within the laparoscope's field of view. Additionally, or alternatively, the image data acquired by the optical scanner 202 may include such data such that the surgical instrument identification module 1412 can compare measurement data associated with a particular instrument with information contained in a database to identify the instrument and load the appropriate calibration file into the controller system.

[0229] Furthermore, given that each specific product name of a laparoscope may have a distinguishable output video feed quality, e.g., xy pixel count, frame rate, noise signature, codec, etc., surgical instrument identification module 1412 may identify the product name of the instrument by comparing the metadata obtained via the output video feed quality with that expected from an authorized laparoscope. Furthermore, given that each specific product name of a surgical instrument may have a distinguishable and precise mass, surgical instrument identification module 1412 may identify the product name of the instrument based on the mass of the surgical instrument, for example, during surgical instrument calibration, as described in further detail below. Additionally, assuming that each specific product name of a surgical instrument may have a distinct and precise shaft diameter, the surgical instrument identification module 1412 may identify the surgical instrument based on a specific magnetic field strength measured by the sensor 414 induced by a magnet displaced within the coupler body due to the diameter of the surgical instrument when the surgical instrument is coupled to the coupler body and the coupler body is coupled to the coupler interface 400, as described above. Assuming that each specific product name of a surgical instrument may have a distinct and precise impedance characteristic, the system may transmit a vibration pulse along the surgical instrument when the surgical instrument is coupled to the robotic arm such that the surgical instrument identification module 1412 may identify the product name of the surgical instrument based on the response data. Furthermore, the surgical instrument identification module 1412 may identify the product name of the surgical instrument based on other measurable characteristics, such as the electrical resistance and / or magnetism of the surgical instrument, assuming such characteristics are distinct for each product name of the surgical instrument.

[0230] In some embodiments, the surgical instrument identification module 1412 can automatically identify surgical instruments coupled to the robotic arm via the coupler body and coupler interface using, for example, an RFID transmitter chip and reader or receiver (e.g., positioning an RFID sticker or transmitter on the surgical instrument that can transmit information about the surgical instrument to a receiver in the system), a near-field communication (“NFC”) device such as a near-field magnetic induction communication device, a barcode and scanner or other optical device, a magnet-based communication system, a reed switch, a Bluetooth transmitter, instrument weight and / or data gathered from an optical scanner and lookup table, an activation code associated with an authorized surgical instrument, and / or any other feature or mechanism described herein or suitable for identifying a surgical instrument. The surgical instrument identification module 1412 can further verify that the surgical instrument is authorized by checking a license and / or hospital inventory.

[0231] In some embodiments, an approved surgical instrument may include the following indicators: invisible ink on the tool shaft or handle that can be illuminated and detected via infrared illumination that can be illuminated and detected via the optical sensor 202, e.g., an IR-sensitive sensor of the optical scanner 202; unique reflective markings that can be illuminated and detected at a specific wavelength of light; unique features on the tool and / or coupling mechanism that facilitate a unique kinematic engagement between the tool and coupling mechanism, e.g., shape, profile, indentation, latching features, etc.; unique features built into a sterile drape that is coupled between the coupler body and the coupler interface; etc. In some embodiments, the system may be operably coupled to a docking station configured to receive the surgical instrument therein, record measurements, and detect identifying indicators on the surgical instrument, thereby updating a calibration file and determining whether the surgical instrument is approved.

[0232] Thus, upon coupling of an unauthorized surgical instrument to the robotic arm, the system may generate an audible, visual, and / or haptic alert to notify the user of such unauthorized use so that corrective action can be taken, e.g., replacing the unauthorized tool with an authorized tool. In some embodiments, the system may apply an increased level of stiffness to the robotic arm when an unauthorized tool is coupled to the robotic arm, notify the user via haptic feedback, and / or prevent movement of the system by engaging the robotic arm's braking mechanism and applying impedance via the system's motors. Additionally, some advanced features of the system, such as instrument centering, may be disabled until an authorized tool is used. In some embodiments, upon attachment of an unauthorized tool prior to the start of a procedure, the system may lock the robotic arm via the braking mechanism and motor until the unauthorized tool is replaced with an authorized tool.

[0233] Furthermore, based on image data captured by the laparoscope as well as data acquired by the optical scanner 202, e.g., the tracked movement of the distal end of the laparoscope coupled to the robotic arm 300, and / or robotic telemetry data acquired by the system 100, e.g., the known position / movement of the robotic arm 300 based on the current kinematics of the robotic arm 300 calculated by the system 100, the system may identify the type of laparoscope coupled to the robotic arm 300. For example, laparoscopes commonly used during laparoscopic procedures include flat-tipped laparoscopes and angled-tipped laparoscopes, e.g., laparoscopes with a 30-degree angled tip. The system can determine the laparoscope type currently coupled to the robotic arm 300 by comparing image data acquired by the optical scanner 202 of a predefined pattern of laparoscope movement and / or known kinematic data of the robotic arm 300 during a predefined pattern of laparoscope movement (e.g., while moving the distal end of the laparoscope in a circular pattern in a plane perpendicular to the longitudinal axis of the laparoscope) with image data acquired by the laparoscope as it is moved in the predefined pattern of movement. For example, with respect to a flat-tipped laparoscope, when the distal end of the laparoscope is moved in a circular pattern in a plane perpendicular to the longitudinal axis of the laparoscope, the image data captured by the laparoscope will move along a circular planar path, e.g., there will be no change in the depth of the laparoscope's field of view; on the other hand, with respect to an angled-tipped laparoscope, when the distal end of the laparoscope is moved in a circular pattern in a plane perpendicular to the longitudinal axis of the laparoscope, the image data captured by the laparoscope will observe a change in the depth of the laparoscope's field of view.

[0234] The surgical instrument calibration module 1414 may be executed by the processor 1402 to calibrate a surgical instrument, for example, a surgical instrument that does not currently have an associated calibration file in the database stored in the memory 1410. Thus, when the surgical instrument calibration module 1414 is coupled to the robotic arm 300 and the system is in calibration mode, it may calculate measurements and specifications of the surgical instrument based on force measurements of the robotic arm 300 applied by the surgical instrument via the force detection module 1422, as described in further detail below with respect to FIG. 19 . For example, the surgical instrument calibration module 1414 may generate a calibration file for the surgical instrument that includes information such as the surgical instrument's instrument type, product name, weight, center of mass, length, instrument shaft diameter, viscosity parameters, etc. At least some of the surgical instrument information in the calibration file, e.g., instrument type / product name, may be provided by user input via the user interface 1408, or, for example, the instrument type, center of mass of the instrument, instrument length, and instrument diameter may be detected by the optical scanner interface module 1428.

[0235] Similarly, memory 1410 may include additional modules, such as a system calibration module, that may be executed by processor 1402 to calibrate a new robotic arm when a current robotic arm is replaced, for example, during a surgical procedure, based on data acquired by optical scanner 202, with or without a tracker at the distal end of the new robotic arm, to ensure that the system accurately knows the kinematics of the new robotic arm. Specifically, the system may calibrate optical scanner 202 to platform 200 when the new robotic arm is coupled to platform 200, calibrate the new robotic arm to a base portion of the new robotic arm, and calibrate the new robotic arm to platform 200. For example, based on telemetry data acquired by the optical scanner 202, the system calibration module may compare the actual real-time movement of the new robotic arm as captured by the optical scanner 202 with the expected movement based on commands sent by the system to the new robotic arm to execute a preprogrammed routine (e.g., intended to move the new robotic arm to a specific location), and generate a measure of error indicative of the deviation between the actual real-time movement of the new robotic arm and the expected movement of the robotic arm based on the preprogrammed routine. The system's calibration module may execute an optimization algorithm to further reduce or eliminate the measure of error between the actual real-time movement and the expected movement (e.g., until the measure of error is below a predetermined threshold). This calibration process may occur after the new robotic arm is coupled to the platform 200, when the system is in a predefined calibration mode, or alternatively, in real time during the surgical procedure.

[0236] The encoder interface module 1416 may be executed by the processor 1402 to receive and process angulation measurement data in real time from multiple encoders (e.g., encoders E1-E7) of the robot arm 300. For example, the encoder interface module 1416 may calculate changes over time in the angulation of a link of the robot arm 300 that is rotatably coupled to a given joint associated with the encoder. As described above, the system may include redundant encoders at each joint of the robot arm 300 to ensure safe operation of the robot arm 300. Furthermore, additional encoders may be positioned on the platform 200 to measure the angulation / position of each robot arm relative to the platform 200, e.g., the vertical and horizontal positions of the robot arm relative to the platform 200. Thus, one encoder may be positioned on the platform 200 to measure movement of the robot arm along the vertical axis of the platform 200, and another encoder may be positioned on the platform 200 to measure movement of the robot arm along the horizontal axis of the platform 200.

[0237] The robotic arm position determination module 1418 may be executed by the processor 1402 in real time to determine the position of the robotic arm 300 and the surgical instrument (if applicable) attached thereto in 3D space based on the angulation measurement data generated by the encoder interface module 1416. For example, the robotic arm position determination module 1418 may determine the positions of the various links and joints of the robotic arm 300 and the position along a surgical instrument coupled to the robotic arm 300. Based on the position data of the robotic arm 300 and / or the surgical instrument, the robotic arm position determination module 1418 may calculate, in real time, the velocity and / or acceleration of the movement of the robotic arm 300 and the surgical instrument attached thereto. For example, by determining the individual velocities of the various joints of the robot arm 300 via encoders associated with each of the various joints, the robot arm position determination module 1418 may determine a resultant velocity of the distal end of the robot arm 300, which may be used by the passive mode determination module 1432 to determine whether the movement of the distal end of the robot arm 300 is within a predetermined threshold for purposes of transitioning the system 100 to passive mode, as described in further detail below.

[0238] The trocar position detection module 1420 can be executed by the processor 1402 to determine the position and / or orientation of one or more trocar ports inserted into a patient. The position and / or orientation of the trocar port can be derived based on data obtained from, for example, an inertial measurement unit and / or accelerometer, an optical scanner, an electromechanical tracking device, a linear encoder, or other sensors and data as described above. For example, the position of the trocar port on the patient can be determined using a laser pointing system that can be mounted on one or more of the system's components, such as the wrist portion 311 of the robotic arm, and controlled by the system to point to an optimal or determined position on the patient's body for trocar insertion. Furthermore, upon insertion of a surgical instrument attached to the robotic arm 300 through a trocar, a virtual line can be continuously established along the longitudinal axis of the surgical instrument, the alignment / orientation of which can be automatically determined in real time as the surgical instrument moves around the trocar point upon attachment of the surgical instrument to the coupler interface 400 via the coupler body via a magnetic connection, as described above. Furthermore, when a surgical instrument is inserted into a trocar port, the surgical instrument points toward the trocar point, and therefore the distal wrist link 316 also points toward the trocar point, the angle of which can be measured by its associated encoder. Thus, the trocar point can be calculated as the intersection of multiple imaginary lines established successively along the longitudinal axis of the surgical instrument. In this manner, the calculated trocar point will remain fixed relative to the patient when the surgical instrument is manipulated around the trocar port, e.g., rotated, or moved in and out of the patient. Additionally, the orientation of the trocar port and its position relative to the robotic arm 300 can be determined based on image data received from one or more optical scanners, e.g., LiDAR cameras and / or RGBD cameras.By measuring the true position and orientation of the trocar port, the system can be provided with an additional safety check to ensure that system-level calculations are correct, for example, ensuring that the actual movement of the robotic arm or instrument matches the commanded movement of the robotic arm or instrument in robot-assisted mode.

[0239] Based on the known position of the distal end of the robotic arm 300 from the robotic arm positioning module 1418, plus the known position and / or orientation of the trocar port, the system can maintain the position of the distal end of the robotic arm 300 relative to the trocar point when the robotic arm 300 moves, for example, via its vertical or horizontal adjustment by the platform 200, or when the height of the patient table is adjusted, thereby moving the height of the patient's abdomen, thereby keeping the surgical instrument within the patient's body and coupled to the robotic arm 300 stationary during these external movements. To accomplish this, the known position of the distal end of the robotic arm 300 from the robotic arm positioning module 1418 is calculated in the global frame of the system by adding the position of the platform 200 to the kinematic calculations (e.g., the "forward kinematics" of the robotic arm 300 in the context of a serial-chain robotic manipulator).

[0240] With respect to the globally understood position of the distal end of the robotic arm 300, the system can hold that position stationary during external movement by applying an appropriate force to the robotic arm 300 that minimizes the error between its current position and the desired position. Thus, for example, if a surgical instrument coupled to the distal end of the robotic arm 300 is inserted through a trocar port so that the tip of the instrument is inside the patient, and the user adjusts the height of the patient table, the system can apply a force / torque to the robotic arm 300 to reconfigure the robotic arm 300 and / or cause a movement of the stage of the platform 200 to maintain the relative position between the distal end of the robotic arm 300 (and thus the surgical instrument) and the trocar port. In some embodiments, prior to reconfiguring the robotic arm 300 and maintaining the relative position between the surgical instrument and the trocar port, the system can slightly retract the distal end of the robotic arm 300 so that the tip of the surgical instrument is positioned within the trocar port and out of contact with anatomical structures within the patient's body.

[0241] The force detection module 1422 can be executed by the processor 1402 to detect forces applied to the robotic arm 300 (e.g., at a joint or link of the robotic arm 300 or along a surgical instrument), as well as forces applied to a trocar (e.g., body wall forces). For example, the force detection module 1422 can receive, in real time, motor current measurements at each motor (e.g., M1, M2, M3) located within the base of the robotic arm 300, each operably coupled to a joint (e.g., base joint 303, shoulder joint 318, elbow joint 322, wrist joint 332) of the robotic arm 300. The motor current measurements indicate the amount of force applied to the associated joint. Thus, the force applied to each joint of the robotic arm 300 and its attached surgical instrument can be calculated based on the motor current measurements and position data generated by the robotic arm position determination module 1418 and / or the trocar position detection module 1420.

[0242] Due to the passive axis at the distal end of the robotic arm 300, the force applied to the trocar by an instrument coupled to the robotic arm can generally remain consistent throughout the robotic arm's workspace. The force on the trocar can be affected by the interaction of the instrument's distal tip with tissue within the body. For example, when a tissue retractor advanced through the trocar engages (e.g., grasps) body tissue or another object inside the body, the force exerted on the end of the instrument from the body tissue or other object can cause a change in the force applied to the trocar. In some aspects, the force on the trocar can be a function of the amount of weight being lifted by the instrument being used.

[0243] The impedance calculation module 1424 may be executed by the processor 1402 to determine the amount of impedance / torque needed to be applied to each joint of the robotic arm 300 to achieve a desired effect (e.g., in a passive mode, holding the robotic arm 300 in a stationary position; in a collaborative manipulation mode, allowing the robotic arm 300 to move freely while compensating for the gravity of the robotic arm 300 and its attached surgical instrument; in a haptic mode, applying an increased impedance to the robotic arm 300 when the robotic arm 300 and / or its attached surgical instrument is within a predefined virtual haptic barrier, etc.).

[0244] For example, the impedance calculation module 1424 can determine the amount of force needed by the robotic arm 300 to achieve a desired effect based on the robotic arm 300 position data generated by the robotic arm position determination module 1418 and the trocar position data generated by the trocar position detection module 1420. For example, by determining the position of the distal end of the robotic arm 300 and the entry point of the surgical instrument into the patient (e.g., trocar position), and with knowledge of one or more instrument parameters (e.g., the mass and center of mass of the surgical instrument stored by the surgical instrument calibration module 1414), the impedance calculation module 1424 can calculate the amount of force needed to compensate for the gravity of the surgical instrument (compensation force), as described in further detail below with respect to FIG. 23A . Thus, the amount of compensation force needed to compensate for the gravity of the surgical instrument can be converted into a torque to be applied at a joint of the robotic arm 300 by a motor operably coupled to the joint of the robotic arm 300, for example, as indicated by motor current measurements.

[0245] Furthermore, by determining the position of the distal end of the robotic arm 300, and thus changes in the position of the distal end of the robotic arm 300 over time (e.g., due to external forces applied to the distal end of the robotic arm 300, e.g., by tissue held by the working end of a surgical instrument), and with knowledge of one or more instrument parameters (e.g., the mass, center of mass, and length of the surgical instrument stored by the surgical instrument calibration module 1414), the impedance calculation module 1424 can calculate the amount of force needed to maintain the surgical instrument in a stationary position (holding force), as described in further detail below with respect to FIG. 23B. Thus, the amount of holding force needed to resist changes in the position of the distal end of the robotic arm 300, plus the amount of compensation force needed to compensate for the gravitational force of the surgical instrument, can be converted into torques to be applied at the joints of the robotic arm 300 to maintain the robotic arm 300 in a stationary position, as indicated, for example, by motors operably coupled to the joints of the robotic arm 300, as indicated by motor current measurements. Additionally, the impedance calculation module 1424 and / or the force detection module 1422 may calculate not only the amount of force applied by the surgical instrument to the patient at the entry point (e.g., trocar) but also the amount of force applied to the working end of the surgical instrument (e.g., the grasper end of the surgical instrument) based on one or more parameters of the surgical instrument, such as the compensation force, the holding force, the mass, center of mass, and length of the surgical instrument, and the distance from the center of mass to the entry point.

[0246] Additionally or alternatively, by determining the force applied to the robot arm 300 via the force detection module 1422 and the position / velocity / acceleration of the distal end of the robot arm 300 in 3D space via the robot arm position determination module 1418, a desired force / impedance to be applied to the robot arm 300 to compensate for the applied force can be calculated, e.g., for gravity compensation or to hold the robot arm 300 in a stationary position in a passive mode. Thus, the desired force can be converted into a torque to be applied at the joints of the robot arm 300, e.g., by motors operably coupled to the joints of the robot arm 300. For example, the robot Jacobian matrix can be used for this purpose.

[0247] The motor interface module 1426 may be executed by the processor 1402 to receive motor current readings at each motor, e.g., M1, M2, M3, disposed within the base of the robotic arm 300, and to actuate each motor, e.g., by applying a predetermined impedance, to achieve a desired result as described herein and / or to move a joint operatively coupled to the respective motor, such as in a robotic assist mode. For example, the motor interface module 1426 may actuate M4 to cause rotation of the distal shoulder link 308 relative to the proximal shoulder link 306.

[0248] As described above, the data stream from the robotic arm, the camera feed from the laparoscope, data obtained from the optical scanner 202 and / or proximity sensor 212, and optionally data captured from one or more imaging devices located on structures adjacent to the robotic arm, walls, ceilings, or other structures within the operating room, may be recorded, stored, and used individually or in combination to understand and control the surgical system and the surgical system's procedures. The foregoing components, devices, and combinations thereof are collectively referred to herein as optical scanners or optical scanning devices.

[0249] The optical scanner interface module 1428 may be executed by the processor 1402 to receive depth data acquired by an optical scanning device, e.g., the optical scanner 202, and process the depth data, e.g., to detect predefined conditions therein. Additionally, the optical scanner interface module 1428 may generate a depth map indicative of the received depth data, which may be displayed to an operator, e.g., via a monitor. Based on the depth map generated by the optical scanning device, the optical scanner interface module 1428 may cluster different groups of (depth) pixels into unique objects, a process referred to as object segmentation. Examples of such algorithms for segmentation may include matching the acquired depth map data to known templates of objects for segmentation; using a combination of depth and RGB color images to identify and isolate relevant pixels for objects; and / or machine learning algorithms trained on real or synthetic datasets for the objects to be identified and segmented. Examples of such segmentation on the depth map may include locating a robotic arm or determining the position of the robotic arm; identifying a patient port (e.g., a trocar port) in 3D space and determining the distance from the instrument to the trocar port; determining the relative distances between, for example, the stage of platform 200, robotic arm 300, its attached surgical instruments, and objects / people in the operating room, such as the operating table, drapes, etc.; identifying the surgeon and distinguishing the surgeon from other operators in the room; and / or identifying the surgeon within the field of view of the sensor. Further, the system may use object segmentation algorithms to uniquely identify the surgeon and track the surgeon relative to, for example, the surgical table, the patient, one or more robotic arms, etc. In addition, the system may use object segmentation algorithms to determine whether the surgeon is touching (or handling) any of the robotic arms and, if applicable, identify the robotic arms being touched (or handled) by the surgeon.

[0250] The optical scanner interface module 1428 may further use an object segmentation algorithm to analyze image data acquired from the laparoscope, locate and track one or more surgical instruments and / or anatomical structures, and distinguish the tracked surgical instruments and / or anatomical structures from other objects and structures within the laparoscope's field of view. For example, the object segmentation algorithm may include a deep learning approach. Specifically, a neural network may be trained for instrument / anatomical structure detection via manually annotated video datasets sampled from multiple laparoscopic surgical procedures involving a variety of surgical instruments and anatomical environments. For example, as shown in FIG. 17A , labeled training data including manual annotations indicating surgical instrument / anatomical structure locations and class labels indicating surgical instrument / anatomical structure types may be fed through a feature extractor to generate class labels and train a neural network to identify surgical instrument / anatomical structure locations within the image dataset. The trained neural network may then be implemented by the system to detect, in real time, target surgical instruments / anatomical structures in image data acquired by the laparoscope via the optical scanner interface module 1428 and provide instrument centering, as described in further detail below.

[0251] The optical scanner interface module 1428 may further receive image data from additional optical scanning devices as defined herein (e.g., including an endoscope operably coupled to the system). Additionally, the optical scanner interface module 1428 may receive depth data acquired by a proximity sensor 212 coupled to the platform 200, as described below with respect to FIG. 33 , process the depth data, and generate a virtual map of the area surrounding the platform 200, which may be displayed to an operator via a monitor, e.g., the display 210. For example, the optical scanner interface module 1428 may generate a graphical representation of the system 100, including the platform 200 and the robotic arms 300 a, 300 b, and any objects and / or people in the area surrounding the platform 200, for display on the virtual map to guide the movement of the platform 200 and the robotic arms 300 a, 300 b through the operating room.

[0252] The gesture detection module 1430 may be executed by the processor 1402 to detect predefined gesture patterns as user input and perform an action associated with the user input. The predefined gesture patterns may include, for example, movement of a surgical instrument (whether attached to the robotic arm 300 or not), movement of the robotic arm 300 or other components of the system, such as foot pedals, buttons, etc., and / or movement of an operator in a predefined pattern. For example, back-and-forth movement of a surgical instrument in a first direction (e.g., left / right, up / down, forward / backward, circular) may be associated with a first user input requesting a first action by the system, and / or back-and-forth movement in a second direction different from the first direction (e.g., left / right, up / down, forward / backward, circular) may be associated with a second user input requesting a second action by the system. Similarly, pressing a foot pedal or button operably coupled to the system in a predefined manner may be associated with a third user input requesting a third action by the system, and repeatedly moving the operator's head back and forth or up and down may be associated with a fourth user input requesting a fourth action by the system. Various predefined gesture patterns associated with different components or operators of the system may be redundant, whereby the associated user input may be the same for different gesture patterns. The predefined gesture patterns may be detected by an optical scanning device, such as a laparoscope or optical scanner 202, via, for example, the optical scanner interface module 1428, or may be detected directly by forces applied to the robotic arm 300 via the force detection module 1422 or other components of the system.

[0253] Actions responsive to user input associated with predefined gesture patterns may include, for example, enabling tool tracking and servoing (i.e., moving) the laparoscope based on the movement of the handheld tool and / or automatically maintaining the handheld tool within the field of view of the laparoscope; applying a braking force on the robotic arm (e.g., preventing its further movement); applying a software lock on the robotic arm; dynamically changing the amount of time it takes the robotic arm to transition between states from a default setting; loading a virtual menu overlay onto the video feed whereby a surgical instrument in the field of view of the laparoscope acts as a pointer to trigger further actions available from the virtual menu; starting / stopping recording of image data; and / or, if applicable, identifying a member of the surgical staff touching the robotic arm. This information can be used to ensure that the system does not move if the surgeon is not touching it, for example, to avoid scenarios where an external force acts on the robotic arm (e.g., an optical cable or other wire is pulled across the robotic arm) and the system perceives the force as intentional from the surgeon. The same information can be used to detect the surgeon's gaze direction, for example, to detect whether the surgeon is looking at the video feed or elsewhere in the room, so that the system can freeze the robotic arm if the surgeon's gaze is not where it should be. Additionally, the system can reposition the camera's field of view based on the direction the surgeon is facing or the object the surgeon is believed to be looking at, for example, based on data from the optical scanner 1100. Furthermore, moving the distal tip of a surgical instrument into a central portion of the laparoscopic field of view (e.g., defined by a predetermined boundary area) and holding that position for above a predetermined time threshold can be associated with a user input to enable tool tracking, as detected by gesture detection module 1430, as described in further detail below.

[0254] Additionally, predefined gesture patterns, such as a double tap of the distal portion of the robotic arm and / or a predetermined sinusoidal movement of the laparoscope's camera head around the trocar, can be associated with user inputs for starting and / or stopping the recording of image / audio data by the optical scanning device, as detected by the gesture detection module 1430. Specifically, there may be key moments during a procedure that the user may want to record, and the user may want to be able to identify them in a quick manner without having to sift through the recording of the entire procedure to find them. By providing an easy way for the user to start and stop recording via a simple predefined gesture pattern, such that the recording is saved in a folder with a timestamp associated with that particular procedure, the user may easily identify the recording for review and / or teaching purposes. This feature may be particularly useful for diagnostic procedures. In some embodiments, in response to detection of a predefined gesture pattern by the gesture detection module 1430, the system may record and save a predetermined portion of image data, e.g., 10 seconds before and 10 seconds after the predefined gesture pattern is detected. Additionally, the user's selected recording of key moments can be used by the system to indicate key phase segmentation for a given procedure. The user can also generate a case record via recording when performing a procedure based on a procedure template accessible via the system, for example, by showing progression through different phases of the procedure.

[0255] As described above, in response to detecting a predefined gesture pattern by the user, for example, a predefined pattern of movement of the distal tip of a surgical instrument within the laparoscope's field of view, the gesture detection module 1430 may cause a virtual menu to be overlaid on the video feed such that the surgical instrument within the laparoscope's field of view acts as a pointer, as shown in Figure 15. Additionally, the gesture detection module 1430 may detect additional predefined patterns of movement of the distal end of the surgical instrument, for example, two quick movements in the same direction or a circular movement over a selection area of ​​the virtual menu, which may be interpreted as a selection actuation, for example, a click on the virtual menu. For example, as shown in FIG. 15 , a virtual menu overlay on the video feed may include menu options at the corners of the video feed, e.g., “hot corners,” for: turning on / off an instrument centering mode in which the system automatically moves a robotic arm coupled to the laparoscope to follow and / or zoom in or out a surgical instrument, changes the field of view of the laparoscope, and maintains the target instrument within a predetermined reference distance from the tip of the laparoscope; adjusting the holding force of a robotic arm coupled to a retractor, e.g., the amount of force that can be applied to the distal tip of a surgical instrument before the system transitions from a passive mode to a collaborative manipulation mode; turning audio on / off; and turning haptic feedback on / off. As will be understood by those skilled in the art, more or fewer menu options may be provided via the virtual menu.

[0256] In some embodiments, initiation of the display of the virtual menu overlay on the video feed may be triggered by, for example, actuation of an external actuator such as a foot pedal, a predefined pattern of force applied to the robotic arm, such as double-tapping the wrist portion 311 and / or a surgical instrument coupled to the robotic arm as detected by an encoder at the distal end of the robotic arm, voice activation, a wireless button, a hot button, etc. In some embodiments, the operator may actively switch the system, for example via the user interface 1408, and certain movements or gestures of the robotic arm, surgical instrument, operator, or other movements or gestures as described herein are monitored by the gesture detection module 1430 to determine whether they match predefined gesture patterns associated with predefined user inputs.

[0257] The passive mode determination module 1432 may be executed by the processor 1402 to analyze the operating characteristics of the robotic arm 300 and determine whether to switch the operating mode of the robotic arm 300 to a passive mode, in which the system applies impedance to the joints of the robotic arm 300 via the motor interface module 1426 in an amount sufficient to maintain the robotic arm 300, and therefore, if applicable, the surgical instrument attached thereto, in a stationary position, thereby compensating for the mass of the robotic arm 300 and the surgical instrument, and any other external forces acting on the robotic arm 300 and / or the surgical instrument. If the robotic arm 300 is moved slightly while in passive mode but not with enough force to switch out of passive mode, the system may adjust the amount of impedance applied to the robotic arm 300 to maintain a stationary position, continuing this process until the robotic arm 300 is held in a stationary position. For example, the passive mode determination module 1432 may determine to switch the operational mode of the robotic arm 300 to passive mode if the movement of the robotic arm due to movement at the handle of a surgical instrument, as determined by the force detection module 1422, is less than a predetermined amount, e.g., 1-5 mm or less, for at least a predetermined dwell time associated with the robotic arm 300. The predetermined dwell time refers to the length of time the robotic arm 300 and / or, if applicable, the attached surgical instrument, are held in a stationary position. For example, the predetermined dwell time may range from, e.g., 0.1 to 3 seconds or more and may be adjusted by the operator. FIG. 16 illustrates a table or example values ​​of threshold dwell times for a range of sample instrument types.

[0258] In some embodiments, the passive mode determination module 1432 may determine to switch the operational mode of the robotic arm 300 to passive mode if movement of the distal end of the robotic arm due to movement at the handle of the surgical instrument, as determined by the force detection module 1422, has a speed that is less than a predetermined dwell speed / velocity. For example, if the passive mode determination module 1432 determines that the distal end of the robotic arm 300 and / or, if applicable, the surgical instrument attached thereto is moving at a speed slower than a predetermined dwell speed for the entire predetermined dwell time, the passive mode determination module 1432 may switch the operational mode of the robotic arm 300 to passive mode. Figure 16 illustrates a table or example values ​​of threshold dwell speeds for a range of sample instrument types. For example, for surgical instruments such as speculum and tissue manipulation devices, the threshold dwell speed may be, for example, 3-5 mm / sec, and for surgical instruments such as suturing instruments, needle holders, high force instruments, staplers, and clip appliers, the threshold dwell speed may be, for example, 1-2 mm / sec. In some embodiments, the passive mode determination module 1432 may determine to switch the operating mode of the robotic arm 300 to the passive mode based on the identification of the surgical instrument upon attachment of the surgical instrument to the robotic arm 300 and / or corresponding detachment of the surgical instrument from the robotic arm 300.

[0259] The collaborative operation mode determination module 1434 may be executed by the processor 1402 to analyze the operating characteristics of the robotic arm 300 and determine whether to switch the operating mode of the robotic arm 300 to a collaborative operation mode, in which the robotic arm 300 is allowed to move freely in response to movements at the handles of surgical instruments to perform laparoscopic surgery using surgical instruments, while the system applies impedance to the robotic arm 300 via the motor interface module 1426 in an amount sufficient to account for the mass of the surgical instruments and the robotic arm 300. Additionally, the impedance applied to the robotic arm 300 may provide a predetermined level of stickiness perceptible by the operator. FIG. 16 illustrates a table or example values ​​of stickiness levels for a range of sample instrument types. In some embodiments, the stickiness level may be a function of the speed at which the surgical instrument is being moved and the distance from the trocar point to the tip of the instrument. For example, the collaborative manipulation mode determination module 1434 may determine to switch the operating mode of the robotic arm 300 to the collaborative manipulation mode if the force applied at the robotic arm 300 due to a force applied at the handle of a surgical instrument exceeds a predetermined threshold (e.g., a "breakaway force") associated with the robotic arm 300. The predefined force threshold may be, for example, at least 7 Newtons, approximately 7 Newtons, at least 7 Newtons, 4-15 Newtons, or 4-10 Newtons. The predefined force threshold may depend on the type of surgical instrument being used and / or whether there is an external force being applied to the surgical instrument.

[0260] FIG. 16 illustrates a table of predefined force thresholds or example values ​​for a range of sample instrument types. As shown in FIG. 16, the predefined force thresholds may reflect typical external tissue forces that may be exerted on a surgical instrument. In some embodiments, the predefined force thresholds may be increased depending on the direction of the breakaway force when the force is exerted on the surgical instrument by the tissue or organ or otherwise. For example, as described in more detail below with respect to FIGS. 27A and 27B, if the breakaway force is in the same direction as the force exerted on the surgical instrument from the tissue or organ, the predefined force threshold may be increased by an amount equal to or corresponding to the force exerted on the surgical instrument from the tissue or organ. In some embodiments, the predefined force thresholds for each robotic arm are adjusted based on the patient's body mass index ("BMI"). For example, patients with higher BMIs may have heavier livers that will likely exert greater forces on the instrument. Thus, the predefined force thresholds may be selected to be higher for patients with higher BMIs. Thus, an operator may activate a "high force mode," for example, via user interface 1408, in which the predefined force threshold is increased to accommodate working with heavier tissue or organs. For example, the predefined force threshold may be selectively increased by 20-100% or more.

[0261] Furthermore, the force exerted by the user on the surgical instrument and any external tissue force applied to the surgical instrument may be direction-dependent. For example, if the force exerted by the user on the surgical instrument is in the same direction as the external tissue force applied to the surgical instrument, the two forces may be additive, such that the amount of force exerted on the surgical instrument required to overcome a predefined force threshold by the user may be reduced by the magnitude of the external tissue force, such that a force below the predefined force threshold would be required to exit the passive mode and enter the collaborative operation mode. On the other hand, if the force exerted on the surgical instrument by the user is in the opposite direction to the external tissue force applied to the surgical instrument, the necessary amount of force exerted on the surgical instrument required to overcome a predefined force threshold by the user may be increased by the magnitude of the external tissue force, such that a force above the predefined force threshold would be required to exit the passive mode and enter the collaborative operation mode.

[0262] Additionally, if the force exerted by the user on the surgical instrument is in a direction perpendicular to the external tissue force applied to the surgical instrument, the required amount of force exerted by the user on the surgical instrument required to overcome the predefined force threshold may not be affected by the magnitude of the external tissue force, such that the required force exerted by the user on the surgical instrument required to exit the passive mode and enter the collaborative manipulation mode will be equal to the predefined force threshold. For other directions, the force vector of the applied force may be added to or offset from the force vector of the external tissue force to overcome the predefined force threshold for the particular surgical instrument coupled with the system or robotic arm, depending on the external tissue force, if applicable, and the direction of the force applied by the user. In some embodiments, the collaborative manipulation mode determination module 1434 may determine to switch the operating mode of the robotic arm 300 to the collaborative manipulation mode based on the identification of the surgical instrument.

[0263] The haptic mode determination module 1436 may be executed by the processor 1402 to analyze the motion characteristics of the robotic arm 300 and determine whether to switch the operational mode of the robotic arm 300 to a haptic mode, in which the system applies impedance to the robotic arm 300 via the motor interface module 1426 at a higher amount than that applied in the collaborative manipulation mode, thereby making the movement of the robotic arm 300 in response to movement at the handle of the surgical instrument more sticky than in the collaborative manipulation mode. For example, the haptic mode determination module 1436 may determine to switch the operational mode of the robotic arm 300 to the haptic mode if at least a portion of the robotic arm 300 and / or a surgical instrument attached thereto is within a predefined virtual haptic boundary. Specifically, a virtual haptic boundary may be established by the system to prevent the robotic arm or a surgical instrument coupled thereto from breaching the boundary. For example, a virtual boundary may be established at the surface of a patient to prevent any portion of the robotic arm or an instrument supported by the robotic arm from contacting the patient except through one or more trocars. Similarly, the virtual haptic boundary may include a haptic funnel to help guide the instrument into the patient as the operator inserts the instrument into the trocar port.

[0264] Additionally, a virtual haptic boundary, e.g., a haptic shell, may be established at a predetermined distance surrounding the workspace to prevent overextension of the robot arm away from the operating site, as well as minimize “runaway” of the robot arm. For example, after an instrument is uncoupled from the coupler body coupled to the robot arm, the magnet in the coupler body described above should return to its maximum position away from the repelling magnet in the coupler interface 400, thereby indicating that the surgical instrument has been removed. However, if the magnet does not return to its position, the system may believe that the surgical instrument is still attached to the robot arm and continue to compensate for the mass of the surgical instrument, thereby causing the distal end of the robot arm to “runaway,” e.g., drift upward. Thus, the virtual haptic boundary may slow the drift of the robot arm to avoid potential collisions with other objects or people. For example, the virtual haptic boundary may be set at the user's chest level to prevent the robot arm from colliding with the user's head.

[0265] Thus, for example, based on position data of the robot arm 300 and / or a surgical instrument coupled thereto received by the robotic arm position determination module 1418 and / or the trocar position detection module 1420, the haptic mode determination module 1436 may determine whether the robot arm 300 and / or the surgical instrument are within a predefined virtual haptic boundary and, therefore, whether to transition the robot arm 300 to a haptic mode, in which the processor 1402 may command associated motors to apply an effective amount of impedance to the joints of the robot arm 300 that is perceptible by the operator to communicate the virtual haptic boundary to the operator. Thus, the stickiness of the robot arm 300 observed by the operator will be much higher than in the collaborative manipulation mode. In some embodiments, the haptic mode determination module 1436 may determine to switch the operating mode of the robot arm 300 to the haptic mode based on the identification of the surgical instrument.

[0266] Additionally, the haptic mode determination module 1436 may generate a temporary local virtual haptic boundary at the distal end of the robotic arm during predetermined phases of the procedure / clinical workflow to prevent “runaway” and further improve system safety. The predetermined phases may include, for example, during draping / during draping tearing, immediately after tool removal is detected, and / or immediately after coupler body removal is detected. For example, during draping / deepening, increasing the stickiness of the robotic arm may help stabilize the robotic arm and prevent its excessive movement / runaway, and during tool / coupler body removal, increasing the stickiness may prevent “runaway” due to forces applied to the robotic arm by the user during the removal process. The local virtual haptic boundary may be temporary in that it may only be applied for a predetermined period of time, e.g., several seconds, after the predetermined phase is identified. Certain aspects of the procedure may be determined / estimated, for example, via user input via the GUI 210 and / or voice commands, uploaded from a database stored within the system, and / or via telemetry of the robotic arm and the identification and / or location of the surgical instrument. For example, the system may collect and analyze telemetry data regarding the forces being applied to the robotic arm to assess or estimate whether the user is attempting to remove a tool from the robotic arm; thus, the haptic mode determination module 1436 may generate a temporary, local virtual haptic boundary at the distal end of the robotic arm to facilitate tool removal.

[0267] Additionally, the kinesthetic mode determination module 1436 may adjust the amount of tenacity, e.g., impedance, applied to the distal end of the robotic arm during a given phase of a surgical procedure to guide specific movements based on the type of surgical instrument coupled to the robotic arm during the given phase (e.g., a wrist-jointed instrument such as a needle holder or grasper, a stapling device, a dissection device, a suturing device, a retraction device such as a fan retractor, a tissue removal device such as a bladder bag, a clip applier device, etc.). For example, during the suturing phase of the procedure, the tenacity at the distal end of the robotic arm may be increased to provide the user with greater tenacity control during actuation of the suturing device. Additionally, during the stapling phase of the procedure, the tenacity at the distal end of the robotic arm may be increased to provide a very tight ground for the force application of the stapling device actuated by the user. Thus, the increased tenacity may facilitate performance of specific movements by the user without actively moving the robotic arm and performing the specific movements.

[0268] The robot-assisted mode determination module 1438 may be executed by the processor 1402 to analyze the motion characteristics of the robot arm 300 and determine whether to switch the motion mode of the robot arm 300 to a robot-assisted mode, wherein in the robot-assisted mode: The processor 1402 may command associated motors via the motor interface module 1426 to cause movement of corresponding links and joints of the robotic arm 300 to achieve a desired result. For example, the robot-assisted mode determination module 1438 may determine, based on data obtained from, for example, the optical scanner interface module 1428, to switch the operational mode of the robotic arm 300 to a robot-assisted mode if a predefined condition exists.

[0269] For example, the robot-assisted mode decision module 1438 may determine that a condition exists, such as that one or more trocars are not in an optimal position due to, for example, patient movement, such that the robotic arm 300 should be repositioned to maintain the trocars in an optimal position (e.g., approximately at the center of the robotic arm 300's range of motion), thereby minimizing the risk of reaching the robotic arm's joint limits during the procedure. Thus, in the robot-assisted mode, the processor 1402 may instruct the system to reposition the robotic arm 300 to better align the surgical instrument workspace, for example, via vertical / horizontal adjustments by the platform 100 or via the robotic arm's 300 joints and links.

[0270] The robotic-assisted mode determination module 1438 may determine, based on image data obtained from the laparoscope or optical scanner 202 via the optical scanner interface module 1428, that a certain condition exists, such as the distance between an object, e.g., equipment or a member of the surgical staff other than the surgeon, and the robotic arm 300 reaching or falling below a predetermined threshold, whereby the robotic arm should be frozen to avoid collision with the object. Thus, in robotic-assisted mode, the processor 1402 may instruct the robotic arm 300 to apply a braking force to slow the robotic arm or block or prevent movement within a predetermined distance from other objects, thereby preventing inadvertent movement of the robotic arm that might otherwise result from such a collision or inadvertent force.

[0271] The robot-assisted mode determination module 1438 may further determine that a condition exists, e.g., the robotic arm 300 has been in an extended position for a period of time during the surgical procedure that exceeds a predetermined threshold, whereby the robotic arm should be repositioned to provide more available workspace for the user in proximity to the surgical instrument coupled to the extended robotic arm. Thus, in the robot-assisted mode, the processor 1402 may instruct the system to reposition the robotic arm 300 and move it closer to the surgical instrument, e.g., via vertical / horizontal adjustment by the plat...

Claims

1. 1. A collaborative surgical system for operating a robotic arm comprising a plurality of links, a plurality of joints, and a distal end configured to be removably coupled to a surgical instrument, the collaborative surgical system comprising at least one processor, the at least one processor comprising: in response to determining that movement of the robotic arm due to movement at the handle of the surgical instrument is less than a predetermined amount for at least a predetermined dwell time, causing the robotic arm to switch to a passive mode, wherein the at least one processor is configured to cause the robotic arm to maintain a stationary position in the passive mode; applying gravity compensation to the robotic arm to compensate for gravity of the surgical instrument; calculating a holding force required to maintain the distal end of the robot arm in the rest position in the passive mode during application of the gravity compensation; establishing a baseline retention force based on the retention force; applying a breakaway force threshold to the robot arm based on the baseline holding force; configured to: The breakaway force threshold is a predetermined amount of force that must be applied to the robotic arm to cause the robotic arm to exit the passive mode.

2. The cooperative surgical system of claim 1 , wherein the magnitude of the withdrawal force threshold is equal in all directions relative to the baseline holding force.

3. 2. The collaborative surgical system of claim 1, wherein a total amount of force that needs to be applied to the robotic arm in a direction to cause the robotic arm to exit the passive mode is the sum of the baseline holding force in that direction and the breakaway force threshold.

4. The collaborative surgical system of claim 1 , wherein the holding force required to maintain the distal end of the robotic arm in the stationary position is continuously calculated in the passive mode.

5. 5. The collaborative surgical system according to claim 4, wherein the at least one processor is configured to determine that the surgical instrument is in contact with one or more anatomical structures in the passive mode if the holding force gradually increases over time.

6. 2. The collaborative surgical system of claim 1, wherein the at least one processor is configured to calculate the holding force required to maintain the distal end of the robotic arm in the rest position in the passive mode when one or more external forces are applied to the surgical instrument by one or more anatomical structures having unknown mass.

7. 2. The collaborative surgical system of claim 1, wherein the at least one processor is configured to determine a force that needs to be applied to the distal end of the robotic arm to move the distal end of the robotic arm from a current position to the rest position, and to calculate the holding force needed to maintain the distal end of the robotic arm at the rest position in the passive mode.

8. 2. The cooperatively manipulated surgical system of claim 1, wherein the at least one processor is configured to automatically cause the robotic arm to switch to a cooperative manipulation mode in response to determining that the holding force required to maintain the distal end of the robotic arm in the stationary position exceeds the breakaway force threshold, and wherein the at least one processor is configured to apply the gravity compensation to the robotic arm in the cooperative manipulation mode to allow the robotic arm to be freely movable in the cooperative manipulation mode in response to movement at a handle of the surgical instrument while compensating for gravity of the surgical instrument.

9. 2. The collaborative surgical system of claim 1, wherein the at least one processor is configured to sense a force applied to the distal end of the robotic arm and calculate the holding force needed to maintain the distal end of the robotic arm in the rest position in the passive mode.

10. 10. The collaborative surgical system of claim 9, wherein the at least one processor is configured to measure currents in a plurality of motors operatively coupled to at least some of the plurality of joints and sense forces applied to the distal end of the robotic arm.

11. 2. The collaborative surgical system of claim 1, wherein the at least one processor is configured to apply torques to at least some of a plurality of joints of the robotic arm, apply the gravity compensation to the robotic arm, and compensate for gravity on the surgical instrument.

12. The collaborative surgical system of claim 1 , wherein the at least one processor is configured to establish the baseline holding force after a predetermined period of time upon initiation of the passive mode.

13. The at least one processor configured to apply a high breakaway force threshold to the robot arm during the predetermined period, the high breakaway force threshold being greater than the breakaway force threshold; 13. The collaborative surgical system of claim 12, wherein the at least one processor is configured to cause the robotic arm to exit the passive mode if the holding force required to maintain the distal end of the robotic arm in the stationary position exceeds the high breakaway force threshold for the predetermined period of time.

14. 14. The collaborative surgical system of claim 13, wherein the high breakaway force threshold is selected to prevent inadvertent disengagement of the robotic arm from the passive mode in response to an inadvertent force applied to the distal end of the robotic arm during the predetermined period of time.

15. The at least one processor applying an initial separation force threshold to the robotic arm for the predetermined period of time; applying an upper separation force threshold for the predetermined period of time if the force applied to the distal end of the robotic arm exceeds the initial separation force threshold for the predetermined period of time; configured to: the high separation force threshold is greater than the separation force threshold; 13. The collaborative surgical system of claim 12, wherein the at least one processor is configured to cause the robotic arm to exit the passive mode if the holding force required to maintain the distal end of the robotic arm in the stationary position exceeds the high breakaway force threshold for the predetermined period of time.

16. The at least one processor configured to apply a default breakaway force threshold to the robot arm if the holding force varies such that, upon initiation of the passive mode, after the predetermined period of time, the baseline holding force cannot be established based on the calculated holding force; 13. The collaborative surgical system of claim 12, wherein the at least one processor is configured to cause the robotic arm to exit the passive mode if the holding force required to maintain the distal end of the robotic arm in the stationary position exceeds the default breakaway force threshold.

17. 17. The collaborative surgical system according to claim 16, wherein the at least one processor is configured to select the default breakaway force threshold from among a default high breakaway force threshold and a default low breakaway force threshold based on user input via a graphical user interface operatively coupled to the at least one processor.

18. 20. The collaborative surgical system of claim 17, wherein the at least one processor is configured to adjust at least one of the default high breakaway force threshold or the default low breakaway force threshold based on user input via the graphical user interface.

19. The at least one processor configured to monitor the position of the distal end of the robotic arm during the predetermined period; 13. The collaborative surgical system of claim 12, wherein the at least one processor is configured to cause the robotic arm to exit the passive mode if a rate of change of position of the distal end of the robotic arm exceeds a predetermined position threshold for the predetermined period of time.

20. The at least one processor calculating a first force applied to the distal end of the robotic arm at a first time during the predetermined period of time; applying a first separation force threshold to the robot arm based on the first force at a second time after the first time during the predetermined period of time; calculating a second force applied to the distal end of the robotic arm at the second time; and configured to:

13. The collaborative surgical system of claim 12, wherein the at least one processor is configured to cause the robotic arm to exit the passive mode if the second force applied to the distal end of the robotic arm at the second time exceeds the first breakaway force threshold.

21. The at least one processor configured to apply a second separation force threshold to the robot arm based on the second force at a third time after the second time during the predetermined period of time; The cooperatively manipulated surgical system of claim 20, wherein the first and second breakaway force thresholds have magnitudes greater than the first and second forces, respectively, by a certain predetermined force amount.

22. The at least one processor applying a first separation force threshold to the robot arm for a first predetermined period during the predetermined period; applying a second separation force threshold to the robot arm for the remainder of the predetermined period of time; and wherein the second separation force threshold is greater than the first separation force threshold and the separation force threshold; 13. The collaborative surgical system of claim 12, wherein the at least one processor is configured to cause the robotic arm to exit the passive mode if the holding force required to maintain the distal end of the robotic arm in the stationary position exceeds the first breakaway force threshold for the first predetermined period of time or exceeds the second breakaway force threshold for the remainder of the predetermined period of time.

23. The at least one processor applying the gravity compensation to the robotic arm to compensate for gravity of the surgical instrument based on estimated instrument parameters associated with the surgical instrument; determining calibrated instrument parameters for the surgical instrument based on the holding force; applying adjusted gravity compensation to the robot arm based on the calibrated instrument parameters; and configured to:

2. The collaborative surgical system of claim 1, wherein the baseline holding force is established based on the holding force required to maintain the distal end of the robotic arm in the rest position in the passive mode upon application of the adjusted gravity compensation to the robotic arm.

24. 24. The cooperatively manipulated surgical system of claim 23, wherein the calibrated instrument parameters are selected such that during application of the adjusted gravity compensation, the holding force is adjusted within a predetermined range associated with known parameters of the surgical instrument.

25. 1. A collaborative surgical system for providing adaptive gravity compensation to a robotic arm having a plurality of links, a plurality of joints, and a distal end configured to be removably coupled to a surgical instrument, the collaborative surgical system comprising at least one processor, the at least one processor comprising: applying an initial gravity compensation to the robotic arm to compensate for gravity of the surgical instrument based on estimated instrument parameters associated with the surgical instrument; calculating a holding force required to maintain the distal end of the robot arm in a stationary position in a passive mode during application of the initial gravity compensation; determining calibrated instrument parameters for the surgical instrument based on the holding force; and configured to: a coordinated manipulation surgical system, wherein the calibrated instrument parameters are selected to adjust the holding force required to maintain the distal end of the robot arm in the rest position in the passive mode during application of adjusted gravity compensation to the robot arm based on the calibrated instrument parameters.

26. 26. The collaborative surgical system of claim 25, wherein the at least one processor is configured to apply torque to one or more motorized joints of a plurality of joints of the robotic arm and apply the initial gravity compensation to the robotic arm to compensate for gravity on the surgical instrument.

27. 26. The collaborative surgical system of claim 25, wherein the estimated and calibrated instrument parameters comprise at least one of a mass or a center of mass associated with the surgical instrument.

28. The at least one processor configured to load a calibration file associated with known parameters of the surgical instrument; The collaborative surgical system of claim 25, wherein the calibration file comprises the estimated instrument parameters.

29. The cooperative surgical system of claim 28, wherein the known parameter comprises a diameter of an elongate shaft of the surgical instrument.

30. 29. The collaborative surgical system of claim 28, wherein the at least one processor is configured to determine the known parameter upon coupling of the surgical instrument to the distal end of the robotic arm via a coupler body removably coupled to the surgical instrument and the distal end of the robotic arm.

31. The cooperatively manipulated surgical system of claim 30, wherein the at least one processor is configured to determine the known parameter based on the coupler body.

32. further comprising an optical sensor configured to collect depth data; The collaborative surgical system of claim 28, wherein the at least one processor is configured to determine the known parameter based on the depth data.

33. further comprising a user interface operably coupled to the at least one processor; The collaborative surgical system of claim 28, wherein the at least one processor is configured to determine the known parameters via user input received by the user interface.

34. 26. The collaborative surgical system of claim 25, wherein the calibrated instrument parameters are selected to adjust the holding force within a predetermined range associated with known parameters of the surgical instrument during application of the adjusted gravity compensation based on the calibrated instrument parameters.

35. 26. The collaborative surgical system of claim 25, wherein the calibrated instrument parameters are selected to adjust the holding force to zero or near zero upon application of the adjusted gravity compensation based on the calibrated instrument parameters when the distal end of the robotic arm is not subjected to any external forces other than gravity on the robotic arm and the surgical instrument in the rest position.

36. 26. The collaborative surgical system of claim 25, wherein the calibrated instrument parameters are selected to adjust the holding force within a predetermined range associated with known parameters of the surgical instrument when the distal end of the robotic arm is subjected to one or more external forces in addition to gravity on the robotic arm and the surgical instrument in the rest position.

37. The at least one processor calculating an adjusted gravity compensation for the surgical instrument based on the calibrated instrument parameters; applying the adjusted gravity compensation to the robotic arm to compensate for the gravity of the surgical instrument; The cooperative surgical system of claim 25 configured to:

38. 38. The collaborative surgical system of claim 37, wherein the at least one processor is configured to apply torque to one or more motorized joints of a plurality of joints of the robotic arm and apply the adjusted gravity compensation to the robotic arm to compensate for gravity on the surgical instrument.

39. 38. The collaborative surgical system of claim 37, wherein the at least one processor is configured to automatically cause the robotic arm to switch to a collaborative manipulation mode in response to determining that a force applied to the robotic arm due to a force applied to the surgical instrument handle exceeds a predetermined force threshold, and wherein the at least one processor is configured to apply the adjusted gravity compensation to the robotic arm in the collaborative manipulation mode to allow the robotic arm to be freely movable in the collaborative manipulation mode in response to movement at the surgical instrument handle while compensating for gravity of the surgical instrument.

40. 26. The collaborative surgical system of claim 25, wherein the at least one processor is configured to calculate the adjusted holding force to maintain the distal end of the robotic arm at the rest position in the passive mode upon application of the adjusted gravity compensation.

41. The at least one processor Upon initiation of the passive mode, after a predetermined period of time, establishing a baseline holding force based on the adjusted holding force; applying a predetermined constant separation force threshold to the robot arm based on the baseline holding force; configured to:

41. The collaborative surgical system of claim 40, wherein the at least one processor does not maintain the distal end of the robotic arm in the rest position if the holding force exceeds the predetermined constant breakaway force threshold.

42. 42. The collaborative surgical system of claim 41, wherein the at least one processor is configured to apply a predetermined high breakaway force threshold for the predetermined period of time, and wherein the at least one processor does not maintain the distal end of the robotic arm in the stationary position if the holding force exceeds the predetermined high breakaway force threshold for the predetermined period of time.

43. 26. The collaborative surgical system of claim 25, wherein the at least one processor is configured to automatically switch the robotic arm to the passive mode in response to determining that movement of the robotic arm due to movement at the handle of the surgical instrument is less than a predetermined amount for at least a predetermined dwell time.

44. The collaborative surgical system of claim 25, wherein the at least one processor is configured to record the calibrated instrument parameters in a calibration file associated with the surgical instrument.

45. 1. A co-operative surgical system for assisting in laparoscopic surgery performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween, the co-operative surgical system comprising: a robotic arm including a proximal end, a distal end configured to be removably coupled to the surgical instrument, a plurality of links, and a plurality of joints between the proximal end and the distal end; a controller operably coupled to the robotic arm; Equipped with the robotic arm is configured to be freely movable in response to movement at the handle of the surgical instrument to perform a laparoscopic surgical procedure; The controller causing the robotic arm to maintain a stationary position in a passive mode in response to determining that movement of the robotic arm due to movement at the handle of the surgical instrument is less than a predetermined amount for at least a predetermined dwell time; when the surgical instrument comprises a laparoscope having a field of view, identifying a target surgical instrument within the field of view of the laparoscope based on the image data from the laparoscope; causing the robotic arm to switch to an instrument centering mode; It is programmed to In the instrument centering mode, the robotic arm moves the laparoscope to maintain the target surgical instrument within the field of view of the laparoscope.

46. 46. ​​The cooperatively manipulated surgical system of claim 45, wherein the controller is configured to automatically cause the robotic arm to switch to a cooperative manipulation mode in response to determining that a force applied to the robotic arm due to a force applied to the handle of the surgical instrument exceeds a predetermined threshold, and wherein the controller is configured to apply an impedance to the robotic arm in the cooperative manipulation mode to allow the robotic arm to be freely movable in the cooperative manipulation mode in response to movement at the handle of the surgical instrument while accounting for a weight of the surgical instrument and the robotic arm.

47. 46. ​​The collaborative surgical system of claim 45, wherein the controller is configured to identify the target surgical instrument within the field of view of the laparoscope by detecting a predefined gesture pattern by the target surgical instrument within the field of view of the laparoscope.

48. 48. The collaborative surgical system of claim 47, wherein the predefined gesture pattern comprises positioning the target surgical instrument within a central portion of the field of view of the laparoscope and maintaining the position of the target surgical instrument within the central portion for at least a predetermined holding period.

49. 46. ​​The collaborative surgical system of claim 45, wherein the controller is configured to identify the target surgical instrument within the field of view of the laparoscope based on user input identifying the target surgical instrument.

50. 46. ​​The collaborative surgical system of claim 45, wherein the controller is configured to identify the target surgical instrument within the field of view of the laparoscope based on a direction in which the target surgical instrument enters the field of view of the laparoscope, the direction corresponding to a predefined direction associated with a known handedness of a user manipulating the target surgical instrument.

51. 51. The collaborative surgical system of claim 50, wherein the known handedness of the user is stored in a user profile associated with the user, and the controller is configured to execute the user profile.

52. 46. ​​The collaborative surgical system of claim 45, wherein the controller is configured to distinguish the target surgical instrument from one or more other surgical instruments within the field of view of the laparoscope.

53. 46. ​​The collaborative surgical system of claim 45, wherein in the instrument centering mode, the controller causes the robotic arm to move the laparoscope to maintain the target surgical instrument within a predefined boundary area within the field of view of the laparoscope, whereby the robotic arm does not move the laparoscope unless the target surgical instrument moves outside the predefined boundary area.

54. 54. The collaborative surgical system of claim 53, wherein the controller is configured to determine whether a motion pattern of the target surgical instrument is small motion, and wherein, in the instrument centering mode, the controller does not cause the robotic arm to move the laparoscope to maintain the target surgical instrument within the predefined boundary area if the motion pattern of the target surgical instrument is small motion.

55. 55. The collaborative surgical system of claim 54, wherein the controller is configured to detect a speed or acceleration of the tracked surgical instrument based on the image data, and wherein the controller determines that the motion pattern of the target surgical instrument is small motion if the tracked surgical instrument moves outside the predefined boundary area at a speed or acceleration that exceeds a predetermined speed or acceleration threshold.

56. 55. The collaborative surgical system of claim 54, wherein the controller determines that the motion pattern of the target surgical instrument is small motion when the tracked surgical instrument moves outside the predefined boundary area but remains within the field of view of the laparoscope for less than a predetermined period of time before returning within the predefined boundary area.

57. 46. ​​The collaborative surgical system of claim 45, wherein in the instrument centering mode, the controller causes the robotic arm to move the laparoscope by executing a trajectory generation algorithm to generate a trajectory from a current position of the laparoscope to a desired position of the laparoscope, and causes the robotic arm to move the laparoscope along the trajectory to maintain a target surgical instrument within the field of view of the laparoscope.

58. The controller In response to determining that a force applied to the robotic arm due to a force applied to the laparoscope exceeds a predetermined threshold, applying an impedance to the robotic arm in the collaborative manipulation mode while allowing the robotic arm to be freely movable to account for a weight of the laparoscope and the robotic arm; recording a trajectory of the freely moving robot arm when the movement of the robot arm deviates from the generated trajectory; updating the trajectory generation algorithm based on the recorded trajectories; 58. The cooperative surgical system of claim 57, configured to:

59. 58. The collaborative surgical system of claim 57, wherein the generated trajectory comprises moving the robotic arm along a longitudinal axis of the laparoscope to maintain the target surgical instrument within the field of view of the laparoscope and within a predetermined resolution threshold.

60. 58. The collaborative surgical system of claim 57, wherein the generated trajectory includes moving the robotic arm along at least one of a longitudinal axis of the laparoscope or an axis perpendicular to the longitudinal axis of the laparoscope to maintain the target surgical instrument within the field of view of the laparoscope.

61. The locus is measuring a current position of the distal end of the robotic arm; determining an entry point for the laparoscope into the patient; calculating a distance required to move the distal end of the robotic arm from its current position to a second position that moves the distal end of the laparoscope from its current position to the desired position based on the entry point and a known length between the distal end of the robotic arm and the distal end of the laparoscope; 58. The cooperatively operated surgical system of claim 57, wherein the system is generated by:

62. The controller Calculating the force required to move the distal end of the robotic arm a distance from its current position to the second position; applying torques to at least some of the joints of the robotic arm based on the calculated force for moving the distal end of the robotic arm the distance from its current position to the second position, thereby moving the distal end of the laparoscope from its current position to the desired position; 62. The collaborative surgical system of claim 61, wherein the robot arm is caused to move the laparoscope along the trajectory by

63. The controller Detecting an offset angle between a camera head of the laparoscope and the laparoscope; calibrating the trajectory to correct the offset angle so that movement of the laparoscope along the calibrated trajectory maintains the target surgical instrument within the field of view of the laparoscope; 58. The cooperative surgical system of claim 57, configured to:

64. The controller moving the robot arm along a predetermined trajectory in a known direction within a robot arm coordinate frame; measuring actual movement of a static object within the field of view of the laparoscope in response to movement of the robotic arm along the predetermined trajectory; comparing the actual movement of the static object with the expected movement of the static object associated with the predetermined trajectory; The cooperatively operated surgical system according to claim 63, wherein the system is configured to detect the offset angle by

65. 46. ​​The collaborative surgical system of claim 45, wherein the controller is configured to identify the target surgical instrument within the field of view of the laparoscope by running an object segmentation algorithm on image data from the laparoscope.

66. The collaborative surgical system of claim 45, wherein the controller is configured to cause the robotic arm to switch to the instrument centering mode in response to a user input.

67. 67. The collaborative surgical system of claim 66, further comprising an actuator operatively coupled to the controller and disposed on a link of the plurality of links of the robotic arm, the actuator configured to receive the user input and be actuated to transmit one or more signals indicative of the user input to the controller.

68. The controller determining an aspect of the laparoscopic surgery; estimating the target surgical instrument based on the aspect of the laparoscopic surgical procedure; identifying the target surgical instrument within the field of view of the laparoscope based on the estimation and the image data from the laparoscope; and 46. ​​The cooperative surgical system of claim 45, configured to:

69. The controller determining an aspect of the laparoscopic surgery; automatically switching to the instrument centering mode in response to the phase of the laparoscopic surgical procedure; 46. ​​The cooperative surgical system of claim 45, configured to:

70. The controller identifying one or more anatomical structures within the field of view of the laparoscope based on the image data from the laparoscope; determining the aspect of the laparoscopic surgical procedure based on the identified one or more anatomical structures; having the robotic arm move the laparoscope in the instrument centering mode to maintain the identified one or more anatomical structures within the field of view of the laparoscope; 70. The cooperatively operated surgical system of claim 69, configured to:

71. The controller generating an overlay indicative of the target surgical instrument; causing the overlay to be displayed over the image data from the laparoscope via a graphical user interface; 46. ​​The cooperative surgical system of claim 45, configured to:

72. The controller causing the robotic arm to move the laparoscope along a predetermined trajectory; comparing an actual trajectory of the image data from the laparoscope during movement along the predetermined trajectory with an expected trajectory of the image data associated with the predetermined trajectory to determine an angle of the distal tip of the laparoscope; 46. ​​The cooperative surgical system of claim 45, configured to:

73. The collaborative surgical system of claim 72, wherein the predetermined trajectory comprises a circular pattern in a single plane.

74. 46. ​​The collaborative surgical system of claim 45, wherein the controller is configured to identify the target surgical instrument within the field of view of the laparoscope based on the image data from the laparoscope using a machine learning algorithm executed in the controller.

75. 75. The collaborative surgical system of claim 74, wherein the machine learning algorithm is trained on a database of image data annotated with associated surgical instruments, and the machine learning algorithm is configured to evaluate pixels of the image data from the laparoscope, indicate whether the pixel corresponds to the target surgical instrument, and identify the target surgical instrument.

76. 46. ​​The collaborative surgical system of claim 45, wherein the controller is configured to identify the target surgical instrument within the field of view of the laparoscope in real time.

77. 46. ​​The collaborative surgical system of claim 45, wherein the controller is configured to cause the robotic arm to move the laparoscope to track the target surgical instrument manually held by a user in the instrument centering mode.

78. 78. The collaborative surgical system of claim 77, further comprising a second robotic arm configured to be removably coupled to the target surgical instrument manually held by the surgeon.

79. The controller determining a size of the tracked surgical instrument relative to the field of view of the laparoscope based on the image data; in the instrument centering mode, moving the laparoscope with the robotic arm at a speed or acceleration based on the size of the tracked surgical instrument relative to the field of view of the laparoscope; 46. ​​The cooperative surgical system of claim 45, configured to:

80. The controller determining a length of the laparoscope within a patient's body; moving the laparoscope with the robotic arm at a speed or acceleration based on the length of the laparoscope within the patient's body in the instrument centering mode; 46. ​​The cooperative surgical system of claim 45, configured to:

81. 81. The collaborative surgical system of claim 80, wherein the controller is configured to determine the length of the laparoscope within the patient's body based on a known length of the laparoscope and a position of the distal end of the robotic arm relative to a trocar positioned on the patient's body through which the laparoscope is inserted.

82. 46. ​​The collaborative surgical system of claim 45, wherein the controller is configured to identify the target surgical instrument within the field of view of the laparoscope by detecting a motion pattern of a trajectory of the target surgical instrument within the field of view of the laparoscope.

83. 1. A collaborative surgical system for assisting a surgical procedure performed using a surgical instrument, the collaborative surgical system comprising: a robotic arm comprising a plurality of links, a plurality of joints, a proximal end operably coupled to a base of the robotic arm, and a distal region having a distal end configured to be removably coupled to the surgical instrument; a platform coupled to the base of the robotic arm; Equipped with the platform includes a stage assembly configured to independently move the base of the robotic arm relative to the platform in at least two degrees of freedom; a stage assembly configured to move the base of the robot arm relative to the platform in a first degree of freedom; ...

84. 84. The collaborative surgical system of claim 83, wherein in the user-guided setup mode, the stage assembly is configured to move the base of the robotic arm in the first degree of freedom when the force applied to the distal region of the robotic arm in the first direction exceeds a predetermined force threshold.

85. 84. The collaborative surgical system of claim 83, wherein in the user-guided setup mode, the stage assembly is configured to stop movement of the base of the robotic arm in the first degree of freedom when the force applied to the distal region of the robotic arm in the first direction falls below a predetermined release threshold.

86. 84. The collaborative surgical system of claim 83, wherein in the user-guided setup mode, the stage assembly is configured to stop movement of the base of the robotic arm in the first degree of freedom upon application of an opposing force on the robotic arm in a second direction opposite the first direction.

87. 84. The collaborative surgical system of claim 83, wherein in the user-guided setup mode, application of a force at a distal region of the robotic arm in a second direction causes the stage assembly to move the base of the robotic arm in a second degree of freedom relative to the platform.

88. 84. The collaborative surgical system of claim 83, further comprising an actuator configured to be actuated to switch the system to the user-guided setup mode.

89. 89. The collaborative surgical system of claim 88, wherein the system remains in the user-guided setup mode only while the actuator is actuated.

90. 90. The cooperatively manipulated surgical system of claim 88, wherein the actuator is disposed on a collar rotatably coupled to a link of the plurality of links, and wherein actuation of the actuator enables rotation of the collar in a first direction to cause rotation of a distal link of the plurality of links adjacent to the set joint of the plurality of joints in a corresponding first direction relative to a proximal link of the plurality of links adjacent to the set joint, and enables rotation of the collar in a second direction to cause corresponding rotation of the distal link adjacent to the set joint in a second direction relative to the proximal link adjacent to the set joint.

91. 84. The collaborative surgical system of claim 83, further comprising a graphical user interface operatively coupled to the stage assembly, the graphical user interface configured to display an actuator, the actuator configured to cause the stage assembly to move the base of the robotic arm in at least one of the at least two degrees of freedom relative to the platform.

92. 92. The collaborative surgical system of claim 91, wherein the actuator comprises a slidable cursor configured to be moved relative to a neutral center point of a cursor pad, and wherein movement of the slidable cursor in a direction relative to the neutral center point in the cursor pad causes the stage assembly to move the base of the robotic arm in a corresponding direction relative to the platform.

93. 93. The collaborative surgical system of claim 92, wherein the stage assembly is configured to move the base of the robotic arm relative to the platform in the corresponding direction at a velocity correlated to a distance of the slidable cursor from the neutral center point.

94. 92. The collaborative surgical system of claim 91, wherein the graphical user interface is configured to display one or more indicators that indicate a configuration of the robotic arm relative to the platform in real time in response to actuation of the actuator.

95. 84. The collaborative surgical system of claim 83, wherein in a collaborative mode, the robotic arm is enabled to be freely movable in response to movement at the handle of the surgical instrument to perform laparoscopic surgery.

96. a plurality of motors disposed within the base, the plurality of motors operably coupled to at least some of the plurality of joints; a controller operably coupled to the plurality of motors; Furthermore, The controller measuring currents in the plurality of motors, the measured currents being indicative of forces applied to the distal region of the robotic arm; in the user-guided setup mode, causing the stage assembly to move the base of the robot arm in at least one of the at least two degrees of freedom based on the measured current; 84. The collaborative surgical system of claim 83, programmed to:

97. The robot arm further includes a controller operably coupled to a set joint of the plurality of joints, the controller comprising: determining whether one or more objects are within a predetermined proximity threshold of the robotic arm; automatically rotating a distal link of the plurality of links adjacent to the set joint relative to a proximal link of the plurality of links adjacent to the set joint to avoid collision with the one or more objects when the stage assembly moves the base of the robot arm relative to the platform in at least one of the at least two degrees of freedom in the user-guided set mode; 84. The collaborative surgical system of claim 83, programmed to:

98. one or more depth sensors configured to detect the one or more objects adjacent to the robotic arm and generate one or more signals indicative of the proximity of the one or more objects to the robotic arm; 98. The collaborative surgical system of claim 97, wherein the controller is configured to determine whether the one or more objects are within a predetermined proximity threshold of the robotic arm based on the one or more signals.

99. 99. The collaborative surgical system of claim 98, wherein the one or more depth sensors comprise one or more proximity sensors disposed within the base of the robotic arm, the one or more proximity sensors comprising at least one of an electromagnetic, capacitive, ultrasonic, or infrared proximity sensor.

100. 99. The collaborative surgical system of claim 98, wherein the one or more depth sensors comprise one or more depth cameras.

101. 99. The collaborative surgical system of claim 98, wherein the controller is configured to stop movement of the base of the robotic arm through the stage assembly when the one or more objects are within the predetermined proximity threshold.

102. 84. The collaborative surgical system of claim 83, wherein the collaborative surgical system is not remotely operated via user input received at a remote surgeon console.

103. 1. A cooperative surgical system for assisting in a surgical procedure performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween, the cooperative surgical system comprising: a robotic arm comprising a plurality of links and a plurality of joints comprising one or more motorized joints, a set joint, and one or more passive joints, a proximal end operably coupled to a base of the robotic arm, and a distal region having a distal end configured to be removably coupled to the surgical instrument; a plurality of motors operably coupled to the one or more motorized joints and the setting joint; an actuator operably coupled to the setting joint; Equipped with the actuator is configured to be actuated to cause rotation of a distal link of the plurality of links adjacent the setting joint relative to a proximal link of the plurality of links adjacent the setting joint from a first setting configuration to a second setting configuration in response to actuation of the actuator; When the actuator is in an unactuated state, the robotic arm is enabled to be freely movable in response to movement at the handle of the surgical instrument to perform a surgical operation via the one or more motorized joints and the one or more passive joints, while the distal link adjacent the setting joint and the proximal link adjacent the setting joint remain in the second setting configuration.

104. 104. The cooperatively manipulated surgical system of claim 103, wherein the actuator comprises a collar rotatably coupled to a link of the plurality of links, the collar configured to be rotated in a first direction relative to the link of the plurality of links to cause rotation of the distal link adjacent the set joint in a corresponding first direction relative to the proximal link adjacent the set joint, and to be rotated in a second direction relative to the link of the plurality of links to cause rotation of the distal link adjacent the set joint in a corresponding second direction relative to the proximal link adjacent the set joint.

105. 105. The cooperatively manipulated surgical system of claim 104, wherein the collar includes a set mode actuator configured to be actuated in response to rotation of the collar to enable rotation of the distal link adjacent the set joint relative to the proximal link adjacent the set joint in the corresponding first and second directions.

106. 106. The collaborative surgical system of claim 105, wherein the set mode actuator is configured to be actuated in a plurality of actuation patterns, each actuation pattern of the plurality of actuation patterns being associated with a unique user input configured to initiate a predetermined function of the collaborative surgical system.

107. 105. The cooperatively operated surgical system according to claim 104, wherein the collar is spring biased such that upon release of the collar at any position, the collar is configured to return to a neutral position relative to the link of the plurality of links.

108. further comprising a graphical user interface operably coupled to the configuration joint; The collaborative surgical system of claim 103, wherein the actuator is configured to be displayed on the graphical user interface.

109. 109. The collaborative surgical system of claim 108, wherein the actuator comprises a slidable cursor configured to be moved relative to a neutral center point, wherein movement of the slidable cursor in a first direction relative to the neutral center point causes rotation of the distal link adjacent the set joint in a first direction relative to the proximal link adjacent the set joint, and movement of the slidable cursor in a second direction relative to the neutral center point causes rotation of the distal link adjacent the set joint in a second direction relative to the proximal link adjacent the set joint.

110. 110. The collaborative surgical system of claim 109, wherein the distal link adjacent the set joint is configured to rotate in the corresponding direction relative to the proximal link adjacent the set joint at a speed that correlates with the distance of the slidable cursor from the neutral center point.

111. 109. The cooperatively manipulated surgical system of claim 108, wherein the graphical user interface is configured to display an indicator that indicates in real time a configuration of the distal link adjacent the set joint relative to the proximal link adjacent the set joint in response to actuation of the actuator.

112. 112. The collaborative surgical system of claim 111, wherein the graphical user interface is configured to display graphical representations of a plurality of configurations of the distal link adjacent the set joint relative to the proximal link adjacent the set joint such that a position of an indicator relative to the graphical representations of a plurality of configurations indicates in real time a configuration of the distal link adjacent the set joint relative to the proximal link adjacent the set joint in response to actuation of the actuator.

113. 104. The collaborative surgical system of claim 103, further comprising a controller operably coupled to the robotic arm, the controller being programmed to, during an operating phase, cause the robotic arm to be freely movable in response to movement at the handle of the surgical instrument to perform a laparoscopic surgical procedure.

114. 114. The cooperatively manipulated surgical system of claim 113, wherein the controller is configured to switch from the operating phase to the setting phase upon actuation of a set mode actuator, actuation of the actuator causing rotation of the distal link adjacent the set joint relative to the proximal link adjacent the set joint only when the set mode actuator is in an actuated state.

115. 104. The cooperatively manipulated surgical system of claim 103, wherein when the actuator is in an actuated state, application of a force at the distal region of the robotic arm in a first direction causes rotation of the distal link adjacent the set joint in the first direction relative to the proximal link adjacent the set joint, and application of a force at the distal region of the robotic arm in a second direction causes rotation of the distal link adjacent the set joint in the second direction relative to the proximal link adjacent the set joint.

116. 104. The cooperatively manipulated surgical system of claim 103, wherein when the actuator is in the unactuated state, the setting joint is configured such that the distal and proximal links adjacent the setting joint are fixed relative to one another in the second setting configuration.

117. 104. The collaborative surgical system of claim 103, wherein all of the motors of the plurality of motors operably coupled to the one or more motorized joints are located within the base of the robotic arm.

118. 104. The cooperatively manipulated surgical system of claim 103, wherein a shoulder link of the plurality of links comprises a distal shoulder link rotatably coupled to a proximal shoulder link via the setting joint, and wherein the motor of the plurality of motors operably coupled to the setting joint is not backdrivable.

119. 119. The cooperatively manipulated surgical system of claim 118, wherein the motor of the plurality of motors operably coupled to the set joint is located on the shoulder link adjacent the set joint.

120. 104. The collaborative surgical system of claim 103, further comprising a platform operably coupled to the base of the robotic arm, the platform comprising a stage assembly configured to independently move the base of the robotic arm horizontally and vertically relative to the platform.

121. 121. The collaborative surgical system of claim 120, wherein, in a user-guided setup mode, application of a force at the distal region of the robotic arm in a first direction causes the stage assembly to move the base of the robotic arm in the horizontal direction relative to the platform, and application of a force at the distal region of the robotic arm in a second direction causes the stage assembly to move the base of the robotic arm in the vertical direction relative to the platform.

122. a setup mode actuator configured to be actuated to switch the system to the user-guided setup mode; 122. The collaborative surgical system of claim 121, wherein the system remains in the user-guided setup mode only while the setup mode actuator is actuated.

123. the actuator includes a collar rotatably coupled to a link of the plurality of links; the setting mode actuator is disposed on the collar; 123. The cooperatively manipulated surgical system of claim 122, wherein actuation of the set mode actuator causes rotation of the distal link adjacent the set joint relative to the proximal link adjacent the set joint in a corresponding first direction by enabling rotation of the collar in a first direction, and causes rotation of the distal link adjacent the set joint relative to the proximal link adjacent the set joint in a corresponding second direction by enabling rotation of the collar in a second direction.

124. The collaborative surgical system of claim 103, wherein the collaborative surgical system is not remotely operated via user input received at a remote surgeon console.

125. 1. A collaborative surgical system for assisting a surgical procedure performed using a surgical instrument, the collaborative surgical system comprising: a robotic arm comprising a proximal region having a base, a plurality of links, an elbow joint, a shoulder joint coupled to the base, and a distal region having a distal end configured to be removably coupled to the surgical instrument; a platform coupled to the base of the robotic arm; Equipped with the platform is configured to be movable on a floor; a link of the plurality of links between the elbow joint and the shoulder joint configured to extend away from the base at a first angle and a link of the plurality of links distal to the elbow joint configured to extend away from the base at a second angle in a drape mode to be more aligned with the floor, thereby facilitating draping of the robotic arm.

126. 126. The collaborative surgical system of claim 125, wherein the first angle is selected such that in the draped mode, the link between the elbow joint and the shoulder joint is configured to be substantially vertical.

127. 126. The collaborative surgical system of claim 125, wherein the second angle is selected such that in the drape mode, the link distal to the elbow joint is configured to be substantially horizontal.

128. 126. The collaborative surgical system of claim 125, wherein the second angle is selected such that in the draped mode, the link distal to the elbow joint is configured to be substantially parallel to the floor.

129. 126. The collaborative surgical system of claim 125, wherein the platform comprises a plurality of wheels configured to allow mobility of the platform over the floor.

130. 126. The collaborative surgical system of claim 125, wherein the robotic arm is not teleoperated via user input received at a remote surgeon console.

131. 126. The collaborative surgical system of claim 125, further comprising a controller operably coupled to the robotic arm, the controller configured to automatically cause the link between the elbow joint and the shoulder joint to extend away from the base at the first angle and the link distal to the elbow joint to extend away from the base at the second angle when activated in the drape mode.

132. 132. The collaborative surgical system of claim 131, wherein, when activated in the drape mode, the controller automatically causes the robotic arm to rotate about the base so that the link between the elbow joint and the shoulder joint extends away from the platform at the first angle and the link distal to the elbow joint extends away from the platform at the second angle.

133. 132. The collaborative surgical system of claim 131, wherein the controller is further configured to enforce a virtual haptic boundary around the robotic arm, whereby when the robotic arm is within the virtual haptic boundary, the controller applies a high level of impedance to the robotic arm such that movement of the robotic arm is more tenacious within the virtual haptic boundary.

134. 132. The collaborative surgical system of claim 131, wherein the controller is further configured to apply a temporary local virtual haptic boundary to the distal region of the robotic arm in the drape mode, whereby movement of the robotic arm is more tenacious, thereby stabilizing the robotic arm in the drape mode.

135. a coupler configured to be removably coupled to the distal end of the robotic arm after application of a sterile drape over the robotic arm, the coupler configured to be removably coupled to the surgical instrument; 135. The collaborative surgical system of claim 134, wherein the controller is further configured to remove the temporary local virtual haptic boundary at the distal region of the robotic arm upon detecting that the coupler is removably coupled to the distal end of the robotic arm.

136. 132. The collaborative surgical system of claim 131, wherein the platform is configured to move the base of the robotic arm in at least two degrees of freedom relative to the floor, and wherein, when activated in the drape mode, the controller causes the platform to automatically move the base of the robotic arm in at least one of the at least two degrees of freedom relative to the floor to align the link distal to the elbow joint with a user's height and facilitate draping of the robotic arm by the user.

137. The controller storing data indicative of the height of the user in a user-specific profile; executing the user-specific profile in response to user input received at a user interface operably coupled to the controller; 137. The cooperatively operated surgical system of claim 136, further configured to:

138. an optical sensor operably coupled to the controller and configured to collect depth data; 137. The collaborative surgical system of claim 136, wherein the controller is configured to determine the height of the user based on depth data collected by the optical sensor.

139. 126. The collaborative surgical system of claim 125, further comprising a coupler configured to be removably coupled to the distal end of the robotic arm after application of a sterile drape over the robotic arm, the coupler configured to be removably coupled to the surgical instrument, thereby removably coupling the surgical instrument to the distal end of the robotic arm.

140. a second robotic arm having a second base coupled to the platform; 126. The collaborative surgical system of claim 125, wherein in the drape mode, the links of the second robotic arm are parallel to the corresponding links of the robotic arm.

141. 141. The collaborative surgical system of claim 140, further comprising a sterile drape having a first portion sized and shaped to cover at least the robotic arm and a second portion sized and shaped to cover at least the second robotic arm.

142. 142. The cooperative surgical system of claim 141, wherein the first and second portions of the sterile drape each include a sealing end portion configured to contact a distal end of the respective robotic arm such that the robotic arm is completely enclosed within the sterile drape.

143. 143. The collaborative surgical system of claim 142, wherein a proximal end of the sterile drape includes an opening sized and shaped to receive at least a portion of the platform.

144. 144. The cooperative surgical system of claim 143, wherein a proximal end of the sterile drape comprises an elastic band along the opening, the elastic band configured to facilitate wrapping of the sterile drape over at least a portion of the platform.

145. 142. The collaborative surgical system of claim 141, wherein the sterile drape comprises one or more straps configured to secure the first and second portions of the sterile drape to the respective robotic arms.

146. 142. The collaborative surgical system of claim 141, wherein the sterile drape comprises one or more rigid guides configured to be grasped by a user to guide the sterile drape over the robotic arm.

147. further comprising first and second couplers configured to be removably coupled to the distal end of the respective robotic arms after application of the sterile drape over the robotic arms, each of the first and second couplers configured to be removably coupled to a surgical instrument; The controller applying a temporary local virtual haptic boundary to the distal region of the robot arm in the drape mode such that movement of the robot arm is more tenacious in the drape mode, thereby stabilizing the robot arm; removing the temporary local virtual haptic boundary at the distal region of the robot arm upon detecting that both the first and second couplers are removably coupled to the distal end of the robot arm; and 142. The cooperatively operated surgical system of claim 141, further configured to:

148. 1. A collaborative surgical system for assisting a surgical procedure performed using a surgical instrument, the collaborative surgical system comprising: a plurality of robotic arms, each robotic arm comprising a plurality of links, a plurality of joints, a proximal region operably coupled to a base, and a distal region configured to be removably coupled to the surgical instrument; a platform coupled to a base of each robotic arm, wherein in a surgical mode, each of the robotic arms is configured to extend away from the platform toward a surgical site; Controller and Equipped with the controller is configured, when operating in a storage mode, to transition each of the robotic arms to a retracted, stored configuration via a plurality of links and joints of each of the robotic arms, and to rotate each of the robotic arms about a base of each of the robotic arms such that each of the robotic arms is within a footprint of the platform in the retracted, stored configuration.

149. 149. The collaborative surgical system of claim 148, wherein, when activated in the retraction mode, the controller causes a first robotic arm of the plurality of robotic arms to rotate about its base in a first direction and a second robotic arm of the plurality of robotic arms to rotate about its base in a second direction, the second direction being opposite the first direction.

150. 149. The collaborative surgical system of claim 148, wherein, when activated in the retraction mode, the controller causes a first robotic arm of the plurality of robotic arms to rotate about its base in a first direction and a second robotic arm of the plurality of robotic arms to rotate about its base in a second direction, the second direction being the same as the first direction.

151. 149. The collaborative surgical system of claim 148, wherein in the retracted stored configuration, a first robotic arm of the plurality of robotic arms has the same configuration and orientation relative to the platform as a second robotic arm of the plurality of robotic arms.

152. 149. The collaborative surgical system of claim 148, wherein in the retracted stored configuration, each of the robotic arms extends in a direction toward a rear of the platform.

153. 149. The collaborative surgical system of claim 148, wherein, during operation of the retraction mode, the controller moves the at least one robotic arm of the plurality of robotic arms to a position via a plurality of links and joints of each of the at least one robotic arm such that a collision between the at least one robotic arm and the platform is avoided upon rotation of the at least one robotic arm about its respective base.

154. 149. The collaborative surgical system of claim 148, wherein the platform includes a stage assembly configured to move a base of each robotic arm independently in at least two degrees of freedom relative to the platform.

155. 155. The collaborative surgical system of claim 154, wherein, during operation in the storage mode, the controller causes the stage assembly to move the base of each robotic arm in an inward direction along a first degree of freedom of the at least two degrees of freedom such that, in the retracted stored configuration, each of the robotic arms rotates about its respective base while each of the robotic arms is within the footprint of the platform.

156. 156. The collaborative surgical system of claim 155, wherein, when activated in the storage mode, the controller causes the stage assembly to move the base of each robotic arm downwardly along a second of the at least two degrees of freedom so that the robotic arms have a minimum height relative to the platform for transport and / or storage of the collaborative surgical system in the retracted storage configuration.

157. 155. The collaborative surgical system of claim 154, wherein, during operation in the retracted mode, the controller causes the stage assembly to move a base of the at least one robotic arm outwardly along a first of the at least two degrees of freedom and / or upwardly along a second of the at least two degrees of freedom so as to avoid collision between the at least one robotic arm and the platform upon rotation of at least one robotic arm of the plurality of robotic arms about its respective base.

158. 155. The collaborative surgical system of claim 154, wherein, during operation in the storage mode, the controller causes the stage assembly to rotate the base of each robotic arm relative to the platform along a horizontal plane toward each other such that, upon rotation of each of the robotic arms about its respective base, each of the robotic arms is within the footprint of the platform in the retracted stored configuration.

159. 149. The collaborative surgical system of claim 148, further comprising a graphical user interface operably coupled to the controller, the graphical user interface configured to display a retraction mode actuator configured to be actuated by a user to activate the retraction mode.

160. 149. The collaborative surgical system of claim 148, wherein in the retracted stored configuration, each of the robotic arms is entirely within the footprint of the platform.

161. 149. The collaborative surgical system of claim 148, wherein, during operation in the compact mode, the controller causes each of the robotic arms to extend away from the platform and transition to a semi-retracted configuration via its respective plurality of links and joints and rotate about its respective base to facilitate transport of the collaborative surgical system, the semi-retracted configuration being less retracted than the retracted storage configuration of the storage mode.

162. 149. The collaborative surgical system of claim 148, wherein, during actuation in a drape mode, the controller causes each of the robotic arms to transition to an extended configuration via its respective plurality of links and joints and rotate about its respective base so that each of the robotic arms extends away from the platform and facilitates draping of the robotic arms.

163. 163. The collaborative surgical system of claim 162, wherein in the extended configuration, a distal link of the plurality of links of each robotic arm is substantially parallel to the ground.

164. 1. A cooperative surgical system for assisting in a surgical procedure performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween, the cooperative surgical system comprising: a robotic arm having a proximal end, a distal end configured to be removably coupled to the surgical instrument, a plurality of links, and a plurality of joints; a controller operably coupled to the robotic arm; Equipped with the controller is configured to enable the robotic arm to be freely movable in response to movement at the handle of the surgical instrument to perform the surgical procedure using the surgical instrument; The controller identifying the type of surgical instrument coupled to the distal end of the robotic arm; applying a first impedance to the robotic arm to account for a weight of the surgical instrument and the robotic arm; applying a second impedance to the robotic arm based on the type of surgical instrument to adjust stiffness at the distal end of the robotic arm to thereby guide movement of the surgical instrument by the user during a predetermined phase of the surgical procedure; A collaborative surgical system programmed to:

165. 165. The system of claim 164, wherein the identified type of surgical instrument comprises a suturing device, the predetermined phase of the surgical procedure comprises a suturing phase, and the second impedance is sufficient to provide greater tenacity control of the suturing device during the suturing phase of the surgical procedure.

166. 165. The system of claim 164, wherein the identified type of surgical instrument comprises a stapling device, the predetermined phase of the surgical procedure comprises a stapling phase, and the second impedance is sufficient to provide firm grounding and facilitate application of force by the stapling device during the stapling phase of the surgical procedure.

167. 165. The system of claim 164, wherein the controller is configured to identify predetermined aspects of the surgical procedure based on the type of the surgical instrument.

168. 165. The system of claim 164, wherein the type of surgical instrument is selected from a list comprising at least one of a wrist instrument, a stapling device, a dissection device, a suturing device, a retraction device, a tissue removal device, or a clip applier device.

169. 165. The system of claim 164, wherein the controller is configured to apply the second impedance to the robotic arm and adjust stiffness at the distal end of the robotic arm based on the type of surgical instrument, thereby guiding movement of the surgical instrument by the user during the predetermined phase of the surgical procedure without actively causing movement of the robotic arm.

170. 1. A computer-implemented system for providing image registration to a robotic arm, the robotic arm comprising a plurality of links, a plurality of joints, and a distal end configured to be removably coupled to a laparoscope having a rotatable camera sensor module, the system comprising at least one processor, the at least one processor comprising: reading a plurality of images from the laparoscope while moving the field of view of the laparoscope; calculating, via computer vision techniques, motion of individual pixels between successive images of the plurality of images, the motion of individual pixels indicating image motion; calculating an average of the motion of individual pixels in the x and y directions of the plurality of images to obtain an image motion direction; calculating an angular offset between the camera sensor module and the distal end of the robot arm based on the image motion direction; 1. A computer-implemented system configured to:

171. The at least one processor synchronizing the image movement with movement of the distal end of the robotic arm associated with movement of the field of view of the laparoscope; comparing the image motion direction to the movement of the distal end of the robot arm to calculate the angular offset between the camera sensor module and the distal end of the robot arm; 171. The system of claim 170, configured to:

172. The at least one processor In a foreground mode, the robotic arm is configured to move the laparoscope along a predetermined trajectory; 172. The system of claim 171, wherein the image movement is synchronized with movement of the distal end of the robotic arm associated with movement of the laparoscope along the predetermined trajectory.

173. 172. The system of claim 171, wherein in background mode, the image movement is synchronized with movement of the distal end of the robotic arm in response to movement of the field of view of the laparoscope by a user.

174. 172. The system of claim 171, wherein the at least one processor is configured to retrieve data indicative of the movement of the distal end of the robotic arm via one or more sensors operably coupled to at least some of the plurality of joints of the robotic arm.

175. 171. The system of claim 170, wherein the at least one processor is configured to verify the image motion direction.

176. The at least one processor calculating a norm of the vector in the direction of the image motion to determine the magnitude of the image motion; comparing the magnitude of the image motion to a magnitude threshold; and configured to:

176. The system of claim 175, wherein the image motion direction is verified if the magnitude of the image motion exceeds the magnitude threshold.

177. The at least one processor calculating a percentage of image pixels that have moved between successive images based on the movement of individual pixels; comparing said percentage to a percentage threshold; configured to:

176. The system of claim 175, wherein the image motion direction is verified if the percentage exceeds the percentage threshold.

178. 178. The system of claim 177, wherein the at least one processor is configured to determine whether the image motion is due to at least one of movement of the field of view of the laparoscope or local movement of one or more tools or tissues within the plurality of images based on the comparison of the percentage with the percentage threshold.

179. The at least one processor calculating a relative angle between each of the individual pixel movements and the image motion direction, and determining whether each of the individual pixel movements is consistent with the image motion direction; comparing the percentage of individual pixel motions that match the image motion direction to a match threshold; configured to:

176. The system of claim 175, wherein the image motion direction is verified if the percentage exceeds the match threshold.

180. The at least one processor causing the robotic arm to move the laparoscope along a predetermined axial trajectory; comparing the image motion direction to a direction threshold; determining whether the laparoscope has a flat tip or an angled tip based on the comparison of the image motion direction with the directional threshold; 171. The system of claim 170, configured to:

181. The movement of the field of view of the laparoscope is due to a zoom of the camera sensor module, and the at least one processor: comparing the image motion direction to a direction threshold; determining whether the laparoscope has a flat tip or an angled tip based on the comparison of the image motion direction with the directional threshold; 171. The system of claim 170, configured to:

182. 171. The system of claim 170, wherein the controller is configured to automatically switch the robotic arm to a collaborative manipulation mode in response to determining that a force applied to the robotic arm due to a force applied to the laparoscope exceeds a predetermined threshold, and wherein the controller is configured to apply an impedance to the robotic arm in the collaborative manipulation mode to account for the weight of the laparoscope and the robotic arm while allowing the robotic arm to be freely movable in response to movements in the laparoscope.

183. 1. A system for robotic surgery, the system comprising: a robotic arm comprising a proximal end operably coupled to the base of the robotic arm, a distal end, a plurality of links, and a plurality of joints between the proximal end and the distal end, the robotic arm being configured to be positioned adjacent to a bed for supporting a patient during a surgical procedure; a platform coupled to the base of the robotic arm, the platform including a stage assembly configured to independently move the base of the robotic arm relative to the platform in at least two degrees of freedom; a graphical user interface having a plurality of predetermined selectable pre-configured configurations; a controller operably coupled to the robotic arm; Equipped with The controller during a surgical setup phase, upon selection of a first preset configuration of the plurality of predetermined selectable preset configurations, automatically positioning the robotic arm in a first configuration associated with the first preset configuration; during the surgical setup phase, upon selection of a second preset configuration from the plurality of predetermined selectable preset configurations, automatically positioning the robotic arm in a second configuration associated with the second preset configuration; configured to: A system for robotic surgery, wherein the first configuration associated with the first preset configuration is different from the second configuration associated with the second preset configuration.

184. 184. The system of claim 183, wherein the first preset configuration or the second preset configuration comprises a storage mode, and wherein, during the surgical setup phase, upon selection of the storage mode, the controller automatically positions the robotic arm via the plurality of links and joints in a retracted storage configuration and rotates the robotic arm about the base such that, in the retracted storage configuration, the robotic arm is within a footprint of the platform.

185. 185. The system of claim 184, wherein the first preset configuration or the second preset configuration comprises a compact mode, and wherein during the surgical setup phase, upon selection of the compact mode, the controller automatically positions the robotic arm via the plurality of links and joints in a semi-retracted configuration such that the robotic arm extends away from the platform to facilitate transport of the system, the semi-retracted configuration being less retracted than the retracted storage configuration of the storage mode.

186. 184. The system of claim 183, wherein the first preset configuration or the second preset configuration comprises a drape mode, and during the surgical setup phase, upon selection of the drape mode, the controller automatically positions the robotic arm in an extended configuration via the plurality of links and joints such that the robotic arm extends away from the platform and facilitates draping of the robotic arm.

187. 187. The system of claim 186, wherein in the extended configuration, a distal link of the plurality of links of the robotic arm is substantially parallel to the ground.

188. 187. The system of claim 186, wherein in the drape mode, the controller is configured to apply a temporary local virtual haptic boundary to the distal region of the robotic arm, such that movement of the robotic arm is more tenacious in the drape mode, thereby stabilizing the robotic arm.

189. 184. The system of claim 183, wherein the first preset configuration or the second preset configuration comprises a custom preset configuration, the custom preset configuration being associated with a custom robotic arm configuration pre-stored in a memory of the controller, and wherein during the surgical setup phase, upon selection of the custom preset configuration, the controller automatically positions the robotic arm in the custom robotic arm configuration.

190. 190. The system of claim 189, wherein the custom robotic arm configuration associated with the custom preset configuration is configured to be stored in a user-specific profile, and wherein the controller is configured to display the custom preset configuration on the graphical user interface upon selection of the user-specific profile.

191. 190. The system of claim 189, wherein the custom robotic arm configuration is selected to facilitate a predetermined surgical procedure.

192. the predetermined surgical procedure is a cholecystectomy, and the controller, upon selection of the custom preset configuration, rotating a shoulder link of the plurality of links of the robot arm in a leftward direction relative to the platform; causing the stage assembly to move the base of the robot arm in a downward direction in a first of the at least two degrees of freedom and in an outward direction in a second of the at least two degrees of freedom; 192. The system of claim 191, configured to:

193. 193. The system of claim 192, further comprising a second robotic arm, wherein the controller is configured to cause a stage assembly of the second robotic arm to move a base of the second robotic arm in an upward direction in a first degree of freedom of the at least two degrees of freedom and in an inward direction in a second degree of freedom of the at least two degrees of freedom when the custom preset configuration is selected.

194. 192. The system of claim 191, wherein the predetermined surgical procedure comprises a cholecystectomy, sleeve gastrectomy, hiatal hernia repair, Nissen-Fandomization, inguinal hernia repair (TEP), right, left, and / or total colectomy, gastric bypass, sigmoid colectomy, umbilical hernia repair, or incisional hernia repair.

Images