Co-operating surgical system with coupling mechanism removably attached to surgical instrument

Abstract

A co-maneuvering robotic system that can be used to assist laparoscopic surgical procedures is described herein. The co-manipulated robotic system allows a surgeon to use commercially available surgical tools while providing benefits associated with surgical robots. Advantageously, the surgical tool can be seamlessly coupled to the robotic arm using a disposable coupler while the reusable portion of the robotic arm remains in the sterile drape. Further, the co-manipulated robotic system may operate in multiple modes to enhance availability and safety while allowing a surgeon to position the instrument directly with the instrument handle and further maintain the desired position of the instrument using the robotic arm.

Description

[0001] Cross - reference to related applications

[0002] This application claims priority to U.S. Patent Application No. 18 / 318,699, filed on May 16, 2023; U.S. Patent Application No. 18 / 057,191, filed on November 18, 2022; U.S. Provisional Patent Application No. 63 / 378,434, filed on October 5, 2022; and European Patent Application No. 22306496.5, filed on October 5, 2022. The entire contents of each of the above - mentioned patent applications are incorporated herein by reference. This application also relates to U.S. Patent Application No. 17 / 815,885, filed on July 28, 2022 (now U.S. Patent No. 11,504,197), which is a continuation application of PCT Patent Application No. PCT / IB2022 / 052989, filed on March 30, 2022, and claims priority to European Patent Application No. 21306904.0, filed on December 22, 2021; European Patent Application No. 21306905.7, filed on December 22, 2021; European Patent Application No. 21305929.8, filed on July 5, 2021; and European Patent Application No. 21305417.4, filed on March 31, 2021. The entire contents of each of the above - mentioned patent applications are incorporated herein by reference. This application also relates to U.S. Patent Application No. 18 / 480,360, filed on October 3, 2023; U.S. Patent Application No. 18 / 331,060, filed on June 7, 2023; and U.S. Patent Application No. 17 / 816,958, filed on August 2, 2022 (now U.S. Patent No. 11,622,826). The entire contents of each of the above - mentioned patent applications are incorporated herein by reference. Technical field

[0003] The present disclosure relates to co - operating robotic systems, such as co - operating robotic systems having a coupling mechanism for removably attaching a surgical instrument. Background art

[0004] Managing vision and access during laparoscopic procedures is a challenge. The surgical assistant paradigm is inherently imperfect because the assistant is required to anticipate and observe from the surgeon's perspective without standing where the surgeon stands, and similarly, anticipate and adjust how the surgeon wants to expose the tissue of interest throughout the procedure. For example, during a laparoscopic procedure, one assistant may be required to hold a retractor device to expose tissue for the surgeon, while another assistant may be required to hold a laparoscope device to provide the surgeon with a field of view of the surgical scene inside the patient during the procedure, and any one of them may be required to hold the corresponding tool in an impractical position (e.g., between the surgeon's arms while the surgeon is actively manipulating another surgical instrument).

[0005] Various attempts have been made to address this problem. For example, rail-mounted orthopedic retractors (which are purely mechanical devices mounted to the patient bed / table) can be used to hold the laparoscope device in place during a laparoscopic procedure, and another rail-mounted orthopedic retractor can be used to hold the retractor device in place during a laparoscopic procedure. However, rail-mounted orthopedic retractors require a significant amount of manual interaction to unlock, reposition, and lock the tools in place.

[0006] Surgeons have used complex robotic-assisted systems such as the Da Vinci Surgical System (available from Intuitive Surgical, Inc. of Sunnyvale, California) to enhance the procedure by allowing the surgeon to perform laparoscopic surgery remotely operatively from a surgeon's console that is separate from the patient console that holds the surgical instruments. Such complex robotic-assisted systems are very expensive and have a very large footprint and take up a significant amount of space in the operating room. In addition, such robotic-assisted systems typically require unique system-specific surgical instruments that are compatible with the system, so surgeons may not be able to use the standard off-the-shelf surgical instruments they are accustomed to. As a result, surgeons need to learn an entirely different way of performing laparoscopic procedures.

[0007] In view of the foregoing disadvantages of the previously known systems and methods, there is a need for a system that provides a surgeon with the ability to seamlessly position and manipulate various surgical instruments as needed, thereby avoiding the workflow limitations inherent in both manual and mechanical solutions. SUMMARY OF THE INVENTION

[0008] The present disclosure overcomes the disadvantages of previously known systems and methods by providing a co-manipulating surgical system for assisting laparoscopic surgery, which is performed using a surgical instrument having a handle, an operating end, and an elongate shaft therebetween. The co-manipulating surgical system may include a robotic arm that includes a proximal end, a distal end configured for detachably coupling to the surgical instrument, a plurality of linkages, and a plurality of joints between the proximal end and the distal end, and the distal end of the robotic arm includes a coupler interface. The co-manipulating surgical system may further include a coupler body configured for detachably coupling to the coupler interface. The coupler body includes a lumen sized and shaped to receive the elongate shaft of the surgical instrument therein, and the coupler body may be configured to transition between an open state and a closed state, in which the elongate shaft is slidably movable within the lumen in the open state and longitudinal movement of the elongate shaft relative to the coupler body is inhibited in the closed state while allowing rotational movement of the elongate shaft relative to the coupler body in response to movement at the handle of the surgical instrument. When the coupler body is coupled to the coupler interface, the coupler body may be configured to rotate relative to the distal end of the robotic arm through the coupler interface to self-align the lumen with the elongate shaft when the elongate shaft is inserted into the lumen. Additionally, when the coupler body is coupled to the coupler interface in the closed state, the robotic arm may be allowed to move freely in response to movement at the handle of the surgical instrument to perform laparoscopic surgery. The coupler body is disposable after a single laparoscopic surgery.

[0009] The co-manipulating surgical system may further include a switch configured to transition between an unlocked position and a locked position. The switch may include an engagement portion configured to engage the elongate shaft when the elongate shaft is placed within the lumen and the switch is in the locked position, thereby fixing the elongate shaft within the lumen. Thus, when the coupler body is coupled to the coupler interface and the elongate shaft is placed within the lumen, the robotic arm may be configured to move freely in response to movement at the handle of the surgical instrument. Additionally, when the elongate shaft is placed within the lumen and the switch is in the locked position, the engagement portion may apply a frictional force to the elongate shaft that is configured to allow rotational movement of the elongate shaft relative to the coupler body while inhibiting longitudinal movement of the elongate shaft relative to the coupler body. The switch may include a handle portion configured to be actuated to transition the switch between the unlocked position and the locked position.

[0010] In addition, the coupler body may include an inclined surface having a first valley configured to engage the switch in the unlocked position, a second valley configured to engage the switch in the locked position, and a peak located between the first valley and the second valley. The peak may be configured to allow the switch to transition between the unlocked and locked positions when a force applied to the switch exceeds a predetermined force threshold. The coupler body may further include a bracket slidably disposed within the coupler body, the bracket including a contact surface configured to define at least a portion of the inner cavity. Additionally, the bracket may be biased in a direction toward the inner cavity such that when the elongate shaft is placed within the inner cavity, the contact surface engages the elongate shaft. In some embodiments, the coupler interface may include a repulsive magnet, and the bracket may include a magnet such that the repulsive magnet applies a magnetic force to the magnet, thereby biasing the bracket in a direction toward the inner cavity. Further, the bracket may include a harness configured to couple to the magnet, the harness sized and shaped to be slidably disposed within a channel of the coupler body. The co-manipulation surgical system may further include a clamp pivotally coupled to the coupler body by a rod. The clamp may be configured to transition between an unlocked state that allows the inner cavity to receive the elongate shaft and a locked state in which the clamp secures the elongate shaft within the inner cavity. Additionally, the bracket may include one or more supports coupled to the contact surface. Each of the one or more supports may have a channel sized and shaped to slidably receive a rod passing therethrough such that the bracket may be slidably disposed within the coupler body along the rod.

[0011] In addition, the coupler interface may include a protrusion, and the coupler body may include a groove configured to receive the protrusion of the coupler interface. Further, the protrusion may include one or more indentations, and the coupler body may include one or more locking arms configured to transition between a locked configuration in which at least a portion of the one or more locking arms extends within the groove of the coupler body and an unlocked configuration in which the one or more locking arms do not extend within the groove of the coupler body. Thus, when the one or more locking arms are in the unlocked configuration, the protrusion of the coupler interface may be received by the groove of the coupler body, and when the protrusion is placed within the groove and the locking arms are in the locked configuration, at least a portion of the one or more locking arms may extend within one or more indentations of the protrusion, thereby securing the coupler body to the coupler interface. The one or more locking arms may be biased toward the locked configuration. Additionally, each of the one or more locking arms may have a handle portion configured to be actuated to transition the one or more locking arms from the locked configuration to the unlocked configuration.

[0012] The protrusion of the coupler interface may have a first geometry, while the groove of the coupler body may have a second geometry corresponding to the first geometry, such that when the protrusion is received by the groove, rotational movement between the coupler body and the coupler interface is prohibited. Additionally, the coupler interface may include one or more additional protrusions having the first geometry, and the coupler body may include one or more additional grooves having the second geometry, such that when the one or more additional protrusions are received by the one or more additional grooves, rotational movement between the coupler body and the coupler interface is prohibited. The coupler body and the coupler interface may be configured to receive a sterile drape therebetween, such that the sterile drape prevents contact between the surgical instrument and the robotic arm during laparoscopic surgery.

[0013] Furthermore, the coupler body may include one or more tapered surfaces configured to guide the elongate shaft into the lumen and, as the elongate shaft is inserted into the lumen along the one or more tapered surfaces, rotate the coupler body relative to the distal end of the robotic arm through the coupler interface, thereby facilitating self-alignment of the lumen with the elongate shaft. The coupler body may further include a clamp configured to transition between an unlocked state allowing the lumen to receive the elongate shaft and a locked state in which the clamp secures the elongate shaft within the lumen. The clamp may be biased toward the locked state. Additionally, at least a portion of the clamp may include a tapered surface configured to guide the elongate shaft into the lumen and facilitate transition of the clamp from the locked state to the unlocked state in response to a force applied by the elongate shaft to the tapered surface as the elongate shaft is inserted into the lumen.

[0014] According to another aspect of the present disclosure, a method of using a robotic arm configured to be removably coupled to a surgical instrument having a handle, an operating end, and an elongate shaft therebetween, e.g., for assisting in laparoscopic surgery, is provided. The method may include: at a distal end of the robotic arm, removably coupling a coupler body to a coupler interface; inserting the elongate shaft of the surgical instrument into a lumen of the coupler body; transitioning the coupler body from an open state, in which the elongate shaft is slidably movable within the lumen, to a closed state, in which longitudinal movement of the elongate shaft relative to the coupler body is inhibited while rotational movement of the elongate shaft relative to the coupler body is permitted in response to movement at the handle of the surgical instrument; and, when the coupler body is coupled to the coupler interface in the closed state, freely moving the robotic arm by moving the handle of the surgical instrument, e.g., to perform laparoscopic surgery. The coupler body may rotate relative to the distal end of the robotic arm through the coupler interface to facilitate self-alignment of the lumen with the elongate shaft as the elongate shaft is inserted into the lumen.

[0015] For example, detachably coupling the coupler body to the coupler interface may include: actuating one or more locking arms of the coupler body to transition from a locked configuration in which at least a portion of the one or more locking arms extends within a recess of the coupler body to an unlocked configuration in which the one or more locking arms do not extend within the recess; inserting a protrusion of the coupler interface into the recess of the coupler body; and releasing the one or more locking arms to transition from the unlocked configuration to the locked configuration such that at least a portion of the one or more locking arms extends within one or more indentations of the protrusion, thereby securing the coupler body to the coupler interface. Additionally, inserting the elongate shaft of a surgical instrument into the lumen of the coupler body may include guiding the elongate shaft along one or more tapered surfaces of the coupler body into the lumen. For example, guiding the elongate shaft along one or more tapered surfaces of the coupler body into the lumen as the elongate shaft is inserted along the one or more tapered surfaces may cause the coupler body to rotate relative to the distal end of the robotic arm through the coupler interface to self-align the lumen with the elongate shaft.

[0016] Additionally, inserting the elongate shaft of a surgical instrument into the lumen of the coupler body may include: actuating a clamp of the coupler body to transition the clamp from a locked state to an unlocked state that permits the lumen to receive the elongate shaft; inserting the elongate shaft of the surgical instrument into the lumen; and releasing the clamp to transition the clamp from the unlocked state to the locked state such that the clamp secures the elongate shaft within the lumen. Further, transitioning the coupler body from an open state to a closed state may include transitioning a switch of the coupler body from an unlocked position, in which the elongate shaft is slidably movable within the lumen, to a locked position, in which an engaging portion of the switch engages the elongate shaft disposed within the lumen, thereby inhibiting longitudinal movement of the elongate shaft relative to the coupler body while permitting rotational movement of the elongate shaft relative to the coupler body. Thus, when the elongate shaft is disposed within the lumen and the switch is in the locked position, the engaging portion may apply a frictional force to the elongate shaft that is configured to permit rotational movement of the elongate shaft relative to the coupler body while inhibiting longitudinal movement of the elongate shaft relative to the coupler body. The method may further include placing a sterile drape between the coupler body and the coupler interface prior to detachably coupling the coupler body to the coupler interface.

[0017] According to another aspect of the present disclosure, a coupler device is provided for detachably coupling a surgical instrument having a handle and an elongate shaft to a distal end of a robotic arm of a robotic surgical system for assisting in laparoscopic surgery using the surgical instrument. The distal end of the robotic arm includes a coupler interface configured to detachably couple to the coupler device. The coupler device may include a lumen sized and shaped to receive the elongate shaft of the surgical instrument therein, and a coupler body configured to be convertible between an open state and a closed state. In the open state, the elongate shaft is slidably movable within the lumen, and in the closed state, longitudinal movement of the elongate shaft relative to the coupler body is inhibited while allowing rotational movement of the elongate shaft relative to the coupler body in response to movement at the handle of the surgical instrument. Thus, when the coupler body is coupled to the coupler interface, the coupler body may be configured to rotate relative to the distal end of the robotic arm through the coupler interface to self-align the lumen with the elongate shaft as the elongate shaft is inserted into the lumen.

[0018] The coupler device may further include a switch configured to be convertible between an unlocked position and a locked position. The switch may include an engagement portion configured to apply a frictional force to the elongate shaft when the elongate shaft is placed within the lumen and the switch is in the locked position, thereby allowing rotational movement of the elongate shaft within the lumen while prohibiting translational movement of the elongate shaft relative to the coupler body. Additionally, the coupler device may further include an inclined surface having a first valley configured to engage the switch in the unlocked position, a second valley configured to engage the switch in the locked position, and a peak disposed between the first valley and the second valley. The peak may be configured to allow the switch to be convertible between the unlocked position and the locked position when a force applied to the switch exceeds a predetermined force threshold. Further, the switch may have a handle configured to be actuated to convert the switch between the unlocked position and the locked position. When the coupler body is coupled to the coupler interface and the elongate shaft is placed within the lumen, the robotic arm may be configured to be freely movable in response to movement at the handle of the surgical instrument.

[0019] In addition, the coupler body may include one or more tapered surfaces configured to guide the elongate shaft into the lumen and, as the elongate shaft is inserted into the lumen along the one or more tapered surfaces, facilitate self-alignment of the lumen and the elongate shaft by rotating the coupler body relative to the distal end of the robotic arm via the coupler interface. The coupler device may further include a clamp configured to transition between an unlocked state allowing the lumen to receive the elongate shaft and a locked state in which the clamp secures the elongate shaft within the lumen. The clamp may be biased toward the locked state. Additionally, at least a portion of the clamp may include a tapered surface configured to guide the elongate shaft into the lumen and, as the elongate shaft is inserted into the lumen, facilitate transition of the clamp from the locked state to the unlocked state in response to a force applied by the elongate shaft to the tapered surface. Further, the clamp may have a handle portion configured to be actuated to transition the clamp from the locked state to the unlocked state.

[0020] In addition, the coupler device may include a bracket slidably disposed within the coupler body, the bracket including a friction pad configured to define at least a portion of the lumen. The bracket may be biased in a direction toward the lumen such that the friction pad engages the elongate shaft when the elongate shaft is disposed within the lumen. In some embodiments, the coupler interface may include a repulsive magnet and the bracket may include a magnet such that the repulsive magnet applies a magnetic force to the magnet to bias the bracket in a direction toward the lumen. The bracket may include a harness configured to couple to the magnet, the harness sized and shaped to be slidably disposed within a channel of the coupler body. Additionally, the coupler device may include a clamp pivotally coupled to the coupler body by a rod, the clamp configured to transition between an unlocked state allowing the lumen to receive the elongate shaft and a locked state in which the clamp secures the elongate shaft within the lumen. Further, the bracket may include one or more pedestals coupled to the friction pad, each of the one or more pedestals including a channel sized and shaped to slidably receive the rod therethrough such that the bracket is configured to be slidably disposed within the coupler body along the rod.

[0021] In addition, the coupler body may include a recess configured to receive a protrusion of the coupler interface. The coupler device may further include one or more locking arms configured to transition between a locked configuration in which at least a portion of the one or more locking arms extends within the recess of the coupler body and an unlocked configuration in which the one or more locking arms do not extend within the recess, such that when the one or more locking arms are in the unlocked configuration, the recess is allowed to receive the protrusion of the coupler interface. Further, when the protrusion is placed within the recess and the locking arms are in the locked configuration, at least a portion of the one or more locking arms may extend within one or more indentations of the protrusion of the coupler interface, thereby fixing the coupler body to the coupler interface. The one or more locking arms may be biased toward the locked configuration. Each of the one or more locking arms may include a handle portion configured to be actuated to transition the one or more locking arms from the locked configuration to the unlocked configuration. In addition, the protrusion of the coupler interface may have a first geometry, and the recess of the coupler body may have a second geometry corresponding to the first geometry such that rotational movement between the coupler body and the coupler interface is prohibited when the protrusion is received by the recess. The coupler body may be configured to receive a sterile drape between the coupler body and the coupler interface when the coupler body is coupled to the coupler interface.

[0022] According to another aspect of the present disclosure, another co-manipulation surgical system is provided to assist in laparoscopic surgery performed using a surgical instrument having a handle, an operating end, and an elongate shaft therebetween. The system may include a robotic arm including a proximal end, a distal end configured to be detachably coupled to the surgical instrument, a plurality of linkages, and a plurality of joints between the proximal and distal ends, the distal end of the robotic arm including a coupler interface, and a coupler body configured to be detachably coupled to the coupler interface. The coupler body may include a lumen sized and shaped to receive the elongate shaft of the surgical instrument therein and may be configured to transition between an open state in which the elongate shaft is slidably movable within the lumen and a closed state in which longitudinal movement of the elongate shaft relative to the coupler body is inhibited while allowing rotational movement of the elongate shaft relative to the coupler body in response to movement at the handle of the surgical instrument. Further, when the coupler body is coupled to the coupler interface in the closed state, the robotic arm may be allowed to move freely in response to movement at the handle of the surgical instrument to perform laparoscopic surgery. In some embodiments, the coupler body may be disposable after a single laparoscopic surgery. Optionally, the coupler body may be sterilizable such that it may be reused in multiple surgical procedures.

[0023] The system may further include a switch configured to transition between an unlocked position and a locked position. The switch may include an engagement portion configured to engage with the elongate shaft when the elongate shaft is placed within the inner cavity and the switch is in the locked position, thereby fixing the elongate shaft within the inner cavity. Thus, when the coupler body is coupled to the coupler interface, the elongate shaft is placed within the inner cavity, and the switch is in the locked position, the robotic arm may be configured to be freely movable in response to movement at the surgical instrument handle. Additionally, when the elongate shaft is placed within the inner cavity and the switch is in the locked position, the engagement portion may be configured to apply a frictional force to the elongate shaft, the frictional force being configured to permit rotational movement of the elongate shaft relative to the coupler body while inhibiting longitudinal movement of the elongate shaft relative to the coupler body. The switch may include a handle portion configured to be actuated to transition the switch between the unlocked position and the locked position.

[0024] The coupler body may further include a bracket slidably disposed within the coupler body. The bracket may include a contact surface configured to define at least a portion of the inner cavity and may be configured to be biased in a direction toward the inner cavity such that when the elongate shaft is placed within the inner cavity, the contact surface is configured to engage with the elongate shaft. Additionally, the coupler interface may include a repulsive magnet, and the bracket may include a magnet such that the repulsive magnet may be configured to apply a magnetic force to the magnet, thereby biasing the bracket in a direction toward the inner cavity. Further, the bracket may include a harness configured to couple to the magnet, the harness sized and shaped to be slidably disposed within a channel of the coupler body. Additionally, when the elongate shaft is placed within the inner cavity and the switch is in the locked position, the contact surface may be configured to apply a frictional force to the elongate shaft such that the frictional force may be configured to facilitate permitting rotational movement of the elongate shaft relative to the coupler body while inhibiting translational movement of the elongate shaft relative to the coupler body.

[0025] The coupler body may further include a clamp configured to transition between an unlocked state that permits the inner cavity to receive the elongate shaft and a locked state in which the clamp fixes the elongate shaft within the inner cavity. The clamp may be pivotally coupled to the coupler body by a rod and may be configured to be biased toward the locked state. Thus, the clamp may further include a handle portion configured to be actuated to transition the clamp from the locked state to the unlocked state, e.g., to release the surgical instrument from the coupler body. Additionally, at least a portion of the clamp may include a tapered surface configured to guide the elongate shaft into the inner cavity and to facilitate transitioning of the clamp from the locked state to the unlocked state in response to a force applied by the elongate shaft to the tapered surface as the elongate shaft is inserted into the inner cavity. Further, the bracket may include one or more supports coupled to the contact surface, each of the one or more supports including a channel sized and shaped to slidably receive the rod therethrough such that the bracket may be configured to be slidably disposed within the coupler body along the rod.

[0026] The coupler interface may further include a protrusion, and the coupler body may include a groove configured to receive the protrusion of the coupler interface. For example, the protrusion may include one or more indentations, and the coupler body may include one or more locking arms configured to transition between a locked configuration in which at least a portion of the one or more locking arms extends within the groove of the coupler body and an unlocked configuration in which the one or more locking arms do not extend within the groove of the coupler body. When the one or more locking arms are in the unlocked configuration, the protrusion of the coupler interface may be configured to be received by the groove of the coupler body, and when the protrusion is placed within the groove and the locking arms are in the locked configuration, at least a portion of the one or more locking arms may extend within the one or more indentations of the protrusion, thereby fixing the coupler body to the coupler interface. The one or more locking arms may be biased toward the locked configuration. Additionally, each of the one or more locking arms may include a handle portion configured to be actuated to transition the one or more locking arms from the locked configuration to the unlocked configuration.

[0027] Furthermore, the protrusion of the coupler interface may have a first geometry, and the groove of the coupler body may have a second geometry corresponding to the first geometry such that rotational movement between the coupler body and the coupler interface is prohibited when the protrusion is received by the groove. In some embodiments, the coupler interface may include one or more additional protrusions having a first geometry, and the coupler body may include one or more additional grooves having a second geometry such that rotational movement between the coupler body and the coupler interface is prohibited when the one or more additional protrusions are received by the one or more additional grooves. Additionally, the coupler body and the coupler interface may be configured to receive a sterile drape therebetween such that the sterile drape prevents contact between the surgical instrument and the robotic arm during a laparoscopic surgical procedure. The coupler body may further include one or more tapered surfaces configured to guide the elongate shaft into the lumen and facilitate self-alignment of the distal end of the robotic arm relative to the surgical instrument by rotating the coupler body and the coupler interface as the elongate shaft is inserted into the lumen along the one or more tapered surfaces to align the lumen with the elongate shaft.

[0028] According to another aspect of the present disclosure, a method is provided for using a robotic arm configured to be detachably coupled to a surgical instrument having a handle, an operating end, and an elongate shaft therebetween, for example, for assisting in laparoscopic surgery. The method may include: detachably coupling a coupler body to a coupler interface at a distal end of the robotic arm; inserting the elongate shaft of the surgical instrument into a lumen of the coupler body; transitioning the coupler body from an open state in which the elongate shaft is slidably movable within the lumen to a closed state in which longitudinal movement of the elongate shaft relative to the coupler body is inhibited while rotational movement of the elongate shaft relative to the coupler body is permitted in response to movement at the handle of the surgical instrument; and freely moving the robotic arm, for example, for performing laparoscopic surgery, by moving the handle of the surgical instrument when the coupler body is coupled to the coupler interface in the closed state.

[0029] For example, detachably coupling the coupler body to the coupler interface may include: actuating one or more locking arms of the coupler body to transition the one or more locking arms from a locked configuration in which at least a portion of the one or more locking arms extends within a groove of the coupler body to an unlocked configuration in which the one or more locking arms do not extend within the groove; inserting a protrusion of the coupler interface into the groove of the coupler body; and releasing the one or more locking arms to transition the one or more locking arms from the unlocked configuration to the locked configuration such that at least a portion of the one or more locking arms extends within one or more indentations of the protrusion, thereby fixing the coupler body to the coupler interface. Additionally, inserting the elongate shaft of the surgical instrument into the lumen of the coupler body may include guiding the elongate shaft along one or more tapered surfaces of the coupler body into the lumen, for example, by rotating the coupler body and the coupler interface to facilitate self-alignment of the lumen with the elongate shaft as the elongate shaft is inserted along the one or more tapered surfaces into the lumen.

[0030] Furthermore, inserting the elongate shaft of the surgical instrument into the lumen of the coupler body may include: actuating a clamp of the coupler body to transition the clamp from a locked state to an unlocked state that permits the lumen to receive the elongate shaft; inserting the elongate shaft of the surgical instrument into the lumen; and releasing the clamp to transition the clamp from the unlocked state to the locked state such that the clamp fixes the elongate shaft within the lumen. Transitioning the coupler body from the open state to the closed state may include transitioning a switch of the coupler body from an unlocked position in which the elongate shaft is slidably movable within the lumen to a locked position in which an engagement portion of the switch engages the elongate shaft placed within the lumen, thereby inhibiting longitudinal movement of the elongate shaft relative to the coupler body while permitting rotational movement of the elongate shaft relative to the coupler body. The method may further include placing a sterile drape between the coupler body and the coupler interface prior to detachably coupling the coupler body to the coupler interface.

[0031] According to another aspect of the present disclosure, there is provided a method of using a robotic arm configured to be removably coupled to a surgical instrument having a handle, an operating end, and an elongate shaft therebetween, for example, for assisting in laparoscopic surgery, the method being performed at a bedside adjacent to a bed accommodating a patient. The method may include: placing the robotic arm at the bedside, for example, for performing laparoscopic surgery; coupling the surgical instrument to the robotic arm, the robotic arm including a proximal end, a distal end, and a plurality of links and a plurality of joints between the proximal end and the distal end; and freely moving the robotic arm by moving the handle of the surgical instrument coupled thereto while the robotic arm remains placed at the bedside, for example, to perform laparoscopic surgery using the surgical instrument. When using the surgical instrument coupled to the robotic arm, the robotic arm may remain at the bedside. Thus, when performing laparoscopic surgery using the surgical instrument coupled to the robotic arm, a physician performing the laparoscopic surgery may remain at the bedside. Coupling the surgical instrument to the robotic arm may include coupling the surgical instrument to the robotic arm using a purely mechanical coupling, coupling the robotic arm only to the elongate shaft of the surgical instrument, coupling the surgical instrument to the robotic arm while keeping the handle of the surgical instrument fully exposed for contact with the surgeon's hand, coupling the surgical instrument to the robotic arm when the robotic arm is placed at the bedside, and / or removably coupling a coupler body to a coupler interface placed at the distal end of the robotic arm, and removably coupling the surgical instrument to the coupler body.

[0032] The method may further include transitioning the coupler body from an open state where the coupler body is slidably movable along the elongate shaft within the lumen of the coupler body to a closed state in which the robotic arm is allowed to move freely in response to movement at the handle of the surgical instrument, e.g., for performing laparoscopic surgery. Thus, when the coupler body is coupled to the coupler interface in the closed state, longitudinal movement of the elongate shaft relative to the coupler body may be inhibited while allowing rotational movement of the elongate shaft relative to the coupler body in response to movement at the handle of the surgical instrument. Further, when the coupler body is coupled to the coupler interface in the closed state, the coupler body may apply a frictional force to the elongate shaft that is sufficient to allow rotational movement of the elongate shaft relative to the coupler body while inhibiting longitudinal movement of the elongate shaft relative to the coupler body. Further, removably coupling the surgical instrument to the coupler body may include removably coupling the coupler body to a fixed point along the elongate shaft to provide a consistent reference point for force calculations of the surgical instrument. The method may further include placing a sterile drape between the coupler body and the coupler interface prior to removably coupling the coupler body to the coupler interface. Further, the method may include discarding the coupler body after a single laparoscopic surgery. The robotic arm may be remotely operated without receiving user input from a remote surgeon console.

[0033] Further, freely moving the robotic arm by moving the handle of the surgical instrument coupled thereto may include applying a force greater than a predetermined threshold to the robotic arm via the surgical instrument to automatically switch the robotic arm to a co-manipulation mode in which impedance is applied to the robotic arm to account for the weight of the surgical instrument and the robotic arm. The method may further include adjusting the predetermined threshold of the force applied to the robotic arm via a graphical user interface operatively coupled to the robotic arm to cause the robotic arm to automatically switch to the co-manipulation mode. Further, the plurality of joints of the robotic arm may include one or more powered joints that are operatively coupled to one or more motors disposed in a base proximal to the robotic arm. Thus, the method may include measuring the current of one or more of the motors, the current indicating the force applied to the robotic arm via the surgical instrument. Further, the impedance applied to the robotic arm to account for the weight of the surgical instrument and the robotic arm may be applied via one or more of the powered joints of the robotic arm.

[0034] The method may further include maintaining movement of the robotic arm within a predetermined amount for at least a predetermined dwell time, thereby automatically switching the robotic arm to a passive mode in which the robotic arm maintains a static position. Additionally, the method may include adjusting at least one of a predetermined amount of movement of the robotic arm or the predetermined dwell time via a graphical user interface operatively coupled to the robotic arm to cause the robotic arm to automatically switch to the passive mode. Additionally, the method may include moving the robotic arm by moving a handle of a surgical instrument beyond a predefined haptic barrier, thereby automatically switching the robotic arm to a haptic mode in which an impedance is applied to the robotic arm to cause movement of the robotic arm in response to movement at the handle of the surgical instrument to be more viscous in the haptic mode. The method may further include adjusting a position of the predefined haptic barrier via a graphical user interface operatively coupled to the robotic arm.

[0035] The method may further include selecting an identification of a surgical instrument coupled to the robotic arm via a graphical user interface operatively coupled to the robotic arm. Additionally, a proximal end of the robotic arm may be coupled to a base that is coupled to a platform via a stage assembly configured to move the base relative to the platform in at least two degrees of freedom. Accordingly, the method may include adjusting at least one of a vertical height or a horizontal position of the robotic arm relative to the platform via the stage assembly. For example, adjusting at least one of a vertical height or a horizontal position of the robotic arm via the stage assembly may include providing user input by at least one of: a graphical user interface operatively coupled to the stage assembly, or a user applying a force in at least one of at least two degrees of freedom in a distal region of the robotic arm.

[0036] The platform may include a plurality of wheels such that positioning the robotic arm at a bedside to perform a laparoscopic surgical procedure may include moving the platform relative to the bed via the plurality of wheels. The method may further include releasing a braking mechanism of the plurality of wheels to permit movement of the platform via the plurality of wheels. Additionally, the method may include displaying a virtual map that includes a graphical representation of the platform relative to the bed within a region surrounding the platform to facilitate positioning the robotic arm at the bedside for performing a laparoscopic surgical procedure. Additionally, the method may include adjusting at least one of a height or an orientation of an optical sensor to optimize a field of view of a surgical scene of the optical sensor, the field of view of the surgical scene of the optical sensor including at least one of the robotic arm, a surgical instrument coupled to the robotic arm, or the bed. The method may further include selecting a laparoscope, a retractor tool, a grasping tool, or a surgical cutting tool such that coupling a surgical instrument to the robotic arm may include coupling the laparoscope, the retractor tool, the grasping tool, or the surgical cutting tool to the robotic arm.

[0037] According to another aspect of the present disclosure, another co-manipulative surgical system is provided. The co-manipulative surgical system may include a robotic arm that includes a proximal end, a distal end configured to be detachably coupled to a surgical instrument, a plurality of linkages, and a plurality of joints; a coupler interface disposed at the distal end of the robotic arm; and a coupler body configured to be detachably coupled to the coupler interface and an elongate shaft of the surgical instrument. The coupler interface may include an iron rod and one or more sensors, such as one or more Hall effect sensors, which are configured to measure a magnetic field of the iron rod. The coupler body may include a magnet slidably disposed within the coupler body. For example, the magnet may be configured to induce a magnetic field in the iron rod based on a position of the magnet within the coupler body relative to the iron rod. Additionally, the co-manipulative surgical system may include a controller operatively coupled to the robotic arm and the one or more sensors such that the controller can be programmed to determine whether the coupler body is coupled to the coupler interface based on the magnetic field of the iron rod measured by the one or more sensors.

[0038] Furthermore, the controller can be programmed to determine whether the surgical instrument is coupled to the coupler body when the coupler body is coupled to the coupler interface, based on the magnetic field of the iron rod measured by the one or more sensors. Additionally, the controller can be programmed to identify a size of the surgical instrument coupled to the coupler body based on the magnetic field of the iron rod measured by the one or more sensors. When the coupler body is coupled to the coupler interface, the magnet is biased in a direction away from the coupler interface.

[0039] When the coupler body is coupled to the coupler interface and no surgical instrument is coupled to the coupler body, the magnet may be placed within a predetermined position within the coupler body such that the controller can be configured to determine that the coupler body is coupled to the coupler interface without a surgical instrument coupled to the coupler body, based on the magnetic field induced in the iron rod by the magnet at the predetermined position measured by the one or more sensors. Additionally, when the coupler body is coupled to the coupler interface and a surgical instrument is coupled to the coupler body, the magnet may be placed within a predetermined position within the coupler body such that the controller can be configured to determine that the coupler body is coupled to the coupler interface and the surgical instrument is coupled to the coupler body, based on the magnetic field induced in the iron rod by the magnet at the predetermined position measured by the one or more sensors.

[0040] The coupler body may include a bracket slidably disposed within the coupler body. The bracket may be configured to couple with a magnet and may include a friction pad configured to engage an elongate shaft when the elongate shaft is coupled to the coupler body and the coupler body is coupled to a coupler interface, such that when the elongate rod is coupled to the coupler body, the elongate shaft applies a force to the friction pad, thereby moving the magnet to a predetermined position within the coupler body through the bracket. The coupler interface may include a repulsive magnet configured to apply a magnetic force to the magnet when the coupler body is coupled to the coupler interface to bias the magnet in a direction away from the coupler interface. Optionally, the magnet may be coupled to a compression spring configured to bias the magnet in a direction away from the coupler interface.

[0041] The controller may be configured to allow the robotic arm to move freely in response to movement at the handle of the surgical instrument when the surgical instrument is coupled to the coupler interface through the coupler body for performing a laparoscopic surgery using the surgical instrument. The co-manipulating surgical system may further include a sterile drape configured to cover the coupler interface and the robotic arm during the surgery such that the coupler body may be configured to couple to the coupler interface with the sterile drape sandwiched therebetween. The coupler body may be configured to be discarded after a single surgery, while the robotic arm and the coupler interface may be configured to be reusable for other surgeries.

[0042] According to another aspect of the present disclosure, a method for using a robotic arm is provided, the robotic arm including a proximal end, a distal end having a coupler interface configured to be detachably coupled to a surgical instrument through a coupler body, a plurality of links, and a plurality of joints, for example, assisting in laparoscopic surgery. The method may include: measuring, by one or more sensors, a magnetic field of an iron rod extending within the distal end of the robotic arm; and determining, by a controller operatively coupled to the one or more sensors, based on the magnetic field of the iron rod measured by the one or more sensors, whether the coupler body is coupled to the coupler interface at the distal end of the robotic arm, wherein the coupler body includes a magnet slidably disposed therein such that the magnetic field of the iron rod measured by the one or more sensors varies based on the position of the magnet relative to the iron rod, wherein when the coupler body is coupled to the coupler interface, the magnet is biased in a direction away from the coupler interface.

[0043] According to another aspect of the present disclosure, there is provided a device for detachably coupling a surgical instrument having an elongate shaft to a distal end of a robotic arm of a co-manipulating surgical system to assist in laparoscopic surgery performed using the surgical instrument. The distal end of the robotic arm includes a coupler interface having a protrusion. The device may include a coupler body configured to be detachably coupled to the coupler interface and the elongate shaft of the surgical instrument. The coupler body may include a groove configured to receive the protrusion of the coupler interface; an opening sized and shaped to receive the elongate shaft therein; and a switch configured to transition between an unlocked position and a locked position. The switch may include an engagement portion configured to engage the elongate shaft when the elongate shaft is placed within the opening and the switch is in the locked position, thereby fixing the elongate shaft within the opening. Thus, when the coupler body is coupled to the coupler interface, the elongate shaft is placed within the opening, and the switch is in the locked position, the robotic arm may be configured to move freely in response to movement at the handle of the surgical instrument.

[0044] In addition, when the elongate shaft is placed within the opening and the switch is in the locked position, the engagement portion may apply a frictional force to the elongate shaft, the frictional force being configured to permit rotational movement of the elongate shaft within the opening while prohibiting translational movement of the elongate shaft relative to the coupler body. Further, the coupler body may include one or more tapered surfaces configured to guide the elongate shaft into the opening. The one or more tapered surfaces may be configured to facilitate self-alignment of the distal end of the robotic arm relative to the surgical instrument by rotating the coupler body and the coupler interface to align the opening with the elongate shaft as the elongate shaft is inserted into the opening along the one or more tapered surfaces. The device may further include a clamp configured to transition between an unlocked state and a locked state, wherein in the unlocked state the opening is configured to receive the elongate shaft, and in the locked state the clamp fixes the elongate shaft within the opening. The clamp may be biased towards the locked state. Additionally, at least a portion of the clamp may include a tapered surface configured to guide the elongate shaft into the opening and to facilitate transition of the clamp from the locked state to the unlocked state in response to a force applied by the elongate shaft to the tapered surface as the elongate shaft is inserted into the opening.

[0045] The coupler body may further include a bracket slidably disposed within the coupler body. The bracket may include a friction pad configured to define at least a portion of the opening, and the bracket may be biased in a direction toward the opening such that when the elongated shaft is placed within the opening, the friction pad engages the elongated shaft. In some embodiments, the coupler interface may include a repulsive magnet, and the bracket may include a magnet such that the repulsive magnet exerts a magnetic force on the magnet to bias the bracket in a direction toward the opening. Additionally, the bracket may include a harness configured to couple the magnet, and the size and shape of the harness may be slidably disposed within a channel of the coupler body. Further, the friction pad may be configured to exert a frictional force on the elongated shaft when the elongated shaft is placed within the opening and the switch is in the locked position, the frictional force being configured to permit rotational movement of the elongated shaft within the opening while prohibiting translational movement of the elongated shaft relative to the coupler body.

[0046] The device may further include a clamp pivotally coupled to the coupler body by a rod, the clamp being configured to transition between an unlocked state in which the opening is configured to receive the elongated shaft and a locked state in which the clamp secures the elongated shaft within the opening. Additionally, the bracket may include one or more supports coupled to the friction pad, each of the one or more supports including a channel sized and shaped to slidably receive the rod therethrough such that the bracket is slidably disposed within the coupler body along the rod. In some embodiments, the bracket may be coupled to a compression spring configured to exert a spring force on the bracket to bias the bracket in a direction toward the opening. The clamp may include a handle portion configured to actuate the clamp to transition from the locked state to the unlocked state. Additionally, the switch may include a handle configured to be actuated to transition the switch between an unlocked position and a locked position.

[0047] The protrusion may include one or more indentations, and the coupler body may include one or more locking arms configured to transition between a locked configuration in which at least a portion of the one or more locking arms extends within a recess of the coupler body and an unlocked configuration in which the one or more locking arms do not extend within the recess of the coupler body. Thus, when the one or more locking arms are in the unlocked configuration, the protrusion of the coupler interface may be received by the recess of the coupler body, and when the protrusion is placed within the recess and the locking arms are in the locked configuration, at least a portion of the one or more locking arms may extend within one or more indentations of the protrusion, thereby securing the coupler body to the coupler interface. The one or more locking arms may be biased toward the locked configuration. Additionally, each of the one or more locking arms may include a handle portion configured to be actuated to transition the one or more locking arms from the locked configuration to the unlocked configuration. The protrusion of the coupler interface may have a first geometry, and the recess of the coupler body may have a second geometry corresponding to the first geometry such that rotational movement between the coupler body and the coupler interface is prohibited when the protrusion is received by the recess. Further, the coupler body may be configured such that when the coupler body is coupled to the coupler interface, a sterile drape may be placed between the coupler body and the coupler interface.

[0048] According to another aspect of the present disclosure, another co-manipulative surgical system is provided. The co-manipulative surgical system may include a robotic arm that includes a proximal end operatively coupled to a base, a distal end configured to be removably coupled to a surgical instrument, a plurality of linkages, and a plurality of joints; a platform coupled to the base and configured to move the base in at least one degree of freedom; a plurality of motors operatively coupled to at least some of the plurality of joints; and one or more sensors configured to collect sensor data including at least one of 3D depth data or pixel image data. The co-manipulative surgical system may further include a controller operatively coupled to the robotic arm and the one or more sensors and configured to allow the robotic arm to move freely in response to movement at a handle of the surgical instrument for performing a laparoscopic surgical procedure using the surgical instrument, the controller programmed to: identify at least one of a position or orientation of one or more objects such as a surgical bed within an operating room based on the sensor data from the one or more sensors; estimate a relative distance between the one or more objects and at least one of the base or the robotic arm when at least one of the base or the robotic arm moves within the operating room; and if the estimated relative distance approaches a predetermined threshold, apply torque or impedance to at least some of the plurality of joints of the robotic arm via the plurality of motors to reposition the robotic arm or stop movement of the robotic arm to avoid a collision between the one or more objects and at least one of the base or the robotic arm.

[0049] In addition, the controller may be configured to: detect movement of the distal end of the robotic arm in a first direction in response to a first force applied by a user to the distal end of the robotic arm; in response to detecting the movement of the distal end of the robotic arm in the first direction, cause the platform to move the base in the first direction; and if the first force applied by the user to the distal end of the robotic arm drops below a predetermined threshold, cause the platform to stop the movement of the base in the first direction. The controller may be configured to cause the platform to move the base in the first direction if the first force applied to the distal end of the robotic arm exceeds a predetermined force threshold. The controller may be configured to identify a plane of one or more objects in the operating room based on sensor data from one or more sensors and estimate a relative distance between the one or more objects and at least one of the base or the robotic arm based on the plane of the one or more objects. In addition, the controller may be configured to: determine the type of laparoscopic surgical procedure to be performed; identify at least one of the position or orientation of the trocar port based on sensor data from one or more sensors; and apply torque to at least some of the multiple joints of the robotic arm through multiple motors to automatically position the robotic arm in a predetermined configuration relative to the trocar port based on the type of laparoscopic surgical procedure to be performed.

[0050] According to another aspect of the present disclosure, another co-manipulated surgical system is provided. The co-manipulated surgical system may include a robotic arm including a proximal end, a distal end configured to be detachably coupled to a surgical instrument, a plurality of linkages, and a plurality of joints; one or more sensors configured to collect sensor data including at least one of 3D depth data or pixel image data; and a controller operatively coupled to the robotic arm and the one or more sensors and configured to allow the robotic arm to move freely in response to movement at the handle of the surgical instrument for performing a laparoscopic surgical procedure using the surgical instrument. The controller may be programmed to: determine at least one of the position or orientation of the trocar port relative to the robotic arm based on sensor data from one or more sensors; detect movement of the trocar port based on sensor data from one or more sensors when the operating end of the surgical instrument is inserted through the trocar port; and reposition the robotic arm during the movement of the trocar port to maintain the position of the operating end of the surgical instrument relative to the trocar port. For example, the controller may be configured to detect movement of the trocar port in response to movement of the surgical bed. In addition, the controller may be configured to detect movement of the trocar port in response to movement of the patient's body caused by the patient's breathing. In some embodiments, the controller may be configured to retract the operating end of the surgical instrument within the trocar port at the distal end of the robotic arm before repositioning the robotic arm to maintain the position of the operating end of the surgical instrument relative to the trocar port during the movement of the trocar port.

[0051] According to another aspect of the present disclosure, there is provided a co-manipulative surgical system that can be configured to calibrate a new robotic arm. The co-manipulative surgical system can include a robotic arm that includes a proximal end configured to be detachably coupled to a cart, a distal end configured to be detachably coupled to a surgical instrument, a plurality of linkages, and a plurality of joints; an optical scanner configured to measure depth data; and a controller operatively coupled to the robotic arm and the optical scanner and configured to allow the robotic arm to move freely in response to movement at the handle of the surgical instrument for performing laparoscopic surgery using the surgical instrument. The controller can be programmed to: move the robotic arm relative to the cart in an expected predefined movement pattern according to a preprogrammed routine; compare depth data from the optical scanner indicating the actual movement of the robotic arm in response to the preprogrammed routine with the expected predefined movement pattern and generate a degree of error indicating a deviation between the actual movement of the robotic arm and the expected predefined movement pattern; and execute an optimization algorithm configured to reduce the degree of error such that the deviation between the actual movement of the robotic arm and the expected predefined movement pattern is reduced. The controller can be configured to allow the robotic arm to move freely in response to movement at the handle of the surgical instrument for performing laparoscopic surgery when the degree of error drops below a predetermined threshold.

[0052] According to another aspect of the present disclosure, there is provided a co-manipulative surgical system that can be configured to track a surgical instrument and overlay a virtual menu on a video feed. The co-manipulative surgical system can include a robotic arm that includes a proximal end, a distal end configured to be detachably coupled to a surgical instrument, a plurality of linkages, and a plurality of joints; and a controller operatively coupled to the robotic arm and a laparoscope configured to generate a video feed. The controller can be configured to allow the robotic arm to move freely in response to movement at the handle of the surgical instrument for performing laparoscopic surgery using the surgical instrument. Additionally, the controller can be programmed to: overlay the virtual menu on the video feed displayed on a display screen; track movement of the operating end of the surgical instrument in the video feed in response to movement at the handle of the surgical instrument to detect one or more predefined movement posture patterns of the operating end; and actuate functions of the co-manipulative surgical system associated with the virtual menu based on detection of the one or more predefined movement posture patterns of the operating end relative to the virtual menu. The virtual menu can include one or more menu options overlaid at at least one corner of the video feed.

[0053] Functions of a co-manipulating surgical system associated with a virtual menu can include, for example, adjusting a holding force threshold that needs to be exceeded for the robotic arm to switch from a passive mode where the controller causes the robotic arm to maintain a static position to a co-manipulating mode where the controller allows the robotic arm to move freely in response to movement at the handle of the surgical instrument to perform laparoscopic surgery using the surgical instrument. Additionally, functions of the co-manipulating surgical system associated with the virtual menu can include actuating an assisted aiming mode where the controller causes the laparoscope to automatically adjust at least one of a field of view or a position to assist in laparoscopic surgery. The controller can be configured to cause the virtual menu to be superimposed on a video feed displayed on a display screen in response to user input received through a graphical user interface operatively coupled to the controller, in response to a voice command of the user, and / or in response to actuation of an actuator placed on the robotic arm. Additionally, the controller can be configured to track movement of an operating end of a surgical instrument in response to user input received through a graphical user interface operatively coupled to the controller, in response to a voice command of the user, and / or in response to actuation of an actuator placed on the robotic arm.

[0054] According to another aspect of the present disclosure, a co-manipulating surgical system configured to provide an indication through haptic feedback is provided. The co-manipulating surgical system can include a robotic arm that includes a proximal end, a distal end configured to removably couple to a surgical instrument, a plurality of linkages, and a plurality of joints; and a controller operatively coupled to the robotic arm. The controller can be programmed to: automatically switch the robotic arm between a co-manipulating mode and a passive mode, where in the co-manipulating mode the controller allows the robotic arm to move freely in response to movement of the surgical instrument handle to perform laparoscopic surgery using the surgical instrument, and in the passive mode the controller causes the robotic arm to maintain a static position; cause a vibration at the distal end of the robotic arm that indicates a switch of the robotic arm from the co-manipulating mode to the passive mode. For example, the vibration can be configured to be perceivable by a user holding the handle of the surgical instrument while causing a negligible movement at the operating end of the surgical instrument. Additionally, the controller can be configured to cause a second vibration at the distal end of the robotic arm when the surgical instrument is coupled to the distal end of the robotic arm, where the second vibration can indicate that the surgical instrument is coupled to the distal end of the robotic arm.

[0055] The controller can be configured to cause the robotic arm to switch to a passive mode in response to determining that movement of the robotic arm due to movement at the handle of the surgical instrument is less than a predetermined amount during at least a predetermined dwell time. Additionally, the controller can be configured to cause the robotic arm to switch to a co-manipulation mode in response to determining that a force applied at the robotic arm due to a force applied at the handle of the surgical instrument exceeds a predetermined threshold. The controller can be configured to apply a first impedance to the robotic arm in the co-manipulation mode to handle the weight of the surgical instrument and the robotic arm. Further, the controller can be configured to generate an audible alert indicating a switch of the robotic arm from the co-manipulation mode to the passive mode. The robotic arm can further include a base operatively coupled to the proximal end of the robotic arm, and the system can further include a plurality of motors disposed within the base, wherein the plurality of motors are operatively coupled to at least some of the plurality of joints. Accordingly, the controller can be programmed to actuate at least one of the plurality of motors to cause vibration at the distal end of the robotic arm.

[0056] According to another aspect of the present disclosure, there is provided a co-manipulation surgical system that can be configured for automatic range detection. The co-manipulation surgical system can include a robotic arm that includes a proximal end, a distal end configured to be removably coupled to a laparoscope, a plurality of links, and a plurality of joints; an optical scanner configured to measure depth data; and a controller operatively coupled to the robotic arm and the optical scanner. The controller can be programmed to: compare movement of the laparoscope based on the depth data from the optical scanner with movement of the laparoscope field of view during movement of the laparoscope, based on a video feed collected from the operative end of the laparoscope; and identify the type of laparoscope based on movement of the laparoscope field of view during movement of the laparoscope. The controller can be configured to execute a pre-programmed routine in a calibration mode to cause the laparoscope to move in a pre-defined movement pattern according to the pre-programmed routine. For example, the pre-defined movement pattern can include circular movement. Optionally, movement of the laparoscope can be responsive to movement by a user at the laparoscope handle.

[0057] The type of laparoscope can be the angular degree of the operating end of the laparoscope. For example, the controller can be configured to identify the type of the laparoscope as a flat-head laparoscope when the movement of the laparoscope includes a circumferential movement and the movement of the laparoscope's field of view during the circumferential movement of the laparoscope includes a corresponding circumferential movement. Additionally, the controller can be configured to identify the type of the laparoscope as a flat-head laparoscope when the movement of the laparoscope includes a circumferential movement and the movement of the laparoscope's field of view during the circumferential movement of the laparoscope includes no change in the depth of the field of view. Further, the controller can be configured to identify the type of the laparoscope as an angled-head laparoscope when the movement of the laparoscope includes a circumferential movement and the movement of the laparoscope's field of view during the circumferential movement of the laparoscope includes a change in the depth of the field of view. The controller can be configured to allow the robotic arm to move freely in response to the movement at the laparoscope handle to perform laparoscopic surgery using the laparoscope. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1A and Figure 1B illustrates a conventional laparoscopic procedure performed by a surgeon and one or more assistants.

[0059] Figure 2 illustrates an exemplary co-manipulation surgical system constructed in accordance with the principles of the present disclosure.

[0060] Figure 3 illustrates an exemplary robotic arm of a system constructed in accordance with the principles of the present disclosure Figure 2 of.

[0061] Figure 4A and Figure 4B illustrates an exemplary wrist portion of a robotic arm constructed in accordance with the principles of the present disclosure.

[0062] Figure 4C is Figure 4A and Figure 4B a close-up view of an exemplary surgical instrument coupling mechanism of the wrist portion of.

[0063] Figure 4D is an exemplary robotic arm coupler interface of a surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure Figure 4C of.

[0064] Figure 5A and Figure 5B illustrates an exemplary surgical instrument coupler body of a surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure Figure 4C of.

[0065] Fig. 6A illustrates an alternative exemplary surgical instrument coupler body constructed in accordance with the principles of the present disclosure.

[0066] Figure 6B-6D Illustrated is attaching the coupler body of Fig. 6A to a surgical retractor device in accordance with the principles of the present disclosure.

[0067] Fig. 7A Illustrated is another alternative exemplary surgical instrument coupler body constructed in accordance with the principles of the present disclosure.

[0068] Figures 7B-7D Illustrated is attaching the coupler body of Fig. 7A to a surgical laparoscope device in accordance with the principles of the present disclosure.

[0069] Fig. 8A and Figure 8B Illustrated is a robotic arm in a sterile drape ready configuration.

[0070] Fig. 9A and Fig. 9B Illustrated is a robotic arm covered in a sterile drape.

[0071] Figures 10A-10D Illustrated is the rotation of the shoulder link of a robotic arm in accordance with the principles of the present disclosure.

[0072] Fig.11A Illustrated is an exemplary co-manipulative surgical system having an optical scanner in accordance with the principles of the present disclosure, and Fig. 11B Illustrated is Fig.11A the optical scanner.

[0073] Fig. 11C Illustrated is an exemplary co-manipulative surgical system having multiple optical scanners in accordance with the principles of the present disclosure.

[0074] Fig.12 Illustrated is a user operating a co-manipulative surgical system of Fig.11A in accordance with the principles of the present disclosure.

[0075] Fig.13A Illustrated is the field of view of an optical scanner during a laparoscopic surgical procedure, and Fig. 13B Illustrated is Fig.13A a depth map of the field of view of the optical scanner.

[0076] Fig.14 Illustrated are some example components that may be included in a co-manipulative robotic platform in accordance with the principles of the present disclosure.

[0077] Fig.15 is a flowchart illustrating the operation of a co-manipulative surgical system in accordance with the principles of the present disclosure.

[0078] Fig.16 is a flowchart illustrating the surgical instrument calibration of a co-manipulative surgical system in accordance with the principles of the present disclosure.

[0079] Fig.17 is a flowchart illustrating the operation of a robotic arm according to the principles of the present disclosure.

[0080] Fig.18A and Fig.18B is a force diagram illustrating the forces applied to a surgical instrument coupled to a robotic arm during a laparoscopic surgical procedure.

[0081] Fig.19 is a table of example values related to some arrangements of the passive mode of a robotic arm according to the principles of the present disclosure.

[0082] Fig. 20 Illustrates an example overview of some features and capabilities of a co-manipulating surgical system according to the principles of the present disclosure.

[0083] Fig.21 is a schematic overview of some electrical components and connections of a co-manipulating surgical system according to the principles of the present disclosure.

[0084] Fig. 22 is a flowchart illustrating an example process of acquiring and processing data from an optical scanner and an example application of the data according to the principles of the present disclosure.

[0085] Fig.23 is a schematic overview of the data flow of a co-manipulating surgical system according to the principles of the present disclosure.

[0086] Fig.24 is another schematic overview of the data flow of a co-manipulating surgical system according to the principles of the present disclosure.

[0087] Fig.25 is a schematic overview of the data flow and output control of a co-manipulating surgical system according to the principles of the present disclosure.

[0088] Fig.26 is a schematic overview of the data flow in the network of a co-manipulating surgical system according to the principles of the present disclosure.

[0089] Figures 27A-27D Illustrates the vertical and horizontal movement of a robotic arm according to the principles of the present disclosure.

[0090] Figures 28A-28D Illustrates an exemplary graphical user interface of a co-manipulating surgical system.

[0091] Fig.29 is a schematic diagram of an alternative co-manipulating surgical system constructed according to the principles of the present disclosure.

[0092] FIG. 30A to FIG. 43Illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure.

[0093] Fig.44A and Fig.44B Illustrate another exemplary surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure.

[0094] Fig.44C is Fig.44A a cross-sectional view of the surgical instrument coupling mechanism of.

[0095] Fig.45A and Fig.45B Illustrate Figures 44A-44C the coupler interface of the surgical instrument coupling mechanism of.

[0096] Fig.45C is Fig.45B a cross-sectional view of the coupler interface of.

[0097] Figures 46A-46C Illustrate Figures 44A-44C the coupler body of the surgical instrument coupling mechanism of.

[0098] Fig.46D Illustrate Figures 46A-46C the magnet bracket of the coupler body of.

[0099] Fig.46E is Fig.46A a cross-sectional view of the coupler body of.

[0100] Fig.47A is when the coupler body is detached from the coupler interface Fig.46A a cross-sectional view of the coupler body of.

[0101] Fig.47B is when the coupler body is coupled to the coupler interface Fig.44A a cross-sectional view of the surgical instrument coupling mechanism of.

[0102] Fig.48A and Fig.48B Illustrate various views of the surgical instrument coupling mechanism when coupling the coupler body to the coupler interface in accordance with the principles of the present disclosure Fig.44A of.

[0103] Fig.49 is when the surgical instrument is coupled to the coupler body Fig.44A a cross-sectional view of the surgical instrument coupling mechanism of.

[0104] Fig.50A and 50B Illustrate various views of the surgical instrument coupling mechanism when coupling the surgical instrument to the coupler body in accordance with the principles of the present disclosure Fig.44A of.

[0105] Fig.51 is a cross-sectional view of an alternative exemplary surgical instrument coupling mechanism when a surgical instrument is coupled to a coupler body constructed in accordance with the principles of the present disclosure.

[0106] Figures 52A to 52E Illustrates an alternative exemplary surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure.

[0107] Fig.53A and Fig.53B Illustrates an alternative exemplary coupler body constructed in accordance with the principles of the present disclosure.

[0108] Fig.54 Illustrates another alternative exemplary coupler body constructed in accordance with the principles of the present disclosure.

[0109] Fig.55 Illustrates an exemplary virtual overlay of a graphical user interface for co-manipulating a surgical system.

[0110] Fig.56 Illustrates an alternative exemplary articulating joint of a robotic arm constructed in accordance with the principles of the present disclosure. DETAILED DESCRIPTION

[0111] Disclosed herein are a co-manipulating surgical robotic system for assisting an operator (e.g., a surgeon) in performing a surgical procedure (e.g., a laparoscopic procedure) and a method of using the same. Currently, laparoscopic procedures typically require a surgeon and one or more assistants. For example, as Figure 1A shown, during a laparoscopic procedure, it may be necessary for assistant A1 to hold the retractor device 12 to expose tissue for the surgeon S, while it may be necessary for another assistant A2 to hold the laparoscope device 10 to provide a field of view of the surgical space within the patient to the surgeon S via a display (not shown) during the procedure. As Figure 1A shown, assistant A2 may need to hold the laparoscope device 10 in an unrealistic position, e.g., between the surgeon S's arms, while the surgeon actively operates additional surgical instruments, e.g., surgical instruments 14 and 16. As Figure 1A further shown, the surgeon S may need to release the surgical instrument 16 in order to guide / reposition the laparoscope device 10 held by assistant A2 in order to achieve the field of view desired by the surgeon.

[0112] As Figure 1B shown, an orthopedic retractor 18 mounted on a rail may be used to hold one or more surgical instruments in place during a laparoscopic procedure in an attempt to free the surgeon's and / or assistant's hands for other tasks as well as for stability. As Figure 1BAs shown, the first rail-mounted orthopedic retractor 18a can include a retractor end 20a for engaging and holding the laparoscopic device 10 in place when the locking member 22a is actuated. For example, the locking member 22a can be disengaged so that the retractor 18a can be manually positioned in a desired orientation relative to the patient and re-engaged to lock the retractor 18a and thus the laparoscopic device 10 coupled thereto in the desired position. As Figure 1B As shown, the second rail-mounted orthopedic retractor 18b having a retractor end 20b can be used during the procedure to engage and hold another surgical instrument in place when the locking member 22b is actuated. Thus, the retractors 18a and 18b require a great deal of manual interaction with the locking members 22a and 22b and the retractors 18a and 18b themselves to reposition and lock the respective tools in place.

[0113] The co-manipulating surgical robotic system described herein provides excellent control and stability such that a surgeon and / or assistant can seamlessly position various off-the-shelf surgical instruments as needed, thereby avoiding the workflow limitations inherent in human and mechanical solutions. For example, the robotic arms of the co-manipulating surgical robotic system can provide surgical assistance by keeping a first surgical instrument (e.g., a laparoscope) stable via a first robotic arm and a second surgical instrument (e.g., a retractor) stable via a second robotic arm throughout the procedure to provide an optimal view of the surgical site and reduce the variability of the forces applied by the surgical instruments to the body wall at the trocar sites. As will be understood by those skilled in the art, the robotic arms of the co-manipulating surgical robotic system described herein can hold any surgical instrument preferably having a long and thin instrument shaft for a surgical procedure such as a laparoscopic procedure, including, for example, an endoscope / laparoscope, a retractor, a grasper, surgical scissors, a needle holder, a needle actuator, a clamp, a suturing instrument, a cautery tool, a stapler, a clip applicator, etc.

[0114] Co-manipulating a surgical robot system also allows a surgeon to easily manipulate two tools when necessary, thus providing excellent control and stability of the procedure as well as overall safety. Any implementation of the system described herein enables a surgeon to directly co-manipulate instruments while remaining sterile at the patient's bedside. For example, the system can include two robotic arms that a surgeon can use to hold both a laparoscope and a retractor. During a surgical procedure, the system can seamlessly reposition either instrument to provide optimal visualization and exposure of the surgical field. The two instruments can be directly coupled to the robotic arms of the system, and the system can continuously monitor and record the positions of the two instruments and / or the two robotic arms throughout the procedure. Additionally, the system can record information such as the position and orientation of the surgical instruments attached to the robotic arms, sensor readings related to the (one or more) forces applied at the proximal and distal ends of the surgical instruments attached to the robotic arms, the forces required to hold each instrument in place, an endoscopic video stream, algorithm parameters, a 3D stream of the operating room captured using an optical scanning device, including for example the (one or more) positions of the (one or more) surgical access ports, the position and movement of the surgeon's hands, the position and orientation of the (one or more) surgical instruments, whether attached to the robotic arms, the patient position, and the patient table orientation and height.

[0115] Such data can be used to develop a database of historical data that can be used to develop algorithms used in some implementations to control one or more aspects of the operation of the system. Additionally, such data can be used during the procedure to control one or more aspects of the operation of the system in accordance with one or more algorithms of the system. For example, the data can be used to evaluate the fatigue level of a user of the system.

[0116] When an operator manipulates a robotic arm of a cooperative surgical robotic system by applying a movement to a surgical instrument coupled to the robotic arm, the system can automatically transition the robotic arm between various operating modes when determining a predetermined condition. For example, the system can transition the robotic arm to a passive mode in response to determining that the movement of the robotic arm caused by a movement at the handle of the surgical instrument is less than a predetermined amount for at least a predetermined dwell period, such that in the passive mode, the robotic arm maintains a static position, e.g., to prevent damage to the device and / or injury to the patient. Additionally, the system can transition the robotic arm to a cooperative mode in response to determining that the force applied at the robotic arm due to a force applied at the handle of the surgical instrument exceeds a predetermined threshold, such that in the cooperative mode, the robotic arm is allowed to move freely in response to a movement at the handle of the surgical instrument to perform a laparoscopic surgical procedure using the surgical instrument, while a first impedance is applied to the robotic arm in the cooperative mode to account for the weight of the surgical instrument and the robotic arm. Further, the system can transition the robotic arm to a haptic mode in response to determining that at least a portion of the robotic arm is outside a predefined haptic barrier, such that in the haptic mode, a second impedance greater than the first impedance is applied to the robotic arm, thereby making the movement of the robotic arm in response to a movement at the handle of the surgical instrument more viscous in the haptic mode than in the cooperative mode. The system can also transition the robotic arm to a robot-assisted mode in response to detecting various conditions that ensure automatic movement of the robotic arm, e.g., to guide a surgical instrument attached thereto along a planned trajectory or to avoid collision with another object or person in the surgical space.

[0117] Now referring to Figure 2 , a cooperative surgical robotic system 200 is provided. As Figure 2 shown, the system 200 can include a platform 100, e.g., a surgical cart, that is sized and shaped to support one or more robotic arms 300, e.g., robotic arm 300a and robotic arm 300b, each of the robotic arms 300 having a surgical instrument coupler interface 400 for removably coupling to a surgical instrument, and a computing system operatively coupled to the platform 100 and the robotic arms 300. As Figure 2 shown, the system 200 can also include a graphical user interface display 110 for displaying operational information and receiving user input.

[0118] Additionally, each of the robotic arms 300 may further include an indicator 334 for visually indicating in real time the operating mode associated with the respective robotic arm. For example, the indicator 334 may be positioned at least on the elbow joint of the robotic arm. Additionally or alternatively, the indicator 334 may be placed elsewhere on the system 200, such as on the platform 100, on the display 110, on one or more linkages and / or at joints, etc. Further, the indicator 334 may include a light, such as an LED light, which may be illuminated in various different colors and different patterns, such as solidly lit or flashing. For example, each operating mode of the system 200 may be associated with a uniquely colored light (such as red, yellow, blue, green, purple, white, orange, etc.). Thus, the indicator 334 may indicate a transition from one operating mode to another. Additionally or alternatively, the transition from one operating mode to another may also be indicated to the user through haptic feedback, for example, by vibrating the distal end of the robotic arm 300 and correspondingly vibrating the surgical instrument coupled thereto. For example, when the robotic arm 300 transitions from the co-manipulation mode to the static mode, the distal end of the robotic arm 300 may vibrate to assure the user that the robotic arm 300 is in the static mode and will remain in position when released by the user. Additionally or alternatively, an audible alarm may be emitted when the robotic arm 300 transitions from one operating mode to another to indicate to the user.

[0119] As Figure 2 shown, the platform 100 may include one or more stages coupled to the base portions of one or more robotic arms (e.g., the base portion 302a of the robotic arm 300a and the base portion 302b of the robotic arm 300b) for providing movement to the respective robotic arms, for example, at least in horizontal and vertical directions relative to the platform 100. Each stage may include a vertical extender, such as the vertical extender 106a or the vertical extender 106b, for independently vertically moving the robotic arm 300a or the robotic arm 300b relative to the platform 100 respectively, and a horizontal extender, such as the horizontal extender 108a or the horizontal extender 108b, for independently horizontally moving the robotic arm 300a or the robotic arm 300b relative to the platform 100 respectively, thereby allowing the operator to flexibly position the robotic arm 300 relative to the patient.

[0120] In addition, the platform 100 can include a plurality of wheels 104 (e.g., casters) to provide mobility of the platform 100 and thus provide mobility of the robotic arms 300 within the operating room. Each of the wheels 104 can include a braking mechanism that can be actuated to prevent the platform 100 from moving via the wheels 104. Thus, the platform 100 can independently move each of the robotic arms 300a and 300b in any direction (including a first or vertical direction toward and away from the floor, a second or horizontal direction toward and away from the patient, and / or a third direction or horizontal direction along the length of the patient). In some embodiments, the platform 100 can move the robotic arms 300a and 300b in the same direction simultaneously and can also cause rotational movement of the robotic arms 300a and 300b. When ready for operation, the platform 100 can be moved to a desired position alongside the hospital bed and locked in place via the wheels 104, and in response to user input received by the graphical user interface display 110, the vertical and horizontal positions of the robotic arms 300a and 300b can be adjusted to an optimal position relative to the patient for the procedure via the vertical extenders 106a, 106b and the horizontal extenders 108a, 108b. As described in further detail below, during a laparoscopic procedure and / or during robotic arm setup, the platform 100 can automatically move the robotic arms 300a and 300b in response to, for example, detection of potential collisions with other objects and / or personnel within the operating room and / or user input applied through the robotic arms.

[0121] The surgical robotic system 200 is configured for co-manipulation such that the system 200 can assist a user or operator, such as a surgeon and / or a surgical assistant, by allowing the user to freely move the robotic arm 300a and / or the robotic arm 300b, which is due to manipulation of one or more surgical instruments coupled to the robotic arm in response to a force input provided by the user to the surgical instrument. Thus, the system 200 can be configured such that it is not remotely controlled, such that the robotic arm 300 moves directly in response to movement of the surgical instrument coupled to it by the operator, while compensating for the mass of the surgical instrument and the corresponding robotic arm and providing local impedance along the robotic arm, thereby increasing the accuracy of the movement or action of the operator when the operator manipulates the surgical instrument.

[0122] System 200 is particularly useful in laparoscopic surgical procedures and / or other surgical procedures that utilize long, thin instruments that can be inserted into a patient's body, for example, via a cannula to allow surgical intervention. As will be understood by those skilled in the art, system 200 can be used in any desired or suitable surgical procedure. Additionally, system 200 can be used in conjunction with or in cooperation with video monitoring provided by one or more cameras and / or one or more endoscopes such that an operator of system 200 can view and monitor the use of instruments coupled to robotic arms 300a, 300b via respective coupler interfaces, which will be described in further detail below. For example, robotic arm 300a can be detachably coupled to and manipulate an endoscope, while robotic arm 300b can be detachably coupled to and manipulate a surgical instrument.

[0123] Now referring to Figure 3 , a surgical support arm is provided, such as robotic arm 300. As described above, system 200 can include a plurality of robotic arms, such as robotic arm 300a and robotic arm 300b. However, since each robotic arm can be identically constructed, for the sake of brevity, only a single robotic arm will be described with respect to Figure 3 collectively referred to as robotic arm 300. Aspects of the robotic arm described herein can utilize the structure of U.S. Patent No. 10,118,289 to Louveau, the entire contents of which are incorporated herein by reference. Robotic arm 300 can include a plurality of arm segments / linkages and a plurality of articulated moving joints extending from a base. For example, robotic arm 300 can include a base portion, a shoulder portion, an elbow portion, and a wrist portion, thereby mimicking the kinematics of a human arm. As Figure 3 shown, robotic arm 300 can include a base that includes a base portion 302 rotatably coupled to a shoulder portion 304 at a base joint 303. For example, shoulder portion 304 can be located on top of base portion 302 and can rotate about an axis Q1 relative to base portion 302 at base joint 303. In some embodiments, robotic arm 300 can be interchanged, swapped, or coupled to the base in any desired arrangement.

[0124] Robotic arm 300 can further include a shoulder linkage 305 that includes a proximal shoulder linkage 306 rotatably coupled to a distal shoulder linkage 308. The proximal end of proximal shoulder linkage 306 can be rotatably coupled to the shoulder portion 304 of the base at a shoulder joint 318 such that proximal shoulder linkage 306 can rotate about an axis Q2 relative to shoulder portion 304 at shoulder joint 318. As Figure 3As shown, axis Q2 can be perpendicular to axis Q1. The distal end of the proximal shoulder link 306 can be rotatably coupled to the proximal end of the distal shoulder link 308 at joint 320 such that the distal shoulder link 308 can rotate about axis Q3 relative to the proximal shoulder link 306 at joint 320. As Figure 3 shown, axis Q3 can be parallel to the longitudinal axis of shoulder link 305. Additionally, robotic arm 300 can include an actuator 330, such as a lever, button, sleeve, or switch, operatively coupled to the distal shoulder link 308 and / or the proximal shoulder link 306 such that the distal shoulder link 308 can rotate relative to the proximal shoulder link 306 only when the actuator 330 is actuated. Thus, axis Q3 can be a "set-up" axis about which the distal shoulder link 308 can rotate and be fixed relative to the proximal shoulder link 306 during a set-up phase prior to an operative phase of the robotic arm 300 for a surgical procedure, as further described in detail with respect to FIG. 10A to FIG. 10D As Figure 3 shown, the actuator 330 can be placed on the elbow link 310.

[0125] In some embodiments, the distal shoulder link 308 can be manually rotated relative to the proximal shoulder link 306 in a predetermined increment when the actuator 330 is actuated. Alternatively, when the actuator 330 is actuated, the distal shoulder link 308 can be automatically rotated relative to the proximal shoulder link 306 until the actuator 330 is released, as further described in detail below with respect to Fig.56 and as described in Noonan's U.S. Patent Application No. 18 / 331,060, the entire contents of which are incorporated herein by reference. For example, the actuator 330 can be a button, sleeve, or switch operatively coupled to a motor that is operatively coupled to the distal shoulder link 308 and / or the proximal shoulder link 306 such that when the actuator 330 is actuated, the associated motor causes the distal shoulder link 308 to rotate relative to the proximal shoulder link 306. The motor is placed within the base of the robotic arm 300 or, alternatively, the motor can be placed adjacent to the link 320, e.g., on the shoulder link 305. Thus, the actuator 330 can be a button, sleeve, or switch that allows dual actuation, e.g., a first actuation causes the distal shoulder link 308 to rotate relative to the shoulder link 306 in a first direction while a second actuation causes the distal shoulder link 308 to rotate in a second direction opposite the first direction. In some embodiments, the button or switch can be located on a graphical user interface such as the display 110. Additionally, in some embodiments, the distal shoulder link 308 can be automatically rotated relative to the proximal shoulder link 306 (e.g., during set-up) by a processor that co-manages the robotic platform to avoid collisions, as will be described in further detail below.

[0126] The robotic arm 300 may further include an elbow link 310. The proximal end of the elbow link 310 may be rotatably coupled to the distal end of the distal shoulder link 308 at an elbow joint 322 such that the elbow link 310 may rotate relative to the distal shoulder link 308 about an axis Q4 at the elbow joint 322. The robotic arm 300 may further include a wrist portion 311 which may include a proximal wrist link 312 rotatably coupled to the distal end of the elbow link 310 at a wrist joint 324, an intermediate wrist link 314 rotatably coupled to the proximal wrist link 312 at a joint 326, and a distal wrist link 316 rotatably coupled to the intermediate wrist link 314 at a joint 328, as Figure 4A and Figure 4B further shown in. Thus, the wrist portion 311 may rotate about an axis Q5 relative to the elbow link 310 at the wrist joint 324, the intermediate wrist portion 314 may rotate about an axis Q6 relative to the proximal wrist link 312 at the joint 326, and the distal wrist link 316 may rotate about an axis Q7 relative to the intermediate wrist link 314 at the joint 328.

[0127] The robotic arm 300 may further include a wrist portion 311 which may include a proximal wrist link 312 rotatably coupled to the distal end of the elbow joint 310 at a wrist joint 324, an intermediate wrist link 314 rotatably coupled to the proximal wrist link 312 at a joint 326, and a distal wrist link 316 coupled to / extending from the intermediate wrist link 314 which may be rotatably coupled to a surgical instrument coupler interface (not shown), e.g., coupler interface 400 (as Figure 4A and 4B further shown), coupler interface 4500 (as Fig.44A and 44B further shown) and / or coupler interface 5200 (as Figures 52A to 52E further shown). Thus, the wrist portion 311 may rotate about an axis Q5 relative to the elbow link 310 at the wrist joint 324, the intermediate wrist portion 314 may rotate about an axis Q6 relative to the proximal wrist link 312 at the joint 326, and the surgical instrument coupler interface may rotate about an axis Q7 relative to the distal wrist link 316 and correspondingly relative to the intermediate wrist link 314 at the joint 328.

[0128] Additionally, as Figure 3 and Figure 4BAs shown, the robotic arm 300 may include an actuator 332, such as a lever, button, or switch, that is operatively coupled to the elbow link 310 and / or the proximal wrist link 312 at the joint 324 such that the proximal wrist link 312 can rotate relative to the elbow link 310 only when the actuator 332 is actuated. Thus, the axis Q5 can be a "set" axis, and when the actuator 332 is actuated, for example, such proximal wrist link 312 can rotate and be fixed relative to the elbow link 310 during a set-up phase prior to an operative phase of the robotic arm 300 for a surgical procedure. When the actuator 332 is in an unactuated state, the set joint 324 prevents relative movement between the proximal wrist link 312 and the elbow link 310, thereby fixing the proximal wrist link 312 relative to the elbow link 310. In some preferred embodiments, when the actuator 332 is actuated, the proximal wrist link 312 can be manually rotated relative to the elbow link 310 in predetermined increments, thereby eliminating the need for additional motors and / or electronics at the distal region of the robotic arm 300. Alternatively, when the actuator 332 is actuated, the proximal wrist link 312 can be automatically rotated relative to the elbow link 310 until the actuator 332 is released, for example, by a motor operatively coupled to the proximal wrist link 312 and / or the elbow link 310 at the joint 324.

[0129] As Figure 3As shown, the robotic arm 300 can include multiple motors, such as motors M1, M2, M3, and M4. M1, M2, and M3 can all be placed within the base of the robotic arm 300, while motor M4 is preferably placed near joint 320. Optionally, motor M4 can also be placed within the base of the robotic arm 300. Each of motors M1, M2, and M3 can be operatively coupled to a respective actuated joint of the robotic arm 300, such as base joint 303, shoulder joint 318, and elbow joint 322, to apply local impedance at the respective joint. For example, motors M1, M2, and M3 can each generate impedance / torque at any one of base joint 303, shoulder joint 318, and elbow joint 322, thereby effectively applying impedance at the distal end of the robotic arm (e.g., at the attachment point with the surgical instrument) to improve the feel experienced by the operator during manipulation of the surgical instrument and the operator's movement during a surgical procedure. For example, impedance can be applied to the distal end of the robotic arm 300 and thus to the surgical instrument coupled thereto to provide the operator manipulating the surgical instrument with a sense of viscosity, stiffness, and / or inertia. Additionally, the applied impedance can simulate tissue density or stiffness, convey surgical boundaries to the operator, and can be used to guide the surgical instrument along a desired path, etc. In some embodiments, the motors can actuate the respective joints, causing the robotic arm 300 to move about the respective joints. Thus, axes Q1, Q2, and Q4 can all be "actuated" axes such that motors M1, M2, and M3 can apply impedance / torque to base joint 303, shoulder joint 318, and elbow joint 322, respectively, to inhibit or actuate rotation about the respective axes. As described in further detail below, motors M1, M2, and M3 can be controlled by a processor that co-manages the robotic platform. With three actuated axes, some embodiments of the robotic arm 300 can apply force / torque at the distal end of the robotic arm 300 in three directions, thereby moving a surgical instrument coupled to the distal end of the robotic arm 300 with three degrees of freedom.

[0130] As described above, the motor M4 is operatively coupled to the articulation 320 to apply torque to the articulation 320 to cause the distal shoulder link 308 to rotate relative to the proximal shoulder link 306 about the axis Q3. Unlike other motorized joints described herein (e.g., the base joint 303, the shoulder joint 318, and the elbow joint 322), the motorized joint 320 is preferably not "reversibly actuable", i.e., the user cannot actuate the motorized joint 320, e.g., when the system is in the co-manipulation mode, the user cannot actuate the motorized joint 320 by the movement of a surgical instrument coupled to the robotic arm. Instead, as described above, the actuation of the motorized joint 320 can be performed by one or more actuators (e.g., the actuator 330), and the actuator 330 can be actuated to automatically cause the rotation of the distal shoulder link 308 relative to the proximal shoulder link 306.

[0131] The axis Q6 and the axis Q7 can each be a "passive" axis, such that the intermediate wrist link 314 can rotate relative to the proximal wrist link 312 at the passive joint 326 without any applied impedance from the system 200, and the surgical instrument coupler interface can rotate relative to the distal wrist link 316 at the passive joint 328 without any applied impedance from the system 200. The distal end of the distal wrist link 316 is rotatably coupled to the surgical instrument coupler interface for detachably coupling to a surgical instrument, e.g., by a coupler body 500 (as further shown in Figure 4A and 4B ), a coupler body 4600 (as further shown in Fig.44A and 44B ), a coupler body 5100 (as further shown in Fig.51 ), a coupler body 5300 (as further shown in Fig.53A and 53B ), and / or a coupler body 5400 (as further shown in Fig.54 ), and the coupler body 5400 can be detachably coupled to the surgical instrument and the coupler interface, as described in further detail below. Optionally, the wrist portion 311 can include a passive spherical joint at the attachment point to the surgical instrument, as described in U.S. Patent No. 10,582,977, the entire disclosure of which is incorporated herein by reference.

[0132] Referring again to Figure 3, the robotic arm 300 may also include a plurality of encoders disposed on at least some of the plurality of joints of the robotic arm 300, such as encoders E1 - E7. For example, encoder E1 for measuring the angle between the base portion 302 and the shoulder portion 304 may be disposed on or near the base joint 303 within the base, encoder E2 for measuring the angle between the shoulder portion 304 and the proximal shoulder link 306 may be disposed on or near the shoulder joint 318 within the base, encoder E3 for measuring the angular rotation between the proximal shoulder link 306 and the distal shoulder link 308 may be disposed on or near joint 320, encoder E4 for measuring the angle between the distal shoulder link 308 and the elbow link 310 may be disposed near the motor M3 operatively coupled to the elbow joint 322 within the base, since the transmission of rotational movement at the elbow joint 322 is achieved via a link extending from the base to the elbow joint 322, encoder E5 for measuring the angular rotation between the elbow link 310 and the proximal wrist link 312 may be disposed on or near the wrist joint 324, encoder E6 for measuring the angle between the proximal wrist link 312 and the intermediate wrist link 314 may be disposed on or near joint 326, and encoder E7 for measuring the angle between the distal wrist link 316 and the surgical instrument coupler interface may be disposed on or near joint 328. Optionally, encoder E4 may be disposed on or near the elbow joint 322. The encoders may be absolute encoders or other position / angle sensors configured to generate data for accurately determining the position and / or angle of the corresponding link at the respective joint and / or the precise position of the surgical instrument coupled to the distal end of the robotic arm 300. Thus, the precise position of each link, joint, and distal end of the robot 300 can be determined based on the measurements obtained from the plurality of encoders. Preferably, redundant encoders are placed at each location where encoders are placed along the robotic arm 300 to provide more accurate position data and to detect fault conditions, as described in further detail below.

[0133] Before attaching to the surgical instrument, the robotic arm 300 may be manually manipulated by a user, e.g., to position the robotic arm 300 at a desired location for coupling to the surgical instrument. For example, the user may manually manipulate the robotic arm 300 via the wrist portion 311, actuator 330, and / or actuator 332. When actuator 330 is actuated, the user may manually rotate the distal shoulder link 308, and when actuator 332 is actuated, the user may manually manipulate the proximal wrist portion 312. Additionally, the robotic arm 300 may be further manually moved by directly applying a force on other links and / or joints of the robotic arm 300.

[0134] In some embodiments, in response to a force applied by a user to the robotic arm 300, e.g., at the wrist portion 311, wrist joint 324, elbow link 310, etc., e.g., a predetermined amount or pattern in a given direction, a processor that co-manipulates the robotic platform can cause a stage coupled to the base portion 302 of the robotic arm 300 to move the robotic arm 300 in the same direction, e.g., move the robotic arm 300 via vertical and horizontal extenders of the stage, until the system detects that the force applied by the user to the robotic arm 300 drops below a predetermined threshold, e.g., when the user releases the robotic arm 300. In some embodiments, when the user applies a reaction force to the robotic arm 300, e.g., in a direction opposite to the direction of movement of the robotic arm 300, the system can stop the robotic arm 300 from moving in the same direction as the force applied by the user to facilitate the positioning of the robotic arm 300 relative to the patient.

[0135] For example, the user can apply a force exceeding a predetermined force threshold to the wrist portion 311 in a first direction, which causes the stages of the platform 100 to move the robotic arm 300 in the same direction until the user stops the movement of the wrist portion 311 (e.g., by releasing the robotic arm 300 or applying a reaction force to the robotic arm 300), causing the system to stop the movement of the stages of the platform 100. Thus, the movement of the distal end of the robotic arm, e.g., the wrist portion 311, wrist joint 324, elbow joint 310, etc., can serve as an input for generating movement in a specific direction of the robotic arm via the stages coupled thereto. This automatic movement of the stages of the platform 100 in response to a force applied by the user to the distal end of the robotic arm 300 may be limited to when the system is in a predefined operating mode, which can be input during setup and / or during a surgical procedure, e.g., input when the GUI 110 and / or voice control is executed.

[0136] When attached to a surgical instrument, the robotic arm 300 can still be manually maneuvered by the user directly applying forces (e.g., one or more linear forces and / or one or more torques) to the robotic arm 300; however, during a laparoscopic procedure, the operator preferably manipulates the robotic arm 300 only via the handle of the surgical instrument, which applies forces / torques to the distal end of the robotic arm 300 and thus to the links and joints of the robotic arm 300. When the operator applies a force to the surgical instrument attached to the robotic arm 300, causing movement of the surgical instrument, the robotic arm 300 will move in response to the movement of the surgical instrument to provide the operator with the ability to move the surgical instrument freely relative to the patient. As described in further detail below, when the operator moves the surgical instrument, the robotic arm 300 can apply impedance to account for the weight of the surgical instrument and the robotic arm 300 itself, such as gravity compensation, thus making it easier for the operator to move the instrument, regardless of the gravitational and / or inertial forces applied to the robotic arm and / or the surgical instrument. As will be understood by those skilled in the art, the robotic arm 300 can include fewer or more articulated moving joints than Figure 3 shown, as well as a corresponding number of motors and encoders / sensors.

[0137] Now referring to Figure 4C , a close-up view of the coupling mechanism of the coupler interface 400 and the coupler body 500 is provided. The coupler interface 400 can be rotatably coupled to the distal end of the distal wrist link 316 using any suitable fastener or connector (e.g., magnets, screws, pins, clamps, welding, adhesives, rivets, and / or any other suitable fastener or any combination of the foregoing). As Figure 4C shown, the coupler interface 400 can be rotatably coupled to the distal end of the distal wrist portion 316 using a fastener 410, which can be threaded or have other features such that the fastener 410 and thus the coupler interface 400 can be selectively attached to the distal wrist portion 316. The fastener 410 can be coupled to an insert element 408, which has an opening therein to receive the fastener 410, and the insert element 408 is positioned at or within the distal end of the distal wrist portion 316. In some embodiments, the fastener 410 can be a pin, or can have other features such as a ball, latch, or otherwise allow the fastener 410 to selectively couple with the distal wrist portion 316.

[0138] The coupler body 500 may be detachably coupled to the coupler interface 400. The coupler body 500 may have an opening 514 sized and shaped to slidably and releasably receive the elongate shaft of a surgical instrument therethrough. For example, the coupler body 500 may be detachably coupled to the coupler body 500 via a magnetic connection, thereby facilitating effective attachment and separation between the coupler body 500 and the coupler interface 400, for example, by overcoming the magnetic coupling force between the coupler body 500 and the coupler interface 400. Thus, as Figure 4C shown, the coupler body 500 may have one or more magnets 506 extending away from the surface of the coupler body 500. In the assembled state, the surface of the coupler body 500 contacts the surface of the coupler interface 400. Alternatively, in embodiments without a coupler interface, the magnet 506 may directly contact the distal end of the distal wrist portion 316. Thus, the distal end of the coupler interface 400 or the distal wrist portion 316 may have an iron-based base member configured to receive and magnetically couple with the magnet 506 of the coupler body 500 such that the coupler body 500 may be detachably coupled to the coupler interface 500 and / or the distal end of the distal wrist portion 316.

[0139] Figure 4D Illustrated is a surgical instrument coupler interface 400. As Figure 4D shown, the coupler interface 400 may have a recessed portion 404 sized and shaped to receive a complementary geometry of the coupler body 500 defined by a ridge 402. Thus, when the complementary geometry of the coupler body 500 is received in the recessed portion 404 in the assembled state, rotational movement of the coupler body 500 relative to the coupler interface 400 may be restricted or otherwise prevented. Figure 4D Illustrated is a surgical instrument coupler interface 400. As Figure 4DAs shown, the coupler interface 400 can have a recessed portion 404 that is sized and shaped to receive a complementary geometry of the coupler body 500 defined by a ridge 402. Thus, when the complementary geometry of the coupler body 500 is received within the recessed portion 404 in an assembled state, rotational movement of the coupler body 500 relative to the coupler interface 400 can be restricted or otherwise prevented. Additionally, the coupler interface 400 can have one or more recesses or depressions 406 that are sized and shaped to receive one or more magnets 506 therein. The coupler interface 400 can have an iron base member or magnet within the recess 406 to magnetically couple with the magnet 506. For example, the magnet within the recess 406 can have a south magnetic pole, and the magnet 506 can have a north magnetic pole, or vice versa. Additionally, the polarity of the magnets can ensure proper coupling orientation. The recess 406 can be sized and shaped to restrict or otherwise prevent movement between the coupler body 500 and the coupler interface 400 in any radial or direction perpendicular to the axial (e.g., longitudinal) centerline of the magnet 506 when the coupler body 500 and the coupler interface 400 are in an assembled state. As will be understood by those skilled in the art, the coupler interface 400 can have fewer or more than two recesses 406 such that the coupler body 500 will have a corresponding amount of magnets.

[0140] Now referring Figure 5A and Figure 5B , a coupler body 500 is provided. As Figure 5A shown, the coupler body 500 can have one or more magnets 506 placed on a portion 502 that has a geometry complementary to the recessed portion 404 of the coupler interface 400, as described above, to facilitate alignment between the coupler body 500 and the coupler interface 400. Additionally, the coupler body 500 can have one or more grooves 504 that are sized and shaped to engage with the complementary ridge 402 of the coupler interface 400. The grooves 504 and the ridge 402 can interact to assist in aligning the coupler body 500 with the coupler interface 400 by restricting or otherwise preventing movement between the coupler body 500 and the coupler interface 400 in at least two directions D1 and D2, as Figure 4C shown. Thus, in an assembled state, movement of the coupler body 500 relative to the coupler interface 400 in any axial direction can be prevented.

[0141] As Figure 5A and Figure 5BAs shown, the coupler body 500 may have a first portion 508 and a second portion 510. The first portion 508 may be coupled to or integrally formed with the second portion 510, for example, via a hinge 512, which may be a movable hinge formed of the same material as and / or integrally formed with the first portion 508 and the second portion 510, such that the second portion 510 may move or rotate relative to the first portion 508 to expand (increase in size) or contract (decrease in size) an opening 514 defined by the first portion 508 and the second portion 510. The first portion 508 and the second portion 510 may form a clamping member that may contract around the elongate shaft of a surgical instrument positioned in the opening 514 when a screw 516 (e.g., a wing screw) is tightened to couple the instrument 112 to the coupler body 141. Thus, the coupler body 500 may transition between a first unfastened / open state or position and a second fastened / closed state or position.

[0142] The diameter of the opening 514 may be selected based on the surgical instrument to be coupled to the coupler body 500. For example, the coupler body may be selected from a plurality of coupler bodies, each having an opening sized and shaped to receive the elongate shaft of a particular surgical instrument (such as a laparoscope or other surgical instrument, including surgical instruments for orthopedic and trauma surgery (OTS), needle holders, clamping members, scissors, etc.) having a predetermined elongate shaft diameter. The coupler body 500 may be coupled to the surgical instrument at any desired axial position on the surgical instrument.

[0143] As Figure 4C shown, the coupler body 500 may include a recess 520 extending through the second portion 510 and a recess 522 extending through at least a portion of the first portion 508. The recess 520 is aligned with the recess 522 for receiving a locking portion 518 of the screw 516. For example, the locking portion 518 may have a male threaded surface, and the recesses 520, 522 may have female threaded surfaces to engage the locking portion 518. The screw 516 may be loosened by hand to open or expand the opening 514 such that the surgical instrument may be removed, repositioned, rotated, and / or slid, etc. Once the coupler body 500 is coupled to the surgical instrument, for example, via the screw 516, the coupler body 500 and the surgical instrument coupled to the coupler body 500 may be removably coupled to the coupler interface 400 via a magnet 506.

[0144] The opening 514 can be defined by a first semi-circular cutout in the first portion 508 of the coupler body 500 and a second semi-circular cutout in the second portion 510, so as to engage with the circular outer surface of the elongate shaft of the surgical instrument. The opening 514 can include, for example, a rubber pad, sheet, bump, O-ring, protrusion, or other component or feature configured to be able to contact and clamp the outer surface of the elongate shaft of the surgical instrument. For example, the rubber material can be silicone rubber or any other suitable type of rubber. Thus, once the coupler body 500 is coupled to the surgical instrument, for example, by the fixing screw 516, axial movement of the surgical instrument (e.g., in the direction along the longitudinal axis of the surgical instrument) can be at least inhibited or otherwise prevented, or in some embodiments, axially and rotationally relative to the coupler body 500 in a fixed state. Preferably, the surgical instrument coupled to the coupler body 500 can be freely rotated by the operator relative to the coupler body 500, while axial movement of the surgical instrument relative to the coupler body 500 is prohibited or otherwise prevented in a fixed state. For example, the frictional force between the outer surface of the elongate shaft of the surgical instrument and the inner surface of the coupler body 500 defining the opening 514 can be selected such that rotation of the surgical instrument relative to the coupler body 500 requires less force than axial movement of the surgical instrument relative to the coupler body 500 in a fixed state. Thus, the coupler 500 can be configured to account for diameter variations and surface variations (including variations in the coefficient of friction of the surface) of the surgical instrument.

[0145] In some embodiments, when at least a threshold force is applied to the surgical instrument relative to the coupler body 500, or when the release or state change of the coupler body 500 is actuated, the surgical instrument can move axially relative to the coupler body 500. For example, such actuation can be achieved by, for example, pressing a button, loosening a locking screw (such as the locking screw 516 or other connector), moving a dial, or otherwise changing the coupler body 500 and / or the coupler interface 400 from a second fixed state to a first non-fixed state. Thus, by loosening the screw 516 or other manually operated fastener or fastening mechanism (such as a clamp in the coupler body 500), repositioning the surgical instrument at a desired axial position, and retightening the screw 516 or other manually operated fastener or fastening mechanism, the surgical instrument can be repositioned axially relative to the coupler body 500. The coupler body 500 can be disposable, or alternatively, can be sterilizable such that it can be sterilized between surgical procedures.

[0146] As described above, the diameter of the opening of the coupler body can be selected based on the surgical instrument to be coupled to the coupler body. The most commonly used laparoscopic surgical instruments have a predefined and known elongate shaft diameter, and thus multiple coupler bodies can be provided, each coupler body having an opening sized and shaped to receive and engage a specific surgical instrument. For example, Fig. 6A FIG. shows a coupler body 600 having an opening 614 sized and shaped to receive a 5 mm diameter surgical instrument, such as a retractor device 12. The coupler body 600 can be constructed similar to the coupler body 500. For example, the coupler body 600 can include a first portion 608 coupled to a second portion 610 via a hinge portion 612, and recesses 620, 622 in a locking portion 618 for securely receiving screws 616. As Figure 6B shown, the coupler body 600 can receive the elongate shaft 12a of the retractor 12 through the opening 614, for example, from the operating end of the retractor 12, such that the coupler body 600 can slide along the elongate shaft 12a until the coupler body 600 engages the proximal portion 12b of the retractor 12, as Figure 6C shown. Preferably, when the coupler body 600 contacts the proximal portion 12b, the coupler body 600 is coupled to the retractor 12 because this point along the retractor 12 is fixed, thereby providing a consistent reference point for calculating force measurements, as described in further detail below. Thus, when the coupler body 600 is in the desired orientation along the elongate shaft of the retractor 12, e.g., adjacent the proximal portion 12b, the screw 616 can be coupled to the coupler body 600 to fix the coupler body 600 to the retractor 12. As described above, the coupler body 600 is fixed to the retractor 12 such that rotational movement of the retractor 12 relative to the coupler body 600 is permitted, while axial movement of the retractor 12 relative to the coupler body 600 is constrained, e.g., the force required to move the retractor 12 relative to the coupler body 600 is much higher than the force required to rotate the retractor 12 relative to the coupler body 600.

[0147] Fig. 7A FIG. shows a coupler body 700 having an opening 714 sized and shaped to receive a 10 mm diameter surgical instrument, such as a laparoscopic device 10. The coupler body 700 can be constructed similar to the coupler body 600. For example, the coupler body 700 can include a first portion 708 coupled to a second portion 710 via a hinge portion 712, and recesses 720, 722 in a locking portion 718 for securely receiving screws 716. As Figure 7BAs shown, the coupler body 700 can receive the elongate shaft 10a of the laparoscopic device 10, for example, through the opening 714 from the operating end of the laparoscope 10, such that the coupler body 700 can slide on the elongate shaft 10a until the coupler body 700 engages the proximal portion 10b of the laparoscope 10, as Figure 7C shown. Preferably, when the coupler body 700 contacts the proximal portion 10b, the coupler body 700 is coupled to the laparoscope 10 because that point along the laparoscope 10 is fixed, thereby providing a consistent reference point for calculating force measurements, as described in further detail below. Thus, when the coupler body 700 is in the desired orientation along the elongate shaft of the laparoscope 10, e.g., adjacent the proximal portion 10b, the screw 716 can be coupled to the coupler body 700 to fix the coupler body 700 to the laparoscope 10. As described above, the coupler body 700 is fixed to the laparoscope 10 such that rotational movement of the laparoscope 10 relative to the coupler body 700 is permitted, while axial movement of the laparoscope 10 relative to the coupler body 700 is constrained, e.g., the force required to move the laparoscope 10 relative to the coupler body 700 is much higher than the force required to rotate the laparoscope 10 relative to the coupler body 700

[0148] In the case where a properly sized coupler body is coupled to a selected surgical instrument, the coupler body can be removably coupled to the coupler interface 400 of the robotic arm 300. The coupler body 500 and the coupler interface 400 can be configured for single-handed coupling such that an operator can use one hand to couple the coupler body 500 and thus the surgical instrument coupled thereto to the coupler interface 400 of the robotic arm 300. Preferably, a surgical drape can be pinched or clamped between the coupler body and the coupler interface 400 and draped over the robotic arm 300 to maintain sterility of the surgical space and prevent contact with non-sterile components of the robotic arm 300. Thus, the sterile drape can pass continuously between the coupler body and the coupler interface (e.g., without holes, slits, or any other type of opening) such that the coupler body is on one side of the sterile drape and the coupler interface, the robotic arm 300, and / or other components of the system 200 are on the other side of the sterile drape. In some embodiments, the coupler body can be integrated with the surgical drape. Additionally or alternatively, the surgical drape can include an adapter integrated therewith such that the coupler body 500 can be coupled to the coupler interface 400 via the adapter, e.g., the adapter can be positioned between the coupler body 500 and the coupler interface 400.

[0149] Now referring to Fig. 8A and Figure 8B , the robotic arm 300 can be positioned in the surgical drape ready configuration. As Fig. 8AAs shown, the robotic arm 300 can be extended such that the wrist portion 311, the elbow link 310, and the shoulder link 305 extend away from the shoulder portion 304 of the base to allow a surgical / sterile drape to cover each component of the robotic arm 300. Additionally, as Figure 8B shown, when there are two robotic arms (e.g., robotic arm 300a and robotic arm 300b), the robotic arm 300a and the robotic arm 300b can be angled away from each other, for example, by rotating the shoulder portion 304a relative to the base portion 302a of the robotic arm 300a and by rotating the shoulder portion 304b relative to the base portion 302b of the robotic arm 300b, such that the wrist portion 311a, the elbow link 310a, and the shoulder link 305a extend away from the wrist portion 311b, the elbow link 310b, and the shoulder link 305b. This configuration allows the corresponding robotic arms to be effectively and accessibly covered with a surgical / sterile drape. Additionally, in the extended position, the robotic arms can be outside of the virtual haptic boundary such that the robotic arms are in haptic mode and a high level of impedance is applied to the robotic arms, making the movement of the robotic arms more viscous, which makes it easier for the operator to drape the robotic arms but provides movement of the robotic arms if needed. For example, Fig. 9A FIG. illustrates a single robotic arm 300 covered with a sterile drape 800, and Fig. 9B FIG. illustrates robotic arms 300a, 300b covered with sterile drapes 800a, 800b, respectively.

[0150] The sterile drape 800 can be completely enclosed at its end portions. In some embodiments, the sterile drape 800 can have an opening in its distal portion (which can optionally have a sterile seal or interface) through which a portion of the robotic arm 300, the coupler interface 400, the coupler body 500, and / or a surgical instrument can pass. A drape having a sealed end portion without any openings and sealed along its length can provide a better sterile barrier for the system 200. Thus, all of the robotic arm 300 can be located inside and / or completely enclosed by the sterile drape 800 except at the opening at the proximal end of the sterile drape 800 (e.g., near the base of the robotic arm 300). In some embodiments, the coupler body 500 and the coupler interface 400 can have electrical connectors to create an electrical connection between the robotic arm 300 and the surgical instrument. Thus, electrical signals can be transmitted through the sterile drape 800. Alternatively, the sterile drape 800 can include an opening such that wires or other components can pass through the opening to provide a wired communication channel to electrical components, which can include, for example, a memory chip for calibration, a radiofrequency probe for ablation, a camera, and other electronic components. The surgical instrument and the coupler body can alternatively be passive or non-electronic such that wires do not need to pass through the sterile drape 800.

[0151] Now referring to FIG. 10A to FIG. 10D , rotation of the distal shoulder link 308 relative to the proximal shoulder link 306 of the shoulder link 305 is provided. As described above, the axis Q3 can be a "set-up" axis such that during the set-up phase of the robotic arm 300, e.g., before operating the robotic arm 300 in a surgical procedure, the distal shoulder link 308 can rotate relative to the proximal shoulder link 306 when the actuator 330 is actuated. As Fig. 10A shown, the shoulder portion 304 can optionally initially be rotated to a desired position relative to the base portion 302, thereby rotating all of the links distal to the proximal shoulder link 306 coupled to the shoulder portion 304 relative to the base portion 302 and providing sufficient space for rotation of the robotic arm 300 about the joint 320. Further, as Fig. 10A shown, the wrist portion 311 can extend at least partially away from the base portion 302 so as not to collide with any components of the robotic arm 300 when the robotic arm 300 rotates about the joint 320. As Fig. 10B shown, the actuator 330 must be actuated to allow the distal shoulder link 308 to rotate relative to the proximal shoulder link 306 at the joint 320. As described above, the construction of the actuator 330 can be referenced in the description of U.S. Patent Application No. 18 / 331,060. Fig. 10CIllustrated is a robotic arm 300 in a desired orientation for a particular laparoscopic procedure when the distal shoulder link 308 rotates relative to the proximal shoulder link 306. Fig. 10D Illustrated is a robotic arm 300a in a desired orientation when the distal shoulder link 308a rotates relative to the proximal shoulder link 306a of the robotic arm 300b. In some embodiments, the joint 320 may be operatively coupled to a motor such that the distal shoulder link 308 can automatically rotate relative to the proximal shoulder link 306, as Fig.56 further described in detail.

[0152] Now referring to Fig.11A and Fig. 11B , an exemplary cooperative robotic surgical system with an optical scanner is provided. As Fig.11A shown, the system may be configured similar to the Figure 2 system 200, having multiple robotic arms, such as robotic arm 300a and robotic arm 300b. As described above, although only two robotic arms are illustrated in Fig.11A , fewer or more robotic arms may be used in combination with the optical scanner 1100. Additionally, the system may include one or more optical scanners 1100, such as, for example, a LiDAR scanner or other suitable optical scanning device, such as an RGBD camera or sensor, an RGB camera with machine learning, a time-of-flight depth camera, structured light, a multi-projection camera, a stereo camera, an ultrasonic sensor, a laser scanner, other types of coordinate measurement area scanners, or any combination of the foregoing. For example, a LiDAR camera / scanner may be capable of recording both the color (RGB) and depth (D) of the surgical field and may include, for example, the Intel RealSense LiDAR Camera L515 or the Intel RealSense Depth Camera D435i (provided by Intel Corporation of Santa Clara, California) or other LiDAR or depth cameras with similar or suitable specifications, including but not limited to any one of the following specifications: (i) range: 25 cm to 500 cm; depth accuracy: 5 mm or approximately 5 mm; depth field of view: 70x55 or approximately 70x55 (degrees); depth output resolution: 1024x768 pixels or approximately 1024x768 pixels; depth / RGB frame rate: 30 frames per second; RGB frame resolution: 1920x1080; and / or RGB field of view: 70x43 degrees or approximately 70x43 degrees. The lidar scanner or optical scanner may also include 1 / 4-20 UNC threads or 2x M3 thread mounting points. As will be understood by those skilled in the art, the optical scanner 1100 may be used in other cooperative robotic surgical systems described herein, such as system 200 or any variant thereof.

[0153] As Fig.11A shown, the platform supporting robotic arms 300a, 300b can support the optical scanner 1100 and any other electronics, wiring, or other components of the system such that the optical scanner 1100 is mounted in a fixed orientation relative to other objects in the surgical space and the position and orientation of the optical scanner 1100 are known or can be determined relative to the global coordinate system of the system and thus the robotic arms. This allows all data streams to be transformed into a single coordinate system for development purposes. For example, the optical scanner 1100 can be supported on a pole or shaft, such as pole 1102, which can have an adjustable height or otherwise be adjustable in any direction (e.g., up / down, left / right, toward / away from the patient) to allow the optical scanner 1100 to obtain an optimal field of view or position relative to other components of the system (e.g., robotic arms 300a, 300b, surgical instruments attached thereto, the surgeon, and / or the surgical assistant). Additionally, telemetry data captured by the optical scanner 1100 (e.g., indicating the movement of the surgeon's hand, other body parts, the hospital bed, trocars, surgical instruments, and other components of the system) can be recorded to provide a rich and detailed dataset describing the precise movements and forces exerted by the surgeon throughout the procedure.

[0154] For example, the data obtained can be used to optimize the procedures performed by the system, including, for example, the automatic servoing (i.e., movement) of one or more portions of the robotic arm 300. By tracking the tendency of the surgeon to hold the tool in a particular region of interest and / or the tendency of the surgeon to avoid moving the tool into a particular region of interest, the system can optimize the automatic servoing algorithm to provide more stability in the particular region of interest. Additionally, the data obtained can be used to optimize the procedures performed by the system, including, for example, the automatic re-centering of the field of view of the optical scanning device of the system. For example, if the system detects that the surgeon has moved or predicts that the surgeon may move out of the field of view, the system can cause the robotic arm supporting the optical scanning device (e.g., laparoscope) to automatically adjust the laparoscope to track the desired orientation of the image while the surgeon performs the desired procedure. This behavior can be surgeon-specific and may require an understanding of the particular surgeon's preferences for the surgical area of interest. Thus, the system can control the robotic arm based on the specific operating requirements and / or preferences of a particular surgeon. Additionally, if the system detects that the robotic arm has been in an extended position for longer than a predetermined threshold, the system can cause the step coupled to the base portion of the robotic arm to move the robotic arm in a manner that facilitates the extension of the robotic arm, thereby providing additional range for the user to extend the robotic arm.

[0155] Now referring Fig. 11C , another exemplary cooperative robotic surgical system having multiple optical sensors is provided. As Fig. 11C As shown, system 200 has a plurality of robotic arms supported by platform 100, such as robotic arms 300a and 300b, and platform 100 has a plurality of wheels for providing mobility to platform 100. As described above, each of the plurality of wheels may include a braking mechanism that can be actuated to engage and prevent platform 100 from moving. For example, the braking mechanism may be operatively coupled to the controller of system 200. Additionally, system 200 may include a plurality of optical sensors placed on platform 100, such as optical scanners 1100a, 1100b, and 1100c. For example, optical scanner 1100a may be placed on top of platform 100, as described above for optical scanner 1100 of Fig.11A and optical scanners 1100b and 1100c may be placed on the sides of platform 100. Additionally or alternatively, one or more optical scanners may be placed below platform 100. Optical scanners 1100a, 1100b, and 1100c are configured to capture depth data. For example, optical scanners 1100a, 1100b, and 1100c may be, for example, depth cameras, stereo RGB cameras, LIDAR devices, and / or electromagnetic, capacitive, or infrared proximity sensors, etc.

[0156] The controller of system 200 may use the depth data generated by the plurality of optical sensors to generate a virtual map of the area around platform 100 (e.g., within an operating room) in real time, such as an "aerial view". For example, the virtual map may illustrate the operating room from a top perspective. Additionally, as Fig. 11CAs shown, the virtual map may include the platform 100 (including the robotic arms 300a, 300b) and graphical representations of one or more objects (e.g., the patient table PT) and / or one or more persons (e.g., the operator O, the person PI, and the person P2) within the area surrounding the platform 100. Specifically, the virtual map may graphically illustrate the proximity between the platform 100 and the one or more objects / persons, e.g., when the operator O is moving the platform 100 through the operating room. The controller may cause the display 110 to display the virtual map such that the operator O can view the virtual map on the display 110 in real time as the operator O moves the platform 100 through the operating room. Thus, the operator O can see the objects and / or persons in the area around the platform 100 that the operator O otherwise could not see with their own eyes, e.g., due to the platform 100 and / or the robotic arms 300a, 300b blocking the operator O's field of view, and avoid collisions between the platform 100 and / or the robotic arms 300a, 300b and the objects / persons in the operating room. Additionally, when the virtual map indicates that the platform 100 and / or the robotic arms 300a, 300b are approaching one or more objects / persons in the operating room or are within a predetermined distance of one or more objects / persons in the operating room, the controller may cause the display 110 to display an alert, e.g., a visual or audible alert.

[0157] In some embodiments, when the platform 100 is moving within the operating room, the controller may cause the display 110 to display the virtual map only. For example, the platform 100 may include one or more actuators, such as buttons, levers, or handles, which may be operatively coupled to the braking mechanism of the wheels of the platform 100 such that when the actuator is actuated, the braking mechanism disengages, allowing mobility of the platform 100. Thus, when the actuator is not actuated, the braking mechanism is engaged, preventing mobility of the platform 100. Accordingly, when the actuator is actuated, the controller may automatically cause the display 110 to display the virtual map such that the operator O can observe the area around the platform 100 before, during, or after the platform 100 moves when the braking mechanism disengages. Once the actuator is released such that the braking mechanism is re-engaged, the display 110 may stop displaying the virtual map. In some embodiments, when the virtual map indicates that the platform 100 and / or the robotic arms 300a, 300b are approaching one or more objects / persons in the operating room or are within a predetermined distance of one or more objects / persons in the operating room, the controller may override the operator's actuation of the actuator and re-engage the braking mechanism, thereby preventing further movement of the platform 100. Thus, the actuator may need to be released and re-actuated by the operator to disengage the braking mechanism and allow further movement of the platform 100.

[0158] In addition, the system can process color and / or depth data obtained from the optical scanners 1100a, 1100b, and / or 1100c to identify objects in the operating room, such as a hospital bed or a trocar, and planes associated with the identified objects. After knowing the positions of the platform 100 and the robotic arms 300a, 300b relative to the identified objects, the system can automatically move (or stop moving) the steps coupled to the base portions of the robotic arms 300a, 300b to avoid collisions with the identified objects during setup, e.g., when the robotic arms 300a, 300b approach a predetermined distance threshold relative to the identified objects. In addition, the system can also generate and emit, for example, an audible alarm indicating the proximity of the steps of the platform 100 and / or the robotic arms 300a, 300b relative to the identified objects. For example, when the distance between the steps of the platform 100 and / or the robotic arms 300a, 300b and the identified objects decreases, as sensed by the system based on depth data, the amplitude and / or frequency of the audible alarm can change.

[0159] In addition, when knowing the positions of the platform 100 and the robotic arms 300a, 300b relative to the patient and the trocar, in combination with the knowledge of where the robotic arms 300a, 300b are to be placed relative to the patient and the trocar for a given surgical procedure, the system can automatically position the robotic arms 300a, 300b in a setup configuration relative to the patient and the trocar for the given surgical procedure. For example, the system can automatically position the distal ends of the robotic arms near the trocar and can further arrange the robotic arms in a predetermined configuration preferably used for the given surgical procedure, e.g., via the articulating joints of the robotic arms.

[0160] In addition, when knowing the positions of the platform 100 and the robotic arms 300a, 300b relative to the trocar, if the system detects that the position of the hospital bed and accordingly the position of the trocar are changing, e.g., via user adjustment, the system can automatically adjust the arrangement of the robotic arms to accommodate the movement of the hospital bed and maintain the relative position between the distal end of the robotic arm and the trocar. In some embodiments, when detecting the movement of the hospital bed, the system can automatically move the robotic arm so as to retract the surgical instrument coupled thereto into the trocar before automatically adjusting the arrangement of the robotic arm to maintain the relative position between the distal end of the robotic arm and the trocar, so that the distal end of the surgical instrument is located within the trocar and away from the anatomical structures within the patient's body.

[0161] Fig.12 Illustration of a system with an optical scanner 1100 operating during a laparoscopic procedure. As Fig.12As shown, an optional additional optical scanner (e.g., camera 1200) can be used to provide additional viewpoints, e.g., redundant measurements of the movement of an instrument held by a robotic arm, and / or to provide a video stream of the surgical scene, e.g., via streaming, for monitoring and analysis. As Fig.12 As shown, the system can include two robotic arms, e.g., robotic arms 300a, 300b, such that when a surgeon operates and manipulates a retractor 12 coupled to the distal end of robotic arm 300b, robotic arm 300a holds the laparoscope 10 in a fixed position relative to the patient. Additionally, during a surgical procedure, robotic arms 300a, 300b can be covered with sterile drapes 800a, 800b, respectively. As described above, when the retractor 12 is coupled to robotic arm 300b, the surgeon can freely manipulate the retractor 12, such that movement of robotic arm 300b is caused by the surgeon moving the retractor 12, and at the same time robotic arm 300b accounts for the weight of the retractor 12 and robotic arm 300b. During a surgical procedure, the optical scanner 1100 can be used to monitor the identity, position, orientation, and / or movement of a surgical instrument (e.g., laparoscope 10) coupled to robotic arm 300a, and the identity, position, orientation, and / or movement of a surgical instrument (e.g., retractor 12) coupled to robotic arm 300b, and whether any surgical instrument has been intentionally or unintentionally detached from the respective robotic arm. Additionally, the optical scanner 1100 can be used to monitor the identity, position, orientation, and / or movement / displacement of any trocar Tr to ensure proper alignment of the robotic arm and / or surgical instrument relative to the respective trocar. The system can be used for surgical procedures having one, two, three, four, or more trocars, depending on the surgical procedure intended to be performed by the system.

[0162] Fig.13A and Fig. 13B illustrates exemplary data generated by the optical scanner 1100. For example, Fig.13A illustrates image data captured by the optical scanner 1100, and Fig. 13BIllustrated is a depth map of at least some objects within a surgical space generated from data captured by an optical scanner 1100. Specifically, the optical scanner 1100 can create a depth map, such as a point cloud, where the value of each pixel is related to the distance from the optical scanner 1100. For example, the difference between the pixels of a first object (such as a first surgical instrument) and a second object (e.g., a trocar) will enable the system to calculate the distance between the surgical instrument and the trocar. Additionally, the difference between the pixels of a first object (such as a first surgical instrument) at a first point in time and the pixels of the first object at a second point in time will enable the system to calculate whether the first object has moved, its movement trajectory, its movement speed, and / or other parameters associated with the changing position of the first object.

[0163] As Fig.13A and Fig. 13B shown, surgeon S is manipulating a surgical tool and / or a covered robotic arm (DA) and an uncovered robotic arm (UA) positioned relative to a pneumoperitoneum abdomen (A). As described above, the data stream from the robotic arm, the camera feed from the laparoscope, the data obtained from the optical scanner 1100, and optionally the data captured from one or more imaging devices placed on structures, walls, ceilings, or other structures within the operating room adjacent to the robotic arm can be recorded, stored, and used individually or in combination to understand and control the surgical system and the procedures of the surgical system. The foregoing components, devices, and their combinations are collectively referred to herein as an optical scanner or an optical scanning device.

[0164] For example, the system can measure and record any of the following within the coordinate space of the system: the movement of a handheld surgical instrument manipulated by the surgeon (attached to or separate from the robotic arm); the presence / absence of other surgical personnel (e.g., scrub nurse, circulating nurse, anesthesiologist, etc.); the height and angular orientation of the surgical table; the patient position and volume on the surgical table; the presence / absence of a drape on the patient; the presence / absence of trocar ports, and if present, their position and orientation; the gestures made by the surgical personnel; the tasks performed by the surgical personnel; the interaction of the surgical personnel with the system; surgical instrument identification; the attachment or detachment "action" of the surgical instrument to / from the system; the position and orientation tracking of specific features of the surgical instrument relative to the system (e.g., camera head, coupler, (one or more) fiducial markers, etc.); the measurement of moving contours or specific features in the scene that allow identification of the stage of the surgery; the position, orientation, identity, and / or movement of any other instruments, features, and / or components of the system or used by the surgical team.

[0165] The system can combine the above measurements and / or other data with any other telemetry data from the system and / or video data from the laparoscope to provide a comprehensive data set, and utilize this comprehensive data set to improve the overall usability, functionality, and safety of the co-manipulation robotic-assisted surgical system described herein. For example, when the system is set to start a procedure, the optical scanner 1100 can detect the height and orientation of the surgical workbench. This information can allow the system to automatically configure the degrees of freedom of the platform 100 that supports the robotic arm 300 to a desired or correct position relative to the surgical workbench. Specifically, the optical scanner 1100 can be used to ensure that the height of the platform 100 is optimally positioned to ensure that the robotic arm 300 overlaps with the intended surgical workspace. Additionally, as described above, the system can automatically reconfigure the degrees of freedom of the platform 100 and the alignment of the robotic arm 300 in response to the movement of the operating table and thus the trocar, so as to maintain the relative position between the distal end of the robotic arm and the trocar.

[0166] Furthermore, based on the data obtained by the optical scanner 1100, the system can warn surgical staff of potential collisions (during setup or during the procedure) between the system and other multiple fixed devices in the operating room (e.g., surgical workbench, laparoscope tower, camera boom, etc.) and between the system and members of the surgical staff, such as inadvertent collisions by the staff. The system can use this information to recommend repositioning of the platform 100 and / or other components of the system, the surgical workbench, and / or the patient, and / or prevent the robotic arm from switching to the co-manipulation mode due to the force applied to the robotic arm by a collision with the staff, even if the force exceeds the predetermined force threshold of the robotic arm.

[0167] In addition, the data obtained from the optical scanner 1100 can be used to monitor the progress of the setup of the surgical procedure and can be combined with the known state of the system to inform remote hospital staff (e.g., surgeons) of the overall readiness to start the procedure. Such progress steps can include: (i) the patient is on the workbench; (ii) the patient is covered; (iii) sterile instruments are available; (iv) the robotic arm is covered; (v) the trocar port is inserted; and (vi) confirmation that instruments (e.g., laparoscope and retractor) are attached to the robotic arm of the system. For example, the data obtained from the optical scanner 1100 can include detected gestures indicating the state of the system (e.g., the system is covered), readiness to start the procedure, etc., and can also be used to prepare the system for attaching or detaching surgical instruments.

[0168] Additionally, the optical scanner 1100 can identify the specific surgeon performing the procedure, enabling the system to load the system profile associated with the specific surgeon into the system using the surgeon's identity. The system profile can include information related to the surgeon's operating parameters and / or preferences, a list of the surgeon's patients with the parameters of each patient, the desired or required algorithm sensitivity of the surgeon, the degrees of freedom positioning of the support platform, and the like. Examples of algorithm sensitivities that can be surgeon-specific include: adapting / modifying the force required to transition from a passive mode to a co-manipulation mode (e.g., from low force to high force), adapting / modifying the viscosity felt by the surgeon when co-manipulating the robotic arm (e.g., from low viscosity to high viscosity), and the like. Additionally, the surgeon's preferences can include the preferred arrangement of the robotic arm 300. For example, the positioning of the links and joints of the robotic arm 300 relative to the patient can be different, and regarding a specific surgical instrument, e.g., the preferred placement between the laparoscope and the retractor can be different.

[0169] In some embodiments, the surgeon's preferences can be learned based on data from past procedures and / or sensors that collect information about the current procedure, including the surgeon's current posture, the surgeon's height, the surgeon's hand preference, and other similar factors. For example, the system can record when the user interacts with the system and also record what the user does to the system, such that the dataset can allow the surgeon's preferences to be "learned" and updated over time. This learning can be accomplished via traditional algorithmic methods (i.e., trends over time, averaging, optical flow, etc.) or via machine learning methods (classification, discrimination, neural networks, reinforcement learning, etc.). Fig.24 A data flow 2400 for updating the system configuration based on learned user behavior is illustrated. As Fig.24 shown, the system can be connected to an online database that can store the surgeon profile and each of a plurality of possible data sources, which can include an optical sensor, an encoder, and / or other sensors, and / or a database of manually entered user inputs. The data sources can be associated with a given surgeon, their preferred robotic arm arrangement and operating parameters, and each procedure performed using the system, which can allow the system configuration to be recorded and analyzed and how it changes between and within procedures. In the case of machine learning, the co-manipulation capabilities of the system can be utilized such that the user's actions can be used to annotate data to create a training dataset.

[0170] Regarding degrees of freedom positioning, the height of the surgical table is typically adjusted to accommodate the height of the surgeon in some operating rooms. Thus, by detecting the surgeon and loading the surgeon's specific profile, the system can position the platform at a height suitable for the corresponding surgeon to accommodate the preferred height of the surgical table. Additionally, the horizontal translation of the robotic arm can depend on the size of the patient. Thus, by accessing a patient list, the system can adjust the position of the arm based on the patient's body mass index ("BMI"). For example, for a patient with a high BMI, the system can move the robotic arm away from the surgical table, and for a patient with a low BMI, the system can move the robotic arm closer to the surgical table. Thus, the system allows the surgical team to fine-tune the position of the robotic arm relative to the patient as needed. The system can also be configured to access a hospital medical record database to access the type of procedure and any other medical data available (e.g., CT scan images, X-ray images, MRI images, and / or other patient-specific information), which can be used to inform the positioning of trocar ports and the position and orientation of the platform 100 relative to the patient.

[0171] Based on data captured by the optical scanner 1100, the system can generate a virtual model of multiple fixed devices and / or other objects in the operating room, the multiple fixed devices and / or other objects being within the movement range of the robotic arm in the same coordinate space as the robotic arm and surgical instruments coupled thereto, such that the virtual model can be stored and monitored, e.g., to detect potential collisions. Additionally, when the objects move relative to each other, the system can track the position and orientation of each virtual model and the objects within the virtual model, such that if the proximity (i.e., the spacing between) of any virtual model or object is below a predefined threshold, e.g., within 50 mm, within 75 mm, from 30 mm or less to 100 mm or more, the system can warn the user. In some embodiments, the distance threshold can be based on the Euclidean distance between the closest points on two virtual models, the normal distance between two surfaces of a virtual model, etc. Additionally, if the proximity of any virtual model or object (e.g., the robotic arm) reaches or is below a predefined threshold relative to the surface of the laparoscopic tower or the surgical table or other objects within the surgical space, the system can stop or inhibit (e.g., prevent) further movement of the robotic arm, e.g., freeze the robotic arm. Additionally, if the system detects that the proximity between objects (e.g., fixed devices or members of the surgical staff other than the surgeon) moving towards the corresponding robotic arm reaches or is below a predefined threshold, the system can freeze the robotic arm, thereby preventing an inadvertent movement of the robotic arm that might otherwise be caused by such a collision or inadvertent force (e.g., an inadvertent collision from a member of the staff or another fixed device, etc.).

[0172] In addition, based on data captured by the optical scanners 1100a, 1100b, 1100c, the system can generate a virtual map having a graphical representation of objects and / or people within a predefined region around the platform and robotic arm in the same coordinate space as the platform and robotic arm in the operating room, such that the virtual map can be stored and displayed to a user, e.g., to detect potential collisions as the user moves the platform throughout the operating room. Additionally, the system can track the position and orientation of the graphical representation within the virtual map such that if the proximity between any of the objects and / or people and the platform and / or robotic arm falls within a predefined threshold, e.g., within 50 mm, 75 mm, from 30 mm or less to 100 mm or greater, the system can warn the user.

[0173] Furthermore, based on data captured by the optical scanner 1100, the system can track the movement of a hand-held surgical instrument that is directly and independently controlled by a surgeon and that is not coupled to the robotic arm. For example, the optical scanner 1100 can track clearly defined features of the instrument, fiducial markers attached to the instrument or to the surgeon's glove (e.g., a sterile glove), a coupler between the robotic arm and the instrument, the distal tip of the instrument, and / or any other defined location on the instrument. For example, the fiducial markers can include a Manus virtual reality glove (provided by Manus of the Netherlands) or other wearable device, and / or an OptiTrack system (provided by NaturalPoint, Inc. of Corvallis, Oregon, USA). The following are examples of the uses and purposes of the movement data: (i) closing the control loop between the hand-held instrument and the robotic arm holding the camera, thereby allowing the surgeon to servo (i.e., move) the camera by "pointing" with the hand-held instrument; (ii) tracking information that can be used independently or in combination with other data streams to identify the stage of a surgical procedure; (iii) identifying the surgeon's dominant hand; (iv) monitoring metrics associated with the surgeon's experience; (v) identifying which tools the surgeon is using and when they are changed to other tools; and / or (vi) tracking the patient's skin surface, as well as the number, location, and orientation of trocar ports. This data and information can also be used and computed by the system as part of a co-manipulation control paradigm. By measuring the true position and orientation of the trocar ports, additional safety checks can be provided to the system to ensure that system-level calculations are correct, e.g., to ensure that the actual movement of the robotic arm or instrument matches the commanded movement of the robotic arm or instrument in the robot-assisted mode.

[0174] Based on the data captured by the optical scanner 1100, the system can also track which instrument is being used in the respective ports, how frequently instruments are exchanged between ports, which ports have manually held instruments versus instruments coupled to the robotic arm, to monitor and determine whether additional trocar ports are added, whether the system holds the instrument in place when the patient or the surgical table is moved (in which case the system can change the operating mode of the robotic arm to a passive mode and by repositioning the robotic arm 300 and / or the platform 100), and / or other conditions or parameters of the operating room or the system. Knowledge of the position and orientation of the skin surface and the trocar ports relative to the robotic arm can facilitate the implementation of "virtual boundaries", as described in further detail below.

[0175] In addition, based on the data obtained by the optical scanner 1100, for example, tracking the movement of the distal end of the laparoscope coupled to the robotic arm 300, and in addition the image data captured by the laparoscope, the system can identify the type of laparoscope coupled to the robotic arm 300. For example, commonly used laparoscopes during laparoscopic procedures include flat-head laparoscopes and angled-head laparoscopes, for example, laparoscopes with a 30-degree angled head. The system can determine which type of laparoscope is currently connected to the robotic arm 300 by comparing the image data of the pre-defined movement pattern of the laparoscope obtained by the optical scanner 1100 (for example, moving the distal end of the laparoscope in a circular pattern in a plane perpendicular to the longitudinal axis of the laparoscope) with the image data obtained by the laparoscope when the laparoscope moves in the pre-defined movement pattern. For example, for a flat-head laparoscope, when the distal end of the laparoscope moves in a circular pattern in a plane perpendicular to the longitudinal axis of the laparoscope, the image data captured by the laparoscope should move along a circular plane path, for example, the depth of field of the laparoscope's field of view does not change; while for an angled-head laparoscope, when the distal end of the laparoscope moves in a circular pattern in a plane perpendicular to the longitudinal axis of the laparoscope, the image data captured by the laparoscope will observe a change in the depth of field of the laparoscope's field of view.

[0176] In addition, when replacing the current robotic arm, e.g., during a surgical procedure, the system can calibrate the new robotic arm (with or without a tracker at the distal end of the new robotic arm) based on data obtained by the optical scanner 1100 to ensure that the system accurately senses the kinematics of the new robotic arm. Specifically, the system can calibrate the optical scanner 1100 to the platform 100, calibrate the new robotic arm relative to the base portion of the new robotic arm, and calibrate the new robotic arm relative to the platform 100 when the new robotic arm is coupled to the platform 100. For example, based on the telemetry data obtained by the optical scanner 1100, the system can compare the actual real-time movement of the new robotic arm captured by the optical scanner 1100 with the movement expected based on commands sent by the system to the new robotic arm (e.g., executing a pre-programmed routine designed to move the new robotic arm to a specific position) and generate an error measure indicating the deviation between the actual real-time movement of the new robotic arm and the robotic arm movement expected based on the pre-programmed routine. The system can further execute an optimization algorithm to reduce or eliminate the error measure between the actual real-time movement and the expected movement, e.g., until the error measure is below a predetermined threshold. This calibration process can occur when the system is in a pre-defined calibration mode or, optionally, can also occur in real-time during a surgical procedure after the new robotic arm is coupled to the platform 100.

[0177] Based on data obtained by the optical scanner 1100, e.g., in the case of knowing the position and orientation of the operating table relative to the platform 100 and / or the cannula port, the system can, during setup, automatically position the robotic arm 300 in a preferred configuration relative to the patient, based on the surgical procedure to be performed, e.g., in response to an operation performed by the user via the GUI 110, while avoiding collisions between the steps of the platform 300, the robotic arm 300, and objects in the operating room such as the operating table. Thus, the system can store preset robotic arm configurations for various surgical procedures, e.g., based on stored surgeon preferences.

[0178] Now referring to Fig.14 , components that can be included in a co-manipulating robotic platform 1400 are described. The platform 1400 can include one or more processors 1402, communication circuitry 1404, a power supply 1406, a user interface 1408, and / or a memory 1410. One or more electrical components and / or circuits can perform some or all of the roles of the various components described herein. Although described separately, it should be understood that the electrical components need not be separate structural elements. For example, the platform 1400 and the communication circuitry 1404 can be embodied in a single chip. Additionally, although the platform 1400 is described as having a memory 1410, (one or more) memory chips can be provided separately.

[0179] Platform 1400 may include a memory and / or be coupled via one or more buses to read information from or write information to the memory. Memory 1410 may include a processor cache, including a multi-level hierarchical cache where different levels have different capacities and access speeds. The memory may also include random access memory (RAM), other volatile storage devices, or non-volatile storage devices. Memory 1410 may be RAM, ROM, flash memory, other volatile storage devices, or non-volatile storage devices, or other known memories, or some combination thereof, and preferably includes a memory that can selectively store data. For example, the storage device may include, for example, a hard disk actuator, an optical disk, flash memory, and a Zip actuator. Programmable instructions may be stored on memory 1410 to execute an algorithm for, for example, calculating a desired force applied along robotic arm 300 and / or a surgical instrument coupled thereto and applying an impedance at corresponding joints of robotic arm 300 to achieve the desired force.

[0180] Platform 1400 may include a processor 1402, which may be composed of one or more processors and may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any suitable combination thereof designed to perform the functions described herein. Platform 1400 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, the combination of one or more microprocessors and a DSP core, or any other such configuration.

[0181] Platform 1400, in combination with firmware / software stored in the memory, may execute an operating system (e.g., operating system 1446), such as, for example, Windows, Mac OS, QNX, Unix, or Solaris 5.10. Platform 1400 also executes software applications stored in the memory. For example, the software may be a program in any suitable programming language known to those skilled in the art, including, for example, C++, PHP, or Java.

[0182] The communication circuit 1404 may include circuitry that allows the platform 1400 to communicate with an image capture device such as an optical scanner and / or an endoscope. The communication circuit 1404 may be configured for wired and / or wireless communication via a network (such as the Internet, a telephone network, a Bluetooth network, and / or a WiFi network) using techniques known in the art. The communication circuit 1404 may be a communication chip known in the art, such as a Bluetooth chip and / or a WiFi chip. The communication circuit 1404 allows the platform 1400 to locally transmit and / or transmit information such as force measurements on the body wall at the trocar insertion point to a remote location such as a server.

[0183] The power supply 1406 may supply alternating current or direct current. In a DC implementation, the power supply may include a suitable battery, such as a replaceable battery or a rechargeable battery, and the device may include circuitry for charging the rechargeable battery and a detachable power cord. The power supply 1406 may be a port to allow the platform 1400 to be plugged into a conventional wall outlet, for example via a wire having an AC to DC power converter and / or a USB port, to power components within the platform 1400. The power supply 1406 may be operatively coupled to an emergency switch such that upon actuation of the emergency switch, power is stopped being supplied to components within the platform 1400, including, for example, a braking mechanism placed on at least some of the multiple joints of the robotic arm 300. For example, the braking mechanism may require power to disengage such that in the absence of power being supplied to the braking mechanism, the braking mechanism engages to prevent the robotic arm 300 from moving in the absence of power.

[0184] The user interface 1408 may be used to receive input from the user and / or provide output to the user. For example, the user interface 1408 may include a touch screen, a display, switches, dials, lights, etc. Thus, the user interface 1408 may display information such as the identity of the selected surgical instrument and force measurements observed during the operation of the robotic arm 300. Additionally, the user interface 1408 may receive user input, including an adjustment of a predetermined amount of movement or a predetermined dwell period at the handle of the surgical instrument that causes the robotic arm to automatically switch to a passive mode, a predetermined threshold of force applied at the handle of the surgical instrument that causes the robotic arm to automatically switch to a co-manipulation mode, the position of a predefined tactile barrier, the identity of the surgical instrument coupled to the distal end of the robotic arm, the vertical height of the robotic arm, the horizontal position of the robotic arm, etc., such that the platform 1400 can adjust the information / parameters accordingly. In some embodiments, the user interface 1408 does not exist on the platform 1400 but is instead provided on a remote external computing device communicatively connected to the platform 1400 via the communication circuit 1404.

[0185] Memory 1410 (which is an example of a non-transitory computer-readable medium) can be used to store an operating system (OS) 1446, a surgical instrument identification module 1412, a surgical instrument calibration module 1414, an encoder interface module 1416, a robotic arm position determination module 1418, a trocar position detection module 1420, a force detection module 1422, an impedance calculation module 1424, a motor interface module 1426, an optical scanner interface module 1428, a gesture detection module 1430, a passive mode determination module 1432, a co-manipulation mode determination module 1434, a haptic mode determination module 1436, a robotic assistance mode determination module 1438, a fault detection module 1440, an indicator interface module 1442, and a fatigue detection module 1444. The modules are provided in the form of computer-executable instructions / algorithms that can be executed by the processor 1402 to perform various operations in accordance with the present disclosure.

[0186] For example, during a procedure, the system can continuously run the algorithms described herein based on data collected by the system. The data can be collected and / or recorded using any of the components and methods disclosed herein, including, for example, from sensors / encoders within the robot, from optical scanning devices that communicate with other components of the robotic system, and / or from manual input of the system's operator. Thus, the algorithms, data, and configuration of the system can enable a user to co-manipulate the robotic arm with minimal impact and influence from the weight, gravity, and other forces that a traditional robotic arm cannot compensate for, of the robotic arm and / or the surgical instrument coupled thereto. Some parameters of the algorithms described herein can control aspects of the behavior of the system, including, for example, the robustness of the detected features, the sensitivity to false positives, the robotic control gain, the number of features to track, the dead zone radius, and the like.

[0187] The surgical instrument identification module 1412 can be executed by the processor 1402 to identify surgical instruments coupled to each robotic arm and load an appropriate calibration file into the controller system. For example, the calibration file for each surgical instrument can be stored in a database accessible to the surgical instrument identification module 1412 and can include information associated with the surgical instrument, such as, for example, instrument type, weight, center of mass, length, instrument shaft diameter, etc. Thus, when the appropriate calibration file is loaded and the associated surgical instrument is coupled to the robotic arm 300, the system will automatically account for the mass of the surgical instrument. For example, when attaching the surgical instrument to the robotic arm 300 based on the data in the calibration file, the gravity on the surgical instrument is compensated such that after the surgical instrument is coupled to the robotic arm and the operator releases the surgical instrument, the robotic arm 300 can hold the surgical instrument in place. For example, the surgical instrument identification module 1412 can identify the surgical instrument based on user input via the user interface 1408. For example, the operator can select the surgical instrument from a database of surgical instruments stored in the memory 1410.

[0188] In some embodiments, the surgical instrument identification module 1412 can automatically identify a surgical instrument coupled to the robotic arm via a coupler body and a coupler interface using, for example, an RFID transmitter chip and reader or receiver (e.g., placing an RFID sticker or transmitter that can transmit information about the surgical instrument to the system on the surgical instrument), a near field communication (“NFC”) device (such as a near field magnetic induction communication device), a barcode and scanner or other optical device, a magnet-based communication system, a reed switch, a Bluetooth transmitter, the weight of the instrument and / or data collected from an optical scanner and a look-up table, and / or any other feature or mechanism described herein or suitable for identifying a surgical instrument. As described above, the coupler body can be selected based on the size and shape of the lumen extending therethrough to accommodate and engage a surgical instrument having a known elongated shaft diameter. Thus, the surgical instrument identification module 1412 can automatically identify the surgical instrument based on the coupler body coupled to the surgical instrument via a magnetic connection between the coupler body and the coupler interface.

[0189] In some embodiments, for example, when using the following with respect to Figures 44A-50BWhen describing the coupling mechanism 4400, the surgical instrument identification module 1412 can identify a surgical instrument based on data obtained from a sensor. For example, the type of the surgical instrument, such as a Hall effect sensor within the distal wrist link 316, which indicates the magnetic field strength induced by a magnet displaced in response to the coupling of the surgical instrument to a coupling body coupled to the coupler interface. Based on the detected magnetic field strength, the system can determine whether the coupling body is coupled to the coupler interface, whether the surgical instrument is coupled to the coupling body, and the diameter of the shaft of the surgical instrument, such as a 5 mm or 10 mm surgical instrument, as described in further detail below.

[0190] The surgical instrument identification module 1412 can further cause the distal end of the robotic arm 300 to move in a predetermined pattern when coupled to a laparoscope. For example, it moves in a circular motion in a plane perpendicular to the longitudinal axis of the laparoscope, and compares the image data captured when the distal end of the laparoscope moves in the predetermined circular pattern with the image data obtained by the optical scanner 1100 of the laparoscope when it moves in the predetermined pattern to identify whether the laparoscope is a flat-head laparoscope or a corner-head laparoscope as described above.

[0191] In some embodiments, the surgical instrument identification module 1412 can identify a surgical instrument, such as the type of the surgical instrument, based on data obtained by the optical scanner 1100 via the optical scanner interface module 1428 described in further detail below. For example, the data can include measurement data associated with a specific instrument, such that the surgical instrument identification module 1412 can compare such data with the information contained in a database to identify the instrument and load an appropriate calibration file into the controller system. Similarly, the surgical instrument identification module 1412 can detect whether the instrument is removed and return the calibration parameters to the default configuration.

[0192] The surgical instrument calibration module 1414 can be executed by the processor 1402 to calibrate a surgical instrument, such as a surgical instrument that currently does not have an associated calibration file in the database stored in the memory 1410. Therefore, the surgical instrument calibration module 1414 can calculate the measurements and specifications of the surgical instrument when the surgical instrument is coupled to the robotic arm 300 and the system is in the calibration mode based on the force measurements of the robotic arm 300 exerted by the surgical instrument via the force detection module 1422, as described below with respect to Fig.16This will be described in further detail. For example, the surgical instrument calibration module 1414 can generate a calibration file for the surgical instrument, and the calibration file includes information such as instrument type, weight, center of mass, length, instrument shaft diameter, viscosity parameters of the surgical instrument, and the like. At least some of the surgical instrument information in the calibration file can be provided by the user input via the user interface 1408, such as the instrument type, or can be detected by the optical scanner interface module 1428, such as the instrument type, the center of mass of the instrument, the instrument length, and the instrument diameter.

[0193] If the surgical instrument calibration module 1414 determines that the recalibration result is always different from the configuration that has been loaded into the system, the surgical instrument calibration module 1414 can replace the existing information or add it to its known tool list without any user input and automatically load them. For example, if the robotic arm moves only due to the gravity acting on the robotic arm and / or the surgical instrument when the surgical instrument is coupled to the robotic arm, the surgical instrument calibration module 1414 can determine that the calibration factor is insufficient to compensate for the gravity, which can be done when the surgical instrument is fully positioned outside the patient's body. In addition, if it is determined that the calibration factor is insufficient to compensate for the gravity, the surgical instrument calibration module 1414 can automatically update or adjust the calibration factor (e.g., the force applied to the joints of the robotic arm). Thus, the surgical instrument calibration module 1414 can update the calibration factor for a specific surgical instrument and store the updated calibration factor for the specific surgical instrument in the associated calibration file for future use.

[0194] The encoder interface module 1416 can be executed by the processor 1402 to receive and process in real time the angular measurement data from multiple encoders (e.g., encoders E1 - E7) of the robotic arm 300. For example, the encoder interface module 1416 can calculate the change in the angle of the link of the robotic arm 300 rotatably coupled to a given joint associated with the encoder over time. As described above, the system can include redundant encoders at each joint of the robotic arm 300 to ensure the safe operation of the robotic arm 300. In addition, additional encoders can be placed on the platform 100 to measure the angle / position of each robotic arm relative to the platform 100, such as the vertical and horizontal positions of the robotic arm relative to the platform 100. Thus, one encoder can be placed on the platform 100 to measure the movement of the robotic arm along the vertical axis of the platform 100, and another encoder can be placed on the platform 100 to measure the movement of the robotic arm along the horizontal axis of the platform 100.

[0195] The robotic arm position determination module 1418 can be executed by the processor 1402 to determine the position of the robotic arm 300 and any attached surgical instruments (if any) in 3D space in real time based on the angular measurement data generated by the encoder interface module 1416. For example, the robotic arm position determination module 1418 can determine the positions of the various links and joints of the robotic arm 300 as well as the position along the surgical instrument coupled to the robotic arm 300. Based on the position data of the robotic arm 300 and / or the surgical instrument, the robotic arm position determination module 1418 can compute the speed and / or acceleration of the movement of the robotic arm 300 and any attached surgical instruments in real time. For example, by determining the respective speeds of the various joints of the robotic arm 300 via encoders associated with each of the various joints, the robotic arm position determination module 1418 can determine the resultant speed of the distal end of the robotic arm 300, which can be used by the passive mode determination module 1432 to determine whether the movement of the distal end of the robotic arm 300 is within a predetermined threshold for the purpose of transitioning the system 200 to the passive mode, as described in further detail below.

[0196] The trocar position detection module 1420 can be executed by the processor 1402 to determine the position and / or orientation of one or more trocar ports inserted into a patient. The position and / or orientation of the trocar port can be derived based on data obtained from, for example, an inertial measurement unit and / or an accelerometer, an optical scanner, an electromechanical tracking instrument, a linear encoder, a sensor, and the data described above. For example, the position of the trocar port on the patient can be determined using a laser pointing system that can be mounted on one or more of the components of the system (e.g., the wrist portion 311 of the robotic arm) and can be controlled by the system to point to an optimal or determined position on the patient's body for inserting the trocar. Additionally, when inserting a surgical instrument attached to the robotic arm 300 through the trocar, a virtual line can be continuously established along the longitudinal axis of the surgical instrument, and when the surgical instrument moves around the trocar point, the alignment / orientation of the virtual line can be automatically determined in real time when the surgical instrument is attached to the coupler interface 400 via the magnetic connection described above. Further, when the surgical instrument is inserted into the trocar port, it will point to the trocar point, and thus, the distal wrist link 316 will also point to the trocar point, and its angle can be measured by an encoder associated therewith. Thus, the trocar point can be calculated as the intersection of a plurality of virtual lines continuously established along the longitudinal axis of the surgical instrument. In this way, when manipulating the surgical instrument around the trocar port (e.g., rotating or moving into or out of the patient), the calculated trocar point will remain fixed relative to the patient. Additionally, the orientation of the trocar port and its position relative to the robotic arm 300 can also be determined based on image data received from one or more optical scanners (e.g., a LiDAR camera and / or an RGBD camera).

[0197] Based on the known position and / or orientation of the trocar port in addition to the known position of the distal end of the robotic arm 300 from the robotic arm position determination module 1418, the system can maintain the position of the distal end of the robotic arm 300 relative to the trocar point when the robotic arm 300 moves (e.g., vertically or horizontally adjusted by the platform 100) or when the height of the patient worktable is adjusted, thus causing the height of the patient's abdomen to move, so as to keep the surgical instrument in the patient's body and stably coupled to the robotic arm 300 during these external movements. To achieve this, by adding the position of the platform 100 to the kinematic calculation (e.g., the "forward kinematics" of the robotic arm 300 in the context of a serial-link robotic manipulator), the known position of the distal end of the robotic arm 300 from the robotic arm position determination module 1418 is calculated in the global frame of the system. With the position of the distal end of the robotic arm 300 globally known, the system can keep this position stable by applying appropriate forces to the robotic arm 300 during external movements that minimize the error between its current position and the desired position.

[0198] With the position of the distal end of the robotic arm 300 globally known, the system can keep this position stable by applying appropriate forces to the robotic arm 300 during external movements that minimize the error between its current position and the desired position. Thus, for example, when a surgical instrument coupled to the distal end of the robotic arm 300 is inserted through the trocar port, with the instrument tip in the patient's body and the user adjusts the height of the patient table, the system can apply forces / torques to the robotic arm 300 to reconfigure the robotic arm 300 and / or move the respective steps of the platform 100, so as to maintain the relative position between the distal end of the robotic arm 300 and thus the surgical instrument and the trocar port. In some embodiments, the system can slightly retract the distal end of the robotic arm 300, so as to position the tip of the surgical instrument within the trocar port and keep it from contacting the anatomical structures in the patient's body before reconfiguring the robotic arm 300 to maintain the relative position between the surgical instrument and the trocar port.

[0199] The force detection module 1422 can be executed by the processor 1402 to detect forces applied to the robotic arm 300 (e.g., at the joints or links of the robotic arm 300 or along the surgical instrument) and forces applied to the trocar, such as body wall forces. For example, the force detection module 1422 can receive motor current measurements in real time at each motor (e.g., M1, M2, M3) placed within the base of the robotic arm 300, each motor being operatively coupled to a joint of the robotic arm 300, such as the base joint 303, shoulder joint 318, elbow joint 322, wrist joint 332. The motor current measurements indicate the amount of force applied to the associated joint. Thus, the forces applied to each joint of the robotic arm 300 and the surgical instrument attached thereto can be calculated based on the motor current measurement results and the position data generated by the robotic arm position determination module 1418 and / or the trocar position detection module 1420.

[0200] Due to the passive axes at the distal end of the robotic arm 300, the force applied to the trocar by the instrument coupled to the robotic arm can remain substantially consistent throughout the workspace of the robotic arm. The force on the trocar can be affected by the interaction of the distal tip of the instrument with the tissue within the body. For example, if a tissue retractor advanced through the trocar engages (e.g., grasps) body tissue or another object within the body, the force applied to the end of the instrument from the body tissue or other object may cause a change in the force applied to the trocar. In some aspects, the force on the trocar can be a function of how much weight is lifted by the instrument being used.

[0201] The impedance calculation module 1424 can be executed by the processor 1402 to determine the amount of impedance / torque that needs to be applied to the corresponding joints of the robotic arm 300 to achieve a desired effect (e.g., holding the robotic arm 300 in a static position in passive mode, allowing the robotic arm 300 to move freely while compensating for the gravity of the robotic arm and the surgical instrument attached thereto in co-manipulation mode, applying increased impedance to the robotic arm 300 when the robotic arm 300 and / or the surgical instrument attached thereto is within a predefined virtual haptic barrier in haptic mode, etc.).

[0202] For example, the impedance calculation module 1424 can determine the amount of force required for the robotic arm 300 to achieve the desired effect based on the position data of the robotic arm 300 generated by the robotic arm position determination module 1418 and the position data of the trocar generated by the trocar position detection module 1420. For example, by determining the position of the distal end of the robotic arm 300 and the entry point of the surgical instrument into the patient (e.g., the trocar position), and knowing one or more instrument parameters (e.g., the mass and center of mass of the surgical instrument stored by the surgical instrument calibration module 1414), the impedance calculation module 1424 can calculate the amount of force (compensation force) required to compensate for the gravity of the surgical instrument, as described in further detail below with respect to Fig.18A Accordingly, the amount of compensation force required to compensate for the gravity of the surgical instrument can be converted into the torque to be applied at the joints of the robotic arm 300, for example, by motors operatively coupled to the joints of the robotic arm 300, as indicated by motor current measurements.

[0203] In addition, by determining the position of the distal end of the robotic arm 300 and thus determining the change over time in the position of the distal end of the robotic arm 300, for example, due to an external force applied to the distal end of the robotic arm 300 by tissue held by the operating end of the surgical instrument, and knowing one or more instrument parameters (e.g., the mass, center of mass, and length of the surgical instrument stored by the surgical instrument calibration module 1414), the impedance calculation module 1424 can calculate the amount of force (holding force) required to maintain the surgical instrument in a static position, as described in further detail below with respect to Fig.18B Accordingly, in addition to the amount of compensation force required to compensate for the gravity of the surgical instrument, the amount of holding force required to resist changes in the position of the distal end of the robotic arm 300 can be converted into the torque to be applied at the joints of the robotic arm 300, for example, by motors operatively coupled to the joints of the robotic arm 300, to maintain the robotic arm 300 in a static position, as indicated by motor current measurements. Additionally, the impedance calculation module 1424 and / or the force detection module 1422 can calculate the amount of force applied by the surgical instrument to the patient at the entry point (e.g., at the trocar) and the amount of force applied to the operating end (e.g., the gripper end of the surgical instrument) of the surgical instrument based on the compensation force, the holding force, one or more parameters of the surgical instrument (such as the mass, center of mass, and length of the surgical instrument), and the distance from the center of mass to the entry point.

[0204] Additionally or alternatively, by determining the force applied to the robotic arm 300 via the force detection module 1422 and the position / velocity / acceleration of the distal end of the robotic arm 300 in 3D space via the robotic arm position determination module 1418, the desired force / impedance to be applied to the robotic arm 300 to compensate for the applied force can be calculated, e.g., for gravity compensation or to hold the robotic arm 300 in a static position in passive mode. Thus, the desired force can be converted, e.g., by motors operatively coupled to the joints of the robotic arm 300, into the torque to be applied at the joints of the robotic arm 300. For example, the robotic Jacobian matrix can be used for this purpose. The Jacobian matrix is a matrix calculated at each given strut of the robotic arm and relates the velocity at the joints to the velocity at the distal end of the robotic arm 300:

[0205] V = J * q dot

[0206] Here, V is the velocity vector at the distal end of the robotic arm 300, J is its Jacobian matrix, and q dot is its joint velocity represented in vector form. Using the energy principle and assuming that the mass of the links of the robotic arm 300 is negligible and the friction / damping is negligible, the power of the system can be determined by multiplying its force and velocity:

[0207] F · V = τ · q dot

[0208] =>

[0209] F · (J * q dot ) = τ · q dot

[0210] Here, F is the generalized force vector at the distal end of the robot 300. Additionally, vector manipulation results in:

[0211] (J t * F) · q dot = τ · q dot

[0212] =>

[0213] τ = J t * F

[0214] Here, t represents the transpose of the matrix, such that the force at the distal end of the robotic arm 300 can be converted into the torque to be applied at the joints using the Jacobian matrix.

[0215] The motor interface module 1426 can be executed by the processor 1402 to receive motor current readings at each motor (e.g., M1, M2, M3) placed within the base of the robotic arm 300 and to actuate the corresponding motor, e.g., by applying a predetermined impedance to achieve the desired results as described herein and / or to move the joints operatively coupled to the corresponding motor, such as in a robotic assist mode. In some embodiments, when the joint 320 is operatively coupled to a motor (e.g., M4) such that the distal shoulder link 308 can rotate automatically relative to the proximal shoulder link 306, as further described with respect to Fig.56 the motor interface module 1426 can actuate M4 to cause the distal shoulder link 308 to rotate relative to the proximal shoulder link 306.

[0216] The optical scanner interface module 1428 can be executed by the processor 1402 to receive depth data obtained by the optical scanner 1100 and process the depth data to detect predefined conditions therein. Additionally, the optical scanner interface module 1428 can generate a depth map indicative of the received depth data, which can be displayed to an operator via a monitor, for example. For instance, the optical scanner interface module 1428 can map the orientation of the trocar port in 3D space such that the mapping of the trocar port can be communicated to the operator via a display or the user interface 1408, for example. Based on the depth data and / or color data received from the optical scanner 1100, the optical scanner interface module 1428 can determine the relative distances, for example, between the steps of the platform 100, the robotic arm 300, any surgical instruments attached thereto, and the objects / people in the operating room (e.g., the operating table, drapes, etc.).

[0217] The optical scanner interface module 1428 can also receive image data from additional optical scanning devices as defined herein, including, for example, an endoscope operatively coupled to the system. Additionally, the optical scanner interface module 1428 can receive depth data obtained by the optical scanners 1100a, 1100b, 1100c coupled to the platform 100 and process the depth data to generate a virtual map of the area around the platform 100, as described above with respect to Fig. 11C which can be displayed to the operator via a monitor (e.g., the display 110). For example, the optical scanner interface module 1428 can generate a graphical representation of the system 200 including the platform 100 and the robotic arms 300a, 300b and any objects and / or people within the area around the platform 100 for display in the virtual map.

[0218] The gesture detection module 1430 can be executed by the processor 1402 to detect predefined pose patterns as user inputs and perform actions associated with the user inputs. The predefined pose patterns can include, for example, the movement of a surgical instrument (whether or not attached to the robotic arm 300), the movement of the robotic arm 300 or other components of the system (e.g., foot pedals, buttons, etc.), and / or the movement of the operator in a predefined pattern. For example, the back-and-forth movement of a surgical instrument in a first direction (e.g., left / right, up / down, forward / backward, in a circle) can be associated with a first user input that requires the system to perform a first action, and / or the back-and-forth movement in a second direction different from the first direction (e.g., left / right, up / down, forward / backward, in a circle) can be associated with a second user input that requires the system to perform a second action. Similarly, pressing a foot pedal or button operatively coupled to the system in a predefined manner can be associated with a third user input that requires the system to perform a third action, and the repeated back-and-forth or up-and-down movement of the operator's head can be associated with a fourth user input that requires the system to perform a fourth action. The various predefined pose patterns associated with different components of the system or the operator can be redundant such that the associated user inputs can be the same for different pose patterns. The predefined pose patterns can be detected by, for example, an optical scanning device (such as a laparoscope or the optical scanner 1100) via the optical scanner interface module 1428, or directly by the force applied to the robotic arm 300 via the force detection module 1422 or other components of the system.

[0219] Actions in response to user inputs associated with predefined pose patterns can include, for example, enabling tool tracking to servo (i.e., move) a laparoscope based on the movement of a handheld tool; engaging a brake on the robotic arm (e.g., to prevent further movement of the robotic arm); engaging a software lock on the robotic arm; dynamically changing the length of time it takes for the robotic arm to transition from a default placement between states; loading a virtual menu overlay on the video feed such that a surgical instrument in the laparoscope field of view can be used as a pointer to trigger further operations in the virtual menu; and / or identifying which member of the surgical staff is touching the robotic arm (if any). This information can be used to ensure that the system does not move if the surgeon is not in contact with the robotic arm, for example, to avoid a situation where an external force acts on the robotic arm (e.g., an optical cable or other line is pulled across the robotic arm) and the system senses an intentional force from the surgeon. The same information can be used to detect the direction of the surgeon's gaze, e.g., whether the surgeon is looking at the video feed or somewhere else in the room, such that the system can freeze the robotic arm if the surgeon's gaze is not in the direction it should be. Additionally, the system can reposition the field of view of the camera based on, for example, the direction the surgeon is facing, or based on the object the surgeon appears to be looking at, based on data from the optical scanner 1100.

[0220] As described above, in response to the user's detection of a predefined pose pattern, e.g., a predefined pattern of the distal end of a surgical instrument within the laparoscopic field of view, the gesture detection module 1430 can cause a virtual menu to be superimposed on the video feed such that the surgical instrument within the laparoscopic field of view acts as a pointer, as shown in FIG. 52. Additionally, the gesture detection module 1430 can detect further predefined movement patterns of the distal end of the surgical instrument, e.g., two quick movements in the same direction or a circular movement over a selection area of the virtual menu, which can be interpreted as a selection execution, e.g., a click on the virtual menu. For example, as shown in FIG. 52, the virtual menu superimposition on the video feed can include menu options in the corners of the video feed, e.g., "hot corners", such as: turn on / off the assisted aiming mode where the system automatically moves the robotic arm coupled to the laparoscope to follow the surgical instrument and / or zooms in or out to change the field of view of the laparoscope; adjust the holding force of the robotic arm coupled to the retractor, e.g., the magnitude of the force that can be applied to the distal end of the surgical instrument before the system switches from the passive mode to the co-manipulation mode; turn on / off audio; and turn on / off haptic feedback. As will be understood by those of ordinary skill in the art, more or fewer menu options can be provided via the virtual menu. In some embodiments, the activation of the virtual menu superimposition display on the video feed can be triggered by, for example, the actuation of an external actuator such as a foot pedal, a predefined force pattern applied to the robotic arm (e.g., double-clicking on the wrist portion 311 and / or the surgical instrument coupled to the robotic arm detected by the distal encoder of the robotic arm), voice activation, a wireless button, a hot button, etc.

[0221] In some embodiments, the operator can actively switch the system to the command mode, e.g., via the user interface 1408, where specific movements or gestures of the robotic arm, the surgical instrument, the operator, or other means as described herein are monitored by the gesture detection module 1430 to determine if they are consistent with a predefined pose pattern associated with a predefined user input.

[0222] The passive mode determination module 1432 can be executed by the processor 1402 to analyze the operating characteristics of the robotic arm 300 to determine whether to switch the operating mode of the robotic arm 300 to the passive mode, where the system applies an impedance to the joints of the robotic arm 300 via the motor interface module 1426 in an amount sufficient to hold the robotic arm 300 and thus any surgical instrument (if any) attached thereto in a static position, thereby compensating for the mass of the robotic arm 300 and the surgical instrument, as well as any other external forces acting on the robotic arm 300 and / or the surgical instrument. If the robotic arm 300 moves slightly in the passive mode but does not have sufficient force to switch out of the passive mode, the system can adjust the amount of impedance applied to the robotic arm 300 to maintain the static position and continue this process until the robotic arm 300 is held in the static position. For example, if the movement of the robotic arm due to movement at the handle of the surgical instrument determined by the force detection module 1422 is less than a predetermined amount (e.g., not exceeding 1 mm to 5 mm) for at least a predetermined dwell period associated with the robotic arm 300, the passive mode determination module 1432 can determine to switch the operating mode of the robotic arm 300 to the passive mode. The predetermined dwell period refers to the length of time that the robotic arm 300 and / or any surgical instrument (if any) attached thereto remains in the static position. For example, the predetermined dwell time can be in the range of, for example, between 0.1 second and 3 seconds or longer and can be adjusted by the operator. Fig.19 A table or exemplary values of threshold dwell times for a series of sample instrument types are illustrated.

[0223] In some embodiments, if the movement of the distal end of the robotic arm due to movement at the handle of the surgical instrument determined by the force detection module 1422 has a speed less than a predetermined dwell speed / rate, the passive mode determination module 1432 can determine to switch the operating mode of the robotic arm 300 to the passive mode. For example, if the passive mode determination module 1432 determines that the distal end of the robotic arm 300 and / or any surgical instrument (if any) attached thereto moves at a rate lower than the predetermined dwell speed during the entire predetermined dwell period, the passive mode determination module 1432 can switch the operating mode of the robotic arm 300 to the passive mode. Fig.19Illustrated is a table or exemplary values of threshold dwell rates for a series of sample instrument types. For example, for surgical instruments such as observation instruments and tissue manipulation devices, the threshold dwell rate can be, for example, 3 - 5 mm / s, and for surgical instruments such as suture instruments, needle actuators, high - force instruments, staplers, and clip appliers, the threshold dwell rate can be, for example, 1 - 2 mm / s. In some embodiments, the passive - mode determination module 1432 can determine to switch the operating mode of the robotic arm 300 to the passive mode based on the identity of the surgical instrument when the surgical instrument is attached to the robotic arm 300 and / or when the surgical instrument is responsive to separation from the robotic arm 300.

[0224] The co - manipulation - mode determination module 1434 can be executed by the processor 1402 to analyze the operating characteristics of the robotic arm 300 to determine whether to switch the operating mode of the robotic arm 300 to the co - manipulation mode, where the robotic arm 300 is allowed to move freely in response to movement at the handle of the surgical instrument to perform laparoscopic surgery using the surgical instrument, while the system applies an impedance to the robotic arm 300 via the motor interface module 1426 in an amount sufficient to account for the mass of the surgical instrument and the robotic arm 300. Additionally, the impedance applied to the robotic arm 300 can provide a predetermined level of viscosity that is perceivable by the operator. Fig.19 Illustrated is a table or exemplary values of viscosity levels for a series of sample instrument types. In some embodiments, the viscosity level can be a function of the rate at which the surgical instrument moves and the distance of the tip of the instrument from the trocar point. For example, if the force applied at the robotic arm 300 due to a force applied at the handle of the surgical instrument exceeds a predetermined threshold associated with the robotic arm 300 (e.g., a "separation force"), then the co - manipulation - mode determination module 1434 can determine to switch the operating mode of the robotic arm 300 to the co - manipulation mode. The predetermined force threshold can be, for example, at least 7 Newtons, approximately 7 Newtons, at least 7 Newtons, 4 - 15 Newtons, 4 - 10 Newtons. The predefined force threshold can depend on the type of surgical instrument being used and / or the presence of an external force applied to the surgical instrument.

[0225] Fig.19 Illustrated is a table or exemplary values of predefined force thresholds for a series of sample instrument types. As Fig.19As shown, the predefined force threshold can reflect the typical external tissue forces that can be applied to a surgical instrument. In some embodiments, if a force is applied to the surgical instrument by tissue or an organ or otherwise, the predefined force threshold can be increased, depending on the direction of the separation force. For example, if the separation force is in the same direction as the force applied to the surgical instrument by the tissue or organ, the predefined force threshold can be increased by an amount equal to or commensurate with the force applied to the surgical instrument by the tissue or organ. In some embodiments, the predefined force threshold of the corresponding robotic arm is adjusted based on the patient's body mass index (“BMI”). For example, a patient with a higher BMI may have a heavier liver, which may apply a greater force on the instrument. Thus, for a patient with a higher BMI, the predefined force threshold can be selected to be higher. Thus, the operation can, for example, actuate a “high force mode” via the user interface 1408, where the predefined force threshold is increased to accommodate engagement with heavier tissue or organs. For example, the predefined force threshold can be selectively increased by 20%-100% or more.

[0226] In addition, the force applied by the user to the surgical instrument and any external tissue forces applied to the surgical instrument can be direction-dependent. For example, if the force applied by the user to the surgical instrument is in the same direction as the external tissue force applied to the surgical instrument, the two forces can be additive, such that the amount of force applied by the user to the surgical instrument required to overcome the predefined force threshold can reduce the magnitude of the external tissue force, such that a lower force than the predefined force threshold will be required to exit the passive mode and enter the co-manipulation mode. On the other hand, if the force applied by the user to the surgical instrument is in the direction opposite to the external tissue force applied to the surgical instrument, the necessary amount of force applied by the user to the surgical instrument required to overcome the predefined force threshold can increase the magnitude of the external tissue force, such that a higher force than the predefined force threshold will be required to exit the passive mode and enter the co-manipulation mode.

[0227] Additionally, if the force applied by the user on the surgical instrument is in a direction perpendicular to the external tissue force applied to the surgical instrument, the necessary amount of force applied by the user on the surgical instrument to overcome the predefined force threshold may not be affected by the magnitude of the external tissue force, such that the force applied by the user on the surgical instrument required to exit the passive mode and enter the co-manipulation mode will be equal to the predefined force threshold. For other directions, the force vector of the applied force may be added to the force vector of the external tissue force or offset by the force vector of the external tissue force to overcome the predefined force threshold of the system coupled to the robotic arm or a particular surgical instrument, depending on the direction of the external tissue force (if any) and the force applied by the user. In some embodiments, the co-manipulation mode determination module 1434 may determine to switch the operation mode of the robotic arm 300 to the co-manipulation mode based on the identity of the surgical instrument.

[0228] The haptic mode determination module 1436 can be executed by the processor 1402 to analyze the operating characteristics of the robotic arm 300 to determine whether to switch the operating mode of the robotic arm 300 to the haptic mode, where the system applies an impedance to the robotic arm 300 via the motor interface module 1426 in an amount higher than that applied in the co-manipulation mode, so that the movement of the robotic arm 300 in response to movement at the handle of the surgical instrument is more viscous in the co-manipulation mode. For example, if at least a portion of the robotic arm 300 and / or the surgical instrument attached thereto is within a predefined virtual haptic boundary, the haptic mode determination module 1436 can determine to switch the operating mode of the robotic arm 300 to the haptic mode. Specifically, a virtual haptic boundary can be established by the system such that the robotic arm or the surgical instrument coupled thereto should not breach the boundary. For example, a virtual boundary can be established at the surface of the patient to prevent any part of the robotic arm or the instrument supported by the robotic arm from contacting the patient, except through one or more trocars. Similarly, the virtual haptic boundary can include a haptic funnel to assist in guiding the instrument into the patient's body when the operator inserts the instrument into the trocar port. Thus, based on the position data of the robotic arm 300 and / or the surgical instrument coupled thereto received, for example, by the robotic arm position determination module 1418 and / or the trocar position detection module 1420, the haptic mode determination module 1436 can determine whether the robotic arm 300 and / or the surgical instrument is within the predefined virtual haptic boundary, and thus transition the robotic arm 300 to the haptic mode, where the processor 1402 can instruct the associated motor to apply an effective amount of impedance to the joints of the robotic arm 300 perceptible to the operator to convey the virtual haptic boundary to the operator. Thus, the viscosity of the robotic arm 300 observed by the operator will be much higher than in the co-manipulation mode. In some embodiments, the haptic mode determination module 1436 can determine to switch the operating mode of the robotic arm 300 to the haptic mode based on the identity of the surgical instrument.

[0229] The robotic assistance mode determination module 1438 can be executed by the processor 1402 to analyze the operating characteristics of the robotic arm 300 to determine whether to switch the operating mode of the robotic arm 300 to the robotic assistance mode, where the processor 1402 can instruct the associated motor via the motor interface module 1426 to cause movement of the corresponding links and joints of the robotic arm 300 to achieve a desired result. For example, if a predetermined condition exists based on data obtained from, for example, the optical scanner interface module 1428, the robotic assistance mode determination module 1438 can determine to switch the operating mode of the robotic arm 300 to the robotic assistance mode.

[0230] For example, the robotic assistance mode determination module 1438 may determine the presence of conditions based on image data obtained via the optical scanner interface module 1428 from the laparoscope or the optical scanner 1100. For example, the field of view of the laparoscope coupled to the robotic arm 300 or the optical scanner 1100 is not optimal for a given surgical procedure, e.g., due to obstruction by the surgeon or an assistant or another component of the system, such that the robotic arm coupled to the laparoscope or the optical scanner 1100 should be repositioned or zoomed in / out to optimize the field of view of the surgical site for the operator. Thus, in the robotic assistance mode, the processor 1402 may automatically / quasi-automatically or in response to a user input from the operator instruct the robotic arm 300 to move to reposition the laparoscope and / or zoom the laparoscope in or out, or increase the resolution of the image, or otherwise. For example, the user input from the operator may be determined by the gesture detection module 1430 as described above, such that movement of the robotic arm or the surgical instrument in a first direction in a predefined pose pattern causes the endoscope to increase the resolution or magnification, and movement in a second direction causes the endoscope to decrease the resolution or magnification, and movement in another predefined pose pattern causes the robotic arm holding the laparoscope to retract away from the patient's body.

[0231] Additionally, the robotic assistance mode determination module 1438 may determine the presence of conditions, e.g., one or more trocars are not in an optimal position, e.g., due to patient movement, such that the robotic arm 300 should be repositioned to keep the trocars in an optimal position, e.g., at approximately the center of the movement range of the robotic arm 300, thereby minimizing the risk of reaching the joint limits of the robotic arm during the procedure. Thus, in the robotic assistance mode, the processor 1402 may instruct the system to reposition the robotic arm 300, e.g., via vertical / horizontal adjustment of the platform 100 or via the joints and linkages of the robotic arm 300, to better align the surgical instrument workspace.

[0232] The robotic assistance mode determination module 1438 may determine the presence of conditions based on image data obtained via the optical scanner interface module 1428 from the laparoscope or the optical scanner 1100. For example, the distance between an object and the robotic arm 300 is within a predetermined threshold, such that the robotic arm should be frozen to avoid collision with the object. Thus, in the robotic assistance mode, the processor 1402 may instruct the robotic arm 300 to apply brakes to slow down the robotic arm or inhibit or prevent movement within a predetermined distance from another object.

[0233] The robot-assisted mode determination module 1438 can further determine that a situation exists, for example, where the robotic arm 300 is in an extended position for a time period exceeding a predetermined threshold during a surgical procedure, such that the robotic arm should be repositioned to provide more available workspace for the user near the surgical instrument coupled to the extended robotic arm. Thus, in the robot-assisted mode, the processor 1402 can instruct the system to reposition the robotic arm 300, for example, by vertical / horizontal adjustment of the platform 100 and / or by the joints and linkages of the robotic arm 300, to move the robotic arm 300 closer to the surgical instrument.

[0234] The fault detection module 1440 can be executed by the processor 1402 to analyze data indicative of the operating characteristics of the system (e.g., position data generated by the robotic arm position determination module 1418 and / or the trocar position detection module 1420 and / or force measurements calculated by the force detection module 1422) to detect whether a fault condition exists. For example, the fault detection module 1440 can detect a fault condition of the system and determine whether the fault condition is a "minor fault", a "major fault", or a "critical fault", where each category of fault condition can be cleared in different predefined ways.

[0235] For example, the fault detection module 1440 may detect a minor fault condition, such as the robotic arm 300 moving at a speed exceeding a predetermined speed threshold, which may be cleared, for example, by slowing down the movement of the robotic arm 300. In some embodiments, when the robotic arm 300 is moving too fast, the system may automatically apply additional impedance to the robotic arm 300, thereby forcing the operator to slow down the movement of the robotic arm 300. Additionally, the fault detection module 1440 may detect a major fault condition, such as an unintentional collision of the robotic arm 300, as indicated by a large force applied to the robotic arm 300 by someone other than the operator. In response to detecting the major fault condition, the fault detection module 1440 may actuate the braking mechanism associated with each motorized joint of the robotic arm 300 (or at least the joints associated with the major fault condition), thereby freezing the robotic arm 300 and inhibiting further movement of the robotic arm 300. Such a major fault condition may be cleared by the operator actuating a "clear" option displayed on the user interface 1408. The fault detection module 1440 may detect a critical fault condition, such as different angle measurements being generated by redundant encoders associated with a given joint of the robotic arm 300 that have a δ exceeding a predetermined threshold. In response to detecting the critical fault condition, the fault detection module 1440 may actuate the braking mechanism associated with each motorized joint of the robotic arm 300, thereby freezing the robotic arm 300 and inhibiting further movement of the robotic arm 300. Such a critical fault condition may be cleared by the operator restarting the system. When the system is restarted, if the fault detection module 1440 still detects the critical fault condition, the robotic arm 300 will remain frozen until the critical fault condition is cleared.

[0236] The indicator interface module 1442 can be executed by the processor 1402 to cause the indicator 334 to convey the state of the system (e.g., the operating mode of the robotic arm 300) to an operator or other user based on determinations made, for example, by the passive mode determination module 1432, the co-manipulation mode determination module 1434, the haptic mode determination module 1436, and / or the robotic assist mode determination module 1438. For example, the indicator interface module 1442 can cause the indicator 334 to illuminate with a specific color light associated with a specific state of the system. For example, the indicator interface module 1442 can cause the indicator 334 to illuminate with a first color (e.g., yellow) to indicate that no surgical instrument is attached to the robotic arm and the robotic arm can move freely such that the system compensates for the mass of the robotic arm; to illuminate with a second color (e.g., purple) to indicate that a surgical tool is attached to the robotic arm and the robotic arm can move freely such that the system compensates for the mass of the robotic arm and the mass of the surgical instrument coupled to the robotic arm; to illuminate with a third color (e.g., blue) to indicate that a surgical instrument is attached to the robotic arm and the robotic arm is in the passive mode determined by the passive mode determination module 1432; to illuminate with a fourth color (e.g., pulsed orange) to indicate that at least a portion of the robotic arm and / or the surgical instrument attached thereto is within a virtual haptic boundary, e.g., 1.4 m or more above the floor; to illuminate with a fifth color (e.g., pulsed red) to indicate that the fault detection module 1440 has detected a fault in the system. As will be understood by those skilled in the art, different colors and patterns can be conveyed by the indicator 334 to indicate the state of the system described above.

[0237] In addition, the indicator 334 can be illuminated with other different colors and / or patterns to convey additional manipulations of the robotic arm 300, e.g., when the robotic arm 300 retracts the surgical arm in the robotic assist mode or performs another robotic assist manipulation in the robotic assist mode. As described above, the indicator 334 can also include means for emitting other alerts, such as an audible alert or a text alert. Thus, the indicator interface module 1442 can cause the indicator 334 to convey the state of the system to the operator using audio or text as well as light or instead of light. For example, when the robotic arm 300 is approaching a possible collision with one or more objects / persons in the operating room, the indicator interface module 1442 can cause one or more speakers to emit a sound alert, e.g., with a changing amplitude and / or frequency.

[0238] Additionally or alternatively, the indicator interface module 1442 can communicate the status of the system via haptic feedback at the distal end of the robotic arm 300 and correspondingly on the surgical instrument coupled thereto, e.g., a transition from the co-manipulation mode to the passive mode. For example, when the surgical instrument is held in position for a predetermined dwell time to cause the system to switch to the passive mode, the user can feel a vibration on the surgical instrument, indicating that the system has switched to the passive mode and the user can release the surgical instrument. As another example, the user can feel a vibration after the surgical instrument is coupled to the coupler body to indicate successful coupling of the surgical instrument to the robotic arm. The vibration may be strong enough for the user to feel, but weak enough that any movement of the distal end of the surgical instrument resulting therefrom is negligible.

[0239] The fatigue detection module 1444 can be executed by the processor 1402 to detect user fatigue that may occur during the operation of the robotic arm 300 during a surgical procedure, as described in further detail below with respect to Fig.25 For example, based on data from, e.g., the robotic arm position determination module 1418, the force detection module 1422, the impedance calculation module 1424, the fatigue detection module 1444 can use the surgical instrument coupled to the robotic arm 300 to determine the operator's fatigue level and compare the fatigue level to a predetermined fatigue threshold. For example, the fatigue detection module 1444 can evaluate an overall score for a given procedure to determine the fatigue level based on, e.g., operator hand tremors, the distance / minimum path traveled by the instrument tip, the time to achieve a procedure step, and / or the time to complete the procedure. Based on the data generated by the fatigue detection module 1444, the impedance calculation module 1422 can determine the amount of impedance to be applied to the robotic arm 300 to, e.g., reduce operator tremors, such that the motor interface module 1426 can cause the associated motor to apply the necessary impedance to the robotic arm 300. Additionally, based on the data generated by the fatigue detection module 1444, the motor interface module 1426 can cause the associated motor to move the linkages of the robotic arm 300 to guide the operator's manipulation of the surgical instrument attached thereto.

[0240] The co-manipulating surgical robot system described herein may include additional modules within the memory 1410 of the platform 200 for performing additional tasks based on the acquired data. For example, the system may determine that a surgical instrument has been attached to the robotic arm 300 by detecting, via the force detection module 1422, a rapid or sudden change in the force applied to the robot (a "snapping movement") caused, for example, by the attractive force of the magnetic connection between the coupler body and the coupler interface 400. For example, when the magnets are aligned, the attractive force of the magnets on the coupler body and the coupler interface 400 may cause a sudden movement on at least the end portion of the robotic arm, and / or a sudden rotation of the last joint of the robotic arm. Thus, such a sudden movement can be detected and can trigger the surgical instrument identification module 1412 to determine that the instrument has been attached to or detached from the robotic arm. Similarly, the surgical instrument identification module 1412 may determine that the surgical instrument has been detached from the robotic arm 300, for example, when subsequent movement of the distal end of the robotic arm 300 is accompanied by little or even no rotation of the most distal joint of the robotic arm 300.

[0241] Additionally, the system may determine whether a surgical instrument has been detached from the robotic arm 300 based on data indicating the position of the distal end of the robotic arm 300 relative to the trocar point generated by the trocar position detection module 1420 and the orientation of the instrument axis and / or the orientation of the most distal link of the robotic arm 300 (e.g., the distal wrist link 316). For example, if the instrument is pointing directly at the trocar, the likelihood that the tool is attached to the robotic arm is higher. Additionally, the axis Q7 of the robotic arm 300 may indicate the direction in which the instrument is pointing, and if the instrument is passing through the trocar port, the distal wrist link 316 will point in the direction of the trocar port. Thus, if the distal wrist link 316 is not pointing directly at the trocar port, the system may determine that the robotic arm is not supporting the instrument or that the instrument is not being advanced through the trocar port. For example, when the instrument is detached from the robotic arm 300 and the robotic arm 300 is moved, the calculated direction of the instrument axis (e.g., the direction in which the instrument would point if attached to the robotic arm 300) may no longer point at the trocar entry point and may not point at the trocar entry point. Thus, if the system determines that no tool is coupled to the robotic arm 300, the user may be warned, for example, via the indicator 334.

[0242] Additionally, the system can recognize when the user may attempt to remove or separate the surgical instrument from the robotic arm 300 and adjust the removal force required to separate the surgical instrument and thus the coupler body from the coupler interface 400. For example, in a case where one or more magnets are used to provide a biasing force to bias the surgical coupler body to the coupler interface, a force greater than the attractive force provided by the one or more magnets must be applied to the surgical instrument and / or the coupler body coupled to the surgical instrument in a direction opposite to the force provided by the one or more magnets to overcome the attractive force and separate the coupler body and the surgical instrument from the coupler interface. For example, the removal force can be 30 - 60 Newtons.

[0243] Furthermore, the system can collect and analyze telemetry data regarding the forces applied to the robotic arm to evaluate or estimate whether the user is attempting to remove the tool from the robotic arm and, if so, reduce the coupling force between the coupler body and the coupler interface to make it easier for the user to disengage the surgical instrument from the robotic arm. For example, the coupling / removal force can be reduced by 50 - 80%. Based on historical data and user feedback, and based on data such as whether the user replaces the instrument without adjusting the orientation of the instrument (which may indicate an unintentional removal of the instrument), the system can estimate the optimal time to reduce the coupling force between the coupler body and the coupler interface. Additionally, the coupling force can be increased during operation to prevent the unintentional removal of the surgical instrument from the robotic arm.

[0244] Moreover, the system can determine the optimal positioning of the robotic arm 300 and its joints, the surgical instrument coupled to the robotic arm, or other components of the robotic arm and / or the system based on data obtained from an optical scanning device used with the system, and provide guidance to the operator of the system to achieve the optimal positioning. The data indicating the optimal positioning can also be used by the processor 1402 to instruct the motors to move the corresponding links and joints of the robotic arm 300, for example, in a robotic assist mode, to automatically reposition the robotic arm 300 and / or the optical scanning device in the optimal position, for example, during or after a setup phase.

[0245] Additionally, the system can collect data from sensors during a procedure (e.g., during the setup or operation of robotic arm 300), such as position data of robotic arm 300 or a surgical instrument attached thereto via an encoder or an optical scanning device and / or position data of an operator via a body sensor or an optical scanning device, such that processor 1402 can detect a deviation of the current user's movement or procedure compared to a model or an optimal movement pattern and communicate the deviation to the current user in real time. For example, processor 1402 can cause a monitor to display the deviation, as well as the optimal and / or actual movement pattern, to the current user in real time. Additionally or alternatively, indicator interface module 1440 can cause indicator 334 to indicate a deviation from the model or the optimal movement pattern, e.g., by illuminating a specific color and / or in a specific pattern. Additionally or alternatively, motor interface module 1426 can apply an operator-perceivable impedance to robotic arm 30 as haptic feedback, including vibration, a restriction of movement, or a sensation indicating a deviation from the model or the optimal movement pattern. Thus, the system can be used as a training tool for new users, as such data can be used to optimize the position of the surgical device in real time.

[0246] The system can also analyze a depth map generated by an optical scanning device and cluster different (depth) pixel groups into unique objects, a process referred to as object segmentation. Examples of such algorithms for segmentation can include: matching the acquired depth map data to a known template of an object to be segmented; using a combination of depth and RGB color images to identify and isolate relevant pixels of the object; and / or training a machine learning algorithm on a real or synthetic data set to identify and segment objects. Examples of such segmentation on a depth map can include: locating a robotic arm or determining the position of a robotic arm; identifying a patient port (e.g., a trocar port) and determining the distance from an instrument to the trocar port; identifying a surgeon and differentiating the surgeon from other operators in the room; and / or identifying a surgeon within the field of view of a sensor. Additionally, the system can use an object segmentation algorithm to uniquely identify a surgeon and track the surgeon relative to, e.g., a surgical table, a patient, one or more robotic arms, etc. Additionally, the system can use an object segmentation algorithm to determine whether a surgeon is touching or manipulating any of the robotic arms and, if so, identify which robotic arm the surgeon is touching or manipulating. The system can also use object segmentation to locate the distal ends of surgical instruments and robotic arms in 3D space such that the system can determine, e.g., based on the proximity between the distal ends of the surgical instrument and the robotic arm, whether the surgical instrument is attached to the robotic arm.

[0247] Now referring to Fig.15 , operation 1500 of the co-manipulating surgical robot system described herein is provided. As Fig.15As shown, in step 1502, the operator can couple the selected surgical instrument to the coupler interface 400 of the robotic arm 300 via a coupler body (e.g., coupler bodies 500, 600, 700). As described above, the operator can select a coupler body that is sized and shaped to couple to the selected surgical instrument, for example, based on the diameter of the elongate shaft of the surgical instrument. When the surgical instrument and the coupler body are ready to be coupled to the robotic arm 300, the operator can load the calibration file of the selected surgical instrument, for example, via the user interface 1408, such that information associated with the selected surgical instrument (e.g., laparoscope or retractor) is loaded into the system. For example, the operator can select a calibration file from a database of calibration files for various surgical instruments. The calibration files can be stored from a previous procedure and can be pre-loaded to include calibration files for commonly used laparoscopic instruments.

[0248] If the calibration file for the selected surgical instrument is not available in the database, the operator can use the system to self-calibrate the surgical instrument. For example, Fig.16 A surgical instrument calibration process 1600 for calibrating a surgical instrument is illustrated, for example, to determine the center of mass of the surgical instrument, which can be used to calculate precise force measurements on the surgical instrument and the robotic arm 300 during operation. At step 1601, the operator can activate the "Start" option on the user interface 1408. At step 1602, the operator can select "Load Tool Calibration" to begin the calibration process. At step 1603, the system does not apply any impedance to the robotic arm 300 for gravity compensation of the surgical instrument. The system can apply impedance to the robotic arm 300 to account for the weight of the robotic arm 300, for example, to prevent the robotic arm 300 from falling to the ground. At step 1604, the surgical instrument is coupled to the coupler interface 400 of the robotic arm 300 via a properly sized coupler body, which can cause the wrist portion 411 of the robotic arm 300 to rotate about axis Q7 to engage the coupler body.

[0249] At step 1605, the system compensates for the gravity of the surgical instrument and the forces applied by the operator's hand. For example, by measuring the force applied to the distal end of the robotic arm 300 due to the mass of the surgical instrument. As described above, the force applied to the distal end of the robotic arm 300 can be measured by measuring the motor current on the motor placed in the base of the robotic arm 300. If the system over - compensates for the gravity of the surgical instrument, at step 1606, the robotic arm 300 may "runaway", for example, drift upwards. The runaway effect can be detected at step 1607, and at step 1608, the indicator 334 can blink to indicate the runaway to the operator. At step 1609, the system can identify the runaway as a minor fault and thus apply additional impedance to the robotic arm 300 and freeze the robotic arm 300 before removing the additional impedance when the robotic arm 300 decelerates. Once the minor fault is resolved, the calibration process 1600 can return to step 1603.

[0250] After step 1605, when the system compensates for the gravity of the surgical instrument, if the surgical instrument is accidentally or manually disengaged by the operator at step 1611, at step 1610, the system detects that the surgical instrument has been disengaged from the robotic arm 300. Consequently, the system will stop compensating for the gravity of the surgical instrument, and the calibration process 1600 can return to step 1603. After step 1605, when the system compensates for the gravity of the surgical instrument, the calibration process 1600 is ready to enter the calibration mode at step 1612. For example, the operator can initiate the calibration mode at step 1613 via the user interface 1408. At step 1614, the system can indicate to the operator, for example, via the user interface 1408 and / or the blinking of the indicator 334, that it is safe to release the surgical instrument, such that the operator can release the surgical instrument at step 1616. At step 1615, the system calibrates the surgical instrument.

[0251] Referring again to Fig.15 , when the surgical instrument and the coupler body are ready to be coupled to the robotic arm 300 and the appropriate calibration file is loaded, the operator can easily place the coupler body near the coupler interface 400 such that the magnetic connection between the coupler body and the coupler interface 400 automatically aligns and couples the surgical instrument to the robotic arm 300. The system will now accurately compensate for the gravity of the selected surgical instrument. At step 1504, the user can use the co - manipulation surgical system by freely manipulating the surgical instrument coupled to the robotic arm 300 in the same manner as the operator would without the robotic arm 300 coupled to it. As Fig.15As shown, when an operator manipulates a surgical instrument and thus the robotic arm 300 coupled thereto, the system can automatically switch between, for example, a co-manipulation mode 1506, a passive mode 1508, a haptic mode 1510, and a robot-assisted mode 1512 (collectively referred to as "operation modes") when a predefined condition is detected, as described below with respect to Fig.17 as described. In some embodiments, the system can automatically switch between only the co-manipulation mode 1506, the passive mode 1508, and the haptic mode 1510. In some embodiments, the operator can select which operation mode to set the system to before using the co-manipulation surgical system at step 1504.

[0252] For example, the operator can apply a specific force on the distal end of the robotic arm 300 by manipulating, for example, a surgical instrument coupled to the robotic arm 300 to indicate that the operator wishes to change the operation mode of a particular robotic arm. Sensor and / or motor current readings can be used to detect the force applied to the distal end of the robotic arm 300, and it can be determined whether the force matches a predefined force signature associated with an operation change, for example, by comparing the force with one or more predefined force signatures stored in the system. If there is a match, the system can change the operation mode of the robotic arm to a particular operation mode that matches the force signature.

[0253] As described above, during the operation of the co-manipulation surgical system, the system can continuously monitor the robotic arm and the force applied thereto to detect a predefined condition that requires switching the operation mode of the system, as Fig.17 described in method 1700 of. As Fig.17 shown, at step 1702, the system continuously collects data related to a first operating characteristic of the robotic arm and / or a surgical instrument coupled to the robotic arm. For example, as described above, the system can measure the motor current of the motors of the joints operatively coupled to the robotic arm and the angles of the links of the robotic arm based on measurements by the encoders of the robotic arm to real-time compute the position of the robotic arm and the surgical instrument and the forces (if any) acting on any part of the robotic arm and the surgical instrument. At step 1704, the system can analyze the data related to the first operating characteristic to determine whether a first condition exists. For example, based on the position and force data of the robotic arm and / or the surgical instrument, the system can determine whether the movement of the robotic arm caused by the movement of the surgical instrument coupled thereto is within a predefined movement threshold of the robotic arm for a period of time longer than a predefined dwell time of the robotic arm. When the first condition is detected, at step 1706, the system can modify a first operating parameter of the robotic arm. For example, the system can switch the operation mode of the robotic arm to the passive mode, in which the robotic arm maintains the surgical instrument in a static position.

[0254] For example, a first robotic arm can be coupled to a laparoscope, and an operator can manipulate the laparoscope within a patient until the laparoscope provides a desired field of view, e.g., via a monitor that displays an image feed from the laparoscope. To freely move the laparoscope coupled to the first robotic arm in a co-manipulation mode, the operator must apply sufficient force to the laparoscope that exceeds a predetermined force threshold. The predetermined force threshold should be low enough such that the operator does not need to apply too much force to freely move the laparoscope. Additionally, the predetermined force threshold can be selected to resist unintentional movement away from the passive mode. When the operator freely moves the laparoscope in the co-manipulation mode as described above, the system will apply sufficient impedance to the first robotic arm to compensate for the effects of the mass (i.e., inertia) and / or gravity of the first robotic arm and the laparoscope during movement, such that the mass or weight of the first robotic arm cannot be detected by the operator or otherwise significantly attenuated. In some embodiments, if the laparoscope is not yet positioned within the patient when the operator couples the laparoscope to the first robotic arm, the system can determine that there are no external forces acting on the surgical instrument and can automatically switch the first robotic arm to a haptic mode to guide the operator, e.g., via a virtual haptic funnel established around a trocar port, to move the laparoscope through the trocar port to an appropriate orientation.

[0255] When the laparoscope is in a desired position relative to the patient and the surgical site within the patient, when it is detected that movement of the first robotic arm caused by movement of the surgical instrument is within a predetermined movement threshold for a period of time exceeding a predetermined dwell time, the system will automatically switch from the co-manipulation mode to the passive mode. For example, upon reaching the desired position, the operator holds the laparoscope in the desired position, e.g., for at least a quarter of a second. Thus, if the predetermined dwell time is a quarter of a second, holding the laparoscope in the desired position for any time longer than the predetermined dwell period will cause the system to automatically switch to the passive mode. Additionally, since the operator may not be able to hold the laparoscope completely still, at least some movement of the laparoscope is allowed during the duration of the predetermined dwell time to enter the passive mode. As described above, in the passive mode, the first robotic arm holds the laparoscope in a static position, e.g., by the system applying sufficient impedance to the first robotic arm to compensate for all external forces acting on the laparoscope.

[0256] Similarly, the second robotic arm can be coupled to the retractor, and the operator can freely manipulate the retractor within the patient in a co-manipulation mode, e.g., to grasp tissue within the patient through the laparoscope coupled to the first robotic arm and retract the tissue by applying sufficient force to the second robotic arm due to the force applied at the retractor exceeding a predetermined force threshold of the second robotic arm to provide a clear field of view of the surgical site. When the operator grasps / lifts / retracts tissue with the retractor, the system can compensate only for the gravity of the second robotic arm and / or the instrument, and not for the gravity of the grasped tissue, such that the operator can sense any other forces acting on the retractor, including but not limited to forces from the tissue acting on the instrument. In this alternative configuration. Thus, the haptic associated with the grasped tissue can be retained.

[0257] When the retractor sufficiently grasps and retracts tissue, the system can automatically transition to a passive mode after a period of time in which the operator holds the retractor in place (e.g., where the movement does not exceed a predetermined movement threshold of the second robotic arm) for a period exceeding a predetermined dwell period of the second robotic arm. Thus, when the retractor retracts tissue within the patient in the passive mode, the second robotic arm will account for the mass of the tissue in addition to the mass of the retractor and the second robotic arm. Thus, the predetermined force threshold for switching the second robotic arm out of the passive mode must be greater than the force applied to the second robotic arm due to the force applied by the tissue to the tip of the retractor, such that if the force applied by the tissue to the surgical instrument exceeds a predetermined first threshold of the second robotic arm, the system will automatically switch the second robotic arm out of the passive mode and into, e.g., a co-manipulation mode. However, the predetermined force threshold should not be so high that it is very difficult for the operator to move the retractor. As described above, the operator can adjust the predetermined force threshold via, e.g., the user interface 1408.

[0258] When retracting tissue via the retractor coupled to the second robotic arm, the operator may need to readjust the field of view of the laparoscope coupled to the first robotic arm. Thus, the operator can apply a force to the laparoscope that exceeds a predetermined force threshold of the first robotic arm, such that the system automatically switches the first robotic arm from the passive mode to the co-manipulation mode. When the new desired position of the laparoscope is reached, if the above predefined conditions are met, the first robotic arm can automatically switch back to the passive mode. Alternatively, to readjust the laparoscope or reposition the linkage of the first robotic arm to avoid potential collisions during the laparoscopic procedure or to switch the laparoscope completely to a different robotic arm, the operator can choose to detach the laparoscope, readjust the robotic arm and / or the laparoscope, and reattach the laparoscope to the first robotic arm (or another robotic arm). When the laparoscope is reattached to the first robotic arm, if the above predefined conditions are met, the first robotic arm can automatically switch to the passive mode.

[0259] In addition, when the operator freely moves the retractor in the co-manipulation mode, for example, before inserting the tip of the retractor through the trocar into the patient, if the operator moves the tip of the retractor too close to the patient's skin away from the trocar port and the system has established a virtual haptic boundary on the patient's skin outside the trocar port, the system can automatically switch to the haptic mode. Thus, the system can apply a much higher impedance to the second robotic arm than the impedance applied to the second robotic arm in the co-manipulation mode to indicate to the operator that they are approaching or within the virtual haptic boundary. For example, the movement of the retractor by the operator may feel more viscous in the haptic mode. The system can remain in the haptic mode until the operator moves the retractor out of the virtual haptic boundary. In some embodiments, in the haptic mode, the second robotic arm can reduce the effect of gravity, eliminate tremors at the tip of the instrument, and apply force feedback to avoid critical structures defined by the virtual haptic boundary. Thus, the system does not replace the operator, but enhances the operator's capabilities through features such as gravity compensation, tremor removal, haptic barriers, force feedback, etc.

[0260] In some embodiments, the system can switch the second robotic arm to the robot-assisted mode. For example, when the operator attempts to retract tissue, if more force than the operator is able or willing to apply to the retractor is required to retract the tissue, the operator can provide user input indicating that the operator wants the second robotic arm to assist in the retraction of the tissue. For example, as described above, the operator can perform a predefined pose pattern that can be detected by, for example, the optical scanner 1100, such that the system switches the second robotic arm to the robot-assisted mode and causes the motor of the second robotic arm to move the second robotic arm and thus the retractor to provide the additional force required to retract the tissue.

[0261] Alternatively, instead of manually manipulating the laparoscope coupled to the first robotic arm as described, the operator can provide to the system another user input indicating that the operator wants the system to reposition the laparoscope. For example, if the operator actively manipulates surgical scissors that may or may not be coupled to the system and the tip of the surgical scissors is within the field of view of the laparoscope coupled to the first robotic arm, the operator can perform a predefined pose pattern with the tip of the surgical scissors, e.g., quickly move the surgical scissors forward in a specific direction. The predefined pose pattern of the surgical scissors can be captured by the laparoscope as image data, and based on this data, the system can detect the predefined pose pattern and associate it with a predefined user input that requires the system to switch the first robotic arm from the passive mode to the robot-assisted mode, and cause the first robotic arm to reposition itself and thus the laparoscope to adjust the field of view in the direction of movement of the pattern of the surgical scissors. As described above, additional pose patterns can be performed via the surgical scissors within the field of view of the laparoscope to cause the first robotic arm to retract the laparoscope and / or cause the laparoscope itself to zoom in or out or increase the resolution. In some embodiments, based on the image data captured by the laparoscope, using object tracking of an additional tool (e.g., surgical scissors actively operated by the operator) within the field of view of the laparoscope, the system can automatically switch the first robotic arm coupled to the laparoscope to the robot-assisted mode and cause the first robotic arm to reposition itself to adjust the field of view, thereby ensuring that the tip of the surgical scissors remains within the optimal position within the field of view of the laparoscope during the procedure.

[0262] The operating mode of any one of the robotic arms can be changed independently of the operating modes of the other robotic arms of the system. Additionally, the operating parameters of each robotic arm can be adapted to the specific surgical instrument coupled to it. For example, the predefined force threshold of the robotic arm coupled to the retractor device can be higher than the predefined force threshold of the robotic arm coupled to the laparoscope because the retractor will withstand higher forces during the procedure. The sensors, motors, etc. of the system can be active in all modes, but can function very differently in each mode, e.g., including functioning as if deactivated. As will be understood by those skilled in the art, the system can include more than two robotic arms such that the operator can couple a third surgical instrument (e.g., a grasper device) to a third robotic arm and a fourth surgical instrument (e.g., a surgical scissors device) to a fourth robotic arm for use during a laparoscopic procedure.

[0263] In some embodiments, the operating mode of the robotic arm can be changed in response to user input provided by an operator. For example, the operator can selectively change the operating mode of the robotic arm by actuating a button, dial, or switch located on the robotic arm, a foot pedal or foot switch, a voice command, an input on a touchscreen, or using a gesture or force signature as described above. In some embodiments, the operating mode of the robotic arm can be changed based solely on a surgical instrument being coupled to a coupler interface via a coupler body. As described above, the system can automatically identify the surgical instrument based on the coupling of the coupler body to the coupler interface. Thus, based on the identity of the surgical instrument coupled to the robotic arm, the system can automatically switch the operating mode of the robotic arm to a predetermined operating mode, e.g., a passive mode if the surgical instrument is an endoscope, or if the robotic arm is already in the passive mode, the system will remain in the passive mode when the endoscope is coupled to the robotic arm.

[0264] Similarly, based on the identity of the surgical instrument when attaching the surgical instrument to the robotic arm, the system can automatically switch the operating mode of the robotic arm to a co-manipulation mode, e.g., whether the surgical instrument identity indicates that it is a tool that will be actively manipulated by the operator during a laparoscopic procedure. Additionally, based on the identity of the surgical instrument when attaching the surgical instrument to the robotic arm, the system can automatically switch the operating mode of the robotic arm to a robot-assisted mode, e.g., if the surgical instrument identity indicates that it is a tool whose operation is desired to be fully robot-controlled, such as an irrigation device. Thus, when an irrigation device is attached to the robotic arm, the system will switch to the robot-assisted mode and cause the robotic arm to position the irrigation device at a desired location within the body.

[0265] Furthermore, the system can be instructed by the operator, e.g., via the user interface 1408, to operate the robotic arm in fewer than the four operating modes described above. For example, the operator can deactivate any one of the operating modes for a given procedure. In some embodiments, the system can cause the robotic arm to operate in an additional operating mode, such as a locked mode, which can be similar to the passive mode, except that the predetermined force threshold for the robotic arm to switch out of the passive / locked mode can be so high that the robotic arm is effectively frozen in order to protect the robotic arm from inadvertently switching out of the passive / locked mode, e.g., to avoid movement due to an inadvertent collision of the robotic arm. In this locked mode, if the force from an inadvertent collision is high enough to cause even a slight movement of the robotic arm, the system can cause the robotic arm to reposition itself to the position it was in prior to the inadvertent collision.

[0266] Additionally, when no surgical instrument is coupled to the distal end of the robotic arm of the system, the system is still capable of automatically switching the operating mode of the robotic arm in response to the movement of the robotic arm by the operator when the above-mentioned predefined conditions are detected. Therefore, the system will apply impedance to the joints of the robotic arm to compensate for the mass of the robotic arm, such that the robotic arm can remain in a static position when in the passive mode, and if the system detects that the force applied by the operator to the robotic arm exceeds a predefined force threshold of the robotic arm, it will allow the robotic arm to be freely moved by the operator in the co-manipulation mode. Additionally, if the operator attempts to move any part of the robotic arm within a predefined virtual haptic barrier, the system will switch the robotic arm to the haptic mode. At step 1514, when the laparoscopic procedure is completed, the operator can remove the surgical instrument from the corresponding robotic arm.

[0267] Now refer Fig.18A to FIGS. 18C, force measurements during the operation of the robotic arm 300 are provided. As described above, when attaching a surgical instrument to the coupler interface 400 via a coupler body coupled to the surgical instrument, the orientation of the surgical instrument can be automatically determined based on the magnetic connection between the coupler interface and the coupler body. Additionally, as described above, the calibration file of the surgical instrument coupled to the robotic arm 300 loaded on the system can include information of the surgical instrument (including, for example, the mass of the surgical instrument, the center of mass of the surgical instrument, and the length of the surgical instrument), such that the distance D3 between the center of mass and the tip of the instrument can be derived. Additionally, as described above, the position of the surgical instrument at the trocar (e.g., where the surgical instrument enters the patient's body) can be calculated in real time, such that the distance D2 between the center of mass of the surgical instrument and the trocar can be derived in real time. Additionally, as described above, the coupler body is preferably coupled to the surgical instrument at a fixed known position along the elongate axis of the surgical instrument (which can be included in the calibration file) (e.g., adjacent the proximal portion of the surgical instrument), and thus the distance D1 between the center of mass of the surgical instrument and the coupler body (e.g., the attachment point to the distal end of the robotic arm 300) can be derived. Alternatively or additionally, as described above, an optical scanning device can be used to determine any one of D1, D2, or D3.

[0268] As Fig.18A shown, when the surgical instrument is positioned through the trocar Tr without any additional external force acting on the surgical instrument other than at the trocar Tr (e.g., the surgical instrument is not lifting or retracting tissue within the patient's body), the force applied by the body wall to the surgical instrument at the trocar Tr (e.g., "body wall force" or "trocar force") can be calculated using the following equation:

[0269] Feff +W + F tr = 0 => F tr = -W - F eff

[0270] where F eff is the force at the distal end of the robotic arm 300 (e.g., the "end - effector force" of the robotic arm 300), W is the weight vector of the surgical instrument (= -mgz), and F tr is the trocar force. Thus, F eff is the desired force sent to the system, which is the sum of all forces generated in the algorithm pipeline (including, for example, gravity compensation, holding, etc.).

[0271] As Fig.18B shown, when the surgical instrument is positioned through the trocar Tr and holds / retracts tissue such that an external force is applied to the tip of the surgical instrument, there are two forces to be solved for: F tr and F tt . Thus, two equations are needed to solve for the two unknown vectors, which can be the balance of forces and the balance of torques about the center - of - mass of the surgical instrument (e.g., L cg ).

[0272] W + F eff + F tr + F tt = 0

[0273] F eff × D1+F tr × D2+F tt × D3 = 0

[0274] Here, the distances D1 and D3 are known as described above, and D2 can be derived based on the known position of the distal end of the robotic arm 300 and the calculated position of the trocar Tr. As Fig.18B shown, the center - of - mass Lcg of the surgical instrument is behind the attachment point of the coupler body to the distal end of the robotic arm 300.

[0275] As described above, if the force (e.g., the force Ftt applied to the tip of the instrument and / or the force Ftr exerted by the instrument at the trocar) is greater than the corresponding threshold force, the system can warn the operator, and thus if the calculated force is greater than the threshold force, freeze the system, and / or reduce the force applied at the trocar point at the body wall or the tip of the instrument by automatically applying a brake or stop force to the robotic arm 300, by slowing down or preventing further movement of the instrument in the direction that would increase the force applied at the tip of the instrument or the trocar, and / or by automatically moving the robotic arm in the direction of reducing the force applied at the tip of the instrument and / or at the trocar point at the body wall.

[0276] Now refer to Fig. 20 for a high-level example 2000 of different combinations of data inputs from the various sensors and devices of the systems disclosed herein (e.g., system 200), and the various features and capabilities that any implementation of the systems disclosed herein can have and can at least partially generate based on multiple possible data inputs. As Fig. 20 shown, some implementations of the system can be configured to collect data from at least three monitoring sources 2002, including telemetry from the system (which can include force data from a robotic arm, position data from a robotic arm, etc.), video from a laparoscopic tower, and / or data from an optical scanner 1100. The data collected from the monitoring sources 2002 can undergo data processing steps 2004 using one or more processors in the system. The data processing steps can include, for example, data fusion (e.g., fusion of data collected from the monitoring sources 2002) and data analysis, which can include algorithmic calculations. Additionally, the data from the monitoring sources 2002 can undergo processing 2004 for developing system usability features 2006, system security features 2008, and system performance features 2010. The system can provide the features in real time. For example, system usability features can include identifying the surgeon and adjusting the platform height based on the surgeon's profile, detecting the patient's skin surface and creating a virtual boundary around the skin surface to prevent unintentional contact with the patient's skin surface, detecting the instrument type and automatically loading a calibration file suitable for the specific instrument, etc. Additionally, system security features can include, for example, displaying a virtual map of the area around the platform 100 as the operator moves the platform 100 throughout the operating room to provide the operator with a view of the area around the platform 100 so that the operator can avoid collisions between the platform 100 and any objects and / or people within the area around the platform 100.

[0277] Refer to Fig.21 for a schematic overview of the electrical components of an electrical system and the connections 2100 of the system. This includes the flow of energy throughout the illustrated portions of the system, the ports that can be used for connections, and other details related to the various electronic components. For example, the system can include a non-real-time computer 2102 that can be used to obtain data from an optical scanning device and perform other functions. The non-real-time computer 2102 can also control the graphical user interface of the system for the surgeon to interact with. As described above, the graphical user interface can include a touch screen. The non-real-time computer 2102 can include, for example, a 10th generation Core TMi7-10700 processor, 32GB of RAM (which can optionally be 2x16GB, DDR4, 2933Mhz), standard keyboard, and 512GB PCIe M.2 SSD + 1TB SATA 7200RPM hard disk drive, wireless and Bluetooth card (such as Killer TM Wi-Fi 6 AX1650i (2x2) 802.11ax wireless and Bluetooth 5.1) and / or GeForce RTX TM 2060 6GB GDDR6 graphics card. The system may also include a real-time computer 2104 that can be used to operate and control the robotic arm and associated robotic controller and / or other functions (such as obtaining data and information from an optical scanning device). The real-time computer 2104 may include, for example, an Intel Core i7 (8th generation) processor, 32GB of RAM for memory, a 500GB SDD hard disk drive, and / or two or more RJ45 connectors for Ethernet connection.

[0278] Now referring to Fig. 22 , a flowchart of a process 2200 for obtaining and processing data from an optical scanning device is provided. As Fig. 22As shown, at step 2202, depth data can be obtained from one or more optical scanning devices (e.g., optical scanner 1100). At step 2204, filtering / other signal processing algorithms can be performed, such as median filtering, Gaussian noise removal, anti-aliasing algorithms, morphological operations, ambient light adjustment, etc. At step 2206, 3D object segmentation can be performed using, for example, template matching, machine learning, brute-force matching, color plus depth segmentation, 2D-3D registration, pixel value thresholding, etc. At step 2208, the object coordinates can be transformed to the task space. For example, at step 2206, 3D object segmentation can be performed using, for example, template matching, machine learning, brute-force matching, color plus depth segmentation, 2D-3D registration, pixel value thresholding, etc. At step 2208, the object coordinates can be transformed to the task space. For example, transforming the object coordinates to the task space can include transforming the position and orientation of the object from the coordinate reference system of the optical scanning device to the coordinate reference system of the desired task (e.g., the robot reference system for robot control, the cart reference system for system setup, etc.). Additionally or alternatively, transforming the object coordinates to the task space can include using known optical scanning device to support platform (e.g., cart) transformations, surgical robot transformations, and / or user interface screen transformations, and generating new transformations for specific tasks such as tracking the surgeon's body (e.g., face, hand, etc.) relative to different elements of the system (e.g., support platform, robot arm, screen, etc.), tracking the surgical workbench relative to the cart platform, tracking the patient orientation for system setup, tracking the trocar port position and orientation for setup, and tracking the position of the operating room staff for safety. At step 2210, the desired task can be performed, such as moving the robot arm near the patient / trocar port for setup, tracking the operating room staff to ensure the system responds only to surgeon commands, recording the hand movements of the surgeon during different stages of the surgery, etc.

[0279] In addition, Fig. 22 FIG. shows a flowchart of a process 2212 for obtaining and processing data from an optical scanning device. At step 2214, depth data can be obtained from one or more optical scanning devices (e.g., optical scanner 1100). At step 2216, specular noise filtering can be performed. At step 2218, patient / trocar port segmentation and identification can be performed. At step 2218, the tracked port coordinates can be transformed to the robot coordinate space. At step 2222, the robot arm can be moved to the desired vicinity of the patient / trocar port.

[0280] Now referring to Fig.23 , an example data flow 2300 of the system is provided. As Fig.23As shown, the non-real-time computer 2302 can collect data from an optical scanning device (e.g., the optical scanner 1100) and / or from the camera feed of a laparoscope. The non-real-time computer 2302 can also receive data from the real-time computer 2308 having a robot controller, including telemetry information such as the position of the robotic arm, the forces applied to the various motors / sensors of the robotic arm, operation mode information, etc. The non-real-time computer 2302 can also receive data from the patient database 2310 having patient-specific information in a program, including, for example, CT scan data, relevant health conditions, and other information that a surgeon may desire.

[0281] The non-real-time computer 2302 can also provide user feedback 2312 to a user via the user interface 2314. The user feedback can include, for example, collision notifications, positioning information and / or recommendations regarding the various components of the system, operation modes detected by the system, etc. The non-real-time computer 2302 can also provide commands 2318 to the real-time computer 2308, such as high-level commands. The high-level commands can include, for example, mode changes, trajectories, haptic barriers, user configurations, etc. The real-time computer 2308 can include a robot controller 2320 programmed to provide robot commands 2322 (e.g., movement or force commands) to one or more robotic arms 2324 (e.g., the robotic arm 300). The robot controller 2320 can receive robot feedback data 2326 from one or more robotic arms 2324, such as movement, force, and / or contact data, etc.

[0282] Now referring to Fig.25 , a method 2500 for estimating user fatigue during a surgical procedure using the robotic arm 300 is provided. As described above, algorithms for gravity compensation, viscosity, and / or mass effects can be used to account for user fatigue. Specifically, during a laparoscopic procedure, as the procedure progresses, the surgeon may experience fatigue and may experience hand tremors or incorrect tool movements of surgical tools (e.g., scissors, needle actuators, cautery tools, graspers). As Fig.25As shown, at step 2502, during a surgical procedure, when an operator manipulates a surgical instrument coupled to the robotic arm 300, the system can receive and monitor data indicative of the operator's performance, such as from an optical scanner 1100 (such as a LiDAR camera), robotic telemetry, and / or an endoscope. At step 2504, learning from a large dataset of clinical procedures and / or collecting and analyzing data during a procedure or a portion of a procedure can allow the system to infer the surgeon's level of proficiency as the procedure progresses and can also allow the system to adjust algorithm parameters to help the surgeon move more effectively when co-manipulating the surgical instrument attached to the robotic arm. For example, at step 2506, the system can adjust one or more operating parameters of the robotic arm 300 to change its behavior. If the fatigue level is above a specific threshold, at step 2608, the system can warn the surgeon. Additionally, a rating procedure can be used to allow the system to provide the surgeon with a summary of their performance for a given procedure and display their overall progress, the procedure following the procedure.

[0283] In some embodiments, the system can collect data during the procedure indicative of at least one of operator hand tremors, the distance / minimum path traveled by the instrument tip, the time to achieve a procedure step, and / or the time to complete the procedure, and compare this data to a threshold or predefined value for each factor to determine if the magnitude of any one factor has reached a level sufficient for the system to warn the operator and / or sufficient for the system to adjust one or more operating parameters to mitigate user fatigue. For example, the system can eliminate or reduce tremors at the instrument tip by applying a force to the instrument to increase its impedance or viscosity, avoid critical structures, and / or apply force feedback. User fatigue can be identified when, for example, the procedure time increases beyond the threshold for a particular procedure, the number of movements of the surgical instrument increases beyond the threshold for a particular procedure, or otherwise indicates an error or uncontrolled movement, if the operator moves the instrument into a haptic barrier a predetermined number of times, if the operator applies too much force to the trocar once or a predetermined number of times, etc. As described above, this data can be collected using sensors on the robotic arm and / or one or more optical scanning devices. When the system identifies a specific level of user fatigue, the system can increase the viscosity or impedance of the instrument and / or the robotic arm associated with the instrument to reduce the magnitude and / or number of movements of the surgical instrument and / or the robotic arm.

[0284] Additionally, the system can collect data on the speed and frequency at which the operator moves various instruments / laparoscopes, as well as an estimate of how much tremor is involved in the movement, and the added viscosity required to reduce tremor without impeding the movement or adding unnecessary fatigue to the operator. In some embodiments, the controller of the robotic arm 300 can iteratively adjust the viscosity value of a particular instrument, collect data related to the movement of the instrument, and evaluate whether additional adjustment to the viscosity applied to the instrument is needed. Additionally, the system can use additional algorithms to employ an iterative approach to optimize particular operating characteristics or parameters of the robotic arm 300, including collecting data related to the particular operating characteristics or parameters, changing the operating characteristics or parameters, collecting additional data related to the operating characteristics or parameters, and analyzing the data to determine whether additional changes should be made to the operating characteristics or parameters, which can be based on, for example, the deviation between the actual data value of the operating characteristics or parameters and the preferred or optimal value.

[0285] Now referring Fig.26 , a data stream 2600 of a distributed network for co-manipulating a surgical robotic system is provided. For example, a distributed network for co-manipulating a robotic ("cobot") surgical system can be used in multiple hospitals, each of which can be connected to an online database. Such an arrangement can provide considerably more data and user information that can be used by any of the cobot systems in operation. The system can aggregate data from the distributed network of the system to identify an optimal configuration based on factors such as procedure type, surgeon experience, patient attributes, etc. Through analysis or clinician input, the cobot system can identify routine procedures and those that may be more complex. This information can be used to provide advice or guidance to novice surgeons.

[0286] Furthermore, centralized procedure data can enable running large data analytics on a wide range of clinical procedures from different users. Analysis of the data can lead to optimized settings for a particular procedure, including, for example, optimized system positioning, optimal port placement, optimal algorithm settings for each robotic arm, and / or detection of procedure anomalies (such as excessive force, time, bleeding, etc.). These optimal settings or parameters can depend on patient and tool characteristics. As described above, a surgeon can load and use the optimal settings from another surgeon or another group of surgeons. In this way, optimal settings can be achieved based on, for example, the level of expertise of the surgeon. To track the various users in the distributed network of the cobot system, it can be beneficial to identify each user. Thus, a user can log in to the cobot system and access their profile online as needed. In this way, users can access their profiles anywhere and will be able to utilize their settings at different hospital locations to perform clinical procedures.

[0287] An example user profile may include a user's specific settings and information, including for example a username; a level of expertise; areas of different procedures and / or clinical practices performed. Additionally, a clinical procedure may require the user to store specific settings, such as a clinical procedure (e.g., cholecystectomy, hernia, etc.), a workbench orientation and height, a preferred port placement, settings for each assist arm for each algorithm, patient characteristics (e.g., BMI, age, gender), and / or surgical tool characteristics and specifications (e.g., weight, length, center of gravity, etc.). A user may be able to enable his own profile and optionally may be able to enable another user's profile, such as a peer's profile, the most representative profile of surgeons in the user's practice area, the most representative profile of surgeons with a specific level of expertise, and / or a recommended profile based on patient characteristics.

[0288] Identification of the user may be performed via a password, an RFID key, facial recognition, etc. Learning from a large number of procedures can lead to a higher level of optimization of the cobot system settings for a given procedure. This may include for example a cart position, positions of individual robotic arms, a surgical workbench height and orientation, port placement, and / or joint positions settings. These settings may be based on patient height, weight, and gender and may also be interdependent. For example, the optimal port placement may depend on the patient workbench orientation.

[0289] Additionally, a clinical procedure may be described as a series of clinical procedure steps. Learning these different steps may allow the cobot system to infer in real time the actual steps of a given procedure. For example, learning clinical steps from a procedure may allow or enable: adjustment of algorithm settings, the system to give actual customized reminders, the system to notify the staff of an estimated procedure end time, the system to warn the staff in the case where necessary equipment is not available in the room, and / or the system to warn the staff of the occurrence of an emergency.

[0290] During a clinical surgery, a surgeon typically performs simple and routine surgical tasks, such as grasping, retracting, cutting, etc. Learning these different tasks may allow the cobot system to infer in real time the preferences and habits of the surgeon regarding the real-time procedure sequence. Some algorithms of the cobot system may be tuned (i.e., adjusted and optimized) based on this sequence identification during the procedure and help the user to better perform this simple surgical task. An example of such a task is automatically retracting the liver during a gallbladder procedure. By aggregating information in many cases, an optimized force vector may be developed.

[0291] In addition, some complications may occur during a clinical procedure, which may lead to unexpected steps or surgical maneuvers. Learning how to distinguish these unexpected events will help the cobot system implement some specific safety features. In an emergency, the robotic arm can stop or limit movement according to the emergency level detected by the system.

[0292] Now referring to FIG. 27A to FIG. 27D , a setup for a co-manipulated surgical system is provided. The platform 2700 can be constructed similarly to the platform 100 such that the platform 2700 supports one or more robotic arms (e.g., robotic arms 300a' and 300b'), and can move the robotic arms relative to the platform 2700. As Fig.27A shown, when the robotic arms 300a', 300b' are in their respective stowed configurations, the platform 2700 can be moved by a user, e.g., via the wheels 104', to a desired position relative to the patient worktable PT.

[0293] When the platform 2700 is moving towards the patient, the scene can be directly observed through a depth mapping sensor (e.g., the optical scanner 1100') that can be mounted on the platform 2700. Based on the depth map observed and generated by the optical scanner 1100', key features can be identified, such as, for example, the height and / or orientation of the patient worktable PT, the surface of the patient's abdomen, the position of the surgeon and other characteristics (including the height of the surgeon), and the (one or more) trocar ports, the bases of the robotic arms 300a', 300b' (e.g., the base portions 302a', 302b' and the shoulder portions 304a', 304b'), the robotic arms 300a', 300b' and / or one or more surgical instruments coupled to the robotic arms. Standard computer vision techniques (such as template matching, feature tracking, edge detection, etc.) can be used to perform the identification of such key features. When each feature is registered, a local coordinate system can be assigned to its position and orientation, and the system can be transformed into a global coordinate system using a standard transformation matrix. Once all features are transformed into a single global coordinate system, optimization algorithms (e.g., least squares and gradient descent) can be used to identify the most appropriate vertical and horizontal positions of the robotic arms 300a', 300b', which can be adjusted via the platform 2700 to maximize the workspace of the robotic arms relative to the insertion points on the patient. The optimal workspace can depend on the surgical operation to be performed and / or the preferred position of the surgeon.

[0294] As Fig.27B shown, when the platform 2700 is in its desired position relative to the patient worktable PT such that the wheels 104' are locked, the robotic arms 300a', 300b' can extend away from their respective stowed configurations. As Fig.27CAs shown, the vertical position of the robotic arm relative to the platform 2700 can be adjusted to a desired position, and as Fig.27D shown, the horizontal position of the robotic arm relative to the platform 2700 can be adjusted to a desired position.

[0295] Now referring to FIG. 28A to FIG. 28D , a screenshot of an exemplary graphical user interface 2800 is provided. The exemplary graphical user interface 2800 can be user-configurable and can be integrated with the display 110. Fig.28A An exemplary start menu is illustrated. The operator can initiate the operation of the co-manipulation system by actuating the "Start" option. Fig.28B An exemplary system settings screen is illustrated. As Fig.28B shown, when the system includes two robotic arms, the graphical user interface 2800 can identify which robotic arm is to be used with which instrument (e.g., the retractor arm 2806 and the endoscope arm 2808), and the procedure to be completed. The graphical user interface 2800 can allow the user to pre-load specific calibration files or set joint positions based on the procedure being performed and / or the surgeon performing the procedure. For example, if the user input procedure is a laparoscopic cholecystectomy, the system can pre-load the tool types known to be associated with that procedure. Filling these pre-loaded settings can be achieved by monitoring which tools the user manually selects for a given procedure. If a given tool is consistently selected for a predetermined number of procedures, the system can automatically pre-fill that tool the next time the user selects the procedure.

[0296] Additionally, the operator can adjust the vertical and horizontal positions of each robotic arm, as shown above in Fig.27C and Fig.27D . As Fig.28B shown, to adjust the vertical and / or horizontal position of the robotic arm that will be or is currently coupled to the retractor device, the operator can switch the adjustment actuator 2802, and to adjust the vertical and / or horizontal position of the robotic arm that will be or is currently coupled to the endoscope device, the operator can switch the adjustment actuator 2804. In some embodiments, the user can adjust the horizontal and vertical positions of the robotic arm by using the robotic arm as a force-sensitive input device. For example, the robotic arm can be configured to sense the user's intent by measuring the force applied by the user to the robotic arm. If the user applies a force in the positive horizontal direction, the platform can move the robotic arm in that direction until the user no longer applies the force. A similar approach is taken for other directions (e.g., negative horizontal, positive vertical, and negative vertical). As Fig.28B shown, the graphical user interface 2800 can indicate via an error notification 2810 whether the system has detected an error, such as a fault condition, during the setup or operation of the system.

[0297] As Fig.28C shown, the graphical user interface 2800 can display information associated with the selected surgical instrument, as described above. For example, for each instrument to be coupled to each robotic arm, the graphical user interface 2800 can display the instrument type, the total length, the distance between the coupler body and the instrument tip, the distance between the center of mass and the instrument tip, the mass, and the preset unlocking force required to unlock the instrument. As Fig.28C shown, the graphical user interface 2800 can allow the operator to select between a high unlocking force or a low unlocking force for the surgical instrument. Additionally, the graphical user interface 2800 can allow the operator to initiate surgical instrument calibration, for example, for a new surgical instrument that does not yet have an associated calibration file stored in the system. Fig.28D illustrates an exemplary screen during system operation (e.g., during a surgical procedure). As Fig.28D shown, the graphical user interface 2800 can display trocar force and the force, for example, applied by tissue within the patient's body to the tip of the surgical instrument.

[0298] Now referring to Fig.29 , an alternative co-manipulating surgical robot system is provided. System 2900 can be constructed similar to Figure 2 system 200. For example, the platform 1400', the base portion 302', the shoulder portion 304', the encoders E1', E2', E3', E5', E6', E7', the motors M1', the shoulder joint 318', the shoulder link 305', the elbow joint 322', the elbow link 310', the wrist portion 311', and the coupler interface 400' for coupling the surgical instrument SI to the robotic arm can be constructed similar to the platform 1400, the base portion 302, the shoulder portion 304, the encoders E1, E2, E3, E5, E6, E7, the motor M1, the shoulder joint 318, the shoulder link 305, the elbow joint 322, the elbow link 310, the wrist portion 311, and the coupler interface 400, respectively. System 2900 differs from system 200 in that system 2900 includes motors placed at the joints of the robotic arm. For example, system 2900 can include a motor M2' placed at the elbow joint 318' and a motor M3' placed at the elbow joint 322', which are configured to rotate the associated links to manipulate the robotic arm. Additionally, the encoder E4' can be positioned on or near the elbow joint 322'.

[0299] Some embodiments of the systems described herein may be configured to be remotely controlled or manipulated, for example, via a joystick or other suitable remote control device, computer vision algorithms, force measurement algorithms, and / or by other means. However, in a preferred embodiment, the systems described herein operate without any telemetry. For example, the robotic arm is not remotely operated via a remote surgeon console separate from the robotic arm, but rather the robotic arm moves in response to movement applied to a surgical instrument coupled thereto. Any robotic-assisted movement applied by the system to the surgical instrument (e.g., in a robotic-assisted mode) does not respond to user input received at the remote surgeon console.

[0300] Fig. 30A A top view of a coupler 3000 for coupling a surgical instrument SI to a robotic arm is illustrated, showing a coupler body 3002 (also referred to herein as the body) coupled to a coupler interface 3001 (also referred to herein as the interface). Figure 2 Figure B illustrates Fig. 30A the coupler 3000 with the coupler body 3002 separated from the coupler interface 3001. As Fig. 30A and Fig. 30B shown, the coupler 3000 may have a coupler body 3002 and a coupler interface 3001. The coupler interface 3001 may be coupled to the robotic arm 300 and may be configured such that the coupler body 3002 may be detachably coupled to the coupler interface 3001. The coupler body 150 may be coupled to the surgical instrument SI at any desired axial position on the surgical instrument SI. Once the coupler body 3002 is coupled to the surgical instrument SI, the coupler body 3002 and the surgical instrument SI coupled to the coupler body 3002 may be coupled to the coupler interface 3001. The coupler body 3002 may be configured such that once the coupler body 3002 is coupled to the surgical instrument SI, axial movement of the surgical instrument SI may be at least inhibited (e.g., prevented), or in some embodiments, axial and rotational movement relative to the coupler body 3002. The coupler 3000 may be configured such that movement of the coupler body 3002 relative to the coupler interface 3001 in any axial direction may be at least inhibited (e.g., prevented). In some embodiments, the coupler 3000 may be configured such that the coupler body 3002 may freely rotate relative to the coupler interface 3001. In such a configuration, the surgical instrument SI coupled to the coupler body 3002 may freely rotate relative to the coupler interface 3001 to which the coupler body 3002 is coupled, and any axial movement relative to the coupler interface 3001 to which the coupler body 3002 is coupled may be at least inhibited (e.g., prevented).

[0301] In other embodiments, the coupler 300 may be configured such that when at least a threshold force is applied to the surgical instrument SI relative to the...

Claims

1. An adapter device for detachably coupling a surgical instrument having a handle and an elongate shaft to a distal end of a robotic arm of a co-manipulating surgical system to assist in performing laparoscopic surgery using the surgical instrument, the distal end of the robotic arm including an adapter interface configured to detachably couple to the adapter device, the adapter device comprising: a lumen sized and shaped to receive the elongate shaft of the surgical instrument therein; and an adapter body configured to transition between an open state and a closed state, in the open state, the elongate shaft being slidably movable within the lumen; in the closed state, inhibiting longitudinal movement of the elongate shaft relative to the adapter body while allowing rotational movement of the elongate shaft relative to the adapter body in response to movement at the handle of the surgical instrument, wherein when the adapter body is coupled to the adapter interface, the adapter body is configured to rotate relative to the distal end of the robotic arm through the adapter interface to self-align the lumen with the elongate shaft as the elongate shaft is inserted into the lumen.

2. An adapter device for detachably coupling a surgical instrument having an elongate shaft to a distal end of a robotic arm of a co-manipulating surgical system to assist in performing laparoscopic surgery using the surgical instrument, the distal end of the robotic arm including an adapter interface having a protrusion, the adapter device comprising: an adapter body configured to detachably couple to the adapter interface and the elongate shaft of the surgical instrument, the adapter body comprising: a groove configured to receive the protrusion of the adapter interface; a lumen sized and shaped to receive the elongate shaft therein; and a switch configured to transition between an unlocked position and a locked position, the switch including an engagement portion configured to engage the elongate shaft when the elongate shaft is placed within the lumen and the switch is in the locked position, thereby fixing the elongate shaft within the lumen, wherein when the adapter body is coupled to the adapter interface and the elongate shaft is placed within the lumen, the robotic arm is configured to be freely movable in response to movement at the handle of the surgical instrument.

3. The adapter device according to claim 1, further comprising a switch configured to transition between an unlocked position and a locked position, the switch including an engagement portion configured to apply a frictional force to the elongate shaft when the elongate shaft is placed within the lumen and the switch is in the locked position, thereby allowing rotational movement of the elongate shaft within the lumen while prohibiting translational movement of the elongate shaft relative to the adapter body.

4. The adapter device according to claim 2 or 3, further comprising an inclined surface including: a first valley configured to engage the switch in the unlocked position; A second valley configured to engage the switch in the locked position; and A peak placed between the first valley and the second valley, the peak configured to allow the switch to transition between the unlocked position and the locked position when a force applied to the switch exceeds a predetermined force threshold.

5. The coupler device according to any one of claims 2 to 4, wherein the switch includes a handle configured to be actuated to transition the switch between the unlocked position and the locked position.

6. The coupler device according to any one of claims 3 to 5, wherein, When the coupler body is coupled to the coupler interface and the elongate shaft is placed within the lumen, the robotic arm is configured to be freely movable in response to movement at the handle of the surgical instrument.

7. The coupler device according to any one of the preceding claims, wherein the coupler body includes one or more tapered surfaces configured to guide the elongate shaft into the lumen and to facilitate self-alignment of the lumen with the elongate shaft by rotating the coupler body relative to the distal end of the robotic arm via the coupler interface as the elongate shaft is inserted into the lumen along the one or more tapered surfaces.

8. The coupler device according to any one of the preceding claims, further comprising a clamp configured to transition between an unlocked state allowing the lumen to receive the elongate shaft and a locked state in which the clamp secures the elongate shaft within the lumen.

9. The coupler device according to claim 8, wherein the clamp is configured to bias towards the locked state.

10. The coupler device according to claim 8 or 9, wherein at least a portion of the clamp includes a tapered surface configured to guide the elongate shaft into the lumen and to facilitate transition of the clamp from the locked state to the unlocked state in response to a force applied by the elongate shaft to the tapered surface as the elongate shaft is inserted into the lumen.

11. The coupler device according to any one of claims 8 to 10, wherein the clamp includes a handle portion configured to be actuated to transition the clamp from the locked state to the unlocked state.

12. The coupler device according to any one of the preceding claims, further comprising: A bracket slidably placed within the coupler body, the bracket including a friction pad configured to define at least a portion of the lumen, wherein the bracket is configured to bias in a direction towards the lumen such that when the elongate shaft is placed within the lumen, the friction pad is configured to engage the elongate shaft.

13. The coupler device according to claim 12, wherein the coupler interface includes a repulsive magnet, and wherein the bracket includes a magnet such that the repulsive magnet is configured to apply a magnetic force to the magnet to bias the bracket in the direction towards the lumen.

14. The coupler device according to claim 13, wherein the bracket includes a harness configured to couple with the magnet, and the size and shape of the harness are slidably placed within a channel of the coupler body.

15. The coupler device according to any one of claims 12 to 14, further comprising: A clamp pivotally coupled to the coupler body by a rod, the clamp configured to transition between an unlocked state allowing the lumen to receive the elongate shaft and a locked state in which the clamp secures the elongate shaft within the lumen, wherein the bracket includes one or more supports coupled to the friction pad, and each of the one or more supports includes a channel sized and shaped to slidably receive the rod therethrough such that the bracket is configured to be slidably placed within the coupler body along the rod.

16. The coupler device according to claim 1, wherein the coupler body includes a groove configured to receive a protrusion of the coupler interface.

17. The coupler device according to any one of claims 2 to 16, further comprising: One or more locking arms configured to transition between a locked configuration in which at least a portion of the one or more locking arms extends within the groove of the coupler body and an unlocked configuration in which the one or more locking arms do not extend within the groove, such that when the one or more locking arms are in the unlocked configuration, the groove is allowed to receive the protrusion of the coupler interface, wherein when the protrusion is placed within the groove and the locking arms are in the locked configuration, at least a portion of the one or more locking arms extends within one or more indentations of the protrusion of the coupler interface, thereby securing the coupler body to the coupler interface.

18. The coupler device according to claim 17, wherein the one or more locking arms are biased toward the locked configuration.

19. The coupler device according to claim 17 or 18, wherein each of the one or more locking arms includes a handle portion configured to be actuated to transition the one or more locking arms from the locked configuration to the unlocked configuration.

20. The coupler device according to any one of claims 2 to 19, wherein the protrusion of the coupler interface has a first geometry and the groove of the coupler body has a second geometry corresponding to the first geometry such that when the protrusion is received by the groove, rotational movement between the coupler body and the coupler interface is prohibited.

21. The coupler device according to any one of the preceding claims, wherein the coupler body is configured to receive a sterile drape between the coupler body and the coupler interface when the coupler body is coupled to the coupler interface.

22. A co-manipulation surgical system comprising a robotic arm and the coupler device according to any one of the preceding claims.

23. The co-manipulated surgical system according to claim 22, wherein the co-manipulated surgical system has two robotic arms, each robotic arm configured to be coupled to a coupler device.

24. A co-manipulated surgical system for assisting in performing laparoscopic surgery using a surgical instrument having a handle, an operating end, and an elongate shaft therebetween, the co-manipulated surgical system comprising: A robotic arm including a proximal end, a distal end configured to be detachably coupled to the surgical instrument, a plurality of linkages, and a plurality of joints between the proximal end and the distal end, the distal end of the robotic arm including a coupler interface; And A coupler body configured to be detachably coupled to the coupler interface, the coupler body including an inner cavity sized and shaped to receive the elongate shaft of the surgical instrument therein, the coupler body configured to transition between an open state and a closed state, in the open state, the elongate shaft being slidably movable within the inner cavity, in the closed state, inhibiting longitudinal movement of the elongate shaft relative to the coupler body while allowing rotational movement of the elongate shaft relative to the coupler body in response to movement at the handle of the surgical instrument, Wherein when the coupler body is coupled to the coupler interface, the coupler body is configured to rotate relative to the distal end of the robotic arm through the coupler interface to self-align the inner cavity with the elongate shaft when the elongate shaft is inserted into the inner cavity, and Wherein when the coupler body is coupled to the coupler interface in the closed state, the robotic arm is allowed to move freely in response to movement at the handle of the surgical instrument to perform laparoscopic surgery.

25. The co-manipulated surgical system according to claim 24, wherein the coupler body is disposable after a single laparoscopic surgery.

26. The co-manipulated surgical system according to claim 24, further comprising: A switch configured to transition between an unlocked position and a locked position, the switch including an engaging portion configured to engage the elongate shaft when the elongate shaft is placed within the inner cavity and the switch is in the locked position, thereby fixing the elongate shaft within the inner cavity, Wherein when the elongate shaft is placed within the inner cavity and the switch is in the locked position, the engaging portion is configured to apply a frictional force to the elongate shaft, the frictional force configured to allow rotational movement of the elongate shaft relative to the coupler body while inhibiting longitudinal movement of the elongate shaft relative to the coupler body.

27. The co-manipulation surgical system according to claim 26, wherein, When the coupler body is coupled to the coupler interface and the elongate shaft is placed within the inner cavity, the robotic arm is configured to move freely in response to movement at the handle of the surgical instrument.

28. The co-manipulation surgical system according to claim 26, wherein the switch includes a handle portion configured to be actuated to switch the switch between the unlocked position and the locked position.

29. The co-manipulation surgical system according to claim 26, wherein the coupler body includes an inclined surface, and the inclined surface includes: a first valley configured to engage with the switch in the unlocked position; a second valley configured to engage with the switch in the locked position; and a peak between the first valley and the second valley, the peak configured to allow the switch to switch between the unlocked position and the locked position when a force applied to the switch exceeds a predetermined force threshold.

30. The co-manipulation surgical system according to claim 24, wherein the coupler body further includes a bracket slidably disposed within the coupler body, the bracket including a contact surface configured to define at least a portion of the inner cavity, and wherein the bracket is configured to be biased in a direction toward the inner cavity such that when the elongated shaft is placed within the inner cavity, the contact surface is configured to engage with the elongated shaft.

31. The co-manipulation surgical system according to claim 30, wherein the coupler interface includes a repulsive magnet, and wherein the bracket includes a magnet such that the repulsive magnet is configured to apply a magnetic force to the magnet to bias the bracket in a direction toward the inner cavity.

32. The co-manipulation surgical system according to claim 31, wherein the bracket includes a harness configured to be coupled to the magnet, and the size and shape of the harness are slidably placed within a channel of the coupler body.

33. The co-manipulation surgical system according to claim 30, further comprising: a clamp pivotally coupled to the coupler body by a rod, the clamp configured to switch between an unlocked state allowing the inner cavity to receive the elongated shaft and a locked state in which the clamp fixes the elongated shaft within the inner cavity, wherein the bracket includes one or more supports coupled to the contact surface, and each of the one or more supports includes a channel sized and shaped to slidably receive the rod therethrough such that the bracket is configured to be slidably placed within the coupler body along the rod.

34. The co-manipulation surgical system according to claim 24, wherein the coupler interface includes a protrusion, and wherein the coupler body includes a groove configured to receive the protrusion of the coupler interface.

35. The co-manipulation surgical system according to claim 34, wherein the protrusion includes one or more indentations, and wherein the coupler body includes: one or more locking arms configured to switch between a locked configuration in which at least a portion of the one or more locking arms extends within the groove of the coupler body and an unlocked configuration in which the one or more locking arms do not extend within the groove of the coupler body. Wherein, when the one or more locking arms are in the unlocked configuration, the protrusion of the coupler interface is configured to be received by the groove of the coupler body, and wherein, when the protrusion is placed within the groove and the locking arms are in the locked configuration, at least a portion of the one or more locking arms extends within the one or more indentations of the protrusion, thereby fixing the coupler body to the coupler interface.

36. The co-manipulating surgical system according to claim 35, wherein the one or more locking arms bias towards the locked configuration.

37. The co-manipulating surgical system according to claim 35, wherein each of the one or more locking arms includes a handle portion configured to be actuated to convert the one or more locking arms from the locked configuration to the unlocked configuration.

38. The co-manipulating surgical system according to claim 34, wherein the protrusion of the coupler interface has a first geometry, and the groove of the coupler body has a second geometry corresponding to the first geometry such that rotational movement between the coupler body and the coupler interface is prohibited when the protrusion is received by the groove.

39. The co-manipulating surgical system according to claim 34, wherein the coupler interface includes one or more additional protrusions having a first geometry, and wherein the coupler body includes one or more additional grooves having a second geometry such that rotational movement between the coupler body and the coupler interface is prohibited when the one or more additional protrusions are received by the one or more additional grooves.

40. The co-manipulating surgical system according to claim 24, wherein the coupler body and the coupler interface are configured to receive a sterile drape therebetween such that the sterile drape prevents contact between the surgical instrument and the robotic arm during laparoscopic surgery.

41. The co-manipulating surgical system according to claim 24, wherein the coupler body includes one or more tapered surfaces configured to guide the elongate shaft into the lumen and, when the elongate shaft is inserted into the lumen along the one or more tapered surfaces, facilitate self-alignment of the lumen with the elongate shaft by rotating the coupler body relative to the distal end of the robotic arm via the coupler interface.

42. The co-manipulating surgical system according to claim 24, wherein the coupler body includes a clamp configured to transition between an unlocked state allowing the lumen to receive the elongate shaft and a locked state in which the clamp secures the elongate shaft within the lumen.

43. The co-manipulating surgical system according to claim 42, wherein the clamp is configured to bias towards the locked state.

44. The co-manipulating surgical system according to claim 43, wherein at least a portion of the clamp includes a tapered surface configured to guide the elongate shaft into the lumen and, in response to a force applied by the elongate shaft to the tapered surface as the elongate shaft is inserted into the lumen, facilitate conversion of the clamp from the locked state to the unlocked state.

45. A method for using a robotic arm configured to be removably coupled to a surgical instrument having a handle, an operating end, and an elongate shaft therebetween, the method comprising: Removably coupling a coupler body to a coupler interface at a distal end of the robotic arm; Inserting the elongate shaft of the surgical instrument into a lumen of the coupler body; Converting the coupler body from an open state, in which the elongate shaft is slidably movable within the lumen, to a closed state, in which longitudinal movement of the elongate shaft relative to the coupler body is inhibited while allowing rotational movement of the elongate shaft relative to the coupler body in response to movement at the handle of the surgical instrument; And When the coupler body is coupled to the coupler interface in the closed state, freely moving the robotic arm by moving the handle of the surgical instrument, Wherein the coupler body rotates relative to the distal end of the robotic arm through the coupler interface to self-align the lumen with the elongate shaft as the elongate shaft is inserted into the lumen.

46. The method according to claim 45, wherein removably coupling the coupler body to the coupler interface comprises: Actuating one or more locking arms of the coupler body to convert the one or more locking arms from a locked configuration in which at least a portion of the one or more locking arms extends within a recess of the coupler body to an unlocked configuration in which the one or more locking arms do not extend within the recess; Inserting a protrusion of the coupler interface into the recess of the coupler body; And Releasing the one or more locking arms to convert the one or more locking arms from the unlocked configuration to the locked configuration such that at least a portion of the one or more locking arms extends within one or more indentations of the protrusion, thereby fixing the coupler body to the coupler interface.

47. The method according to claim 45, wherein inserting the elongate shaft of the surgical instrument into the lumen of the coupler body comprises guiding the elongate shaft into the lumen along one or more tapered surfaces of the coupler body.

48. The method according to claim 47, wherein guiding the elongate shaft into the lumen along one or more tapered surfaces of the coupler body causes the coupler body to rotate relative to the distal end of the robotic arm through the coupler interface as the elongate shaft is inserted along the one or more tapered surfaces to self-align the lumen with the elongate shaft.

49. The method according to claim 45, wherein inserting the elongate shaft of the surgical instrument into the lumen of the coupler body comprises: Actuating a clamp of the coupler body to transition the clamp from a locked state to an unlocked state, in which the lumen permits receipt of the elongate shaft; Inserting the elongate shaft of the surgical instrument into the lumen; and Releasing the clamp to transition the clamp from the unlocked state to the locked state such that the clamp secures the elongate shaft within the lumen.

50. The method according to claim 45, wherein transitioning the coupler body from the open state to the closed state comprises transitioning a switch of the coupler body from an unlocked position in which the elongate shaft is slidably movable within the lumen to a locked position in which an engagement portion of the switch engages the elongate shaft disposed within the lumen, thereby inhibiting longitudinal movement of the elongate shaft relative to the coupler body while permitting rotational movement of the elongate shaft relative to the coupler body.

51. The method according to claim 50, wherein, When the elongate shaft is disposed within the lumen and the switch is in the locked position, the engagement portion applies a frictional force to the elongate shaft, the frictional force configured to permit rotational movement of the elongate shaft relative to the coupler body while inhibiting longitudinal movement of the elongate shaft relative to the coupler body.

52. The method according to claim 45, wherein the coupler interface includes a repulsive magnet, and wherein the coupler body includes a bracket slidably disposed within the coupler body, the bracket including a magnet and a contact surface configured to define at least a portion of the lumen such that when the coupler body is removably coupled to the coupler interface, the repulsive magnet applies a magnetic force to the magnet to bias the bracket in a direction toward the lumen.

53. The method according to claim 45, further comprising placing a sterile drape between the coupler body and the coupler interface prior to removably coupling the coupler body to the coupler interface.

54. A co-manipulation surgical system for assisting in laparoscopic surgery using a surgical instrument, the surgical instrument having a handle, an operating end, and an elongate shaft therebetween, the co-manipulation surgical system comprising: A robotic arm including a proximal end, a distal end configured to be removably coupled to the surgical instrument, a plurality of linkages, and a plurality of joints between the proximal end and the distal end, the distal end of the robotic arm including a coupler interface; and The coupler body is configured to be detachably coupled to the coupler interface. The coupler body includes a lumen sized and shaped to receive the elongate shaft of the surgical instrument therein. The coupler body is configured to transition between an open state and a closed state. In the open state, the elongate shaft is slidably movable within the lumen. In the closed state, longitudinal movement of the elongate shaft relative to the coupler body is inhibited while rotational movement of the elongate shaft relative to the coupler body is permitted in response to movement at the handle of the surgical instrument. Wherein, when the coupler body is coupled to the coupler interface in the closed state, the robotic arm is permitted to move freely in response to movement at the handle of the surgical instrument to perform laparoscopic surgery.

55. The co-manipulated surgical system according to claim 54, wherein the coupler body is disposable after a single laparoscopic surgery.

56. The co-manipulated surgical system according to claim 54, further comprising: A switch configured to transition between an unlocked position and a locked position. The switch includes an engagement portion configured to engage the elongate shaft when the elongate shaft is placed within the lumen and the switch is in the locked position, thereby fixing the elongate shaft within the lumen. Wherein, when the coupler body is coupled to the coupler interface, the elongate shaft is placed within the lumen and the switch is in the locked position, the robotic arm is configured to move freely in response to movement at the handle of the surgical instrument.

57. The co-manipulation surgical system according to claim 56, wherein, When the elongate shaft is placed within the lumen and the switch is in the locked position, the engagement portion is configured to apply a frictional force to the elongate shaft. The frictional force is configured to permit rotational movement of the elongate shaft relative to the coupler body while inhibiting longitudinal movement of the elongate shaft relative to the coupler body.

58. The co-manipulated surgical system according to claim 56, wherein the switch includes a handle portion configured to be actuated to transition the switch between the unlocked position and the locked position.

59. The co-manipulated surgical system according to claim 56, wherein the coupler body further includes a bracket slidably disposed within the coupler body. The bracket includes a contact surface configured to define at least a portion of the lumen. And wherein the bracket is configured to be biased in a direction toward the lumen such that when the elongate shaft is placed within the lumen, the contact surface is configured to engage the elongate shaft.

60. The co-manipulated surgical system according to claim 59, wherein the coupler interface includes a repulsive magnet, and wherein the bracket includes a magnet such that the repulsive magnet is configured to apply a magnetic force to the magnet, thereby biasing the bracket in a direction toward the lumen.

61. The co-manipulation surgical system according to claim 60, wherein the bracket includes a harness configured to be coupled to the magnet, and the harness is sized and shaped to be slidably placed within a channel of the coupler body.

62. The co-manipulation surgical system according to claim 59, wherein, When the elongate shaft is placed within the lumen and the switch is in the locked position, the contact surface is configured to apply a frictional force to the elongate shaft, and wherein the frictional force is configured to facilitate allowing rotational movement of the elongate shaft relative to the coupler body while inhibiting translational movement of the elongate shaft relative to the coupler body.

63. The co-manipulation surgical system according to claim 59, further comprising: a clamp pivotally coupled to the coupler body by a rod, the clamp configured to transition between an unlocked state that allows the lumen to receive the elongate shaft and a locked state that fixes the elongate shaft within the lumen, wherein the bracket includes one or more supports coupled to the contact surface, each of the one or more supports including a channel sized and shaped to slidably receive the rod therethrough such that the bracket is configured to be slidably placed within the coupler body along the rod.

64. The co-manipulation surgical system according to claim 54, wherein the coupler interface includes a protrusion, and wherein the coupler body includes a groove configured to receive the protrusion of the coupler interface.

65. The co-manipulation surgical system according to claim 64, wherein the protrusion includes one or more indentations, and wherein the coupler body includes: one or more locking arms configured to transition between a locked configuration in which at least a portion of the one or more locking arms extends within the groove of the coupler body and an unlocked configuration in which the one or more locking arms do not extend within the groove of the coupler body, wherein, when the one or more locking arms are in the unlocked configuration, the protrusion of the coupler interface is configured to be received by the groove of the coupler body, and wherein, when the protrusion is placed within the groove and the locking arms are in the locked configuration, at least a portion of the one or more locking arms extends within the one or more indentations of the protrusion, thereby fixing the coupler body to the coupler interface.

66. The co-manipulation surgical system according to claim 65, wherein the one or more locking arms bias towards the locked configuration.

67. The co-manipulation surgical system according to claim 65, wherein each of the one or more locking arms includes a handle portion configured to be actuated to transition the one or more locking arms from the locked configuration to the unlocked configuration.

68. The co-manipulating surgical system according to claim 64, wherein the protrusion of the coupler interface has a first geometry, and the groove of the coupler body has a second geometry corresponding to the first geometry, such that when the protrusion is received by the groove, rotational movement between the coupler body and the coupler interface is prohibited.

69. The co-manipulating surgical system according to claim 64, wherein the coupler interface includes one or more additional protrusions having a first geometry, and wherein the coupler body includes one or more additional grooves having a second geometry, such that when the one or more additional protrusions are received by the one or more additional grooves, rotational movement between the coupler body and the coupler interface is prohibited.

70. The co-manipulating surgical system according to claim 54, wherein the coupler body and the coupler interface are configured to receive a sterile drape therebetween, such that the sterile drape prevents contact between the surgical instrument and the robotic arm during laparoscopic surgery.

71. The co-manipulating surgical instrument system according to claim 54, wherein the coupler body includes one or more tapered surfaces configured to guide the elongate shaft into the lumen and, as the elongate shaft is inserted into the lumen along the one or more tapered surfaces, facilitate self-alignment of the distal end of the robotic arm relative to the surgical instrument by rotating the coupler body and the coupler interface to align the lumen with the elongate shaft.

72. The co-manipulating surgical system according to claim 54, wherein the coupler body includes a clamp configured to transition between an unlocked state that permits the lumen to receive the elongate shaft and a locked state in which the clamp secures the elongate shaft within the lumen.

73. The co-manipulating surgical system according to claim 72, wherein the clamp is configured to bias towards the locked state.

74. The co-manipulating surgical system according to claim 73, wherein at least a portion of the clamp includes a tapered surface configured to guide the elongate shaft into the lumen and, as the elongate shaft is inserted into the lumen, facilitate transition of the clamp from the locked state to the unlocked state in response to a force applied by the elongate shaft to the tapered surface.

75. A method for using a robotic arm configured to be detachably coupled to a surgical instrument, the surgical instrument having a handle, an operating end, and an elongate shaft therebetween, the method comprising: detachably coupling a coupler body to a coupler interface at a distal end of the robotic arm; inserting the elongate shaft of the surgical instrument into a lumen of the coupler body; Convert the coupler body from an open state, in which the elongate shaft is slidably movable within the inner cavity, to a closed state in which longitudinal movement of the elongate shaft relative to the coupler body is inhibited while rotational movement of the elongate shaft relative to the coupler body is permitted in response to movement at the handle of the surgical instrument; and When the coupler body is coupled to the coupler interface in the closed state, freely move the robotic arm by moving the handle of the surgical instrument.

76. The method according to claim 75, wherein detachably coupling the coupler body to the coupler interface comprises: Actuating one or more locking arms of the coupler body to transition the one or more locking arms from a locked configuration in which at least a portion of the one or more locking arms extends within a groove of the coupler body to an unlocked configuration in which the one or more locking arms do not extend within the groove; Inserting a protrusion of the coupler interface into the groove of the coupler body; and Releasing the one or more locking arms to transition the one or more locking arms from the unlocked configuration to the locked configuration such that at least a portion of the one or more locking arms extends within one or more indentations of the protrusion, thereby fixing the coupler body to the coupler interface.

77. The method according to claim 75, wherein inserting the elongate shaft of the surgical instrument into the inner cavity of the coupler body comprises guiding the elongate shaft into the inner cavity along one or more tapered surfaces of the coupler body.

78. The method according to claim 77, wherein guiding the elongate shaft into the inner cavity along one or more tapered surfaces of the coupler body comprises rotating the coupler body and the coupler interface to facilitate self-alignment of the inner cavity with the elongate shaft as the elongate shaft is inserted into the inner cavity along the one or more tapered surfaces.

79. The method according to claim 75, wherein inserting the elongate shaft of the surgical instrument into the inner cavity of the coupler body comprises: Actuating a clamp of the coupler body to transition the clamp from a locked state to an unlocked state in which the inner cavity is permitted to receive the elongate shaft; Inserting the elongate shaft of the surgical instrument into the inner cavity; and Releasing the clamp to transition the clamp from the unlocked state to the locked state such that the clamp secures the elongate shaft within the inner cavity.

80. The method according to claim 75, wherein converting the coupler body from the open state to the closed state comprises transitioning a switch of the coupler body from an unlocked position in which the elongate shaft is slidably movable within the inner cavity to a locked position in which an engagement portion of the switch engages the elongate shaft placed within the inner cavity, thereby inhibiting longitudinal movement of the elongate shaft relative to the coupler body while permitting rotational movement of the elongate shaft relative to the coupler body.

81. The method according to claim 80, wherein, When the elongate shaft is placed within the inner cavity and the switch is in the locked position, the engagement portion applies a frictional force to the elongate shaft, the frictional force being configured to permit rotational movement of the elongate shaft relative to the coupler body while inhibiting longitudinal movement of the elongate shaft relative to the coupler body.

82. The method of claim 75, wherein the coupler interface includes a repulsive magnet, and wherein the coupler body includes a bracket slidably disposed within the coupler body, the bracket including a magnet and a contact surface configured to define at least a portion of the inner cavity such that when the coupler body is detachably coupled to the coupler interface, the repulsive magnet applies a magnetic force to the magnet to bias the bracket in a direction toward the inner cavity.

83. The method of claim 75, further comprising placing a sterile drape between the coupler body and the coupler interface prior to detachably coupling the coupler body to the coupler interface.

84. A method of using a robotic arm configured to be detachably coupled to a surgical instrument having a handle, an operating end, and an elongate shaft therebetween, the method being performed adjacent a patient-containing bedside, the method comprising: Positioning the robotic arm at the bedside; Coupling the surgical instrument to the robotic arm, the robotic arm including a proximal end, a distal end, a plurality of links, and a plurality of joints between the proximal end and the distal end; and Moving the robotic arm freely by moving the handle of the surgical instrument coupled thereto while the robotic arm remains positioned at the bedside.

85. The method of claim 84, wherein the robotic arm remains at the bedside when using the surgical instrument coupled to the robotic arm.

86. The method of claim 84, wherein coupling the surgical instrument to the robotic arm includes coupling the surgical instrument to the robotic arm using a purely mechanical coupling.

87. The method of claim 84, wherein coupling the surgical instrument to the robotic arm includes coupling the robotic arm only to the elongate shaft of the surgical instrument.

88. The method of claim 84, wherein coupling the surgical instrument to the robotic arm includes coupling the surgical instrument to the robotic arm while keeping the handle of the surgical instrument fully exposed for contact with a surgeon's hand.

89. The method of claim 84, wherein coupling the surgical instrument to the robotic arm includes coupling the surgical instrument to the robotic arm while positioning the robotic arm at the bedside.

90. The method of claim 84, wherein coupling the surgical instrument to the robotic arm includes detachably coupling a coupler body to a coupler interface disposed at the distal end of the robotic arm, and detachably coupling the surgical instrument to the coupler body.

91. The method according to claim 90 further includes transitioning the coupler body from an open state, in which the elongate shaft is slidably movable within the lumen of the coupler body, to a closed state, in which the robotic arm is allowed to move freely in response to movement at the handle of the surgical instrument.

92. The method according to claim 91, wherein, When the coupler body is coupled to the coupler interface in the closed state, longitudinal movement of the elongate shaft relative to the coupler body is inhibited while rotational movement of the elongate shaft relative to the coupler body is allowed in response to movement at the handle of the surgical instrument.

93. The method according to claim 92, wherein, When the coupler body is coupled to the coupler interface in the closed state, the coupler body applies a frictional force to the elongate shaft that is sufficient to allow rotational movement of the elongate shaft relative to the coupler body while inhibiting longitudinal movement of the elongate shaft relative to the coupler body.

94. The method according to claim 90, wherein, Detachably coupling the surgical instrument to the coupler body includes detachably coupling the coupler body to a fixed point along the elongate shaft to provide a consistent reference point for force calculations of the surgical instrument.

95. The method according to claim 90 further includes placing a sterile drape between the coupler body and the coupler interface prior to detachably coupling the coupler body to the coupler interface.

96. The method according to claim 90 further includes processing the coupler body after a single laparoscopic surgical procedure.

97. The method according to claim 84, wherein the robotic arm is not remotely operated by user input received at a remote surgeon console.

98. The method according to claim 84, wherein freely moving the robotic arm by moving the handle of the surgical instrument coupled thereto includes applying a force greater than a predetermined threshold at the robotic arm by the surgical instrument, thereby automatically switching the robotic arm to a co-manipulation mode in which impedance is applied to the robotic arm to handle the weight of the surgical instrument and the robotic arm.

99. The method according to claim 98 further includes adjusting the predetermined threshold of the force applied to the robotic arm by an operatively coupled graphical user interface to cause the robotic arm to automatically switch to the co-manipulation mode.

100. The method according to claim 98, wherein the plurality of joints of the robotic arm include one or more powered joints that are operatively coupled to one or more motors disposed in a base coupled to the proximal end of the robotic arm, the method further including measuring the current of the one or more motors, the current indicating the force applied to the robotic arm by the surgical instrument.

101. The method according to claim 100, wherein the impedance applied to the robotic arm to handle the weight of the surgical instrument and the robotic arm is applied through the one or more actuated joints of the robotic arm.

102. The method according to claim 84, further comprising maintaining the movement of the robotic arm within a predetermined amount for at least a predetermined dwell time, thereby automatically switching the robotic arm to a passive mode in which the robotic arm maintains a static position.

103. The method according to claim 102, further comprising adjusting at least one of a predetermined movement amount or a predetermined dwell time of the robotic arm through a graphical user interface operatively coupled to the robotic arm to cause the robotic arm to automatically switch to the passive mode.

104. The method according to claim 84, further comprising moving the robotic arm by moving the handle of the surgical instrument outside a predefined haptic barrier, thereby automatically switching the robotic arm to a haptic mode in which impedance is applied to the robotic arm to make the movement of the robotic arm in response to movement at the handle of the surgical instrument more viscous in the haptic mode.

105. The method according to claim 104, further comprising adjusting the position of the predefined haptic barrier through a graphical user interface operatively coupled to the robotic arm.

106. The method according to claim 84, further comprising selecting an identification of the surgical instrument coupled to the robotic arm through a graphical user interface operatively coupled to the robotic arm.

107. The method according to claim 84, wherein a proximal end of the robotic arm is coupled to a base, and the base is coupled to the platform through a step assembly configured to move the base relative to the platform with at least two degrees of freedom, the method further comprising adjusting at least one of a vertical height or a horizontal position of the robotic arm relative to the platform through the step assembly.

108. The method according to claim 107, wherein adjusting at least one of a vertical height or a horizontal position of the robotic arm through the step assembly includes providing user input by at least one of: a graphical user interface operatively coupled to the step assembly, or a user applying a force on the robotic arm in a distal region of the robotic arm in at least one of the at least two degrees of freedom.

109. The method according to claim 107, wherein the platform includes a plurality of wheels, and wherein positioning the robotic arm at a bedside to perform a laparoscopic surgical procedure includes moving the platform relative to the bed through the plurality of wheels.

110. The method according to claim 109, further comprising releasing a braking mechanism of the plurality of wheels to allow the platform to move through the plurality of wheels.

111. The method according to claim 107, further comprising displaying a virtual map, the virtual map including a graphical representation of the platform relative to the bed within an area surrounding the platform to facilitate positioning the robotic arm at the bedside to perform laparoscopic surgery.

112. The method according to claim 84, further comprising adjusting at least one of a height or a direction of an optical sensor so as to optimize a field of view of a surgical scene of the optical sensor, the field of view of the surgical scene of the optical sensor including at least one of the robotic arm, the surgical instrument coupled to the robotic arm, or the bed.

113. The method according to claim 84, further comprising: selecting a laparoscope, a retraction tool, a grasping tool, or a surgical cutting tool, wherein coupling the surgical instrument to the robotic arm includes coupling the laparoscope, the retraction tool, the grasping tool, or the surgical cutting tool to the robotic arm.

114. A co-manipulation surgical system for assisting in laparoscopic surgery using a surgical instrument, the surgical instrument having a handle, an operating end, and an elongate shaft therebetween, the co-manipulation surgical system comprising: a robotic arm including a proximal end operatively coupled to a base, a distal end configured to be detachably coupled to the surgical instrument, a plurality of links, and a plurality of joints; a platform coupled to the base, the platform configured to move the base in at least one degree of freedom; a plurality of motors operatively coupled to at least some of the plurality of joints; and one or more sensors configured to collect sensor data, the sensor data including at least one of 3D depth data or pixel image data; and a controller operatively coupled to the robotic arm and the one or more sensors and configured to allow the robotic arm to move freely in response to movement at the handle of the surgical instrument to perform laparoscopic surgery using the surgical instrument, the controller being programmed to: identify at least one of a position or a direction of one or more objects in the operating room based on the sensor data from the one or more sensors; estimate a relative distance between the one or more objects and at least one of the base or the robotic arm when at least one of the base or the robotic arm moves in the operating room; and if the estimated relative distance is close to a predetermined threshold, apply torque or impedance to at least some of the plurality of joints of the robotic arm via the plurality of motors to reposition the robotic arm or stop movement of the robotic arm so as to avoid a collision between the one or more objects and at least one of the base or the robotic arm.

115. The co-manipulation surgical system according to claim 114, wherein the controller is configured to: detect movement of the distal end of the robotic arm in a first direction in response to a first force applied by a user to the distal end of the robotic arm; In response to detecting movement of the distal end of the robotic arm in a first direction, move the base along the first direction by the platform; and If the first force applied by the user to the distal end of the robotic arm is below a predetermined threshold, stop the platform from moving the base along the first direction.

116. The co-manipulative surgical system according to claim 115, wherein the controller is configured to move the base along the first direction by the platform if the first force applied to the distal end of the robotic arm exceeds a predetermined force threshold.

117. The co-manipulative surgical system according to claim 114, wherein the controller is configured to identify a plane of the one or more objects in the operating room based on the sensor data from the one or more sensors, and wherein the controller is configured to estimate a relative distance between the one or more objects and at least one of the base or the robotic arm based on the plane of the one or more objects.

118. The co-manipulative surgical system according to claim 114, wherein the controller is configured to: Determine the type of laparoscopic surgery to be performed; Based on the sensor data from the one or more sensors, determine at least one of the position or orientation of the trocar port; And Apply torque to at least some of the plurality of joints of the robotic arm through the plurality of motors to automatically position the robotic arm in a predetermined configuration relative to the trocar port based on the type of laparoscopic surgery to be performed.

119. The co-manipulative surgical system according to claim 114, wherein the one or more objects include an operating table.

120. A co-manipulative surgical system for assisting in laparoscopic surgery using a surgical instrument, the surgical instrument having a handle, an operating end, and an elongate shaft therebetween, the co-manipulative surgical system comprising: A robotic arm including a proximal end, a distal end configured to be detachably coupled to the surgical instrument, a plurality of links, and a plurality of joints; One or more sensors configured to collect sensor data, the sensor data including at least one of 3D depth data or pixel image data; And A controller operatively coupled to the robotic arm and the one or more sensors and configured to allow the robotic arm to move freely in response to movement at the handle of the surgical instrument to perform laparoscopic surgery using the surgical instrument, the controller programmed to: Based on the sensor data from the one or more sensors, determine at least one of the position or orientation of the trocar port relative to the robotic arm; When the operating end of the surgical instrument is inserted through the trocar port, detect movement of the trocar port based on the sensor data from the one or more sensors; And Relocate the robotic arm to maintain the position of the operating end of the surgical instrument relative to the trocar port during movement of the trocar port.

121. The co-manipulated surgical system according to claim 120, wherein the controller is configured to detect movement of the cannula port in response to movement of the operating table.

122. The co-manipulated surgical system according to claim 120, wherein the controller is configured to detect movement of the cannula port in response to movement of the patient's body due to the patient's breathing.

123. The co-manipulated surgical system according to claim 120, wherein the controller is configured to retract the operating end of the surgical instrument within the cannula port by the distal end of the robotic arm before repositioning the robotic arm, so as to maintain the position of the operating end of the surgical instrument relative to the cannula port during movement of the cannula port.

124. A co-manipulated surgical system for assisting in laparoscopic surgery using a surgical instrument, the surgical instrument having a handle, an operating end, and an elongate shaft therebetween, the co-manipulated surgical system comprising: A robotic arm including a proximal end configured to be detachably coupled to a cart, a distal end configured to be detachably coupled to the surgical instrument, a plurality of linkages, and a plurality of joints; An optical scanner configured to measure depth data; And A controller operatively coupled to the robotic arm and the optical scanner and configured to allow the robotic arm to move freely in response to movement at the handle of the surgical instrument, so as to perform laparoscopic surgery using the surgical instrument, the controller being programmed to: Move the robotic arm in an expected predefined movement pattern relative to the cart according to a preprogrammed routine; Compare depth data from the optical scanner indicating the actual movement of the robotic arm in response to the preprogrammed routine with the expected predefined movement pattern, and generate a degree of error indicating a deviation between the actual movement of the robotic arm and the expected predefined movement pattern; And Execute an optimization algorithm configured to reduce the degree of error such that the deviation between the actual movement of the robotic arm and the expected predefined movement pattern is reduced.

125. The co-manipulated surgical system according to claim 124, wherein the controller is configured to allow the robotic arm to move freely in response to movement at the handle of the surgical instrument to perform laparoscopic surgery using the surgical instrument when the degree of error drops below a predetermined threshold.

126. A co-manipulated surgical system for assisting in laparoscopic surgery using a surgical instrument, the surgical instrument having a handle, an operating end, and an elongate shaft therebetween, the co-manipulated surgical system comprising: A robotic arm including a proximal end and a distal end configured to be detachably coupled to the surgical instrument, a plurality of linkages, and a plurality of joints; And A controller operatively coupled to the robotic arm and the laparoscope, the laparoscope configured to generate a video feed, the controller configured to allow the robotic arm to be freely movable in response to movement at the surgical instrument handle to perform laparoscopic surgery using the surgical instrument, the controller being programmed to: Overlay a virtual menu onto the video feed displayed on a display screen; Track, in the video feed, the movement of the operating end of the surgical instrument in response to movement at the surgical instrument handle to detect one or more predetermined gesture movement patterns of the operating end; And Based on detecting the one or more predetermined gesture movement patterns of the operating end relative to the virtual menu, actuate a function of the co-manipulation surgical system associated with the virtual menu.

127. The co-manipulation surgical system according to claim 126, wherein the virtual menu includes one or more menu options superimposed in at least one corner of the video feed.

128. The co-manipulation surgical system according to claim 126, wherein the function of the co-manipulation surgical system associated with the virtual menu includes adjusting a holding force threshold that needs to be exceeded to switch the robotic arm from a passive mode in which the controller holds the robotic arm in a static position to a co-manipulation mode in which the controller allows the robotic arm to be freely movable in response to movement at the surgical instrument handle to perform laparoscopic surgery using the surgical instrument.

129. The co-manipulation surgical system according to claim 126, wherein the function of the co-manipulation surgical system associated with the virtual menu includes actuating an assisted aiming scope mode in which the controller causes the laparoscope to automatically adjust at least one of a field of view or a position to assist in laparoscopic surgery.

130. The co-manipulation surgical system according to claim 126, wherein the controller is configured to overlay the virtual menu onto the video feed displayed on the display in response to user input received through a graphical user interface operatively coupled to the controller.

131. The co-manipulation surgical system according to claim 126, wherein the controller is configured to overlay the virtual menu onto the video feed displayed on the display in response to a voice command of a user.

132. The co-manipulation surgical system according to claim 126, wherein the controller is configured to overlay the virtual menu onto the video feed displayed on the display in response to actuation of an actuator placed on the robotic arm.

133. The co-manipulation surgical system according to claim 126, wherein the controller is configured to track the movement of the operating end of the surgical instrument in response to user input received through a graphical user interface operatively coupled to the controller.

134. The co-manipulation surgical system according to claim 126, wherein the controller is configured to track the movement of the operating end of the surgical instrument in response to a user's voice command.

135. The co-manipulation surgical system according to claim 126, wherein the controller is configured to track the movement of the operating end of the surgical instrument in response to the actuation of an actuator placed on the robotic arm.

136. A co-manipulation surgical system for assisting in laparoscopic surgery using a surgical instrument having a handle, an operating end, and an elongate shaft therebetween, the co-manipulation surgical system comprising: A robotic arm including a proximal end, a distal end configured to be detachably coupled to the surgical instrument, a plurality of links, and a plurality of joints; And A controller operatively coupled to the robotic arm, the controller being programmed to: Automatically switch the robotic arm between a co-manipulation mode and a passive mode, wherein in the co-manipulation mode, the controller allows the robotic arm to move freely in response to movement at the handle of the surgical instrument for performing laparoscopic surgery using the surgical instrument; and in the passive mode, the controller causes the robotic arm to maintain a static position; And Cause vibration at the distal end of the robotic arm, the vibration indicating a switch of the robotic arm from the co-manipulation mode to the passive mode.

137. The co-manipulation surgical system according to claim 136, wherein the vibration is configured to be perceivable by a user holding the handle of the surgical instrument while causing negligible movement at the operating end of the surgical instrument.

138. The co-manipulation surgical system according to claim 136, wherein the controller is configured to cause a second vibration at the distal end of the robotic arm when the surgical instrument is coupled to the distal end of the robotic arm, the second vibration indicating that the surgical instrument is coupled to the distal end of the robotic arm.

139. The co-manipulation surgical system according to claim 136, wherein the controller is configured to cause the robotic arm to switch to the passive mode in response to determining that the movement of the robotic arm due to movement at the handle of the surgical instrument is less than a predetermined amount during at least a predetermined dwell time.

140. The co-manipulation surgical system according to claim 136, wherein the controller is configured to cause the robotic arm to switch to the co-manipulation mode in response to determining that the force applied to the robotic arm due to a force applied to the handle of the surgical instrument exceeds a predetermined threshold.

141. The co-manipulation surgical system according to claim 136, wherein the controller is configured to apply a first impedance to the robotic arm in the co-manipulation mode to handle the weights of the surgical instrument and the robotic arm.

142. The co-manipulated surgical system according to claim 136, wherein the controller is configured to generate an audible alarm indicating that the robotic arm has switched from the co-manipulation mode to the passive mode.

143. The co-manipulated surgical system according to claim 136, wherein the robotic arm includes a base operatively coupled to the proximal end of the robotic arm, and the system further includes: a plurality of motors disposed within the base, the plurality of motors being operatively coupled to at least some of the plurality of joints, wherein the controller is programmed to actuate at least one of the plurality of motors to cause vibration at the distal end of the robotic arm.

144. A co-manipulated surgical system for assisting in laparoscopic surgery performed using a laparoscope, the laparoscope having a handle, an operating end configured to collect a video feed including a field of view, and an elongate shaft therebetween, the co-manipulated surgical system including: a robotic arm including a proximal end, a distal end configured to be detachably coupled to the laparoscope, a plurality of links, and a plurality of joints; an optical scanner configured to measure depth data; and a controller operatively coupled to the robotic arm and the optical scanner, the controller being programmed to: during movement of the laparoscope, compare movement of the laparoscope based on the depth data from the optical scanner with movement of the field of view of the laparoscope based on the video feed collected from the operating end of the laparoscope; and during movement of the laparoscope, identify the type of the laparoscope based on movement of the field of view of the laparoscope.

145. The co-manipulated surgical system according to claim 144, wherein the controller is configured to execute a pre-programmed routine in a calibration mode to move the laparoscope in a pre-defined movement pattern according to the pre-programmed routine.

146. The co-manipulated surgical system according to claim 145, wherein the pre-defined movement pattern includes circular motion.

147. The co-manipulated surgical system according to claim 144, wherein movement of the laparoscope is responsive to movement by a user at the laparoscope handle.

148. The co-manipulated surgical system according to claim 144, wherein the type of the laparoscope includes the angle of the operating end of the laparoscope.

149. The co-manipulated surgical system according to claim 148, wherein the controller is configured to identify the type of the laparoscope as a flat-head laparoscope when movement of the laparoscope includes circular motion and movement of the field of view of the laparoscope during the circular movement of the laparoscope includes a corresponding circular movement.

150. The co-manipulated surgical system according to claim 148, wherein the controller is configured to identify the type of the laparoscope as a flat-head laparoscope when movement of the laparoscope includes circular motion and movement of the field of view of the laparoscope during the circular movement of the laparoscope includes no change in field-of-view depth.

151. The co-manipulative surgical system according to claim 148, wherein the controller is configured to identify the type of the laparoscope as an angled head laparoscope when the movement of the laparoscope includes a circular motion and the movement of the field of view of the laparoscope during the circular movement of the laparoscope includes a change in the depth of the field of view.

152. The co-manipulative surgical system according to claim 144, wherein the controller is configured to allow the robotic arm to move freely in response to a movement at the handle of the laparoscope to perform laparoscopic surgery using the laparoscope.

Images