Systems and methods for visual sensing and interfacing with a trocar

Abstract

A surgical robotic system is provided having a tool drive coupled to a distal end of a robotic arm, the robotic arm having a plurality of actuators. The tool drive has a docking interface to receive a trocar. The system also includes one or more sensors operable to visually sense surface features of the trocar. One or more processors determine a position and orientation of the trocar based on the visually sensed surface features. In response, the processors control the actuators to orient the docking interface to the determined orientation of the trocar and to guide the robotic arm toward the determined position of the trocar. Other aspects are described and claimed.

Description

Technical Field

[0001] This disclosure relates generally to the field of robotic surgery, and more specifically to docking systems for use in surgical robots or in robotic-assisted surgical systems where the surgical robot arm needs to be docked with a cannula. Background Technology

[0002] Minimally invasive surgical procedures (MIS), such as laparoscopic surgery, involve techniques designed to minimize tissue damage during surgical procedures. For example, a laparoscopic procedure typically involves creating multiple small incisions inside the patient (e.g., in the abdomen) and introducing one or more instruments and at least one endoscopic camera through these incisions. The surgical procedure is then performed using the introduced instruments, with visualization aids provided by the camera. Generally, MIS offers multiple beneficial effects, such as reduced patient scarring, reduced patient pain, shorter recovery times, and lower medical costs associated with patient recovery. In some implementations, MIS can be performed using a robotic system comprising one or more robotic arms for manipulating surgical instruments based on commands from an operator.

[0003] In MIS procedures, access to the patient's body cavity is provided via a cannula. Once the distal end of the cannula (e.g., through the patient's abdominal wall) is correctly positioned and inserted through the tissue into the patient's internal region, the operator manually manipulates a surgical robotic arm, or a tool drive attached thereto, with a cannula mating interface at the distal end, until the mating interface aligns with the attachment portion (e.g., a mating interface) on the proximal end of the cannula (outside the patient's body). The operator then manually or automatically latches the cannula mating interface and the cannula mating interface together, thereby rigidly attaching the arm to the cannula. Once mated in this manner, a surgical tool (e.g., scissors, gripping jaws, needle, energy emitter, or camera) with an end effector at the distal end is inserted into the top opening of the cannula and then attached to the arm, allowing further surgical procedures to be performed using the tool while the arm is remotely controlled. Summary of the Invention

[0004] In MIS procedures, once the cannula is correctly positioned and inserted through the tissue into the patient's internal region, a robotic arm or a tool drive attached thereto is docked with the cannula to provide a rigid mechanical attachment between the robotic arm and the cannula. This attachment of the robotic arm and the cannula to each other allows the robotic arm to move as a unit with the cannula and one or more surgical instruments, which have been inserted through the lumen of the cannula into the patient's internal region. A docking interface located on the distal block of the robotic arm or on the tool drive attached to the arm is manipulated via actuators in the control arm until the docking interface aligns with and is positioned at the attachment portion (e.g., mating interface) of the cannula (which is exposed outside the patient). The robotic arm / tool ​​drive docking interface is then latched onto the attachment portion of the cannula, thus providing a rigid mechanical attachment between the robotic arm / tool ​​drive and the cannula.

[0005] Systems and methods for docking robotic arms with trocars are needed that avoid the challenges associated with certain docking methods of trocars. In one aspect, a vision sensor system or imaging system, such as one or more cameras positioned on a tool drive or elsewhere on the robotic arm, generates a series of digital images capturing the trocar. These images are processed by a data processor to determine the position and orientation of the trocar, i.e., the pose of the trocar relative to the camera and the tool drive. In response, the robotic arm (driving its actuators) is guided by a surgical robotic arm control system until the docking interface is aligned with and in position with the trocar, at which point mechanical coupling between the two is achieved.

[0006] In one aspect, a surgical robotic system has a robotic arm with multiple joints and associated joint actuators, and a tool drive coupled to the distal end of the robotic arm. The tool drive has a mating interface to receive an attachment portion of a cannula. The system also has one or more sensors operable to visually sense surface features of the cannula. These sensors may include imaging sensors, such as a camera, as part of an imaging system. In one variation, the imaging sensor may be disposed within a chamber of the mating interface. In another variation, a sterile adapter is coupled to the front portion of the mating interface, and the imaging sensor is mounted on the sterile adapter.

[0007] One or more processors are configured to determine the position and orientation of a trocar by interpreting surface features of the sensed trocar. In other words, the processors determine the sensing pose of the trocar based on digital image processing (including pattern recognition) of image sequences generated by an imaging sensor. In one variant, the surface features of the trocar are an encoded data payload that is detected and interpreted as an indication of the sensing pose of the trocar.

[0008] Once the sensing posture of the cannula is determined, the processor controls the robotic arm actuators to guide the arm as the mating interface moves toward the attachment portion of the cannula. The arm is guided by the processor driving the actuators to orient the mating interface to the determined orientation of the attachment portion of the cannula. In one aspect, the arm is also guided by one or more processors driving the actuators to move the mating interface to a determined position of the attachment portion of the cannula.

[0009] The one or more processors can be configured to generate a planned trajectory between the current position of the docking interface of the tool drive and one or more of the sensing position and sensing orientation of the cannula. The planned trajectory is a path along which the docking interface of the tool drive can travel and reorient itself (as the arm is being guided by the control system) until the posture of the docking interface matches the sensing posture of the cannula (resulting in a docking state).

[0010] In one variant, the robotic arm is automatically and fully driven along a planned trajectory by actuators (controlled by one or more processors). In this case, the operator does not need to manually push the arm (move along the trajectory). In another variant, the robotic arm is manually guided (by the operator's hand) while being assisted by actuators (controlled by one or more processors). In yet another variant, the robotic arm is manually guided by the operator along a planned trajectory, and whenever the operator's manual guidance is directing or causing the robotic arm to deviate from the planned trajectory (especially from the docking interface), the actuators, controlled by one or more processors, resist the operator's manual guidance of the robotic arm. In this case, the actuators resist the operator's manual guidance of the robotic arm with a force that may be proportional to the distance between the docking interface and the planned trajectory (or the distance between the docking interface and the planned trajectory). This is also referred to herein as the virtual spring operating mode.

[0011] A mating interface defines a receiving space between a chamber and one or more clamping members positioned within the chamber. In one variation, one or more clamping members are movably coupled to the mating interface and configured to move to secure an attachment portion of the trocar (such as an upper projection) within the chamber of the mating interface. In another variation, a lever is supported on the mating interface, and movement of the lever (e.g., by an operator's hand) causes one or more clamping members to move toward a locked or unlocked position that rigidly secures the attachment portion of the trocar to the mating interface. In yet another variation, a switch is provided that, when actuated, signals a processor to activate one or more sensors and / or determines the position and orientation of the trocar based on sensed surface features of the trocar, and then drives an actuator to guide the mating interface toward the determined orientation and position of the trocar. The switch can be positioned such that movement of the same lever used to latch the mating interface to the trocar also actuates the switch.

[0012] According to this disclosure, a method for docking a robotic arm of a surgical robotic system with a cannula includes the following operations (performed in part by one or more processors): Receiving an image of surface features on the cannula captured by sensors coupled to the robotic arm (e.g., a docking interface coupled to a tool drive mechanism of the arm). In one variation, the surface features may be an encoded data payload. The processor determines a sensed pose of the cannula based on digital image processing of the image (e.g., detection and interpretation of the sensed surface features). The sensed pose may include the position and orientation of the cannula, for example, six degrees of freedom (DOF), including three DOFs about position and three DOFs about orientation. The sensed cannula pose may be calculated relative to a known pose of the docking interface, wherein the latter may be determined using sensors and the history of previous movements of the arm.

[0013] In addition, one or more processors calculate a planned trajectory for the mating interface to travel and rotate until it matches the sensed cannula posture. Then, one or more processors drive actuators in the robotic arm to guide the robotic arm (its mating interface) along the planned trajectory toward the sensed cannula posture. The robotic arm can be guided in different ways under processor control. For example, guidance can be fully automatic (without operator intervention) until the arm (its mating interface) engages with the cannula. Alternatively, processor control can assist operator manual force applied to the arm to reduce the force required for operator movement, or processor control can counteract operator manual force whenever the arm deviates from the planned trajectory.

[0014] In one variant, the processor determines the distance from the docking interface to the planned trajectory. Based on this distance, the processor drives actuators in the robot arm to guide the robot arm toward the planned trajectory (e.g., moving the docking interface back onto the planned trajectory, also known as heading correction). This drive to return the robot arm to the planned trajectory can be initiated based on a virtual spring modeled by the processor, where the force applied to the robot arm by the actuators (under the processor's control) is proportional to the distance from the docking interface to the planned trajectory.

[0015] In one aspect of this disclosure, the processor determines, for example, a component of the hand force applied to the robot arm by an operator along a planned trajectory by means of signals received from force and / or torque sensors in the robot arm. An actuator in the robot arm is then driven such that the robot arm is guided along the planned trajectory based on that component of the hand force applied by the operator, thereby assisting the operator's hand force. In a variation, the actuator is driven such that the robot arm is guided with a force determined by the processor based on the product of the component of the hand force applied by the operator along the planned trajectory and a predetermined scalar value.

[0016] In another aspect of this disclosure, the processor determines a component of the operator's manual force applied to the robotic arm, deviating from the planned trajectory (again detected by signals received from force / torque sensors in the arm). The processor then drives actuators in the robotic arm based on this component, causing the arm to resist the manual force (which deviates from the planned trajectory). In other words, the actuators are driven to generate a force on the arm opposite to the manual force, and this can be determined by the processor by calculating the product of the component of the manual force applied by the operator that deviates from the trajectory and a predetermined scalar value.

[0017] The foregoing summary does not constitute an exhaustive list of all aspects of the invention. It is contemplated that the invention encompasses all systems and methods that can be implemented by all suitable combinations of the aspects outlined above, as well as those disclosed in the detailed description below and specifically pointed out in the claims filed with this patent application. Such combinations have specific advantages not specifically described in the foregoing summary. Attached Figure Description

[0018] Embodiments of the invention are illustrated by way of example and not by way of limitation in the accompanying drawings, wherein similar reference numerals indicate similar elements. It should be noted that references to "an" or "one" embodiment of the invention in this disclosure do not necessarily refer to the same embodiment, and that they refer to at least one. Furthermore, for the sake of brevity and to reduce the total number of drawings, a given drawing may be used to illustrate features of more than one embodiment of the invention, and not all elements in the drawing may be necessary for a given embodiment.

[0019] Figure 1 This is an overview diagram of the operating room layout equipped with a surgical robot system.

[0020] Figure 2 This is a perspective view of a portion of a robotic arm according to one aspect of this disclosure.

[0021] Figure 3 yes Figure 2 A schematic perspective view of the tool drive mechanism of a robotic arm.

[0022] Figure 4 yes Figure 3 The tool drive device docking interface and includes a perspective view of the cross-section of the sensor system.

[0023] Figure 5 yes Figure 3 The tool drive device interface and include a perspective view of a variant sensor system according to this disclosure.

[0024] Figures 6 to 8 This is a drawing view of the operation of a method for engaging a tool drive device of a robotic arm attached to a surgical robot system with a cannula, according to one aspect of this disclosure.

[0025] Figure 9 yes Figure 6 An enlarged schematic diagram of region 9 identified in the image.

[0026] Figure 10 This is a process flow for a method of engaging a tool drive device of a robotic arm attached to a surgical robot system with a cannula, according to one aspect of this disclosure.

[0027] Figure 11 This is a process flow for a method of engaging a tool drive device of a robotic arm attached to a surgical robot system with a cannula, according to one aspect of this disclosure.

[0028] Figure 12A This is a process flow for a method of engaging a tool drive device of a robotic arm attached to a surgical robot system with a cannula, according to one aspect of this disclosure.

[0029] Figure 12B This is a process flow for a method of engaging a tool drive device of a robotic arm attached to a surgical robot system with a cannula, according to one aspect of this disclosure. Detailed Implementation

[0030] Several embodiments of the invention will now be explained with reference to the accompanying drawings. Where the shape, relative position, and other aspects of the parts described in the embodiments are not explicitly defined, the scope of the invention is not limited to the parts shown, which are shown for illustrative purposes only. Furthermore, while many details have been set forth, it should be understood that some embodiments of the invention may be practiced without these details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0031] See Figure 1 This is a drawing view of an exemplary surgical robotic system 1 in a surgical setting. The robotic system 1 includes a user console 2, a control tower 3, and one or more surgical robotic arms 4 at a surgical robotic platform 5 (e.g., a table, bed, etc.). The system 1 can be combined with any number of devices, tools, or accessories for performing surgery on a patient 6. For example, the system 1 may include one or more surgical tools 7 for performing surgery. The surgical tool 7 may be an end effector attached to the distal end of the surgical robotic arm 4 for performing surgical procedures.

[0032] Each surgical tool 7 can be manually manipulated, robotically manipulated, or both during surgery. For example, a surgical tool 7 can be a tool for accessing, viewing, or manipulating the internal anatomy of the patient 6. In one embodiment, the surgical tool 7 is a gripper capable of grasping patient tissue. The surgical tool 7 can be manually controlled by a bedside operator 8; or it can be robotically controlled via actuated movement of its attached surgical robotic arm 4. The robotic arm 4 is shown as a tabletop system, but in other configurations, the arm 4 can be mounted on a trolley, ceiling, or sidewall, or in another suitable structural support.

[0033] Generally, a remote operator 9 (such as a surgeon) can use the user console 2 to remotely manipulate the arm 4 and / or attached surgical instruments 7, for example, through remote operation. The user console 2 may be located in the same operating room as the rest of the system 1, such as... Figure 1 As shown. However, in other environments, the user console 2 may be located in an adjacent or nearby room, or it may be located in a remote location, such as in different buildings, cities, or countries. The user console 2 may include a seat 10, foot controls 13, one or more handheld user input devices UID 14, and at least one user display 15 configured to display a view of, for example, a surgical site within the body of a patient 6. In the exemplary user console 2, a remote operator 9 sits in the seat 10 and views the user display 15 while manipulating the foot controls 13 and the handheld UID 14 to remotely control the arm 4 and the surgical instrument 7 (which is mounted on the distal end of the arm 4).

[0034] In some variations, the bedside operator 8 can also operate the system 1 in a "bed" mode, where the bedside operator 8 is now positioned to one side of the patient 6 and simultaneously manipulates robot-driven tools (end-effectors attached to arm 4), for example, holding a handheld UID 14 and a manual laparoscopic tool with one hand. For example, the bedside operator's left hand can manipulate the handheld UID to control the robotic components, while the bedside operator's right hand can manipulate the manual laparoscopic tool. Thus, in these variations, the bedside operator 8 can perform both robot-assisted minimally invasive surgery and manual laparoscopic surgery on the patient 6.

[0035] During the exemplary procedure (surgical operation), the surgical team may perform preoperative setup before initiating surgery using the surgical robotic system 1. During preoperative setup, the main components of the surgical robotic system (platform 5 and robotic arm 4, control tower 3 and user console 2) are positioned, connected, and powered in the operating room. The robotic arm 4 may be in a fully retracted configuration, with the arm 4 positioned below the platform 5 for storage and / or transport purposes. The surgical team may extend the arm 4 from its retracted position for sterile draping, such as covering one or more portions of system 1, such as portions of the arm 4, with a sterile barrier to minimize, inhibit, or prevent the spread of pathogens. After draping, the arm 4 may be partially retracted until needed. Several routine laparoscopic procedures can then be performed, including cannulation and ventilation. For example, each cannula may be inserted into a small incision and through the body wall using an occluder. The cannulas and occluders allow optical access for visualization of tissue layers during insertion, thereby minimizing the risk of injury during placement. Typically, an endoscope is placed first to provide visualization with a handheld camera for placing other cannulas or other tools or equipment.

[0036] In one embodiment, a remote operator 9 holds and moves UID 14 to provide input commands, thereby driving one or more robotic arm actuators 17 in the robot system 1 for remote operation. UID 14 may be communicatively coupled to the rest of the robot system 1, for example, via a console computer system 16. UID 14 may generate spatial state signals corresponding to the movement of UID 14, such as the position and orientation of the handheld housing of UID 14, and the spatial state signals may be input signals for controlling the movement of the robotic arm actuators 17. The robot system 1 may use control signals derived from the spatial state signals to control the proportional movement of the actuators 17. In one embodiment, a console processor of the console computer system 16 receives the spatial state signals and generates corresponding control signals. Based on these control signals controlling how the actuators 17 are energized to drive segments or connectors of the arm 4, the movement of a corresponding surgical tool attached to the arm may simulate the movement of UID 14. Similarly, the interaction between the remote operator 9 and UID 14 can generate, for example, a gripping control signal that causes the jaws of the gripper of the surgical tool 7 to close and grip the tissue of the patient 6.

[0037] The surgical robot system 1 may include a plurality of UIDs 14, wherein a corresponding control signal is generated for each UID that controls the actuators and surgical instruments (end-effectors) of a respective arm 4. For example, a remote operator 9 may move a first UID 14 to control the movement of an actuator 17 located in the left robotic arm, wherein the actuator responds by moving links, gears, etc. in the arm 4. Similarly, movement of a second UID 14 by the remote operator 9 controls the movement of another actuator 17, which in turn drives other links, gears, etc. of the robot system 1. The robot system 1 may include a right arm 4 fixed to a bed or table on the right side of the patient, and a left arm 4 located on the left side of the patient. The actuators 17 may include one or more motors, which are controlled such that they drive the joints of the arm 4 to rotate, for example, to change the orientation of the endoscope or gripper of the surgical instrument 7 attached to the arm relative to the patient. The movement of a plurality of actuators 17 in the same arm 4 may be controlled by spatial state signals generated from a particular UID 14. The UID 14 may also control the movement of the corresponding surgical instrument gripper. For example, each UID 14 can generate a corresponding gripping signal to control the movement of an actuator (e.g., a linear actuator) that opens or closes the jaws of the gripper at the distal end of the surgical tool 7 to grip tissue in the patient 6.

[0038] In some respects, communication between platform 5 and user console 2 can be achieved via control tower 3, which translates user commands received from user console 2 (and more specifically from console computer system 16) into robot control commands transmitted to arm 4 on robot platform 5. Control tower 3 can also transmit status and feedback from platform 5 back to user console 2. The communication connection between robot platform 5, user console 2, and control tower 3 can be via wired and / or wireless links, using any suitable data communication protocol from a variety of data communication protocols. Any wired connection can optionally be integrated into the floor and / or walls or ceiling of the operating room. Robot system 1 can provide video output to one or more displays, including displays within the operating room and remote displays accessible via the Internet or other networks. Video output or feeds can also be encrypted to ensure privacy, and all or part of the video output can be stored on a server or electronic healthcare record system.

[0039] As described above, in order to form a port for introducing surgical instruments into the patient 6, the cannula assembly can be inserted into the patient through an incision or access point within the patient's body (e.g., in the abdominal wall). The cannula assembly may include a cannula or a cannula 63 ( Figure 6 The cannula assembly may include an occluder and / or a seal. In some variations, the cannula assembly may include an occluder, such as a needle having a sharp tip for penetrating the patient's skin. It should be understood that the cannula 63 as described herein includes at least a cannula and may optionally include an occluder or other components. The occluder may be positioned within the lumen of the cannula 63 upon insertion into the patient 6 and then removed from the cannula 63, allowing surgical instruments to be inserted through the lumen of the cannula 63. Once positioned within the patient 6, the cannula 63 may provide a channel for holding one or more surgical instruments within a body cavity or other site within the patient 6, and for docking the arm 4 to move the instruments during remote operations.

[0040] Go to Figure 2A portion of an exemplary robotic arm 4 according to one aspect of this disclosure is shown. The robotic arm 4 may include a plurality of connectors (e.g., connectors 20A-20E) and a plurality of engagement modules (e.g., engagements 21A-21E) for actuating the plurality of connectors relative to each other. The engagement modules may include various engagement types, such as pitch engagements or roll engagements, any of which can be actuated manually or by a robotic arm actuator 17, and any of which can substantially constrain the movement of adjacent connectors about certain axes relative to other axes. Also shown, a tool drive 23 is attached to the distal end of the robotic arm 4. As described herein, the tool drive 23 may be configured with a mating interface 27 to receive an attachment portion (e.g., a mating interface) of a cannula 63, such that the cannula 63 can then be rigidly secured to the robotic arm 4. In this configuration, the distal elongated portion of one or more surgical instruments (e.g., endoscopes, suture devices, etc.) may be guided through the lumen of the cannula 63, and the instrument may be attached to the tool drive. The multiple joint modules 21A-21E of the robotic arm 4 can then be actuated under the control of the control system to position and orient the arm 4, the tool drive 23, and the surgical instruments thus attached, so as to enable remote operation during robotic surgery.

[0041] Figure 3 This is a schematic diagram illustrating an exemplary tool drive 23 without a loaded tool 7 according to various aspects of the subject matter. In one variation, the tool drive 23 may include an elongated base (or “shelf”) 24 having a longitudinal rail 25 and a tool holder 26 slidably engaged with the longitudinal rail 25. The shelf 24 may be configured to be coupled to the distal end of a robotic arm 4 such that articulated movements of the robotic arm 4 position and / or orient the tool drive 23 in place. The tool holder 26 may be configured to receive the tool 7 (the distal portion of which is inserted through a cannula needle 63). Once the tool 7 has been attached to the tool holder 26, the tool holder can actuate a set of articulated movements of the tool 7 (as an end effector) via an actuator driven by a control system in the tool holder 26, through any suitable mechanical transmission, such as a system of cables or wires and / or gears.

[0042] See also: Figure 4 and Figure 5 The cannula 63 can be coupled to the tool drive 23 or another component of the surgical robot system 1, such as, for example, at the docking station or docking interface 27 located at the distal block of the elongated base 24. Figure 4 As seen in the diagram. The docking interface 27 is configured to receive the cannula 63 as part of the surgical robotic arm 4, such that the docking interface 27 is configured as a cannula docking interface, a cannula attachment device, or a cannula mounting device. The docking interface 27 provides a reliable and quick way to attach the cannula 63 to the surgical robotic arm 4.

[0043] like Figure 4 As shown, the docking interface 27 can define a chamber 29, which can be accessed through the opening 31 of the docking interface 27 (see...). Figure 5 The receiver 37 may include a first clamping member 33 and a second clamping member 35 (e.g., an arm, plate, lever, member) arranged around the receiver 37, wherein the receiver defines a receiving space 38 for receiving a portion of the cannula 63 (e.g., a mating interface, such as the attachment portion of the cannula located in the proximal portion of the cannula). At least one of the clamping members 33, 35 may pivot between an open position and a closed position; in the closed position, the attachment portion 69 of the cannula 63 (see...) Figure 6 (It has been inserted into the receiving space 38 between the clamping members 33 and 35) and is held in place at least partially by the first clamping member 33 and the second clamping member 35.

[0044] In one variation, the docking interface 27 may include an eccentric mechanism comprising a lever 45 or other suitable locking member, which mechanically engages with the clamping member 33 between an open and closed position, for example, by means of a pin and slot or by means of another pivotable or movable connector. The lever 45 may move, for example, along a track or slot defined in the body or housing of the docking interface 27 between a forward-locked position (e.g., a locked eccentric position) and a rearward-unlocked position. As the lever 45 moves toward the locked position, it may push the clamping member 33 downward toward the receiving space 38 and lock the clamping member 33 in the closed position, such that a portion of the cannula 63 is securely held between the first clamping member 33 and the second clamping member 35. In some variations, the second clamping member 35 may be stationary or fixed. In one variation, the lever 45 may be driven by an electric motor or actuator (controlled by a processor or by a switch under operator manual control), or it may be manually driven by the operator's hand.

[0045] like Figure 5 As shown, the docking interface 27 can also provide a sterile barrier between sterile components such as the cannula 63 and non-sterile components such as the first clamping member 33 and the second clamping member 35 (or other non-sterile components of the surgical system). The sterile barrier can be provided, for example, by a sterile adapter 44 formed of a surgical-grade polymer or other surgical-grade material inserted between the cannula 63 and the first clamping member 33 and the second clamping member 35. In this respect, the sterile adapter 44 can be coupled to the front portion of the docking interface 27 such that the holes in the sterile adapter 44 are aligned with the opening 31 on the front of the docking interface 27, as shown. Figure 5 As shown. The attachment portion 69 of the cannula (see...) Figure 6It passes through the hole, then through the front opening 31, and is then positioned within the receiving space 38 of the chamber 37.

[0046] For example, in such Figure 4 and Figure 5 A sensor system is provided in the docking interface 27 shown. This sensor system may be at least partially recessed into the chamber 29 or may be otherwise coupled to or supported by the docking interface 27. The sensor system may include an imaging sensor 49 and a lens 51. The sensor system can generate a series of digital images (e.g., video) captured within the field of view of the lens 51 in front of the docking interface 27, as shown. The lens 51 may be a polymer or composite element that can protect the imaging sensor 49 from, for example, fluids, particles, or accidental contact with the operator or surgical equipment. The imaging sensor 49 and optionally the lens 51 may be housed within an additional housing (not shown) to provide additional shock or vibration protection, particle or fluid resistance, etc. Figure 5 As shown, the sensor system can be covered by a sterile adapter 44, wherein a protective cover portion 43 that is transparent to visible light is positioned on the front surface of the lens 51 (to allow the sensor system to view the scene in front of the docking interface 27).

[0047] The sensor system can be positioned such that the lens 51 is fully positioned within the chamber 29 (and therefore does not protrude from the foremost plane of the docking interface 27). The sensor system should be mounted to the docking interface 27 so as not to impede or interfere with other operations of the docking interface 27, such as the movement of the lever 45 and the movement of one or more of the clamping components 33, 35, as described above, and the reception of one or more portions of the cannula 63.

[0048] Despite Figure 5 Not shown, but a sterile drape or barrier may also be present, which is attached to the robotic arm 4 at a portion spaced apart from the docking interface 27 and covers the docking interface 27 to maintain a sterile barrier with the cannula 63, and has a visible light portion aligned with the imaging path surface of the lens 51 to provide the sensor system with an unobstructed view of the scene in front of the docking interface 27.

[0049] A processor or controller that can be part of control tower 3 (see...) Figure 1The processor will process the digital image sequence generated by the sensor system to determine how to guide arm 4 (by providing drive commands, such as force or velocity commands, to various actuators 17 in arm 4) to guide the movement of docking interface 27. It should be understood that such a processor may be part of other parts of the surgical robot system 1, wherein the sensor system communicates electrically with one or more of such processors. Furthermore, such processing can be triggered by an actuation switch 61 or other user-selectable control mounted on arm 4, for example, mounted on tool drive 23, particularly mounted on... Figure 5 The switch 61 is positioned behind lever 45 in this case, allowing lever 45 to be pushed into contact with switch 61, thereby actuating switch 61, as further described herein. Switch 61 is in electrical communication with the processor in control tower 3 and, when actuated, signals the processor to excite or activate the sensor system and / or begin processing the image data generated by the sensor system to determine, according to an algorithm, a planned trajectory for guiding the robotic arm 4 (and its attached tool drive 23) toward cannula 63, as further described herein. Switch 61 here generally refers to any suitable mechanism that can be triggered by the operator, such as a momentary mechanical switch, a proximity sensor, a virtual switch as part of a touchscreen or touchpad, etc. Placing switch 61 on or near docking interface 27 ensures that the operator activates the sensor system to guide docking interface 27 only when approaching arm 4 and when a separate control interface is not required (e.g., via user console 2, which may be positioned too far from robotic arm 4 to allow the operator sitting on user console 2 to see how docking interface 27 moves toward cannula 63).

[0050] See also: Figures 6 to 9The figure illustrates the guidance and docking of a tool drive 23 with a docking interface 27 according to one aspect of the present disclosure and a cannula 63 at least partially inserted into a patient 6 (preferably in a constant position). As shown, the cannula 63 includes a generally tubular body 64 having a flanged upper portion or head 67 and an attachment portion 69 protruding from the head 67 to mate with the docking interface 27. In one variation, the attachment portion 69 may be configured, for example, to have a nose, a clasp, or a pin arrangement, and may have one or more surface features, such as notches, ridges, protrusions, angles, hooks, etc., for interlocking with a receiver 37 of the docking interface 27. The cannula 63 may have different arrangements without departing from the present disclosure. Target markings or surface features 71 (e.g., textured surfaces, markings, printing, or other visible markings) may be provided on the upper portion of the cannula 63, such as one side of the head 67 of the cannula 63. Surface feature 71 may be provided, for example, by etching, stamping, printing, or as an accessory such as a sticker or label, and may have an arrangement corresponding to an encoded data payload. For example, surface feature 71 may have an arrangement of a barcode, a two-dimensional barcode, or a matrix barcode, which may include data such as numeric data, alphanumeric data, byte / binary data, or other data. In one variation, the surface feature may correspond to information associated with or a link providing that information for a specific algorithm used to guide or drive the robotic arm 4 and its docking interface 27 toward the cannula 63. The control system may load such an algorithm in response to detecting the surface feature and execute the algorithm to guide the robotic arm 4 and its docking interface 27 toward the cannula 23. Without departing from this disclosure, the cannula 63 and its surface feature 71 may have different arrangements. For example, in another variation, surface feature 71 itself may be a visible property of the external structure of one or more portions of the cannula 63, such as its shape, size, etc.

[0051] The docking interface 27 of the robotic arm 4 can be guided from a first posture (e.g., a parking posture or an unknown posture) to... Figure 6The second posture shown is close to but physically separated from the cannula 63. This guidance can be, for example, manually pushed by an operator, or it can be driven by the robot arm actuator 17. In the second posture, the robot arm 4 / dock interface 27 is positioned such that the cannula 63 is within the field of view V of the sensor system. The field of view V can include a straight line of sight to at least a portion of the cannula 63. In one variation, the appropriate proximity or arrangement of the robot arm 4 / dock interface 27 relative to the cannula 63 can be indicated to the operator (by the processor), for example, in the form of an audible beep or alarm, indicator light, or other visual marker and / or tactile indicator, such as tactile or vibrational feedback on a portion of the robot arm 4. In this regard, the imaging sensor 49 can be activated by the processor, for example, during the initial setup or preparation of the robot arm 4 and the tool drive 23, or via operator input, before positioning the robot arm 4 / tool ​​drive 23 into the second posture. If the proximity of the docking interface 27 to the sensor system is not appropriate (to establish a field of view V surrounding the cannula 63), the robotic arm 4 may be further guided toward the cannula 63, for example, by manual pushing by the operator, by automatic guidance under processor control, or some combination thereof, until the processor determines that the cannula 63 is within the field of view of the sensor system. Figure 7 This demonstrates how to move arm 4 closer to cannula needle 63 (with) Figure 6 (Compared to the initial posture).

[0052] The docking interface 27, particularly the sensor system, is positioned and oriented to receive light from the field of view V, and in response, a corresponding image data electrical signal is generated and transmitted to the processor in the control tower 3. By analyzing the image data, for example, when identifying the surface features 71 of the cannula 63 according to an object recognition algorithm, the processor calculates the position and orientation of the cannula 63 relative to the docking interface 27. The initialization or initiation of such an algorithm can be prompted, for example, by activating switch 61. In a variant, switch 61 can be activated by moving lever 45 backward to the unlocked (rearward) position, causing lever 45 to contact and actuate switch 61. Thus, the processor in the control tower 3 is signaled by switch 61 to apply an algorithm to determine the posture (e.g., spatial position and orientation) of the attachment portion 69 of the cannula 63 relative to the docking interface 27. The processor can then calculate a transformation, such as a transformation matrix, that can be used to guide or drive the robot arm 4, and the docking interface 27 of the tool drive device 23 attached thereto, toward the cannula 63. In this regard, the processor generates a planned trajectory for the robot arm 4 / dock interface 27, along which the robot arm 4 / dock interface 27 should move and reorient to achieve a posture that matches the posture of the cannula 63 sensed by the sensor system (e.g., Figure 8As shown, in this posture, the docking interface 27 appears to have reached or docked within the head 67 of the pin 63. Note that "matching" as used herein does not mean exactly identical, but within tolerances. An algorithm can be a set of computer-implemented instructions, for example, as part of a computer program product, firmware, etc., which may be stored on a non-transitory computer-readable medium for processing by the processor of the control tower 3, and will be collectively referred to herein as an algorithm. The processor's initialization of the algorithm can be considered as the start of the docking procedure for the robot arm 4 or the attached tool drive 23.

[0053] The object recognition algorithm applied by the processor can be, for example, a feature-based algorithm that identifies surface features 71 or other features of the cannula 63 within the field of view V of the sensor system. Such algorithms may include, for example, a Harris affine zone detector or a scale-invariant feature transform (SIFT). In one variant, in the presence of multiple cannulas, the processor can uniquely identify and distinguish the cannula 63 by recognizing surface features 71, where each cannula has a unique surface feature. In an environment with multiple robotic arms, each robotic arm 4 can be designated to recognize a predetermined surface feature 71. In one aspect, the processor determines the orientation of the attachment portion 69 of the cannula 63 by analyzing image data output from the sensor system, to determine one or more of the following: the depth distance (e.g., X-axis distance) between the cannula 63 and the docking interface 27, the horizontal distance (e.g., Y-axis distance) between the cannula 63 and the docking interface 27, the vertical distance (e.g., Z-axis distance) between the cannula 63 and the docking interface 27, and the rotational orientation about one or more of the X-axis, Y-axis, and Z-axis.

[0054] Once surface features 71 have been identified and the orientation of the cannula 63 has been calculated based on them, a processor in the control tower 3 can generate a tracking path or planned trajectory T for the extension of the robotic arm 4 / dock interface 27 toward the attachment portion 69 of the cannula 63. The planned trajectory T can be generated by the processor based at least on image data received from the sensor system. In one variation, at least a portion of the planned trajectory T may include a predetermined path generated independently of the image data from the sensor system. In this respect, the planned trajectory T can begin from a known orientation of the dock interface 27 and can be calculated based on records of previous movements, signals received from the F / T sensor 73, or other inputs. The planned trajectory T can be designed to navigate around one or more objects that may be located between the robotic arm 4 / dock interface 27 and the cannula 63, and to enable the dock interface 27 to match the sensed orientation of the cannula 63. In this respect, the planned trajectory T can provide a path that bypasses and thus avoids collisions with, for example, parts of the patient's anatomy, the surgical platform where the patient rests, bedside staff, cables or pipes, the attached robotic arm, or other equipment or personnel in the surgical environment. The planned trajectory T can be provided relative to a three-axis coordinate system, such as a coordinate system with mutually perpendicular X, Y, and Z axes, and can include translational movements along one or more of the X, Y, and Z axes, as well as rotational orientations, such as roll, pitch, and yaw, about one or more of the X, Y, and Z axes. Although the planned trajectory T is shown as a curve in the figure, it should be understood that the planned trajectory T can include one or more straight, angled, or discontinuous sections, and can be provided in one or more segments or shelves.

[0055] As described herein, guiding the robotic arm 4 along a planned trajectory T toward the cannula 63 can be accomplished in several modalities. For example, in one variant, the robotic arm 4 / dock interface 27 is guided to dock with the cannula 63 in a process that is at least partially automated, wherein a processor in the control tower 3 drives the robotic arm actuator 17 to guide the robotic arm 4 / dock interface 27 in response to sensing manual push or guide by the operator. Such guidance can be implemented using a control algorithm that may include admittance control, wherein external forces (e.g., gravity and operator hand force) applied to the robotic arm 4 are sensed, and the algorithm uses the measured docking position and docking velocity together as feedback to determine the command to drive the robotic arm actuator 17. In this regard, the robotic arm 4 may include an F / T (force / torque) sensor 73 to receive input manually applied to the robotic arm 4 by the operator and to generate a corresponding electrical signal as an output to the processor in the control tower 3. The F / T sensor 73 may also receive the force applied to the robotic arm 4 by the robotic arm actuator 17 as input. Therefore, the F / T sensor 73 can be configured to receive linear or rotational forces, such as torque, as input. Although the F / T sensor 73 is schematically shown as being mounted or integrated at a specific joint of the robot arm 4, more than one such F / T sensor 73 may be present and integrated into various joints or other parts of the robot arm 4 without departing from this disclosure.

[0056] As described herein, the guidance of the robotic arm 4 / dock interface 27 toward the cannula 63 can be at least partially achieved by manual pushing by the operator. However, since the forces required to manipulate the robotic arm 4 (e.g., due to weight, friction, etc.) are typically large, manual guidance by the operator is assisted by the robotic arm actuator 17 under processor control. In one variant, the processor can generate or model a virtual spring for a guidance control algorithm that corrects for or resists any manual pushing in a direction deviating from the planned trajectory T. This virtual spring generated by the processor indicates the magnitude and direction of the required force on the robotic arm 4 according to a predetermined virtual spring constant (k), which tends to return the robotic arm 4 / dock interface 27 toward the direction aligned with the planned trajectory T. These forces can be made proportional to the distance deviating from the planned trajectory T (through the corresponding portion of the robotic arm 4 / dock interface 27). In this respect, the virtual spring constant (k) is a predefined function that takes the distance and direction of the dock interface 27 of the tool drive from the planned trajectory T as input. In one variant, the processor can signal the robot arm actuator 17 to counteract any manual push of the robot arm 4 / dock interface 27 in a direction away from the planned trajectory T.

[0057] In this respect, based on the virtual spring generated by the processor, the operator can encounter resistance applied by the robot arm actuator 17, such that the resistance increases with the distance from the planned trajectory T. In this respect, the processor in the control tower 3 provides the planned trajectory T as a virtual fixture; deviation from this trajectory results in corrective movements of the robot arm actuator 17 and forces applied to the robot arm 4, which tend to return the robot arm 19 / dock interface 27 in a direction aligned with the planned trajectory.

[0058] Alternatively or concurrently, the robot arm actuator 17 may assist (e.g., enhance, amplify, elevate, etc.) the manual guidance of the robot arm 4 / dock interface 27 along the planned trajectory T. For example, the robot arm actuator 17 may assist the operator in manually pushing the robot arm 4 / dock interface 27 in a direction along the planned trajectory T (exactly along or close to the planned trajectory T (within a predetermined tolerance)). In this case, the processor receives a signal from the F / T sensor 73 as input, corresponding to the force applied to the robot arm 4 by the operator, and in response drives the robot arm actuator 17 to assist manual guidance. As further described herein, this assisting force on the robot arm 4 / dock interface 27 provided by the processor-controlled robot arm actuator 17 may be consistent with the spring constant (k) or may be based at least in part on different factors.

[0059] Providing the aforementioned planned trajectory T / virtual fixture and associated virtual springs modeled by the processor can significantly reduce the force required by the operator when guiding the robot arm 4 / dock interface 27 toward the cannula 63 and maintaining the alignment of the robot arm 4 / dock interface 27 with the planned trajectory T. Two virtual fixture methods are described below that can facilitate the docking process.

[0060] Assuming the surgical plan is known, such as the type of surgery, patient size, patient position and orientation on the operating table, and the location of the cannula to be inserted into the patient, the position of the cannula head (including its attachment portion 69) can be estimated (as calculated by the processor using a physical model). In this case, the processor can guide the robotic arm 4 to approach the cannula 63, for example, until it reaches... Figure 6 The second posture is shown. Alternatively, the processor can allow the operator to manually push arm 4 to... Figure 6 The robot operates in the position shown in the mode (with only active gravity compensation and anti-drive to overcome friction in the joint of arm 4). Once robot arm 4 is in... Figure 6The indicated position allows the processor to execute any suitable sensing method to more accurately estimate the position of the cannula 63 (and particularly its attachment portion 69) relative to the arm 4 (particularly its docking interface 27). Possible sensing methods include magnetic sensing, structured light camera sensing, and, as described herein, sensing by analyzing image data from a visible light camera. Once the robotic arm 4 reaches the appropriate position, the processor can respond by providing an alert or feedback to the operator of the system using any of a variety of techniques at which the sensing method is expected to be effective in more accurately estimating the cannula position (calculating the “sensing or measuring posture” of the cannula 63). At this point, the operator has the option to select one of at least two virtual gripper operation modes for controlling the system to manually guide the arm to “fine-tune” the posture of the robotic arm 4 / docking interface 27 closer to docking before actual docking (via the actuator 17 of the drive arm 4 to assist manual guidance).

[0061] The selection between the two virtual clamping modes can be made by the bedside operator 8 pressing the button switch on the arm 4 or the foot pedal switch on the platform 5, which can activate a predetermined one of the virtual clamping modes or switch between virtual clamping modes. Figure 10 This is a process flow for an active virtual fixture mode, in which the processor automatically guides arm 4 along a planned trajectory T by appropriately actuating actuator 17 of arm 4 to influence the movement of arm 4 along the planned trajectory T. In a sub-mode referred to herein as fully automatic, the processor controls actuator 17 such that the docking interface 27 is automatically driven to approach and dock with the cannula 63 (along trajectory T) without any manual pushing by the operator. This sub-mode is useful because it allows for fine-tuning of the orientation of the docking interface 27 as it approaches docking. In another sub-mode of the active virtual fixture mode, the processor controls actuator 17 to drive arm 4 (producing movement of arm 4) to the extent necessary only to assist manual guidance by the operator; that is, the processor controls actuator 17 to pause arm 4 in its last position in response to detecting that the operator has stopped manually pushing arm 4.

[0062] refer to Figure 10 The diagram illustrates the process flow for an active virtual fixture operation mode. In active virtual fixture mode, the control system generates or drives the arm 4 to move forward along a planned trajectory T. The processor generates the planned trajectory T based on an algorithm planned at the joint level or task level. Figure 10 (Box 101). This algorithm determines the appropriate path for the extension of the mating interface 27 to achieve the sensing or measuring posture of the cannula 63, such as... Figure 8As shown. The sensing or measurement posture of the cannula may have been determined by the processor analyzing image data from the sensor system, or it may have been determined by the processor analyzing other sensor data generated by any other sensor system (in the presence of...). Figure 6 The planned trajectory T can be based on one or more inputs as signals from other sensors (e.g., proximity sensors, accelerometers for detecting nearby objects), as information provided by the operator (e.g., via user console 2), and as parameters associated with the operating environment such as characteristics of the patient 6 (e.g., the patient's body size or shape) or characteristics of the platform 5 supporting the patient 6. In one variant, the planned trajectory T can begin from the current pose of the robot arm 4 / dock interface 27, which can be determined by the processor via signals received from the F / T sensor 73 or other sensor inputs, or it can be a known value, for example, as determined by previous movement logs.

[0063] Although the planned trajectory T has been described as the output of the processor in control tower 3, it should be understood that one or more parts of the planned trajectory T may be manually entered by the operator, for example, received via a touchscreen.

[0064] Still referencing Figure 10 At box 103, the processor, for example, determines whether the robot arm 4 / docking interface 27 is positioned along the planned trajectory T based on the determined current pose of the docking interface 27. If not, the processor determines the distance and direction of the docking interface 27 from the planned trajectory T, for example, as the distance and direction to the nearest point along the planned trajectory T (box 105). The planned trajectory T may have been provided as a series of incrementally spaced points. The distance and direction may be referred to as a heading correction vector or sequence of vectors having one or more of X and Y or Z components, and may represent the path along which the robot arm 4 / docking interface 27 can travel to become aligned with the nearest point of the planned trajectory T.

[0065] At box 107, the processor in control tower 3 determines the virtual force / torque vector to be applied to robot arm 4 based on the sum of the heading correction vector and the planned trajectory vector of the planned trajectory T. The resulting sum vector can be scaled in box 108 based on the spring constant of a virtual spring modeled or generated by the processor. This operation provides the virtual force / torque vector applied to robot arm 4 (by driving robot arm actuator 17), which guides robot arm 4 / dock interface 27 to align with the planned trajectory T.

[0066] Returning to box 103, if the processor determines that the robot arm 4 / dock interface 27 is positioned along the planned trajectory T (i.e., within acceptable tolerances), then the processor determines at box 109 that the heading correction vector to be applied to the robot arm 4 should be negligible or zero.

[0067] Then, at box 111, (by the processor) the following operations are performed: i) summing the virtual force / torque vector determined by the processor in box 108 and ii) summing the detected hand force / torque vector applied to the robotic arm 4 by the operator (e.g., detected by the processor based on signals from one or more F / T sensors 73, as shown in box 113). It should be understood that one or more components of the hand force / torque vector may be represented by negative values, for example, force components in directions away from the planned trajectory T or in directions away from the virtual force / torque vector.

[0068] The summation in block 111 causes the processor to determine the input force / torque vector applied to the robot arm actuator 17 via admittance control (block 115). Admittance control is a feedback control algorithm implemented by the processor in control tower 3 to guide the docking interface 27 of the arm 4. It receives the summation generated in block 111 as input and issues a speed command to the robot arm actuator 17 based on feedback from the force / torque sensors in the arm 4 to drive one or more parts (e.g., joints or connectors) of the robot arm 4. The speed command may contain compensation calculated by the admittance control algorithm and is updated as the arm / docking interface is driven toward alignment with and along the planned trajectory T. In a variant, the signal provided by the processor to the robot arm actuator 17 under admittance control may include a force or torque command. The activation of the aforementioned admittance control of the robot arm 4 by the processor-controlled activation of the robot arm actuator 17 (including blocks 103-111) can be performed as an iterative process, for example, such that the input virtual force / torque vector and the hand force / torque vector summed by the processor to achieve admittance control of the robot arm can be repeatedly updated, for example, at predetermined intervals, to provide an updated signal to drive the robot arm actuator 17, which reflects the change of the input force / torque vector over time.

[0069] Another operating mode exists in which the robotic arm 4 moves solely due to manual guidance applied by the operator – the movement of the robotic arm 4 along the planned trajectory is substantially unaffected by the robotic arm actuator 17 (and is under processor control). This mode can be an alternative to any admittance control mode (instead of block 115) and can be selected by the operator at any point during the docking process as needed. For example, the operator may wish to deviate from the planned trajectory T or wish to stop automatic guidance for some reason, switching to operator-only or fully manual operation mode. This can be achieved via button 75 located on or near the docking interface 27 (e.g., see...). Figure 6This triggers or selects the switch between admittance control and fully manual control of robot arm 4. Button 75 can actuate a switch (not shown) that communicates electronically with the processor. Button 75 generally refers to any suitable manual control interface that can be selected by the operator's hand or foot to switch between, for example, two modes. For example, it could be a foot pedal located on platform 5 (see...). Figure 1 Button 75, when pressed or otherwise selected, can signal the processor in control tower 3 to stop the admittance control drive of robot arm actuator 17 and instead enter an alternative operating mode that allows individual operator manual force to move robot arm 4, even deviating from the planned trajectory. In this mode, robot arm 4 is considered to be operator-guided, deviating from the planned trajectory T if necessary. However, note that even in this fully manual operating mode, the processor may need to drive actuator 17 to perform active gravity compensation and active counter-drive (to overcome gravity and gear friction at the various joints of arm 4, so that arm 4 moves smoothly and remains stationary in response to operator hand movements without operator force). In another variation, button 75 can be a direct disengagement or deactivation of mechanical or electromechanical control of robot arm actuator 17.

[0070] During fully manual guidance of the robotic arm 4, feedback can be provided to the operator regarding the positioning of the robotic arm 4 / dock interface 27 relative to the planned trajectory T. Such feedback may include, for example, audible beeps or alarms (e.g., whenever arm 4 deviates from the planned trajectory T), indicator lights or other visual markers (e.g., a green light as long as the arm remains on the planned trajectory T), graphical indicators displayed on a monitor adjacent to platform 5 or on the user display 15 of user console 2, tactile indicators, such as tactile or vibrational feedback regarding a portion of the robotic arm 4, or other suitable technologies to provide the operator with feedback on the progress of manual guidance.

[0071] Turn now Figure 11 The diagram illustrates the process flow of a passive virtual gripper operation mode, where the processor constrains the robot arm 4 to a planned trajectory T, and if it detects that an operator is applying hand force to the arm 4 (in the direction of the planned trajectory T), it drives the arm 4 forward only along the planned trajectory T. At box 101, the planned trajectory T is generated by the processor in response to signals received from the sensor system or by manual input, as described above. At box 113, as described above, the processor detects the hand force / torque vector applied to the robot arm 4 by the operator based on signals received from the F / T sensor 73, and at box 117, the processor calculates the dot product of the planned trajectory T and the hand force / torque vector to determine, at box 119, the component of the hand force / torque vector provided by the operator that is parallel to or otherwise along the planned trajectory T (hereinafter referred to as "F").X At box 121, the total hand force / torque vector F applied to the robotic arm 4 by the operator and measured by the F / T sensor 73 is subtracted from the hand force / torque vector F along the planned trajectory T. X The component of the hand force / torque vector perpendicular to the planned trajectory T is provided at box 123 (hereinafter referred to as "F"). Y (”).

[0072] Subsequently, at box 125, the processor will assign the manual force / torque vector F X Multiplying the parallel component by a scalar value, for example, a scalar value greater than 1 (hereinafter referred to as "scalar value N1", as shown in box 127), makes this parallel hand force / torque vector component F X The value is amplified or otherwise increased. At box 129, the hand force / torque vector F... Y The vertical component is also multiplied by a scalar value, such as a scalar value less than 1 (hereinafter referred to as "N2", as shown in box 131), so that the vertical hand force / torque vector component F Y The value decays or decreases in other ways.

[0073] It should be understood that the scalar values ​​N1 and N2 can be related to the spring constant (k) modeled or generated by the processor as described above, and can be predetermined values ​​or inputs determined by the processor or manually provided by the operator. In one variation, N2 is less than 1 (decreased or scaled down), while N1 is greater than 1 (increased or scaled up). This means that the operator will feel "easy" pushing arm 4 along trajectory T, but will feel resistance when pulling or pushing arm 4 in a direction away from trajectory T. In another variation, one or both of the scalar values ​​N1 and N2 can be 1, such that the corresponding hand force / torque vector component F determined by the processor... X F Y The value is not modified by the processor.

[0074] At box 133, the force / torque vector component F, multiplied by the corresponding scalar values ​​N1 and N2, is... X F Y The signals are added by the processor to generate a signal at block 115 to the robot arm actuator 17 under the control of the processor, which enables the robot arm actuator 17 to control the admittance of the robot arm 4 as described above.

[0075] In this respect, the robot arm actuator 17, under the control of the processor, can assist the operator in manually guiding the robot arm 4 along a planned trajectory T, for example, by modulating the vector component of the manual force / torque vector applied to the robot arm 4 by the operator relative to the planned trajectory T, so as to assist the operator in manually pushing the robot arm 4 along the planned trajectory T and resist the operator from manually pushing the robot arm 4 away from the planned trajectory T. In a variation, the scalar value N2 can be zero, such that the force / torque vector component F Y The value decays to zero or is canceled out, for example, causing the action of the robot arm actuator 17 under processor control to substantially suppress or prevent the operator from manually guiding the robot arm 4 away from the planned trajectory T.

[0076] according to Figure 11 The processor-controlled activation of the robot arm actuator 17 to conduct admittance control of the robot arm 4 can be performed as an iterative process, including through corresponding scalar values ​​N1, N2 to the hand force / torque vector F. X The parallel component and the vertical component F of the hand force / torque vector Y The above modulation is performed to achieve admittance control of the robotic arm 19. In this regard, the input to admittance control (block 115) can be repeatedly updated, for example at predetermined intervals, by executing blocks 113-133, to provide an updated signal to the robotic arm actuator 17 in response to the change of the input hand force / torque vector over time.

[0077] Figure 12A The process flow for mates a tool drive to a cannula is illustrated, in which, during the guidance or actuation of the robotic arm 4 by the robotic arm actuator 17 (based on the planned trajectory T), the processor checks the orientation of the mating interface 27, for example, to confirm whether the orientation of the mating interface 27 matches the entry orientation, i.e., the orientation and orientation of the cannula 63 to be mated by only linear translational movement (box 135). If the processor determines that the mating interface 27 does not match the determined orientation of the cannula 63, the processor can control the robotic arm actuator 17 to further drive or guide the robotic arm 4 toward such orientation (box 137). Once the orientation of the mating interface 27 matches the orientation of the cannula 63, the processor can drive the robotic arm actuator 17 to further drive or guide the robotic arm 4, for example, by only translational movement (now without needing to change the orientation or orientation of the mating interface 27), until the tool drive 23 mates with the cannula 63 (box 139). In this process, the orientation of the docking interface 127 during guidance along the planned trajectory T does not need to match the orientation of the cannula 63 until the entry posture is achieved.

[0078] Figure 12BAnother exemplary process flow for mates a tool drive mechanism of a robotic arm attached to a surgical robotic system with a cannula is shown. The process begins by determining a planned trajectory T (box 101). The processor also determines the orientation of the cannula 63 and then determines whether the current orientation of the docking interface 27 matches the current orientation of the cannula 63 (e.g., the "entry orientation," which is the orientation of the docking interface 27 in which linear translation of the docking interface 27 would be sufficient to dock with the head of the cannula 63). If not, the processor adjusts the orientation of the docking interface (by appropriately actuating the actuator 17 of the arm 4) such that it matches the entry orientation. At this point, guidance of the arm along the planned trajectory T can begin or resume (box 141). The processor can repeatedly, for example periodically, check the orientation of the docking interface 27 while guiding the arm 4 (box 141), for example, to confirm whether the docking interface 27 matches the entry orientation during guidance. If not, the processor determines how to adjust the orientation of the docking interface 27 and accordingly actuates the actuator 17 in the arm 4 (box 141), while continuing to guide the arm along the planned trajectory T as described above. The process continues until the mating interface has reached the head of the cannula 63 and matches the orientation of the head of the cannula 63, for example, where the attachment portion 69 of the cannula 63 at least partially surrounds or is received in the receiving space 38 of the receiver 37, such that the mating interface 27 and the cannula 63 have the same posture, such as the same position and orientation, i.e., within tolerance (box 143). Once the orientation and position of the mating interface 27 match the orientation and position of the cannula 63, the processor can assert the mating state (and the process ends). In one variant, the operator can visually confirm the mating engagement of the mating interface 27 with the cannula 63. If the processor determines that the mating interface 27 is mated with the cannula 63, the algorithm can be considered complete, and in one variant, visual, auditory, or tactile confirmation can be provided to the operator by the processor. If the processor determines that the docking interface 27 has not yet docked with the cannula 63, the processor may signal the drive robot arm actuator 17 to further guide or drive the robot arm 4 until the tool drive device 23 docks with the cannula 63.

[0079] Such guidance or actuation of the robotic arm 19 into the cannula 63 can also be performed by processor-controlled assistance or resistance from the robotic arm actuator 17 to the operator’s manual guidance of the robotic arm 19 according to the virtual spring described above, or can be performed entirely manually by the operator, as described above.

[0080] In a docking posture, the docking interface 27 may be locked (e.g., rigidly mechanically coupled) to the cannula 63, for example, via clamping members 33, 35, as further described below. In a variation, the state in which the attachment portion 69 of the cannula 63 is at least partially surrounded or received in the receiving space 38 of the receiver 37 can be considered a ready-to-dock state, and the locked or mechanically coupled engagement of the docking interface 27 can be considered a docking state or a final docking state.

[0081] When the attachment portion 69 of the cannula 63 is positioned in the receiving space 38 of the receiver 37 of the docking interface 27, the lever 45 can be moved, for example manually or under processor control by an actuator, to a forward-locked position to push the clamping member 33 ( Figure 4 The attachment portion 69 of the cannula 63 can be press-engaged, such that the attachment portion 69 is secured (e.g., latched, clamped, or locked) to the docking interface 27 in a docking position. Furthermore, the lever 45 can be moved rearward to an unlocked position to disengage the clamping member 33 from the attachment portion 69, thereby disengaging the robotic arm 19 / docking interface 27 from the cannula 63.

[0082] When the cannula 63 is locked to the docking interface 27, one or more surgical instruments can be coupled to the tool drive 23 and inserted through the cannula 63 to enter the body cavity of the patient 6 (to perform subsequent surgical procedures therein). The surgical robot system 1 has the ability to uniquely identify each tool (endoscope and surgical instrument) immediately upon attachment and displays the tool type and arm position, for example, on the display 15 at the user console 2. The corresponding tool function is then enabled and can be activated using the UID 14 and foot control 13. A patient-side assistant (e.g., bedside operator 8) can attach and detach tools as needed throughout the procedure. The surgeon (operator 9) seated at the user console 2 can begin performing surgery using the tools controlled by operator 9 via manipulating the UID 14 and foot control 13. The system 1 translates the surgeon's hand, wrist, and finger movements into precise, real-time movements of the surgical instruments via the UID 14 and foot control 13.

[0083] The aforementioned arrangement of the sensor system and robotic arm 4, algorithmically controlled using inputs from at least one sensor system, and optionally provided by a manual force / torque applied to the robotic arm 4 by a bedside operator 8, offers smooth, at least partially processor-controlled guidance for docking with the cannula 63. Once docked, a rigid or stable mechanical connection between the cannula 63 and the docking interface 27 of the robotic arm 4 / tool ​​drive 23 can be formed by a latching mechanism (e.g., under manual operator control).

[0084] On the other hand, when performing the docking process in either passive or active virtual clamping mode, arm 4 can still move to a position or orientation that might lead to loss of visual tracking. This can happen, for example, when the cannula 63 leaves the field of view of the arm's imaging system, or when other objects obstruct the line of sight from the imaging sensor in the arm to the cannula 63, making the control system potentially lack sufficient confidence that its planned trajectory T will result in docking with the cannula. To avoid this tracking loss problem, another layer of virtual clamping can be used to keep the sensed surface features of the cannula within the arm's sensing range throughout the arm's movement along the planned trajectory. For example, in the case of visually tracking the cannula using a camera integrated into the arm as described above, the control system can generate a virtual spring (e.g., such as...) between the sensed (tracked) surface features on the cannula and the center of the arm's sensing range. Figure 6 (The virtual spring is located in the center of the camera's field of view, as shown). This virtual spring ensures that the tracked / sensed surface feature remains close to the center of the camera's field of view when following the planned trajectory. Therefore, this prevents the sensed surface feature from leaving the sensing range of the imaging system throughout the robot arm's movement during the docking process. The active / passive virtual gripper described above for pulling or guiding the arm along the planned trajectory towards the pin can be superimposed on this second virtual gripper. The latter will resist any movement of the arm that would cause the sensed surface feature to leave the sensing range of the arm's imaging system, thus ensuring a more reliable and uninterrupted docking process.

[0085] For purposes of explanation, the foregoing description uses specific nomenclature to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that specific details are not required to practice the invention. The foregoing description of specific embodiments of the invention has been provided for illustrative and descriptive purposes. These are not intended to be exhaustive or to limit the invention to the specific forms disclosed; various modifications and alterations can be made to this disclosure in light of the foregoing teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, thereby enabling others skilled in the art to best utilize the invention and its various embodiments with various modifications suitable for the contemplated particular uses.

Claims

1. A surgical robot system, comprising: A tool drive device coupled to the distal end of a surgical robot arm, the tool drive device including a docking interface for receiving a cannula, wherein the docking interface defines a chamber and one or more clamping members disposed in the chamber and configured to move to secure an attachment portion of the cannula to the docking interface. One or more sensors, the one or more sensors being operable to visually sense surface features of the cannula; One or more processors, said one or more processors being configured to: The position and orientation of the cannula are determined based on visually sensed surface features, and Signals are sent to multiple actuators to guide the robotic arm toward a defined position of the cannula, while simultaneously orienting the docking interface toward a defined orientation of the cannula. and A switch, when actuated, signals the processor to i) activate the one or more sensors, ii) determine the position and orientation of the cannula based on visually sensed surface features, or iii) guide the robotic arm toward the determined position of the cannula.

2. The surgical robot system according to claim 1, wherein, The one or more processors are configured to generate a planned trajectory to a determined location of the cannula and guide the robotic arm along the planned trajectory.

3. The surgical robot system according to claim 2, wherein, The one or more processors are configured to guide the robotic arm by controlling the plurality of actuators to automatically drive the arm along the planned trajectory.

4. The surgical robot system according to claim 2, wherein, The one or more processors are configured to guide the robotic arm by controlling the plurality of actuators to assist an operator who manually pushes the arm.

5. The surgical robot system according to claim 2, wherein, The one or more processors are configured to control the plurality of actuators to resist manual pushing of the robotic arm by the operator when the operator manually pushes the robotic arm away from the planned trajectory.

6. The surgical robot system according to claim 5, wherein, The actuator, controlled by the one or more processors, resists manual guidance by the operator to move the robotic arm away from the planned trajectory with a force proportional to the distance between the robotic arm and the planned trajectory.

7. The surgical robot system according to claim 2, wherein, The planned trajectory avoids collisions between the robotic arm and one or more of the patient, the table on which the patient rests, bedside staff, cables, tubing, and other surgical robotic arms.

8. The surgical robot system according to claim 1, wherein, The arm includes a switch that changes operating mode between a first mode and a second mode when manually actuated by an operator. In the first mode, the one or more processors guide the robotic arm under the admittance control of the actuator; in the second mode, the one or more processors control the actuator to achieve fully manual guidance of the robotic arm.

9. The surgical robot system according to claim 1, wherein, The attachment portion of the cannula is a protrusion extending from the upper portion of the cannula.

10. The surgical robot system of claim 1, further comprising a lever supported on the docking interface, wherein movement of the lever causes movement of the one or more clamping components.

11. The surgical robot system of claim 10, wherein the switch is positioned such that movement of the lever in one direction actuates the switch, and movement of the lever in the opposite direction causes movement of the one or more clamping members to secure the mating interface to the cannula.

12. The surgical robot system according to claim 1, wherein, The one or more sensors are disposed in the docking interface.

13. The surgical robot system according to claim 12, wherein, The docking interface includes a sterile adapter coupled to the front portion of the docking interface, and the one or more sensors are mounted on the sterile adapter.

14. A method for engaging a robotic arm of a surgical robot system with a cannula, the method comprising: An image of surface features on the cannula is generated by one or more sensors, the one or more sensors being coupled to a docking interface of a tool drive device, which is coupled to the robotic arm; The sensing posture of the cannula, including the position and orientation of the cannula, is determined by one or more processors based on the image of the surface features. The planned trajectory of the sensed posture of the cannula is calculated by the one or more processors; The one or more processors drive multiple actuators in the robotic arm to guide the docking interface of the robotic arm along the planned trajectory to the sensed posture of the cannula; as well as When the switch is actuated, the switch signals the one or more processors to i) activate the one or more sensors, ii) determine the position and orientation of the cannula based on the image of the surface features, or iii) guide the robotic arm toward the determined position of the cannula.

15. The method of claim 14, further comprising determining a distance between the docking interface and the planned trajectory by the one or more processors, and controlling the plurality of actuators in the robotic arm based on the distance to guide the robotic arm toward the planned trajectory.

16. The method of claim 14, further comprising determining, by the processor, a component of a manual force applied by an operator to the robotic arm in the direction of the planned trajectory, and controlling, based on the component of the manual force applied by the operator along the planned trajectory, the plurality of actuators in the robotic arm to guide the robotic arm along the planned trajectory.

17. The method according to claim 16, wherein, The processor controls the plurality of actuators in the robotic arm to guide the robotic arm along the planned trajectory using a force determined by i) a component of the manual force applied by the operator along the planned trajectory and ii) a predetermined scalar value.

18. The method of claim 14, further comprising determining, by the processor, a component of a manual force applied by an operator in a direction away from the planned trajectory on the robot arm, and controlling, based on the component of the manual force applied by the operator in the direction away from the planned trajectory, the plurality of actuators in the robot arm to drive the robot arm toward the planned trajectory.

19. The method according to claim 18, wherein, The processor controls the plurality of actuators in the robotic arm to drive the robotic arm toward the planned trajectory using a force determined by i) a component of the hand force applied by the operator in a direction away from the planned trajectory and ii) a predetermined scalar value.

20. The method of claim 14, wherein, The one or more sensors are part of a camera located on the arm, and the one or more processors control the plurality of actuators in the robotic arm to maintain the surface feature at the center of the camera's field of view while the arm moves along the planned trajectory.

Images