Method and system for providing assistance to a user of a surgical robotic system

Abstract

A system for providing assistance to a user of a surgical robot system, the system comprising: a surgical robot system, the surgical robot system comprising at least one surgical robot, the at least one surgical robot having a base and an arm extending from the base to an attachment of an instrument, the arm comprising a plurality of joints, whereby the configuration of the arm can be changed; and an auxiliary module, the auxiliary module comprising a list of task steps that can be provided with program assistance, the auxiliary module being configured to: receive status data indicating the status of the surgical robot system when the surgical robot system is being used to perform the task; determine from the status data whether the surgical robot system is in a program-assisted state, the surgical robot system being in a program-assisted state when the surgical robot system is being used or is to be used to perform one of the task steps that can be provided with program assistance; and in response to determining that the surgical robot system is in a program-assisted state, provide the user with program assistance in performing the task steps when the surgical robot system is being used to perform the task.

Description

Background Art

[0001] It is known to use robots to assist and perform surgery. Figure 1 A typical surgical robot 100 is shown, comprising a base 108, an arm 102, and an instrument 105. The base supports the robot and is itself rigidly attached, for example, to an operating room floor, an operating room ceiling, or a cart. The arm extends between the base and the instrument. The arm is articulated along its length by means of a plurality of flexible joints 103, which are used to position the surgical instrument in a desired position relative to the patient. The surgical instrument is attached to the distal end 104 of the robot arm. The surgical instrument penetrates the body of the patient 101 at a port 107 in order to access the surgical site. At its distal end, the instrument comprises an end effector 106 for participating in a medical procedure.

[0002] Figure 2 A typical surgical instrument 200 for performing robotic laparoscopic surgery is shown. The surgical instrument includes a base 201 by which the surgical instrument is connected to a robotic arm. A shaft 202 extends between the base 201 and an articulation 203. The articulation 203 terminates in an end effector 204. Figure 2 , a pair of serrated jaws are shown as end effector 204. Articulation 203 allows end effector 204 to move relative to shaft 202. It is desirable to provide at least two degrees of freedom to the movement of end effector 204 by means of the articulation.

[0003] All surgical procedures, including those performed via surgical robots, are associated with patient risks. Many risks can be mitigated by taking certain precautions before, during, and after the procedure. However, many of the mitigation techniques currently implemented are limited by the information available before, during, and after the procedure. When a surgical robot is used to perform all or a portion of a surgical procedure, more information inherent to the procedure is available. In particular, there is at least information about the surgical robot—how it interacts with the patient and how the user (e.g., the surgical team) interacts with it. It would be advantageous to be able to use this information to improve the efficiency and / or efficacy of surgical procedures performed via surgical robots.

[0004] The embodiments described below are provided by way of example only and are not intended to limit implementations that address any or all of the disadvantages of known surgical robotic systems. Summary of the invention

[0005] This Summary is provided to introduce a selection of concepts that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0006] The present invention describes a method and system for providing assistance to one or more users of a surgical robot system. The system includes a surgical robot system and an auxiliary module. The surgical robot system includes at least one surgical robot having a base and an arm extending from the base to an attachment of an instrument, the arm including a plurality of joints, whereby the configuration of the arm can be changed. The auxiliary module is configured to: receive status data indicating the status of the surgical robot system when the surgical robot system is being used to perform the task; determine from the status data whether the surgical robot system is in an auxiliary state; and in response to determining that the surgical robot system is in an auxiliary state, the surgical robot system provides assistance to one or more of the users when the surgical robot system is being used to perform the task.

[0007] A first aspect provides a system for providing assistance to a user of a surgical robot system, the system comprising: a surgical robot system, the surgical robot system comprising at least one surgical robot, the at least one surgical robot having a base and an arm extending from the base to an attachment of an instrument, the arm comprising a plurality of joints, whereby the configuration of the arm can be changed; and an auxiliary module, the auxiliary module comprising a list of task steps that can be provided with program assistance, the auxiliary module being configured to: receive status data indicating the status of the surgical robot system when the surgical robot system is being used to perform the task; determine from the status data whether the surgical robot system is in a program-assisted state, the surgical robot system being in a program-assisted state when the surgical robot system is being used or is to be used to perform one of the task steps that can be provided with program assistance; and in response to determining that the surgical robot system is in a program-assisted state, the surgical robot system provides program assistance to the user in performing the task steps when the surgical robot system is being used to perform the task.

[0008] The assistance module may be configured to cause the surgical robotic system to provide procedural assistance by causing the surgical robotic system to output information providing guidance on how to perform the task steps.

[0009] The information providing guidance on how to perform the task steps may include one or more of the following: information indicating one or more steps to be performed next; and information indicating which instrument should be attached to the at least one surgical robot next.

[0010] The system may also include a command interface, which includes one or more input devices, so that an operator of the surgical robotic system can request movement of an instrument attached to the at least one surgical robot in a desired manner; and the information providing guidance on how to perform the task may include information indicating which of the at least one surgical robot will be associated with each of the one or more input devices.

[0011] The information providing guidance on how to perform the task step may include information indicating a path that the active surgical robotic arm should follow in order to perform the task step in an efficient manner.

[0012] The auxiliary module may be configured to cause the surgical robotic system to display on a display screen a recommended path that the active surgical robotic arm should follow in order to perform the task step.

[0013] The assistance module may be configured to dynamically determine guidance information from status data related to previously performed tasks.

[0014] A second aspect provides a system for providing assistance to a user of a surgical robot system, the system comprising: a surgical robot system, the surgical robot system comprising at least two surgical robots, the at least two surgical robots each having a base and an arm extending from the base to an attachment of an instrument, each arm comprising a plurality of joints, whereby the configuration of the arm can be changed, wherein a powered instrument is attached to one of the at least two surgical robot arms and an endoscope is attached to another of the at least two surgical robot arms; and an auxiliary module, the auxiliary module being configured to: receive status data indicating the status of the surgical robot system when the surgical robot system is being used to perform a task; dynamically determine from the status data whether the surgical robot system is in a powered instrument dangerous state, the surgical robot system being in a powered instrument dangerous state when the powered instrument is in motion, when the powered instrument is not in the field of view of the endoscope, and / or when the powered instrument is in a dangerous position relative to a patient and / or another instrument; and in response to determining that the surgical robot system is in a powered instrument dangerous state, the surgical robot system provides powered instrument dangerous assistance to the user when the surgical robot system is being used to perform the task.

[0015] The powered instrument may be an electrosurgical instrument or an electrocautery instrument, which is powered by an electric current to perform a surgical function.

[0016] The status data may include information regarding the status of the powered device.

[0017] The information about the status of the powered device may include information about whether the powered device is currently powered on.

[0018] The assist module may be configured to cause the surgical robotic system to provide powered instrument hazard assist by causing the surgical robotic system to prevent the powered instrument from being energized.

[0019] The auxiliary module may be configured to cause the surgical robotic system to prevent the powered instrument from being powered on by sending a control signal to a control unit of the surgical robotic system, the control signal causing the control unit to prevent a power current from being sent to the powered instrument.

[0020] The auxiliary module may also be configured to request user input regarding whether the power-on prevention is to be overridden; and in response to receiving the user input that the power-on prevention is to be overridden, cause the surgical robotic system to allow the powered instrument to be powered on.

[0021] The auxiliary module can be configured to enable the surgical robot system to provide powered instrument hazard assistance by enabling the surgical robot system to output auxiliary information, wherein the auxiliary information notifies one or more users of the surgical robot system that the powered instrument hazard state has been detected and / or the powered instrument cannot be powered on.

[0022] Outputting the auxiliary information to the user may include displaying at least a portion of the auxiliary information on a display of the surgical robot system.

[0023] Outputting the auxiliary information to the user may include transmitting at least a portion of the auxiliary information to the user audibly.

[0024] The assistance module can also be configured to determine whether the surgical robot system is in a dangerous state from the status data; and in response to determining that the surgical robot system is in a dangerous state, the surgical robot system provides dangerous assistance to the user when the surgical robot system is being used to perform the task.

[0025] The auxiliary module may be configured to determine that the surgical robot system is in a dangerous state when the auxiliary module determines from the state data that the surgical robot system is in at least one of a collision state, a faulty instrument state, and an incorrect instrument state.

[0026] The surgical robot system may include at least two surgical robots, each surgical robot being attached to an instrument, and the auxiliary module may be configured to determine that the surgical robot system is in a collision state when the auxiliary module determines from the status data that at least two surgical robot arms have collided or are about to collide, at least two instruments have collided or are about to collide, or an instrument and a surgical robot arm have collided or are about to collide.

[0027] The auxiliary module can be configured to determine that the surgical robotic system is in an incorrect instrument state when the auxiliary module determines from the state data that an instrument that is not suitable for the task or the current step of the task has been attached to the surgical robotic arm.

[0028] The auxiliary module can be configured to cause the surgical robot system to automatically perform instrument replacement in response to determining that the surgical robot system is in an incorrect instrument state, so that the instrument that is unsuitable for the task or the current step of the task is replaced by an instrument that is suitable for the task or the current step of the task.

[0029] The auxiliary module can also be configured to determine whether the surgical robot system is in a predicted dangerous state from the status data, and the surgical robot system is in a predicted dangerous state when one or more danger indicators are detected from the status data; and in response to determining that the surgical robot system is in a predicted dangerous state, the surgical robot system provides predicted danger assistance to the user when the surgical robot system is being used to perform the task.

[0030] The one or more hazard indicators may include events or patterns that precede the hazard in the status data of previously performed tasks.

[0031] The task may be a surgical procedure performed on a patient, and at least one of the one or more risk indicators may include one or more of the following: one or more vital signs and / or one or more other health metrics of the patient falling outside a predetermined range; one or more vital signs and / or one or more other health metrics of an operator of the surgical robotic system falling outside a predetermined range; speaking loudly to one or more individuals present at the task; and speaking a warning word or phrase to one or more individuals present at the task.

[0032] The auxiliary module can also be configured to determine from the status data whether the surgical robot system is in a skill-assistance state, and when the surgical robot system is currently being used or is about to be used to perform a known skill, the surgical robot system is in the skill-assistance state; and in response to determining that the surgical robot system is in the skill-assistance state, the auxiliary module is configured to control one or more surgical robot arms to automatically perform the known skill.

[0033] The level of assistance provided may be based on the skill level of the operator of the surgical robotic system.

[0034] The assistance module may be configured to dynamically determine a skill level of the operator based on the status data.

[0035] A third aspect provides a computer-implemented method for providing assistance to a user of a surgical robot system, the method comprising: receiving status data indicating the status of the surgical robot system when the surgical robot system is being used to perform a task comprising multiple steps, the surgical robot system comprising at least one surgical robot, the at least one surgical robot system having a base and an arm extending from the base to an attachment of an instrument, the arm comprising multiple joints, thereby enabling the configuration of the arm to be changed; determining from the status data whether the surgical robot system is in a program-assisted state, the surgical robot system being in the program-assisted state when the surgical robot system is being used or is to be used to perform one of the task steps for which program assistance can be provided; and in response to determining that the surgical robot system is in the program-assisted state, the surgical robot system provides program assistance to the user in performing the task steps while the surgical robot system is being used to perform the task.

[0036] A fourth aspect provides a computer-implemented method for providing assistance to a user of a surgical robot system, the method comprising: receiving status data indicating the status of the surgical robot system when the surgical robot system is being used to perform a task, the surgical robot system comprising at least two surgical robots, each surgical robot having a base and an arm extending from the base to an attachment of an instrument, each arm comprising a plurality of joints, thereby enabling the configuration of the arm to be changed, wherein a powered instrument is attached to one of the at least two surgical robot arms and an endoscope is attached to another of the at least two surgical robot arms; dynamically determining from the status data whether the surgical robot system is in a powered instrument dangerous state, the surgical robot system being in a powered instrument dangerous state when the powered instrument is in motion, when the powered instrument is not in the field of view of the endoscope, and / or when the powered instrument is in a dangerous position relative to a patient and / or another instrument; and in response to determining that the surgical robot system is in a powered instrument dangerous state, the surgical robot system provides powered instrument dangerous assistance to the user when the surgical robot system is being used to perform the task.

[0037] A fifth aspect provides a system for providing assistance to a user of a surgical robot system, the system comprising: a surgical robot system, the surgical robot system comprising at least one surgical robot, the at least one surgical robot having a base and an arm extending from the base to an attachment of an instrument, the arm comprising a plurality of joints, thereby enabling the configuration of the arm to be changed; and an auxiliary module, the auxiliary module being configured to receive status data indicating the status of the surgical robot system when the surgical robot system is being used to perform a task; determine from the status data whether the surgical robot system is in an auxiliary state; and in response to determining that the surgical robot system is in an auxiliary state, enable the surgical robot system to provide assistance to the user when the surgical robot system is being used to perform the task.

[0038] The assist module may be configured to determine whether the surgical robotic system is in an assist state by comparing the state data with state data of a previously performed task.

[0039] The assistance module may be configured to enable the surgical robot system to provide assistance by controlling movement of a surgical robot arm.

[0040] The auxiliary module may be configured to control movement of the surgical robotic arm by causing the surgical robotic arm to move to automatically perform at least a portion of the task.

[0041] The auxiliary module may be configured to control the movement of the surgical robotic arm by inhibiting the movement of the surgical robotic arm.

[0042] The auxiliary module may be configured to control movement of the surgical robotic arm only in response to receiving input from a user confirming that the surgical robotic arm may be automatically controlled.

[0043] The assistance module may be configured to cause the surgical robot system to provide assistance by causing the surgical robot system to output assistance information to the user.

[0044] Outputting the auxiliary information to the user may include displaying at least a portion of the auxiliary information on a display of the surgical robot system.

[0045] Outputting the auxiliary information to the user includes transmitting at least a portion of the auxiliary information to the user in an audible manner.

[0046] The auxiliary module may be configured to determine that the surgical robot system is in an auxiliary state when the auxiliary module determines from the state data that the surgical robot system is in a dangerous state.

[0047] The auxiliary module can be configured to determine that the surgical robot system is in a dangerous state when the auxiliary module determines from the status data that the surgical robot system is in at least one of a collision state, a faulty instrument state, an incorrect instrument state, a powered instrument dangerous state, and a potentially dangerous state.

[0048] The surgical robot system may include at least two surgical robots, and the auxiliary module may be configured to determine that the surgical robot system is in a collision state when the auxiliary module determines from the status data that the at least two surgical robot arms have collided or are about to collide, at least two instruments have collided or are about to collide, or an instrument and a surgical robot arm have collided or are about to collide.

[0049] The auxiliary module can be configured to determine that the surgical robotic system is in an incorrect instrument state when the auxiliary module determines from the state data that an instrument that is not suitable for the task or the current step of the task has been attached to the surgical robotic arm.

[0050] The auxiliary module may be configured to cause the surgical robot system to automatically perform instrument replacement in response to determining that the surgical robot system is in an incorrect instrument state, so that the incorrect instrument is replaced with a suitable instrument.

[0051] The surgical robot system may include at least two surgical robots; a powered instrument may be attached to one of the surgical robot arms and an endoscope may be attached to another of the surgical robot arms; the auxiliary module may be configured to determine that the surgical robot system is in a powered instrument dangerous state when the auxiliary module determines from the state data that the powered instrument is in motion, the powered instrument is not in the field of view of the endoscope, and / or the powered instrument is in a dangerous position relative to a patient and / or another instrument.

[0052] The auxiliary module can be configured to determine that the surgical robot system is in an auxiliary state when the auxiliary module determines from the state data that the surgical robot system is in a predicted dangerous state, and the surgical robot system is in a predicted dangerous state when one or more danger indicators are detected from the state data.

[0053] The one or more hazard indicators may include events or patterns that precede the hazard in the status data of previously performed tasks.

[0054] The task may be a surgical procedure on a patient, and at least one of the one or more risk indicators may include one or more of the following: one or more of the patient's vital signs and / or other health metrics falling outside a predetermined range; one or more of the operator's vital signs and / or other health metrics falling outside a predetermined range; speaking loudly to one or more individuals present at the task; and speaking a warning word or phrase to one or more individuals present at the task.

[0055] The auxiliary module can be configured to determine that the surgical robot system is in the auxiliary state when the auxiliary module determines from the state data that the surgical robot system is in the skill-assistance state, and that the surgical robot system is in the skill-assistance state when the surgical robot is currently being used or is to be used to perform a known skill; and in response to determining that the surgical robot system is in the skill-assistance state, the auxiliary module is configured to control one or more surgical robot arms to automatically perform the known skill.

[0056] The level of assistance provided may be based on the skill level of the operator of the surgical robotic system.

[0057] The assistance module may be configured to dynamically determine a skill level of the operator based on the status data.

[0058] The task may be a surgical procedure.

[0059] A sixth aspect provides a computer-implemented method for providing assistance to a user of a surgical robot system, the method comprising: receiving status data indicating the status of the surgical robot system when the surgical robot system is being used to perform a task, the surgical robot system comprising at least one surgical robot having a base and an arm extending from the base to an attachment of an instrument, the arm comprising a plurality of joints, thereby enabling the configuration of the arm to be changed; determining from the status data whether the surgical robot system is in an assistive state; and in response to determining that the surgical robot system is in an assistive state, causing the surgical robot system to provide assistance to the user when the surgical robot system is being used to perform the task.

[0060] The above features may be combined as appropriate, as will be apparent to the skilled person, and may be combined with any aspects of the examples described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Examples will now be described in detail with reference to the accompanying drawings, in which:

[0062] Figure 1 is a schematic diagram of an exemplary surgical robot performing a surgical procedure;

[0063] Figure 2 is a schematic diagram of an exemplary surgical instrument;

[0064] Figure 3 is a block diagram of an exemplary system for providing assistance to one or more users of a surgical robotic system;

[0065] Figure 4 is a schematic diagram of an exemplary surgical robotic system; and

[0066] Figure 5 is a flow chart of an exemplary method of providing assistance to one or more users of a surgical robotic system.

[0067] The accompanying drawings illustrate various examples. It will be appreciated by the skilled person that the element boundaries (e.g., boxes, groups of boxes, or other shapes) shown in the figures represent an example of boundaries. In some examples, an element may be designed as multiple elements, or multiple elements may be designed as one element. Where appropriate, common reference numerals are used throughout the accompanying drawings to indicate similar features. DETAILED DESCRIPTION

[0068] The following description is presented by way of example to enable those skilled in the art to make and use the invention. The invention is not limited to the embodiments described herein, and various modifications to the disclosed embodiments will be apparent to those skilled in the art. The embodiments are described only by way of example.

[0069] The present invention describes a method and system for dynamically providing assistance to one or more users of a surgical robot system while the surgical robot system is being used to perform a task. The system includes a surgical robot system, the surgical robot system including one or more surgical robots; and an auxiliary module that communicates with the surgical robot system. The auxiliary module is configured to: receive state data indicating the state of the surgical robot system while the surgical robot system is being used to perform a task; determine from the state data that the surgical robot system is in an auxiliary state; and in response to determining that the surgical robot system is in an auxiliary state, cause the surgical robot system to provide assistance to one or more users of the surgical robot system while the surgical robot is being used to perform a task. Users of the surgical robot system include an operator (e.g., a surgeon) who controls the surgical robot system and / or other individuals who are part of a team (e.g., a surgical or operating room team) that is present and participates in a task (e.g., a surgery).

[0070] First reference Figure 3, which illustrates an exemplary system 300 for providing assistance to one or more users of a surgical robotic system. The system 300 includes a surgical robotic system 302 and an assistance module 304 in communication with the surgical robotic system 302. The surgical robotic system 302 includes at least one surgical robot having a base and an arm extending from the base to an attachment for a surgical instrument, the arm including a plurality of joints whereby the configuration of the arm can be changed. Figure 4 An exemplary surgical robotic system 302 is described.

[0071] The auxiliary module 304 is configured to receive status data indicating a status of the surgical robotic system 302. As described in more detail below, the status data may include any information about the surgical robotic system 302 or about the environment in which the surgical robotic system 302 operates. The auxiliary module 304 may receive status data from the surgical robotic system itself (e.g., data related to the position and / or movement of the surgical robotic arm) and / or receive status data from an external source. The auxiliary module 304 may receive status data from the surgical robotic system 302 and / or from an external source via any suitable means. For example, the auxiliary module 304 may be connected via a wireless or wired communication connection, such as, but not limited to, an Ethernet connection, connect, The auxiliary module 304 may receive status data from the surgical robotic system 302 and / or from an external source via a surgical robotic system 302 connection, a near field communication (NFC) connection, etc. At least a portion of the status data is provided to the auxiliary module 304 in real time (or substantially real time) while the surgical robotic system is performing a task (e.g., a surgical procedure).

[0072] The assist module 304 is configured to analyze the state data to determine whether the surgical robotic system is in an assist state (i.e., a state in which assistance can be provided to one or more users of the surgical robotic system). As described in more detail below, the assist module 304 can determine that the surgical robotic system is in an assist state by identifying certain events or patterns in the state data and / or by comparing the state data with state data of previously performed tasks. For example, the assist module 304 can be configured to determine that the surgical robotic system is in an assist state when the assist module 304 determines that a collision is about to occur or has occurred between the arms of the surgical robotic system.

[0073] If the assist module 304 determines that the surgical robot system 302 is in an assist state, the assist module 304 causes the surgical robot system to dynamically provide assistance to one or more users of the surgical robot system in performing a task. As described in more detail below, causing the surgical robot system to provide assistance to one or more users of the surgical robot may include controlling all or part of the movement of one or more surgical robot arms and / or causing the surgical robot system to provide assistance information to the user (e.g., audibly or through a display). The type and form of assistance provided to the user can be based on the specific assist state in which the surgical robot system 302 is located. For example, if the assist module 304 determines that a collision is about to or has occurred between the arms of the surgical robot system, the assist module 304 can be configured to cause the surgical robot system to suppress the movement of the robot arms. In some cases, before causing the surgical robot system 302 to provide assistance to the user of the surgical robot system in performing a task, the assist module 304 can be configured to request input from the user as to whether the user wishes to provide assistance. In these cases, the assist module 302 can be configured to cause the surgical robot system 302 to provide assistance to the user of the surgical robot system only in response to receiving input indicating that assistance will be provided. Reference is made below Figure 5 Exemplary methods are described that may be performed by the assistance module 304 to provide assistance in performing a task to one or more users of the surgical robotic system.

[0074] The auxiliary module 304 may include a processor 306 and a memory 308. The memory 308 stores software in a non-transitory manner that can be executed by the processor 306 to receive state data, determine from the state data whether the surgical robot system 302 is in an auxiliary state, and if the auxiliary state is detected, cause the surgical robot system to provide assistance to one or more users of the surgical robot system. In some cases, the auxiliary module 304 may also include a data repository 310 for storing state data of previously performed tasks. As described above, in some cases, the auxiliary module 304 may be configured to identify that the surgical robot system is in an auxiliary state by comparing the received state data with the state data of previously performed tasks.

[0075] Reference now Figure 4 , which shows an exemplary surgical robotic system 400. Figure 4The surgical robot system 400 includes a surgical robot having an arm 402 extending from a base 404. The arm 402 includes a plurality of rigid branches 406. The branches are connected by revolute joints 408. The most proximal branch 406a is connected to the base 404 by joint 408a. It is connected in series with the other branches by further joints 408. In some cases, the wrist 410 may be comprised of four separate revolute joints. The wrist 410 connects one branch (406b) to the most distal branch (406c) of the arm 402. The most distal branch 406c carries an attachment 412 for an instrument 414. Each joint 408 of the arm 402 has: one or more motors 416 that can be operated to cause rotational motion at the corresponding joint; and one or more position and / or torque sensors 418 that provide information about the current configuration and / or load at the joint. Appropriately, the motors 416 are arranged proximal to the joints whose motion they drive in order to improve weight distribution. For clarity, Figure 4 Only some of the motors and sensors are shown. The arm may generally be as described in our co-pending patent application PCT / GB2014 / 053523.

[0076] The arms terminate in attachment means 412 for connection with an instrument 414. The instrument 414 may be provided with reference to Figure 2 The form described. The diameter of the instrument can be less than 8mm. In some cases, the instrument can have a diameter of 5mm. In other cases, the diameter of the instrument can be less than 5mm. The instrument diameter can be the diameter of the shaft. The instrument diameter can be the diameter of the articulation profile. Suitably, the diameter of the articulation profile matches the diameter of the shaft or is narrower than the diameter of the shaft. Attachment 412 includes an articulated drive assembly for driving the instrument. The removable interface element of the drive assembly interface mechanically engages the corresponding removable interface element of the instrument interface so as to transmit drive from the robot arm to the instrument. During a typical operation, an instrument is replaced several times. Therefore, during operation, the instrument can be attached to the robot arm and can be removed from the robot arm. The features of the drive assembly interface and the instrument interface facilitate their alignment when they are engaged with each other, thereby reducing the accuracy required for the operator (e.g., surgeon) to align.

[0077] Instrument 414 includes an end effector for performing an operation. The end effector can take any suitable form. For example, the end effector can be a smooth jaw, a serrated jaw, a clamp, a pair of shears, a needle for suturing, a camera, a laser, a knife, a stapler, a cauterizer, an aspirator. Figure 2As described, the instrument comprises an articulated portion between the instrument shaft and the end effector. The articulated portion comprises several joints that allow the end effector to move relative to the instrument shaft. The joints in the articulated portion are actuated by drive elements, such as cables or the like. These drive elements are fixed to interface elements of the instrument interface at the other end of the instrument shaft. Thus, the robot arm transmits drive to the end effector in the following manner: movement of the drive assembly interface element moves the instrument interface element, which moves the drive element, which moves the articulated joint, and the articulated joint moves the end effector.

[0078] Controllers for motors, torque sensors, and encoders are distributed with the robot arm. The controller is connected to a control unit 420 via a communication bus. The control unit 420 includes a processor 422 and a memory 424. The memory 424 stores software in a non-transient manner, which can be executed by the processor 422 to control the operation of the motor 416 so that the arm 402 operates in the manner described herein. Specifically, the software can control the processor 422 to drive the motor (e.g., via a distributed controller) according to inputs from the sensor 418 and inputs from the operator (e.g., surgeon) command interface 426. The control unit 420 is connected to the motor 416 to drive them according to the output generated by the execution of the software. The control unit 420 is connected to the sensor 418 for receiving sensing inputs from the sensor, and the control unit is connected to the command interface 426 to receive inputs from the command interface. For example, the corresponding connection can be a cable or an optical cable, or can be provided by a wireless connection. The command interface 426 includes one or more input devices, whereby the operator (e.g., surgeon) can request the movement of the end effector in a desired manner. The input device can be, for example, a manually operable mechanical input device, such as a control handle or joystick, or a contactless input device, such as an optical gesture sensor. The software stored in the memory 424 is configured to respond to these inputs according to a predetermined control strategy and to move the joints of the arm and the instrument accordingly. The control strategy can include a safety feature that regulates the movement of the arm and the instrument in response to the command input. Thus, in general, an operator (e.g., a surgeon) at the command interface 426 can control the instrument 414 to move in a manner to perform a desired surgical procedure. The control unit 420 and / or the command interface 426 can be remote from the arm 402.

[0079] although Figure 4The exemplary surgical robotic system 400 includes a single surgical robot having a single robotic arm, but other surgical robotic systems may include multiple surgical robots and / or multiple robotic arms. For example, other exemplary surgical robotic systems may include a surgical robot having multiple robotic arms that can each receive and manipulate surgical instruments; or they may include multiple surgical robots that each have a robotic arm that can receive and manipulate surgical instruments.

[0080] Reference now Figure 5 , which illustrates an example method 500 that may be performed by Figure 3 The assist module 304 of the method 500 is implemented to provide assistance to one or more users of the surgical robotic system while the surgical robotic system is being used to perform a task (e.g., perform surgery). The method 500 begins at block 502, where the assist module 304 receives state data indicating a state of the surgical robotic system 302. The state data may include any information about the surgical robotic system 302 or about the environment in which the surgical robotic system 302 is operating. At least a portion of the state data is received in real time (or substantially real time) while the surgical robotic system is being used to perform a task.

[0081] For example, the state data may include information or data describing the current state of the robotic arm, such as, but not limited to, position and / or torque information indicating the position and / or movement of the robotic arm or its joints. Specifically, the state data may include position and / or torque data generated by the position sensor and / or torque sensor 418. It will be apparent to those skilled in the art that this is merely an example and that the surgical robot may include other sensors that provide information about the state of the robotic arm.

[0082] The state data may also or alternatively include data or information describing the state of the instrument attached to the robot arm. For example, the state data may include information identifying whether the instrument is attached to the robot arm, and if attached, information identifying the type of instrument attached to the robot arm. In particular, the surgical robot system may include: a device for detecting whether the instrument is attached to the arm (for example, the surgical robot system may include an instrument engagement device, such as described in the applicant's published patent application GB2552855 A, which is incorporated herein by reference); a device for automatically detecting the type of instrument (for example, each instrument may include an RFID or other component that is configured to automatically provide information about its identity to the surgical robot system when it is attached to the robot arm), and this information may be provided to the auxiliary module 304 as state data. The state data may also or alternatively include such information, such as but not limited to position and / or torque information indicating the position and / or movement of the instrument attached to the robot arm. It is obvious to those skilled in the art that this is just an example, and the surgical robot or the instrument itself may include other sensors that provide information about the state of the instrument.

[0083] When at least one instrument attached to the robotic arm is a powered instrument, such as, but not limited to, an electrocautery instrument or an electrosurgical instrument that supplies an electric current to perform a surgical function (e.g., cauterization, cutting, etc.), the status data may include information about the status of the powered instrument, such as, but not limited to, whether the powered instrument is currently powered, and if so, the waveform of the current used to power the instrument.

[0084] When at least one of the instruments attached to the robotic arm is an endoscope that is inserted into the body and captures images (e.g., video) of the patient, the state data may include images (e.g., video) captured by the endoscope. As described in applicant's co-pending UK patent application entitled "AUTOMATIC ENDOSCOPE VIDEO AUMENTATION" filed on the same day as the present application (which is incorporated herein by reference), in some cases, when a task (e.g., a surgical procedure) is being performed using a surgical robotic system and an event is detected during the task, the images (e.g., video) captured by the endoscope may be automatically enhanced with information identifying the detected event. The state data may also or alternatively include information indicating the position, movement, and / or field of view of the endoscope.

[0085] The state data may also or alternatively include data describing the state of an operator input device used to control the operation of the robot arm and the instrument. As described above, the operator input device may be, for example, a manually operable mechanical input device such as a control handle or a joystick. In these cases, the state data may include, for example, but not limited to, data indicating position and / or torque data of the position and / or movement of the input device.

[0086] The state data may also or alternatively include information about the operating mode of the surgical robotic system 302. For example, in some cases, the surgical robotic system is capable of operating in one of a plurality of modes, such as, but not limited to, an active mode, in which an operator (e.g., a surgeon) controls one or more of the robotic arms to perform a task; a selection mode, in which an operator (e.g., a surgeon) is selecting or switching the robotic arm they will control from a command interface; and an instrument change mode, in which at least one of the instruments attached to the robotic arm is removed, attached, and / or replaced. There may also be several types of active modes. For example, in some cases, there may be an endoscope action mode, in which an operator controls an endoscope attached to a robotic arm; and an instrument action mode, in which an operator controls one or more other instruments attached to a robotic arm. In these cases, the state data may include information about the current operating mode of the surgical robotic system. It will be apparent to one skilled in the art that these are exemplary operating modes of the surgical robotic system, and that additional or different operating modes may exist for the surgical robotic system.

[0087] When the task being performed is a surgical procedure performed on a patient, the state data may also or alternatively include data related to the patient. For example, the state data may include patient health information or data describing the patient's health status. For example, the state data may include information about the patient's vital signs (e.g., body temperature, pulse rate, respiratory rate) or other health metrics (e.g., blood pressure, oxygen saturation, blood sugar / sugar). Typically, during surgery, the patient's vital signs and other health metrics are measured by special medical measurement devices, such as heart rate monitors, pulse oximeters, continuous glucose monitors (CGM) devices, etc. In these cases, the medical measurement device can be configured to send vital sign information or other patient measurement information to the auxiliary module via, for example, a communication network (e.g., a wired or wireless communication network). However, in other cases, information about the patient's vital signs and / or other patient health metrics can be obtained in another way. For example, in some cases, the operating room where the surgical procedure is being performed may be equipped with a video and / or audio recording device, and vital sign data and / or other patient measurement data may be obtained from the video and / or audio recording. For example, where a medical measurement device produces an audible representation of a measurement (e.g., a set of audible tones or "beeps"), an estimate of the measurement may be obtained by analyzing the audio recorded by a video and / or audio recording device. The status data may also or alternatively include other information about the patient, such as, but not limited to, demographic information, such as age and weight, and other medical information, such as previous scans (e.g., X-rays, MRIs), previous results, medical history, diagnostic history, and / or treatment plans.

[0088] In some cases, the state data may also include surgeon health information or data describing the health status of the surgeon controlling the surgical robot system. For example, the state data may include information or data about one or more vital signs (e.g., body temperature, pulse rate, respiratory rate) and / or other health metrics (e.g., blood pressure) of the surgeon. For example, in some cases, where the input device is a handheld controller, the handheld controller may include a sensor that measures the surgeon's heart rate, the surgeon's tremors in movement, and / or the surgeon's sweat rate, and provides information or data related to the measured heart rate, tremors, and / or sweat rate to the auxiliary module 304. In other cases, there may be external medical devices configured to measure the surgeon's vital signs or other health metrics, etc. As described above with respect to patient vital signs and other health metrics, the medical device that measures the surgeon's vital signs or other health metrics may be configured to provide data about the measured metrics to the auxiliary module 304 via, for example, a communication network, or an estimate of the measured metrics may be obtained by other means (e.g., by analyzing the video and / or audio of the operating room).

[0089] As described above, in some cases, the room where the task is being performed (e.g., an operating room) may be equipped with audio and / or video recording equipment to record the sounds in the room and / or the visual images of the room while the task is being performed. In these cases, the state data may also or alternatively include video and audio data captured by the audio and / or video recording equipment.

[0090] In some cases, the state data may also or alternatively include information about the task being performed. For example, the state data may include information about the task (e.g., a surgical procedure) being performed by the surgical robotic system. In some cases, where the task includes several steps, the state information may include information indicating the step of the task currently being performed by the surgical robotic system. For example, an operator of the surgical robotic system (e.g., a surgeon) may be able to indicate which step of the task he / she is currently performing, and this information may be provided to the auxiliary module 304 as state data.

[0091] In some cases, the status data may also or alternatively include information about a user of the surgical robotic system (e.g., an operator (e.g., a surgeon) controlling the surgical robot and / or other members of a task team (e.g., a surgical team)). For example, in some cases, before starting a task, information may be provided to the surgical robotic system and / or an auxiliary module indicating to the user that the task will be performed. In other cases, the user (e.g., a team of surgeons) performing the task (e.g., surgery) may be equipped with a device (e.g., an RFID device, a mobile phone, a tablet computer, etc.) that automatically transmits information identifying the user to the surgical robotic system and / or the auxiliary module 304 when the user is in the vicinity of the surgical robotic system. An example of such a system is described in the applicant's co-pending UK patent application entitled "DEVICE INTEROPERATION" filed on the same day as the present application, which is incorporated herein by reference.

[0092] Once the auxiliary module 304 has received the status data, the method 500 proceeds to block 504. In some cases, the auxiliary module 304 may be configured to collect status data within a predetermined amount of time before proceeding to block 504. The predetermined time period may be short (e.g., less than 50 ms) so that the auxiliary module 304 can quickly respond to the detected auxiliary status. In other cases, the auxiliary module 304 may only have a fixed amount of memory for storing status data, and may be configured to collect status data until the memory for storing status data becomes full.

[0093] At block 504, the assist module 304 determines whether the surgical robot system 302 is in an assist state based on the state data. The term "assist state" is used herein to mean that the surgical robot system is in a state where assistance can be provided to one or more users of the surgical robot system via the surgical robot system. The assist state includes, but is not limited to, a dangerous state where a hazard or potential hazard has been detected and assistance can be provided to avoid or minimize the hazard; and an assist state where a hazard or potential hazard has not been detected but assistance can be provided to one or more users in performing a task (e.g., surgery).

[0094] Examples of hazardous states include, but are not limited to; a collision state in which a collision has occurred (or is about to occur) between multiple robotic arms, multiple instruments, or a robotic arm and an instrument; an incorrect ended instrument state in which an incorrect instrument for the current task or step of a task is attached to the robotic arm; a faulty instrument state in which one of the instruments attached to the robotic arm fails or needs to be replaced; a powered instrument hazardous state in which a hazard associated with a powered instrument is detected; and a potentially hazardous state in which a hazard is predicted based on, for example, an event or pattern detected from state data.

[0095] Examples of assistance states include, but are not limited to, a skill assistance state in which a particular skill is being performed using the surgical robotic system, where assistance may be provided to perform the skill, and a procedure assistance state in which the surgical robotic system may provide procedure assistance in performing and / or preparing for a task or a portion of a task. The term "skill" is used herein to refer to different activities that may be performed using the surgical robotic system, such as, but not limited to, suturing or knot tying. A skill may be performed one or more times during a task (e.g., a surgical procedure).

[0096] The auxiliary module 304 can be configured to determine whether the surgical robot system 302 is in an auxiliary state independently based on the state data (e.g., by detecting an event or pattern from the state data) and / or based on a comparison of the state data with state data of a previously performed task. For example, as described above, in some cases, the auxiliary module 304 may include or be able to access a data repository 310 of state data of previously performed tasks. The previously performed tasks may be performed by the same surgical robot system and / or another surgical robot system. The previously performed tasks may be performed by the same operator (e.g., a surgeon) and / or other operators (e.g., a surgeon). The previously performed tasks may include the same type of tasks (e.g., the same type of surgical procedure) and / or other types of tasks (e.g., different types of surgical procedures or different surgical procedures) relative to the task currently being performed.

[0097] As described above, in some cases, the state data may include data or information indicating the position and / or motion of the robotic arm and / or the instrument attached thereto. In these cases, the auxiliary module 304 may, for example, be configured to detect that the surgical robot system is in a collision state (i.e., the robotic arm or instrument is about to or has collided with another robotic arm or instrument) based on the data or information indicating the position and / or motion of the robotic arm and / or the instrument. For example, the auxiliary module 304 may be configured to estimate the trajectory of the robotic arm and / or the instrument based on the position and / or motion data of the robotic arm and / or the instrument, and estimate that a collision will occur based on the estimated trajectory. The auxiliary module 304 may also or alternatively be configured to determine that a collision has occurred between arms, between instruments, or between an arm and an instrument, for example by (i) determining or estimating the position of the arms and / or instruments from position and / or motion data, and determining that the positions of one or more instruments and / or one or more arms are the same or substantially similar; or (ii) determining from position and / or motion information (e.g., torque data) that equal and opposite forces have been detected on two arms and / or instruments, as described in the applicant's UK patent application No. 1810754.0 (which application is incorporated herein by reference).

[0098] The auxiliary module 304 can be configured, for example, to detect that the surgical robot system 302 is in an incorrect instrument state when the auxiliary module 304 detects from the state data that a particular type of instrument has been attached to the arm and the particular type of instrument is not suitable for the task being performed or the particular step of the task currently being performed. As described above, in some cases, the state data may include information indicating the type of instrument currently attached to the robot arm. In these cases, the auxiliary module 304 can be configured to detect that the surgical robot system 304 is in an incorrect instrument state by comparing the type of instrument currently attached to the robot arm with a list of instruments suitable for performing the current task or the current step of the task. In some cases, the list of instruments suitable for performing a particular task or the step of a task can be predetermined and loaded into the auxiliary module 304 before the task begins. In other cases, the auxiliary module 304 can be configured to dynamically generate a list of instruments suitable for performing a task or the step of a task based on the state data of a previously performed task. For example, the auxiliary module 304 can be configured to determine a group of instruments that have been used for a particular task or a particular step of a task from the state data of a previously performed task.

[0099] As described above, in some cases, the state data may include information identifying the task being performed and optionally identifying the steps of the task being performed. In these cases, the auxiliary module 304 can directly identify the current task and optionally the current step of the task from the task / step identification information. In the case where the information that explicitly identifies the task and / or the step of the task currently being performed is not received as part of the state data, the auxiliary module 304 can be configured to dynamically detect the task and / or the current step of the task from other state data. For example, the auxiliary module 304 can be configured to detect the task and / or the step of the task based on the pattern matching of the pattern of the state data of the previously performed task detected in the state data. In particular, the auxiliary module 304 or based on another computing device can be configured to analyze the state data related to the task of the same type as the current task previously performed (for example, the state data related to the same surgical procedure or the same type of surgical procedure); identify a set of common steps in the task based on the analysis; and generate a map linking the identified steps. In some cases, the step of the task (and therefore the current step of the task) can be identified based on one or more of the combination of instruments used and / or the movement of the instrument and / or robot arm.

[0100] The auxiliary module 304 can be configured to detect that the surgical robot system 302 is in a faulty instrument state if the auxiliary module 304 detects that, for example, an instrument attached to the robot arm has been used a predetermined number of times. Specifically, many surgical instruments can only be safely used a predetermined number of times before they should be replaced. The number of times a surgical instrument can be used before it must be replaced can vary depending on the instrument type and / or instrument manufacturer. As described above, in some cases, the state data may include information identifying the instrument attached to the arm. For example, each instrument can be identified by a unique identifier. In these cases, the auxiliary module 304 can maintain or access a list of each instrument that can be used to perform a task (e.g., each instrument owned by a hospital performing a surgical procedure), which indicates how many times the instrument has been used. When the auxiliary module 304 detects (via an instrument identifier) ​​that a particular instrument has been used in the current task, it updates the list to reflect that the instrument has been used for an additional time. If the auxiliary module 304 detects that the number of uses of a particular instrument attached to the arm has exceeded a predetermined threshold, the auxiliary module 304 can determine that the surgical robot system is in a faulty instrument state.

[0101] The auxiliary module 304 can be configured to detect that the surgical robot system 302 is in a powered instrument dangerous state if the state data indicates that the surgical robot system is in an unsafe state to power on a powered instrument and the operator (e.g., a surgeon) has attempted to power on the powered instrument. As described above, the term "powered instrument" herein refers to surgical instruments, such as, but not limited to, electrosurgical instruments and electrocautery instruments that are powered on to perform surgical functions (e.g., cutting, cauterization, etc.). The auxiliary module 304 can be configured to detect that the surgical robotic system 302 is in an unsafe state with a powered instrument powered on when the auxiliary module 304 detects the following from the status data: the powered instrument attached to one of the robotic arms is outside the field of view of the endoscope; the powered instrument attached to one of the robotic arms is not in contact with the patient's flesh; the powered instrument attached to one of the robotic arms is not in a static position (e.g., it is being moved by the corresponding robotic arm); the powered instrument attached to one of the robotic arms is within a predetermined distance of certain parts of the patient's body (e.g., a blood vessel); and / or the powered instrument attached to one of the robotic arms is in contact with another instrument (which may be another powered instrument or may be a non-powered instrument).

[0102] The auxiliary module 304 can be configured to detect that a powered instrument attached to one of the robotic arms is outside of the field of view (FOV) of the endoscope by, for example, determining the field of view of the endoscope from information about the position and / or movement of the endoscope, determining the position of the powered instrument from information about the position and / or movement of the arm attached to the powered instrument and / or the powered instrument, and comparing the determined position of the powered instrument to the determined FOV of the endoscope. Alternatively or in addition, the auxiliary module 304 can be configured to detect that a powered instrument is outside of the FOV of the endoscope by analyzing the endoscope video using video analytics techniques to determine whether the powered instrument is visible in the endoscope video.

[0103] The auxiliary module 304 can be configured to detect that a powered instrument attached to one of the arms is not in contact with the patient's flesh or is within a predetermined distance of certain parts of the patient's body, for example by analyzing the endoscopic video using video analysis techniques to determine whether the powered instrument is in contact with the patient's flesh or whether the powered instrument is within a predetermined distance of certain parts of the patient's body.

[0104] The auxiliary module 304 can be configured to detect that a powered instrument attached to one of the robotic arms is not stationary by analyzing position and / or movement information related to the robotic arm and / or instrument to determine whether the associated arm is in motion. If the auxiliary module 304 determines that the associated arm is in motion, the auxiliary module 304 can determine that the powered instrument is in an unsafe condition for energization, and if an operator (e.g., a surgeon) attempts to energize the powered instrument in this unsafe condition, the auxiliary module 304 can determine that a powered instrument hazard exists. In some cases, the auxiliary module 304 can be configured to determine that the powered instrument is in an unsafe condition for energization only when the powered instrument is in motion and the movement or motion is not expected or is outside of an acceptable range.

[0105] The auxiliary module 304 can be configured to detect that the surgical robotic system 302 is in a potentially dangerous state if the auxiliary module detects one or more danger indicators from the state data of the current task. Danger indicators can be conditions that do not clearly indicate that a problem exists by themselves, but can indicate that a problem exists. Danger indicators can include, but are not limited to: one or more patient vital signs and / or other health metrics that fall outside of a predetermined range; one or more operator (e.g., surgeon) vital signs and / or other health metrics that fall outside of a predetermined range; one or more individuals present for the task (e.g., one or more people on the surgical team) speaking loudly; and one or more individuals present for the task (e.g., one or more people on the surgical team) speaking a warning word or phrase.

[0106] As described above, in some cases, the state data may include patient health information or data indicating the state of one or more of the patient's vital signs and / or one or more other health metrics. In these cases, the auxiliary module 304 may be configured to detect a danger indicator when the patient health information or data indicates that one or more of the patient's health metrics (e.g., vital signs) have fallen outside a predetermined range. The predetermined range can vary between health metrics (e.g., different ranges can exist for respiratory rate and heart rate). Similarly, in some cases, the state data may include surgeon health information or data indicating the state of one or more of the surgeon's vital signs and / or one or more other health metrics. In these cases, the auxiliary module 304 may be configured to detect a danger indicator when the surgeon health information or data indicates that one or more of the surgeon's health metrics (e.g., vital signs) have fallen outside a predetermined range. In some cases, before starting a task, a predetermined range of different medical metrics of the patient and / or surgeon can be preloaded into the auxiliary module 304. In some cases, the predetermined range can be learned from the state data of a previously performed task. For example, the auxiliary module 304 or another computing device can identify the levels of different health metrics just before the negative outcome occurred from the status data of the previously performed task, and generate a predetermined range from the identified levels (e.g., a range that covers the lowest and highest identified levels can be selected). The predetermined range can be determined for all tasks, or different predetermined ranges can be determined for different tasks (i.e., different surgical procedures). For example, it can be determined from the status data of the previously performed task that a heart rate within the range of X to Y is normal for surgical type A, but abnormal for surgical type B.

[0107] As described above, in some cases, the state data may include a video and / or audio recording of the room (e.g., an operating room) in which the task is performed. In these cases, the auxiliary module 304 may be configured to detect a danger indicator when the auxiliary module 304 detects from the video and / or audio recording that an individual in the room has spoken loudly. Specifically, if a person speaks loudly, this may indicate that there is some friction between the people in the room and / or that there is a dispute or problem in the room. The auxiliary module 304 can detect the elevated voice using any known speech analysis technology. In the case where the state data includes a video and / or audio recording, the auxiliary module 304 may also or alternatively be configured to detect a danger indicator when the auxiliary module 304 detects from the video and / or audio recording that an individual in the room has spoken a warning word or phrase (i.e., a word or phrase indicating that there may be a problem). The auxiliary module 304 may be configured to detect the warning word and / or phrase in the video and / or audio recording using any known speech recognition or analysis technology. Before starting a task, a list of warning words and / or phrases may be preloaded into the auxiliary module 304. In some cases, the list of warning words and / or phrases may be learned from the state data of previously performed tasks. For example, the auxiliary module 304 or another computing device can identify from the state data of each of a plurality of previously performed tasks any words or phrases spoken within a predetermined time period prior to a negative outcome, and generate a list of the identified words or phrases that are common between a predetermined number of previously performed tasks.

[0108] It will be apparent to those skilled in the art that these are exemplary risk indicators, and there may be other conditions and / or events, etc., that indicate potential problems or disputes. In some cases, risk indicators may be learned from state data of previously performed tasks (e.g., surgeries). For example, in some cases, state data related to previously performed tasks may be stored in the data repository 310, and an event or series of events prior to a negative outcome may be identified, and those identified events or series of events that are common in a predetermined number of previously performed tasks may be identified as risk indicators. For example, if the state data of multiple previously performed tasks indicate that steps A, B, and C are performed before a negative outcome or consequence occurs, the risk indicator may be identified as the execution of steps A and B, and step C may be performed. The state data of previously performed tasks may include state data of previously performed tasks performed by the same operator (e.g., a surgeon) and / or by other operators (e.g., a surgeon). The state data of previously performed tasks may include only state data related to the same task (e.g., a surgical procedure), or may include state data related to multiple different tasks (e.g., different surgical procedures).

[0109] The auxiliary module 304 may also or alternatively be configured to detect that the surgical robot system 302 is in a potentially dangerous state based on an evaluation of the performance of an operator (e.g., a surgeon) controlling the surgical robot system. For example, if the auxiliary module 304 detects that the performance of the operator (e.g., a surgeon) has fallen below a predetermined threshold, the auxiliary module 304 may detect that the surgical robot system 302 is in a potentially dangerous state. In some cases, the auxiliary module 304 may be configured to detect that the performance of the operator (e.g., a surgeon) has fallen below a predetermined threshold by comparing the state data of the current task with the state data of a previously performed task (e.g., a task of the same type). For example, the auxiliary module 304 may access a data repository 310 in which state data of previously performed tasks is stored and in which the performance of the operator (e.g., a surgeon) of the surgical robot system has been evaluated or identified (e.g., the performance may be sorted by a predetermined scale); and the state data may be compared with the state data stored in the repository to determine the performance level of the current operator (e.g., a surgeon) of the surgical robot system.

[0110] In some cases, the auxiliary module 304 can determine the performance level of the current operator (e.g., surgeon) operating the surgical robot system by comparing one or more performance metrics of the current operator (e.g., surgeon) with the performance metrics of operators (e.g., surgeons) of previously performed tasks. In some cases, the performance metrics may include one or more of the following: the path taken by the robot arm or instrument when performing a task or task step; the smoothness of the movement of the robot arm and / or instrument when performing a task or task step; and / or the time taken to perform a task or task step. In the case where the state data includes information describing the position and / or movement of the arm and / or instrument of the surgical robot system 302, the auxiliary module 304 can be configured to determine the path of the robot arm and / or instrument from the information describing the position and / or movement of the arm and / or instrument. Once the path taken has been identified, the auxiliary module 304 can measure the smoothness of the path based on whether the path has a smooth curve or a jagged edge. As described above, in some cases, the state information may include information that explicitly identifies the task and / or step of the task currently being performed. In other cases, auxiliary module 304 can dynamically detect the current task and / or the current step of the task based on other state data. In either case, auxiliary module 304 can be configured to determine the time spent to perform the task or the step of the task by measuring the time between the tasks or the steps of the task.

[0111] In some cases, the auxiliary module 304 may include a list of skills for which assistance can be provided, and the auxiliary module 304 can be configured to detect that the surgical robot system is in a skill assistance state when the auxiliary module detects from the state data that one of the skills in the skill list is about to be performed by the surgical robot system or the surgical robot system has begun to perform one of the skills in the skill list. For example, the list of skills for which assistance can be provided can include, but is not limited to, performing suturing or performing knotting. In these cases, the auxiliary module 304 can be configured to detect that the operator (e.g., a surgeon) is about to perform suturing using the surgical robot system if a suturing instrument has been attached to one of the robot arms. It is obvious to those skilled in the art that these are merely examples of skills for which assistance can be provided, and there may be other skills for which assistance can be provided.

[0112] In some cases, the auxiliary module 304 may include a list of tasks or task steps for which program assistance can be provided, and the auxiliary module 304 may be configured to detect that the surgical robot system is in a program assistance state when the auxiliary module 304 detects from the state data that one of the tasks or task steps for which program assistance can be provided is about to be performed or the operator (e.g., a surgeon) has begun to perform one of the tasks or task steps on the list. As described above, in some cases, the state data may include information that explicitly identifies the task or step currently being performed. In other cases, the auxiliary module 304 is able to dynamically determine the task or task step currently being performed from other state data.

[0113] If the auxiliary module 304 determines that the surgical robot system is not in the auxiliary state, the method returns to box 502, where the auxiliary module 304 receives the next set of state data and determines from the next set of state data whether the surgical robot system is in the auxiliary state. However, if the auxiliary module 304 determines that the surgical robot system is in the auxiliary state, the method 500 proceeds to box 506.

[0114] At block 506, in response to determining that the surgical robot system is in the assist state, the assist module 304 is configured to cause the surgical robot system to provide assistance in performing a task to one or more users. The assist module 304 may cause the surgical robot system to provide assistance in performing a task to one or more users by (i) controlling all or part of the movement of one or more robot arms and / or (ii) causing the surgical robot system to output assist information that can assist the one or more users in performing a task to the one or more users.

[0115] The auxiliary module 304 can be configured to control the movement of one or more robotic arms by transmitting control signals to, for example, the control unit 420, which causes the control unit 420 to control the operation of the motor 416 of one or more arms to cause the one or more arms 402 to operate in a particular manner. In some cases, the auxiliary module 304 can be configured to control the movement of one or more robotic arms to (i) inhibit or limit the movement of the robotic arms; or (ii) cause the robotic arms to move in a predetermined manner. Inhibiting or limiting the movement of the robotic arms may include preventing the robotic arms from moving in one or more degrees of freedom and / or directions, and / or constraining the movement of the robotic arms in one or more degrees of freedom and / or directions (e.g., limiting the movement of the robotic arms to movement in a predetermined manifold). The auxiliary module 304 can be configured to inhibit or limit the movement of one or more robotic arms to, for example, prevent a hazard from occurring or to limit the use of an instrument or robotic arm under unsafe conditions. Causing a robotic arm to move in a predetermined substance may include, for example, causing the robotic arm to follow a predetermined path. The assist module 304 can be configured to cause one or more robotic arms to move in a predetermined manner to, for example, (i) perform motion smoothing; (ii) perform automated surgical actions (e.g., perform knot tying); (iii) perform automated non-surgical actions (e.g., perform instrument replacement).

[0116] The auxiliary module 304 can be configured to cause the surgical robot system to output auxiliary information by transmitting one or more control signals to, for example, the control unit 420, which causes the control unit 420 to output the auxiliary information to one or more users of the surgical robot system 302. The auxiliary information can be output to the user in any suitable form, such as but not limited to audible and / or visual form. For example, in the case where the surgical robot system includes a speaker or other audio output device, the auxiliary module 304 can be configured to send a control signal to the control unit 420, which causes the control unit 420 to audibly output the auxiliary information via the speaker or other audio output device. In the case where the surgical robot system 302 includes one or more devices (such as a display) capable of visually transmitting information to the user, the auxiliary module 304 can be configured to send a control signal to the control unit 420, which causes the control unit 420 to visually output the auxiliary information via the display or other visual output device. In some cases, the surgical robot system 302 may include a light on the robot arm, which can be used to transmit information to the user of the surgical robot system. In these cases, all or part of the auxiliary information can be transmitted or presented to the user via the light on the robot arm.

[0117] Assistive information may include any information that may assist a user in performing a task. Examples of assistive information include, but are not limited to: a list of steps to be performed to complete a task or task step; instructions for proposed instrument changes; proposed port settings; and an alarm that a potentially dangerous condition has been detected.

[0118] In some cases, before causing the surgical robotic system 302 to provide assistance to one or more users of the surgical robotic system in performing a task, the assist module 304 may be configured to request input from the user as to whether the user wishes to provide assistance. In these cases, the assist module 302 may be configured to cause the surgical robotic system 302 to provide assistance to the user of the surgical robotic system only in response to receiving input indicating that assistance will be provided. Whether the surgical robotic system 302 requests user input before providing assistance may depend on the type of assistance being provided and / or the assistance state detected by the assist module 304 in block 504. For example, the assist module 302 may be configured to request user input before controlling all or a portion of the movement of the robotic arm, but not request user input before providing assistance information.

[0119] The type of assistance provided may be based on the assistance state detected by the assistance module 304 in block 504. For example, for certain types of assistance states, the assistance module 304 may only be able to provide one type of assistance (e.g., outputting assistance information), while for other types of assistance states, the assistance module may be able to provide multiple types of assistance (e.g., outputting assistance information and controlling movement of one or more robotic arms).

[0120] When the auxiliary module 304 has determined (at block 504) that the surgical robotic system 302 is in a collision state, the auxiliary module 304 can be configured to cause the surgical robotic system to output auxiliary information notifying one or more users of the surgical robotic system that a collision (or predicted collision) has been detected, and optionally, to provide information about the collision (or predicted collision), such as, but not limited to, the type of collision (e.g., whether the collision is between arms, between instruments, or between an arm and an instrument) and the elements involved in the collision (e.g., the robotic arm and / or the instrument). For example, in the case where the robotic arm includes a controllable light (e.g., a set of controllable light emitting diodes (LEDs)), the auxiliary module 302 can be configured to cause the light of the associated robotic arm to turn a predetermined color (e.g., red) or emit light in a predetermined pattern to indicate the robotic arm involved in the collision.

[0121] Alternatively or in addition, the auxiliary module 304 may be configured to inhibit movement of the relevant robotic arms (the robotic arms involved in the collision or predicted collision) to prevent the components that predict the collision or prevent the collision from being damaged by further moving toward each other. For example, the auxiliary module 304 may be configured to prevent the relevant arms (the arms involved in the collision or predicted collision) from moving at all until they are manually moved apart. In other examples, the auxiliary module may be configured to prevent the relevant robotic arms from moving toward each other, but allow the relevant robotic arms to move in other directions (e.g., away from each other). In some cases, once the auxiliary module 304 has prevented the relevant arms from moving (completely or toward each other) for a predetermined period of time, the auxiliary module 304 may be configured to automatically move the relevant arms apart (e.g., in the direction opposite to the collision). In some cases, the auxiliary module 304 may first request input from a user of the surgical robot system before moving the relevant arms apart, and only move the relevant arms apart after the user confirms that the relevant arms are to be moved apart.

[0122] When the auxiliary module 304 has determined (at box 504) that the surgical robot system 302 is in a faulty instrument state or an incorrect instrument state, the auxiliary module 304 can be configured to cause the surgical robot system to output auxiliary information notifying one or more users of the surgical robot system that a faulty instrument and / or an incorrect instrument has been detected. The auxiliary module 302 can also be configured to cause the surgical robot system to: output information about a faulty or incorrect instrument, such as, but not limited to, which instrument is faulty and / or the arm to which the faulty instrument is currently attached; and / or output information that suggests instrument replacement to one or more users of the surgical robot system. For example, in the case where the robotic arm includes a controllable light, the auxiliary module can be configured to indicate a robotic arm to which an instrument is attached that is faulty or an instrument is incorrect by causing the light of the relevant arm to become a specific color (e.g., orange) or to emit light in a specific pattern.

[0123] Alternatively or in addition, the auxiliary module 304 can be configured to automatically perform instrument replacement (i.e., automatically replace a faulty or incorrect instrument). When the auxiliary module 304 has determined (at block 504) that the surgical robotic system is in a faulty instrument state 304, the automatic instrument replacement replaces the faulty instrument with an instrument of the same type. In contrast, when the auxiliary module 304 has determined (at block 504) that the surgical robotic system is in an incorrect instrument state, the automatic instrument replacement replaces the incorrect instrument with an instrument of a type suitable for performing the current task or task step.

[0124] To be able to perform automatic instrument changes, the surgical robotic system may include: an instrument rack in which instruments that can be attached to the robotic arms of the surgical robotic system are located; and a loading robotic arm that is capable of (i) removing or disassembling instruments from a selected robotic arm; (ii) retrieving a particular instrument from the instrument rack; and (iii) attaching a particular instrument to the selected robotic arm. In some cases, the auxiliary module 304 may first request input from a user of the surgical robotic system (via, for example, the command interface 426) as to whether to perform automatic instrument changes, and the auxiliary module 304 causes the automatic instrument changes to be performed only when the user confirms that the automatic instrument changes will be performed. In the event that the auxiliary module 304 has determined (at box 504) that the surgical robotic system is in an incorrect instrument state and there are multiple suitable instrument types, the auxiliary module 304 may be configured to request input from the user of the surgical robotic system as to which of the multiple suitable instruments to use.

[0125] When the auxiliary module 304 has determined (at box 504) that the surgical robot system 302 is in a dangerous state of an energized instrument, the auxiliary module 304 can be configured to cause the surgical robot system to prevent the relevant energized instrument from being energized. The auxiliary module 304 can be configured to cause the surgical robot system to prevent the relevant energized instrument from being energized by sending a control signal to the control unit 420, and the control signal causes the control unit 420 to prevent the energized current from being sent to the relevant energized instrument. The auxiliary module 304 can also be configured to cause the surgical robot system 302 to output information to notify one or more users of the surgical robot system that a dangerous state of an energized instrument has been detected and / or that the relevant energized instrument cannot be energized. In some cases, the auxiliary module 304 can also be configured to inquire about whether the user input of the power-on prevention should be overwritten (e.g., via a command interface). If the user provides an input that the power-on prevention should be overwritten, the auxiliary module 304 can be configured to cause the surgical robot system to allow the relevant energized instrument to be energized.

[0126] When the auxiliary module 304 has determined (at box 504) that the surgical robot system 302 is in a potentially dangerous state, the auxiliary module 304 can be configured to cause the surgical robot system 302 to output information notifying one or more users of the surgical robot system that a potentially dangerous state has been detected. The auxiliary module 302 can also be configured to cause the surgical robot system to output information about which danger indicators are detected (e.g., whether the patient's or surgeon's vital signs or other health metrics indicate that there is a problem, or whether the volume of voices or actual words spoken by individuals or teams in the room indicate that there may be a problem). The auxiliary module 304 can also be configured to request input from the user of the surgical robot system (via, for example, a command interface) as to whether the task should be paused. If the user provides input indicating that the task should be paused, the auxiliary module 304 can be configured to suppress the movement of one or more of the robotic arms.

[0127] When the auxiliary module 304 has determined (at block 504) that the surgical robot system is in a skill assistance state, the auxiliary module 304 can be configured to cause the surgical robot system 302 to output information notifying one or more users of the surgical robot system that a skill assistance state has been detected. The auxiliary module 304 can also or alternatively be configured to control one or more robotic arms of the surgical robot system 302 to perform all or part of a skill and / or inhibit movement of one or more surgical robotic arms in order to assist an operator (e.g., a surgeon) in performing a skill. For example, as described above, the auxiliary module 304 can be configured with an ideal or preferred path for the robotic arm / instrument attached thereto to take to perform knotting, and when the auxiliary module 304 has determined that the surgical robot system is to be used to perform knotting, the robotic arm can be controlled to follow the ideal or preferred path to perform knotting. In some cases, the auxiliary module 304 can be configured to request input from the user (via, for example, a command interface) as to whether a skill should be automatically performed, and the auxiliary module 304 controls the robotic arm to automatically perform the skill only when the user indicates that the skill should be automatically performed.

[0128] In some cases, alternatively to controlling a robotic arm to perform all or a portion of a skill, the assist module 304 may be configured to inhibit movement of one or more surgical robotic arms in order to assist an operator (e.g., a surgeon) in performing the skill. For example, as described above, the assist module may be configured with an ideal path or preferred path for a robotic arm / instrument attached thereto to take to perform knot tying, and the assist module 304 may restrict movement of the associated robotic arm to within a predetermined distance of the ideal path. This would allow the surgeon to control movement of the associated robotic arm / instrument within a predetermined distance of the ideal path and control the rate at which the associated robotic arm / instrument follows the path.

[0129] In some cases, good knotting paths may have been learned from state data of previously performed tasks involving performing knot tying. For example, auxiliary module 304 or another computing device may receive state data related to multiple tasks involving performing knot tying, where quality assessments of knot tying are provided (e.g., knots may have been scored according to a standard), and identify common paths or paths that link best knotting paths. A common or linking best knotting paths may be a path that has a maximum mean square distance to a path of knots at or above a particular level.

[0130] When the assist module 304 has determined (at block 504) that the surgical robot system is in a program assist state, the assist module may be configured to cause the surgical robot system to output information that provides guidance on how to perform a task or steps of a task. The information that provides guidance on how to perform a task may include one or more of the following: information indicating one or more steps to be performed next; information indicating which instrument should be selected next; information indicating which manual controller should be associated with each active arm; information indicating a path that the active robotic arm should follow in order to perform the task in an efficient manner (e.g., a recommended path for the active robotic arm / instrument to follow may be displayed on a display screen).

[0131] In some cases, guidance on how to perform a task or task step is determined by state data related to a previously performed task. For example, the auxiliary module 304 or based on another computing device can be configured to identify preferred steps of a procedure, preferred instruments, preferred manual controllers, and / or preferred paths taken by instruments by comparing the state data of previously performed tasks to identify patterns therein. For example, in the case where the state data includes result information indicating the result of a task (e.g., the result of a patient) or is augmented with result information, the auxiliary module 304 or based on another computing device can be configured to identify the steps of a particular task, instruments, manual controllers, and / or paths taken by instruments that produce the best results from the state data of previously performed tasks. The analyzed state data may include state data for similar tasks (e.g., for the same or similar surgical procedures); state data for tasks performed by similar operators (e.g., surgeons); and / or state data for patients similar to the current patent. It is obvious to those skilled in the art that these are examples of guidance information and examples of how guidance information can be obtained, and other guidance information can be provided, and guidance information can be generated in another way. For example, in other cases, guidance information can be preconfigured or predetermined.

[0132] In some cases, the level of assistance provided to the user can be based on the skill level of the operator (e.g., surgeon) controlling the surgical robotic system. For example, more assistance (e.g., more auxiliary information) can be provided to operators (e.g., surgeons) with lower skill levels, and less assistance (e.g., less auxiliary information) can be provided to operators (e.g., surgeons) with higher skill levels. For example, detailed instructions on how to perform specific steps of a skill or task can be provided to operators (e.g., surgeons) with lower skill levels, while only a few hints or prompts can be provided to operators (e.g., surgeons) with higher skill levels. In some cases, the skill level of the operator (e.g., surgeon) can be explicitly provided to the auxiliary module 304 as part of the state data or in another way (e.g., as part of the task setting). In other cases, the skill level of the operator (e.g., surgeon) controlling the surgical robotic system can be dynamically determined from the state data of the current task (e.g., surgical operation) or from the state data of a previous task performed by the operator (e.g., surgeon).

[0133] As described above, the performance of an operator (e.g., a surgeon) controlling a surgical robotic system can be evaluated or determined by comparing one or more performance metrics of the current operator (e.g., a surgeon) with performance metrics of other operators (e.g., surgeons) of previously performed tasks. In some cases, the performance metrics may include one or more of the following: the path taken by the robot arm or instrument when performing a task or task step; the smoothness of the movement of the robot arm and / or instrument when performing a task or task step; and / or the time taken to perform a task or task step. In the case where the state data includes information describing the position and / or movement of the arm and / or instrument of the surgical robotic system 302, the auxiliary module 304 can be configured to determine the path of the robot arm and / or instrument from the information describing the position and / or movement of the arm and / or instrument. Once the path taken has been identified, the auxiliary module 304 can measure the smoothness of the path based on whether the path has a smooth curve or a jagged edge. The state data may include information that explicitly identifies the current task or the current step of the task, or the auxiliary module 304 may dynamically detect the current task and / or the current step of the task based on other state data. In either case, the assistance module 304 can be configured to determine the time taken to perform a task or steps of a task by measuring the time between tasks or between steps of a task.

[0134] In some cases, in addition to providing assistance to one or more users of the surgical robot system in response to detecting that the surgical robot system is in an assist state, the assist module 304 can also be configured to provide other feedback information to the user of the surgical robot system during a task (e.g., surgery) or after a task (e.g., surgery). For example, the assist module 304 can be configured to provide feedback about the operator's performance during a task, indicating how the operator (e.g., surgeon) performed relative to his previous performance of the same task and / or relative to other operators (e.g., surgeons) who have performed the same task (e.g., surgical procedure). For example, the assist module 304 can be configured to measure one or more operator performance metrics, such as, but not limited to, smoothness, the path taken, and the time it takes to perform a task or steps of a task. The measured performance metrics can then be compared with the operator's previous performance of the task or other operators' previous performance of the task to provide feedback about the operator's performance. For example, the assist module 304 can be configured to provide feedback to the operator, such as, but not limited to: "You are faster than usual" or "You are slower than your colleagues."

[0135] In another example, the auxiliary module 304 may also or alternatively be configured to measure metrics related to other users of the surgical robotic system (e.g., other members of the surgical team) and generate feedback information related thereto. For example, the auxiliary module 304 may be configured to measure how many instruments are used during a task, the number of instrument changes, the time it takes to change each instrument, the time it takes to prepare (drape) the surgical robotic system for a task (e.g., surgery). The auxiliary module 304 may be configured to cause the surgical robotic system to output the measured metrics, or may provide the measured metrics to another party.

[0136] In yet another example, the auxiliary module 304 may also or alternatively be configured to measure metrics about the surgical robotic system and generate feedback information related thereto. For example, the auxiliary module 304 may be configured to detect from the status data when an instrument replacement occurs, and if the auxiliary module 304 has detected that a first instrument attached to the arm is replaced by a second instrument of the same type within a predetermined time period of the first instrument attached to the arm, the auxiliary module 304 may be configured to automatically detect a first instrument failure. This information may be recorded and reported to another party so that, for example, such a new instrument may be ordered.

[0137] In another example, the auxiliary module 304 or based on another computing device can also or alternatively be configured to analyze the state data to generate feedback information for use in a process or program performed before or after a task or similar task. Specifically, the auxiliary module 304 can be configured to analyze the state data to identify information that can be used to schedule future tasks or other programs. For example, the auxiliary module 304 can be configured to predict when the task will be completed from the state data, and notify another party (e.g., an operating room manager) if the predicted completion time is lower than a predetermined threshold (e.g., 1 hour) so that the next task in the operating room can be planned, and the next patient can be prepared, and / or the operator of the surgical robot system (e.g., a surgeon) can be scheduled to perform another task. In another example, the auxiliary module 304 can be configured to compare the state data (including the results) with the state data of a similar program performed by the same operator (e.g., a surgeon) after the task is completed to determine the best time for the operator to perform the specific task (e.g., time of day and / or day of the week), and output feedback information indicating the best time for the operator to perform the specific task.

[0138] It will be apparent to those skilled in the art that these are examples of additional feedback information that may be generated and provided by auxiliary module 304, and that any information that may be obtained from the state data and / or comparison of the state data to state data of a previously performed task (e.g., a surgery) may be used to generate additional feedback information.

[0139] The applicant hereby independently discloses each individual feature described herein and any combination of two or more such features, as long as these features or combinations can be implemented according to the common knowledge of those skilled in the art based on this specification as a whole, regardless of whether these features or combinations of features solve any problems disclosed herein. In view of the above description, it is obvious to those skilled in the art that various modifications can be made within the scope of the present invention.

Claims

1. A system (300) for providing assistance to a user of a surgical robot system, the system (300) comprising: A surgical robot system (302, 400), the surgical robot system comprising at least one surgical robot having a base and an arm extending from the base to an attachment of an instrument, the arm comprising a plurality of joints whereby the configuration of the arm can be changed; as well as An auxiliary module (304), the auxiliary module being configured to: While a task is being performed using the surgical robotic system (302, 400), receiving status data, the status data comprising a video and / or audio recording of a room in which the task is being performed; determining that the surgical robotic system is in a potentially hazardous state when one or more hazard indicators are detected from the state data, the one or more hazard indicators comprising speaking a warning word or phrase for one or more individuals present at the task, wherein the warning word or phrase is learned from the state data of a previously performed task; and In response to determining that the surgical robot system (302, 400) is in a potentially dangerous state, the surgical robot system provides potentially dangerous assistance to the user when the surgical robot system (302, 400) is being used to perform the task, Wherein, the auxiliary module is configured to identify any words or phrases spoken within a predetermined time period prior to a negative output from the status data of each of a plurality of previously performed tasks, and to make any identified words or phrases common between a predetermined number of previously performed tasks into warning words or phrases.

2. The system (300) according to claim 1, wherein: Causing the surgical robotic system to provide potentially hazardous assistance includes outputting information regarding one or more detected hazardous indicators.

3. The system (300) according to claim 2, wherein: When the one or more detected danger indicators include a warning word or phrase spoken by one or more individuals present for the task, outputting information regarding the one or more detected danger indicators includes identifying the warning word or phrase.

4. The system (300) according to any one of claims 1 to 3, wherein: Causing the surgical robotic system to provide potentially dangerous assistance includes requesting input from a user of the surgical robotic system as to whether the task should be paused.

5. The system (300) according to claim 4, wherein: Causing the surgical robotic system to provide potentially dangerous assistance further includes causing movement of one or more of the at least one arm to be inhibited in response to a user of the surgical robotic system providing input that the task should be paused.

6. The system (300) according to any one of claims 1 to 3, wherein: The status data includes data indicative of a status of one or more health metrics of an operator of the surgical robotic system.

7. The system (300) according to claim 6, wherein: The one or more risk indicators include one or more health metrics of the operator of the surgical robotic system that fall outside of a predetermined range.

8. The system (300) according to claim 6, wherein: Each of the one or more health metrics of the operator of the surgical robotic system is associated with a predetermined range, and each predetermined range has been learned from state data of previously performed tasks.

9. The system (300) according to any one of claims 1 to 3, wherein: The one or more hazard indicators include events or patterns in the state data of previously executed tasks prior to the hazard.

10. The system (300) according to any one of claims 1 to 3, wherein: The auxiliary module is configured to determine that the surgical robot system is in a potentially dangerous state when the auxiliary module determines, based on the state data, that the performance of an operator of the surgical robot system has fallen below a predetermined threshold.

11. The system (300) according to claim 10, wherein: The assist module is configured to determine that the performance of the operator of the surgical robotic system has fallen below a predetermined threshold by comparing the status data with status data of a previously performed task.

12. The system (300) according to claim 10, wherein: The auxiliary module is configured to determine, based on the status data, that the performance of the operator of the surgical robotic system has fallen below a predetermined threshold by determining one or more performance metrics of the operator of the surgical robotic system based on the status data, the one or more performance metrics including one or more of: a path taken by a robotic arm or instrument when performing the task or a step of the task, the smoothness of the movement of the robotic arm when performing the task or a step of the task, and the time taken to perform the task or a step of the task.

Images