Robotic surgical systems and methods for implementing customized user training procedures

Abstract

A robotic surgical system and method for implementing a customized user training program is provided. In one embodiment, the robotic surgical system determines whether a user's skill in moving a robotic arm with a user input device needs improvement. In response to determining that the user's skill in moving the robotic arm with the user input device needs improvement, the robotic surgical system provides a customized simulated training program to improve the skill. Other embodiments are provided.

Description

[0001] Related applications

[0002] This patent document claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 595,054, filed November 1, 2023, pursuant to 35 USC §119(e), which is hereby incorporated by reference. Technical Field

[0003] The following implementation plan relates to the field of robotic surgery as a whole, and more specifically to a customized training program for users of robotic surgical systems. Background Technology

[0004] Minimally invasive surgery (MIS), such as laparoscopic surgery, involves techniques designed to minimize tissue damage during surgical procedures. For example, laparoscopic surgery typically involves making multiple small incisions inside the patient (e.g., in the abdomen) and introducing one or more surgical instruments (e.g., end effectors, at least one camera, etc.) through these incisions. Surgical procedures can then be performed using the introduced surgical instruments, with visualization aids provided by the camera.

[0005] Generally, medical interventions (MIS) offer multiple beneficial effects, such as reducing patient scarring, alleviating patient pain, shortening patient recovery time, and reducing medical costs associated with patient recovery. In some implementations, MIS can be performed using a robotic system comprising one or more robotic arms for manipulating surgical instruments based on commands from an operator. The robotic arms may, for example, support various devices at their distal ends, such as surgical end effectors, imaging devices, cannulas for providing access to the patient's body cavities and organs.

[0006] In some implementations, the operator can provide commands for manipulating surgical instruments while viewing images provided by a camera and displayed to the user on a monitor. Attached Figure Description

[0007] Figure 1A An example of an operating room layout with a robotic surgical system and a user console is depicted in the implementation scheme.

[0008] Figure 1B This is a schematic illustration of an exemplary variant of the robotic arm manipulator, tool actuator, and cannula with surgical tools in the implementation scheme.

[0009] Figure 1C This is a schematic illustration of an exemplary user console for the implementation scheme.

[0010] Figure 2This is a schematic illustration of an exemplary variant of a user console for an implementation of a robotic surgical system used to communicate with one or more third-party devices.

[0011] Figure 3 This is a schematic diagram of a surgical robot platform with a graphical user interface (GUI) module, wherein the surgical robot platform communicates with multiple medical data resources.

[0012] Figure 4A and Figure 4B These are, respectively, a perspective view and a longitudinal cross-sectional view of an exemplary variant of the handheld user input device of the implementation scheme.

[0013] Figure 5 This is a flowchart of an implementation method for carrying out customized user training programs on robotic surgical systems.

[0014] Figure 6 It is a diagram of a graphical user interface for a customized user training program that provides an implementation plan.

[0015] Figure 7 It is an illustration of a graphical user interface that provides scores for customized user training programs based on the implementation plan.

[0016] Figure 8 This is a flowchart of a method for implementing a customized user training program.

[0017] Figure 9 This is a flowchart of an implementation method for carrying out customized user training programs for surgeons.

[0018] Figure 10 This is a flowchart of an implementation method for carrying out customized user training programs for resident physicians.

[0019] Figure 11 This is a flowchart of a method for implementing a customized user training program based on collected endoscopic positioning statistics. Detailed Implementation

[0020] Non-limiting examples of various aspects and variations of the implementation scheme are described herein and illustrated in the accompanying drawings.

[0021] Overview of Robotic Surgical Systems

[0022] Figure 1A This is an illustration of an exemplary operating room environment with a robotic surgical system. Typically, as... Figure 1AAs shown, the robotic surgical system includes a user console 100 (sometimes referred to herein as a “surgeon’s bridge” or “bridge”), a control tower 133, and one or more robotic arms 160 located on a robotic platform (e.g., a table, bed, etc.), wherein surgical instruments (e.g., having end effectors) are attached to the distal end of the robotic arm 160 to perform surgical procedures. The robotic arm 160 is shown as a tabletop-mounted system, but in other configurations, one or more robotic arms may be mounted on a trolley, ceiling, or sidewall, or other suitable support surface.

[0023] As further illustration, such as Figure 1B As illustrated in the exemplary schematic, a robotic surgical system may include at least one robotic arm 160 and a tool actuator 170 typically attached to the distal end of the robotic arm 160. A cannula 180 coupled to the end of the tool actuator 170 may receive and guide surgical instruments 190 (e.g., end effectors, cameras, etc.). Furthermore, the robotic arm 160 may include a plurality of actuated links to position and orient the tool actuator 170, which actuates the surgical instruments 190.

[0024] Usually, such as Figure 1A As shown, the user console 100 can be used to interface with the robotic surgical system 150. A user (such as a surgeon or other operator) can use the user console 100 to remotely manipulate the robotic arm 160 and / or surgical instruments (e.g., in remote operation). The user console 100 may be located in the same operating room as the robotic system 150, such as... Figure 1A As shown. In other embodiments, the user console 100 may be located in an adjacent or nearby room, or be remotely operated from a remote location in a different building, city, or country. In one example, the user console 100 may include a seat 110, foot pedal controls (pedals) 120, one or more handheld user input devices 122, and at least one user display 130 configured to display, for example, a view of a surgical site inside a patient's body (e.g., captured by an endoscopic camera), and / or other surgical or medical information.

[0025] exist Figure 1CIn the exemplary user console shown, a user seated in seat 110 and viewing user display 130 can manipulate foot pedal controls 120 and / or handheld user input device 122 to remotely control robotic arm 160 and / or surgical instruments mounted to the distal end of that arm. Foot pedal controls 120 and / or handheld user input device 122 can also be used, alternatively, to control other aspects of user console 100 or robotic system 150. For example, in a variation where the user typically controls (at any given time) a designated “left-hand” robotic arm / instrument and a designated “right-hand” robotic arm / instrument, foot pedal controls 120 allow the user to specify which robotic arms / instruments include both “left-hand” and “right-hand” robotic arms / instruments from a larger set of available robotic arms / instruments (e.g., by switching or rotating when selecting among available robotic arms / instruments). Other examples include adjusting or configuring seat 110, foot pedal controls 120, user input device 122, and / or user display 130.

[0026] In some variants, a user can operate the surgical robotic system in an "OTB" (over-the-bed) mode, where the user is positioned to one side of the patient and simultaneously manipulates both robot-driven instruments / end-effectors attached to the patient (e.g., a handheld user input device 122 held in one hand) and manual laparoscopic tools. For example, the user's left hand can manipulate the handheld user input device 122 to control the robotic surgical components, while the user's right hand can manipulate the manual laparoscopic tools. Thus, in these variants, the user can perform both robot-assisted MIS and manual laparoscopic surgery on the patient.

[0027] During the exemplary procedure or surgery, the patient is prepared and covered aseptically, and anesthesia is administered. Initial approach to the surgical site can be manually performed using the robotic system 150 in a retracted or collapsed configuration to facilitate access. Once approach is complete, initial positioning and / or preparation of the robotic system can be performed. During the surgical procedure, the surgeon or other user at the user console 100 can manipulate various end effectors and / or imaging systems to perform the procedure using foot pedal controls 120, user input devices 122, and / or other suitable controls. Manual assistance can be provided by other personnel at the procedure table, who can perform tasks including, but not limited to, retracting tissue, or performing manual repositioning or tool changes involving one or more robotic arms 160. Other personnel may be present to assist the user at the user console 100. Medical and surgical information assisting other medical personnel (e.g., nurses) can be provided on additional displays, such as a display 134 on the control tower 133 (e.g., a control system for the robotic surgical system) and / or a display 132 located near the patient's bedside. For example, as described further in detail herein, some or all of the information displayed to the user in the user console 100 may also be displayed on at least one additional monitor of other personnel and / or provide additional pathways for inter-personnel communication. Upon completion of a procedure or surgery, the robotic system 150 and / or the user console 100 may be configured or set to facilitate one or more postoperative procedures, including but not limited to robotic system cleaning and / or sterilization, and / or medical record input or printout, whether electronic or hard copy, via the user console 100.

[0028] In some variations, communication between the robotic system 150, the user console 100, and any other displays can be achieved via a control tower 133, which translates user commands from the user console 100 into robot control commands and transmits these commands to the robotic system 150. The control tower 133 can also transmit status and feedback from the robotic system 150 back to the user console 100 (and / or other displays). The connection between the surgical robotic system 150, the user console 100, other displays, and the control tower 133 can be via wired and / or wireless connections, and can be proprietary and / or use any of a variety of data communication protocols. Any wired connection can be integrated into the operating room floor and / or walls or ceiling. The robotic surgical system can provide video output to one or more displays, including displays within the operating room and remote displays accessible via the Internet or other networks. Video output or feeds can be encrypted to ensure privacy, and all or one or more portions of the video output can be stored on a server, an electronic health record system, or other suitable storage medium.

[0029] In some variations, an additional user console 100 may be provided, for example, to control additional surgical instruments and / or to control one or more surgical instruments on the main user console. This would allow, for example, a surgeon to take over or explain a technique to medical students and physicians in training during surgical procedures, or to assist during complex surgeries that require multiple surgeons to act simultaneously or in a coordinated manner.

[0030] In some variations, such as Figure 2 As illustrated in the schematic diagram, one or more third-party devices 240 may be configured to communicate with the user console 210 and / or other suitable parts of the robotic surgical system. For example, as described elsewhere herein, a surgeon or other user may be seated in the user console 210, which may communicate with the robotic instruments in the control tower 230 and / or the robotic system 220. Medical data (e.g., endoscopic images, patient vital signs, tool status, etc.) may be displayed on the user console 210, the control tower 230, and / or other displays. At least a subset of surgical and other medically relevant information may also be displayed on the third-party device 240, such as a remote computer monitor, which may be viewed by surgical collaborators in or outside the same room. Other communications, such as teleconferences using audio and / or visual communications, may be further provided to and from the third-party device. Surgical collaborators may be, for example, supervisors or trainers, medical colleagues (e.g., radiologists), or other third parties who may view and communicate, for example, via the third-party device 240, to assist in surgical procedures.

[0031] Figure 3 This is a schematic illustration of an exemplary variant of system 300, which includes a robotic surgical system and its interaction with other devices and parties. Although Figure 3 The document describes specific architectures for various connection and communication systems, but it should be understood that other suitable architectures can be used in other variations, and Figure 3 The arrangement shown is for illustrative purposes. System 300 may include a surgical robot platform 302 that facilitates the integration of medical data from discrete medical data resources generated from various parties. Data from discrete medical data resources can, for example, be used to form temporally coordinated medical data. As further described herein, a multi-panel display for presenting the temporally coordinated medical data can be configured and presented.

[0032] Platform 302 may be, for example, a machine having one or more processors 310 connected via a bus 314 to one or more input / output devices 312. At least one processor may include, for example, a central processing unit, a graphics processing unit, an application-specific integrated circuit, a field-programmable logic device, or a combination thereof.

[0033] The surgical robot platform 302 may include one or more input ports to receive medical data from discrete medical data resources. For example, surgical robot port 329 may receive surgical robot data from surgical robot 330. Such data may include, for example, position data or other suitable status information. Imaging port 331 may receive imaging data from imaging device 332 (such as an endoscope) configured to capture images (e.g., still images, video images) of the surgical site. The endoscope may be inserted, for example, through a natural orifice or through a hole in the patient's body. As another example, one or more medical instrument ports 333 may receive patient vital signs information from medical instrument 334 (e.g., pulse oximeter, electrocardiogram device, ultrasound device, etc.). Additionally, as another example, one or more user control data ports 335 may receive user interaction data from one or more control devices that receive user input from the user to control the system. For example, one or more handheld user input devices, one or more foot pedals, and / or other suitable devices (e.g., eye tracking, head tracking sensors) may receive user input.

[0034] The surgical robot platform 302 may also include one or more output ports 337 configured to connect to one or more displays 338. For example, displays 338 may include open displays (e.g., monitor screens) in a user console, immersive displays or head-mounted devices with displays, supplemental displays such as those on control tower displays (e.g., team displays), bedside displays (e.g., nurse displays), overhead "stadium" style screens, etc. For example, the graphical user interface disclosed herein may be presented on one or more displays 338. One or more displays 338 may present three-dimensional images. In some variations, one or more displays 338 may include touchscreens. One or more displays 138 may be a single display with multiple panels, each displaying different content. Alternatively, one or more displays 138 may include a collection of individual displays, each individual display presenting at least one panel.

[0035] In some variations, network interface 316 may also be connected to bus 314. For example, network interface 316 may provide connectivity to network 317, which may be any combination of one or more wired and / or wireless networks. For example, network 317 may facilitate communication between the surgical robot platform 302 and other data sources or other devices. For example, one or more third-party data sources 340 may also be connected to network 317. Third-party sources 340 may include third-party devices (e.g., another computer operated by a third party such as another physician or medical specialist), a repository of video surgical procedure data (e.g., data that may be related to a procedure performed by a surgeon), or other suitable additional information sources related to the surgical procedure. For example, third-party device data may be transmitted to a panel displayed to the surgeon before, during, or after the procedure.

[0036] As another example, one or more application databases 342 may be connected to network 317 (or alternatively, locally stored in memory 320 within the surgical robot platform 302). Application database 342 may include software applications (e.g., as described further below), which the surgeon may be interested in during the procedure. For example, the software application may provide access to stored patient medical records, provide a checklist for surgical tasks within the surgical procedure, perform machine vision techniques to assist the procedure, perform machine learning tasks to improve the surgical task, etc. Any suitable number of applications may be invoked. During the procedure, information associated with the applications may be displayed on a multi-panel display or other suitable display. Alternatively or additionally, information provided by one or more applications may be provided through a separate resource (e.g., a machine learning resource) that is also suitably in communication with the surgical robot platform 302.

[0037] In some variations, one or more software applications within a software application can function as separate processes that use an application interface (API) to draw objects and / or images on a display. APIs of varying complexity can be used. For example, a simple API might include several templates with fixed widget sizes and positions, which can be used by a GUI module to customize text and / or images. As another example, a more complex API could allow the software application to create, place, and delete different widgets, such as labels, lists, buttons, and images.

[0038] Alternatively, one or more software applications may render themselves for display. This can, for example, allow for advanced customization and complex behavior of the application. For instance, this method can be implemented by allowing the application to pass frames rendered by a graphical user interface (GUI) module 324, which may be computer-readable program code executed by processor 310. Alternatively, an image buffer may be used as a store for the application to render itself.

[0039] In some variations, one or more software applications can run and render themselves independently of the GUI module 324. However, the GUI module can still launch such applications, instruct applications or the operating system where to position applications on the display, and so on.

[0040] As another approach, in some variations, one or more applications can run completely independently of the GUI rendered by the GUI module. For example, such applications can have physical video and data connections to the system (e.g., via suitable input / output devices, networks, etc.). The data connection can be used to configure the application's video feed to appropriate pixel dimensions (e.g., full-screen, half-screen, etc.).

[0041] like Figure 3 As shown, in some variations, memory 320 may also be connected to bus 314. Memory 320 may be configured to store data processed according to implementations of the methods and systems described herein.

[0042] In some variations, memory 320 may be configured to store other types of data and / or software modules for execution. For example, a user console may include memory 320 storing a GUI module 324 with executable instructions to perform the operations disclosed herein. The GUI module may, for example, combine and aggregate information from various software applications and / or other medical data resources for display. In some exemplary variations, one or more software applications may be incorporated into the base code of the GUI module, causing the module to draw graphics and display text at appropriate locations on the display. For example, the module may retrieve images from a database or push images from instruments (e.g., endoscopic cameras) in the operating room to the interface via a wired or wireless interface.

[0043] In some variations, medical data can be collected from discrete medical data resources, such as surgical robots 330, endoscopes 332, medical instruments 334, control devices 336, third-party data sources 340, application databases 342, etc. Furthermore, at least some of the medical data can be temporally coordinated, such that time-sensitive information from different medical data resources is aligned on a common timeline when necessary. For example, surgical robot position data can be temporally coordinated with endoscope data, which in turn is coordinated with operator interaction data from the control device. Similarly, network resources, such as information provided by one or more software applications, can be presented at appropriate points in time along with other temporally coordinated data. Multi-panel displays and / or other suitable displays can be configured to convey medical information (e.g., including temporally coordinated medical data) as part of a graphical user interface (GUI).

[0044] This document describes various exemplary aspects of the GUI (User Interface) for robotic surgical systems. In some variations, the GUI may be displayed on a multi-panel display at the user console controlling the robotic surgical system. Alternatively, the GUI may be displayed at one or more additional displays, such as at the control tower of the robotic surgical system, at the patient's bedside, etc. Typically, the GUI can provide more efficient information communication to the user and / or other personnel at the user console, as well as more efficient communication and collaboration between different parties involved in surgical procedures, as further described below.

[0045] Graphical User Interface (GUI) Interaction

[0046] In one implementation, the GUI is displayed on a monitor 130 in a user console 100, which is used (e.g., by a surgeon) to control the robotic surgical system 150. At least some of the interactive graphical objects displayed on the monitor 130 can be controlled, selected, or otherwise interacted with via one or more user controls that are also used to control aspects of the surgical system (e.g., surgical instruments). For example, a user can selectively control aspects of the robotic surgical system 150 and selectively interact with the GUI using one or more handheld user input devices 122 and / or one or more foot pedals 120. By enabling the use of the same user controls to control both the robotic surgical system 150 and the GUI, the user can advantageously avoid having to switch between two different types of user controls. Enabling the user to use the same input devices to control both the robotic system 150 and the GUI streamlines surgical procedures and improves efficiency, as well as helping the user maintain aseptic technique throughout the surgical procedure.

[0047] like Figure 4A and Figure 4BAs generally shown, an exemplary variant of the handheld user input device 122 for controlling a robotic system may include a component 410, a housing 420 at least partially disposed around the component 410 and configured to be held in a user's hand, and a tracking sensor system 440 configured to detect the position and / or orientation of at least a portion of the device. The housing 420 may be flexible (e.g., made of silicone). In some instances, the detected position and / or orientation of the device may be associated with controls of the robotic system. For example, the user input device 122 may control at least a portion of a robotic arm, an end effector or tool (e.g., a gripper or clamp) coupled to the distal end of the robotic arm, a GUI, or other suitable aspects or features of the robotic surgical system 150. Additionally, in some instances, the detected position and / or orientation of the device 122 may be associated with controls of the GUI. Furthermore, in some variants, the user input device 122 may include one or more sensors for detecting other manipulations of the user input device 122, such as squeezing the housing 420 (e.g., via one or more pressure sensors, one or more capacitive sensors, etc.).

[0048] Typically, the user interface for controlling a robotic surgical system may include at least one handheld user input device 122, or may include at least two handheld user input devices 122 (e.g., a first user input device to be held by the user's left hand and a second user input device to be held by the user's right hand), or any suitable number. Each user input device 122 may be configured to control one or more different aspects or features of the robotic system. For example, a user input device held in the user's left hand may be configured to control an end effector represented on the left side of a camera view provided to the user, while a user input device held in the user's right hand may be configured to control an end effector represented on the right side of a camera view.

[0049] In some variations, the handheld user input device 122 can be an ungrounded user input device configured to be held in the hand and manipulated in free space. For example, the user input device 122 can be configured to be held between the user's fingers and moved freely by the user (e.g., translation, rotation, tilting, etc.) as the user moves his or her arm, hand, and / or fingers. Alternatively, the handheld user input device 122 can be a body-grounded user input device, since the user input device 122 can be attached directly or via any suitable mechanism (such as a glove, hand strap, sleeve, etc.) to a part of the user (e.g., the user's fingers, hand, and / or arm). Such a body-grounded user input device still allows the user to manipulate the user input device in free space. Thus, in variations where the user input device 122 is ungrounded or body-grounded (as opposed to permanent mounting or grounding to a fixed console, etc.), the user input device 122 can be ergonomic and provide dexterous control, such as by allowing the user to control the user input device with natural body movements unimpeded by the fixed characteristics of a grounding system.

[0050] The handheld user input device 122 may include a wired connection, for example, to provide power to the user input device 122 and carry sensor signals (e.g., from a tracking sensor assembly and / or other sensors such as capacitive sensors, optical sensors, etc.). Alternatively, the user input device may be wireless, such as... Figure 4A As shown, commands and other signals are transmitted via radio frequency signals (e.g., WiFi or short-range signals such as 400mm-500mm range) or other suitable wireless communication protocols (such as Bluetooth). Further wireless connectivity can be facilitated using optical reader sensors and / or cameras (configured to detect optical marks on user input device 122), infrared sensors, ultrasonic sensors, or other suitable sensors.

[0051] Handheld user input devices may include a clutch mechanism for switching between controlling a robotic arm or end effector and controlling a graphical user interface, and / or switching between other control modes. One or more of the various user inputs described further below can be used as a clutch in any suitable combination. For example, a gesture touch area of ​​a touch device, squeezing the housing, flicking or rotating the user input device can act as an engaging clutch. As another example, a combination of squeezing and holding the user input device with rotating the user input device can be used as a clutch. However, any suitable combination of gestures can be used as a clutch. Additionally or alternatively, user input to other user input devices (e.g., foot pedal assemblies) can be used as a clutch, either alone or in combination with user input to the handheld user input device.

[0052] In some variations, the engagement and disengagement of the clutch mechanism allows for switching between using a handheld user input device as a controller for the robotic system and using it as a controller for the GUI (e.g., operating a cursor displayed on a screen). When the clutch mechanism is engaged so that the user input device is used to control the GUI, the position or orientation of the robot arm can be essentially locked in place to "pause" the operation of the robotic system, ensuring that subsequent movement of the user input device when the clutch is engaged will not unintentionally cause movement of the robot arm.

[0053] Simulator application

[0054] Another variation of GUI applications is the simulator application. Simulator applications may, for example, communicate with a database storing simulated surgical robot experiences or training, such as to teach user-specific psychomotor skills for robot simulations (e.g., games for practicing scrolling movements and / or other skills on a handheld user input device). Simulated surgical robot experiences may include, for example, simulations and training with simulated patients. Simulator applications can load such simulation experiences into the GUI, including simulated endoscopic views and other patient parameters. Simulation experiences may also include simulated events, such as robot arm collisions, patient emergencies, and other appropriate events that can help new users (e.g., surgeons in training) learn how to respond appropriately and resolve problems. Simulations may be generated independently of the simulator application (e.g., using simulation-based software development), or alternatively, within the simulator application itself.

[0055] In some variations, simulator applications can rate users based on their performance in simulated training, such as by providing scores. These scores can be tracked over time to determine the trainee's progress and proficiency in using the robotic surgical system. In other variations, simulator applications can display the user's progress throughout a set course (e.g., indicating that the user has completed three out of ten training sessions), assess the user's baseline skills to customize or adjust the course, and / or provide recommendations for specific simulated training based on the user's performance.

[0056] Generally speaking, a "simulator" refers to the hardware and / or software components (e.g., a processor that executes computer-readable instruction code) that allow simulations to be performed on the surgical bridge 100 to allow users to practice using the user input device of the surgical bridge 100. In one embodiment, the simulator's hardware / software components are integrated into the user console / surgical bridge 100 of the robotic surgical system 150 (e.g., in the same housing as other components of the console / surgical bridge 100). For example, simulator software may be stored on the surgical bridge 100 and activated for training purposes. Previous simulators were separate components removably attached to the outer surface of the user console, with a data cable plugged into the user console and a power cable plugged into a power outlet. Therefore, setting up this external component before use could take several minutes, and additional time was required to disassemble it after use. Furthermore, the extra cables could cause tripping or other hazards. Compared to these master-slave training stations, with the simulator integrated into the surgical bridge 100, the user simply clicks a button on the GUI to use the simulator application, and the simulator is ready to use (possibly after a short reset time), without the user needing to worry about plugging in data and power cables or waiting for other components to start up. In clinical mode, manipulation of the user input device produces movement of the robotic arm, while in simulation mode, signals from the user input device are sent only to the simulator and not to the robotic arm controller.

[0057] In addition to the integration of the simulator into the robotic surgical system, the simulator in this implementation differs from previous simulators in that it provides customized user training programs. Current skills training for surgeons is neither personalized nor responsive to their needs, nor does it have the ability to incorporate real-world surgical performance data to build customized training programs for individual surgeons. For example, user training sessions typically focus on navigating various skill areas to achieve certification. It is desirable to focus training on the user's real-life performance (e.g., during surgery or other training) to enhance skills the user lacks. In one implementation, the simulator may select training for the user based on performance feedback from past surgical and / or training sessions. As discussed in more detail below, customized training programs may be provided, which may be based on the sequence of robotic training or a customized sequence, such as skills assessment, progress tracking, and specific case studies. Data recorded by the robotic system may include, but is not limited to, the categories, types, and examples of data used for performance evaluation in both remote operation and simulation scenarios.

[0058] Return to the attached image. Figure 5This is a flowchart 500 of a method for implementing a customized user training procedure on a robotic surgical system 150. The method may be executed by one or more processors in the system 150. For example, a single processor may be used to perform all steps, or multiple processors may be used, each performing one or more different steps. The phrase "process" is used herein to refer to either alternative. Furthermore, the one or more processors used to perform these steps may be used to perform other functions in the system 150, or may be specifically used for simulated training.

[0059] like Figure 5 As shown, in this embodiment, the processor in the robotic surgical system 150 maintains a skill level record for the user, which tracks the user's skill level (actions 510) in manipulating the robotic arm 160 using the user input device 122. The skill level record can be any suitable data structure with any suitable format and can be stored in the memory of the robotic surgical system 150 or in external memory. The user-specific skill level record can be maintained by collecting data acquired during previous actual surgeries performed by the user using the robotic surgical system 150 and / or by collecting data acquired from previous simulation training performed by the user.

[0060] Data collected to track a user's skill level can take any suitable form. For example, if the data comes from an actual surgery, it could be positioning data of a robotic arm (e.g., a carrying tool or endoscope), which could include the duration for which the robotic arm is positioned in a given location or the frequency of its adjustments. The data may also include one or more of the following: data on the movement of the robotic arm, user-related physiological data, manually entered data, video review data, and video data. It should be noted that the phrase "one or more of the following" is intended to refer to one or more of those items (and therefore not all items need to be included), rather than one or more examples of each of those items (where all items are included). Some of this data (described in more detail below) may be generated only during clinical / remote operation mode, only during simulation mode (e.g., missed targets, falling objects, and tissue tears), or during either mode.

[0061] Data on robotic arm movement can refer to data from robotic arm 160 or other components of robotic surgical system 150. For example, such data may include, but is not limited to, user interaction, UID 122 location, joint angles, instrument and camera positions, instrument movement outside the field of view, indications of interlock engagement, workspace size, and the smoothness level of instrument movement. User-related physiological data may include data collected from sensors monitoring the user (e.g., blink count, heart rate, or other measurements related to cognitive load or stress), as well as self-reported task load index data. Manually entered data refers to data recorded by the instructor or supervisor at the time of occurrence (errors), such as tissue damage events. Video review data refers to data derived from subsequent case reviews using a common assessment framework, such as the Global Assessment of Robotic Skills (GEARS) or the Robotic Skills Assessment (RSA) scale. Video data can refer to video of events, errors, techniques, and movements detected by computer vision.

[0062] Return to reference Figure 5 Based on the user's skill level record, the processor creates a customized simulation training program (Action 520) for the user. For example, the processor can create a customized simulation training program by analyzing collected data and then mapping / transforming the data (indicating user skills that need improvement) to a specific simulation training program. The resulting customized simulation training program can be a sequence of steps performed during surgery, or a customized sequence of steps that differs from the sequence performed during surgery (e.g., based on skills assessment, progress tracking, and specific case studies). Furthermore, the customized simulation training program can include one or more of the following: basic skills training cycle, responsive training cycle, skills refresh cycle, career advancement cycle, upcoming case cycle, and system update cycle. Each of these cycles is designed to provide the right training at the right time, so that the user can effectively acquire and practice skills and avoid training that is not helpful to them.

[0063] The basic skills training cycle is an initial training simulation that includes a sequence of core lessons related to teaching basic robot control, safety, psychomotor, and sensory skills. Based on user performance metrics, the system can suggest that the user either repeat the training, try alternative training to enhance their skills, or continue to the next training session. In this way, even for prescribed basic skills lessons, the system can respond to the user's skills and guide them through the shortest path if they meet the baseline. The responsive training cycle responds to metrics collected during surgery that indicate the surgeon's weakest skill neighborhoods. The system can then recommend additional training to the surgeon by using simulation training or laboratory tasks to enhance those areas. Once the system sees an increase in a skill area during training, it can look for increases in the same skill area during surgery.

[0064] When surgeons are not frequent users of the robot, the skills refresh cycle can be used to detect skills decline. The system can detect when there are long time intervals between scheduled patient cases and require the user to take a rapid skills assessment test. If the user's metrics show decline in any skill area, the system can recommend simulation training or other training until the metrics return to their baseline. The career advancement cycle recognizes that at different stages of a surgeon's career, it is important for the surgeon to enhance different skills. The system can view aggregate metrics for surgeons across different career profiles, stages, and specialties and recommend which types of training might be useful for individual surgeons. The upcoming case cycle can be used when the system views a surgeon's upcoming cases. If there are any upcoming surgeries that the surgeon does not perform frequently, the system can recommend review training. This could be for complex procedures or the use of instruments of a different type than usual. The system update cycle can be used to notify surgeons of training when new instruments, new best practice guidelines, etc., are available.

[0065] After creating a customized simulation training program for the user, it can be provided to the user (Action 530). Figure 6 This is an illustration of the graphical user interface of a simulator that provides customized user training programs for implementation plans. For example... Figure 6 As shown in this example, the training in the program consists of games that are visually unrelated to surgery. For example, one game requires the user to pick up jacks and place them in bowls of different colors, while another requires the user to pick up rings and place them on nails. These training exercises help users develop skills that will be useful in surgery.

[0066] After the user completes training, the user's skill level record (Movement 540) is updated based on the training results. In one implementation, the user can be evaluated based on completion time, number of dropped objects, movement economy, excessive force, equipment collisions, and missed targets, and these scores are provided to the user (see [link to implementation details]). Figure 7 This information is used to update users' skill level records. The updated records can then be used to create additional, customized simulation training programs for the future.

[0067] Figures 8 to 11 Example implementations are provided and will be discussed below. It should be understood that these are merely examples and other implementations may be used.

[0068] First go to Figure 8 , Figure 8 This is a flowchart 800 illustrating a method for implementing a customized user training program. (Example) Figure 8 As shown, the processor of system 150 receives the user's surgical performance scores across various surgical skills (action 810). The processor then imports these surgical performance scores into the training simulator (action 820). Next, the processor integrates the surgical performance scores with the training scores (action 830). Then, when the user begins a training session using the simulator, the processor automatically selects the surgical skill with the lowest score for the user (action 840). Finally, the processor updates the user's training scores after the training session (action 850).

[0069] Figure 9 This is a flowchart 900 illustrating a method for implementing a customized user training program for surgeons. (Example) Figure 9 As shown, in this example, the surgeon experiences frequent instrument collisions during surgery (Action 910). System 150 identifies skills to be improved (e.g., hand dexterity) to avoid this problem (Action 920). System 150 examines which training sessions the user experienced the most instrument collisions (Action 930). The user practices those training sessions (e.g., nail board, camera aiming) until the score improves (Action 940). System 150 then observes the user's future surgeries for skill transfer (Action 950).

[0070] Figure 10 This is a flowchart 1000 of a method for implementing a customized user training program for resident physicians. (e.g.) Figure 10As shown, in this example, the resident receives feedback that they need to focus on endoscopic localization (Action 1010). System 150 then identifies skills to be improved (Action 1020), and in this example, recommends endoscopic localization training (Action 1030). The resident repeats the endoscopic localization training until a score reaches a baseline (Action 1040). System 150 provides feedback to the resident during future surgeries (Action 1050).

[0071] Figure 11 This is a flowchart 1100 of a method for implementing a customized user training program based on collected endoscopic positioning statistics. (e.g.) Figure 11 As shown, system 150 collects endoscopic positioning statistics (e.g., duration, adjustment frequency, etc.) of the surgeon during robotic surgery (action 1110). If system 150 determines that the surgeon's positioning skills need improvement (action 1120), the system recommends endoscopic positioning training during the surgeon's training (action 1130). This can be done by selecting one or more of a predetermined number of training exercises to be included in a customized simulation training program based on the skills the user needs to improve. If the surgeon passes the training exercises (action 1140), system 150 updates the surgeon's endoscopic positioning skills training report (action 1150).

[0072] Other exemplary embodiments include the following. Exemplary embodiments for one type of claim (e.g., system, method, computer program, or computer-readable storage medium) may be provided for other types (e.g., system as a method). Exemplary embodiments for one set (e.g., exemplary embodiments 1 to 9) may be used in other sets.

[0073] Example Implementation 1. A robotic surgical system comprising: a user console including a display device and a user input device; a robotic arm; and a processor configured to: maintain a skill level record for a user, the skill level record tracking the user's skill level in manipulating the robotic arm using the user input device; create a customized simulation training program for the user based on the user's skill level record; provide the customized simulation training program to the user; and update the user's skill level record based on the results of the customized simulation training program.

[0074] Exemplary Implementation 2. The robotic surgical system according to Exemplary Implementation 1, wherein the skill level record for the user is maintained by collecting data collected during previous actual surgeries performed by the user using the robotic surgical system.

[0075] Exemplary Embodiment 3. The robotic surgical system according to any one of Exemplary Embodiments 1 to 2, wherein the skill level record for the user is maintained by collecting data from previous simulation training performed by the user.

[0076] Exemplary Implementation 4. The robotic surgical system according to any one of Exemplary Implementations 1 to 3, wherein the skill level record for the user is maintained by collecting one or more of the following: data on the movement of the robotic arm, physiological data related to the user, manually entered data, video review data, and video data.

[0077] 5. The robotic surgical system according to any one of the exemplary embodiments 1 to 4, wherein the customized simulation training program is created by mapping the skills that need to be improved to a specific simulation training program.

[0078] 6. The robotic surgical system according to any one of the exemplary embodiments 1 to 5, wherein the customized simulation training program is based on the sequence of steps performed during surgery.

[0079] 7. The robotic surgical system according to any one of the exemplary embodiments 1 to 6, wherein the customized simulation training program provides a customized sequence of steps that differs from the sequential sequence of steps performed during surgery.

[0080] 8. The robotic surgical system according to any one of illustrative embodiments 1 to 7, wherein the customized simulation training program includes one or more of the following: a basic skills training cycle, a responsive training cycle, a skills refresh cycle, a career advancement cycle, an upcoming case cycle, and a system update cycle.

[0081] Example Implementation 9. A method for implementing a customized user training program on a robotic surgical system, the method comprising: performing the following operations in a robotic surgical system including a robotic arm and a user console including a display device and a user input device: determining whether a user's skill in moving the robotic arm using the user input device needs improvement; and in response to determining that the user's skill in moving the robotic arm using the user input device needs improvement, providing a customized simulation training program to improve the skill.

[0082] Example 10. The method according to Example 9, further comprising updating the records for the user based on the results of the customized simulation training program.

[0083] Exemplary Implementation Scheme 11. The method according to any one of Exemplary Implementation Schemes 9 to 10, wherein the user's skill is determined based on the user's surgical performance score, the user's simulation score, or both the user's surgical performance score and the user's simulation score.

[0084] Exemplary Implementation Scheme 12. The method according to any one of Exemplary Implementation Schemes 9 to 11, wherein the user's skill is determined by collecting one or more of the following: data on the movement of the robotic arm, physiological data related to the user, manually entered data, video review data, and video data.

[0085] Exemplary Implementation 13. The method according to any one of Exemplary Implementations 9 to 12, wherein providing the customized simulation training program includes selecting one or more of a predetermined number of training programs to be included in the customized simulation training program based on the skills the user needs to improve.

[0086] Exemplary Implementation Scheme 14. The method according to any one of Exemplary Implementation Schemes 9 to 13, wherein providing the customized simulation training program includes selecting one or more of the following based on the skills the user needs to improve: basic skills training cycle, responsive training cycle, skills refresh cycle, career advancement cycle, upcoming case cycle, and system update cycle.

[0087] Example 15. A robotic surgical system comprising: a user console including a display device and a user input device; a robotic arm; means for maintaining a skill level record for a user, the skill level record tracking the user's skill level in manipulating the robotic arm using the user input device; means for creating a customized simulation training program for the user based on the user's skill level record; means for providing the customized simulation training program to the user; and means for updating the user's skill level record based on the results of the customized simulation training program.

[0088] Exemplary Embodiment 16. The robotic surgical system according to Exemplary Embodiment 15, wherein the skill level record for the user is maintained by collecting data collected during previous actual surgeries performed by the user using the robotic surgical system.

[0089] Exemplary Embodiment 17. The robotic surgical system according to any one of Exemplary Embodiments 15 to 16, wherein the skill level record for the user is maintained by collecting data from previous simulation training performed by the user.

[0090] Exemplary Embodiment 18. The robotic surgical system according to any one of Exemplary Embodiments 15 to 17, wherein the skill level record for the user is maintained by collecting one or more of the following: data on the movement of the robotic arm, physiological data related to the user, manually entered data, video review data, and video data.

[0091] Exemplary Implementation 19. The robotic surgical system according to any one of Exemplary Implementations 15 to 18, wherein the customized simulation training program includes one or more of the following: a basic skills training cycle, a responsive training cycle, a skills refresh cycle, a career advancement cycle, an upcoming case cycle, and a system update cycle.

[0092] For purposes of explanation, the foregoing description uses specific nomenclature to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that specific details are not required to practice the invention. The foregoing description of specific embodiments of the invention has been provided for illustrative and descriptive purposes. These are not intended to be exhaustive or to limit the invention to the specific forms disclosed; various modifications and alterations can be made to this disclosure in light of the foregoing teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, thereby enabling others skilled in the art to best utilize the invention and various embodiments with various modifications suitable for the contemplated particular uses. The following claims and their equivalents are intended to define the scope of the invention.

Claims

1. A robotic surgical system, comprising: The user console includes a display device and a user input device; Robotic arm; and Processor, the processor being configured to: Maintain a skill level record for the user, which tracks the user's skill level in manipulating the robotic arm using the user input device; A customized simulation training program is created for the user based on the user's skill level record; Provide the customized simulation training program to the user; as well as The user's skill level record is updated based on the results of the customized simulation training program.

2. The robotic surgical system according to claim 1, wherein, The skill level record for the user is maintained by collecting data during previous actual surgeries performed by the user using the robotic surgical system.

3. The robotic surgical system according to claim 1, wherein, The skill level record for the user is maintained by collecting data from previous simulated training performed by the user.

4. The robotic surgical system according to claim 1, wherein, The skill level record for the user is maintained by collecting one or more of the following: data on the movement of the robotic arm, physiological data related to the user, manually entered data, video review data, and video data.

5. The robotic surgical system according to claim 1, wherein, The customized simulation training program is created by mapping the skills that need improvement to a specific simulation training program.

6. The robotic surgical system according to claim 1, wherein, The customized simulation training program is based on the sequence of steps performed during surgery.

7. The robotic surgical system according to claim 1, wherein, The customized simulation training program provides a customized sequence of steps that differs from the sequence of steps performed during surgery.

8. The robotic surgical system according to claim 1, wherein, The customized simulation training program includes one or more of the following: basic skills training cycle, responsive training cycle, skills refresh cycle, career advancement cycle, upcoming case cycle, and system update cycle.

9. A method for implementing a customized user training program on a robotic surgical system, the method comprising: Perform the following operations in a robotic surgical system that includes a robotic arm and a user console that includes display devices and user input devices: Determine whether the user's skill in moving the robotic arm using the user input device needs improvement; as well as In response to the determination that the user's skill in moving the robotic arm using the user input device needs improvement, a customized simulation training program is provided to improve the skill.

10. The method of claim 9, further comprising updating the records for the user based on the results of the customized simulation training program.

11. The method according to claim 9, wherein, The user's skill is determined based on the user's surgical performance score, the user's simulation score, or both the user's surgical performance score and the user's simulation score.

12. The method according to claim 9, wherein, The user's skills are determined by collecting one or more of the following: data on the movement of the robotic arm, physiological data related to the user, manually entered data, video review data, and video data.

13. The method according to claim 9, wherein, Providing the customized simulation training program includes selecting one or more of a predetermined number of training programs to be included in the customized simulation training program based on the skills the user needs to improve.

14. The method according to claim 9, wherein, The customized simulation training program includes selecting one or more of the following based on the skills the user needs to improve: basic skills training cycle, responsive training cycle, skills refresh cycle, career advancement cycle, upcoming case cycle, and system update cycle.

15. A robotic surgical system, comprising: The user console includes a display device and a user input device; Robotic arm; A device for maintaining a skill level record for a user, the skill level record tracking the user's skill level in manipulating the robotic arm using the user input device; A device for creating a customized simulation training program for the user based on the user's skill level record; A device for providing the customized simulation training program to the user; and A device for updating the user's skill level record based on the results of the customized simulation training program.

16. The robotic surgical system of claim 15, wherein the skill level record for the user is maintained by collecting data acquired during previous actual surgeries performed by the user using the robotic surgical system.

17. The robotic surgical system of claim 15, wherein the skill level record for the user is maintained by collecting data from previous simulated training performed by the user.

18. The robotic surgical system of claim 15, wherein the skill level record for the user is maintained by collecting one or more of the following: data on the movement of the robotic arm, physiological data related to the user, manually entered data, video review data, and video data.

19. The robotic surgical system of claim 15, wherein the customized simulation training program includes one or more of the following: a basic skills training cycle, a responsive training cycle, a skills refresh cycle, a career advancement cycle, an upcoming case cycle, and a system update cycle.

Images