Integrated robotic surgical system simulator and method for performing psychomotor memory training

Abstract

Integrated robotic surgical system simulators and methods for providing psychomotor memory training are provided. In one embodiment, a robotic surgical system user console is provided that includes a plurality of user input devices; a simulator that provides simulated training; and a switch. When in a clinical mode, the switch causes signals generated in response to user manipulation of the plurality of user input devices to cause movement of a robotic arm of the robotic surgical system. However, when in a simulation mode, the switch causes signals generated in response to user manipulation of the plurality of user input devices to be provided to the simulator for simulated training, but not to cause movement of the robotic arm of the robotic surgical system. Other embodiments are provided.

Description

Related applications

[0001] This patent document claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 595,045, filed November 1, 2023, and U.S. Provisional Patent Application Serial No. 63 / 595,050, filed November 1, 2023, pursuant to 35 USC §119(e), which are hereby incorporated by reference. Technical Field

[0002] The following implementation scheme relates to the field of robotic surgery as a whole, and more specifically to a robotic surgical system simulator. Background Technology

[0003] Minimally invasive surgery (MIS), such as laparoscopic surgery, involves techniques designed to minimize tissue damage during surgical procedures. For example, laparoscopic surgery typically involves making multiple small incisions inside the patient (e.g., in the abdomen) and introducing one or more surgical instruments (e.g., end effectors, at least one camera, etc.) through these incisions. Surgical procedures can then be performed using the introduced surgical instruments, with visualization aids provided by the camera.

[0004] Generally, medical interventions (MIS) offer multiple beneficial effects, such as reducing patient scarring, alleviating patient pain, shortening patient recovery time, and reducing medical costs associated with patient recovery. In some implementations, MIS can be performed using a robotic system comprising one or more robotic arms for manipulating surgical instruments based on commands from an operator. The robotic arms may, for example, support various devices at their distal ends, such as surgical end effectors, imaging devices, cannulas for providing access to the patient's body cavities and organs.

[0005] Simulators are available to assist surgeons in becoming familiar with the use of robotic surgical systems. Attached Figure Description

[0006] Figure 1A An example of an operating room layout with a robotic surgical system and a user console is depicted in the implementation scheme.

[0007] Figure 1B This is a schematic illustration of an exemplary variant of the robotic arm manipulator, tool actuator, and cannula with surgical tools in the implementation scheme.

[0008] Figure 1C This is a schematic illustration of an exemplary user console for the implementation scheme.

[0009] Figure 2 This is a schematic illustration of an exemplary variant of a user console for an implementation of a robotic surgical system used to communicate with one or more third-party devices.

[0010] Figure 3 This is a schematic diagram of a surgical robot platform with a graphical user interface (GUI) module, wherein the surgical robot platform communicates with multiple medical data resources.

[0011] Figure 4A and Figure 4B These are, respectively, a perspective view and a longitudinal cross-sectional view of an exemplary variant of the handheld user input device of the implementation scheme.

[0012] Figure 5A and Figure 5B This is a schematic diagram of an exemplary variant of a robotic surgical system with an integrated simulator.

[0013] Figure 6 This is a flowchart of a method for implementing training on a simulator.

[0014] Figure 7 This is a diagram illustrating an implementation of the workspace of a tool for determining the given location of a user input device (UID).

[0015] Figure 8 This is an illustration of an implementation scheme in which moving the UID during active teleoperation moves the tool without moving the workspace.

[0016] Figure 9 This is an illustration of an implementation scheme in which moving the UID during gripping causes the tool workspace to move in the opposite direction.

[0017] Figure 10 This is a diagram illustrating an implementation where the UID workspace to the tool workspace is independent for each tool.

[0018] Figure 11 This is a diagram illustrating an implementation scheme of a moving camera grasping device. Detailed Implementation

[0019] Non-limiting examples of various aspects and variations of the implementation scheme are described herein and illustrated in the accompanying drawings.

[0020] Overview of Robotic Surgical Systems

[0021] Figure 1A This is an illustration of an exemplary operating room environment with a robotic surgical system. Typically, as... Figure 1AAs shown, the robotic surgical system includes a user console 100 (sometimes referred to herein as a “surgeon’s bridge” or “bridge”), a control tower 133, and one or more robotic arms 160 located on a robotic platform (e.g., a table, bed, etc.), wherein surgical instruments (e.g., having end effectors) are attached to the distal end of the robotic arm 160 to perform surgical procedures. The robotic arm 160 is shown as a tabletop-mounted system, but in other configurations, one or more robotic arms may be mounted on a trolley, ceiling, or sidewall, or other suitable support surface.

[0022] As further illustration, such as Figure 1B As illustrated in the exemplary schematic, a robotic surgical system may include at least one robotic arm 160 and a tool actuator 170 typically attached to the distal end of the robotic arm 160. A cannula 180 coupled to the end of the tool actuator 170 may receive and guide surgical instruments 190 (e.g., end effectors, cameras, etc.). Furthermore, the robotic arm 160 may include a plurality of actuated links to position and orient the tool actuator 170, which actuates the surgical instruments 190.

[0023] Usually, such as Figure 1A As shown, the user console 100 can be used to interface with the robotic surgical system 150. A user (such as a surgeon or other operator) can use the user console 100 to remotely manipulate the robotic arm 160 and / or surgical instruments (e.g., in remote operation). The user console 100 may be located in the same operating room as the robotic system 150, such as... Figure 1A As shown. In other embodiments, the user console 100 may be located in an adjacent or nearby room, or be remotely operated from a remote location in a different building, city, or country. In one example, the user console 100 may include a seat 110, foot pedal controls (pedals) 120, one or more handheld user input devices 122, and at least one user display 130 configured to display, for example, a view of a surgical site inside a patient's body (e.g., captured by an endoscopic camera), and / or other surgical or medical information.

[0024] exist Figure 1CIn the exemplary user console shown, a user seated in seat 110 and viewing user display 130 can manipulate foot pedal controls 120 and / or handheld user input device 122 to remotely control robotic arm 160 and / or surgical instruments mounted to the distal end of that arm. Foot pedal controls 120 and / or handheld user input device 122 can additionally or alternatively be used to control other aspects of user console 100 or robotic system 150. For example, in a variation where the user typically controls (at any given time) a designated “left-hand” robotic arm / instrument and a designated “right-hand” robotic arm / instrument, foot pedal controls 120 allow the user to specify which robotic arms / instruments include both “left-hand” and “right-hand” robotic arms / instruments from a larger set of available robotic arms / instruments (e.g., by switching or rotating when selecting available robotic arms / instruments). Other examples include adjusting or configuring seat 110, foot pedal controls 120, user input device 122, and / or user display 130.

[0025] In some variants, a user can operate the surgical robotic system in an "OTB" (over-the-bed) mode, where the user is positioned to one side of the patient and simultaneously manipulates both robot-driven instruments / end-effectors attached to the patient (e.g., a handheld user input device 122 held in one hand) and manual laparoscopic tools. For example, the user's left hand can manipulate the handheld user input device 122 to control the robotic surgical components, while the user's right hand can manipulate the manual laparoscopic tools. Thus, in these variants, the user can perform both robot-assisted MIS and manual laparoscopic surgery on the patient.

[0026] During the exemplary procedure or surgery, the patient is prepared and covered aseptically, and anesthesia is administered. Initial approach to the surgical site can be manually performed using the robotic system 150 in a retracted or collapsed configuration to facilitate access. Once approach is complete, initial positioning and / or preparation of the robotic system can be performed. During the surgical procedure, the surgeon or other user at the user console 100 can manipulate various end effectors and / or imaging systems to perform the procedure using foot pedal controls 120, user input devices 122, and / or other suitable controls. Manual assistance can be provided by other personnel at the operating table, who can perform tasks including, but not limited to, retracting tissue, or performing manual repositioning or tool changes involving one or more robotic arms 160. Other personnel may be present to assist the user at the user console 100. Medical and surgical information assisting other medical personnel (e.g., nurses) can be provided on additional displays, such as a display 134 on a control tower 133 (e.g., a control system for a robotic surgical system) and / or a display 132 located near the patient's bedside. For example, as described further in detail herein, some or all of the information displayed to the user in the user console 100 may also be displayed on at least one additional monitor of other personnel and / or provide additional pathways for inter-personnel communication. Upon completion of a procedure or surgical operation, the robotic system 150 and / or the user console 100 may be configured or set to facilitate one or more postoperative procedures, including but not limited to robotic system cleaning and / or sterilization, and / or medical record input or printout, whether electronic or hard copy, via the user console 100.

[0027] In some variations, communication between the robotic system 150, the user console 100, and any other displays can be achieved via a control tower 133, which translates user commands from the user console 100 into robot control commands and transmits these commands to the robotic system 150. The control tower 133 can also transmit status and feedback from the robotic system 150 back to the user console 100 (and / or other displays). The connection between the surgical robotic system 150, the user console 100, other displays, and the control tower 133 can be via wired and / or wireless connections, and can be proprietary and / or use any of a variety of data communication protocols. Any wired connection can be integrated into the operating room floor and / or walls or ceiling. The robotic surgical system can provide video output to one or more displays, including displays within the operating room and remote displays accessible via the Internet or other networks. Video output or feeds can be encrypted to ensure privacy, and all or one or more portions of the video output can be stored on a server, an electronic health record system, or other suitable storage medium.

[0028] In some variations, an additional user console 100 may be provided, for example, to control additional surgical instruments and / or to control one or more surgical instruments on the main user console. This would allow, for example, a surgeon to take over or explain a technique to medical students and physicians in training during surgical procedures, or to assist during complex surgeries that require multiple surgeons to act simultaneously or in a coordinated manner.

[0029] In some variations, such as Figure 2 As illustrated in the schematic diagram, one or more third-party devices 240 may be configured to communicate with the user console 210 and / or other suitable parts of the robotic surgical system. For example, as described elsewhere herein, a surgeon or other user may be seated in the user console 210, which may communicate with the robotic instruments in the control tower 230 and / or the robotic system 220. Medical data (e.g., endoscopic images, patient vital signs, tool status, etc.) may be displayed on the user console 210, the control tower 230, and / or other displays. At least a subset of surgical and other medically relevant information may also be displayed on the third-party device 240, such as a remote computer monitor, which may be viewed by surgical collaborators in or outside the same room. Other communications, such as teleconferences using audio and / or visual communications, may be further provided to and from the third-party device. Surgical collaborators may be, for example, supervisors or trainers, medical colleagues (e.g., radiologists), or other third parties who may view and communicate, for example, via the third-party device 240, to assist in surgical procedures.

[0030] Figure 3 This is a schematic illustration of an exemplary variant of system 300, which includes a robotic surgical system and its interaction with other devices and parties. Although Figure 3 The document describes specific architectures for various connection and communication systems, but it should be understood that other suitable architectures can be used in other variations, and Figure 3 The arrangement shown is for illustrative purposes. System 300 may include a surgical robot platform 302 that facilitates the integration of medical data from discrete medical data resources generated from various parties. Data from discrete medical data resources can, for example, be used to form temporally coordinated medical data. As further described herein, a multi-panel display for presenting the temporally coordinated medical data can be configured and presented.

[0031] Platform 302 may be, for example, a machine having one or more processors 310 connected via a bus 314 to one or more input / output devices 312. At least one processor may include, for example, a central processing unit, a graphics processing unit, an application-specific integrated circuit, a field-programmable logic device, or a combination thereof.

[0032] The surgical robot platform 302 may include one or more input ports to receive medical data from discrete medical data resources. For example, surgical robot port 329 may receive surgical robot data from surgical robot 330. Such data may include, for example, position data or other suitable status information. Imaging port 331 may receive imaging data from imaging device 332 (such as an endoscope) configured to capture images (e.g., still images, video images) of the surgical site. The endoscope may be inserted, for example, through a natural orifice or through a hole in the patient's body. As another example, one or more medical instrument ports 333 may receive patient vital signs information from medical instrument 334 (e.g., pulse oximeter, electrocardiogram device, ultrasound device, etc.). Additionally, as another example, one or more user control data ports 335 may receive user interaction data from one or more control devices that receive user input from the user to control the system. For example, one or more handheld user input devices, one or more foot pedals, and / or other suitable devices (e.g., eye tracking, head tracking sensors) may receive user input.

[0033] The surgical robot platform 302 may also include one or more output ports 337 configured to connect to one or more displays 338. For example, displays 338 may include open displays (e.g., monitor screens) in a user console, immersive displays or head-mounted devices with displays, supplemental displays such as those on control tower displays (e.g., team displays), bedside displays (e.g., nurse displays), overhead "stadium" style screens, etc. For example, the graphical user interface disclosed herein may be presented on one or more displays 338. One or more displays 338 may present three-dimensional images. In some variations, one or more displays 338 may include touchscreens. One or more displays 138 may be a single display with multiple panels, each displaying different content. Alternatively, one or more displays 138 may include a collection of individual displays, each individual display presenting at least one panel.

[0034] In some variations, network interface 316 may also be connected to bus 314. For example, network interface 316 may provide connectivity to network 317, which may be any combination of one or more wired and / or wireless networks. For example, network 317 may facilitate communication between the surgical robot platform 302 and other data sources or other devices. For example, one or more third-party data sources 340 may also be connected to network 317. Third-party sources 340 may include third-party devices (e.g., another computer operated by a third party such as another physician or medical specialist), a repository of video surgical procedure data (e.g., data that may be related to a procedure performed by a surgeon), or other suitable additional information sources related to the surgical procedure. For example, third-party device data may be transmitted to a panel displayed to the surgeon before, during, or after the procedure.

[0035] As another example, one or more application databases 342 may be connected to network 317 (or alternatively, locally stored in memory 320 within the surgical robot platform 302). Application database 342 may include software applications (e.g., as described further below), which the surgeon may be interested in during the procedure. For example, the software application may provide access to stored patient medical records, provide a checklist for surgical tasks within the surgical procedure, perform machine vision techniques to assist the procedure, perform machine learning tasks to improve the surgical task, etc. Any suitable number of applications may be invoked. During the procedure, information associated with the applications may be displayed on a multi-panel display or other suitable display. Additionally or alternatively, information provided by one or more applications may be provided through a separate resource (e.g., a machine learning resource) that is also suitably in communication with the surgical robot platform 302.

[0036] In some variations, one or more software applications within a software application can function as separate processes that use an application interface (API) to draw objects and / or images on a display. APIs of varying complexity can be used. For example, a simple API might include several templates with fixed widget sizes and positions, which can be used by a GUI module to customize text and / or images. As another example, a more complex API could allow the software application to create, place, and delete different widgets, such as labels, lists, buttons, and images.

[0037] Alternatively or additionally, one or more software applications may render themselves for display. This can, for example, allow for advanced customization and complex behavior of the applications. For instance, this method can be implemented by allowing the application to pass frames rendered by a graphical user interface (GUI) module 324, which may be computer-readable program code executed by processor 310. Alternatively, an image buffer may be used as a store for the application to render itself.

[0038] In some variations, one or more software applications can run and render themselves independently of the GUI module 324. However, the GUI module can still launch such applications, instruct applications or the operating system where to position applications on the display, and so on.

[0039] As another approach, in some variations, one or more applications can run completely independently of the GUI rendered by the GUI module. For example, such applications can have physical video and data connections to the system (e.g., via suitable input / output devices, networks, etc.). The data connection can be used to configure the application's video feed to appropriate pixel dimensions (e.g., full-screen, half-screen, etc.).

[0040] like Figure 3 As shown, in some variations, memory 320 may also be connected to bus 314. Memory 320 may be configured to store data processed according to implementations of the methods and systems described herein.

[0041] In some variations, memory 320 may be configured to store other types of data and / or software modules for execution. For example, a user console may include memory 320 storing a GUI module 324 with executable instructions to perform the operations disclosed herein. The GUI module may, for example, combine and aggregate information from various software applications and / or other medical data resources for display. In some exemplary variations, one or more software applications may be incorporated into the base code of the GUI module, causing the module to draw graphics and display text at appropriate locations on the display. For example, the module may retrieve images from a database or push images from instruments (e.g., endoscopic cameras) in the operating room to the interface via a wired or wireless interface.

[0042] In some variations, medical data can be collected from discrete medical data resources, such as surgical robots 330, endoscopes 332, medical instruments 334, control devices 336, third-party data sources 340, application databases 342, etc. Furthermore, at least some of the medical data can be temporally coordinated, such that time-sensitive information from different medical data resources is aligned on a common timeline when necessary. For example, surgical robot position data can be temporally coordinated with endoscope data, which in turn is coordinated with operator interaction data from the control device. Similarly, network resources, such as information provided by one or more software applications, can be presented at appropriate points in time along with other temporally coordinated data. Multi-panel displays and / or other suitable displays can be configured to convey medical information (e.g., including temporally coordinated medical data) as part of a graphical user interface (GUI).

[0043] This document describes various exemplary aspects of the GUI (User Interface) for robotic surgical systems. In some variations, the GUI may be displayed on a multi-panel display at the user console controlling the robotic surgical system. Alternatively or additionally, the GUI may be displayed at one or more additional displays, such as at the control tower of the robotic surgical system, at the patient's bedside, etc. Typically, the GUI can provide more efficient information communication to the user and / or other personnel at the user console, as well as more efficient communication and collaboration between different parties involved in surgical procedures, as further described below.

[0044] Graphical User Interface (GUI) Interaction

[0045] In one implementation, the GUI is displayed on a monitor 130 in a user console 100, which is used (e.g., by a surgeon) to control the robotic surgical system 150. At least some of the interactive graphical objects displayed on the monitor 130 can be controlled, selected, or otherwise interacted with via one or more user controls that are also used to control aspects of the surgical system (e.g., surgical instruments). For example, a user can selectively control aspects of the robotic surgical system 150 and selectively interact with the GUI using one or more handheld user input devices 122 and / or one or more foot pedals 120. By enabling the use of the same user controls to control both the robotic surgical system 150 and the GUI, the user can advantageously avoid having to switch between two different types of user controls. Enabling the user to use the same input devices to control both the robotic system 150 and the GUI streamlines surgical procedures and improves efficiency, as well as helping the user maintain aseptic technique throughout the surgical procedure.

[0046] like Figure 4A and Figure 4BAs generally shown, an exemplary variant of the handheld user input device 122 for controlling a robotic system may include a component 410, a housing 420 at least partially disposed around the component 410 and configured to be held in a user's hand, and a tracking sensor system 440 configured to detect the position and / or orientation of at least a portion of the device. The housing 420 may be flexible (e.g., made of silicone). In some instances, the detected position and / or orientation of the device may be associated with controls of the robotic system. For example, the user input device 122 may control at least a portion of a robotic arm, an end effector or tool (e.g., a gripper or clamp) coupled to the distal end of the robotic arm, a GUI, or other suitable aspects or features of the robotic surgical system 150. Additionally, in some instances, the detected position and / or orientation of the device 122 may be associated with controls of the GUI. Furthermore, in some variants, the user input device 122 may include one or more sensors for detecting other manipulations of the user input device 122, such as squeezing the housing 420 (e.g., via one or more pressure sensors, one or more capacitive sensors, etc.).

[0047] Typically, the user interface for controlling a robotic surgical system may include at least one handheld user input device 122, or may include at least two handheld user input devices 122 (e.g., a first user input device to be held by the user's left hand and a second user input device to be held by the user's right hand), or any suitable number. Each user input device 122 may be configured to control one or more different aspects or features of the robotic system. For example, a user input device held in the user's left hand may be configured to control an end effector represented on the left side of a camera view provided to the user, while a user input device held in the user's right hand may be configured to control an end effector represented on the right side of a camera view.

[0048] In some variations, the handheld user input device 122 can be an ungrounded user input device configured to be held in the hand and manipulated in free space. For example, the user input device 122 can be configured to be held between the user's fingers and moved freely by the user (e.g., translation, rotation, tilting, etc.) as the user moves his or her arm, hand, and / or fingers. Alternatively, the handheld user input device 122 can be a body-grounded user input device, since the user input device 122 can be attached directly or via any suitable mechanism (such as a glove, hand strap, sleeve, etc.) to a part of the user (e.g., the user's fingers, hand, and / or arm). Such a body-grounded user input device still allows the user to manipulate the user input device in free space. Thus, in variations where the user input device 122 is ungrounded or body-grounded (as opposed to permanent mounting or grounding to a fixed console, etc.), the user input device 122 can be ergonomic and provide dexterous control, such as by allowing the user to control the user input device with natural body movements unimpeded by the fixed characteristics of a grounding system.

[0049] The handheld user input device 122 may include a wired connection, for example, to provide power to the user input device 122 and carry sensor signals (e.g., from a tracking sensor assembly and / or other sensors such as capacitive sensors, optical sensors, etc.). Alternatively, the user input device may be wireless, such as... Figure 4A As shown, commands and other signals are transmitted via radio frequency signals (e.g., WiFi or short-range signals such as 400mm-500mm range) or other suitable wireless communication protocols (such as Bluetooth). Further wireless connectivity can be facilitated using optical reader sensors and / or cameras (configured to detect optical marks on user input device 122), infrared sensors, ultrasonic sensors, or other suitable sensors.

[0050] Handheld user input devices may include a clutch mechanism for switching between controlling a robotic arm or end effector and controlling a graphical user interface, and / or switching between other control modes. One or more user inputs, among the various user inputs described further in detail below, can be used as a clutch in any suitable combination. For example, a gesture touch area of ​​a touch device, squeezing the housing, flicking or rotating the user input device can act as an engaging clutch. As another example, a combination of squeezing and holding the user input device with rotating the user input device can be used as a clutch. However, any suitable combination of gestures can be used as a clutch. Additionally or alternatively, user input to other user input devices (e.g., foot pedal assemblies) can be used as a clutch, either alone or in combination with user input to the handheld user input device.

[0051] In some variations, the engagement and disengagement of the clutch mechanism allows for switching between using a handheld user input device as a controller for the robotic system and using it as a controller for the GUI (e.g., operating a cursor displayed on a screen). When the clutch mechanism is engaged so that the user input device is used to control the GUI, the position or orientation of the robot arm can be essentially locked in place to "pause" the operation of the robotic system, ensuring that subsequent movement of the user input device when the clutch is engaged will not unintentionally cause movement of the robot arm.

[0052] simulator

[0053] Another variation of the GUI application is the simulator application. Surgeons and operating room staff need to be trained in realistic simulations to acquire the skills to use and successfully employ the robotic surgical system 150. The simulator application may, for example, communicate with a database storing simulated surgical robot experiences or simulation training, such as to teach user-specific psychomotor skills used for robot simulation (e.g., games for practicing scrolling movements and / or other skills on a handheld user input device). Running the simulator on the surgeon's bridge 100 provides the most realistic and full-featured training because the surgeon uses the same UID and GUI as during actual surgical procedures. Simulated surgical robot experiences may include, for example, simulations and training with simulated patients. The simulator application can load such simulation experiences into the GUI, including simulated endoscopic views and other patient parameters. The simulator allows surgeons to build the psychomotor skills required for surgical procedures and provides performance metrics for the simulation. For example, surgeons can be trained to manipulate instruments for pick-up and placement, camera aiming, needle driving, suturing, and knot tying using the simulator. Simulation experiences may also include simulated events such as robotic arm collisions, patient emergencies, and other appropriate events that can help new users (e.g., surgeons in training) learn how to respond appropriately and resolve problems. Simulations can be generated independently of simulator applications (such as using simulation development software), or alternatively, they can be generated within the simulator application itself.

[0054] In some variations, simulator applications can rate users based on their performance in simulated training, such as by providing scores. These scores can be tracked over time to determine the trainee's progress and proficiency in using the robotic surgical system. In other variations, simulator applications can display the user's progress throughout a set course (e.g., indicating that the user has completed three out of ten training sessions), assess the user's baseline skills to customize or adjust the course, and / or provide recommendations for specific simulated training based on the user's performance.

[0055] In one implementation, the simulator's hardware / software components are integrated into the user console / surgeon bridge 100 of the robotic surgical system 150 (e.g., in the same housing as other components of the console / surgeon bridge 100). Previous simulators were separate components removably attached to the outer surface of the user console, with data cables plugged into the console and power cables plugged into a power outlet. Therefore, setting up this external component could take several minutes before use and additional time was required to disassemble it after use. Furthermore, the extra cables could pose tripping or other hazards. In contrast to these master-slave training stations, with the simulator integrated into the surgeon bridge 100, the user simply clicks a button on the GUI to use the simulator application, and the simulator is ready to use (possibly after a short reset time), without the user needing to worry about plugging in data and power cables or waiting for other components to start up.

[0056] Return to the attached image. Figure 5A and Figure 5B These figures illustrate a surgeon's bridge 100 with an integrated simulator implementation. As shown in these figures, in this implementation, the surgeon's bridge 100 includes a surgeon's bridge mode switch 510, a surgeon's bridge computer 520 (which may be a first processor), a simulator control computer 540 (which may be a second processor), and a simulator rendering computer 550 (which may be a third processor). The surgeon's bridge mode switch 510 may be positioned to communicate with a control tower switch 530 (which provides communication with a robot arm controller), and the simulator control computer 540 may be positioned to communicate with a network cloud 580 (e.g., the Internet).

[0057] The surgeon bridge computer 520 receives signals from the surgeon bridge user input devices (e.g., UID, pedals, and displays) and provides these signals to the surgeon bridge mode switch 510. During actual surgical procedures, these signals are provided to a control tower switch 530, which uses these signals to manipulate the surgical robot. The control tower switch 530 can also provide feedback and other signals to the surgeon bridge computer 520 for display on the GUI of the surgeon bridge 100.

[0058] In this embodiment, the surgeon bridge computer 520 and the simulator control computer 540 form a simulator. In operation, the simulator control computer 540 provides the hardware / software to create the simulation, accepts user input from the surgeon bridge computer 520 (typically used to control the robot), and responds to it. For example, the simulator control computer 540 may connect to a cloud 580 (e.g., the Internet) to download simulation software and upload simulation results. A simulator rendering computer 550 communicates with the simulator control computer 540 to provide visual rendering of the simulation on a GUI displayed on the monitor of the surgeon bridge 100.

[0059] In this embodiment, the surgeon bridge computer 520 and the simulator control computer 540 are integrated within the surgeon bridge 100 because those components are housed in the same enclosure as other components of the surgeon bridge computer 520 and the surgeon bridge 100 (some of which are not shown in the figures). In one embodiment, the simulator control computer 540 is a separate computer system with a processor, memory, and storage devices independent of the surgeon bridge computer 520. In an alternative embodiment, the simulator control computer 540 and the surgeon bridge computer 520 share some components. In yet another alternative embodiment, the simulator runs as a virtual machine on the surgeon bridge computer 520.

[0060] It should be noted that while the simulator in this embodiment includes a surgeon control computer 540 and a simulator control computer 550, the simulator may include any suitable components. Generally, "simulator" refers to hardware and / or software components (e.g., a processor executing computer-readable instruction code) that allow simulations to be performed on the surgeon bridge 100 to allow a user to practice using the user input device of the surgeon bridge 100. As discussed in more detail below, simulations may be simulated surgical procedures and / or field training that, while not simulating surgery, still teaches the user specific psychomotor skills / memory, which may be helpful in performing actual surgical procedures using the robotic surgical system 150.

[0061] As described above, during actual surgical procedures, user input signals are transmitted from the surgeon bridge controller 520 to the control tower switch 530, but during simulation training, user input signals are transmitted to the simulator (in this example, the simulator control computer 540). In this embodiment, the surgeon bridge mode switch 510 is used to direct this information traffic. Generally, the surgeon bridge mode switch 510 is a management hub that creates communication channels between various ports and can be used to switch between clinical and simulation modes. In one embodiment, the surgeon bridge mode switch 510 is a management switch that creates virtual local area networks (VLANs) between different ports in the switch 510. (Although VLANs are used in these examples, it should be noted that other types of channels can be used.) Through management, the surgeon bridge mode switch 510 is configured to allow connectivity between some ports while restricting connectivity between other ports.

[0062] For example, such as Figure 5A As shown, when the robotic surgical system 150 is in simulation mode, the surgeon bridge mode switch 510 creates a VLAN 560 between the simulator control computer 540 and the surgeon bridge computer 520. As indicated by the "X" in the figure, switch 510 does not create a VLAN communication channel between the simulator control computer 540 and the control tower switch 530 (other channels, such as power management channels, remain open). That is, in simulation mode, switch 510 is configured to establish a VLAN (or other type of communication channel) between the surgeon bridge computer 520 and the simulator control computer 540, such that signals generated by the surgeon bridge computer 520 in response to user manipulation of the multiple user input devices are provided to the simulator control computer 540 for simulation training, but do not cause movement of the robotic arm of the robotic surgical system 150. Therefore, during simulation mode, the surgeon bridge computer 520 is disconnected from the control tower switch 530, providing a safety feature to ensure that any user input from the surgeon bridge computer 520 only drives the simulation and not the actual surgical robot. Thus, this implementation replicates the user experience of a real robot while ensuring that the simulation / training does not interact with the robotic arm.

[0063] Similarly, for security and reliability, it may be desirable to isolate the simulator rendering computer 550 (e.g., a Windows-based personal computer (PC)) from clinical components. This implementation achieves this by creating another VLAN 570 between only the simulator rendering computer 550 and the simulator control computer 540.

[0064] like Figure 5BAs shown, when a user switches modes (e.g., by selecting a virtual button on the GUI to end the simulation), the robotic surgical system 150 disables VLAN 560 between the surgeon bridge computer 510 and the simulator control computer 540 via switch 510, and establishes VLAN 590 (or other type of communication channel) between the surgeon bridge computer 510 and the robotic arm controller of the robotic surgical system via the control tower computer 530, thereby switching from simulation mode to clinical mode. In this mode, signals generated by the surgeon bridge computer in response to user manipulation of the multiple user input devices cause movement of the robotic arm of the robotic surgical system 150.

[0065] As described above, because the simulator in this implementation is integrated into the surgeon bridge 100, switching between modes can be accomplished through a simple user interface selection, avoiding the installation and disassembly hassles associated with using a separate machine and its associated wiring. Furthermore, since the simulator is integrated into the surgeon bridge 100, any surgeon bridge can be used as a standalone simulator. Additionally, simulation and training can be synchronized based on user login information and profiles, thus not being tied to any specific simulator / surgeon bridge.

[0066] As described above, simulators can be used to provide training activities, procedural steps, and evaluation metrics to practice psychomotor skills on the same surgeon's bridge 100 that will be used to perform actual surgical procedures. The robotic surgical system 150 includes an ergonomic layout on which surgeons need to perform specific psychomotor skills to achieve success during procedures. Therefore, simulation helps surgeons learn the location of controllers (buttons, pedals, etc.), their movements (depending on system state), and feedback from the system. Training with simulators allows surgeons to progressively build safety, efficiency, and confidence, thereby shortening the learning curve on real robotic systems.

[0067] Repetition is a highly effective method for mastering psychomotor skills, and simulators can provide core skills training for foundational training, enabling users to perform simple tasks in a 3D simulation environment after several runs. The following paragraphs describe solutions for making the granularity of repetitive steps finer. In one implementation, training is designed to have users repeat atomic actions (e.g., pedal presses) or sequences of actions (e.g., engagements) to build psychomotor memory. Ultimately, associating the concept of a controller or action with the correct movement of (e.g., the user's feet) will become second nature. This is facilitated by ergonomics based on user profiles.

[0068] As mentioned above, a primary goal of training simulators is to reduce the length of the learning curve for new users on the robotic surgical system 150. Mastering psychomotor skills and memory are factors that play a role in the initial stages of the learning curve, and training simulators can be used to increase the slope of the learning curve. For example, when using an immersive display, the head must be turned back to see the pedals. When using an open display, looking at the pedals may cause the robot to disengage due to eye trackers. Knowing the location of the pedals and having the ability to move the feet to the correct position becomes crucial for improving the user's efficiency and confidence. Furthermore, simulators can be used to refine skills after initial training.

[0069] To maintain user engagement, training provided by the simulator can be designed as engaging games that incentivize users to return to training for better scores and participate in additional simulated training beyond the initial setup. For example, the simulator could present a memory game where the user must perform a list of actions, with actions added to replays of the game. Alternatively, the simulator could present a scrolling / running list of actions to perform, similar to Guitar Hero. ™ Video games. Of course, these are just examples, and other types of training can be used.

[0070] Turn to the attached image again. Figure 6 This is a flowchart 600 of a method for implementing training on a simulator. (Example) Figure 6 As shown, first, the GUI prompts the user to perform an action on the surgeon bridge 120 (e.g., by the name of the controller or by an action associated with the controller) (Action 610). Next, the user performs the action as quickly as possible (Action 620). The GUI then indicates whether the action has been performed correctly (Action 630). This loop can be repeated multiple times.

[0071] Depending on the required level of difficulty, several variations of this workflow are possible. For example, there could be a "no-timeout mode," where the user can take as much time as they need to perform an action; a "timeout mode," where the user has a limited time to perform an action (if the action times out, the loop can proceed to the next action); a "timeout duration reduction mode," where the user has increasingly less time to perform an action; and an "infinite loop mode," where the timeout duration decreases and the loop continues until the user's action fails. Furthermore, to increase the level of difficulty, the prompted action could be an atomic action (e.g., pedal press), followed by a "routine" action if identified from typical clinical use (e.g., grasping after moving a camera), or a sequence of actions (e.g., engagement).

[0072] While training can be performed in a 3D simulation environment, other options are possible. For example, the GUI could display an image of the surgeon's bridge 100 to provide cues to the user, or it could display a series of actions for the user to anticipate the next move. In other implementations, no visual cues are provided.

[0073] As mentioned above, user performance can be evaluated based on criteria. Such criteria may include, but are not limited to, the number of correctly executed actions, the number of incorrect actions, the number of timeouts, the reaction time for each action, and performance compared to peer trainees or benchmarks.

[0074] The list / order of actions to be performed in the simulation can be completely random or semi-random (to cover the entire surgeon bridge 100), or simple data analysis can be used to identify actions that are difficult for the user to perform (e.g., those with longer reaction times or that lead to incorrect actions), and these actions can be repeatedly trained when the user trains again.

[0075] Training on understanding the workspace of ungrounded input devices

[0076] Another example of simulation is training to understand the workspace of ungrounded input devices. Simulators for this type of training may or may not be integrated into the Surgeon Bridge 100. For a surgeon to successfully control the end effector instruments of a robotic arm during robotic surgery, he needs to learn how to use the input devices within the constraints of a specific robotic platform. As users transition between robotic platforms, it becomes increasingly important to enable surgeons to quickly and easily familiarize themselves with any system-specific constraints affecting their ability to control instruments.

[0077] In one implementation, some user input devices of the robotic surgical system 150 are ungrounded input devices (UIDs) held in the user's hand to move surgical instruments. These are "ungrounded" because there are no mechanical constraints on the user moving the device to any location. In contrast, "grounded" input devices can be on an arm with hard stops to prevent movement of the device beyond mechanical limits set by the arm. Ungrounded input devices can be tracked in space using an electromagnetic (EM) tracker. The movement of the UID is then translated into instrument movement via a robot control (RC) algorithm. The accuracy of EM tracking decreases if the UID is too far from or too close to the tracker. The user needs to keep his hand in the available workspace so that the system can track the UID accurately enough for teleoperation. The size and shape of this workspace are typically characterized by the tracker manufacturer and are referred to herein as the "EM (or UID) workspace".

[0078] In one implementation, the boundaries of the workspace are invisible, and system 150 does not physically prevent the user from moving the input device outside the workspace. This presents a system-specific constraint that the surgeon user must be familiar with in order to effectively control the robot. The size of the workspace does not allow the user to move the instrument throughout its reach and approach range without repositioning the user's hand, depending on the motion scaling. As used herein, "instrument reach and approach" refers to the space defined by the arm joint constraints, and "instrument workspace" refers to the space defined by the EM workspace.

[0079] In one implementation, the "grip" operation is used to reposition a hand within the workspace without moving the robotic arm. For example, in one specific implementation, the user presses the clutch pedal, which allows the user to move the UID without moving the robotic arm. In this way, the user can reposition their hand without moving the robotic arm. Once the user's hand is in the desired position, the user can release the clutch pedal, after which the movement of the UID will again move the robotic arm.

[0080] The following implementation scheme can be used to allow users to quickly familiarize themselves with system constraints, understand the concept of EM workspace boundaries, understand the impact of hand positioning on instrument control, adjust the tool workspace through gripping, and understand the impact of camera movement on the tool workspace (if the camera movement is controlled by hand movement).

[0081] In one implementation, if the user moves one or both input devices outside the EM workspace boundary, the robotic surgical system 150 will automatically disengage the user from active teleoperation. The user will then have to reposition their hand within the EM workspace and re-engage. This can disrupt or slow down surgical procedures. The concept of gripping and repositioning the hand to move instruments further is part of the teleoperation of the surgical robot in this implementation. Novice surgeons may struggle to understand how hand movements translate into instrument movements and how hand positioning affects the tool workspace. Similarly, when controlling endoscope movement with a handheld input device, new users may struggle to understand the impact of camera movement on the tool workspace. The camera's movement is opposite to the hand movement (moving back to zoom in); therefore, the user's hand matches the relative movement of the instrument with respect to the camera view, thereby automatically redefining the tool workspace (indirectly achieving the same effect as gripping).

[0082] To address the aforementioned issues, simulators can provide simulated training to allow surgeons to learn how to control user input devices within the system's available workspace and ergonomics. Training can be a 3D scene of the endoscope, with its camera view. The scene may include two instruments (e.g., one controlled by the left UID and the other by the right UID). The 3D scene can be empty (with some background for zooming) or it may have objects to interact with.

[0083] For simplicity, the accompanying figures are 2D, and the tool workspace is rectangular. However, it should be noted that in operation, the simulator can render the tool workspace as a parallelepiped. Also for simplicity, the motion scaling is 1:1 (i.e., the tool workspace is the same size as the EM workspace). For further simplification, some figures will show only a single UID and instrument.

[0084] Return to the attached image. Figure 7 This is an illustration of a simulation training implementation shown on a display device of the surgical bridge 100. (See diagram.) Figure 7 As shown, the simulation illustrates the left UID and its EM workspace, as well as the left tool and its reach. As the illustration shows, a given position of the UID within the EM workspace determines the corresponding instrument's workspace. If the user's hand is "aligned" with the tool, the tool's workspace is "mapped" into the EM workspace. This is in... Figure 7 As shown, the position of the left UID in the EM workspace is the same as or approximately the same as the position of the left tool in the left tool arrival box. In active telemetry, when the left UID is moved, the tool follows, and the tool workspace does not move. This is in... Figure 8 As shown in the figure, when the left UID is close to the right edge of the EM workspace, the left tool is close to the right edge of the tool reach range.

[0085] If the user expects to move the tool further to the right, they will encounter a problem because they are approaching the rightmost edge of the EM workspace. In this situation, the user will press the clutch pedal and move their hand to the left. With the clutch pedal depressed, moving the left UID will not move the tool, but it will indeed move the tool's workspace. This is in Figure 9 As shown in the diagram, this repositioning operation places the left UID in the bottom left corner of the EM workspace and the left tool in the bottom left corner of the tool arrival box. This allows the user to move the left UID and left tool further to the right.

[0086] In this implementation, when using two instruments, the relationship between the UID workspace and the tool workspace is independent for each tool. This is in Figure 10As shown in the diagram, the left and right UIDs can be moved or grabbed independently.

[0087] In addition to instruments, the UID can also be used to move the endoscopic camera to change the field of view. In one embodiment, moving the camera (e.g., by moving the UID while one of the pedals is depressed) simultaneously grasps two instruments. Therefore, in this embodiment, moving the camera has the same effect on the tool workspace as grasping and repositioning the UID, but also changes the surgeon's viewpoint. Because the camera moves in the opposite direction to the UID, the tool workspace moves with the camera, which has no effect on the camera unless the hands are separated or close together, but will change the tool workspace. This is illustrated in... Figure 11 middle.

[0088] There are many alternatives that can be used with these implementations. These alternatives include, but are not limited to, GUI indication of EM workspace boundaries, 3D visualization of the EM workspace and on-screen input devices, augmented reality (AR) glasses for visualizing the EM workspace boundaries around the user's hands, and transparent plastic training boxes attached to trackers to physically represent / materialize the EM workspace boundaries.

[0089] Other exemplary embodiments include the following. Exemplary embodiments for one type of claim (e.g., system, method, computer program, or computer-readable storage medium) may be provided for other types (e.g., system as a method). Exemplary embodiments for one set (e.g., exemplary embodiments 1 to 9) may be used in other sets.

[0090] Example 1. A user console for a robotic surgical system, the user console comprising: a plurality of user input devices; a processor configured to generate signals in response to user manipulation of the plurality of user input devices; a simulator providing simulation training; and a switch configured to operate in a clinical mode or a simulation mode; wherein, in the clinical mode, the switch is configured to establish a connection between the processor and a robotic arm controller of the robotic surgical system such that signals generated by the processor in response to user manipulation of the plurality of user input devices cause movement of the robotic arm of the robotic surgical system; and wherein, in the simulation mode, the switch is configured to establish a connection between the processor and the simulator such that signals generated by the processor in response to user manipulation of the plurality of user input devices are provided to the simulator for the simulation training, but do not cause movement of the robotic arm of the robotic surgical system.

[0091] Example 2. The robotic surgical system user console according to Example 1, wherein the simulator is integrated into the robotic surgical system user console.

[0092] Example 3. A user console for a robotic surgical system according to any one of example 1 to 2, wherein the simulation training teaches the psychomotor skills required in surgical procedures.

[0093] Exemplary Implementation Scheme 4. A user console for a robotic surgical system according to any one of Exemplary Implementation Schemes 1 to 3, wherein the simulation training enables a user to repeat atomic actions or action sequences using the plurality of user input devices to build psychomotor memory.

[0094] Example 5. A robotic surgical system user console according to any one of example 1 to 4, wherein, in the clinical mode, the switch is configured to establish a virtual local area network between the processor and the robotic arm controller, and wherein, in the simulation mode, the switch is configured to establish a virtual local area network between the processor and the simulator.

[0095] 6. A user console for a robotic surgical system according to any one of 1 to 5, wherein the simulator includes a simulator control computer and a simulator rendering computer.

[0096] Exemplary Implementation Scheme 7. The robotic surgical system user console according to Exemplary Implementation Scheme 6, wherein the switch is further configured to establish a virtual local area network between the simulator control computer and the simulator rendering computer.

[0097] Example 8. A user console for a robotic surgical system according to any one of example 1 to 7, wherein the simulator is configured to communicate with a network cloud.

[0098] Example 9. A robotic surgical system user console according to any one of example 1 to 8, wherein the switch is configured to establish the connection between the processor and the robotic arm controller via a control tower switch.

[0099] Example 10. A method for performing a simulation of a robotic surgical system, the method comprising: performing the following operations in a surgeon's bridge of a robotic surgical system, the robotic surgical system including a plurality of user input devices; a simulator providing simulation training; and a switch configured to operate in a clinical mode or a simulation mode: in response to receiving a command to operate in the clinical mode, establishing a connection with a robotic arm controller of the robotic surgical system using the switch such that signals generated in response to user manipulation of the plurality of user input devices cause movement of a robotic arm of the robotic surgical system; and in response to receiving a command to operate in the simulation mode, establishing a connection with the simulator using the switch such that signals generated in response to user manipulation of the plurality of user input devices are provided to the simulator for the simulation training, but do not cause movement of the robotic arm of the robotic surgical system.

[0100] Example 11. The method according to example 10, wherein the simulator is integrated into the user console of the robotic surgical system.

[0101] Exemplary Embodiment 12. The method according to any one of Exemplary Embodiments 10 to 11, wherein the simulation training teaches the psychomotor skills required in surgical procedures.

[0102] Exemplary Implementation Scheme 13. The method according to any one of Exemplary Implementation Schemes 10 to 12, wherein the simulation training causes a user to repeat atomic actions or action sequences using the plurality of user input devices to construct psychomotor memory.

[0103] Exemplary Implementation 14. The method according to any one of Exemplary Implementations 10 to 13, wherein the connection with the robot arm controller and the connection with the simulator are implemented using different virtual local area networks.

[0104] Example 15. A user console for a robotic surgical system, the user console comprising: a plurality of user input devices; means for allowing user manipulation of the plurality of user input devices to cause movement of the robotic arm of the robotic surgical system in a clinical mode; and means for preventing user manipulation of the plurality of user input devices from causing movement of the robotic arm of the robotic surgical system in a simulation mode.

[0105] Exemplary Implementation 16. The user console of the robotic surgical system according to Exemplary Implementation 15, wherein the permissioning component establishes a virtual local area network with the robotic arm controller.

[0106] Example 17. A user console for a robotic surgical system according to any one of example 15 to 16, wherein the component for preventing [something] establishes a virtual local area network with the simulator but not with the robotic arm controller.

[0107] Example 18. The robotic surgical system user console according to any one of example 15 to 17 further includes components for providing simulation training that teaches psychomotor skills required in surgical procedures.

[0108] Exemplary Embodiment 19. The robotic surgical system user console according to any one of Exemplary Embodiments 15 to 18 further includes components for providing simulation training, the simulation training enabling a user to repeat atomic actions or action sequences using the plurality of user input devices to build psychomotor memory.

[0109] Example 20. The robotic surgical system user console according to any one of example embodiments 15 to 19 further includes: a simulator, the simulator including a simulator control computer and a simulator rendering computer; and components for establishing a virtual local area network between the simulator control computer and the simulator rendering computer.

[0110] Example 21. A user console for a robotic surgical system, the user console comprising: a user input device; an electromagnetic tracker configured to track the position of the user input device; a display device; and a processor configured to provide simulation training, the simulation training displaying on the display device: a representation of an electromagnetic workspace; an indication of the position of the user input device in the electromagnetic workspace; a representation of a tool workspace; and an indication of the position of a tool in the tool workspace; wherein movement of the user input device causes movement of the indication of the position of the user input device in the representation of the electromagnetic workspace and the indication of the position of the tool in the representation of the tool workspace.

[0111] Exemplary Embodiment 22. The user console of the robotic surgical system according to Exemplary Embodiment 21 further includes a clutch pedal, wherein in response to the clutch pedal being pressed, movement of the user input device causes movement of the indication of the position of the user input device in the representation of the electromagnetic workspace, but does not cause movement of the indication of the position of the tool in the representation of the tool workspace.

[0112] Exemplary Embodiment 23. The user console of the robotic surgical system according to Exemplary Embodiment 22, wherein in response to the release of the clutch pedal, movement of the user input device causes movement of the indication of the position of the user input device in the representation of the electromagnetic workspace, and causes movement of the indication of the position of the tool in the representation of the tool workspace.

[0113] Exemplary Embodiment 24. A user console for a robotic surgical system according to any one of Exemplary Embodiments 21 to 23, wherein the representation of the electromagnetic workspace and the representation of the tool workspace are displayed in three dimensions.

[0114] Example 25. The robotic surgical system user console according to any one of example 21 to 24 further includes a second user input device, wherein the relationship between the electromagnetic workspace and the tool workspace of the user input device is independent of the relationship between the electromagnetic workspace and the tool workspace of the second user input device.

[0115] Exemplary Embodiment 26. A user console for a robotic surgical system according to any one of Exemplary Embodiments 21 to 25, wherein moving the camera of the robotic surgical system to change the field of view also causes a change in the indication of the position of the user input device in the representation of the electromagnetic workspace.

[0116] Example 27. A user console for a robotic surgical system according to any one of example 21 to 26, wherein the user input device is not grounded.

[0117] Exemplary Embodiment 28. A method for performing a simulation of a robotic surgical system, the method comprising: performing the following operations in a surgeon's bridge of a robotic surgical system, the robotic surgical system including a user input device; an electromagnetic tracker configured to track the position of the user input device; and a display: displaying the simulation on the display device, wherein the simulation displays a representation of an electromagnetic workspace, an indication of the position of the user input device in the representation of the electromagnetic workspace, a representation of a tool workspace, and an indication of the position of a tool in the representation of the tool workspace; and, in response to movement of the user input device, moving the indication of the position of the user input device in the representation of the electromagnetic workspace, and moving the indication of the position of the tool in the representation of the tool workspace.

[0118] Example 29. The method according to example 28 further includes, in response to the depressing of the clutch pedal and the movement of the user input device, moving the indication of the position of the user input device in the representation of the electromagnetic workspace, but not moving the indication of the position of the tool in the representation of the tool workspace.

[0119] Exemplary Embodiment 30. The method according to Exemplary Embodiment 29, wherein in response to the release of the clutch pedal and the movement of the user input device, the indication of the position of the user input device in the representation of the electromagnetic workspace and the indication of the position of the tool in the representation of the tool workspace are moved.

[0120] Exemplary Embodiment 31. The method according to any one of Exemplary Embodiments 28 to 30, wherein the representation of the electromagnetic workspace and the representation of the tool workspace are displayed in three dimensions.

[0121] Example 32. The method according to any one of example embodiments 28 to 31, wherein the relationship between the electromagnetic workspace of the user input device and the tool workspace is independent of the relationship between the electromagnetic workspace of the second user input device and the tool workspace.

[0122] Exemplary Embodiment 33. The method according to any one of Exemplary Embodiments 28 to 32 further includes, in response to moving the camera of the robotic surgical system to change the field of view, moving the indication of the position of the user input device in the representation of the electromagnetic workspace.

[0123] Exemplary Implementation 34. The method according to any one of Exemplary Implementations 28 to 33, wherein the user input device is not grounded.

[0124] Example 35. A user console for a robotic surgical system, the user console comprising: a display device; a user input device; an electromagnetic tracker configured to track the position of the user input device; and components for providing simulation training, the simulation training displaying on the display device: a representation of an electromagnetic workspace; an indication of the position of the user input device in the representation of the electromagnetic workspace; a representation of a tool workspace; and an indication of the position of a tool in the representation of the tool workspace; wherein movement of the user input device causes movement of the indication of the position of the user input device in the representation of the electromagnetic workspace, and causes movement of the indication of the position of the tool in the representation of the tool workspace.

[0125] Exemplary Embodiment 36. The user console of the robotic surgical system according to Exemplary Embodiment 35 further includes a clutch pedal, wherein in response to the clutch pedal being pressed, movement of the user input device causes movement of the indication of the position of the user input device in the representation of the electromagnetic workspace, but does not cause movement of the indication of the position of the tool in the representation of the tool workspace.

[0126] Exemplary Embodiment 37. A user console for a robotic surgical system according to any one of Exemplary Embodiments 35 to 36, wherein, in response to the release of the clutch pedal, movement of the user input device causes movement of the indication of the position of the user input device in the representation of the electromagnetic workspace, and causes movement of the indication of the position of the tool in the representation of the tool workspace.

[0127] Exemplary Embodiment 38. A user console for a robotic surgical system according to any one of exemplary embodiments 35 to 37, wherein the representation of the electromagnetic workspace and the representation of the tool workspace are displayed in three dimensions.

[0128] Example 39. The robotic surgical system user console according to any one of example 35 to 38 further includes a second user input device, wherein the relationship between the electromagnetic workspace and the tool workspace of the user input device is independent of the relationship between the electromagnetic workspace and the tool workspace of the second user input device.

[0129] Exemplary Embodiment 40. A user console for a robotic surgical system according to any one of exemplary embodiments 35 to 39, wherein moving the camera of the robotic surgical system to change the field of view also causes a change in the indication of the position of the user input device in the representation of the electromagnetic workspace.

[0130] For purposes of explanation, the foregoing description uses specific nomenclature to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that specific details are not required to practice the invention. The foregoing description of specific embodiments of the invention has been provided for illustrative and descriptive purposes. These are not intended to be exhaustive or to limit the invention to the specific forms disclosed; various modifications and alterations can be made to this disclosure in light of the foregoing teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, thereby enabling others skilled in the art to best utilize the invention and various embodiments with various modifications suitable for the contemplated particular uses. The following claims and their equivalents are intended to define the scope of the invention.

Claims

1. A user console for a robotic surgical system, the user console comprising: Multiple user input devices; A processor configured to generate signals in response to user manipulation of the plurality of user input devices; A simulator configured to provide simulated training; and A switch configured to operate in clinical or simulation mode; In the clinical mode, the switch is configured to establish a connection between the processor and the robotic arm controller of the robotic surgical system, such that signals generated by the processor in response to user input from the plurality of user input devices cause movement of the robotic arm of the robotic surgical system; and In the simulation mode, the switch is configured to establish a connection between the processor and the simulator, such that signals generated by the processor in response to user manipulation of the plurality of user input devices are provided to the simulator for the simulation training, but without causing movement of the robotic arm of the robotic surgical system.

2. The robotic surgical system user console according to claim 1, wherein the simulator is integrated into the robotic surgical system user console.

3. The user console of the robotic surgical system according to claim 1, wherein the simulation training teaches the psychomotor skills required in surgical procedures.

4. The user console of the robotic surgical system according to claim 1, wherein the simulation training enables the user to repeat atomic actions or action sequences using the plurality of user input devices to build psychomotor memory.

5. The user console for the robotic surgical system according to claim 1, wherein in the clinical mode, the switch is configured to establish a virtual local area network between the processor and the robotic arm controller, and wherein, In the simulation mode, the switch is configured to establish a virtual local area network between the processor and the simulator.

6. The user console of the robotic surgical system according to claim 1, wherein the simulator includes a simulator control computer and a simulator rendering computer.

7. The robotic surgical system user console of claim 6, wherein the switch is further configured to establish a virtual local area network between the simulator control computer and the simulator rendering computer.

8. The user console for the robotic surgical system according to claim 1, wherein the simulator is configured to communicate with a network cloud.

9. The user console of the robotic surgical system of claim 1, wherein the switch is configured to establish the connection between the processor and the robotic arm controller via a control tower switch.

10. A method for performing a robotic surgical system simulation, the method comprising: Perform the following operations in the surgeon's bridge of a robotic surgical system, which includes multiple user input devices; A simulator configured to provide simulated training; and a switch configured to operate in clinical mode or simulation mode: In response to receiving a command to operate in the clinical mode, the switch is used to establish a connection with the robotic arm controller of the robotic surgical system, such that signals generated in response to user manipulation of the plurality of user input devices cause movement of the robotic arm of the robotic surgical system. as well as In response to receiving a command to operate in the simulation mode, the switch is used to establish a connection with the simulator, such that signals generated in response to user manipulation of the plurality of user input devices are provided to the simulator for the simulation training, without causing movement of the robotic arm of the robotic surgical system.

11. The method of claim 10, wherein the simulator is integrated into the user console of the robotic surgical system.

12. The method of claim 10, wherein the simulation training teaches the psychomotor skills required in surgical procedures.

13. The method of claim 10, wherein the simulation training causes a user to repeat atomic actions or action sequences using the plurality of user input devices to construct psychomotor memory.

14. The method of claim 10, wherein the connection to the robot arm controller and the connection to the simulator are implemented using different virtual local area networks.

15. A user console for a robotic surgical system, the user console comprising: Multiple user input devices; Components for allowing user manipulation of the multiple user input devices to cause movement of the robotic arm of the robotic surgical system when in clinical mode; and A component for preventing movement of the robotic arm of the robotic surgical system caused by user manipulation of the multiple user input devices when in analog mode.

16. The user console of the robotic surgical system according to claim 15, wherein the permissioning component establishes a virtual local area network with the robotic arm controller.

17. The user console of the robotic surgical system according to claim 15, wherein the component for preventing [something] establishes a virtual local area network with the simulator but not with the robotic arm controller.

18. The user console of the robotic surgical system of claim 15, further comprising components for providing simulation training that teaches psychomotor skills required in surgical procedures.

19. The user console of the robotic surgical system of claim 15, further comprising components for providing simulation training, the simulation training enabling a user to repeat atomic actions or sequences of actions using the plurality of user input devices to build psychomotor memory.

20. The user console of the robotic surgical system according to claim 15, further comprising: The simulator includes a simulator control computer and a simulator rendering computer; and Components for establishing a virtual local area network between the simulator control computer and the simulator rendering computer.

Images