Surgical robotic system user interface

Abstract

A surgical robotic system includes a surgical table, a plurality of movable carts oriented toward the surgical table, each of the plurality of movable carts including a robotic arm and an alignment unit configured to determine an orientation of the movable cart and the robotic arm relative to the surgical table, and a computer coupled to each of the plurality of movable carts and configured to calculate a yaw angle of each of the plurality of movable carts.

Description

BACKGROUND 1. TECHNICAL FIELD

[0002] The present disclosure relates generally to a surgical robotic system having one or more modular cart of arms, each of the one or more modular cart of arms supporting a robotic arm and a surgical console for controlling the cart and its respective arm. More particularly, the present disclosure relates to a system and method for registration of a modular cart of arms in a surgical robotic system with respect to a surgical table, height adjustment of controls of a surgical console, and a graphical user interface for displaying an orientation of an endoscope camera coupled to one of the robotic arms.

[0003] 2. BACKGROUND

[0004] Surgical robotic systems are currently used in minimally invasive medical procedures. Some surgical robotic systems include a surgical console that controls surgical robotic arms and surgical instruments having end effectors (e.g., clamping or grasping instruments) coupled to and actuated by the robotic arms. In operation, the robotic arms are moved into position over a patient, and the robotic arms then guide the surgical instruments into small incisions via surgical operation ports or natural orifices of the patient to position the end effectors at a work site within the patient.

[0005] Prior to utilizing the robotic arms, the robotic arms need to be oriented. Accordingly, a system is needed to properly orient the robotic arms, and a user interface is needed to indicate the status of the robotic arms to the operating room staff. Furthermore, an adjustable surgical console and a graphical user interface for displaying an orientation of an endoscope camera coupled to one of the robotic arms is needed. SUMMARY

[0006] According to one embodiment of the present disclosure, a surgical robotic system includes a surgical table, a plurality of movable carts oriented toward the surgical table, each of the plurality of movable carts including a robotic arm and an alignment unit configured to determine an orientation of the movable cart and the robotic arm with respect to the surgical table, and a computer coupled to each of the plurality of movable carts and configured to calculate a yaw angle of each of the plurality of movable carts.

[0007] According to one aspect of the above embodiment, each of the plurality of movable carts is aligned based on an alignment pattern projected onto a surface by the alignment unit. The computer is configured to set a status of each of the plurality of movable carts to an aligned status in response to confirmation from the alignment unit.

[0008] According to another aspect of the above embodiments, each of the plurality of movable carts includes a plurality of wheels and a plurality of brakes. Each of the plurality of movable carts includes a cart controller configured to identify a corresponding movable cart as registered in response to the plurality of brakes being engaged, the corresponding movable cart being aligned, and the robotic arm docked to the access port. The cart controller is configured to identify the corresponding movable cart as unregistered in response to at least one of the plurality of brakes having been disengaged or the robotic arm having undocked from the access port.

[0009] According to further aspects of the above embodiments, the computer is configured to output a user interface having a representation of the surgical table and a plurality of graphical representations of the plurality of movable carts. Each of the plurality of graphical representations displays the yaw angle. The computer is configured to determine whether two adjacent movable carts of the plurality of movable carts are spaced apart by a predetermined distance based on a difference between the yaw angles of the two adjacent movable carts.

[0010] According to another embodiment of the present disclosure, a method of aligning a robotic arm with a surgical table is disclosed. The method includes placing a plurality of movable carts around a surgical table, each of the plurality of movable carts including a robotic arm; projecting an alignment pattern from an alignment unit onto a surface, the alignment unit operatively coupled to a movable cart of the plurality of movable carts; and prompting a user to manipulate the alignment pattern by adjusting the alignment unit. The method further includes receiving an input indicating that the adjustment to the alignment unit has been completed; determining an orientation of the alignment pattern relative to a representative coordinate system; determining an orientation of each of the plurality of movable carts based on the determined orientation of the alignment pattern; and calculating, at a computer coupled to the plurality of movable carts, a yaw angle of each of the plurality of movable carts.

[0011] According to one aspect of the above embodiments, projecting the alignment pattern includes projecting at least two portions of the alignment pattern and is configured to indicate an alignment direction.

[0012] According to another aspect of the above embodiments, the method further includes activating an input device disposed on the alignment unit to confirm that the adjustment to the alignment unit has been completed. The method can further include setting a status of each of the plurality of movable carts to an aligned status in response to the confirmation from the alignment unit.

[0013] According to another aspect of the above embodiments, each of the plurality of movable carts includes a plurality of wheels and a plurality of brakes, and the method further includes identifying the movable cart as registered in response to the plurality of brakes being engaged, the movable cart of the plurality of movable carts being aligned, and the robotic arm docked to the access port. The method can further include identifying the movable cart as unregistered in response to at least one of the plurality of brakes having disengaged or the robotic arm having undocked from the access port.

[0014] According to further aspects of the above embodiments, the method further includes outputting a user interface having a representation of a surgical table and a plurality of graphical representations of the plurality of movable carts. The method can further include displaying the yaw angle with each of the plurality of graphical representations. The method can further include determining whether two adjacent movable carts of the plurality of movable carts are spaced apart by a predetermined distance based on a difference between the yaw angles of the two adjacent movable carts.

[0015] According to one embodiment of the present disclosure, a surgical robotic system is disclosed. The surgical robotic system includes a surgical table, a control tower including a first display, and a surgical console coupled to the control tower and including a second display. The surgical robotic system further includes a plurality of movable carts coupled to the control tower and configured to be controllable by the surgical console. Each of the movable carts includes a robotic arm having an instrument. The surgical robotic system further includes a user interface displayed on the first display and the second display. The user interface is configured to display an orientation of the movable carts and the robotic arms relative to the surgical table.

[0016] According to one aspect of the above embodiments, the user interface includes a graphical arm representation of each of the movable carts of the plurality of movable carts. One or more of a contour, a fill, or a color of the graphical arm representation can be used to designate a status of the movable cart. The graphical arm representation can include a yaw angle and a numerical identifier. The graphical arm representation can include an identifier designating a robotic arm having a camera.

[0017] According to another aspect of the above embodiments, the user interface displayed on the first display is configured to transition between a setup view and a surgical view. The user interface can include a plurality of views. One of the plurality of views can be a pre-setup view showing the surgical table without graphical arm representations.

[0018] According to another embodiment of the present disclosure, a method for graphical representation of a surgical robotic system is disclosed. The method includes displaying a first user interface on a first display coupled to a control tower of a plurality of movable carts oriented toward a surgical table. Each of the movable carts includes a robotic arm. The method further includes displaying a second user interface on a second display coupled to a surgical console of the control tower and the plurality of movable carts. The surgical console is configured to control each of the movable carts and the robotic arms, wherein each of the first user interface and the second user interface is configured to display an orientation of the movable carts and the robotic arms relative to the surgical table.

[0019] According to one aspect of the above embodiment, the method further includes displaying a graphical arm representation of each of the movable carts of the plurality of movable carts. The method can further include modifying at least one of a contour, a fill, or a color of the graphical arm representation to a status of the movable cart. The method can further include displaying a yaw angle and a numerical identifier as part of the graphical arm representation. The method can further include displaying a camera identifier designating a robotic arm having a camera as part of the graphical arm representation.

[0020] According to another aspect of the above embodiment, the method further includes transitioning between a settings view and a surgical view of the first user interface.

[0021] According to one embodiment of the present disclosure, a surgical robotic system is disclosed. The surgical robotic system includes a movable cart including a robotic arm having a camera and a surgical console coupled to the movable cart. The surgical console is configured to move the camera, the surgical console including a display configured to display a video feed from the camera and an orientation indicator displaying an orientation of the camera.

[0022] According to another embodiment of the present disclosure, a method for displaying an orientation of a camera in a surgical robotic system is disclosed. The method includes controlling movement of a camera of a robotic arm coupled to a movable cart by a surgical console. The method further includes displaying a video feed from the camera on a display of the surgical console and displaying an orientation indicator on the display of the surgical console, the orientation indicator displaying an orientation of the camera.

[0023] According to one aspect of the above two embodiments, the orientation indicator includes a rotation indicator and a pitch indicator. The rotation indicator includes an arrow that is rotatable within a bounded region to indicate rotation of the camera about a longitudinal axis defined by the camera. The pitch indicator displays an absolute value of a pitch of the camera. The pitch indicator can also include a line that bifurcates a bounded region into a lower portion and an upper portion. The absolute value of the pitch of the camera can be represented by a vertical movement of the line within the bounded region.

[0024] According to one embodiment of the present disclosure, a surgical robotic system is disclosed. The surgical robotic system includes a surgical table, a control tower including a first display, and a surgical console coupled to the control tower and including a second display. The surgical robotic system further includes a plurality of movable carts, each of the movable carts including a robotic arm and coupled to the control tower and configured to be controllable by the surgical console. The plurality of movable carts are oriented with the robotic arms facing the surgical table. The surgical robotic system further includes a user interface displayed on the first display and the second display. The user interface is configured to display an orientation of the movable carts and the robotic arms relative to the surgical table.

[0025] According to one aspect of the above embodiments, the user interface includes a graphical arm representation of each of the movable carts. At least one of a contour, a fill, or a color of the graphical arm representation can be used to designate a status of a movable cart. The graphical arm representation can include a yaw angle and a numerical identifier. The graphical arm representation can include a camera identifier designating a robotic arm having a camera.

[0026] According to another aspect of the above embodiments, the user interface displayed on the first display is configured to transition between a setup view and a surgical view. The user interface can include a plurality of views, where one of the plurality of views is a pre-setup view showing the surgical table without graphical arm representations.

[0027] According to further aspects of the above embodiments, the surgical system further includes a third display coupled to the surgical console, the third display configured to display an orientation of the movable carts and the robotic arms relative to the surgical table. The third display can be configured to display an identification number and an instrument of each of the robotic arms.

[0028] According to one embodiment of the present disclosure, a method for graphical representation of a surgical robotic system is disclosed. The method displays a first user interface on a first display coupled to a control tower of a plurality of movable carts, each of the movable carts including a robotic arm and oriented with the robotic arm facing a surgical table. The method also includes displaying a second user interface on a second display of a surgical console. The surgical console is coupled to the control tower and the plurality of movable carts and configured to control each of the movable carts and the robotic arms. Each of the first user interface and the second user interface is configured to display an orientation of the movable carts and the robotic arms relative to the surgical table.

[0029] According to one aspect of the above embodiment, the method further includes displaying a graphical arm representation of each of the movable carts of the plurality of movable carts. The method can also include modifying at least one of a contour, a fill, or a color of the graphical arm representation to reflect a status of the movable cart. The method can also include displaying a yaw angle and a numerical identifier as part of the graphical arm representation.

[0030] According to another aspect of the above embodiment, the method further includes displaying a camera identifier specifying a robotic arm having a camera as part of the graphical arm representation.

[0031] According to further aspects of the above embodiment, the method further includes transitioning between a setup view and a surgical view of the first user interface. The method can also include displaying a third user interface on a third display of the surgical console. The third user interface can be configured to display an orientation of the movable carts and the robotic arms relative to the surgical table. The third user interface can additionally or alternatively be configured to display an identification number and an instrument of each of the robotic arms.

[0032] According to one embodiment of the present disclosure, a surgical robotic system includes a movable cart including a robotic arm having a camera and a surgical console coupled to the movable cart. The surgical console is configured to move the camera. The surgical console further includes a display configured to display a video feed from the camera and an orientation indicator displaying an orientation of the camera.

[0033] According to one aspect of the above embodiment, the orientation indicator includes a rotation indicator and a pitch indicator. The rotation indicator includes an arrow rotatable within a bounded region, the arrow indicating a rotation of the camera about a longitudinal axis defined by the camera. The pitch indicator displays an absolute value of a pitch of the camera. The pitch indicator can also include a line bifurcating the bounded region into a lower portion and an upper portion. The absolute value of the pitch of the camera can be represented by a vertical movement of the line within the bounded region.

[0034] According to one embodiment of the present disclosure, a method for displaying an orientation of a camera in a surgical robotic system is disclosed. The method includes controlling movement of a camera coupled to a robotic arm of a movable cart by a surgical console. The method also includes displaying a video feed from the camera on a display of the surgical console and displaying an orientation indicator on the display of the surgical console, the orientation indicator displaying an orientation of the camera.

[0035] According to one aspect of the above embodiment, the orientation indicator includes a roll indicator and a pitch indicator. The roll indicator includes an arrow rotatable within a bounded region, the arrow configured to indicate a roll of the camera about a lateral axis defined by the camera. The pitch indicator displays an absolute value of a pitch of the camera. The pitch indicator includes a line bifurcating a bounded region into a lower portion and an upper portion. The absolute value of the pitch of the camera can be represented by a vertical movement of the line within the bounded region.

[0036] According to one embodiment of the present disclosure, a surgical robotic system is disclosed. The surgical robotic system includes a control tower and a plurality of movable carts coupled to the control tower, each of the movable carts including a robotic arm. At least one of the robotic arms has a camera. The surgical robotic system also includes a surgical console coupled to the control tower and configured to control each of the robotic arms and move the camera. The surgical console also includes a display configured to display a video feed from the camera and an orientation indicator displaying an orientation of the camera.

[0037] According to one aspect of the above embodiment, the orientation indicator includes a roll indicator and a pitch indicator. The roll indicator includes an arrow rotatable within a bounded region, the arrow configured to indicate a roll of the camera about a lateral axis defined by the camera. The pitch indicator displays an absolute value of a pitch of the camera. The pitch indicator also can include a line bifurcating a bounded region into a lower portion and an upper portion. The absolute value of the pitch of the camera can be represented by a vertical movement of the line within the bounded region.

[0038] According to one embodiment of the present disclosure, a surgical console for controlling a surgical robotic system is disclosed. The surgical console includes a pair of hand controller configured to control the surgical robotic system and an armrest configured to support an arm of a clinician during operation of the pair of hand controllers. The armrest is movable along a vertical axis. The surgical console further includes a plurality of foot pedals configured to control the surgical robotic system. The plurality of foot pedals is movable along a horizontal axis. The surgical console further includes a display movable along the vertical axis. The display is configured to display a view of a surgical site. The console further includes a user interface for adjusting at least one of a height of the display along the vertical axis, a height of the armrest along the vertical axis, or a depth of the foot pedals along the horizontal axis.

[0039] According to one aspect of the above embodiment, the user interface includes a clinician height adjustment input for inputting a clinician height. The surgical console can further include a computer configured to automatically calculate at least one of the height of the display along the vertical axis, the height of the armrest along the vertical axis, or the depth of the foot pedals along the horizontal axis based on the clinician height.

[0040] According to another aspect of the above embodiment, the user interface includes an armrest height adjustment input, a foot pedal depth adjustment input, and a display height adjustment input. Each of the armrest height adjustment input, the foot pedal depth adjustment input, and the display height adjustment input includes an up arrow and a down arrow configured to select a parameter. The up arrow or the down arrow can be disabled upon reaching a respective limit of the parameter.

[0041] According to one embodiment of the present disclosure, a method for adjusting a surgical console of a surgical robotic system is disclosed. The method includes inputting a clinician height through a user interface displayed on a display of the surgical console. The display is movable along a vertical axis. The surgical console further includes an armrest movable along a vertical axis and a plurality of foot pedals movable along a horizontal axis. The method further includes adjusting at least one of a height of the display along the vertical axis, a height of the armrest along the vertical axis, or a depth of the foot pedals along the horizontal axis based on the clinician height.

[0042] According to one aspect of the above embodiments, the method further includes displaying a handrail height adjustment input, a footswitch depth adjustment input, and a display height adjustment input on the user interface. The method further includes adjusting at least one of the handrail height adjustment input, the footswitch depth adjustment input, and the display height adjustment input. The method can further include adjusting at least one of the height of the display along the vertical axis, the height of the handrail along the vertical axis, or the depth of the footswitch along the horizontal axis based on which input is adjusted.

[0043] According to another aspect of the above embodiments, the method further includes, for each of the handrail height adjustment input, the footswitch depth adjustment input, and the display height adjustment input, displaying an up arrow and a down arrow configured to select a parameter. The method can further include disabling the up arrow or the down arrow upon reaching a respective limit of the parameter. BRIEF DESCRIPTION OF DRAWINGS

[0044] Various embodiments of the present disclosure are described herein with reference to the accompanying drawings, in which:

[0045] FIG. 1 is a schematic illustration of a surgical robotic system including a control tower, a control console, and one or more surgical robotic arms according to the present disclosure;

[0046] FIG. 2 is a perspective view of a surgical robotic arm of the surgical robotic system of FIG. 1

[0047] FIG. 3 is a perspective view of a setup arm of the surgical robotic arm of the surgical robotic system of FIG. 1

[0048] FIG. 4 is a perspective view of a setup arm and a robotic arm of the surgical robotic system of FIG. 1

[0049] FIG. 5 is a schematic illustration of a computer architecture of the surgical robotic system of FIG. 1

[0050] FIG. 6 is a schematic illustration of an alignment pattern of the surgical robotic system of FIG. 1

[0051] FIG. 7 is a block diagram illustrating components of an alignment unit according to the present disclosure;

[0052] FIG. 8 is a flowchart illustrating a method according to the present disclosure;

[0053] ​​​​​FIG. 9 is a schematic of a yaw angle of a robotic arm relative to a surgical table;

[0054] FIG. 10 is a graphical user interface showing a yaw angle of a robotic arm relative to a surgical table;

[0055] FIG. 11 is a graphical user interface displayed on a display of a control tower according to an embodiment of the present disclosure;

[0056] FIG. 12 is a graphical user interface displayed on a display of a control tower according to an embodiment of the present disclosure;

[0057] FIG. 13 is a graphical user interface displayed on a second display of a surgical console according to an embodiment of the present disclosure;

[0058] FIG. 14 is a graphical user interface showing a plurality of views during setup of a surgical robotic arm;

[0059] FIG. 15A to FIG. 15D is a graphical user interface representation of a rotation indicator for a camera according to an embodiment of the present disclosure;

[0060] FIG. 16A to FIG. 16D is a graphical user interface representation of a pitch indicator for a camera according to an embodiment of the present disclosure;

[0061] FIG. 17A to FIG. 17B is a graphical user interface representation of a combined orientation indicator having a rotation indicator and a pitch indicator according to an embodiment of the present disclosure; and

[0062] FIG. 18 is a graphical user interface for adjusting ergonomic parameters of a surgical console according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0063] Embodiments of the disclosed surgical robotic system are described in detail with reference to the drawings, wherein like reference numerals represent like elements throughout the several views. As used herein, the term “distal” refers to portions of the surgical robotic system and / or surgical instruments coupled to the patient, while the term “proximal” refers to portions further from the patient.

[0064] The term“application” can include a computer program designed to perform a function, task, or activity for the benefit of a user. For example, an application can refer to software running as a standalone program or locally or remotely in a web browser, or other software understood by those skilled in the art as an application. An application can run on a controller or user device, including for example on a mobile device, IOT device, or server system.

[0065] As will be described in detail below, the present disclosure relates to a surgical robotic system that includes a surgical console, a control tower, and one or more movable carts having surgical robotic arms coupled to setup arms. The surgical console receives user input through one or more interface devices that are interpreted by the control tower as movement commands for moving the surgical robotic arms. The surgical robotic arms include controllers configured to process the movement commands and generate torque commands for activating one or more actuators of the robotic arms, which in turn will move the robotic arms in response to the movement commands.

[0066] Reference FIG. 1 The surgical robotic system 10 includes a control tower 20 that is connected to all components of the surgical robotic system 10, including a surgical console 30 and one or more robotic arms 40. Each of the robotic arms 40 includes a surgical instrument 50 removably coupled thereto. Each of the robotic arms 40 is also coupled to a movable cart 60.

[0067] The surgical instrument 50 is configured for use during a minimally invasive surgical procedure. In embodiments, the surgical instrument 50 can be configured for open surgical procedures. In embodiments, the surgical instrument 50 can be an endoscope configured to provide a video feed to a user, such as an endoscope camera 51. In further embodiments, the surgical instrument 50 can be an electrosurgical forceps configured to seal tissue by compressing tissue between jaw members and applying electrosurgical current thereto. In yet another embodiment, the surgical instrument 50 can be a surgical stapler including a pair of jaws configured to grasp and clamp tissue while deploying a plurality of tissue fasteners (e.g., staples) and cutting the stapled tissue.

[0068] One of the robotic arms 40 can include a camera 51 configured to capture video of a surgical site. The surgical console 30 includes a first display 32 that displays a video feed of the surgical site provided by the camera 51 of the surgical instrument 50 disposed on the robotic arm 40 and a second display 34 that displays a user interface for controlling the surgical robotic system 10. The first display 32 and the second display 34 are touchscreens that allow for the display of various graphical user inputs.

[0069] The surgical console 30 also includes a plurality of user interface devices, such as a foot pedal 36 and a pair of hand controllers 38a and 38b used by the user to remotely control the robotic arms 40. The surgical console also includes an arm rest 33 for supporting the clinician’s arms while operating the hand controllers 38a and 38b.

[0070] The control tower 20 includes a display 23, which can be a touch screen, and outputs on a graphical user interface (GUI). The control tower 20 also serves as an interface between the surgical console 30 and the one or more robotic arms 40. Specifically, the control tower 20 is configured to control the robotic arms 40 to move the robotic arms 40 and corresponding surgical instruments 50, such as based on a set of programmable instructions and / or input commands from the surgical console 30, to cause the robotic arms 40 and surgical instruments 50 to perform a desired sequence of movements in response to input from the foot pedal 36 and hand controllers 38a and 38b.

[0071] Each of the control tower 20, surgical console 30, and robotic arms 40 includes a respective computer 21, 31, 41. The computers 21, 31, 41 are interconnected to one another using any suitable communication network based on wired or wireless communication protocols. As used herein, the term “network,” whether singular or plural, means a data network, including but not limited to the Internet, an intranet, a wide area network, or a local area network, and is not limited to the full extent of the definition of communication networks encompassed by the present disclosure. Suitable protocols include, but are not limited to, Transmission Control Protocol / Internet Protocol (TCP / IP), User Datagram Protocol / Internet Protocol (UDP / IP), and / or Datagram Congestion Control Protocol (DCCP). Wireless communication can be implemented via one or more wireless configurations, for example, radio frequency, optical, Wi-Fi, Bluetooth (an open wireless protocol for exchanging data over short distances from fixed devices and mobile devices, thus creating personal area networks (PANs)), ZigBee (a suite of high-level communication protocols using small, low-power digital radios based on the IEEE 122.15.4-2003 standard for wireless personal area networks (WPANs)). ZigBee (a suite of high-level communication protocols using small, low-power digital radios based on the IEEE 122.15.4-2003 standard for wireless personal area networks (WPANs)).

[0072] The computers 21, 31, 41 can include a suitable processor (not shown) operably connected to memory (not shown), which can include one or more of volatile, nonvolatile, magnetic, optical, or electrical media such as read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM (EEPROM), non-volatile RAM (NVRAM), or flash memory. The processor can be any suitable processor (e.g., control circuitry) adapted to perform the operations, calculations, and / or instructions described in this disclosure, including, but not limited to, a hardware processor, a field programmable gate array (FPGA), a digital signal processor (DSP), a central processing unit (CPU), a microprocessor, and combinations thereof. It will be understood by those skilled in the art that the processor can be replaced by any logic processor (e.g., control circuitry) adapted to perform the algorithms, calculations, and / or instructions described herein.

[0073] Referring to FIG. 2 Each robotic arm 40 can include a plurality of links 42a, 42b, 42c interconnected at joints 44a, 44b, 44c, respectively. Joint 44a is configured to secure robotic arm 40 to a movable cart 60 and defines a first longitudinal axis. Referring to FIG. 3 Movable cart 60 includes a riser 61 and a setup arm 62 that provides a base for mounting robotic arm 40. Riser 61 allows setup arm 62 to move vertically. Movable cart 60 also includes a display 69 for displaying information related to robotic arm 40.

[0074] Setup arm 62 includes a first link 62a, a second link 62b, and a third link 62c that provide lateral maneuverability of robotic arm 40. Links 62a, 62b, 62c are interconnected at joints 63a and 63b, each of which can include an actuator (not shown) for rotating links 62b and 62b relative to each other and link 62c. Specifically, links 62a, 62b, 62c can move in their respective lateral planes parallel to each other, thereby allowing robotic arm 40 to extend relative to a patient (e.g., a surgical table). In embodiments, robotic arm 40 can be coupled to a surgical table (not shown). Setup arm 62 includes a controller 65 for adjusting the movement of links 62a, 62b, 62c and riser 61.

[0075] The third link 62c includes a rotatable base 64 having two degrees of freedom. Specifically, the rotatable base 64 includes a first actuator 64a and a second actuator 64b. The first actuator 64a is rotatable about a first fixed arm axis that is perpendicular to a plane defined by the third link 62c, and the second actuator 64b is rotatable about a second fixed arm axis that is transverse to the first fixed arm axis. The first and second actuators 64a, 64b allow for full three-dimensional orientation of the robotic arm 40.

[0076] Referring to FIG. 2 , the robotic arm 40 also includes a holder 46 that defines a second longitudinal axis and is configured to receive an IDU 52 FIG. 1 ) that is configured to couple to an actuation mechanism and a camera 51 of a surgical instrument 50 and to move (e.g., rotate) and actuate the instrument 50 and / or the camera 51. The IDU 52 transmits actuation forces from its actuators to the surgical instrument 50 to actuate components (e.g., an end effector) of the surgical instrument 50. The holder 46 includes a sliding mechanism 46a that is configured to move the IDU 52 along the second longitudinal axis defined by the holder 46. The holder 46 also includes an engagement portion 46b that rotates the holder 46 relative to the link 42c.

[0077] The robotic arm 40 also includes a plurality of manual override buttons 53 disposed on the IDU 52 and on the set arm 62 that are usable in a manual mode. A user can press one or buttons 53 to move the component associated with the button 53.

[0078] The engagement portions 44a and 44b include actuators 48a and 48b that are configured to drive the engagement portions 44a, 44b, 44c relative to each other through a series of belts 45a and 45b or other mechanical linkages (such as drive rods, cables, or levers, etc.). Specifically, the actuator 48a is configured to rotate the robotic arm 40 about the longitudinal axis defined by the link 42a.

[0079] Actuator 48b of junction 44b is coupled to junction 44c via belt 45a, and junction 44c is in turn coupled to junction 46c via belt 45b. Junction 44c can include a transfer case that couples belts 45a and 45b, such that actuator 48b is configured to rotate each of links 42b, 42c and holder 46 relative to one another. More specifically, links 42b, 42c and holder 46 are passively coupled to actuator 48b, which enforces rotation about a pivot point “P” that is at the intersection of a first axis defined by link 42a and a second axis defined by holder 46. Thus, actuator 48b controls the angle Θ between the first and second axes, allowing for orientation of surgical instrument 50. As links 42a, 42b, 42c and holder 46 are interconnected via belts 45a and 45b, the angle between links 42a, 42b, 42c and holder 46 is also adjusted in order to achieve the desired angle Θ. In embodiments, some or all of junctions 44a, 44b, 44c can include actuators to eliminate the need for mechanical linkages.

[0080] With reference to FIG. 4 Each of the computers 21, 31, 41 of the surgical robotic system 10 can include a plurality of controllers that can be embodied in hardware and / or software. The computer 21 of the control tower 20 includes a controller 21a and a safety observer 21b. The controller 21a receives data from the computer 31 of the surgical console 30 regarding the current positions and / or orientations of the hand controllers 38a and 38b and the states of the footswitch 36 and other buttons. The controller 21a processes these input positions to determine the desired drive commands for each junction of the robotic arm 40 and / or the IDU 52, and transmits these commands to the computer 41 of the robotic arm 40. The controller 21a also receives the actual junction angles and uses this information to determine force feedback commands that are transmitted back to the computer 31 of the surgical console 30 to provide haptic feedback through the hand controllers 38a and 38b. The safety observer 21b performs validity checks on the data coming into and out of the controller 21a, and if an error in data transmission is detected, notifies a system fault handler to place the computer 21 and / or the surgical robotic system 10 into a safe state.

[0081] The computer 41 includes multiple controllers, namely a master cart controller 41a, a setup arm controller 41b, a robot arm controller 41c, and an instrument drive unit (IDU) controller 41d. The master cart controller 41a receives and processes joint commands from the controller 21a of the computer 21 and communicates these commands to the setup arm controller 41b, the robot arm controller 41c, and the IDU controller 41d. The master cart controller 41a also manages instrument exchange and the overall state of the movable cart 60, the robot arm 40, and the IDU 52. The master cart controller 41a also communicates actual joint angles back to the controller 21a.

[0082] The setup arm controller 41b controls each of the joints 63a and 63b, as well as the rotatable base 64 of the setup arm 62, and computes desired motor movement commands (e.g., motor torques) for the pitch axis and controls brakes. The robot arm controller 41c controls each joint 44a and 44b of the robot arm 40 and computes desired motor torques needed for gravity compensation, friction compensation, and closed loop position control of the robot arm 40. The robot arm controller 41c computes movement commands based on the computed torques. The computed motor commands are then communicated to one or more of the actuators 48a and 48b in the robot arm 40. Actual joint positions are then transmitted back to the robot arm controller 41c through the actuators 48a and 48b.

[0083] The IDU controller 41d receives desired joint angles of the surgical instrument 50, such as wrist and jaw angles, and computes desired currents for the motors in the IDU 52. The IDU controller 41d computes actual angles based on motor positions and transmits actual angles back to the master cart controller 41a.

[0084] The robot arm 40 is controlled as follows. First, the pose of the hand controllers (e.g., hand controllers 38a) controlling the robot arm 40 is converted to a desired pose of the robot arm 40 by a hand-eye transformation function performed by the master cart controller 21a. The hand-eye function, as well as other functions described herein, are embodied in software executable by the controller 21a or any other suitable controller described herein. The pose of one of the hand controllers 38a can be embodied as a coordinate position and roll-pitch-yaw (“RPY”) orientation relative to a coordinate frame fixed to the surgical console 30. The desired pose of the instrument 50 is relative to a fixed frame on the robot arm 40. The pose of the hand controller 38a is then scaled by a scaling function performed by the controller 21a. In embodiments, by the scaling function, the coordinate position is scaled down and the orientation is scaled up. Additionally, the controller 21a also performs a clutching function that disengages the hand controller 38a from the robot arm 40. Specifically, if certain movement limits or other thresholds are exceeded, the controller 21a stops transmitting movement commands from the hand controller 38a to the robot arm 40 and essentially acts like a virtual clutch mechanism, e.g., limiting mechanical input from affecting mechanical output.

[0085] The desired pose of the robot arm 40 is based on the pose of the hand controller 38a and is then passed through an inverse kinematics function performed by the controller 21a. The inverse kinematics function calculates the angles of the joints 44a, 44b, 44c of the robot arm 40 that achieve the scaled and adjusted pose input by the hand controller 38a. The calculated angles are then passed to the robot arm controller 41c, which includes a joint axis controller with a proportional-derivative (PD) controller, a friction estimator module, a gravity compensator module, and a double-sided saturation block configured to limit the commanded torques of the motors of the joints 44a, 44b, 44c.

[0086] With reference to FIG. 5 The robot arm 40 is coupled to a setup arm 300, which is substantially identical to the setup arm 62. The setup arm 300 is further mounted to the movable cart 60. The setup arm 300 includes a setup arm base 302 coupled to the movable cart 60. The setup arm 300 further includes a plurality of movable links coupled to one another by actuators (not shown), allowing the setup arm 300 to move into various configurations. In particular, the setup arm 300 includes a first setup link 304, a second setup link 306, and a coupling assembly 308. The coupling assembly 308 is configured to be coupled to the robot arm 40.

[0087] The setup arm base 302 is configured to fasten the setup arm 300 to a surgical table (not shown) or a movable cart 12. The first setup link 304 is rotatable 360° about an axis "A-A" at joint 310 relative to the setup arm base 302. The second setup link 306 is rotatable about an axis "B-B" at joint 312 relative to the first setup link 304. The coupling assembly 308 is rotatable about an axis "C-C" at joint 314 relative to the second setup link 306. The coupling assembly 308 is further rotatable about an axis "D-D" from about 0° to about 90°.

[0088] The setup arm 300 includes an alignment unit 316 coupled to the setup arm 300 and specifically to the joint 314. The alignment unit 316 is in operable communication with the control tower 20. In embodiments, the alignment unit 316 can be directly coupled to the coupling assembly 308. The alignment unit 316 is configured to determine the orientation of the setup arm 300 and the robotic arms 40 relative to a representative coordinate system 11 that is constructed by the computer 21 and used to place and orient each of the robotic arms 40 to the clinician's perspective, for example, by a camera and / or endoscope. Specifically, the alignment unit 316 is used to create a common reference alignment for the robotic arms 40 and determine the yaw orientation of the robotic arms 40 relative to the representative coordinate system 11. As used herein, the term "yaw" denotes movement of the robotic arms 40 about a longitudinal axis that is perpendicular to the ground.

[0089] The orientation of each link of the robotic arms 40 and each setup link of the setup arm 300 is used in calculations to align the movement of the robotic arms 40 with the movement of an input device at the surgical console 30, such as the manual input 18. The alignment unit 316 includes a light unit 412 (see FIG. 6 ) configured to project an alignment pattern 318 onto a horizontal surface. The alignable pattern 318 is projected onto a horizontal surface, such as a surgical table, floor, patient, or any other surface. The surface can not be perfectly horizontal as long as the alignment pattern 318 projected onto the surface is visible and discernible by the clinician or the computing device. Thus, any non-vertical surface can be used.

[0090] The alignment unit 316 has a rotatable body 320 that allows a user to manually rotate the alignment unit 316 and adjust the angle of the alignment pattern 318 in order to align the alignment pattern 318 with the coordinate system 11. In embodiments, the alignment unit 316 can include an indicator 316a, such as a printed mark or image on its surface to indicate a forward direction or a direction relative to the patient. In further embodiments, the alignment pattern 318 can be a line with a direction indication. In embodiments, the alignment pattern 318 can include a first portion 324 and a second portion 322. The second portion 322 of the alignment pattern 318 can indicate a forward direction or a portion of the surgical instrument 50 and the robotic arm 40 closest to the patient, and the second portion 322 can indicate a rearward direction or a portion of the surgical instrument 50 and the robotic arm 40 farthest from the patient. The second portion 322 and the first portion 324 can be visually distinct, such as different in color and / or pattern to allow for easier differentiation. In an exemplary embodiment, the second portion 322 can be green and the first portion 324 can be red. In embodiments, the second portion 322 can be blue and the first portion 324 can be yellow to allow for easier differentiation by colorblind individuals. In further embodiments, the second portion 322 and the first portion 324 can have different patterns, such as one of the first portion 324 or the second portion 322 can be solid and the other can be dashed.

[0091] With reference to FIG. 6 The surgical table 100 is shown with a patient "P" disposed thereon. FIG. 6Also shown are a plurality of alignment patterns 318a, 318b, 318c, 318d oriented with respect to the surgical table 100. The surgical table 100 can serve as a reference point for orienting the robotic arms 40 by aligning each of their respective alignment units 316. The reference point can be any object that remains stationary during alignment; such as the surgical table 100, the patient “P”, a wall, a mark on the floor, or even any of the other alignment patterns 318. The alignment units 316 of the four robotic arms 40 project alignment patterns 318a, 318b, 318c, 318d. The alignment pattern 318a is projected by the alignment unit 316 attached to the robotic arm 40 holding the camera and / or endoscope. When correctly oriented, the alignment patterns 318b, 318c, 318d are parallel to and face in the same direction as the alignment pattern 318a, which is projected from the robotic arm 40 holding the camera and / or endoscope as shown in patterns 402, 404, and 406. Pattern 408 shows misaligned alignment patterns 318a, 318b, 318c, 318d, with alignment pattern 318c being lateral with respect to alignment patterns 318a and 318b and alignment pattern 318d oriented in the opposite direction of alignment patterns 318a and 318b. While pattern 406 shows parallel aligned patterns 318a, 318b, 318c, 318d, the system 10 will correctly coordinate the surgeon controlled motion with respect to the camera view, it will produce an incorrect bed map graphic, as all arms will be shown 90 degrees from their actual yaw.

[0092] In embodiments, the alignment unit 316 includes an input device 326 disposed on the alignment unit 316, which can be a button or any other user interface device. The input device 326 can be actuated by a user to indicate to the control tower 20 and / or the surgical console 30 that an adjustment to the setup arm 300 and / or the alignment unit 316 has been completed. As FIG. 7As depicted in FIG. 3, alignment unit 316 includes light unit 412, sensor 414, and connector 416. Alignment unit 316 can also include a printed circuit board that incorporates various electronic components. Sensor 414 can be any suitable encoder, potentiometer, rotary variable differential transformer, or any other kind of rotary position sensor. In embodiments, light unit 412 projects a number of different alignment patterns 318 in one or more colors, including various shapes, numbers, letters, and / or symbols, to help identify the orientation and / or direction of alignment unit 316. Light unit 412 can include a light source, such as one or more light-emitting diodes, configurable to emit a laser, and an optional projection pattern or lens that shapes the emitted light into alignment patterns 318. Sensor 414 is used to determine the angle of alignment patterns 318. Sensor 414 can be configured to measure the rotation of alignment unit 316, which is then used to determine the orientation of robotic arm 40 relative to representative coordinate system 11. In particular, as alignment unit 316 is rotated by a user, sensor 414 determines the angle of alignment patterns 318 and correlates this angle with the position of robotic arm 40.

[0093] Connector 416 operably couples alignment unit 316 with computers 21, 31, and 41 of control tower 20, surgical console 30, and robotic arm 40, and allows data and information to be transmitted to and from alignment unit 316 and control tower 20, surgical console 30, and robotic arm 40. In embodiments, connector 416 can be a wired connection (e.g., USB), or connector 416 can include a wireless transmitter / receiver that communicates wirelessly with control tower 20 and / or surgical console 30, which can also include a wireless transmitter / receiver. The wireless communication can be radio frequency, optical, (Open Wireless Protocol for exchanging data across short distances using short length radio waves), and the like. Through connector 416, control tower 20 and / or surgical console 30 can transmit data and / or real-time data from alignment unit 316, and more specifically from sensor 414. Sensor 414 senses the orientation of alignment patterns 318 and sends data about the angle of alignment patterns 318 back to control tower 20 and / or surgical console 30. Control tower 20 or surgical console 30 utilizes this information to correlate movement of robotic arm 40 relative to representative coordinate system 11 with movement of an input device, such as manual input 18, from surgical console 30.

[0094] FIG. 8A flowchart 500 depicting an illustrative method for aligning a robotic arm 40 in alignment with a representative coordinate system 11. In practice, when the system is set up at step 502, the user is shown instructions for positioning a movable cart 60 proximate to the representative coordinate system 11, the movable cart including a setup arm 300, a robotic arm 40, and a surgical instrument 50. The user then adjusts the setup arm 300 by manipulating the setup links 304, 306 and coupling assembly 308 to align the setup arm 300 with the representative coordinate system 11. In an embodiment, the setup links 304, 306 can be manually adjusted by the user. In another embodiment, the setup links 304, 306 can include a plurality of actuators (not shown) configured to actuate the setup links 304, 306. The plurality of motors can be controlled by a control device (not shown) that can be operated by the user. If the robotic arm 40 is repositioned, the input device 326 is activated, or if the alignment unit 316 detects a change in yaw of the robotic arm 40, the user can be prompted to realign the robotic arm 40 with the representative coordinate system 11 according to the disclosed method. In step 504, after the user adjusts the setup arm 300 relative to the representative coordinate system 11, the alignment unit 316 is configured to project an alignment pattern 318 onto the representative coordinate system 11 by a light unit. The projected alignment pattern 318 can have a high intensity such that the alignment pattern 318 is visible to the user. The alignment unit 316 can be activated automatically once the movable cart 60 is stopped and cart brakes (not shown) are engaged.

[0095] In step 506, the user manipulates the alignment pattern 318 by adjusting the alignment unit 316. In particular, the user can rotate the alignment unit 316, which causes the alignment pattern 318 to also rotate. In embodiments, the alignment pattern 318 can be a straight line. In further embodiments, the light unit projects light of two or more colors to indicate orientation and / or direction. At step 508, once the user has finished adjusting the alignment unit 316, the user activates the input device 326 disposed on the alignment unit 316 to indicate to the control tower 20 and / or the surgical console 30 that the adjustment has been completed and that the setup arm 300 is properly aligned with the representative coordinate system 11. In embodiments, the movable cart 60 is docked or otherwise connected to the control tower 20. At step 510, the control tower 20 and / or the surgical console 30 determines the orientation of the alignment pattern 318 relative to the representative coordinate system 11. In particular, the alignment unit 316 includes a sensor (not shown) to determine the angle of the projected alignment pattern 318 relative to the position of the alignment unit 316. At step 512, based on the orientation of the alignment pattern 318 relative to the representative coordinate system 11, the control tower 20 and / or the surgical console 30 determines the position and orientation of the setup arm 300 and / or the robotic arm 40 relative to the representative coordinate system 11. At step 514, upon determining the orientation of the robotic arm 40, the control tower 20 and / or the surgical console 30 correlates movement and orientation of the robotic arm 40 relative to the representative coordinate system with movement of the manual input 18 configured to manipulate the robotic arm.

[0096] The surgical robotic system 10 according to the present disclosure is configured to perform a registration process to correlate (e.g., register) the orientation of each of the plurality of movable carts 60 and attached robotic arms 40 relative to a central point in space, such as the surgical table 100 FIG. 9 During the registration process, the system 10 determines the relative orientation of the robotic arms based on the yaw angle φ (in the horizontal plane) and the pitch angle θ FIG. 2 of the robotic arms 40. As described above, the yaw angle can be controlled by the robotic arms 40 and the setup arms 62, and the pitch angle (e.g., angle) can be controlled by the robotic arms 40. Registration of the yaw angle φ and the pitch angle θ of each of the robotic arms 40 ensures that handle movements of the hand controllers 38a and 38b from the surgical console 30 correctly map to natural movements of the surgical instrument 50 on the first display 32 (e.g., left movement of the hand controller 38a corresponds to left movement of the surgical instrument 50 on the first display 32). The system 10, and in particular the computer 21 of the control tower 20, is configured to execute an algorithm that correlates the yaw angle φ and the pitch angle θ based on the inputs from the alignment units 316 to determine the relative orientation of the robotic arms 40. In particular, the computer 21 of the control tower 20 is configured to execute an algorithm that correlates the yaw angle φ and the pitch angle θ based on the inputs from the alignment units 316 to determine the relative orientation of the robotic arms 40. FIG. 5 to FIG. 8and the pitch angle Θ based on the position of the junctions 44a, 44b, 44c. After the angles have been registered, the algorithm also determines when the robot arm 40 and the corresponding movable cart 60 have been registered and processes the event logic for controlling the alignment unit 316 and displaying the yaw angle φ on the arm cart display 69 FIG. 2 Additionally, notifications of registered and unregistered are sent to the control tower 20 and the surgical console 30 to indicate the registration status of each movable cart 60. The operating room staff also confirms registration prior to performing remote robot operations of the surgical robot system 10.

[0097] The master cart controller 41a is configured to perform the registration process and handle the setting of various registration states of the movable cart 60 and the robot arms 40. The master cart controller 41a is configured to set the movable cart 60 to a registered state when the following conditions are met: 1) one or more of the brakes 68 are activated to prevent movement of the movable cart 60; 2) the robot arms 40 attached to the movable cart 60 are aligned relative to the surgical table 100; and 3) the surgical instruments 50 of the robot arms 40 are coupled to access ports or trocars (not shown) inserted into the patient’s abdominal cavity.

[0098] Conversely, the master cart controller 41a is configured to set the movable cart 60 to an unregistered state when the following conditions are met: 1) one or more of the brakes 68 are deactivated to allow movement of the movable cart 60; and 2) the surgical instruments 50 of the robot arms 40 are decoupled from the ports or trocars.

[0099] The master cart controller 41a is also configured to set the movable cart 60 to an aligned state when the alignment unit 316 is aligned (as described above with respect to FIG. 5 to FIG. 8 ) and the input device 326 is activated. In the aligned state, the alignment unit 316 is deactivated and ceases to emit the alignment pattern 318.

[0100] The controller 21a coordinates communication between the operating room team interface (ORTI) and the master cart controller 41a of the movable cart 60. The ORTI is displayed on the display 23 of the control tower 20 as well as the second display 34. The controller 21a is also configured to confirm that each of the robot arms 40 is registered prior to enabling remote operation of the robot arms 40 and is further configured to determine when two adjacent robot arms 40 are too close to each other based on the registered angles. The controller 21a receives the registration status of each robot arm 40 and publishes data to the master cart controller 41a of each of the movable cart 60 and the ORTI indicating which robot arms 40 have been confirmed by the user and issues a warning if the robot arms 40 are placed too close together.

[0101] The controller 21a determines when the movable cart 60 and the robot arm 40 have registered, calculates the yaw angle φ of the registration, and processes event logic for controlling the alignment unit 316 and displaying the registered yaw angle on the cart display 69 of the movable cart 60. In addition, notifications of registered and unregistered are sent to indicate the registration status of the movable cart 60 and the robot arm 40. The registration of the movable cart 60 and the robot arm 40 is confirmed by the operating room staff prior to teleoperation.

[0102] FIG. 9 A schematic view of the system 10 and in particular the movable cart 60 and the robot arm 40 is shown as represented by the controller 21a for storing the yaw angle φ of each of the robot arms 40 (e.g., the longitudinal axis of the first link 42a of the robot arm 40) relative to the surgical table 100. Although FIG. 9 In the present embodiment only one set of movable cart 60 and robot arm 40 is shown, but multiple movable carts 60 and corresponding robot arms 40 can be used. FIG. 9 A circular scale 102 with a scale from 0° to 360° is shown, which is oriented with the top of the surgical table 100. In the present embodiment, the circular scale 102 is shown as a circle with a diameter of 10 cm. The scale is shown in the center of the circular scale 102. FIG. 9 In the present embodiment, the robot arm 40 is shown with a yaw angle φ of about 60°.

[0103] The circular scale 102 and the alignment angle shown thereon follow the right-hand rule (e.g., counterclockwise) and is defined based on the angle from the alignment pattern 318 to the first link 42a of the robot arm 40. The angle is zero when the second portion 322 of the alignment pattern 318 is aligned in the forward direction with the longitudinal axis defined by the first link 42a. Conversely, the alignment angle is defined clockwise for the system setup and user interface 110 FIG. 10 ). The angle is zero when the second portion 322 is aligned in the reverse direction of the first link 42a of the robot arm 40.

[0104] The yaw angle is determined by converting the original alignment angle of the alignment pattern 318 relative to the surgical table 100 to a converted alignment angle using the following equation (I):

[0105] (I) alignment angle = mod(3 * π - original alignment angle, 2 * π)

[0106] In equation (I), the mod function is the modulo operation, which takes the remainder of the difference between 3 * π and the original alignment angle divided by 2 * π. The converted alignment angle is then used to calculate the yaw angle using equation (II):

[0107] (II) yaw angle = converted alignment angle - sum(current vector - initial vector)

[0108] In equation (II), the initial vector is a 3x1 vector of the initial set arm angles between the links 62a, 62b, 62c of the set arm 62 before alignment, and the current vector is a 3x1 vector corresponding to the set arm 62 in the post-alignment state. As the robotic arms 40 move after their alignment, the current vector is updated, thereby calculating a new yaw angle. As shown in FIG. 11, the yaw angle for each of the robotic arms 40 is displayed on a user interface 110 (e.g., a bed map). The user interface 110 can be displayed on the first display 32 of the control tower 20 and / or the cart display 69 of the movable cart 60. FIG. 10

[0109] When the movable cart 60 is initially transitioned to the aligned state along with the robotic arms 40, the yaw angle is equal to the alignment angle. As the set arm 62 moves during manual planar motion to position the robotic arms 40 relative to the surgical table 100, the rotational joints 63a and 63b rotate about their separate rotational axes that are perpendicular to the floor, thus each joint 63a and 63b additionally contributes to the rotation of the base joint 44a of the robotic arms 40.

[0110] Referring to FIG. 10 , the user interface 110 is part of the ORTI and includes a graphical arm representation 112 for each of the robotic arms 40. Each of the graphical arm representations 112 displays an arm identification number 114 and a registered yaw angle 116. Additionally, the graphical arm representations 112 are displayed in various colors and / or other indicators to indicate the state of the robotic arms 40.

[0111] During set-up, the initial default state of each of the robotic arms 40 and the movable cart is set to a disabled state until the brakes 68 have been engaged, at which time the master cart controller 41a transitions to an enabled state and enters an unaligned substate. Upon entering the unaligned substate, the laser intensity of the alignment unit 316 is set to high. Once the alignment pattern 318 is aligned with the surgical table 100 and the input device 326 (e.g., a laser button) is pressed by the user, the laser intensity is set to off and the master cart controller 41a transitions to an aligned state. Upon entering the aligned state, the raw alignment angle is converted using equation (I) described above. The converted angle is then used to calculate the yaw angle using equation (II).

[0112] ​When in the aligned state, the master cart controller 41a transitions to an unregistered substate in which the flag indicating when to display the yaw angle on the user interface 110 is set to true upon entry. When the robot arm 40 is docked, i.e., the surgical instrument 50 of the robot arm 40 is coupled to a port or trocar, the master cart controller 41a transitions to a registered state, the display yaw angle flag is set to false, and the registration flag is set to true. If at any time, in the aligned state, the manual mode of the movable cart 60 and the robot arm 40 is activated, the display yaw angle flag is set to true. Otherwise, if the manual mode is not activated and the master cart controller 41a is in the registered state, the display yaw angle is set to false. Each time the master cart controller 41a enters the registered state, the registration count value is incremented by one.

[0113] In summary, the robot arm 40 is registered if all of the following conditions are met: the brake 68 is on, the robot arm 40 is aligned with the surgical table 100, and the robot arm 40 is docked, e.g., the surgical instrument 50 of the robot arm 40 is coupled to a port or trocar. Conversely, the robot arm 40 can become unregistered if any of the following conditions are met: the robot arm 40 becomes undocked or the brake 68 has been released.

[0114] The controller 21a of the control tower 20 provides a notification when any robot arm 40 is registered too close to each other, e.g., spaced apart by a minimum distance, and the controller 21a handles the receipt of registration acknowledgments from the ORTI. If the difference between two yaw angles is less than a predetermined threshold, the controller 21a determines whether the two adjacent robot arms 40 are too close to each other. If the robot arms 40 are too close to each other, the user can fix the bedside configuration or ignore the notification.

[0115] The controller 21a sends a notification to the user specifying which two of the robot arms 40 are too close. The notification can be sent for robot arms 40 that are registered and not in manual mode. The controller 21a clears the notification for robot arms 40 that are unregistered, in manual plane, or no longer too close.

[0116] Referring to FIG. 11 and FIG. 12 , another embodiment of the user interface 120 includes similar elements of the user interface 110. Like the interface 110 of FIG. 10 , the user interface 120 includes a bed map 121 that shows a graphical arm representation 122 for each of the robot arms 40 disposed about the surgical table 100.

[0117] The bed map 121 allows the user to quickly identify the relationship of the arms 40 to the patient. The bed map 121 shows the placement of the arms 40 relative to the surgical table 100 and the patient. Each of the graphical arm representations 122 displays a cart identification number 124, i.e., 1-4,FIG. 11 The trolley identification number indicates the movable trolley 60, the robotic arm 40, and the setting arm 62. The user interface 120 displayed on the monitor 23 of the control tower 20 includes, for example: FIG. 11 The settings view shown and as FIG. 12 The surgical view shown. FIG. 11 The settings view shows a magnified view of the user interface 120, and FIG. 12 The surgical view is a minimized view of the user interface 120, so that the user interface 120 does not interfere with the surgical view from the camera 51. Additionally, with... FIG. 13 Similar to the user interface 150, the user interface 120 includes a top function area 125 that displays the status of each robotic arm in the robotic arm 40.

[0118] Three instances of bed diagram 121 are displayed during the use of system 10. During setup ( FIG. 11 ) and during surgery ( FIG. 12 The first instance is displayed to the operations team on monitor 23 of control tower 20. During setup, the user can touch arm setup guide button 127 to switch to... FIG. 11 The settings view, where bed diagram 121 is magnified. (See...) FIG. 13 As shown, the bed diagram 121 is also displayed on the second display 34 of the surgical console 30 as part of the surgeon-assisted user interface 150. The user interface allows clinicians to view the bed diagram 121, which includes a graphical arm representation 122 of each of the robotic arms 40, similar to the ribbon 125.

[0119] refer to FIG. 14 This shows the registration and assigned progress of each robotic arm in the robotic arm 40 during its use. The setup view is available on the display 23 of the control tower 20. FIG. 11 This progress will be displayed during the period. Specifically, FIG. 14 A progress view of the user interface 120 is shown. The first view 130 is a preset view that shows the surgical table 100 without the graphical arm display 122.

[0120] The intermediate setup view includes a second view 132, in which the first robotic arm 40 is connected to the control tower 20. Identification number 124 is assigned to a graphical arm representation 122, which is shown in a dashed outline because the robotic arm 40 has not yet been confirmed. Additionally, the yaw angle 126 of the graphical arm representation 122 is also shown. In the third view 134, the robotic arm 40 has been confirmed but is still not assigned. This is shown by the transition from a dashed outline to a solid-filled graphical arm representation 122.

[0121] The intermediate initialization view includes a fourth view 138 in which the graphical arm representation 122 transitions from a solid-filled representation to a solid-unfilled representation. This indicates that the robotic arm has been confirmed and assigned to a primary point. A fifth view 140 shows all of the robotic arms 40 that have been connected, registered, and assigned, in which each of the graphical arm representations 122a-122d is numbered 1-4, respectively. Additionally, the fourth graphical arm representation 122d includes a designation 123 for the camera holding the robotic arm 40, for example, a solid-filled circle designated using the number surrounding the arm.

[0122] The internal operations view includes a sixth view 142 with four graphical arm representations 122a-122d in which the active robotic arms 40 are designated using a first fill or color scheme, while the standby robotic arms 40 maintain the same designation as in the fifth view 140. The fill color (i.e., graphical arm representations 122a and 122b) showing the active robotic arms 40 can be a different color combination than the solid-filled representation of the third view 134 to avoid confusion. The graphical user interface 120 displayed on each of the displays 23, 32, and 34 can utilize the designations of FIG. 14 to show the status of each of the robotic arms 40.

[0123] Referring to FIG. 13 , in addition to the bed map 121, the surgeon assist user interface 150 shows a graphical representation 152a-152c of each of the three robotic arms 40. Each of the graphical representations 152a-152c includes an identification number 154a-154c and an instrument type 156a-156c, as only three robotic arms 40 are connected in the depicted scenario. The user interface 150 also includes an orientation indicator 160. The orientation indicator 160 shows a roll and pitch indication of the camera 51, which shows a roll and pitch indication of the camera 51. The camera 51 can be a stereo camera and provides a live video stream of the surgical site. The pitch and roll of the camera 51 are controlled by one of the hand controllers 38a or 38b. Thus, as the hand controller 38a or 38b is rotated about its longitudinal axis, moved vertically, and / or moved laterally, that motion is replicated by the camera 51 (i.e., by the robotic arm 40 and / or the IDU 52). Since the camera 51 is disposed within a confined surgical site, the orientation of the camera 51 can be confusing to the clinician viewing its feed on the display 23 or first display 32.

[0124] The orientation indicator 160 is a combination of a roll and pitch indicator. The roll indicator 161 of the orientation indicator 160 includes an arrow 162 disposed within a bounded area 164. As the camera 51 is rolled, the arrow 162 rotates within the bounded area 164, showing the roll of the camera 51. Referring to FIG. 15A to FIG. 15DAs the camera 51 is rotated, the view on the first display 32 of the surgical console 30 is also rotated. The rotation of the camera 51 is also shown by the arrow 162 on the orientation indicator 160. In FIG. 15A , the camera 51 is rotated in an upward orientation. In FIG. 15B , the camera 51 is rotated 90° clockwise from the position of FIG. 15A . In FIG. 15C , the camera 51 is rotated 90° clockwise from the position of FIG. 15B . In FIG. 15D , the camera 51 is rotated 90° clockwise from the position of FIG. 15C . As shown in FIG. 15A to FIG. 15D , the direction of the arrow 162 matches the view of the camera 51.

[0125] Referring to FIG. 16A to FIG. 16D , the pitch indicator 163 of the orientation indicator 160 includes a line 166 that bifurcates a bounded region 164. The pitch indicator 163 shows the absolute value of the pitch of the camera 51. The lower portion 166A below the line 166 represents the floor or ground, while the upper portion 166b above the line 166 represents the ceiling or sky. Thus, when the camera 51 is pitched horizontally as shown in FIG. 16A , the portions 166a and 166b are the same. The "absolute value" of the pitch indicates that the lower portion 166a is always shown at the bottom. Thus, when the camera 51 is pitched forward beyond 0°+10°, and when the camera is pitched backward beyond 0°-10° in the opposite direction (i.e., pitched at any position on the cone about its vertical axis), the amount of the lower portion 166a shown on the indicator is the same. As shown in FIG. 13 , FIG. 16A to FIG. 16D , and FIG. 17A to FIG. 17B , the roll indicator 161 and the pitch indicator 163 each form part of the orientation indicator 160.

[0126] Referring to FIG. 17A to FIG. 17B , the clinician "drags the workspace" when rotating the camera 51 and when translating the camera 51. In FIG. 17A , the clinician uses the hand controllers 38a and 38b to "turn the steering wheel counterclockwise" 15° and then lowers them to raise the view of the camera 51. The camera 51 is rotated clockwise (when viewed from behind the camera 51), but the direction of the rotation of the roll indicator and the image on the screen is the same as the direction of the clinician's hands. In FIG. 17B , the clinician turns the hand controllers 38a and 38b clockwise until the camera 51 has rotated 90° and then raises the hand controllers 38a and 38b to lower the endoscope view. The camera 51 itself is rotated counterclockwise, but the roll indicator 161 and the image on the screen follow the clinician's hands.

[0127] Referring to FIG. 13In addition to the bed diagram 121, the clinician-assisted user interface 150 shows additional elements. Specifically, the user interface 150 shows graphical representations 152a-152d of each of the four robotic arms 40. Each graphical representation 152a-152d includes identification numbers 154a-c and instrument types 156a-156c, since only three robotic arms 40 are connected in the depicted scenario. The user interface 150 also includes an orientation indicator 160. The orientation indicator 160 shows rotation and pitch indications for camera 51. Camera 51 may be a stereo camera and provides a real-time video stream of the surgical site. The pitch and rotation of camera 51 are controlled by one of the hand controllers 38a or 38b. Thus, as hand controller 38a or 38b rotates, moves vertically, and / or laterally about its longitudinal axis, this movement is replicated by camera 51 (i.e., by robotic arms 40 and / or IDU 52). Because the camera 51 is positioned within a restricted surgical site, its orientation may confuse clinicians observing its feed on the display 23 or the first display 32.

[0128] Orientation indicator 160 is a combination of rotation and pitch indicators. The rotation indicator 161 of orientation indicator 160 includes an arrow 162 disposed within a bounded region 164. Arrow 162 is rotatable from 0° to 360° such that as camera 51 rotates about its longitudinal axis, arrow 162 rotates within the bounded region 164, indicating the rotation of camera 51. (Reference) FIG. 15A to FIG. 15D As the camera 51 rotates, the view on the first display 32 of the surgical console 30 also rotates. The rotation of the camera 51 is also indicated by arrow 162 on the orientation indicator 160.

[0129] refer to FIG. 16A to FIG. 16D The pitch indicator 163 of the orientation indicator 160 includes a line 166 of a bifurcated bounded region 164. The pitch indicator 163 shows the absolute value of the pitch of the camera 51. The lower portion 166A below the line 166 represents the floor or ground, while the upper portion 166b above the line 166 represents the ceiling or sky. Therefore, when the camera 51 is in the orientation region, the pitch indicator 163 indicates the absolute value of the camera 51's pitch. FIG. 16A When the camera is tilted horizontally as shown, portions 166a and 166b are identical. The "absolute value" of the tilt indicates that the lower portion 166a is always displayed at the bottom. Therefore, when the camera 51 tilts forward by, for example, more than 0° + 10°, and when the camera 51 tilts backward in the opposite direction by, for example, more than 0° - 10° (i.e., tilting at any position on the cone about its vertical axis), the amount of the lower portion 166a shown on the indicator is the same. Therefore, FIG. 16B and FIG. 16DThe pitch indicator 163 shows the same amount (e.g., absolute value) of the lower portion 166a and the upper portion 166b, only the direction of the arrow 162 is different from the direction of the camera 51. As the camera 51 is tilted, the line 166 moves vertically within the bounded region 164. As shown in FIG. 13 、 FIG. 16A to FIG. 16D and FIG. 17A to FIG. 17B The rotation indicator 161 and the pitch indicator 163 each form part of the orientation indicator 160.

[0130] Referring to FIG. 17A to FIG. 17B , during use, the clinician can drag the workspace shown on the display 32 of the surgical console 30. The workspace is dragged while the camera 51 is both rotated and translated. In FIG. 17A , the clinician turns the hand controllers 38a and 38b counterclockwise 15° and, because the vertical axis control is reversed, also lowers the hand controllers 38a and 38b to raise the field of view of the camera 51. The camera 51 is rotated clockwise (when viewed from behind the camera 51), but the direction of rotation of the rotation indicator and the image on the screen is the same as the direction of the clinician’s hands. In FIG. 17B , the clinician turns the hand controllers 38a and 38b clockwise until the camera 51 has been rotated 90°, and then raises the hand controllers 38a and 38b to lower the endoscope field of view. The camera 51 itself is rotated counterclockwise, but the rotation indicator 161 and the image on the screen follow the clinician’s hands.

[0131] Referring to FIG. 1 , certain components of the surgical console 30 can be reconfigured to provide a customized ergonomic fit to the clinician. Specifically, the height of the first display 32, which provides the clinician with a 3D view from the camera 51 to the surgical site, is adjustable. Additionally, the height of the armrest 33 is also adjustable. Furthermore, the depth of the foot pedal 36 is adjustable. Any combination of modifying the ergonomic parameters, i.e., the height of the first display 32, the height of the armrest 33, and the depth of the foot pedal 36, allows the clinician to obtain a comfortable position. Each of the first display 32, the armrest 33, and the foot pedal 36 can be disposed on one or more tracks or other mechanisms that provide movement of the first display 32, the armrest 33, and the foot pedal 36 along their respective axes of movement. In embodiments, the first display 32 and the armrest 33 can move along a vertical axis, and the foot pedal 36 can move along a horizontal axis. In further embodiments, the first display 32, the armrest 33, and the foot pedal 36 can move along multiple axes and / or can rotate about a pivot point using a spherical joint, an arm, and other suitable mechanisms.

[0132] Each of the ergonomic parameters can be adjusted during initialization of the surgical console 30, e.g., after calibration of the surgical console 30. Referring to FIG. 18 The user interface 170 for configuring the surgical console 30 is displayed on the first display 32. The clinician can use the up arrow 172a and the down arrow 172b to input his / her height by the clinician height adjustment input 172, respectively. The computer 31 of the surgical console 30 includes a lookup table that the computer 31 uses to set each of the ergonomic parameters to recommended parameters from the lookup table based on the input height. Specifically, for a particular height range, the lookup table stores each of the ergonomic parameters, which the computer 31 then utilizes to automatically calculate and / or adjust one or more of the following ergonomic parameters: the height of the first display 32, the height of the armrest 33, and the depth of the footswitch 36. In embodiments, the user interface 170 can also include physical buttons (not shown) for adjusting each of the ergonomic parameters. These buttons can be provided on the armrest 33.

[0133] The clinician can also manually change each of the ergonomic parameters. The user interface 170 includes an armrest height adjustment input 174, a footswitch depth adjustment input 176, and a display height adjustment input 178. Each of the adjustment inputs 174, 176, 178 includes an up arrow 174a, 176a, 178a, a down arrow 174b, 176b, 178b, and a graphical representation 174c, 176c, 178c of the armrest 33, the footswitch 36, and the display 32, respectively. The clinician can adjust each of the adjustment inputs 174, 176, 178.

[0134] The ergonomic parameters can be represented on a 0-10 scale or any other suitable range. During adjustment, when the value reaches the lower or upper limit, the corresponding button for adjusting higher or lower (e.g., the up arrow 174a, 176a, 178a or the down arrow 174b, 176b, 178b) is grayed out or otherwise disabled, where any input (e.g., touch) has no effect on the ergonomic parameter. The user interface 170 also includes a next step button 180 and an adjustment button 182. The next step button 180 can be initially touchable if the clinician does not adjust any settings. Once the clinician adjusts a setting, the next step button 180 becomes disabled and the clinician engages the adjustment button 182 so that the surgical console 30 can make the inputted adjustment. Similarly, if the computer 31 is used to automatically calculate the ergonomic parameters, pressing the adjustment button 182 initiates the automatic adjustment of the surgical console 30 rather than the computer 31.

[0135] It should be understood that various modifications can be made to the disclosed embodiments. In embodiments, the sensor can be disposed on any suitable portion of the robotic arm. Accordingly, the above description should not be construed as limiting, but merely as illustrative of various embodiments. Those skilled in the art will be able to devise many other modifications that fall within the scope and spirit of the claims appended hereto.

Claims

1. A surgical robotic system, the surgical robotic system comprising: a surgical table; a plurality of movable carts oriented toward the surgical table, each of the plurality of movable carts including a robotic arm and an alignment unit configured to determine an orientation of each of the robotic arms relative to the surgical table; and a computer coupled to each of the plurality of movable carts and configured to calculate a yaw angle of each of the robotic arms; wherein the computer is configured to output a user interface having a surgical table representation and a graphical arm representation of each of the robotic arms, wherein the graphical arm representation of each of the robotic arms displays the calculated yaw angle.

2. The surgical robotic system of claim 1, wherein each of the robotic arms is aligned based on an alignment pattern projected onto a surface by the alignment unit.

3. The surgical robotic system of claim 2, wherein the computer is configured to set a status of each of the robotic arms to an aligned status in response to confirmation from the alignment unit.

4. The surgical robotic system of claim 3, wherein each of the plurality of movable carts includes a plurality of wheels and a plurality of brakes.

5. The surgical robotic system of claim 4, wherein each of the plurality of movable carts includes a cart controller configured to identify a corresponding movable cart as registered in response to the plurality of brakes being engaged, the corresponding movable cart being aligned, and the robotic arm being docked to an access port.

6. The surgical robotic system of claim 5, wherein the cart controller is configured to identify the corresponding movable cart as unregistered in response to at least one of the plurality of brakes having been disengaged or the robotic arm having undocked from the access port.

7. The surgical robotic system of claim 1, wherein the computer is configured to determine whether two adjacent robotic arms are spaced apart by a predetermined distance based on a difference between yaw angles of the two adjacent robotic arms.

8. A method of aligning robotic arms with a surgical table, the method comprising: positioning a plurality of movable carts around a surgical table, each of the plurality of movable carts including a robotic arm; projecting an alignment pattern from an alignment unit onto a surface, the alignment unit operatively coupled to a movable cart of the plurality of movable carts; prompting a user to manipulate the alignment pattern by adjusting the alignment unit; receiving an input indicating that adjustments to the alignment unit have been completed; determining an orientation of the alignment pattern relative to a representative coordinate system; determining an orientation of each of the robotic arms based on the determined orientation of the alignment pattern; calculating a yaw angle of each of the robotic arms at a computer coupled to the plurality of movable carts; ​ outputting, at the computer, a user interface having a representation of a surgical table and a graphical arm representation of each of the robotic arms; and displaying, with each of the graphical arm representations, the computed yaw angle.

9. The method of claim 8, wherein projecting the alignment pattern comprises projecting at least two portions of the alignment pattern and is configured to indicate an alignment direction.

10. The method of claim 8, further comprising activating an input device disposed on the alignment unit to confirm that adjustment to the alignment unit has been completed.

11. The method of claim 10, further comprising setting a status of each of the plurality of movable carts to an aligned status in response to confirmation from the alignment unit.

12. The method of claim 11, wherein each of the plurality of movable carts comprises a plurality of wheels and a plurality of brakes.

13. The method of claim 12, further comprising identifying the movable cart as registered in response to the plurality of brakes being engaged, the movable cart of the plurality of movable carts being aligned, and the robotic arm docked to an access port.

14. The method of claim 13, further comprising identifying the movable cart as unregistered in response to at least one of the plurality of brakes having been disengaged or the robotic arm having undocked from the access port.

15. The method of claim 8, further comprising determining whether two adjacent movable carts of the plurality of movable carts are spaced apart by a predetermined distance based on a difference between yaw angles of the two adjacent movable carts.

16. A surgical robotic system, the surgical robotic system comprising: a surgical table; a control tower comprising a first display; a surgical console coupled to the control tower and comprising a second display; a plurality of movable carts, each of the movable carts comprising a robotic arm and coupled to the control tower and configured to be controllable by the surgical console, the plurality of movable carts oriented with the robotic arms facing the surgical table; and a user interface displayed on the first display and the second display, the user interface configured to display an orientation of the movable carts and the robotic arms relative to the surgical table; wherein the user interface comprises a representation of a surgical table and a graphical arm representation of each of the robotic arms, wherein the graphical arm representation comprises a yaw angle.

17. The surgical robotic system of claim 16, wherein at least one of a contour, a fill, or a color of the graphical arm representation is used to designate a status of a movable cart.

18. The surgical robotic system of claim 16, wherein the graphical arm representation comprises a numerical identifier.

19. The surgical robotic system of claim 16, wherein the graphical arm representation comprises a camera identifier designating a robotic arm having a camera.

20. The surgical robotic system of claim 16, wherein the user interface displayed on the first display is configured to transition between a setup view and a surgical view.

21. The surgical robotic system of claim 16, wherein the user interface comprises a plurality of views.

22. The surgical robotic system of claim 21, wherein one of the plurality of views is a pre-setup view showing the surgical table without graphical arm representations.

23. The surgical robotic system of claim 16, further comprising a third display coupled to the surgical console, the third display configured to display an orientation of the movable carts and the robotic arms relative to the surgical table.

24. The surgical robotic system of claim 23, wherein the third display is configured to display an identification number and an instrument of each of the robotic arms.

25. A method for graphical representation of a surgical robotic system, the method comprising: displaying a first user interface on a first display coupled to a control tower of a plurality of movable carts, each of the movable carts comprising a robotic arm and oriented with the robotic arm facing a surgical table; displaying a second user interface on a second display of a surgical console coupled to the control tower and the plurality of movable carts and configured to control each of the movable carts and the robotic arms, wherein each of the first and second user interfaces are configured to display an orientation of the movable carts and the robotic arms relative to the surgical table; displaying a surgical table representation and a graphical arm representation of each of the robotic arms; and displaying a yaw angle as part of the graphical arm representation.

26. The method of claim 25, further comprising: modifying at least one of a contour, a fill, or a color of the graphical arm representation to a status of a movable cart.

27. The method of claim 25, further comprising: displaying a numerical identifier as part of the graphical arm representation.

28. The method of claim 25, further comprising: displaying a camera identifier designating a robotic arm having a camera as part of the graphical arm representation.

29. The method of claim 25, further comprising: transitioning between a setup view and a surgical view of the first user interface.

30. The method of claim 25, further comprising: displaying a third user interface on a third display of the surgical console.

31. The method of claim 30, wherein the third user interface is configured to display an orientation of the movable carts and the robotic arms relative to the surgical table.

32. The method of claim 31, wherein the third user interface is configured to display an identification number and an instrument of each of the robotic arms. ​

Images