Multi-arm surgical robot system for remote surgery

Abstract

The present invention relates to a multi-arm surgical robotic system (100) for telesurgery, comprising a local system (102), a remote system (104), a private network (N1), and a public network (N2). The local system (102) comprises a plurality of robotic arms (106a, 106b, 106c, 106d, 106e) connected to an endoscopic camera (C), each of the remaining robotic arms being connected to one of surgical robotic instruments (110, 112, 114, 116), a local surgeon's console (118) coupled to a master controller (136) that provides control inputs to the plurality of robotic arms (106a, 106b, 106c, 106d, 106e), and a conferencing system (138). The remote system 104 includes a remote surgeon console 140 coupled to a master controller 158 and a conferencing system 160 operably connected to the conferencing system 138 via a public network N2. Communication between the local surgeon console 118 and the remote surgeon console 140 over a private network N1 allows the remote surgeon 154 to control and monitor the robotic arms 106a, 106b, 106c, 106d, and 106e, enabling the remote surgeon 154 to remotely perform the robotic surgical procedure. In the event of a failure of the private network N1, control can be shifted from the remote surgeon console 140 to the local surgeon console 118, enabling control of the robotic surgical procedure.

Description

[Technical Field]

[0001] The present disclosure relates generally to multi-arm robotic surgical systems for minimally invasive surgery, and more particularly, the present disclosure relates to systems and methods for performing telesurgery in multi-arm robotic surgical systems. [Background technology]

[0002] This section is intended to introduce the reader to various aspects of technology that may be related to various aspects of the present disclosure, as described below. This disclosure is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light, and not merely as admissions of prior art.

[0003] Robotic-assisted surgery systems are being adopted worldwide to gradually replace traditional surgical procedures, such as open and laparoscopic procedures. Robotic-assisted surgery provides various benefits to patients during surgery and post-operative recovery. Robotic-assisted surgery similarly offers many advantages to surgeons in terms of improved ability to perform surgery precisely, reduced fatigue, and enhanced, clear, three-dimensional (3D) views of the surgical site. Furthermore, in robotic-assisted surgery, surgeons typically operate with hand controllers / master controllers / surgeon input devices / joysticks at a surgeon console system that seamlessly captures and communicates the complex movements performed by the surgeon, giving the perception that they are directly articulating surgical tools / instruments to perform the surgery.

[0004] A robotic-assisted surgery system can include multiple modular robotic arms that assist in performing robotic-assisted surgery. A surgeon uses a surgeon console to control the robotic arms and the instruments attached to them. The surgeon console includes a visualization system that enables the surgeon to perform the surgery. Additionally, hand controllers / master controllers / surgeon input devices are integrated with the surgeon console, which the surgeon manipulates to perform the surgery. The greatest challenge is requiring the presence of a skilled surgeon in the operating room, which may not be possible due to the surgeon's prior constraints or geographic distance. Telesurgery can solve this problem by allowing the surgeon operating on the surgeon console system to be located remotely from the surgical site or inside the operating room where the patient is being operated on.

[0005] Performing remote surgery creates new challenges. The main challenge is the unavailability of multi-arm robotic surgical systems to perform remote surgery. Another challenge is how to securely transmit the surgeon's commands from a remote location to the robotic surgical system without interruption while performing remote surgery. Furthermore, existing systems do not allow the local surgeon to take over and require conversion to open surgery in the event of a transmission network failure.

[0006] Yet another challenge is that information about the surgical instruments being utilized may not be known to the remote surgeon. Additionally, a 2D scan of the patient may not be available to the remote surgeon, who may want to review the patient's 2D scan before performing the surgery.

[0007] Also, sometimes an inexperienced local surgeon may want to have guidance or supervision from a more experienced remote surgeon. Furthermore, existing systems do not allow both the local and remote surgeons to jointly control the robotic arm.

[0008] In light of the aforementioned challenges, there is a need to provide a multi-arm robotic surgical system that solves the aforementioned problems associated with telesurgery. Summary of the Invention

[0009] It is proposed that some or all of the above-mentioned problems associated with performing telesurgery in a multi-arm robotic surgical system are addressed by certain embodiments of the present disclosure.

[0010] According to an aspect of the present invention, a multi-arm surgical robotic system for telesurgery includes a local system, a remote system, a private network, and a public network, the local system including a plurality of robotic arms arranged along an operating table, one of the robotic arms connected to an endoscopic camera and the remaining robotic arms each connected to one of the surgical robotic instruments; a left hand controller, a right hand controller, foot pedals, a two-dimensional (2D) touchscreen monitor, a three-dimensional (3D) HD monitor, and a head tracking camera for tracking head movements of a local surgeon wearing trackable glasses, each coupled to a master controller and connected to a master controller. the local surgeon console includes a master controller that provides control inputs received from the local surgeon to the plurality of robotic arms; and a conferencing system, wherein the remote system includes remote surgeon consoles including a left hand controller, a right hand controller, foot pedals, a two-dimensional (2D) touchscreen monitor, a three-dimensional (3D) HD monitor, and a head tracking camera for tracking head movements of a remote surgeon wearing tracked glasses, each coupled to a master controller, the master controllers being operably coupled to the master controller of the local surgeon console via a private network; and the conferencing system operably connected to the conferencing system via a public network;A system is disclosed, characterized in that the private network is configured to transmit control inputs provided by the remote surgeon to a master controller at the local surgeon console, transmit a real-time encrypted 3D video stream of the surgical site from the local surgeon console to a three-dimensional (3D) HD monitor at the remote surgeon console, retrieve DICOM images related to the patient from a hospital server connected to the private network and provide them to the master controller for display on a 2D display at the remote surgeon console and to the master controller for display on the 2D display at the local surgeon console, and transmit information regarding the surgical instruments and the status of each robotic arm to the master controller for display on the three-dimensional (3D) HD monitor, the public network is configured to facilitate bidirectional transmission of real-time audio and video data between a conference system at the remote surgeon console and a conference system at the local surgeon console, and communication between the local surgeon console and the remote surgeon console enables the remote surgeon to control and monitor the robotic arms, enables the remote surgeon to perform the robotic surgery remotely, can shift control from the remote surgeon console to the local surgeon console, and enables the local surgeon to assume control of the robotic surgical procedure in the event of a failure of the private network.

[0011] According to embodiments of the present invention, in a multi-arm surgical robotic system having at least five robotic arms, partial control from a remote surgeon console can be shifted to a local surgeon console, allowing the remote surgeon to operate any three of the five or more robotic arms, with the remaining arms being operated by the local surgeon acting as an assistant surgeon.

[0012] According to another embodiment of the present invention, the left hand controller, the right hand controller, and the foot pedals are configured to capture control inputs from a local surgeon.

[0013] According to yet another embodiment of the present invention, a two-dimensional (2D) touchscreen monitor coupled to the master controller is configured to serve as a graphical user interface for capturing control inputs from the local surgeon.

[0014] According to yet another embodiment of the present invention, a real-time 3D video stream of the surgical site is displayed on a three-dimensional (3D) HD monitor at the local surgeon console.

[0015] According to yet another embodiment of the present invention, the left hand controller, the right hand controller, and the foot pedal are configured to receive control inputs from a remote surgeon.

[0016] According to yet another embodiment of the present invention, a two-dimensional (2D) touchscreen monitor coupled to the master controller is configured for use as a graphical user interface for capturing control inputs from the remote surgeon.

[0017] According to yet another embodiment of the present invention, each conferencing system comprises a processor, a camera, a 2D monitor, a speaker, and a microphone.

[0018] According to yet another embodiment of the present invention, real-time two-way video data transmitted between the remote surgeon console conferencing system and the local surgeon console conferencing system can be displayed on separate 2D monitors at the remote location and in the local operating room.

[0019] According to yet another embodiment of the present invention, there is provided a method of telesurgery using a multi-arm surgical robotic system comprising a local system, a remote system, a private network, and a public network, the method comprising: establishing, using the private network, a communication link between the local system and the remote system, the local system including a plurality of robotic arms and a local surgeon console, and the remote system including a remote surgeon console; transmitting, using the private network, control inputs from the remote surgeon to the local system, the control inputs controlling the movement and operation of the plurality of robotic arms; transmitting, using the private network, a 3D image of the remote surgeon console. providing real-time encrypted 3D video of the surgical site from a local system to a remote system for visualization on an HD monitor using a private network; enabling real-time two-way audio and video communication between the local surgeon console and the remote surgeon console over a public network using a local surgeon console conferencing system and a remote surgeon console conferencing system to facilitate remote collaboration during a surgical procedure; transmitting DICOM images of the patient from a hospital server using a private network for display on the remote surgeon console's 2D touchscreen monitor and the local surgeon console's 2D touchscreen monitor; displaying information regarding the status of the robotic arm and surgical robotic instruments using a three-dimensional (3D) HD monitor of the local surgeon console and the remote surgeon console's three-dimensional (3D) HD monitor; enabling control of the robotic arm movement from a remote surgeon console, where the remote surgeon is located remotely from the patient, using control inputs from a master controller; shifting control from the remote surgeon console to the local surgeon console using the remote surgeon console, thereby enabling the local surgeon to assume direct control over the robotic arm movement during a surgical procedure in the event of a private network failure;performing a surgical procedure on a patient using the robotic arm based on control inputs from a remote surgeon, thereby enabling remote surgery;

[0020] According to yet another embodiment of the present invention, in a multi-arm surgical robotic system having at least five robotic arms, partial control from a remote surgeon console can be shifted to a local surgeon console, allowing the remote surgeon to operate any three of the multiple robotic arms, while the remaining arms can be operated by the local surgeon acting as an assistant surgeon.

[0021] Other embodiments, systems, methods, apparatus aspects, and features of the present invention will become apparent to those skilled in the art from the following detailed description, the accompanying drawings, and the appended claims.

[0022] The foregoing summary, as well as the following detailed description of the present disclosure, will be better understood when read in conjunction with the accompanying drawings. For the purpose of illustrating the present disclosure, there are shown in the drawings exemplary configurations of the present disclosure. However, the disclosure is not limited to the particular methods and instrumentalities disclosed herein. Moreover, those skilled in the art will appreciate that the drawings are not to scale. Wherever possible, like elements will be designated by like numerals. Embodiments of the present disclosure will now be described, by way of example only, with reference to the following figures: [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 illustrates an exemplary implementation of a multi-arm surgical robotic system for telesurgery using a remote surgeon console, according to an embodiment of the present disclosure. [Figure 2] FIG. 2 illustrates an exemplary implementation of a multi-arm teleoperated robotic surgical system held at a surgical site, according to an embodiment of the present disclosure. [Figure 3]FIG. 3 illustrates an implementation of a local surgeon console connected to a remote surgeon console via a private network, according to an embodiment of the present disclosure. [Figure 4] FIG. 4 illustrates the communication flow of control inputs from a remote surgeon console (140) to a local surgeon console (118) located in a local operating room, according to an embodiment of the present disclosure. [Figure 5a] FIG. 5a shows a block diagram illustrating communication of control inputs from a remote surgeon at a remote location to a local operating room according to an embodiment of the present disclosure. [Figure 5b] FIG. 5b shows a block diagram illustrating the communication of a video signal having a 3D endoscopic view of an actual surgical site from a local surgeon console to a remote surgeon console according to an embodiment of the present disclosure. [Figure 5c] FIG. 5c shows a block diagram illustrating bidirectional transmission of audio and video data between a conferencing system at a remote location and a local operating room, according to an embodiment of the present disclosure. [Figure 6] FIG. 6 illustrates steps for transmitting data from a remote surgeon console to a local surgeon console according to an embodiment of the present disclosure. [Figure 7] FIG. 7 shows a flow diagram of a method for performing remote surgery according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0024] For the purposes of promoting an understanding of the principles of the present disclosure, reference will now be made to the embodiments illustrated in the drawings and specific language will be used to describe the same. Nonetheless, it will be understood that no limitation of the scope of the disclosure is thereby intended, and that such changes and further modifications in the illustrated systems, and such further applications of the principles of the present disclosure as set forth therein, as would normally occur to one skilled in the art to which the present disclosure pertains, are contemplated.

[0025] It will be understood by those skilled in the art that the foregoing general description and the following detailed description are exemplary and explanatory of the present disclosure and are not intended to be limiting of the present disclosure. A common convention throughout this patent specification is that like reference numerals refer to like elements in the accompanying drawings.

[0026] Reference throughout this specification to "an embodiment," "another embodiment," "an implementation," "another implementation," or similar language means that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the present disclosure. Thus, throughout this specification, the appearances of "in an embodiment," "in another embodiment," "in one embodiment," "in another embodiment," and similar language may, but do not necessarily, all refer to the same embodiment.

[0027] The terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process or method comprising a list of steps does not include only those steps, but may also include other steps not expressly listed or inherent in such process or method. Similarly, one or more devices or subsystems or elements or structures preceding "comprises...a" does not, without further constraints, exclude the presence of other devices or subsystems or elements or structures or additional devices or subsystems or structures.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The devices, systems, and examples provided herein are illustrative only and are not intended to be limiting.

[0029] The use of the terms "a" and "an" herein does not denote a limitation of quantity, but rather denote the presence of at least one of the referenced item. Additionally, the terms sterile barrier and sterile adapter are synonymous and may be used interchangeably throughout the description.

[0030] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0031] 1 illustrates an exemplary implementation of a multi-arm surgical robotic system 100 for telesurgery in accordance with an embodiment of the present disclosure. The system 100 includes a local system 102, a remote system 104, a private network NI, and a public network NI.

[0032] FIG. 2 illustrates an exemplary implementation of a multi-arm teleoperated robotic surgical system maintained at a surgical site, having one or more features according to embodiments of the present disclosure. Specifically, FIG. 2 illustrates a local system with multiple robotic arms (106a), (106b), (106c), (106d), (106d), (106d), (106d), and (106e), each mounted on a robotic arm cart around a surgical table (108). As an exemplary embodiment, a configuration with five robotic arms (106a), (106b), (106c), (106d), (106d), and (106e) is depicted in FIG. 1. This depiction is for illustrative purposes only; the number of robotic arms may vary depending on the type of surgery. The five exemplary robotic arms (106a), (106b), (106c), (106d), (106d), and (106e) are positioned along the operating table (108) and may be positioned in different ways, including but not limited to, robotic arms (106a), (106b), (106c), (106d), (106d), and (106e) positioned along the operating table (108). The robotic arms 106a, 106b, 106c, 106d, 106d, and 106e may be separately mounted on five robotic arm carts, or the robotic arms 106a, 106b, 106c, 106d, 106d, and 106e may be mechanically and / or operatively connected to one another, or the robotic arms 106a, 106b, 106c, 106d, 106d, and 106e may be connected to a central body (not shown) such that the robotic arms 106a, 106b, 106c, 106d, 106d, and 106e branch out from the central body (not shown). One robotic arm of the plurality of robotic arms (106a, 106b, 106c, 106d, 106e) is connected to an endoscopic camera (C), and the remaining robotic arms are each connected to one of the surgical robotic instruments (110, 112, 114, 116). The local system further includes a local surgeon's console (118), a vision cart (VC), surgical instruments and accessory tables (not shown), and a conference system (138).

[0033] 3 illustrates an implementation of a local surgeon console connected to a remote surgeon console via a private network (N1) according to an embodiment of the present disclosure. The local surgeon console (118) includes a left hand controller (120), a right hand controller (122), foot pedals (124), a two-dimensional (2D) touchscreen monitor (126), a three-dimensional (3D) HD monitor (128), and head-tracking cameras (130) for tracking head movements of a local surgeon (132) wearing trackable glasses (134), each of which is coupled to a master controller (136), which provides control inputs to multiple robotic arms (106a, 106b, 106c, 106d, 106e). The local surgeon (132) controls the surgeon console (118). Additionally, a chair is provided for the local surgeon (132). The remote system (104) includes a remote surgeon console (140) and a conferencing system (160) (shown in FIG. 1). The remote surgeon console (140) includes a left hand controller (142), a right hand controller (144), foot pedals (146), a two-dimensional (2D) touchscreen monitor (148), a three-dimensional (3D) HD monitor (150), and head-tracking cameras (152) for tracking the head movements of a remote surgeon (154), each of whom wears trackable glasses (156) coupled to a master controller (158), which is operably coupled to the master controller (136) of the local surgeon console (118) via a private network (N1). Both the local surgeon console (118) and the remote surgeon console (140) are identical. The remote surgeon (154) may be located remotely from the actual operating room. The telesurgery uses a private network (N1) to communicate data between the server and the client, which is a secure network with end-to-end encryption and may be a P2P, MNLS, or leased line.The local surgeon console (118) can function as a slave surgeon console, and the remote surgeon console (140) functions as a master surgeon console. In this scenario, the remote surgeon (154) has complete control over the local surgeon console (118) (as shown in FIG. 1) to perform the remote surgery.

[0034] The remote surgeon console 140 can utilize a portable chair to accommodate the remote surgeon 154 during robotic telesurgery. Essential components for controlling the surgical robot are integrated into both the local surgeon console 118 and the remote surgeon console 140, ensuring a compact and easy-to-use design. The remote surgeon console 140 can be utilized by a skilled surgeon 154 seated remotely from the local operating room. In the event of a loss of electrical communication signal, the local surgeon 132 seated in the local operating room can assume control. The local surgeon console 118 serves as an interface for the remote surgeon console 140 within the local operating room. Both surgeons 132, 154 wear tracked 3D glasses 134, 156, respectively, for use with their respective surgeon consoles 118, 140. The surgeon's trackable 3D glasses 134, 156 are tracked by respective head tracking cameras 130, 152, which may be fixed to the respective 3D HD monitors 128, 150. Preferably, the head tracking cameras 130, 152 may be fixed to the top of the 3D HD monitors 128, 150. This is a safety feature to avoid inadvertent use of the multi-arm robotic surgical system 100 for telesurgery and unintended movement while the surgeon's attention is not focused on the 3D HD monitors 128, 150.

[0035] 4 illustrates the communication flow of control inputs from a remote surgeon console 140 to a local surgeon console 118 located in a local operating room, according to an embodiment of the present disclosure. A remote surgeon 154, seated at a remote location, performs a surgical procedure using hand controllers 142, 144. All control input data related to the remote surgeon's 154 movement of the hand controllers 142, 144, data from the various foot pedals 146, and any other sensor-based data for the robotic surgical instruments 110, 112, 114, 116, are collected and communicated via a serial port to the master controller 158 of the remote surgeon console 140. A start byte and a stop byte are appended to the received data. The complete control input data thus received is stored in a frame on the master controller 158 of the remote surgeon console 140. The remote surgeon console (140) acts as a client and sends data frames to the local surgeon console (118) over a transmission network, which is a private network (N1) using the TCP / IP communication protocol. The TCP / IP communication protocol is a connection-oriented protocol that ensures seamless data transmission between the client (remote surgeon console (140)) and the server (local surgeon console (118)). The local surgeon console (118) communicates with the patient-side arms (106a, 106b, 106c, 106d, 106e). The received data enables the movement of the different robotic surgical instruments (110, 112, 114, 116) used to perform the procedure at the surgical site.

[0036] Both the conferencing system (138) of the local surgeon console (118) and the conferencing system (160) of the remote surgeon console (140) may include a processor (not shown), a camera (not shown), a 2D monitor (not shown), a sound device such as a speaker (not shown), and a microphone (not shown). The local surgeon console (118) and the remote surgeon console (140) may feature an omnidirectional microphone and a wireless station. These components cooperate to enable audio and video interaction between the local surgeon (132) in the local operating room and the remote surgeon (154) at the remote location. The processor, camera, 2D monitor, sound device, microphone, and wireless station facilitate real-time communication using a public network (N2), as shown in FIG. 1, allowing the remote surgeon (154) to supervise, guide, and instruct the local surgeon (132) during surgery.

[0037] Instead of simple tracked 3D glasses, a mixed reality headset holder consisting of an omnidirectional microphone (not shown) and speakers (not shown) can be provided. The remote surgeon (154) can have an augmented environment with pass-through or see-through capabilities to easily view the local operating room. The headset (not shown) can feature a virtual screen for viewing the endoscopic feed from the endoscope, a virtual control screen for operating the chair ergonomics, a panel for patient details, a virtual screen for patient vitals monitoring, a virtual screen for remote proctoring access, and a virtual screen for robotic system control. The headset can include the provision of 3D notifications for important troubleshooting and surgical status data.

[0038] Figure 5a shows a block diagram illustrating the communication of control inputs from a remote surgeon at a remote location to a local operating room in accordance with an embodiment of the present disclosure. The remote surgeon (154) provides control inputs that are transmitted to a master controller (136) of a local surgeon console (118) in the local operating room over a private network (N1). Figure 5b shows a block diagram illustrating the communication of a video signal having a 3D endoscopic view of the actual surgical site from the local surgeon console to a remote surgeon console in accordance with an embodiment of the present disclosure. Figure 5c shows a block diagram illustrating the bidirectional transmission of audio and video data between a conferencing system at a remote location and a local operating room in accordance with an embodiment of the present disclosure. The local surgeon (132) and the remote surgeon (154) can interact over a public network (N2).

[0039] FIG. 6 illustrates steps for transmitting data from a remote surgeon console to a local surgeon console according to an embodiment of the present disclosure. First, control input data relating to the remote surgeon's (154) hand movements, hand control frames, data from various foot pedals, and any other sensor-based data is collected in step (602). The collected control input data is stored in frames in step (604). Next, in step (606), a connection is established between the remote surgeon console (140) and the patient-side arm cart via the master controller (136) of the local surgeon console (118) using a private network (N1) of the TCP / IP communication protocol. Received data frames with control inputs are transmitted from the remote surgeon console (140) to the master controller (136) of the local surgeon console (118) in step (608). The transmitted control input data is received by the master controller (136) of the local surgeon console (118) in step (610). The received control input data is converted into the required format in step (612). The control input data in the required format is sent to the patient side arm cart in step 614. The robotic arm is moved according to the control input using the required format data frames to match the hand movements of the remote surgeon 154 in step 616.

[0040] FIG. 7 shows a flow diagram of a method for performing telesurgery according to an embodiment of the present disclosure. Robotic surgical instruments are finalized to perform the surgical procedure. The number of robotic arms is determined for the selected robotic surgical procedure, and pre-operative planning is performed. Intra-operative planning can be performed following discussions between the local surgeon (132) and the remote surgeon (154). In step (702), a communication link is established between the local system (102) and the remote system (104) using a private network (N1). In step (704), control inputs from the remote surgeon (154) are transmitted to the local system (102) using the private network (N1). In step (706), real-time encrypted 3D video of the surgical site from the local system (102) is provided to the remote system (104) for visualization on the 3D HD monitor (150) of the remote surgeon console (140). Using the conferencing system (138) of the local surgeon console (118) and the conferencing system (160) of the remote surgeon console (140) via the public network (N2), two-way audio and video communication is established between the local surgeon console (118) and the remote surgeon console (140) in step (708). This two-way audio and video communication facilitates remote collaboration during the surgical procedure. Using the private network (N1), in step (710), DICOM images (DI) of the patient from the hospital server are transmitted for display on the 2D touchscreen monitor (148) of the remote surgeon console (140) and the 2D touchscreen monitor (126) of the local surgeon console (118). In step (712), information (SORA) regarding the status of the robotic arms (106a, 106b, 106c, 106d, 106e) and surgical robotic instruments (110, 112, 114, 116) is obtained from the hospital server via the private network (N1) and displayed using a three-dimensional (3D) HD monitor (128) on the local surgeon console (118) and a three-dimensional (3D) HD monitor (150) on the remote surgeon console (140).In step 714, control of the movement of the robotic arms (106a, 106b, 106c, 106d, 106e) is enabled using control inputs from the master controller (158) of the remote surgeon console (140). In step 716, control from the remote surgeon console (140) can be shifted to the local surgeon console (118), thereby allowing the local surgeon (132) to assume direct control over control of the movement of the robotic arms (106a, 106b, 106c, 106d, 106e) during the surgical procedure in the event of a failure of the private network (N1). Remote surgery or telesurgery is performed using the robotic arms (106a, 106b, 106c, 106d, 106e) based on control inputs from the remote surgeon (154) in step 718.

[0041] In an exemplary embodiment of the present disclosure, a remote surgeon (154) located remotely from the local operating room and a local surgeon (132) located in the local operating room can exchange system control and collaboratively perform a surgery. Also, portions of the remote surgery may be performed by the local surgeon (132) and portions of the surgery may be performed by the remote surgeon (154). Thus, the expertise of both surgeons may be utilized in performing the remote surgery.

[0042] In another exemplary embodiment, if the telesurgery multi-arm surgical robotic system (100) has a minimum of five robotic arms (106a, 106b, 106c, 106d, 106e), the telesurgery system (100) can be utilized to function as this system, and in a dual-console system, partial control from the remote surgeon console (140) can be shifted to the local surgeon console (118). The skilled surgeon / remote surgeon (154) can then operate any three of the five robotic arms (106a, 106b, 106c, 106d, 106e), and the remaining arms can be operated by the local surgeon (132) acting as a surgical assistant.

[0043] The present disclosure has the following advantages: both the local surgeon console (118) and the remote surgeon console (140) of the present disclosure are portable and have an ergonomic design. Furthermore, surgeon comfort and control during robotic surgery are improved. Integration of essential components is compact. Precise instrument control is obtained through the use of various sensors. The multi-arm robotic surgery system (100) for remote surgery can also be utilized for remote supervision. Furthermore, in any adverse scenario where there is a loss of communication between the remote surgeon console (140) and the local surgeon console (118) (due to any failure in the transmission network (N)), the local surgeon console (118) can take over overall control of the surgical procedure. Furthermore, the system (100) can be utilized in such a way that the remote surgeon (154) can operate any three of the five arms, and the remaining arms can be operated by the local surgeon (132), who may act as a surgeon assistant present in the local operating room.

[0044] The foregoing description of exemplary embodiments of the present disclosure has been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the disclosure to the precise forms disclosed, as obviously many modifications and variations are possible in light of the above teachings. The exemplary embodiments were chosen and described to best explain the principles of the present disclosure and its practical application, thereby enabling others skilled in the art to best utilize the present disclosure and various embodiments with various modifications suited to the particular uses contemplated. It is understood that various omissions and substitutions of equivalents are contemplated as circumstances may suggest or render expedient, but are intended to cover applications or implementations without departing from the spirit or scope of the claims of the present disclosure.

[0045] Benefits, other advantages, and solutions to problems have been described above with respect to particular embodiments. However, the benefits, advantages, solutions to problems, and any elements that may cause or make more pronounced any benefit, advantage, or solution should not be construed as critical, necessary, or essential features or elements of any or all of the claims.

[0046] Although specific language has been used to describe this disclosure, no limitations are intended to arise therefrom. As will be apparent to those skilled in the art, various functional modifications can be made to the device to implement the inventive concepts taught herein.

Claims

1. A multi-arm surgical robotic system (100) for telesurgery, comprising a local system (102), a remote system (104), a private network (N1), and a public network (N2); The local system (102) a plurality of robotic arms (106a, 106b, 106c, 106d, 106e) arranged along an operating table (108), wherein one robotic arm of the plurality of robotic arms (106a, 106b, 106c, 106d, 106e) is connected to an endoscopic camera (C), and the remaining robotic arms are each connected to one of surgical robotic instruments (110, 112, 114, 116); a local surgeon's console (118) comprising a left hand controller (120), a right hand controller (122), foot pedals (124), a two-dimensional (2D) touchscreen monitor (126), a three-dimensional (3D) HD monitor (128), and a head tracking camera (130) for tracking head movements of a local surgeon (132) wearing trackable glasses (134), each coupled to a master controller (136), which provides control inputs received from the local surgeon (132) to the plurality of robotic arms (106a, 106b, 106c, 106d, 106e); and Conference system (138) Equipped with The remote system (104) a remote surgeon console (140) comprising a left hand controller (142), a right hand controller (144), a foot pedal (146), a two-dimensional (2D) touchscreen monitor (148), a three-dimensional (3D) HD monitor (150), and a head tracking camera (152) for tracking head movements of a remote surgeon (154) wearing trackable glasses (156), each coupled to a master controller (158), which is operably coupled to the master controller (136) of the local surgeon console (118) via the private network (N1); and a conferencing system (160) operatively connected to said conferencing system (138) via said public network (N2); Equipped with The private network (N1) transmitting the control inputs provided by the remote surgeon (154) to the master controller (136) of the local surgeon console (118) and transmitting a real-time encrypted 3D video stream of the surgical site from the local surgeon console (118) to the three-dimensional (3D) HD monitor (150) of the remote surgeon console (140); obtaining patient-related DICOM images from a hospital server connected to the private network (N1) and providing them to the master controller (158) for display on the 2D display (148) of the remote surgeon console (140) and to the master controller (136) for display on the 2D display (126) of the local surgeon console (118); Transmitting information about the surgical instruments (110, 112, 114, 116) and the status of each of the robotic arms (106a, 106b, 106c, 106d, 106e) to the master controller (136, 158) for display on the three-dimensional (3D) HD monitor (128, 150). It is configured as follows: the public network (N2) is configured to facilitate bidirectional transmission of real-time audio and video data between the conferencing system (160) of the remote surgeon console (140) and the conferencing system (138) of the local surgeon console (118); communication between the local surgeon console (118) and the remote surgeon console (140) over the private network (N1) enables the remote surgeon (154) to control and monitor the robotic arms (106a, 106b, 106c, 106d, 106e), enabling the remote surgeon (154) to remotely perform robotic surgical procedures; The system (100) can shift the control from the remote surgeon console (140) to the local surgeon console (118), allowing the local surgeon (132) to assume control of the robotic surgical procedure in the event of a failure of the private network (N1).

2. 10. The system of claim 1, wherein in the multi-arm surgical robotic system having at least five robotic arms, partial control from the remote surgeon console can be shifted to the local surgeon console, allowing the remote surgeon to operate any three of the five or more robotic arms, with the remaining arms being operated by the local surgeon acting as an assistant surgeon.

3. 2. The system (100) of claim 1, wherein the left hand controller (120), the right hand controller (122), and the foot pedal (124) are configured to capture the control input from the local surgeon (132).

4. 10. The system of claim 1, wherein the two-dimensional (2D) touchscreen monitor (126) coupled to the master controller (136) is configured to function as a graphical user interface for capturing the control inputs from the local surgeon (132).

5. 2. The system (100) of claim 1, wherein the real-time 3D video stream of the surgical site is displayed on the three-dimensional (3D) HD monitor (128) of the local surgeon console (118).

6. The system (100) of claim 1, wherein the left hand controller (142), the right hand controller (144), and the foot pedal (146) are configured to receive the control input from the remote surgeon (154).

7. 10. The system of claim 1, wherein the two-dimensional (2D) touchscreen monitor (148) coupled to the master controller (158) is configured for use as a graphical user interface for capturing the control inputs from the remote surgeon (104).

8. The system (100) of claim 1, wherein each conferencing system (138, 160) comprises a processor, a camera, a 2D monitor, a speaker, and a microphone.

9. 10. The system of claim 1, wherein the real-time, two-way video data transmitted between the conferencing system of the remote surgeon console and the conferencing system of the local surgeon console can be displayed on separate 2D monitors at the remote location and in the local operating room.

10. A method of telesurgery using a multi-arm surgical robotic system (100) comprising a local system (102), a remote system (104), a private network (N1), and a public network (N2), comprising: establishing a communication link between the local system (102) and the remote system (104) using the private network (N1), the local system (102) including a plurality of robotic arms (106a, 106b, 106c, 106d, 106e) and a local surgeon console (118), and the remote system (104) including a remote surgeon console (140); transmitting control inputs from a remote surgeon (154) to the local system (102) using the private network (N1), the control inputs controlling the movement and operation of the plurality of robotic arms (106a, 106b, 106c, 106d, 106e); providing real-time encrypted 3D video of the surgical site from the local system (102) to the remote system (104) using the private network (N1) for visualization on a 3D HD monitor (150) of the remote surgeon console (140); using a conferencing system (138) of the local surgeon console (118) and a conferencing system (160) of the remote surgeon console (140) to enable the real-time two-way audio and video communication between the local surgeon console (118) and the remote surgeon console (140) over a public network (N2) to facilitate remote collaboration during a surgical procedure; transmitting patient DICOM images from a hospital server using the private network (N1) for display on a 2D touchscreen monitor (148) of the remote surgeon console (140) and on the 2D touchscreen monitor (126) of the local surgeon console (118); displaying information regarding the status of the robotic arms (106a, 106b, 106c, 106d, 106e) and surgical robotic instruments (110, 112, 114, 116) using the three-dimensional (3D) HD monitor (128) of the local surgeon console (118) and the three-dimensional (3D) HD monitor (150) of the remote surgeon console (140); enabling the control of the movement of the robotic arms (106a, 106b, 106c, 106d, 106e) from the remote surgeon console (140) using the control inputs from the master controller (158), wherein the remote surgeon (154) is located remotely from the patient; using the remote surgeon console (140) to shift the control from the remote surgeon console (140) to the local surgeon console (118), thereby enabling the local surgeon (132) to assume direct control over the control of the movement of the robotic arms (106a, 106b, 106c, 106d, 106e) during a surgical procedure in the event of a failure of a transmission network (N); and performing a surgical procedure on the patient using the robotic arms (106a, 106b, 106c, 106d, 106e) based on the control inputs from the remote surgeon (154), thereby enabling remote surgery; A method comprising:

11. 11. The method of claim 10, wherein in the multi-arm surgical robotic system having at least five robotic arms, partial control from the remote surgeon console can be shifted to the local surgeon console, allowing the remote surgeon to operate any three of the robotic arms, with the remaining arms being operated by the local surgeon acting as an assistant surgeon.

Images