Remote surgery support system and guiding doctor side operation device

Abstract

To provide a remote surgery support system capable of remotely operating a surgical robot from a guiding doctor side operation device in accordance with the state of communication while the guiding doctor is grasping the state of communication between the guiding doctor side operation device and the surgical robot.SOLUTION: A remote surgery support system 100 includes: a surgical robot 1 installed in a first facility B1; a doctor side operation device 2; a guiding doctor side operation device 3 installed in a second facility B2 different from the first facility B1 for a guiding doctor A2 to operate the surgical robot 1 through an external network N; and a control device. The control device acquires the state of communication between the guiding doctor side operation device 3 and the surgical robot 1. The control device superimposes a graphical user interface including information on the state of communication acquired by the control device on an endoscope image, and displays it in a scope type display device 34.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a remote surgery support system and a teaching doctor's operation device, and more particularly to a remote surgery support system and a teaching doctor's operation device for performing surgery under the guidance of a teaching doctor. [Background technology]

[0002] 2. Description of the Related Art Conventionally, a robot system for performing surgery under the guidance of a supervising physician is known (see, for example, Patent Document 1).

[0003] The above-mentioned Patent Document 1 discloses a robot system including a surgeon master control station, an instructor master control station, and a surgical robot operated by the surgeon master control station and the instructor master control station. In this robot system, the instructor instructs the surgeon by audio communication between the surgeon master control station and the instructor master control station using a headset. In this robot system, the surgeon master control station and the instructor master control station are located in the same facility where the surgical robot is installed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-181670 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the robot system described in the above Patent Document 1, the surgeon master control station and the instructor master control station are located in the same facility where the surgical robot is installed. Therefore, the instructor cannot provide guidance on surgery unless he or she goes to the facility where the surgery is being performed, so if the instructor is located remotely, it is difficult for the instructor to provide guidance on surgery using the robot system. Therefore, in order for the instructor to provide guidance on surgery using the robot system remotely, it is possible to connect the instructor's operating device and the surgical robot via a network. In this case, the state of communication between the instructor's operating device and the surgical robot may cause delays in operation, etc. In this case, it becomes difficult for the instructor to smoothly operate the surgical robot due to delays and other effects caused by the state of communication.

[0006] The present invention has been made to solve the above-mentioned problems, and one object of the present invention is to provide a remote surgery support system and a teaching physician's operating device that enable the teaching physician to grasp the communication status between the teaching physician's operating device and the surgical robot, and to remotely operate the surgical robot from the teaching physician's operating device in accordance with the communication status. [Means for solving the problem]

[0007] A remote surgery support system according to a first aspect of the present invention comprises: a surgical robot installed in a first facility and including a first manipulator arm for holding an endoscope and a second manipulator arm for holding a surgical instrument; a doctor-side operation device installed in the first facility and allowing a doctor to operate the first manipulator arm and the second manipulator arm; a teaching physician-side operation device installed in a second facility different from the first facility and allowing a teaching physician to operate the surgical robot via an external network; and one or more control devices, wherein the doctor-side operation device includes a first display unit for displaying an endoscopic image acquired by the endoscope and a first operation handle for operating the first manipulator arm and the second manipulator arm, the teaching physician-side operation device includes a second display unit for displaying an endoscopic image transmitted via the external network and a second operation handle for operating the first manipulator arm and the second manipulator arm, and the control device is configured to acquire a state of communication between the teaching physician-side operation device and the surgical robot, and to display a graphical user interface including information regarding the acquired state of communication on the second display unit by superimposing it on an endoscopic image.

[0008] In the remote surgery support system according to the first aspect of the present invention, as described above, the control device acquires the state of communication between the operating device on the instructor's side and the surgical robot, and displays a graphical user interface including information on the acquired communication state on the second display unit, superimposed on the endoscopic image. In this way, when the operating device on the instructor's side and the surgical robot are remotely connected via an external network, even if a delay in operation occurs due to the communication state, the graphical user interface including information on the communication state displayed on the second display unit allows the instructor to easily check the communication state. As a result, the instructor can remotely operate the surgical robot from the operating device on the instructor's side in accordance with the communication state while grasping the communication state between the operating device on the instructor's side and the surgical robot.

[0009] A supervising physician's side operating device according to a second aspect of the present invention is a supervising physician's side operating device installed in a second facility different from the first facility in order to operate a surgical robot of a surgical system including a doctor's side operating device and a surgical robot installed in the first facility via an external network, wherein the surgical robot includes a first manipulator arm that holds an endoscope and a second manipulator arm that holds a surgical instrument, the doctor's side operating device includes a first display unit for displaying an endoscopic image acquired by the endoscope and a first operating handle for operating the first manipulator arm and the second manipulator arm, the supervising physician's side operating device includes a second display unit for displaying an endoscopic image transmitted via the external network and a second operating handle for operating the first manipulator arm and the second manipulator arm, and the second display unit displays an endoscopic image superimposed with a graphical user interface including information regarding the state of communication between the supervising physician's side operating device and the surgical robot.

[0010] In the instructor's side operation device according to the second aspect of the present invention, as described above, the second display unit displays an endoscopic image on which a graphical user interface including information on the state of communication between the instructor's side operation device and the surgical robot is superimposed. This allows the instructor to easily check the communication state by the graphical user interface including information on the communication state displayed on the second display unit, even if the instruction doctor's side operation device and the surgical robot are remotely connected via an external network and a delay in operation occurs due to the communication state. As a result, the instructor can operate the surgical robot remotely from the instructor's side operation device in accordance with the communication state while grasping the communication state between the instruction doctor's side operation device and the surgical robot. Effect of the Invention

[0011] According to the present invention, the instructor can grasp the communication status between the instructor's operation device and the surgical robot, and can remotely operate the surgical robot from the instructor's operation device in accordance with the communication status. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing the configuration of a remote surgery support system according to one embodiment. [Diagram 2] FIG. 1 illustrates a surgical robot according to one embodiment. [Diagram 3] FIG. 2 is a diagram showing a doctor-side operation device and a supervising doctor-side operation device according to an embodiment. [Figure 4] 1 is a block diagram showing a configuration of a remote surgery support system according to one embodiment. [Diagram 5] FIG. 1 is a diagram showing a scope-type display device of a remote surgery support system according to an embodiment. [Figure 6] FIG. 2 is a block diagram showing a communication configuration of a remote surgery support system according to one embodiment. [Figure 7] FIG. 2 is a diagram showing an example of a display screen of the remote surgery support system according to one embodiment. [Figure 8] FIG. 2 is a diagram showing a first example of display of information regarding communication according to an embodiment. [Figure 9] FIG. 11 is a diagram showing a second example of display of information regarding communication according to an embodiment. [Figure 10] FIG. 13 is a diagram illustrating a third example of display of information regarding communication according to an embodiment. [Figure 11] FIG. 13 is a diagram showing a fourth example of display of information regarding communication according to an embodiment. [Figure 12] FIG. 13 is a diagram showing a fifth example of display of information regarding communication according to one embodiment. [Figure 13] FIG. 13 is a diagram showing a sixth example of display of information regarding communication according to an embodiment. [Figure 14] FIG. 13 is a diagram showing a seventh example of display of information regarding communication according to an embodiment. [Figure 15] 11 is a flowchart illustrating a communication status acquisition process according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

[0014] The configuration of a remote surgery support system 100 according to one embodiment will be described with reference to FIGS.

[0015] As shown in FIG. 1, the remote surgery support system 100 includes a surgical robot 1 installed near an operating table 101 on which a patient P is placed, a doctor-side operation device 2 for a doctor A1 to operate the surgical robot 1, and a supervising doctor-side operation device 3 for a supervising doctor A2 to operate the surgical robot 1 via an external network N. The surgical robot 1 and the doctor-side operation device 2 are installed at a first facility B1. The supervising doctor-side operation device 3 is installed at a second facility B2 different from the first facility B1. At the first facility B1, an assistant A3 is placed as a surgical staff member near the surgical robot 1. Also, an information sharing device 4 for sharing information with the assistant A3 is installed at the first facility B1. Also, at the second facility B2, a clinical engineer A4 is placed as a remote surgery staff member near the supervising doctor-side operation device 3.

[0016] As shown in FIG. 2, the surgical robot 1 includes one manipulator arm 11 that holds an endoscope 5, and multiple (three) manipulator arms 12 that hold a surgical instrument 6. The endoscope 5 captures an image of a surgical site inside a patient P. The surgical instrument 6 has an end effector such as forceps attached to its tip. The doctor-side operation device 2 and the instructor-side operation device 3 are provided to operate the manipulator arms 11 and 12. The manipulator arms 11 and 12 are examples of the "first manipulator arm" and "second manipulator arm" in the claims, respectively.

[0017] The surgical robot 1 is placed in an operating room of the first facility B1. The surgical robot 1 also includes a medical cart 13, a positioner 14, and an arm base 15. The surgical robot 1 is configured to be movable by the medical cart 13. The medical cart 13 is provided with a control device 16 that controls the operation of the surgical robot 1. The control device 16 controls the operation of the surgical robot 1 based on a command input to the doctor's operation device 2 or the instructor's operation device 3. The control device 16 includes a processing unit such as a CPU that executes a program, and a storage unit such as a memory in which the program is stored. The control device 16 is an example of the "control device" in the claims.

[0018] The medical cart 13 is also provided with an input device 17. The input device 17 is configured to receive operations for moving the manipulator arm 11, the manipulator arm 12, the positioner 14, and the arm base 15 and changing their postures, mainly for preparation of surgery before the procedure.

[0019] The positioner 14 is configured, for example, by a seven-axis articulated robot. The positioner 14 is placed on a medical cart 13. The positioner 14 moves the arm base 15. Specifically, the positioner 14 is configured to move the position of the arm base 15 three-dimensionally.

[0020] The arm base 15 is attached to the tip of the positioner 14. In addition, the base portions of the manipulator arms 11 and 12 are attached to the arm base 15. The arm base 15, the manipulator arm 11, and the manipulator arm 12 are covered with a sterile drape when in use.

[0021] As shown in FIG. 1, the doctor-side operating device 2 is disposed, for example, in or outside the operating room of the first facility B1. The doctor-side operating device 2 includes a doctor-side operating device main body 2a. As shown in FIG. 3, the doctor-side operating device main body 2a includes an operating handle 21, a foot pedal 22, a touch panel 23, a scope-type display device 24, an armrest 25, a voice communication device 26, a sensor 27, a control device 28, and a support arm 30. The operating handle 21 is an example of a "first operating handle" in the claims. The touch panel 23 is an example of a "switching unit" in the claims. The scope-type display device 24 is an example of a "first display unit" in the claims. The voice communication unit 26 is an example of a "first voice communication unit" in the claims.

[0022] The operation handle 21 constitutes an operation handle for the operator (doctor A1) to input commands. The operation handle 21 is provided for the operator (doctor A1) to operate the manipulator arms 11 and 12 (endoscope 5 and surgical instrument 6). The operation handle 21 also receives the amount of operation for the manipulator arms 11 and 12 (endoscope 5 and surgical instrument 6). The operation handle 21 includes a pair of operation handles, a left-hand operation handle 21L operated with the left hand of the operator (doctor A1) and a right-hand operation handle 21R operated with the right hand of the operator (doctor A1). The operation handle 21L is used to operate the surgical instrument 6 held by one manipulator arm 12, and the operation handle 21R is used to operate the surgical instrument 6 held by the other manipulator arm 12.

[0023] Furthermore, the operating handle 21 changes (scales) the movement amount of the endoscope 5 and the surgical instrument 6 in response to the operation amount received by the operating handle 21. For example, when the magnification of the movement amount is set to 1 / 2, the endoscope 5 and the surgical instrument 6 are controlled to move a movement distance that is 1 / 2 the movement distance of the operating handle 21. This allows delicate surgery to be performed accurately.

[0024] The foot pedal 22 includes a plurality of pedals for executing functions related to the endoscope 5 and the surgical instrument 6. The plurality of pedals include, for example, a coagulation pedal, a cutting pedal, a camera pedal, and a clutch pedal. The plurality of pedals are operated by the foot of the operator (doctor A1). When the camera pedal is operated, the endoscope held by the manipulator arm 11 can be moved by the operation handle 21. At this time, the endoscope is moved by operating both the operation handle 21L and the operation handle 21R. The touch panel 23 accepts setting operations related to the doctor-side operation device 2. The touch panel 23 is disposed on the armrest 25. The scope-type display device 24 displays an endoscopic image (see FIG. 7) acquired by the endoscope 5. The scope-type display device 24 is an immersive display device that allows only the operator (doctor A1) to view the displayed image. The operator (doctor A1) operates the operation handle 21 and the foot pedal 22 while visually checking the affected area through the scope-type display device 24. The support arm 30 supports the scope-type display device 24 so that the height of the scope-type display device 24 can be adjusted to match the height of the face of an operator such as a doctor.

[0025] The armrest 25 is formed in a bar shape, and is configured so that the operator (doctor A1) can place his / her arm thereon when operating the operation handle 21. The voice communication device 26 is disposed at a position where the operator (doctor A1) can perform voice communication with a voice communication device 36 (described later) of the supervising doctor side operation device 3 while looking into the scope-type display device 24. Specifically, the voice communication device 26 is disposed so as to be located near the head of the operator (doctor A1) while looking into the scope-type display device 24. The voice communication device 26 is disposed integrally with the scope-type display device 24, and the height of the voice communication device 26 can be adjusted integrally with the scope-type display device 24 by the support arm 30.

[0026] The sensor 27 is configured to detect whether or not the scope-type display device 24 is in a used state in which it is being looked into. The sensor 27 is disposed at a position where the head of the operator (doctor A1) can be detected when the operator (doctor A1) is looking into the scope-type display device 24. The sensor 27 is disposed integrally with the scope-type display device 24. The control device 28 controls the operation of the doctor-side operating device 2. The control device 28 includes a processing unit such as a CPU that executes a program, and a storage unit such as a memory in which the program is stored.

[0027] The doctor-side operation device 2 has been described above, but the supervising doctor-side operation device 3 has the same structure as the doctor-side operation device 2. That is, as shown in FIG. 4, the supervising doctor-side operation device 3 includes a supervising doctor-side operation device main body 3a. The supervising doctor-side operation device main body 3a includes an operation handle 31, a foot pedal 32, a touch panel 33, a scope-type display device 34, an armrest 35, a voice communication device 36, a sensor 37, a control device 38, and a support arm 30. The scope-type display device 34 displays an endoscopic image (see FIG. 7) transmitted via an external network N (see FIG. 1). The doctor-side operation device 2 and the supervising doctor-side operation device 3 are basically of the same specification, but are used by switching between a normal mode and a remote mode via the touch panel 23 (33). The operation handle 31 is an example of a "second operation handle" in the claims. The touch panel 33 is an example of a "switching unit" in the claims. Moreover, the scope-type display device 34 is an example of a "second display section" in the claims. Moreover, the audio communication device 36 is an example of a "second audio communication section" in the claims.

[0028] As shown in Figs. 1 and 4, the supervising physician side operation device 3 includes a touch panel display device 51 that displays an endoscopic image (see Fig. 7) transmitted via an external network N, and an audio communication device 52 that is arranged closer to the touch panel display device 51 than the scope-type display device 34 in order to perform audio communication with the audio communication device 26. The touch panel display device 51 is an example of a "third display unit" in the claims. The control device 43 is also an example of a "control device" in the claims. The audio communication device 52 is an example of a "third audio communication unit" in the claims.

[0029] The touch panel display device 51 and the voice communication device 52 are arranged near the main body 3a of the operating device on the supervising physician side as a unit independent of the main body 3a of the operating device on the supervising physician side. The touch panel display device 51 is, for example, a display device having a touch panel function. The touch panel display device 51 is a flat panel display device or a curved panel display device. The supervising physician A2 uses a finger or a touch panel pen or the like to cause the touch panel display device 51 to accept instruction input, thereby generating an image for guidance. The image for guidance can be, for example, a line, a circle, an arrow, or the like.

[0030] As shown in FIG. 4 and FIG. 5, the audio communication device 26, the audio communication device 36, and the audio communication device 52 each include a microphone (261, 361, 521) and a speaker (262, 362, 522). The microphone 261 (361, 521) accepts audio input. The speaker 262 (362, 522) outputs audio. Also, as shown in FIG. 5, the speaker 262 (362) is disposed at a position close to the ear of the operator (doctor A1 or supervising doctor A2) when the operator looks into the scope-type display device 24 (34). Also, the microphone 261 (361) is disposed at a position close to the mouth of the operator (doctor A1 or supervising doctor A2) when the operator looks into the scope-type display device 24 (34). Also, the microphone 261 (361) and the speaker 262 (362) are disposed integrally with the scope-type display device 24 (34).

[0031] Audio communication device 52 is disposed integrally with touch panel display device 51. Audio communication device 52, for example, has microphone 521 and speaker 522 built into touch panel display device 51. Note that in FIG. 1, touch panel display device 51 and audio communication device 52 are illustrated separately.

[0032] The doctor-side operation device 2 also includes the same units as the supervising doctor-side operation device 3. That is, the doctor-side operation device 2 includes a touch panel display device 61 that displays an endoscopic image (see FIG. 7), and an audio communication device 62 that is arranged closer to the touch panel display device 61 than the scope-type display device 24 in order to perform audio communication with the audio communication devices 36 and 52. By providing the touch panel display device 61, annotation can be performed bidirectionally between the doctor A1 and the supervising doctor A2 during surgery.

[0033] The touch panel display device 61 and the voice communication device 62 are arranged near the doctor's side operation device main body 2a as units independent of the doctor's side operation device main body 2a. The touch panel display device 61 is, for example, a flat panel display device or a curved panel display device having a touch panel function.

[0034] Moreover, the audio communication device 62 includes a microphone 621 and a speaker 622. The microphone 621 accepts audio input. The speaker 622 outputs audio. The audio communication device 62 is disposed integrally with the touch panel display device 61. For example, the audio communication device 62 has the microphone 621 and the speaker 622 built into the touch panel display device 61. Note that in FIG. 1, the touch panel display device 61 and the audio communication device 62 are illustrated separately.

[0035] As shown in FIG. 1 and FIG. 4, the information sharing device 4 is a movable monitor cart. The information sharing device 4 includes a display device 41 for displaying an endoscopic image (see FIG. 7) acquired by an endoscope 5 (see FIG. 2), and an audio communication device 42 for audio communication with the audio communication device 26 of the doctor-side operation device 2. The information sharing device 4 is disposed closer to the surgical robot 1 than the doctor-side operation device 2. This allows the assistant A3, who is disposed as a surgical staff member near the surgical robot 1, to check the endoscopic image and to have a conversation with the doctor A1. The audio communication device 42 can perform audio communication with both the audio communication device 36 and the audio communication device 52. This allows the conversation between the assistant A3, who is disposed as a surgical staff member near the surgical robot 1, and the doctor A1 to be heard by the supervising doctor A2 and the clinical engineer A4. The assistant A3 can also have a conversation with the supervising doctor A2 and the clinical engineer A4. The audio communication device 42 includes a microphone 421 and a speaker 422. The microphone 421 accepts audio input. The speaker 422 outputs audio. The display device 41 is, for example, a flat panel display device or a curved panel display device.

[0036] Further, the information sharing device 4 is provided with a control device 43. The control device 43 performs image processing. The control device 43 includes a processing unit such as a CPU that executes a program, and a storage unit such as a memory in which the program is stored. Further, the control device 43 is provided in the first facility B1.

[0037] The control device 43 displays the same image on each of the scope-type display device 24, the scope-type display device 34, the display device 41, the touch panel display device 51, and the touch panel display device 61. That is, the same endoscopic image is displayed on each of the scope-type display device 24, the scope-type display device 34, the display device 41, the touch panel display device 51, and the touch panel display device 61.

[0038] As shown in Figs. 3 to 5, the supervising physician side operation device 3 includes a sensor 37 that detects whether the scope-type display device 34 is in a used state in which it is being looked into, as described above. When the sensor 37 does not detect a used state, the control device 16 prohibits the operation of the manipulator arms 11 and 12 by the operation handle 31, and when the sensor 37 detects a used state, the control device 16 permits the operation of the manipulator arms 11 and 12 by the operation handle 31. When the sensor 37 does not detect a used state, the control device 16 prohibits the operation of the manipulator arms 11 and 12 by not accepting an input by the operation handle 31. Also, when the sensor 37 detects a used state, the control device 16 accepts a predetermined input by the operation handle 31 to operably connect the manipulator arm 12 to the operation handle 31.

[0039] The same applies to the doctor-side operating device 2. That is, when the sensor 27 does not detect a use state, the control device 16 prohibits the operation of the manipulator arms 11 and 12 by the operating handle 21, and when the sensor 27 detects a use state, the control device 16 accepts a predetermined input by the operating handle 21 to operably connect the manipulator arm 12 to the operating handle 21.

[0040] The doctor-side operating device 2 and the instructor-side operating device 3 each include a switching unit for switching the operating authority for operating the surgical robot 1. The switching unit is disposed in the armrests 25 and 35. The switching unit is a touch panel 23 (33) disposed in the armrest 25 (35). This allows the operation of the surgical robot 1 to be more easily switched between the doctor-side operating device 2 and the instructor-side operating device 3 by operating the touch panel 23 (33) disposed in the armrest 25 (35). The switching unit may also be provided as a changeover switch in the operating handle 21 (31). This allows the operation of the surgical robot 1 to be more easily switched between the doctor-side operating device 2 and the instructor-side operating device 3 by the changeover switch provided in the operating handle 21 or the operating handle 31.

[0041] 5, the surgical robot 1, doctor-side operation device 2, and information sharing device 4 installed at the first facility B1 are connected to the supervising doctor-side operation device 3 installed at the second facility B2 via an external network N. Specifically, the first facility B1 is provided with an encoder 71 that converts an image generated by the information sharing device 4 into transmittable data, a hub 72, and a router 73. The second facility B2 is provided with a decoder 81 that restores the transmitted data to an image, a hub 82, and a router 83.

[0042] The image acquired by the endoscope 5 is input to the information sharing device 4. In the information sharing device 4, a graphical user interface G1 having display areas g1, g2, g3, g4, and g5 is generated as shown in FIG. 7, and superimposed on the endoscope image. The display area g1 displays information on the surgical instrument 6 operated by the operation handle 21R, the display area g2 displays information on the replacement surgical instrument 6, the display area g3 displays information on the surgical instrument 6 operated by the operation handle 21L, the display area g4 displays information on the operation of the camera pedal, and the display area g5 displays information on the operation of the clutch pedal. The endoscope image on which the graphical user interface G1 is superimposed by the information sharing device 4 is displayed on the display device 41 and is also sent to the doctor-side operation device 2 and displayed on the scope-type display device 24. In addition, the endoscope image on which the graphical user interface G1 is superimposed, generated by the information sharing device 4, is sent to the encoder 71 and converted into transmittable data. The data converted by the encoder 71 is sent to the second facility B2 via the external network N through the hub 72 and the router 73. In the second facility B2, the converted data is sent to a decoder 81 via a router 83 and a hub 82. The decoder 81 restores the received data to an endoscopic image on which a graphical user interface G1 is superimposed, and sends the image to the operating device 3 on the supervising physician's side to be displayed on the scope-type display device 34.

[0043] By operating the doctor's operating device 2 at the first facility B1, a signal based on the operation is sent to the surgical robot 1, and the surgical instrument 6 and the endoscope 5 are operated.

[0044] By operating the operating device 3 on the supervising physician's side at the second facility B2, a signal based on the operation is sent to the surgical robot 1 via the hub 82, the router 83, the external network N, the router 73, and the hub 72, causing the surgical instrument 6 and the endoscope 5 to operate.

[0045] In addition, voice communication between the doctor's side operation device 2, information sharing device 4 and voice communication device 62 installed at the first facility B1 and the supervising doctor's side operation device 3 and voice communication device 52 installed at the second facility B2 is carried out by transmitting and receiving information via the hub 72, router 73, external network N, router 83 and hub 82.

[0046] In this embodiment, the control device 16 acquires the state of communication between the operating device 3 on the supervising physician's side and the surgical robot 1. Then, as shown in Fig. 7 to Fig. 14, the control device 43 generates a graphical user interface G1 including information about the state of communication acquired by the control device 16, and displays it on the scope-type display device 34 by superimposing it on the endoscopic image.

[0047] As a result, when the operating device 3 on the operating physician side and the surgical robot 1 are remotely connected via the external network N, even if a delay in operation occurs due to the communication state, the operating physician A2 can easily check the communication state by the graphical user interface G1 including information on the communication state displayed on the scope-type display device 34. As a result, the operating physician A2 can operate the surgical robot 1 remotely from the operating device 3 on the operating physician side in accordance with the communication state while grasping the communication state between the operating device 3 on the operating physician side and the surgical robot 1. For example, when a delay in operation occurs, the operating physician A2 can recognize the degree of the delay by the graphical user interface G1 and operate the surgical robot 1 while taking it into consideration.

[0048] Furthermore, the control device 43 is disposed in the first facility B1, and displays the endoscopic image with the graphical user interface G1 superimposed on it on the scope-type display device 34 via the external network N. As a result, the control device 43 disposed in the first facility B1 where the surgery is being performed performs a process of superimposing the graphical user interface G1 on the endoscopic image, so that the graphical user interface G1 can be superimposed on the endoscopic image without delay.

[0049] In addition, the control device 43 displays the endoscopic image with the graphical user interface G1 superimposed on it on the scope-type display device 24 of the doctor-side operation device 2. This allows not only the supervising doctor A2 but also the doctor A1 on the side where the surgery is being performed to understand the communication status, thereby preventing a difference in information recognition between the supervising doctor A2 and the doctor A1.

[0050] Furthermore, the control device 43 displays the endoscopic image with the graphical user interface G1 superimposed on it on the touch panel display device 61 of the first facility B1, the display device 41 of the information sharing device 4, and the touch panel display device 51 of the second facility B2. This allows the assistant A3 and the clinical engineer of the first facility B1, and the clinical engineer of the second facility B2 to visually grasp the communication status.

[0051] The graphical user interface G1 including information on the state of communication includes at least one of a communication delay time, an icon representing the degree of the communication delay, and a display using a color representing the degree of the communication delay. This allows the supervising physician A2 to easily visually grasp the state of the communication delay using the icon representing the degree of the communication delay time or the degree of the communication delay or the color representing the degree of the communication delay.

[0052] For example, a graphical user interface G1 including information regarding the state of communication is displayed in an area G1a near the center of the upper portion of the screen of the scope-type display device 34, as shown in FIG.

[0053] For example, the graphical user interface G1 may display a display G2 indicating the communication delay time as a numerical value, as shown in FIG.

[0054] 9 to 11, the color indications G3a, G3b, and G3c representing the degree of delay may include a first color (see FIG. 9) indicating that the delay time is equal to or less than the allowable value, a second color (see FIG. 10) indicating that the delay time exceeds the allowable value within a range equal to or less than a first predetermined value, and a third color (see FIG. 11) indicating that the delay time exceeds the allowable value by more than the first predetermined value. This allows the color to be displayed in stages according to the communication delay time, so that the supervising physician A2 can intuitively grasp the degree of the current communication delay from the color.

[0055] For example, a first color may be green or blue to indicate a small degree of delay, a second color may be yellow to indicate a medium degree of delay, and a third color may be red to indicate a large degree of delay.

[0056] Also, as shown in FIG. 12, the information on the communication state may include a display G4 of information showing the change over time of the delay time. This allows the supervising physician A2 to easily grasp the trend of the communication delay state by checking the information on the change over time of the delay time. In this case, the information showing the change over time of the delay time may include a graph showing the trend of the delay time. This allows the supervising physician A2 to visually and easily grasp the trend of the communication delay state by the graph display. Also, in this graph display, the graph may be displayed in different colors depending on the degree of delay, as shown in FIG. 12.

[0057] Furthermore, as shown in FIG. 13, the graphical user interface G1 may be displayed in an area G1b surrounding the screen of the scope-type display device .

[0058] Furthermore, as shown in FIG. 14, the graphical user interface G1 may be displayed in an area G1c near the top edge of the screen of the scope-type display device .

[0059] The control device 16 may obtain the state of communication between the operating device 3 on the instructor's side and the surgical robot 1 from a device that monitors the communication state of the hub 72 (82). The control device 16 may also obtain the state of communication between the operating device 3 on the instructor's side and the surgical robot 1 from the router 73 (83). The control device 16 may also obtain the state of communication between the operating device 3 on the instructor's side and the surgical robot 1 from the decoder 81.

[0060] (Communication status acquisition process) For example, the control device 16 performs control so as to obtain line performance information from the router 83 by the communication status obtaining process shown in FIG.

[0061] In step S1 of Fig. 15, the router 83 transmits a Ping command to the router 73. When transmitting the Ping command, the following options may be set. Note that the option settings are not fixed and can be changed. The option settings include, for example, the destination (router 73), the number of repetitions, the repetition interval (in milliseconds), the contents of the measurement packet, and the size of the measurement packet (in bytes).

[0062] In step S2, the result of the Ping command is received by the router 73. In step S3, the received result is processed by the router 73. The processed received result data includes, for example, the measurement date, the measurement time, the average performance value, and the packet loss value.

[0063] In step S4, the communication status is transmitted to the control device 16. As a result, the control device 16 acquires the communication status.

[0064] (Effects of this embodiment) In this embodiment, the following effects can be obtained.

[0065] In this embodiment, as described above, the control device 16 acquires the state of communication between the operating device 3 on the supervising physician side and the surgical robot 1. The control device 43 displays the graphical user interface G1 including information on the state of communication acquired by the control device 16 on the endoscope image on the scope-type display device 34 of the second facility B2. In this way, when the operating device 3 on the supervising physician side and the surgical robot 1 are remotely connected via the external network N, even if a delay in operation occurs due to the communication state, the supervising physician A2 can easily check the communication state by the graphical user interface G1 including information on the communication state displayed on the scope-type display device 34. As a result, the supervising physician A2 can operate the surgical robot 1 remotely from the operating device 3 on the supervising physician side according to the communication state while grasping the communication state between the operating device 3 on the supervising physician side and the surgical robot 1.

[0066] (Modification) The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is indicated by the claims, not by the description of the embodiments above, and further includes all modifications (variations) within the meaning and scope of the claims.

[0067] For example, in the above embodiment, both the doctor side operation device and the supervising doctor side operation device are provided with a touch panel display device and a voice communication device, but the present invention is not limited to this. For example, only the supervising doctor side operation device may be provided with a touch panel display device and a voice communication device.

[0068] In the above embodiment, the control process of the present invention is performed by a plurality of control devices, but the present invention is not limited to this. For example, the control process of the present invention may be performed by a single control device.

[0069] In the above embodiment, the first and second display units of the present invention are provided as scope-type display devices, but the present invention is not limited to this. For example, the first and second display units of the present invention may be provided as curved panel display devices or flat panel display devices other than scope-type display devices.

[0070] In the above embodiment, a switching unit for switching the operation authority for operating the surgical robot is provided on both the doctor-side operating device and the instructor-side operating device, but the present invention is not limited to this. For example, in the present invention, a switching unit for switching the operation authority for operating the surgical robot may be provided on either the doctor-side operating device or the instructor-side operating device.

[0071] In the above embodiment, the communication status is notified by displaying the graphical user interface G1 including the information on the communication status, but the present invention is not limited to this. For example, in addition to displaying the graphical user interface G1 including the information on the communication status, the communication status may be notified by sound or a voice message using the voice communication device 36. When notifying by sound, the communication delay level is notified by dividing it into stages such as "beep", "beep beep", and "beep beep", and when notifying by voice, a message such as "the line delay is large" is played, so that the supervising doctor A2 and the clinical engineer A4 in the vicinity can be notified. Furthermore, the communication status may be notified to the doctor A1 and the assistant A3 by sound or a voice message using the voice communication device 26 and the voice communication device 42. In addition to displaying the graphical user interface G1 including the information on the communication status, the communication status may be notified by vibrating the operation handle 31 and the operation handle 21. When notifying by vibration, the communication delay level can be notified by dividing the magnitude of the vibration into stages.

[0072] In the above embodiment, an example of a configuration in which an audio communication device is provided in the information sharing device to perform audio communication is shown, but the present invention is not limited to this. In the present invention, an audio communication device may be provided in the surgical robot to perform audio communication.

[0073] In the above embodiment, the supervising physician side operation device 3 is installed in the second facility B2, but the present invention is not limited to this. In the present invention, the doctor side operation device and the surgical robot are arranged in the second facility B2, and when the normal mode is selected in the mode selection, the surgical robot in the second facility B2 is operated by the doctor side operation device in the second facility B2, and when the remote mode is selected in the mode selection, the doctor side operation device in the second facility B2 functions as the supervising physician side operation device, and the surgical robot in the first facility B1 is operated by the doctor side operation device in the second facility B2.

[0074] [Aspects] It will be appreciated by those skilled in the art that the exemplary embodiments described above are examples of the following aspects.

[0075] (Item 1) a surgical robot installed in a first facility and including a first manipulator arm for holding an endoscope and a second manipulator arm for holding a surgical instrument; A doctor-side operation device that is installed in the first facility and allows a doctor to operate the first manipulator arm and the second manipulator arm; a surgeon's side operation device that is installed in a second facility different from the first facility and that allows a surgeon to operate the surgical robot via an external network; one or more control devices; the doctor-side operation device includes a first display unit for displaying an endoscopic image acquired by the endoscope, and a first operation handle for operating the first manipulator arm and the second manipulator arm, the instructor's side operation device includes a second display unit for displaying the endoscopic image transmitted via the external network, and a second operation handle for operating the first manipulator arm and the second manipulator arm, The control device is configured to acquire a communication status between the supervising physician's operating device and the surgical robot, and to display a graphical user interface including information regarding the acquired communication status on the second display unit, superimposed on the endoscopic image.

[0076] (Item 2) The remote surgical support system of item 1, wherein the graphical user interface including information regarding the status of the communication includes at least one of a communication delay time, an icon representing the degree of the communication delay, and a color indication representing the degree of the communication delay.

[0077] (Item 3) 3. The remote surgical support system according to item 2, wherein the colors representing the degree of delay include a first color indicating that the delay time is equal to or less than a tolerance, a second color indicating that the delay time exceeds the tolerance by a range equal to or less than a first predetermined value, and a third color indicating that the delay time exceeds the tolerance by more than the first predetermined value.

[0078] (Item 4) 4. The remote surgery support system according to any one of items 1 to 3, wherein the information on the communication state includes information indicating a change over time in delay time.

[0079] (Item 5) 5. The remote surgery support system according to item 4, wherein the information indicating the change in delay time over time includes a graph indicating the trend in delay time.

[0080] (Item 6) The control device is disposed in the first facility, and displays the endoscopic image with the graphical user interface superimposed on it on the second display unit via the external network.

[0081] (Item 7) 7. The remote surgery support system according to item 6, wherein the control device displays the endoscopic image with the graphical user interface superimposed thereon on the first display unit.

[0082] (Item 8) A remote surgery support system described in any one of items 1 to 7, wherein at least one of the doctor's side operation device and the supervising doctor's side operation device includes a switching unit that switches operating authority for operating the surgical robot.

[0083] (Item 9) 9. The remote surgery support system according to item 8, wherein the switching unit has a changeover switch provided on at least one of the first operating handle and the second operating handle.

[0084] (Item 10) At least one of the doctor-side operation device and the supervising doctor-side operation device includes an armrest, 10. The remote surgery support system according to claim 8 or 9, wherein the switching unit has a touch panel arranged on the armrest.

[0085] (Item 11) and a third display unit disposed in the second facility for displaying the endoscopic image acquired by the endoscope and transmitted via the external network. The remote surgery support system according to any one of items 1 to 10, wherein the first display unit and the second display unit are immersive scope-type display devices, and the third display unit is a flat panel display device or a curved panel display device.

[0086] (Item 12) The doctor-side operating device further includes a first voice communication unit, the instructor side operation device further includes a second voice communication unit and a third voice communication unit for performing voice communication with the first voice communication unit, the third audio communication unit is disposed at a position closer to the third display unit than the second audio communication unit; Item 12. The remote surgery support system according to item 11, wherein the second audio communication unit is positioned closer to the second display unit than the third audio communication unit.

[0087] (Item 13) Item 13. The remote surgery support system according to item 12, wherein the control device notifies at least the second voice communication unit of a sound or voice message corresponding to the status of the communication.

[0088] (Item 14) 14. The remote surgery support system according to any one of claims 1 to 13, wherein the control device notifies the state of the communication by vibrating at least a second operating handle in response to the state of the communication.

[0089] (Item 15) A surgeon-side operation device installed in a first facility and a surgical robot of a surgical system including the surgeon-side operation device and the surgical robot, the surgeon-side operation device being installed in a second facility different from the first facility to operate the surgical robot via an external network, the surgical robot includes a first manipulator arm that holds an endoscope and a second manipulator arm that holds a surgical instrument; the doctor-side operation device includes a first display unit for displaying an endoscopic image acquired by the endoscope, and a first operation handle for operating the first manipulator arm and the second manipulator arm, the instructor's side operation device includes a second display unit for displaying the endoscopic image transmitted via the external network, and a second operation handle for operating the first manipulator arm and the second manipulator arm, The second display unit displays the endoscopic image superimposed with a graphical user interface including information regarding the state of communication between the instructor's side operation device and the surgical robot. [Explanation of symbols]

[0090] 1. Surgical robots 2. Doctor's operation device 3 Instructor side operating device 5 Endoscopy 6 Surgical instruments 11 Manipulator arm (first manipulator arm) 12 Manipulator arm (second manipulator arm) 16 Control device 21 Operating handle (first operating handle) 23 Touch panel (switching section) 25 Armrest 24 Scope-type display device (first display unit) 26 Voice communication device (first voice communication unit) 31 Operating handle (second operating handle) 33 Touch panel (switching section) 34 Scope-type display device (second display unit) 35 Armrest 36 Voice communication device (second voice communication unit) 43 Control device (control device) 51 Touch panel display device (third display unit) 52 Voice communication device (third voice communication unit) 100 Remote surgery support system A1 Doctor A2 Instructor B1 Facility 1 B2 Second Facility G1 Graphical User Interface N External Network

Claims

1. a surgical system installed in a first facility, the surgical system including a first manipulator arm for holding a first surgical instrument, a second manipulator arm for holding a second surgical instrument, a third manipulator arm for holding an endoscope, and a doctor-side operation device for a doctor to operate the first to third manipulator arms; an operating device on the supervising physician side that is installed in a second facility different from the first facility and that allows the supervising physician to operate the first to third manipulator arms via an external network; one or more controllers; the doctor-side operating device includes a first display unit for displaying an endoscopic image acquired by the endoscope, a first operating handle for operating the first to third manipulator arms, a first armrest, and a first switching unit disposed on the first armrest and switching operation authority for operating the first to third manipulator arms, the supervising physician side operation device includes a second display unit for displaying the endoscopic image transmitted via the external network, a second operation handle for operating the first to third manipulator arms, a second armrest, and a second switching unit disposed on the second armrest and switching the operation authority, a control device configured to acquire the status of communication between the supervising physician's operating device and the surgical system, and to display a graphical user interface containing information regarding the acquired communication status on the first and second display units, superimposed on the endoscopic image.

2. 2. The remote surgical support system of claim 1, wherein the graphical user interface including information regarding the communication status includes at least one of a communication delay time, an icon representing the degree of communication delay, and a color indication representing the degree of communication delay.

3. 3. The remote surgery support system of claim 2, wherein the colors representing the degree of delay include a first color indicating that the delay time is equal to or less than a tolerance value, a second color indicating that the delay time exceeds the tolerance value by a range equal to or less than a first predetermined value, and a third color indicating that the delay time exceeds the tolerance value by more than the first predetermined value.

4. The remote surgery support system according to claim 1 , wherein the information relating to the communication state includes information indicating a change in delay time over time.

5. The remote surgery support system according to claim 4 , wherein the information indicating the change in the delay time over time includes a graph indicating a trend in the delay time.

6. 2. The remote surgery support system according to claim 1, wherein the control device is located at the first facility and displays the endoscopic image with the graphical user interface superimposed on it on the second display unit via the external network.

7. a third display unit disposed in the second facility for displaying the endoscopic image acquired by the endoscope and transmitted via the external network; The remote surgery support system according to claim 1 , wherein the first display unit and the second display unit are immersive scope-type display units, and the third display unit is a flat panel display unit or a curved panel display unit.

8. The doctor-side operating device further includes a first voice communication unit, the supervising physician side operation device further includes a second voice communication unit and a third voice communication unit for performing voice communication with the first voice communication unit, the third audio communication unit is disposed closer to the third display unit than the second audio communication unit; The remote surgery support system according to claim 7 , wherein the second audio communication unit is disposed closer to the second display unit than the third audio communication unit.

9. The remote surgery support system according to claim 8 , wherein the control device notifies at least the second voice communication unit of a sound or a voice message corresponding to the state of the communication.

10. A remote surgical support system as described in claim 1, wherein the control device notifies the state of the communication by vibrating at least the second operating handle in response to the state of the communication.

11. An information sharing device is further provided, the information sharing device being disposed in the first facility, including a fourth display unit, and being disposed closer to the first to third manipulator arms than the doctor-side operation device; The remote surgery support system according to claim 1 , wherein the control device displays the endoscopic image on the fourth display unit, with a graphical user interface including information about a state of the communication superimposed thereon.

12. The remote surgical assistance system according to claim 1 , wherein the surgical system comprises a surgical robot supporting the first to third manipulator arms.

13. a supervising physician-side operation device that includes a physician-side operation device, a first manipulator arm that holds a first surgical instrument, a second manipulator arm that holds a second surgical instrument, and a third manipulator arm that holds an endoscope, and that is installed in a second facility different from the first facility to operate the first to third manipulator arms of a surgical system installed in the first facility via an external network, the doctor-side operating device includes a first display unit for displaying an endoscopic image acquired by the endoscope, a first operating handle for operating the first to third manipulator arms, a first armrest, and a first switching unit disposed on the first armrest and switching operation authority for operating the first to third manipulator arms, the supervising physician side operation device includes a second display unit for displaying the endoscopic image transmitted via the external network, a second operation handle for operating the first to third manipulator arms, a second armrest, and a second switching unit disposed on the second armrest and switching the operation authority, The second display unit displays the endoscopic image on which a graphical user interface including information regarding the state of communication between the instructor's side operation device and the surgical system is superimposed.

Images