Remote surgical support system and operating device for supervising physicians

Abstract

To provide a remote surgery support system capable of preventing a manipulation arm from being operated unintentionally while a guiding doctor is carrying out annotation for a doctor, and improving safety during the annotation.SOLUTION: A remote surgery support system 100 includes a surgical robot 1, a doctor side operation device 2, a guiding doctor side operation device 3, and a control device 43. The doctor side operation device 2 includes a scope type display device 24. The guiding doctor side operation device 3 includes a scope type display device 34 and a sensor 37 for detecting whether or not the scope type display device 34 is in a use state of being looked into, and a touch panel display device 51 for receiving an instruction input for generating an annotation image G2. The control device 43 prohibits the instruction input by the touch panel display device 51 when the use state is detected by the sensor 37.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a remote surgery support system and an instructor-side operation device, and particularly to a remote surgery support system and an instructor-side operation device in which a doctor and an instructor perform voice communication.

Background Art

[0002] Conventionally, a remote surgery support system in which a doctor and an instructor perform voice communication has been known (see, for example, Patent Document 1).

[0003] The medical robot system (remote surgery support system) of Patent Document 1 includes a surgeon master control station operated by a surgeon performing surgery, a slave cart having a plurality of slave robot mechanisms, and an instructor master control station operated by an instructor. In this medical robot system, between the surgeon master control station and the instructor master control station, the surgeon and the instructor perform voice communication using a headset. Further, the instructor performs telescribing to generate an annotation to be superimposed on an image of an anatomical structure using a touch screen connected to the instructor master control station.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, Patent Document 1 does not mention placing the instructor master control station in a separate facility from the surgeon master control station and slave cart, and using it via an external network. Furthermore, it does not mention at all installing the instructor master control station in a separate facility from the surgeon master control station and slave cart, and performing "remote surgical support" as described in the remote surgery guidelines (Non-Patent Document 1) via an external network. In addition, Patent Document 1 does not specifically describe actually equipping the instructor master control station with a master input device and using it to operate a slave robot mechanism located in a separate facility to perform surgery. Therefore, Patent Document 1 has a problem in that when an instructor is performing telestration (annotation), there is a need to improve safety regarding the operation of the slave robot mechanism (manipulator arm) via an external network using a master input device (operating handle).

[0007] This invention was made to solve the above-mentioned problems, and one objective of this invention is to provide a remote surgical support system and a supervising physician's control device that can prevent the manipulator arm from being unintentionally operated while the supervising physician is performing annotations for a physician, thereby improving safety during annotation. [Means for solving the problem]

[0008] A remote surgical assistance system according to the first aspect of this invention includes a surgical robot installed in a first facility, equipped with a first manipulator arm for holding an endoscope and a second manipulator arm for holding surgical instruments, A first display device that displays endoscopic images acquired by an endoscope. The physician operates the first and second manipulator arms. It is equipped with a first operating handle and a first switching device for switching the operating authority for operating the first and second manipulator arms, and is installed in the first facility. The physician's control device, A scope-type display device that displays endoscopic images transmitted via an external network, a sensor that detects whether or not the scope-type display device is in use and being looked into, via external network First and second manipulator arms The supervising physician operates it. It is equipped with a second operating handle, a second switching device for switching operating authority, and a touch panel display device that displays endoscopic images transmitted via an external network and accepts instruction input for generating annotation images, and is installed at a second facility different from the first facility. The supervising physician's control device, The system is configured to prohibit the operation of the first and second manipulator arms by the second operating handle if the supervising physician has operating authority and the sensor does not detect any usage status.A set comprising one or more control devices, ru .

[0009] In the remote surgical support system according to the first aspect of this invention, as described above, to constitute This will ,finger This can improve the safety of annotation by preventing the manipulator arm from being unintentionally manipulated while the supervising physician is performing annotations for other physicians.

[0010] The supervising physician's operating device according to the second aspect of this invention is: It includes a first manipulator arm for holding an endoscope and a second manipulator arm for holding surgical instruments. Surgical robots and A physician-side operating device comprising a first display device for displaying endoscopic images acquired by the endoscope, a first operating handle for operating the first and second manipulator arms, and a first switching device for switching the operating authority for operating the first and second manipulator arms, Includes The first facility was Surgical system First and second manipulator arms To operate it via an external network, the supervising physician's control device is installed at a second facility different from the first facility, and displays endoscopic images transmitted via the external network. Rus A cope-type display device and a sensor that detects whether the scope-type display device is in use and being looked at. ,finger The attending physician operates the first and second manipulator arms. The first Two operating handles, A second switching device for switching operating authority, and external The system includes a touch panel display device that displays endoscopic images transmitted via a network and accepts instruction input for generating annotation images. , has operational authority, and The usage status is detected by the sensor. do not have case, Second operating handle by Operation of the first and second manipulator arms Prohibit It is configured in such a way .

[0011] In the supervising physician's operating device according to the second aspect of this invention, as described above, to constitute This makes it possible to provide a physician-side operating device that prevents the manipulator arm from being unintentionally operated while the supervising physician is performing annotations for other physicians, thereby improving the safety of annotation. [Effects of the Invention]

[0012] According to the present invention, it is possible to prevent the manipulator arm from being operated unintentionally while the instructor is making an annotation to the doctor, and improve the safety of the annotation.

Brief Description of the Drawings

[0013] [Figure 1] It is a diagram showing the configuration of a remote surgery support system according to an embodiment. [Figure 2] It is a diagram showing the configuration of a surgical robot according to an embodiment. [Figure 3] It is a diagram showing the configuration of a doctor-side operation device (instructor-side operation device) according to an embodiment. [Figure 4] It is a block diagram showing the configuration of a remote surgery support system according to an embodiment. [Figure 5] It is a diagram showing a scope-type display device of a remote surgery support system according to an embodiment. [Figure 6] It is a block diagram showing the configuration related to the control device of an information sharing device according to an embodiment. [Figure 7] It is a diagram showing an endoscopic image with an annotation image superimposed according to an embodiment. [Figure 8] It is a diagram for explaining a request for switching the operation authority of a surgical robot according to an embodiment. [Figure 9] It is a diagram for explaining the notification of a request for switching the operation authority according to an embodiment. [Figure 10] It is a diagram for explaining the response to a request for switching the operation authority according to an embodiment. [Figure 11] It is a diagram for explaining the display of the operation authority according to an embodiment. [Figure 12] It is a flowchart for explaining the control process related to the switching of the operation authority according to an embodiment. [Figure 13] It is a block diagram showing the configuration of a remote surgery support system according to a modification example.

Modes for Carrying Out the Invention

[0014] The following describes embodiments of the present invention based on the drawings.

[0015] Referring to Figures 1 to 12, the configuration of a remote surgical support system 100 according to one embodiment will be described.

[0016] As shown in Figure 1, the remote surgical support system 100 comprises a surgical robot 1 installed near the operating table 101 on which patient P is placed, a physician-side control device 2 for physician A1 to operate the surgical robot 1, and a supervising physician-side control device 3 for supervising physician A2 to operate the surgical robot 1 via an external network N. The surgical robot 1 and physician-side control device 2 are installed at Facility 1 B1. The supervising physician-side control device 3 is installed at Facility 2 B2, which is different from Facility 1 B1. At Facility 1 B1, an assistant A3 is positioned near the surgical robot 1 as part of the surgical staff. Also at Facility 1 B1, an information sharing device 4 is installed for sharing information with assistant A3. At Facility 2 B2, a clinical engineer A4 is positioned near the supervising physician-side control device 3 as part of the remote surgical staff.

[0017] As shown in Figure 2, the surgical robot 1 comprises one manipulator arm 11 for holding an endoscope 5 and a plurality (three) manipulator arms 12 for holding surgical instruments 6. The endoscope 5 images the surgical site inside the patient P. The surgical instruments 6 have end effectors such as forceps attached to their tips. The physician's control device 2 and the supervising physician's control device 3 are provided for operating the manipulator arms 11 and 12. The manipulator arms 11 and 12 are examples of the "first manipulator arm" and "second manipulator arm" as defined in the claims, respectively.

[0018] The surgical robot 1 is located in the operating room of Facility B1. The surgical robot 1 also includes a medical cart 13, a positioner 14, and an arm base 15. The surgical robot 1 is configured to be movable by the medical cart 13. The medical cart 13 is equipped with a control device 16 that controls the operation of the surgical robot 1. The control device 16 controls the operation of the surgical robot 1 based on commands input to the physician's operating device 2 or the supervising physician's operating device 3. The control device 16 includes a processing unit such as a CPU that executes a program, and a storage unit such as a memory in which the program is stored. Note that the control device 16 is an example of "one or more control devices" in the claims.

[0019] Furthermore, the medical trolley 13 is equipped with an input device 17. The input device 17 is configured to accept commands for movement and posture changes of the manipulator arm 11, manipulator arm 12, positioner 14, and arm base 15, primarily for preparing for surgery before the procedure.

[0020] The positioner 14 is composed of, for example, a 7-axis articulated robot. The positioner 14 is also positioned on a medical trolley 13. The positioner 14 moves the arm base 15. Specifically, the positioner 14 is configured to move the position of the arm base 15 in three dimensions.

[0021] The arm base 15 is attached to the tip of the positioner 14. The arm base 15 has a relatively long rod shape (elongated shape). The manipulator arms 11 and 12 are each attached to the arm base 15 at their bases. The arm base 15, manipulator arms 11, and manipulator arms 12 are used covered with a sterile drape.

[0022] As shown in Figure 1, the physician-side operating device 2 is located, for example, inside or outside the operating room of the first facility B1. The physician-side operating device 2 also comprises a physician-side operating device body 2a. As shown in Figure 3, the physician-side operating device body 2a comprises an operating handle 21, a foot pedal 22, a touch panel 23, a scope-type display device 24, an armrest 25, a voice communication device 26, a sensor 27, a control device 28, and a support arm 30. The operating handle 21 is an example of the "first operating handle" in the claims. The touch panel 23 is an example of the "switching device" in the claims. The scope-type display device 24 is an example of the "first display device" and the "second scope-type display device" in the claims. The voice communication device 26 is an example of the "first voice communication device" in the claims. The sensor 27 is an example of the "second sensor" in the claims. Furthermore, the control device 28 is an example of "one or more control devices" as defined in the claims.

[0023] The operating handle 21 constitutes an operating handle for the operator (physician A1) to input commands. The operating handle 21 is provided for the operator (physician A1) to operate the manipulator arms 11 and 12 (endoscope 5 and surgical instrument 6). The operating handle 21 also receives the amount of manipulation applied to the manipulator arms 11 and 12 (endoscope 5 and surgical instrument 6). The operating handle 21 includes a pair of operating handles: a left-handed operating handle 21L operated by the operator's (physician A1) left hand, and a right-handed operating handle 21R operated by the operator's (physician A1) right hand. The operating handle 21L is used to operate the surgical instrument 6 held on one manipulator arm 12, and the operating handle 21R is used to operate the surgical instrument 6 held on the other manipulator arm 12.

[0024] Furthermore, the operating handle 21 modifies (scales) the movement of the endoscope 5 and surgical instruments 6 in relation to the amount of manipulation received by the operating handle 21. For example, if the movement scaling factor is set to 1 / 2, the endoscope 5 and surgical instruments 6 are controlled to move a distance equal to half the movement distance of the operating handle 21. This allows for precise and detailed surgical procedures to be performed accurately.

[0025] The foot pedal 22 includes multiple pedals for performing functions related to the endoscope 5 and surgical instruments 6. These multiple pedals include, for example, a coagulation pedal, a cutting pedal, a camera pedal, and a clutch pedal. The multiple pedals are operated by the operator's (physician A1) feet. When the camera pedal is operated, the endoscope 5 held by the manipulator arm 11 can be moved using the operating handle 21. At this time, the endoscope 5 is operated and moved using both the operating handle 21L and the operating handle 21R. The touch panel 23 accepts setting operations related to the physician's operating device 2. The touch panel 23 is located on the armrest 25. The scope-type display device 24 displays the endoscopic image G1 (see Figure 7) acquired by the endoscope 5. The scope-type display device 24 is an immersive display device that allows only the operator (physician A1) to view the displayed image. The operator (physician A1) operates the operating handle 21 and the foot pedal 22 while visualizing the affected area using the scope-type display device 24. The support arm 30 supports the scope-type display device 24 so that its height can be adjusted to match the face height of the operator, such as physician A1.

[0026] The armrest 25 is formed in a bar shape and is configured so that the operator (doctor A1) can rest their arm on it when operating the operating handle 21. The voice communication device 26 is positioned so that the operator (doctor A1) can communicate with the voice communication device 36 of the supervising physician's operating device 3 (described later) while looking into the scope-type display device 24. Specifically, the voice communication device 26 is positioned near the operator's (doctor A1) head when the operator (doctor A1) is looking into the scope-type display device 24. The voice communication device 26 is integrally mounted on the scope-type display device 24 and its height can be adjusted integrally by the support arm 30.

[0027] Sensor 27 is configured to detect whether the scope-type display device 24 is in use with the user looking into it. Sensor 27 is positioned to detect the operator's (doctor A1) head when the operator is looking into the scope-type display device 24. Sensor 27 is integrally mounted on the scope-type display device 24. Control device 28 controls the operation of the doctor's operating device 2. Control device 28 includes a processing unit such as a CPU that executes a program and a storage unit such as memory where the program is stored. Note that the use state in which the scope-type display device 24 is being looked into is an example of the "second use state" in the claims.

[0028] The above describes the physician-side operating device 2, but the supervising physician-side operating device 3 has a similar structure to the physician-side operating device 2. That is, as shown in Figure 4, the supervising physician-side operating device 3 comprises the supervising physician-side operating device main body 3a. The supervising physician-side operating device main body 3a comprises an operating handle 31, a foot pedal 32, a touch panel 33, a scope-type display device 34, an armrest 35, an audio communication device 36, a sensor 37, a control device 38, and a support arm. The scope-type display device 34 displays the endoscopic image G1 (see Figure 7) transmitted via the external network N (see Figure 1). Although the physician-side operating device 2 and the supervising physician-side operating device 3 have basically the same specifications, they are used by switching between normal mode and remote mode via the touch panel 23 (33). Note that the operating handle 31 is an example of the "second operating handle" in the claims. Also, the scope-type display device 34 is an example of the "first scope-type display device" in the claims. Furthermore, the touch panel 33 is an example of the "switching device" in the claims. Also, the voice communication device 36 is an example of the "second voice communication device" in the claims. Also, the sensor 37 is an example of the "first sensor" in the claims. Also, the usage state in which the scope-type display device 34 is being looked at is an example of the "first usage state" in the claims.

[0029] In this embodiment, as shown in Figures 1 and 4, the supervising physician's operating device 3 includes a touch panel display device 51 that displays the endoscopic image G1 (see Figure 7) transmitted via the external network N and accepts instruction input for generating an annotation image G2 (see Figure 7). Furthermore, the control device 43 of the information sharing device 4, described later, displays the annotation image G2 on the scope-type display device 24 while updating it according to the instruction input from the touch panel display device 51 if the sensor 37 does not detect a usage status, and does not update the annotation image G2 displayed on the scope-type display device 24 if the sensor 37 detects a usage status, as instruction input from the touch panel display device 51 is prohibited. In addition, the control device 38 of the supervising physician's operating device 3 allows instruction input from the touch panel display device 51 if the sensor 37 does not detect a usage status, and prohibits instruction input from the touch panel display device 51 if the sensor 37 detects a usage status. As a result, by allowing instruction input from the touch panel display device 51, supervising physician A2 can instruct physician A1 to add the annotation image G2 as an annotation to the endoscopic image G1 during surgery. Furthermore, it is possible to prevent the use of the touch panel display device 51 for annotation when supervising physician A2 is operating the surgical robot 1, thereby improving safety during annotation. As a result, it is possible to prevent the manipulator arm from being unintentionally operated while supervising physician A2 is performing annotation on physician A1, thereby improving the safety of annotation. Note that the touch panel display device 51 is an example of the "first touch panel display device" in the claims. Also, the control devices 43 and 38 are examples of "one or more control devices" in the claims.

[0030] Furthermore, in this embodiment, the control device 16 of the surgical robot 1 prohibits the operation of the manipulator arms 11 and 12 by the operating handle 31 when the sensor 37 does not detect a usage state, and permits the operation of the manipulator arms 11 and 12 by the operating handle 31 when the sensor 37 detects a usage state. As a result, when the robot is not in use, even if the supervising physician A2 unintentionally touches the operating handle 31, it is possible to prevent the manipulator arms 11 and 12 from being operated. The control device 16 prohibits the operation of the manipulator arms 11 and 12 by not accepting input from the operating handle 31 when the sensor 37 does not detect a usage state. The control device 16 also permits the operation of the manipulator arms 11 and 12 by accepting a predetermined input from the operating handle 31 when the sensor 37 detects a usage state.

[0031] Furthermore, in this embodiment, the control device 38 of the supervising physician's operating device 3 prohibits instruction input via the touch panel display device 51 when the sensor 37 detects that it is in use, and allows instruction input via the touch panel display device 51 when the sensor 37 does not detect that it is in use. As a result, when supervising physician A2 is operating the manipulator arms 11 and 12 with the operating handle 31, instruction input via the touch panel display device 51 is prohibited, preventing the manipulator arms 11 and 12 from being operated unintentionally while supervising physician A2 is performing annotations for physician A1, thereby improving the safety of the annotation.

[0032] Furthermore, in this embodiment, the supervising physician's operating device 3 includes a voice communication device 52 for voice communication with the voice communication device 26. The voice communication device 52 is positioned closer to the touch panel display device 51 than to the scope-type display device 34. This allows voice communication between the voice communication device 52 and the voice communication device 26, enabling supervising physician A2 to perform annotations while interacting with physician A1 during surgery, and allowing clinical engineer A4, positioned as a remote surgical staff member on supervising physician A2's side, to interact with physician A1. In addition, supervising physician A2 can also interact with physician A1 by communicating voice between the voice communication device 36 and the voice communication device 26.

[0033] The touch panel display device 51 and the voice communication device 52 are positioned near the main body 3a of the supervising physician's control device 3a as annotation units independent of the main body 3a of the supervising physician's control device 3a. The touch panel display device 51 is, for example, a flat panel display device with touch panel functionality. Supervising physician A2 uses their finger or a stylus for the touch panel to input instructions into the touch panel display device 51, thereby generating an annotation image G2 with a shape corresponding to the input. The annotation image G2 may be, for example, a line, a circle, or an arrow.

[0034] Furthermore, as shown in Figures 4 and 5, the voice communication devices 26, 36, and 52 each include microphones (261, 361, 521) and speakers (262, 362, 522). Microphones 261 (361, 521) receive voice input. Speakers 262 (362, 522) output sound. Also, as shown in Figure 5, speakers 262 (362) are positioned close to the operator's ear when the operator (physician A1 or supervising physician A2) is looking into the scope-type display device 24 (34). Microphone 261 (361) is positioned close to the operator's mouth when the operator (physician A1 or supervising physician A2) is looking into the scope-type display device 24 (34). Microphones 261 (361) and speakers 262 (362) are integrally mounted on the scope-type display device 24 (34).

[0035] Furthermore, in this embodiment, the voice communication device 52 is integrally arranged with the touch panel display device 51. This saves space for both the voice communication device 52 and the touch panel display device 51 compared to when the voice communication device 52 is arranged separately from the touch panel display device 51. The voice communication device 52, for example, has a microphone 521 and a speaker 522 built into the touch panel display device 51. Note that in Figure 1, the touch panel display device 51 and the voice communication device 52 are shown separately.

[0036] Furthermore, in this embodiment, the supervising physician's control device 3 (supervising physician's control device main unit 3a) is equipped with an interface 39 (see Figure 4) for connecting external devices. The touch panel display device 51 is also connected to the supervising physician's control device 3 (supervising physician's control device main unit 3a) via the interface 39. This allows the voice communication device 52 and the touch panel display device 51 to be connected to an external network N via the supervising physician's control device 3 (supervising physician's control device main unit 3a). As a result, the communication configuration can be simplified. The interface 39 is a terminal capable of inputting and outputting image data and audio data. The touch panel display device 51 is connected to the interface 39 via a cable capable of inputting and outputting image data and audio data. The touch panel display device 51 and the voice communication device 52 input and output image data and audio data via the interface 39.

[0037] Furthermore, in this embodiment, the physician-side operating device 2, like the supervising physician-side operating device 3, is equipped with an annotation unit. Specifically, the physician-side operating device 2 includes a touch panel display device 61 that displays the endoscopic image G1 (see Figure 7) and receives instruction input for generating the annotation image G2 (see Figure 7), and a voice communication device 62 that is positioned closer to the touch panel display device 61 than the scope-type display device 24 for voice communication with the voice communication devices 36 and 52. By providing the touch panel display device 61, bidirectional annotation can be performed between physician A1 and supervising physician A2 during surgery. In addition, by providing the voice communication device 62, physician A1 can also perform annotation while interacting with supervising physician A2 during surgery.

[0038] Furthermore, the control device 28 of the physician-side operating device 2 allows instruction input via the touch panel display device 61 when the sensor 27 detects that the device is in use, and prohibits instruction input via the touch panel display device 61 when the sensor 27 detects that the device is in use. In addition, the control device 43 of the information sharing device 4 allows instruction input via the touch panel display device 61 when the sensor 27 does not detect that the device is in use, and updates the annotation image G2 according to the instruction input from the touch panel display device 61, displaying it on the touch panel display device 51 and the scope-type display device 34.

[0039] Furthermore, in this embodiment, if the usage status is not detected by sensors 27 and 37, the control device 43 of the information sharing device 4 displays annotation images G2 based on instruction input from the touch panel display device 51 and annotation images G2 based on instruction input from the touch panel display device 61 on the touch panel display devices 51 and 61. This allows annotation images G2 by physician A1 and supervising physician A2 to be displayed on the touch panel display devices 51 and 61 when neither physician A1 nor supervising physician A2 is operating the surgical robot 1. As a result, annotation can be performed bidirectionally between physician A1 and supervising physician A2. If the usage status is not detected by sensors 27 and 37, instruction input from either the touch panel display devices 51 or 61 is permitted.

[0040] Furthermore, the control device 16 of the surgical robot 1 prohibits the operation of the manipulator arms 11 and 12 by the operating handle 21 when the sensor 27 does not detect an operating state, and permits the operation of the manipulator arms 11 and 12 by the operating handle 21 when the sensor 27 detects an operating state. The control device 16 prohibits the operation of the manipulator arms 11 and 12 by not accepting input from the operating handle 21 when the sensor 27 does not detect an operating state. The control device 16 also permits the operation of the manipulator arms 11 and 12 by accepting a predetermined input from the operating handle 21 when the sensor 27 detects an operating state.

[0041] The touch panel display device 61 and the voice communication device 62 are positioned near the physician's operating device 2a as annotation units independent of the physician's operating device 2a. The touch panel display device 61 is, for example, a flat panel display device with touch panel functionality. Physician A1 uses their finger or a touch panel pen to input instructions to the touch panel display device 61, thereby generating an annotation image G2 in a shape corresponding to the input. The annotation image G2 may be, for example, a line, a circle, or an arrow.

[0042] Furthermore, the voice communication device 62 includes a microphone 621 and a speaker 622. The microphone 621 receives voice input. The speaker 622 outputs voice. The voice communication device 62 is also integrally arranged with the touch panel display device 61. For example, the microphone 621 and speaker 622 of the voice communication device 62 are built into the touch panel display device 61. Note that in Figure 1, the touch panel display device 61 and the voice communication device 62 are shown separately.

[0043] Furthermore, the physician-side control device 2 (physician-side control device main unit 2a) is equipped with an interface 29 (see Figure 4) for connecting external devices. The touch panel display device 61 is also connected to the physician-side control device 2 (physician-side control device main unit 2a) via interface 29. Interface 29 is a terminal capable of inputting and outputting image data and audio data. The touch panel display device 61 is connected to interface 29 via a cable capable of inputting and outputting image data and audio data. The touch panel display device 61 and the audio communication device 62 input and output image data and audio data via interface 29.

[0044] As shown in Figures 1 and 4, the information sharing device 4 is a movable monitor cart. The information sharing device 4 includes a display device 41 for displaying endoscopic images G1 (see Figure 7) acquired by the endoscope 5 (see Figure 2), and a voice communication device 42 for voice communication with the voice communication device 26 of the physician's control device 2. The information sharing device 4 is positioned closer to the surgical robot 1 than the physician's control device 2. This allows assistant A3, positioned as surgical staff near the surgical robot 1, to view the endoscopic images G1 and interact with physician A1. The voice communication device 42 is also capable of voice communication with both voice communication device 36 and voice communication device 52. This allows the conversation between assistant A3 and physician A1, positioned as surgical staff near the surgical robot 1, to be heard by supervising physician A2 and clinical engineer A4. Assistant A3 can also interact with supervising physician A2 and clinical engineer A4. The voice communication device 42 also includes a microphone 421 and a speaker 422. The microphone 421 accepts voice input. The speaker 422 outputs sound. The display device 41 is, for example, a flat panel display device. Note that the display device 41 is an example of the "second display device" in the claims. The voice communication device 42 is an example of the "fourth voice communication device" in the claims.

[0045] Furthermore, a control device 43 is located within the information sharing device 4. The control device 43 performs image processing on the endoscopic image G1 and the annotation image G2. The control device 43 includes a processing unit such as a CPU that executes a program and a storage unit such as memory where the program is stored. The control device 43 is located in the first facility B1. As shown in Figures 6 and 7, the control device 43 overlays the annotation image G2 based on instruction input from the touch panel display device 51 received via the external network N and / or the annotation image G2 based on instruction input from the touch panel display device 61 onto the endoscopic image G1 and displays it on the scope-type display device 24. This makes it easy to perform annotation for physician A1.

[0046] The control device 43 displays the same image on each of the following displays: the scope-type display device 24, the scope-type display device 34, the display device 41, the touch panel display device 51, and the touch panel display device 61. In other words, the same endoscopic image G1, or an endoscopic image G1 with the same annotation image G2 superimposed, will be displayed on each of the following displays: the scope-type display device 24, the scope-type display device 34, the display device 41, the touch panel display device 51, and the touch panel display device 61.

[0047] In other words, when an instruction for annotation image G2 is given via the touch panel display device 51, the annotation image G2 is displayed on each display device (scope-type display device 24, scope-type display device 34, display device 41, touch panel display device 51, and touch panel display device 61) while being updated according to the instruction given via the touch panel display device 51. Also, if the usage status is detected by the sensor 37 and the display of annotation image G2 by the touch panel display device 51 is prohibited, the annotation image G2 is displayed on each display device without being updated by the touch panel display device 51. Furthermore, if the usage status is not detected by the sensor 27 and an instruction for annotation image G2 is given via the touch panel display device 61, the annotation image G2 is displayed on each display device while being updated according to the instruction given via the touch panel display device 61.

[0048] Furthermore, voice communication devices 26, 36, 42, 52, and 62 are capable of communicating with each other via voice.

[0049] Furthermore, in this embodiment, the physician-side operating device 2 and the supervising physician-side operating device 3 are equipped with a switching device for switching the operating authority for operating the surgical robot 1. This allows for easy switching of operating authority using the switching device at the desired time. In addition, in this embodiment, the switching device is located on the armrests 25 and 35. This allows the switching device to be placed in a position that is easy for physician A1 and supervising physician A2 to operate (a position where they can easily rest their hands). As a result, the operation of switching operating authority can be easily performed. The switching device is a touch panel 23 (33) located on the armrest 25 (35).

[0050] As shown in Figures 8 to 10, the physician-side control device 2 and the supervising physician-side control device 3 send a request to switch control authority from the person without control authority to the person with control authority. When the person with control authority approves the request, control authority is switched from the person with control authority to the person without control authority. This allows control authority to be switched only after obtaining approval from the person with control authority, thus preventing unintended changes in control authority.

[0051] As shown in Figure 8, when switching operating authority between the physician's operating device 2 and the supervising physician's operating device 3, the one of the two operating devices that does not have operating authority displays indicators 71 and 72 on the touch panel (23, 33) to allow the user to choose whether or not to send an operating authority switch request. If the indicator 71 for sending an operating authority switch request is selected on the touch panel (23, 33) by the one of the two operating devices that does not have operating authority, the user sends the operating authority switch request to the one of the two operating devices that does have operating authority via the external network N.

[0052] As shown in Figure 9, the person with operating authority over either the physician's control device 2 or the supervising physician's control device 3 notifies the system that a request to switch operating authority has been received. For example, the person with operating authority over either the physician's control device 2 or the supervising physician's control device 3 displays an endoscopic image G1 on the scope-type display device (24, 34) with a display 73 superimposed to notify the system that a request to switch operating authority has been received. The process of superimposing the display 73 onto the endoscopic image G1 is performed by the control device 43.

[0053] Furthermore, as shown in Figure 10, the person with operating authority over either the physician's control device 2 or the supervising physician's control device 3 displays indicators 74 and 75 on the touch panel (23, 33) to allow the user to choose whether or not to approve the request to switch operating authority. If the person with operating authority over either the physician's control device 2 or the supervising physician's control device 3 selects indicator 74 on the touch panel (23, 33) to approve the request to switch operating authority, they transfer the operating authority to the person who does not have operating authority over either the physician's control device 2 or the supervising physician's control device 3.

[0054] Furthermore, if the person with operating authority over one of the physician-side operating devices 2 or the supervising physician-side operating device 3 selects display 75 on the touch panel (23, 33) to reject the request to switch operating authority, the physician-side operating device 2 or the supervising physician-side operating device 3 that does not have operating authority will send a notification of rejection of the switch operating authority to the person without operating authority over one of the physician-side operating device 2 or the supervising physician-side operating device 3. In this case, the person without operating authority over one of the physician-side operating device 2 or the supervising physician-side operating device 3 will notify that they have received the notification of rejection of the switch operating authority. For example, the person without operating authority over one of the physician-side operating device 2 or the supervising physician-side operating device 3 will display an endoscopic image G1 on the scope-type display device (24, 34) with a display superimposed to notify that they have received the notification of rejection of the switch operating authority.

[0055] Furthermore, as shown in Figure 11, the physician-side control device 2 and the supervising physician-side control device 3 display an indicator 76 to show that they have operating authority when they do. For example, the physician-side control device 2 and the supervising physician-side control device 3 display the endoscopic image G1 with the indicator 76 superimposed on it on the scope-type display device (24, 34). The process of superimposing the indicator 76 onto the endoscopic image G1 is performed by the control device 43.

[0056] Referring to Figure 12, the control process for switching operating authority will be explained based on a flowchart. Here, we will describe an example where the supervising physician's operating device 3 has operating authority, and the operating authority is switched from the supervising physician's operating device 3 to the physician's operating device 2. Note that each process in the flowchart is performed by the control device 28 of the physician's operating device 2 or the control device 38 of the supervising physician's operating device 3.

[0057] As shown in Figure 12, in step S1, the touch panel 23 is operated, sending a request to switch control authority from the physician's control device 2 to the supervising physician's control device 3. In step S2, the supervising physician's control device 3 is notified that it has received the request to switch control authority. In step S3, the supervising physician's control device 3 decides whether to approve or reject the request to switch control authority. If the touch panel 33 is operated and the request to switch control authority is rejected, the process proceeds to step S4. In step S4, the supervising physician's control device 3 sends a notification to the physician's control device 2 that the request to switch control authority has been rejected. In step S5, the physician's control device 2 displays a notification that it has received the notification to reject the request to switch control authority. The control process then ends.

[0058] Furthermore, if, in step S3, the touch panel 33 is operated and the user chooses to approve the request to switch control authority, the process proceeds to step S6. In step S6, control authority is transferred from the supervising physician's control device 3 to the physician's control device 2. Then, in step S7, control authority is granted to the physician's control device 2. Finally, the control process is completed.

[0059] [Differentiation] It should be noted that the embodiments disclosed herein are illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims rather than by the description of the embodiments above, and further includes all modifications (exceptions) within the meaning and scope equivalent to the claims.

[0060] For example, in the above embodiment, an example was shown in which both the physician-side operating device and the supervising physician-side operating device are equipped with annotation units (touch panel display device and voice communication device), but the present invention is not limited to this. For example, only the supervising physician-side operating device may be equipped with annotation units (touch panel display device and voice communication device).

[0061] Furthermore, while the above embodiment shows an example where instruction input via the touch panel display device 51 and touch panel display device 61 is permitted when the usage status is not detected by sensors 37 and 27, the present invention is not limited thereto. For example, when the usage status is not detected by sensors 37 and 27, the system may switch between permitting or prohibiting instruction input via the touch panel display device 51 and touch panel display device 61 based on whether the supervising physician's operating device 3 or the physician's operating device 2 has the authority to operate.

[0062] Furthermore, although the above embodiment shows an example in which the control processing of the present invention is performed by multiple control devices, the present invention is not limited thereto. For example, the control processing of the present invention may be performed by a single control device.

[0063] Furthermore, although the above embodiment shows an example in which the supervising physician's operating device includes a voice communication device (third voice communication device) positioned closer to the touch panel display device than the scope-type display device, the present invention is not limited to this. For example, the supervising physician's operating device does not need to include a voice communication device (third voice communication device) positioned closer to the touch panel display device than the scope-type display device.

[0064] Furthermore, although the above embodiment shows an example in which a scope-type display device is provided as the first display device of the present invention, the present invention is not limited thereto. For example, a flat panel display device other than a scope-type display device may be provided as the first display device of the present invention.

[0065] Furthermore, while the above embodiments show examples in which the first and second voice communication devices of the present invention are voice communication devices integrally arranged with the scope-type display device, the present invention is not limited thereto. For example, the first and second voice communication devices of the present invention may be voice communication devices separate from the scope-type display device.

[0066] Furthermore, although the above embodiment shows an example in which a third voice communication device is integrally arranged with the touch panel display device as the third voice communication device of the present invention, the present invention is not limited thereto. For example, a separate voice communication device may be provided as the third voice communication device of the present invention. That is, a separate voice communication device including a microphone and a speaker may be provided as the third voice communication device of the present invention. Also, when a separate third voice communication device is provided from the touch panel display device, the third voice communication device and the touch panel display device may each be connected to the supervising physician's operating device via different interfaces.

[0067] Furthermore, although the above embodiment shows an example in which both the physician-side operating device and the supervising physician-side operating device are equipped with a switching device, the present invention is not limited to this. For example, only one of the physician-side operating device or the supervising physician-side operating device may be equipped with a switching device.

[0068] Furthermore, although the above embodiment shows an example where the switching device is a touch panel located on the armrest, the present invention is not limited to this. For example, the switching device may be a switch located on the armrest. Also, the switching device does not have to be located on the armrest. For example, the switching device may be a changeover switch provided on the first operating handle of the physician's operating device and / or the second operating handle of the supervising physician's operating device. In this case, the physician and / or supervising physician can easily switch operating authority using a switch at hand.

[0069] Furthermore, in the above embodiment, an example was shown in which an endoscopic image G1 with display 73 superimposed on it is displayed on a scope-type display device (24, 34) as a means of notifying that a request to switch operating authority has been received, but the present invention is not limited thereto. For example, notification can be made using sound or voice via an audio communication device as a means of notification.

[0070] Furthermore, while the above embodiment shows an example in which an information sharing device is equipped with a voice communication device, the present invention is not limited thereto. For example, a voice communication device may be provided in a surgical robot as an alternative to the voice communication device of the information sharing device. In the modified example shown in Figure 13, the surgical robot 1 is equipped with a voice communication device 18 for voice communication with the voice communication device 26 of the physician's control device 2. This also allows an assistant A3, positioned as surgical staff near the surgical robot 1, to communicate with physician A1. In another modified example, the voice communication device 18 is capable of voice communication with both the voice communication device 36 and the voice communication device 52. This also allows the conversation between assistant A3, positioned as surgical staff near the surgical robot 1, and physician A1 to be heard by supervising physician A2 and clinical engineer A4. Assistant A3 can also communicate with supervising physician A2 and clinical engineer A4. The voice communication device 18 also includes a microphone 181 and a speaker 182. The microphone 181 receives voice input. The speaker 182 outputs sound. The voice communication device 18 is an example of the "fifth voice communication device" in the claims.

[0071] Furthermore, although the above embodiment shows an example in which the supervising physician's control device 3 is installed in the second facility B2, the present invention is not limited thereto. In the present invention, a physician's control device and a surgical robot are placed in the second facility B2, and when the normal mode is selected from the mode selection, the surgical robot in the second facility B2 is operated by the physician's control device in the second facility B2, and when the remote mode is selected from the mode selection, the physician's control device in the second facility B2 is made to function as a supervising physician's control device, and the surgical robot in the first facility B1 is operated by the physician's control device in the second facility B2.

[0072] [Aspect] Those skilled in the art will understand that the exemplary embodiments described above are specific examples of the following embodiments.

[0073] (Item 1) A surgical robot installed in the first facility, equipped with a first manipulator arm for holding an endoscope and a second manipulator arm for holding surgical instruments, A physician-side operating device installed in the first facility for a physician to operate the first manipulator arm and the second manipulator arm, A second facility, different from the first facility, is installed, and includes a physician-side control device for a supervising physician to operate the surgical robot via an external network, It comprises one or more control devices, The physician-side operating device is, A first display device for displaying endoscopic images acquired by the aforementioned endoscope, First voice communication device and It comprises a first operating handle for operating the first and second manipulator arms, The aforementioned operating device on the supervising physician's side is, A first scope-type display device for displaying the endoscopic image transmitted via the external network, A first sensor for detecting whether the first scope-type display device is in a first usage state where it is being peered into, When the first operating state is detected by the first sensor, a second operating handle for the supervising physician to operate the first and second manipulator arms, A second voice communication device is positioned so that the supervising physician can communicate with the first voice communication device while looking into the first scope-type display device, The system includes a first touch panel display device that displays the endoscopic image transmitted via the external network and accepts instruction input for generating an annotation image, A remote surgical support system in which one or more control devices prohibit the input of instructions by the first touch panel display device when the first usage state is detected by the first sensor.

[0074] (Item 2) The remote surgical support system according to item 1, wherein the one or more control devices permit the instruction input by the first touch panel display device when the first usage state is not detected by the first sensor.

[0075] (Item 3) The remote surgical assistance system according to item 1 or 2, wherein one or more control devices prohibit the operation of the first and second manipulator arms by the second operating handle when the first operating state is not detected by the first sensor, and permit the operation of the first and second manipulator arms by the second operating handle when the first operating state is detected by the first sensor.

[0076] (Item 4) The remote surgical support system according to any one of items 1 to 3, wherein the one or more control devices superimpose the annotation image based on the instruction input received by the first touch panel display device onto the endoscopic image and display it on the first display device.

[0077] (Item 5) The remote surgical support system according to any one of items 1 to 4, wherein the physician-side operating device and / or the supervising physician-side operating device are equipped with a switching device for switching the operating authority for operating the surgical robot.

[0078] (Item 6) The remote surgical assistance system according to item 5, wherein the switching device is a changeover switch provided on the first operating handle and / or the second operating handle.

[0079] (Item 7) The remote surgical assistance system according to item 5, wherein the physician-side operating device and / or the supervising physician-side operating device are equipped with an armrest, and the switching device is located on the armrest.

[0080] (Item 8) The supervising physician's operating device includes a third voice communication device for communicating with the first voice communication device. The remote surgical assistance system according to any one of items 1 to 7, wherein the third voice communication device is located closer to the first touch panel display device than the first scope-type display device.

[0081] (Item 9) The remote surgical support system according to item 8, wherein the third voice communication device is integrally arranged with the first touch panel display device.

[0082] (Item 10) The aforementioned supervising physician's control device is equipped with an interface for connecting external devices, The remote surgical assistance system according to item 8 or 9, wherein the third voice communication device and / or the first touch panel display device are connected to the supervising physician's operating device via the interface.

[0083] (Item 11) A second display device for displaying the endoscopic image acquired by the endoscope, An information sharing device comprising a first voice communication device for the physician's operating device and a fourth voice communication device for voice communication is installed in the first facility. The remote surgical support system according to any one of items 1 to 10, wherein the information sharing device is positioned closer to the surgical robot than the physician's operating device.

[0084] (Item 12) The remote surgical support system according to any one of items 1 to 11, wherein the surgical robot comprises a fifth voice communication device for voice communication with the first voice communication device of the physician-side operating device.

[0085] (Item 13) The physician-side operating device is, The first display device is a second scope-type display device, A second sensor detects whether the second scope-type display device is in a second usage state where it is being peered into, The system includes a second touch panel display device that displays the endoscopic image and accepts instruction input for generating the annotation image, The remote surgical support system according to any one of items 1 to 12, wherein the one or more control devices prohibit the instruction input by the second touch panel display device when the second usage state is detected by the second sensor.

[0086] (Item 14) A surgical system including a physician-side operating device and a surgical robot installed at the first facility, and for operating the surgical robot via an external network, a supervising physician-side operating device installed at a second facility different from the first facility, The surgical robot comprises a first manipulator arm for holding an endoscope and a second manipulator arm for holding surgical instruments. The physician-side operating device comprises a first display device for displaying endoscopic images acquired by the endoscope, a first audio communication device, and a first operating handle for operating the first and second manipulator arms. The aforementioned operating device on the supervising physician's side is, A scope-type display device for displaying the endoscopic image transmitted via the external network, A sensor that detects whether the scope-type display device is in use and being peered into, When the usage state is detected by the sensor, a second operating handle for the supervising physician to operate the first and second manipulator arms, A second voice communication device is positioned so that the supervising physician can communicate with the first voice communication device while looking through the scope-type display device, The system includes a touch panel display device that displays the endoscopic image transmitted via the external network and accepts instruction input for generating an annotation image, The supervising physician's operating device is a supervising physician's operating device that prohibits the input of instructions via the touch panel display device when the usage status is detected by the sensor. [Explanation of Symbols]

[0087] 1. Surgical robot 2. Physician's Control Device 3 Instructor side operating device 4 Information sharing device 5 Endoscope 6 Surgical instruments 11. Manipulator arm (first manipulator arm) 12. Manipulator arm (second manipulator arm) 16 Control device (one or more control devices) 18. Voice communication device (5th voice communication device) 21 Operating handle (first operating handle) 23. Touch panel (switching device) 25 Armrest 24. Scope-type display devices (first display device, second scope-type display device) 26. Voice communication device (first voice communication device) 27 Sensor (Second Sensor) 28 Control device (one or more control devices) 31 Operating handle (second operating handle) 33. Touch panel (switching device) 34. Scope-type display device (First scope-type display device) 35 Armrest 36. Voice communication device (second voice communication device) 37 Sensor (1st sensor) 38 Control device (one or more control devices) 39 Interfaces 41 Display device (second display device) 42. Voice communication device (4th voice communication device) 43 Control device (one or more control devices) 51 Touch panel display device (first touch panel display device) 52. Voice communication device (third voice communication device) 61 Touch panel display device (second touch panel display device) 100 Remote surgical support systems A1 Doctor A2 Instructor B1 Facility 1 B2 Facility 2 G1 Endoscopic image G2 Annotation Images N External Network

Claims

1. A surgical robot installed in the first facility, equipped with a first manipulator arm for holding an endoscope and a second manipulator arm for holding surgical instruments, A physician-side operating device installed at the first facility comprises a first display device for displaying endoscopic images acquired by the endoscope, a first operating handle for a physician to operate the first manipulator arm and the second manipulator arm, and a first switching device for switching the operating authority for operating the first and second manipulator arms. A supervising physician's operating device installed at a second facility different from the first facility, comprising: a scope-type display device that displays the endoscopic image transmitted via an external network; a sensor that detects whether the scope-type display device is in use and being peered into; a second operating handle for the supervising physician to operate the first and second manipulator arms via the external network; a second switching device for switching the operating authority; and a touch panel display device that displays the endoscopic image transmitted via the external network and accepts instruction input for generating annotation images. A remote surgical assistance system comprising one or more control devices configured to prohibit the operation of the first and second manipulator arms by the second operating handle when the supervising physician's operating device has the operating authority and the sensor does not detect the usage state.

2. The remote surgical support system according to claim 1, wherein one or more control devices prohibit the operation of the first and second manipulator arms by the second operating handle regardless of the detection result of the sensor if the supervising physician's operating device does not have the operating authority.

3. The remote surgical assistance system according to claim 1, wherein one or more control devices permit the operation of the first and second manipulator arms by the second operating handle when the supervising physician's operating device has the authority to operate and the usage state is detected by the sensor.

4. The remote surgery support system according to claim 1, wherein the one or more control devices superimpose the annotation image based on the instruction input received by the touch panel display device onto the endoscopic image and display it on the first display device.

5. The remote surgical support system according to claim 1, wherein the first switching device is provided on the first operating handle, and the second switching device is provided on the second operating handle.

6. The remote surgical support system according to claim 1, wherein the physician-side operating device is equipped with a first armrest on which the first switching device is located, and the supervising physician-side operating device is equipped with a second armrest on which the second switching device is located.

7. The physician's operating device comprises a first voice communication device, The supervising physician's operating device comprises a second voice communication device positioned so that the supervising physician can communicate with the first voice communication device while looking into the scope-type display device, and a third voice communication device for communicating with the first voice communication device. The remote surgery support system according to claim 1, wherein the third voice communication device is located closer to the touch panel display device than the scope-type display device.

8. The remote surgery support system according to claim 7, wherein the third voice communication device is integrally arranged with the touch panel display device.

9. The aforementioned supervising physician's control device is equipped with an interface for connecting external devices, The remote surgical support system according to claim 7, wherein the third voice communication device and / or the touch panel display device are connected to the supervising physician's operating device via the interface.

10. The physician-side operating device comprises a first voice communication device, The supervising physician's operating device includes a second voice communication device positioned so that the supervising physician can communicate with the first voice communication device while looking through the scope-type display device. A second display device for displaying the endoscopic image acquired by the endoscope, An information sharing device comprising the first and second voice communication devices and a fourth voice communication device for performing voice communication is installed in the first facility. The remote surgical support system according to claim 1, wherein the information sharing device is positioned closer to the surgical robot than the physician-side operating device.

11. A surgical robot comprising a first manipulator arm for holding an endoscope and a second manipulator arm for holding surgical instruments; a physician-side operating device comprising a first display device for displaying endoscopic images acquired by the endoscope, a first operating handle for operating the first and second manipulator arms, and a first switching device for switching the operating authority for operating the first and second manipulator arms, wherein the supervising physician-side operating device is installed at a second facility different from the first facility in order to operate the first and second manipulator arms of a surgical system installed at a first facility via an external network, A scope-type display device that displays the endoscopic image transmitted via the external network, A sensor that detects whether the scope-type display device is in use and being peered into, A second operating handle used by the supervising physician to operate the first and second manipulator arms, A second switching device for switching the aforementioned operating authority, The system includes a touch panel display device that displays the endoscopic image transmitted via the external network and accepts instruction input for generating an annotation image, A supervising physician's operating device configured to prohibit the operation of the first and second manipulator arms by the second operating handle if the supervising physician has the aforementioned operating authority and the sensor does not detect the aforementioned usage state.

Images