Surgery system and method of operating a surgery system

Abstract

A surgical system is provided that can prevent a surgical instrument from unintentionally coming into contact with a site inside a patient's body outside the field of view of an endoscope. [Solution] This surgical system 100 (surgical system) includes arms 60a-60d for supporting an endoscope 6 and a medical instrument 4, a remote control device 2 that receives commands to operate the medical instrument 4, a touch panel 23 for setting an operation change mode, and a control unit 31. When the operation change mode is set and the tip position of the medical instrument 4 is outside the field of view of the endoscope 6, the control unit 31 changes the operation of the medical instrument 4 corresponding to the received command from the operation when the tip of the medical instrument 4 is within the field of view of the endoscope 6, and when the operation change mode is not set and the tip position of the medical instrument 4 is outside the field of view of the endoscope 6, the control unit 31 operates the medical instrument 4 without changing the operation of the medical instrument 4 corresponding to the command.

Description

[Technical Field]

[0001] The present invention relates to a surgical system and a method of operating a surgical system. [Background technology]

[0002] BACKGROUND ART Conventionally, a surgical system is known (see, for example, Patent Document 1).

[0003] The robotic surgery system disclosed in Patent Document 1 includes a plurality of manipulators that respectively support an endoscope and a tool (surgical instrument), an operating device that receives commands to operate the tool, and a control device that controls the operation of the tool. In the surgery system of Patent Document 1, the control device is configured to determine whether the tip position of the tool (surgical instrument) operated by the operating device is outside the field of view of the endoscope, and to display information about the tool that is outside the field of view as a symbol (sign) in a boundary area outside the display area of ​​the monitor screen. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-188574 Summary of the Invention [Problem to be solved by the invention]

[0005] In conventional surgical systems such as that described in Patent Document 1, even when the tip of a tool (surgical instrument) is outside the field of view, the surgeon can operate the tool using the operating device in the same way as when it is within the field of view. Therefore, if the tip of the tool (surgical instrument) is outside the field of view of the endoscope and cannot be visually confirmed, and the tool is operated in the same way as normal, there is a risk that the tip of the tool may unintentionally come into contact with a part inside the patient's body outside the field of view of the endoscope.

[0006] The present invention has been made to solve the above-mentioned problems, and one object of the present invention is to provide a surgical system and an operating method of the surgical system that can prevent a surgical instrument from unintentionally coming into contact with a part of the patient's body outside the field of view of the endoscope. [Means for solving the problem]

[0007] A surgical system according to a first aspect of the present invention comprises a first manipulator that supports an endoscope, a second manipulator that supports a surgical instrument, an operating device that receives commands to operate the surgical instrument, a mode setting device that sets an operation change mode, and a control device that includes one or more processors and controls the operation of the surgical instrument so that it operates in accordance with the commands received by the operating device. The control device determines whether the tip position of the surgical instrument is outside the field of view of the endoscope based on the relationship between the field of view of the endoscope and the tip position of the surgical instrument, and when the operation change mode is set by the mode setting device and the tip position of the surgical instrument is outside the field of view of the endoscope, changes the operation of the surgical instrument corresponding to the command received by the operating device to the operation when the tip of the surgical instrument is within the field of view of the endoscope, and when the operation change mode is not set by the mode setting device and the tip position of the surgical instrument is outside the field of view of the endoscope, operates the surgical instrument without changing the operation of the surgical instrument corresponding to the command.

[0008] In the surgical system according to the first aspect of the present invention, as described above, the control device determines whether the tip position of the surgical instrument is outside the field of view of the endoscope based on the relationship between the field of view of the endoscope and the tip position of the surgical instrument, and when the operation change mode is set and the tip position of the surgical instrument is outside the field of view of the endoscope, changes the operation of the surgical instrument corresponding to the command received by the operating device to the operation when the tip of the surgical instrument is within the field of view of the endoscope. This makes it possible to restrict the operation of the surgical instrument, such as by not operating the surgical instrument, when the tip position of the surgical instrument is outside the field of view of the endoscope and the tip of the surgical instrument cannot be visually confirmed, thereby preventing the surgical instrument from unintentionally coming into contact with a site inside the patient's body outside the field of view of the endoscope.

[0009] A method for operating a surgical system according to a second aspect of the present invention is a method for operating a surgical system executed by a control device provided in the surgical system, comprising the steps of: the control device receiving input of an operation change mode via a mode setting device; the control device determining whether the tip position of the surgical instrument is outside the field of view of the endoscope based on the relationship between the field of view of the endoscope supported by the first manipulator and the tip position of the surgical instrument supported by the second manipulator; and, when the control device has received input of the operation change mode via the mode setting device and the tip position of the surgical instrument is outside the field of view of the endoscope, the control device changing the operation of the surgical instrument corresponding to a command received by an operating device that receives commands to operate the surgical instrument to the operation when the tip of the surgical instrument is within the field of view of the endoscope; and when the control device has not received input of the operation change mode via the mode setting device and the tip position of the surgical instrument is outside the field of view of the endoscope, the control device operates the surgical instrument without changing the operation of the surgical instrument corresponding to the command.

[0010] In a second aspect of the present invention, a method for operating a surgical system includes a step of changing the operation of a surgical instrument corresponding to a command received by an operating device for operating the surgical instrument when the operation change mode is set and the tip position of the surgical instrument is outside the field of view of the endoscope, to the operation when the tip of the surgical instrument is within the field of view of the endoscope. This makes it possible to restrict the operation of the surgical instrument, such as by not operating the surgical instrument, when the tip position of the surgical instrument is outside the field of view of the endoscope and the tip cannot be visually confirmed, thereby preventing the surgical instrument from unintentionally coming into contact with a site inside the patient's body outside the field of view of the endoscope. [Effects of the Invention]

[0011] According to the present invention, it is possible to prevent a surgical instrument from unintentionally coming into contact with a site inside a patient's body outside the field of view of the endoscope. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing the configuration of a surgical operation system according to an embodiment. [Figure 2] 1A and 1B are diagrams illustrating a configuration of a medical manipulator according to an embodiment. [Figure 3] FIG. 1 is a diagram illustrating a configuration of an input device according to an embodiment. [Figure 4] 3A and 3B are diagrams illustrating a configuration of an operating handle according to an embodiment. [Figure 5] 1A and 1B are diagrams illustrating the configuration of a foot pedal according to an embodiment. [Figure 6] 1A and 1B are diagrams illustrating the configuration of an arm of a medical manipulator according to an embodiment. [Figure 7] FIG. [Figure 8] FIG. 2 is a perspective view showing the configuration of an operation unit of the medical manipulator according to the embodiment. [Figure 9] FIG. 1 is a diagram showing an endoscope. [Figure 10] FIG. 10 is a diagram showing a pivot position teaching tool. [Figure 11] FIG. 2 is a block diagram showing the configuration of a control unit of the medical manipulator according to the embodiment. [Figure 12] FIG. 1 shows an image captured by an endoscope and a graphical user interface. [Figure 13] FIG. 2 illustrates areas of a graphical user interface. [Figure 14] 10A and 10B are diagrams for explaining the setting of an operation change mode of a touch panel of a remote control device. [Figure 15] FIG. 10 is a diagram for explaining selection of an operation site in the input device of the medical cart. [Figure 16] FIG. 10 is a diagram for explaining a sign according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, specific embodiments of the present invention will be described with reference to the accompanying drawings.

[0014] 1 to 16, the configuration of a surgical system 100 according to an embodiment will be described. The surgical system 100 includes a medical manipulator 1, which is a patient-side device, and a remote control device 2, which is an operator-side device for operating the medical manipulator 1. The medical manipulator 1 includes a medical cart 3 and is configured to be movable. The remote control device 2 is located at a distance from the medical manipulator 1, and the medical manipulator 1 is configured to be remotely controlled by the remote control device 2. An operator (such as a doctor) inputs commands to the remote control device 2 to cause the medical manipulator 1 to perform a desired operation. The remote control device 2 transmits the input commands to the medical manipulator 1. The medical manipulator 1 operates based on the received commands. The medical manipulator 1 is located in an operating room, which is a sterilized sterile field. The surgical system 100 is an example of a "surgical system" defined in the claims.

[0015] The remote control device 2 is disposed independently of the medical manipulator 1. The remote control device 2 is disposed, for example, inside or outside an operating room. The remote control device 2 receives commands from an operator to operate the medical instrument 4. The remote control device 2 includes an operating handle 21, a foot pedal 22, a touch panel 23, a monitor 24, a support arm 25, and a support bar 26. The operating handle 21 constitutes an operating handle for an operator (such as a doctor) to input commands. The operating handle 21 and the foot pedal 22 are examples of an "operating device" in the claims. The touch panel 23 is an example of a "mode setting device" in the claims. The monitor 24 is an example of a "display device" in the claims.

[0016] The operating handle 21 is configured to operate the medical instrument 4. The operating handle 21 also receives an operating amount for the medical instrument 4. The operating handle 21 includes an operating handle 21L that is located on the left side as viewed from an operator (such as a doctor) and is operated with the operator's left hand, and an operating handle 21R that is located on the right side and is operated with the operator's right hand.

[0017] 4, the operating handle 21 includes link portion 21a, link portion 21b, link portion 21c, and link portion 21d operated by an operator (such as a doctor). Link portion 21a rotates around the A4 axis. When link portion 21a rotates around the A4 axis, an arm portion 61 (described later) rotates around the JT4 axis. Link portion 21b rotates around the A5 axis relative to link portion 21a. When link portion 21b rotates around the A5 axis, an arm portion 61 (described later) rotates around the JT5 axis. Link portion 21c rotates around the A6 axis relative to link portion 21b. When link portion 21c rotates around the A6 axis, an arm portion 61 rotates around the JT6 axis. Link portion 21d rotates around the A7 axis relative to link portion 21c. When the link portion 21d is rotated around the A7 axis, the arm portion 61 is rotated around the JT7 axis. The medical instrument 4 is an example of a "surgical instrument" in the claims.

[0018] Operating handle 21 includes a pair of grip members 21f that is opened and closed by the operator. Grip members 21f are made of elongated, plate-like lever members, and the proximal ends of the pair of grip members 21f are rotatably connected to proximal ends 21g of link portions 21d. Cylindrical finger insertion portions 21e are disposed on grip members 21f. The operator inserts his or her fingers into the pair of finger insertion portions 21e to operate operating handle 21. The base ends of the pair of grip members 21f are connected to link portion 21d, and the opening angle between jaw member 104a and jaw member 104b is changed by increasing or decreasing the angle between the pair of grip members 21f.

[0019] Furthermore, the operating handle 21 changes (scales) the movement amount of the arm 60 (medical instrument 4) according to the operation amount received by the operating handle 21. In other words, the medical instrument 4 is set to move in accordance with a movement command at a predetermined scaling. For example, when the magnification of the movement amount is set to 1 / 2, the medical instrument 4 is controlled to move a distance that is 1 / 2 of the movement distance of the operating handle 21. This allows delicate surgery to be performed accurately.

[0020] As shown in FIG. 5 , a plurality of foot pedals 22 are provided to perform functions related to the medical instrument 4. The plurality of foot pedals 22 are arranged on a base 28. The foot pedals 22 include a switching pedal 22a, a clutch pedal 22b, a camera pedal 22c, an incision pedal 22d, and a coagulation pedal 22e. The switching pedal 22a, the clutch pedal 22b, the camera pedal 22c, the incision pedal 22d, and the coagulation pedal 22e are operated by the operator's feet. The incision pedal 22d includes a incision pedal 22dR for the right arm 60 and a incision pedal 22dL for the left arm 60. The coagulation pedal 22e includes a coagulation pedal 22eR for the right arm 60 and a coagulation pedal 22eL for the left arm 60.

[0021] The switching pedal 22a is configured to switch the arm 60 operated by the operating handle 21. The clutch pedal 22b is configured to perform a clutch operation that temporarily disconnects the operational connection between the arm 60 and the operating handle 21. While the clutch pedal 22b is depressed by the operator, the operation by the operating handle 21 is not transmitted to the arm 60.

[0022] The camera pedal 22c is operated to cause the two operating handles 21 to move the endoscope 6. Specifically, the camera pedal 22c is provided for inputting a command to move the endoscope 6. Specifically, the camera pedal 22c is depressed by the operator, thereby inputting a command to move the endoscope 6. Furthermore, when a command to enable movement of the endoscope 6 is input by the camera pedal 22c (that is, while the camera pedal 22c is depressed by the operator), the endoscope 6 is configured to move by operating both the operating handles 21R and 21L.

[0023] While the incision pedal 22d (coagulation pedal 22e) is being depressed by the operator, a high frequency current for incising or coagulating tissue flows from an electrosurgical device (not shown) to the medical instrument 4 (electric scalpel).

[0024] As shown in FIG. 1, the monitor 24 is a scope-type display device for displaying an endoscopic image (see FIG. 12) captured by the endoscope 6. A support arm 25 supports the monitor 24 so that the height of the monitor 24 is adjusted to the height of the face of an operator (such as a doctor). The touch panel 23 is disposed on a support bar 26. The medical manipulator 1 can be operated by the remote control device 2 when a sensor (not shown) provided near the monitor 24 detects the operator's head. The operator operates the operation handle 21 and the foot pedal 22 while visually checking the affected area on the monitor 24. This inputs commands to the remote control device 2. The commands input to the remote control device 2 are transmitted to the medical manipulator 1.

[0025] The medical cart 3 is provided with a control unit 31 that controls the operation of the medical manipulator 1 and a storage unit 32 that stores programs and the like for controlling the operation of the medical manipulator 1. The control unit 31 controls the operation of the medical instrument 4 so that it operates in accordance with commands received by the operating handle 21, the foot pedal 22, or the arm operating unit 80. Specifically, the control unit 31 of the medical cart 3 controls the operation of the medical manipulator 1 based on commands input to the remote control device 2. The control unit 31 is an example of a "control device" in the claims.

[0026] The medical cart 3 is also provided with an input device 33. The input device 33 is configured to receive operations for moving and changing the posture of the positioner 40, arm base 50, and multiple arms 60, mainly for preparing for surgery before the procedure. The input device 33 is also configured to receive input of information necessary for preparing for surgery. The input device 33 is an example of the "operative site input device" in the claims.

[0027] As shown in FIG. 3, the input device 33 of the medical cart 3 is provided with a display unit 33a consisting of a touch panel display, and a joystick 33b for controlling the movement of the positioner 40 is provided near the display unit 33a. The positioner 40 can be operated three-dimensionally by selecting an operation mode displayed on the display unit 33a using the touch panel and operating the joystick 33b. An enable switch 33c for permitting or prohibiting the movement of the positioner 40 is provided near the joystick 33b. When the enable switch 33c is pressed down to permit the movement of the positioner 40, the positioner 40 is moved by operating the joystick 33b. The touch panel of the display unit 33a also allows the selection of the surgical site (anatomy: "prostate", etc., see FIG. 15) and the insertion direction of the medical manipulator 1 relative to the patient P (e.g., "from the right side").

[0028] The medical manipulator 1 shown in FIGS. 1 and 2 is placed in an operating room. The medical manipulator 1 includes a medical cart 3, a positioner 40, an arm base 50, and multiple arms 60. The arm base 50 is attached to the tip of the positioner 40. The arm base 50 has a relatively long rod shape (long shape). The base of each of the multiple arms 60 is attached to the arm base 50. The multiple arms 60 are configured to be able to take a folded position (storage position). The arm base 50 and the multiple arms 60 are used covered with a sterile drape (not shown). The arms 60 support a medical instrument 4.

[0029] The positioner 40 is configured, for example, by a seven-axis articulated robot. The positioner 40 is placed on the medical cart 3. The positioner 40 moves the arm base 50. Specifically, the positioner 40 is configured to move the position of the arm base 50 in three dimensions.

[0030] The positioner 40 also includes a base portion 41 and a plurality of link portions 42 connected to the base portion 41. The plurality of link portions 42 are connected to each other by joint portions 43.

[0031] 1, a medical instrument 4 is attached to the tip of each of the multiple arms 60. The medical instrument 4 includes, for example, replaceable instruments, an endoscope 6 (see FIG. 9) for capturing an image of the surgical site, and the like.

[0032] As shown in FIG. 1, the surgical system 100 is also provided with a monitor cart 8. The monitor cart 8 is provided with a display unit 8a. The display unit 8a is provided separately from the monitor 24 of the remote control device 2. The monitor cart 8 is also provided with an image processing device 8b. The display unit 8a of the monitor cart 8 displays the same image as the image displayed on the monitor 24 of the remote control device 2. In other words, the image displayed on the monitor 24 and viewed by the surgeon can be viewed by workers (nurses, assistants, etc.) near the medical manipulator 1 and the patient P on the display unit 8a of the monitor cart 8. The image processing device 8b is an example of a "control device" in the claims. The display unit 8a is an example of a "display device" in the claims.

[0033] 6, the instrument is provided with a driven unit 4a that is driven by a servo motor M2 provided on a holder 71 of the arm 60. Furthermore, a forceps 4b is provided at the tip of the instrument.

[0034] As shown in FIG. 7 , the instrument includes a first support 4e that supports the proximal ends of jaw members 104a and 104b at the distal end so that they can rotate about the JT11 axis; a second support 4f that supports the proximal end of first support 4e at the distal end so that they can rotate about the JT10 axis; and a shaft 4c connected to the proximal end of second support 4f. Jaw members 104a and 104b are openable and closable. Driven unit 4a, shaft 4c, second support 4f, first support 4e, and forceps 4b are arranged along the Z direction. The JT11 axis is perpendicular to the direction in which shaft 4c extends (Z direction). The JT10 axis is spaced from the JT11 axis in the direction in which shaft 4c extends and is perpendicular to the direction in which shaft 4c extends and the JT11 axis.

[0035] The forceps 4b are attached to the first support 4e so as to rotate about the axis of the JT11 shaft. The second support 4f supports the first support 4e rotatably about the JT10 shaft. That is, the first support 4e is attached to the second support 4f so as to rotate about the axis of the JT10 shaft. The tip end (Z1 direction side) of the first support 4e has a U-shape. A tool center point (TCP1, clevis) is set in the center of the axis of the JT11 shaft of the tip end of the U-shape of the first support 4e.

[0036] The medical instrument 4 (forceps 4b) also includes a JT9 axis as the rotation axis of the shaft 4c (the axis along the extension direction of the shaft 4c) and a JT12 axis as the opening / closing axis of the forceps 4b. Note that a plurality of (for example, four) servo motors M2 are provided on the holder 71 of the arm 60, and the rotating body of the driven unit 4a is driven by the plurality of servo motors M2. As a result, the medical instrument 4 is driven around the J9 to J12 axes.

[0037] As shown in FIG. 9, the TCP2 of the endoscope 6 is set at the tip of the endoscope 6.

[0038] Next, the configuration of the arm 60 will be described in detail.

[0039] As shown in Fig. 6, the arm 60 includes an arm section 61 (a base section 62, a link section 63, and a joint section 64) and a translational movement mechanism section 70 provided at the tip of the arm section 61. The arm 60 is configured to move the tip side of the arm 60 in three dimensions relative to the base side (arm base 50) of the arm 60. The arm section 61 is also configured as a seven-axis articulated robot arm. The multiple arms 60 have similar configurations.

[0040] As shown in Fig. 6, the arm 60 has axes JT1 to JT7 as rotation axes and axis J8 as a linear motion axis. The axes JT1 to JT7 correspond to the rotation axes of the joint 64 of the arm section 61. The JT7 axis corresponds to the base end link section 72 of the translational movement mechanism 70. The JT8 axis corresponds to an axis that moves the tip end link section 73 of the translational movement mechanism 70 relative to the base end link section 72 in the Z direction. That is, the servo motor M1 shown in Fig. 11 is provided to correspond to the JT1 to JT7 axes of the arm 60. The servo motor M3 is provided to correspond to the JT8 axis.

[0041] The translational movement mechanism 70 is provided on the distal end side of the arm 61, and has the medical instrument 4 attached thereto. The translational movement mechanism 70 translates the medical instrument 4 in the direction of insertion into the patient P. The translational movement mechanism 70 is configured to translate the medical instrument 4 relative to the arm 61. Specifically, the translational movement mechanism 70 is provided with a holder 71 that holds the medical instrument 4. The holder 71 houses a servo motor M2 (see FIG. 11).

[0042] As shown in FIG. 8 , the medical manipulator 1 is attached to the arm 60 and includes an arm operation unit 80 that operates the arm 60. The arm operation unit 80 receives commands from an operator to operate the medical instrument 4. The arm operation unit 80 includes an enable switch 81, a joystick 82, and a switch unit 83. The enable switch 81 permits or prohibits movement of the arm 60 using the joystick 82 and the switch unit 83. When an operator (such as a nurse or an assistant) holds and presses the arm operation unit 80, the enable switch 81 enters a state that permits movement of the medical instrument 4 by the arm 60. The joystick 82 and the switch unit 83 are used to operate the arm 60. The arm operation unit 80 is an example of an "operation device" in the claims.

[0043] The switch unit 83 also includes a switch unit 83a that moves the medical instrument 4 in the direction along the longitudinal direction of the medical instrument 4 toward the direction in which the medical instrument 4 is inserted into the patient P, and a switch unit 83b that moves the medical instrument 4 in the direction opposite to the direction in which the medical instrument 4 is inserted into the patient P. Both the switch unit 83a and the switch unit 83b are configured as push button switches.

[0044] 8, the arm operating unit 80 includes a pivot button 85 that sets a pivot position PP, which serves as a fulcrum for movement of the medical instrument 4 attached to the arm 60. The pivot button 85 is provided on a surface 80b of the arm operating unit 80 adjacent to the enable switch 81. When the tip of the endoscope 6 (see FIG. 9) or the pivot position teaching instrument 7 (FIG. 10) is moved to a position corresponding to the insertion position of the trocar T inserted into the body surface S of the patient P, the pivot button 85 is pressed to set the pivot position PP, which is then stored in the memory unit 32. Note that when setting the pivot position PP, the pivot position PP is set as a single point (coordinates).

[0045] As shown in FIG. 1 , an endoscope 6 is attached to one arm 60 (for example, arm 60c) of the multiple arms 60, and medical instruments 4 other than the endoscope 6 are attached to the remaining arms 60 (for example, arms 60a, 60b, and 60d). Specifically, during surgery, an endoscope 6 is attached to one arm 60 of the four arms 60, and medical instruments 4 other than the endoscope 6 (such as forceps 4b) are attached to three arms 60. A pivot position PP is set for the arm 60 to which the endoscope 6 is attached, with the endoscope 6 attached. Furthermore, a pivot position PP is set for the arm 60 to which the medical instrument 4 other than the endoscope 6 is attached, with the pivot position teaching instrument 7 attached. The endoscope 6 is attached to one of the two arms 60 (arms 60b and 60c) located in the center of the four arms 60 arranged adjacent to each other. That is, the pivot position PP is set individually for each of the multiple arms 60. The arm 60c is an example of a "first manipulator" in the claims, and the arms 60a, 60b, and 60d are an example of a "second manipulator" in the claims.

[0046] 11, the arm 60 is provided with a plurality of servo motors M1, an encoder E1, and a reducer (not shown) so as to correspond to the plurality of joints 64 of the arm section 61. The encoder E1 is configured to detect the rotation angle of the servo motor M1. The reducer is configured to reduce the rotation speed of the servo motor M1 to increase the torque.

[0047] 11, the translational movement mechanism 70 is provided with a servo motor M2 for rotating a rotor provided in the driven unit 4a of the medical instrument 4, a servo motor M3 for translationally moving the medical instrument 4, encoders E2 and E3, and a reducer (not shown). The encoders E2 and E3 are configured to detect the rotation angles of the servo motors M2 and M3, respectively. The reducers are configured to decelerate the rotation of the servo motors M2 and M3 to increase the torque.

[0048] The positioner 40 is also provided with a plurality of servo motors M4, an encoder E4, and a reducer (not shown) to correspond to the plurality of joints 43 of the positioner 40. The encoder E4 is configured to detect the rotation angle of the servo motor M4. The reducer is configured to reduce the rotation speed of the servo motor M4 to increase the torque.

[0049] The medical cart 3 is also provided with a servomotor M5, an encoder E5, a reducer, and a brake (not shown) that drive each of the multiple front wheels (not shown) of the medical cart 3. The reducer is configured to reduce the rotation speed of the servomotor M5 and increase the torque. A potentiometer P1 (see FIG. 1) is provided on the throttle unit 34a of the medical cart 3, and the servomotor M5 of the front wheels is driven based on the rotation angle detected by the potentiometer P1 in response to the twist of the throttle unit 34a. The rear wheels (not shown) of the medical cart 3 are dual-wheel type, and are steered based on the left and right rotation of the operating handle 34. A potentiometer P2 (see FIG. 2) is provided on the operating handle 34 of the medical cart 3, and the rear wheels of the medical cart 3 are provided with a servomotor M6, an encoder E6, and a reducer (not shown). The reducer is configured to reduce the rotation speed of the servomotor M6 and increase the torque. The servo motor M6 is driven based on the rotation angle detected by the potentiometer P2 in response to the left and right rotation of the operating handle 34. In other words, the steering of the rear wheels caused by the left and right rotation of the operating handle 34 is configured to be power-assisted by the servo motor M6.

[0050] The front wheels of the medical cart 3 are driven to move forward and backward, and the rear wheels are steered by turning the operating handle 34 of the medical cart 3, causing the medical cart 3 to turn left and right.

[0051] The control unit 31 of the medical cart 3 includes an arm control unit 31a that controls the movement of the multiple arms 60 based on commands, and a positioner control unit 31b that controls the movement of the positioner 40 and the drive of the front wheels (not shown) of the medical cart 3 based on commands. A servo control unit C1 that controls a servo motor M1 that drives the arm 60 is electrically connected to the arm control unit 31a. An encoder E1 that detects the rotation angle of the servo motor M1 is also electrically connected to the servo control unit C1.

[0052] The arm control unit 31a is also electrically connected to a servo control unit C2 for controlling a servo motor M2 for driving the medical instrument 4. The servo control unit C2 is also electrically connected to an encoder E2 for detecting the rotation angle of the servo motor M2. The arm control unit 31a is also electrically connected to a servo control unit C3 for controlling a servo motor M3 for translationally moving the translational movement mechanism 70. The servo control unit C3 is also electrically connected to an encoder E3 for detecting the rotation angle of the servo motor M3.

[0053] The operation command input to the remote control device 2 is input to the arm control unit 31a. The arm control unit 31a generates a position command based on the input operation command and the rotation angle detected by the encoder E1 (E2, E3), and outputs the position command to the servo control unit C1 (C2, C3). The servo control unit C1 (C2, C3) generates a torque command based on the position command input from the arm control unit 31a and the rotation angle detected by the encoder E1 (E2, E3), and outputs the torque command to the servo motor M1 (M2, M3). This causes the arm 60 to move in accordance with the operation command input to the remote control device 2. The control unit 31 also controls the arm 60c so that the endoscope 6 moves from the pivot position PP to the fulcrum. The control unit 31 also controls the arm 60a (60b, 60d) so that the medical instrument 4 moves from the pivot position PP to the fulcrum.

[0054] 11, the control unit 31 (arm control unit 31a) is configured to operate the arm 60 based on an input signal from a joystick 82 of the arm operation unit 80. Specifically, the arm control unit 31a generates a position command based on the input signal (operation command) input from the joystick 82 and the rotation angle detected by the encoder E1, and outputs the position command to the servo control unit C1. The servo control unit C1 generates a torque command based on the position command input from the arm control unit 31a and the rotation angle detected by the encoder E1, and outputs the torque command to the servo motor M1. As a result, the arm 60 is moved in accordance with the operation command input to the joystick 82.

[0055] The control unit 31 (arm control unit 31a) is configured to operate the arm 60 based on an input signal from a switch unit 83 of the arm operation unit 80. Specifically, the arm control unit 31a generates a position command based on the input signal (operation command) input from the switch unit 83 and the rotation angle detected by the encoder E1 or E3, and outputs the position command to the servo control unit C1 or C3. The servo control unit C1 or C3 generates a torque command based on the position command input from the arm control unit 31a and the rotation angle detected by the encoder E1 or E3, and outputs the torque command to the servo motor M1 or M3. As a result, the arm 60 is moved in accordance with the operation command input to the switch unit 83.

[0056] 11, the positioner control unit 31b is electrically connected to a servo control unit C4 for controlling a servo motor M4 that moves the positioner 40. The servo control unit C4 is also electrically connected to an encoder E4 for detecting the rotation angle of the servo motor M4. The positioner control unit 31b is also electrically connected to a servo control unit C5 for controlling a servo motor M5 that drives the front wheels (not shown) of the medical cart 3. The servo control unit C5 is also electrically connected to an encoder E5 for detecting the rotation angle of the servo motor M5.

[0057] Furthermore, an operation command related to setting a standby position or the like is input from the input device 33 to the positioner control unit 31b. The positioner control unit 31b generates a position command based on the operation command input from the input device 33 and the rotation angle detected by the encoder E4, and outputs the position command to the servo control unit C4. The servo control unit C4 generates a torque command based on the position command input from the positioner control unit 31b and the rotation angle detected by the encoder E4, and outputs the torque command to the servo motor M4. This causes the positioner 40 to move in accordance with the operation command input to the input device 33. Similarly, the positioner control unit 31b moves the medical cart 3 based on the operation command from the input device 33.

[0058] The surgical system 100 includes an image processing device 8b. The image processing device 8b executes processing based on a predetermined program. The image processing device 8b is configured with a computer. The image processing device 8b includes a processing unit such as a CPU that executes the program and a storage unit such as a memory in which the program is stored. The image processing device 8b is configured to generate a graphical user interface G (see FIG. 13) and display the graphical user interface G on the monitor 24 of the remote control device 2, superimposed on an endoscopic image (see FIG. 12) captured by the endoscope 6. The image processing device 8b is also configured to display the graphical user interface G on the display unit 8a, superimposed on an endoscopic image captured by the endoscope 6. The image processing device 8b is also configured to capture an image from the endoscope 6. The image processing device 8b is also configured to be able to communicate with the control unit 31.

[0059] 13, the graphical user interface G includes a clutch area G1, which indicates the state of the clutch pedal 22b.

[0060] 13, in this embodiment, the graphical user interface G includes a camera area G2 that displays information related to the endoscope 6. The camera area G2 is displayed in an area near the lower end ed (see FIG. 12) of the screen gr of the monitor 24.

[0061] As shown in FIG. 13, the graphical user interface G includes a hand area G3. The hand area G3 displays information about each medical instrument 4, each arm 60, and the status of the coagulation pedal 22e and the incision pedal 22d. The hand area G3 includes a hand area G3a displaying information about the medical instrument 4 and arm 60a (numbered "4" in the hand area G3) operated by the left-hand operating handle 21L, a hand area G3b displaying information about the replacement medical instrument 4 and arm 60b (numbered "3" in the hand area G3), and a hand area G3c displaying information about the medical instrument 4 and arm 60d (numbered "1" in the hand area G3) operated by the left-hand operating handle 21L. The hand area G3 is displayed in an area near the lower end ed of the screen gr of the monitor 24. The clutch area G1 is also displayed in an area near the lower end ed of the screen gr. Specifically, as shown in FIG. 13, the clutch area G1, camera area G2, and hand area G3 are displayed between the lower end ed of the screen gr and a position 1 / 10 (L11) of the vertical length of the screen gr from the end ed.

[0062] In addition, the information about the arm 60 includes the arm number (arm number such as "1" or "2") and an arrow icon that is displayed when the arm 60 is set as the replacement arm for the attached medical instrument 4, the information about the medical instrument 4 includes the name of the medical instrument 4, and the information about the status of the coagulation pedal 22e and the incision pedal 22d includes the operation status of the clutch pedal 22b, the operation status of the incision pedal 22d, and the operation status of the coagulation pedal 22e.

[0063] 12, the hand areas (hand area G3a and hand area G3c) of the arm 60 to be operated are displayed in dark gray with a white oval inside and the number of the arm 60 (1 and 4) displayed inside the white oval. The hand area (hand area G3b) of the arm 60 not to be operated is displayed in light gray with the number of the arm 60 (3) displayed in even lighter gray.

[0064] As shown in FIG. 13, the graphical user interface G includes a medical instrument status information area G4, which is a pop-up area. The medical instrument status information area G4 displays the current number of uses / maximum number of uses of the medical instrument 4 attached to each arm 60 in a pop-up. When the current number of uses equals the maximum number of uses, the current number of uses is displayed in red. When an error occurs in the medical instrument 4 attached to each arm 60, error information is displayed in a pop-up. When no medical instrument 4 is attached to the arm 60, nothing is displayed in the medical instrument status information area G4. The medical instrument status information area G4 is displayed in an area adjacent to and above the clutch area G1, camera area G2, and hand area G3 on the monitor 24.

[0065] 13, the graphical user interface G includes a spirit level area G5. In the spirit level area G5, angle information of the endoscope 6 is displayed. Furthermore, the spirit level area G5 is displayed only while the camera pedal 22c is being depressed. That is, when the image processing device 8b receives a command to enable movement of the endoscope 6, it displays the spirit level LV of the endoscope 6 in the spirit level area G5.

[0066] 13, the graphical user interface G includes a left pop-up area G6. In the left pop-up area G6, an icon representing the foot pedal 22 on which the foot is placed is displayed when the left pop-up area G6 is in a hover state with the foot placed on the foot pedal 22. The left pop-up area G6 is displayed on the left side of the monitor 24.

[0067] 13, the graphical user interface G includes a right pop-up area G7. Icons are displayed in the right pop-up area G7 when the feet are placed on the coagulation pedal 22eR and the incision pedal 22dR. The right pop-up area G7 is displayed on the right side of the monitor 24.

[0068] 13, the graphical user interface G includes a first area G8 that displays the operable range of the arm 60 and the operable range of the arm 60 that can be operated by the operating handle 21. The graphical user interface G also includes a second area G9 that displays the operating direction of the operating handle 21 required to return the operating handle 21 to the operable range and / or (in this embodiment, "and") to return the arm 60 to the operable range.

[0069] There are two arms 60 that can be operated by the operating handle 21. For example, the left arm 60L (e.g., arm 60a, see FIG. 1) for supporting the medical instrument 4 is operated by the operating handle 21L. The right arm 60R (e.g., arm 60d, see FIG. 1) for supporting the medical instrument 4 is operated by the operating handle 21R. The first area G8 (first area G8L, first area G8R) and the second area G9 (second area G9L, second area G9R) are provided separately for the left arm 60L and the right arm 60R, respectively.

[0070] 13, the graphical user interface G also includes an error notification area G15 (G15a, G15b). The error notification area G15a pops up to display warning or error information when a warning or error occurs. The error notification area G15b pops up to display detailed notes about the warning or error displayed in the error notification area G15a.

[0071] As shown in Fig. 13, the graphical user interface G includes a status area G10. In the status area G10, information such as the remaining charge of the built-in battery of the medical manipulator 1, the brightness / contrast of the monitor 24, lap time, and elapsed time of surgery is displayed. Specifically, as shown in Fig. 13, the status area G10 is displayed between the upper end eu of the screen gr and a position 1 / 10 (L11) of the vertical length of the screen gr from the end eu.

[0072] Here, the control unit 31 determines whether the tip position of the medical instrument 4 is outside the field of view of the endoscope 6 based on the relationship between the field of view of the endoscope 6 and the tip position of the medical instrument 4. Specifically, the control unit 31 is configured to determine whether each medical instrument 4 is located outside the field of view of the endoscope 6 based on imaging range information of the endoscope 6 and position information of the tip of each of the multiple medical instruments 4 supported by the multiple arms 60. That is, the control unit 31 acquires the position of the medical instrument 4 based on the attitude and position of the arm 60. The control unit 31 also acquires the imaging direction of the endoscope 6 based on the attitude and position of the arm 60. The control unit 31 also acquires the angle of view (field of view range) of the endoscope 6 based on the zoom state of the endoscope 6. The control unit 31 acquires the angle of view (field of view range) of the endoscope 6 using values ​​set as the mechanical mechanism (lens, etc.) of the endoscope 6. Then, the control unit 31 acquires the coordinates of the tip of the medical instrument 4 relative to the field of view of the endoscope 6 based on the field of view of the endoscope 6, the attitude and position of the endoscope 6, and the position information of the arm 60. As a result, the control unit 31 determines whether the medical instrument 4 is located outside the field of view of the endoscope 6 or not.

[0073] In addition, when the tip position of the medical instrument 4 is outside the field of view of the endoscope 6, the control unit 31 changes the operation of the medical instrument 4 corresponding to the command received by the operating handle 21, foot pedal 22 (incision pedal 22d or coagulation pedal 22e) or arm operating unit 80 to the operation when the tip of the medical instrument 4 is within the field of view of the endoscope 6.

[0074] Specifically, the commands received by the operating handle 21 or the arm operating unit 80 include a movement command to move the medical instrument 4. Furthermore, the commands received by the incision pedal 22d or the coagulation pedal 22e include an energization command to supply electrical energy to the medical instrument 4. Furthermore, the commands received by the pair of grip members 21f of the operating handle 21 include an opening / closing command to open or close the jaws at the tip of the medical instrument 4.

[0075] Furthermore, when the tip position of the medical instrument 4 changes from a state where it is outside the field of view of the endoscope 6 to a state where it is within the field of view, the control unit 31 cancels the process of changing the operation of the medical instrument 4 corresponding to the command received by the operating handle 21, the foot pedal 22, or the arm operating unit 80. That is, the control unit 31 restricts the operation of the medical instrument 4 when the tip position is outside the field of view of the endoscope 6 due to movement of the medical instrument 4 or a change in the field of view of the endoscope 6, and cancels the restriction on the operation of the medical instrument 4 when the tip position of the medical instrument 4 is within the field of view of the endoscope 6 due to movement of the medical instrument 4 or a change in the field of view of the endoscope 6. Specifically, with the camera pedal 22c depressed by the operator, the endoscope 6 is moved by operating both the operating handles 21R and 21L so that the tip position of the medical instrument 4, which is outside the field of view, comes within the field of view.

[0076] As an example of this embodiment, when the tip position of the medical instrument 4 is outside the field of view of the endoscope 6, the control unit 31 controls the arms 60a, 60b, and 60d so that the medical instrument 4 does not move in response to a command to move the medical instrument 4. In other words, when the tip position of the medical instrument 4 is outside the field of view of the endoscope 6 due to the movement of the medical instrument 4, the control unit 31 does not move the medical instrument 4 that is outside the field of view, even if an attempt is made to move the medical instrument 4 that is outside the field of view by operating the operating handle 21.

[0077] As another example of the present embodiment, when the tip position of the medical instrument 4 is outside the visual field of the endoscope 6, the control unit 31 controls the arms 60a, 60b, and 60d to move the medical instrument 4 at a scaling smaller than a predetermined scaling in response to a movement command. For example, when the predetermined scaling is set to 1 / m times, the scaling for moving the medical instrument 4 outside the visual field is set to 1 / n times (where m < n).

[0078] As another example of the present embodiment, when the tip position of the medical instrument 4 is outside the visual field of the endoscope 6, the control unit 31 controls the energization of the medical instrument 4 so as not to supply electrical energy in response to an energization command. For example, even when a cutting or coagulation energization command is given to the medical instrument 4 outside the visual field of the endoscope 6, the control unit 31 does not energize the medical instrument 4 outside the visual field.

[0079] As another example of the present embodiment, when the tip position of the medical instrument 4 is outside the visual field of the endoscope 6, the control unit 31 controls the operation of the jaw so as not to open or close the jaw of the medical instrument 4 in response to an opening / closing command.

[0080] As shown in FIG. 14, the touch panel 23 is used to set an operation change mode. As another example of the present embodiment, when the operation change mode is turned on, the operation of the medical instrument 4 outside the visual field of the endoscope 6 is restricted. Also, when the operation change mode is turned off, the operation of the medical instrument 4 is not restricted even if it is outside the visual field of the endoscope 6.

[0081] That is, when the operation change mode is set by the touch panel 23, the control unit 31 changes the operation of the medical instrument 4 corresponding to a command when the tip position of the medical instrument 4 is outside the visual field of the endoscope 6. Also, when the operation change mode is not set by the touch panel 23, the control unit 31 operates the medical instrument 4 without changing the operation of the medical instrument 4 corresponding to a command when the tip position of the medical instrument 4 is outside the visual field of the endoscope 6.

[0082] 15, the input device 33 is used to receive input of the surgical site, and the medical manipulator 1 is controlled to perform an appropriate operation according to the selected surgical site.

[0083] As another example of this embodiment, when the first surgical site is set by the input device 33, the control unit 31 changes the operation of the medical instrument 4 corresponding to the command when the distal end position of the medical instrument 4 is outside the field of view of the endoscope 6. For example, the first surgical site is a region such as the lungs, which is close to blood vessels such as the aorta and vena cava. When the first surgical site is selected, the operation of the medical instrument 4 outside the field of view is restricted. In this case, the operation restriction is preferably such that movement of the medical instrument 4 outside the field of view is prohibited in response to a movement command for the medical instrument 4 outside the field of view. The setting of the surgical site to be restricted and the type of operation restriction can be performed using the touch panel 23.

[0084] When a second surgical section different from the first surgical section is set by the input device 33, the control section 31 operates the medical instrument 4 without changing the operation of the medical instrument 4 corresponding to the command when the distal end position of the medical instrument 4 is outside the field of view of the endoscope 6. For example, the second surgical section is a region such as the prostate gland, which is not close to blood vessels such as the aorta or vena cava. When the second surgical section is selected, the operation of the medical instrument 4 is not restricted even if it is outside the field of view of the endoscope 6.

[0085] Furthermore, the image processing device 8b acquires information on the determination result as to whether or not the medical instrument 4 is located outside the field of view of the endoscope 6 from the control unit 31, and generates a graphical user interface G based on the acquired information. When the distal end position of the medical instrument 4 is outside the field of view of the endoscope 6, the image processing device 8b displays an indicator MK on the display unit 8a and the monitor 24, indicating that the medical instrument 4 is outside the field of view. The indicator MK is an example of a "first indication" in the claims.

[0086] As shown in FIG. 12, when the tip of the medical instrument 4 operated by the operating handle 21 is located outside the field of view of the endoscope 6, the image processing device 8b displays a marker MK indicating the medical instrument 4 located outside the field of view in an inner area of ​​the graphical user interface G that does not include the area near the edge of the screen of the display unit. That is, the image processing device 8b acquires information that the medical instrument 4 operated by the operating handle 21 is located outside the field of view of the endoscope 6 from the control unit 31, and generates and displays the graphical user interface G including a marker MK indicating the medical instrument 4 located outside the field of view. Also, as shown in FIG. 13, the image processing device 8b is configured to display the marker MK in an area G11 near the outer edge of the level area G5 of the graphical user interface G. That is, the marker MK (graphical user interface G) is displayed in the area G11 near the outer edge (marker display area) of the level area G5 that includes the center CN1 but does not include the area near the edge e of the screen gr of the monitor 24. The area G11 near the outer edge is an area located outside and adjacent to the level LV, which will be described later.

[0087] 12 and 16, the sign MK includes an arrow indicating the direction in which a medical instrument 4 located outside the field of view is located. Specifically, when the tip position of the medical instrument 4 operated by the left-hand operating handle 21L is located outside the field of view, a number for identifying the medical instrument 4 located outside the field of view (for example, the number "4" in the hand area G3, which is information related to the arm 60a) is displayed inside the arrow. The number may be located outside the arrow. The inside of the arrow is displayed transparently or semi-transparently so that the image can be seen through it. For a medical instrument 4 to be operated, the number of the sign MK is displayed as a black number for the arm 60 in a white circle. For a medical instrument 4 that is not to be operated, the number of the sign MK is displayed as a gray number for the arm 60 in a black circle.

[0088] Furthermore, as shown in FIG. 16, the image processing device 8b is configured to display a sign MK indicating the direction corresponding to the area where the medical instrument 4 outside the field of view is located, among the eight areas outside the field of view that are divided according to the shape of the rectangular display unit whose vertical and horizontal lengths are not equal.

[0089] 16, the width of the display unit is greater than the height. The area outside the field of view is divided into eight areas according to the shape of the display unit, and the angles of the upper area G12c and the lower area G12g are larger than the angles of the right area G12a, the left area G12e, the upper right area G12b, the upper left area G12d, the lower right area G12h, and the lower left area G12f.

[0090] The image processing device 8b is also configured to display a marker MK at a predetermined position within each of the eight divided regions of the outer edge vicinity area G11.

[0091] Furthermore, when the tip position of the medical instrument 4 is outside the field of view of the endoscope 6, the image processing device 8b displays a pull back display PU, which indicates that the tip position of the medical instrument 4 will be brought into the field of view by pulling the endoscope 6 toward you. The pull back display PU is an example of the "second display" in the claims.

[0092] 16, the image processing device 8b is configured to display the draw display PU in an area adjacent to the outer edge vicinity area G11 of the spirit level area G5 of the graphical user interface G. For example, the image processing device 8b displays the draw display PU above the outer edge vicinity area G11. The image processing device 8b also displays the draw display PU in an area adjacent to the outer edge vicinity area G11 in the left-right direction.

[0093] 13, when the image processing device 8b receives a command to enable movement of the endoscope 6 via the control unit 31, it displays the spirit level LV of the endoscope 6 in the spirit level area G5. That is, while a camera movement operation is being performed using the camera pedal 22c, the image processing device 8b displays the spirit level LV of the endoscope 6 in the spirit level area G5 of the graphical user interface G. Then, the image processing device 8b displays on the monitor 24 a graphical user interface G for displaying a marker MK in an area G11 near the outer edge outside the spirit level area G5. The spirit level LV represents the inclination of the field of view of the endoscope 6 relative to the patient P.

[0094] 13, the level area G5 is a rectangular region (horizontally elongated rectangular shape) including an upper side G5a, a left side G5b, a right side G5c, and a lower side G5d. Displaying the indicator MK in an area G11 near the outer edge of the level area G5 of a predetermined size including the center of the screen gr can improve the operator's visibility. The upper side G5a is located between a position 1 / 10 (L11) of the vertical length of the screen gr above the center CN1 of the screen gr of the monitor 24 and a position 1 / 10 (L11) of the vertical length of the screen gr below the upper edge eu of the screen gr. Preferably, the upper edge portion G5a is positioned between a position 1 / 8 of the vertical length of the screen gr above the center CN1 of the screen gr of the monitor 24 and a position 1 / 8 of the vertical length of the screen gr below the upper end eu of the screen gr, and more preferably, the upper edge portion G5a is positioned between a position 1 / 6 of the vertical length of the screen gr above the center CN1 of the screen gr of the monitor 24 and a position 1 / 6 of the vertical length of the screen gr below the upper end eu of the screen gr.

[0095] The top side G5a is disposed below a status area G10 in which the remaining charge of the built-in battery of the medical manipulator 1 and the like are displayed.

[0096] The bottom side G5d of the spirit level area G5 is located above the hand areas G3a, G3b, G3c, and the camera area G2. Furthermore, the bottom side G5d of the spirit level area G5 is located above the medical instrument status information area G4.

[0097] The center CN1 of the screen gr of the monitor 24 and the center CN2 of the level area G5 are at approximately the same position.

[0098] Furthermore, a sign MK indicating a medical instrument 4 located outside the field of view is disposed between the spirit level area G5 and at least one (in this embodiment, all) of the medical instrument status information area G4, the left pop-up area G6, the right pop-up area G7, and the status area G10. The medical instrument status information area G4 indicates the status of the medical instrument 4. The left pop-up area G6 is displayed when the foot pedal 22 is operated (when the operator's foot is placed on the foot pedal 22 in a hover state). The right pop-up area G7 is displayed when the coagulation pedal 22eR and the incision pedal 22dR are operated (when the operator's foot is placed on them). The status area G10 indicates the status of the surgical system 100.

[0099] 12, the mark MK includes a number that identifies the arm 60 that supports the medical instrument 4 that is located outside the field of view. In the example shown in Fig. 12, a mark MK including the number "4" is displayed, which indicates that the medical instrument 4 supported by the arm 60a (number "4" in the hand area G3) is located outside the field of view of the endoscope 6.

[0100] [Effects of this embodiment] In this embodiment, the following effects can be obtained.

[0101] In this embodiment, as described above, the control unit 31 determines whether the tip position of the medical instrument 4 is outside the field of view of the endoscope 6 based on the relationship between the field of view of the endoscope 6 and the tip position of the medical instrument 4, and when the tip position of the medical instrument 4 is outside the field of view of the endoscope 6, changes the operation of the medical instrument 4 corresponding to the command received by the operating handle 21, the foot pedal 22, or the arm operating unit 80 to the operation when the tip of the medical instrument 4 is within the field of view of the endoscope 6. As a result, when the tip position of the medical instrument 4 is outside the field of view of the endoscope 6 and the tip of the medical instrument 4 cannot be visually confirmed, the operation of the medical instrument 4 can be restricted, for example, by not operating the medical instrument 4, thereby preventing the medical instrument 4 from unintentionally coming into contact with a site inside the body of the patient P outside the field of view of the endoscope 6.

[0102] Furthermore, in this embodiment, as described above, when the tip position of the medical instrument 4 changes from a state where it is outside the field of view of the endoscope 6 to a state where it is inside the field of view, the control unit 31 cancels the process of changing the operation of the medical instrument 4 corresponding to the command received by the operating handle 21, the foot pedal 22, or the arm operating unit 80. This allows the operation of the medical instrument 4 to be returned to normal when the tip position of the medical instrument 4 changes from outside the field of view to inside the field of view of the endoscope 6, so that the operator can operate the medical instrument 4 while visually checking the tip of the medical instrument 4.

[0103] Furthermore, in this embodiment, as described above, when the tip position of the medical instrument 4 is outside the field of view of the endoscope 6, the control unit 31 controls the arms 60a, 60b, and 60d so that the medical instrument 4 does not move in response to a command to move the medical instrument 4. This prevents the medical instrument 4 outside the field of view of the endoscope 6 from moving, effectively preventing the medical instrument 4 from unintentionally coming into contact with a site inside the body of the patient P.

[0104] In this embodiment, as described above, the medical instrument 4 is set to move in response to a movement command with a predetermined scaling. Furthermore, when the distal end position of the medical instrument 4 is outside the field of view of the endoscope 6, the control unit 31 controls the arms 60a, 60b, and 60d to move the medical instrument 4 with a scaling smaller than the predetermined scaling in response to the movement command. This causes the movement of the medical instrument 4 outside the field of view of the endoscope 6 to be slower than the movement within the field of view, thereby preventing the medical instrument 4 from unintentionally coming into contact with a site inside the body of the patient P.

[0105] Furthermore, in this embodiment, as described above, the control unit 31 controls the supply of electricity to the medical instrument 4 so as not to supply electrical energy in response to a power supply command when the distal end position of the medical instrument 4 is outside the field of view of the endoscope 6. This makes it possible to prevent treatment from being performed on a site inside the body of the patient P outside the field of view of the endoscope 6 by supplying electricity to the medical instrument 4 outside the field of view of the endoscope 6.

[0106] Furthermore, in this embodiment, as described above, the control unit 31 controls the operation of the jaws of the medical instrument 4 so that the jaws of the medical instrument 4 are not opened or closed in response to an open / close command when the distal end position of the medical instrument 4 is outside the field of view of the endoscope 6. This makes it possible to prevent the jaws of the medical instrument 4 from coming into contact with a site inside the body of the patient P due to opening or closing of the jaws of the medical instrument 4 outside the field of view of the endoscope 6.

[0107] Furthermore, in this embodiment, as described above, when the operation change mode is set by the touch panel 23, the control unit 31 changes the operation of the medical instrument 4 corresponding to the command when the distal end position of the medical instrument 4 is outside the field of view of the endoscope 6. As a result, when an aorta, a vena cava, or the like is near the surgical site and cannot be visually confirmed, and it is therefore not desired to operate the medical instrument 4, the operation of the medical instrument 4 outside the field of view of the endoscope 6 can be restricted by setting the operation change mode.

[0108] Furthermore, in this embodiment, as described above, if the operation change mode is not set by the touch panel 23, the control unit 31 operates the medical instrument 4 without changing the operation of the medical instrument 4 corresponding to the command when the tip position of the medical instrument 4 is outside the field of view of the endoscope 6. As a result, when the area around the surgical site is less susceptible to contact with the medical instrument 4, the medical instrument 4 can be operated outside the field of view of the endoscope 6 in the same way as when it is within the field of view, thereby enabling efficient surgery.

[0109] Furthermore, in this embodiment, as described above, when the first surgical site is set by the input device 33, the control unit 31 changes the operation of the medical instrument 4 corresponding to the command when the distal end position of the medical instrument 4 is outside the field of view of the endoscope 6. This makes it possible to restrict the operation of the medical instrument 4 outside the field of view of the endoscope 6 when performing surgery on the first surgical site where it is not desired to operate the medical instrument 4 when it cannot be visually confirmed.

[0110] Furthermore, in this embodiment, as described above, when a second surgical site different from the first surgical site is set by the input device 33, the control unit 31 operates the medical instrument 4 without changing the operation of the medical instrument 4 corresponding to the command when the tip position of the medical instrument 4 is outside the field of view of the endoscope 6. As a result, when performing surgery on the second surgical site, which is less affected by contact with the medical instrument 4, the medical instrument 4 can be operated outside the field of view of the endoscope 6 in the same way as when it is within the field of view, allowing the surgery to be performed efficiently.

[0111] Furthermore, in this embodiment, as described above, when the distal end position of the medical instrument 4 is outside the field of view of the endoscope 6, the image processing device 8b displays an indicator MK indicating that the medical instrument 4 is outside the field of view on the display unit 8a and the monitor 24. This allows the operator to easily recognize that the medical instrument 4 is outside the field of view of the endoscope 6.

[0112] Furthermore, in this embodiment, as described above, when the tip position of the medical instrument 4 is outside the field of view of the endoscope 6, the image processing device 8b displays a pull back display PU, which indicates that the tip position of the medical instrument 4 will come into the field of view by pulling the endoscope 6 toward you. This allows the operator to easily recognize that the medical instrument 4 that is outside the field of view of the endoscope 6 will come into the field of view by pulling the endoscope 6 toward you.

[0113] [Variations] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims rather than the description of the above embodiments, and further includes all modifications (variations) within the meaning and scope of the claims.

[0114] For example, in the above embodiment, an example was shown in which the image processing device 8b captured an image from the endoscope 6 and generated the graphical user interface G, but the present invention is not limited to this. For example, the control unit 31 of the medical manipulator 1 may generate the graphical user interface G, or a control unit (not shown) of the remote control device 2 may generate the graphical user interface G. Furthermore, an image processing device that captures an image from the endoscope 6 and performs image processing and an image processing device that generates the graphical user interface G and overlays it on the image of the endoscope 6 may be provided separately.

[0115] Furthermore, in the above embodiment, an example has been shown in which the control unit 31 of the medical manipulator 1 determines whether or not the tip position of the medical instrument 4 is outside the field of view of the endoscope 6 based on the relationship between the field of view of the endoscope 6 and the tip position of the medical instrument 4, but the present invention is not limited to this. In the present invention, for example, the image processing device 8b or a control unit (not shown) of the remote control device 2 may determine whether or not the tip position of the medical instrument 4 is outside the field of view of the endoscope 6 based on the relationship between the field of view of the endoscope 6 and the tip position of the medical instrument 4. Furthermore, the above processing may be performed by multiple control units (such as the control unit 31, the image processing device 8b, and the remote control device 2).

[0116] Furthermore, in the above embodiment, an example was shown in which, when the tip position of the medical instrument 4 is outside the field of view of the endoscope 6, a sign MK including an arrow indicating that the medical instrument 4 is outside the field of view of the endoscope 6 is displayed, but the present invention is not limited to this. In the present invention, the sign indicating that the medical instrument 4 is outside the field of view of the endoscope 6 may be a sign that does not include an arrow.

[0117] Furthermore, in the above embodiment, an example was shown in which the pull indication PU and the sign MK are displayed on the graphical user interface G when the tip of the medical instrument 4 operated by the operating handle 21 moves out of the field of view, but the present invention is not limited to this. For example, the pull indication PU and the sign MK may also be displayed on the graphical user interface G when a predetermined input is made by a predetermined input device. For example, the pull indication PU and the sign MK may also be displayed on the graphical user interface G when a predetermined input is made by the camera pedal 22c or the enable switch 81, or when a predetermined input is made by a switch provided on the operating handle 21.

[0118] In the above embodiment, an example was shown in which the command received by the incision pedal 22d or the coagulation pedal 22e was an energization command to supply electrical energy to the medical instrument 4, but the present invention is not limited to this. For example, a switch may be provided on the link portion 21d or the grip member 21f of the operating handle 21, and an energization command to supply electrical energy to the medical instrument 4 may be issued by operating the switch.

[0119] In the above embodiment, an example was shown in which the operation of a surgical instrument is restricted in response to a command received by the operating device according to a set surgical site, but the present invention is not limited to this. For example, a surgical procedure and the type of operation restriction may be set in advance using the touch panel 23, and the operation of a surgical instrument may be restricted in response to a command received by the operating device according to the set surgical procedure. The set surgical procedure can be input using the input device 33 or the touch panel 23.

[0120] The functions of the elements disclosed herein can be performed using circuits or processing circuits, including general-purpose processors, special-purpose processors, integrated circuits, application-specific integrated circuits (ASICs), conventional circuits, and / or combinations thereof, configured or programmed to perform the disclosed functions. A processor is considered a processing circuit or circuit because it includes transistors and other circuitry. In this disclosure, a circuit, unit, or means is hardware that performs the recited functions or hardware that is programmed to perform the recited functions. The hardware may be hardware disclosed herein or other known hardware that is programmed or configured to perform the recited functions. Where the hardware is a processor, which is considered a type of circuit, the circuit, means, or unit is a combination of hardware and software, and the software is used to configure the hardware and / or processor.

[0121] [Aspect] It will be appreciated by those skilled in the art that the exemplary embodiments described above are examples of the following aspects.

[0122] (Item 1) a first manipulator that supports the endoscope; a second manipulator supporting a surgical instrument; an operating device that receives a command to operate the surgical instrument; a control device including one or more processors and controlling the operation of the surgical instrument so that the operating device operates in accordance with the command received by the operating device; The control device determines whether the tip position of the surgical instrument is outside the field of view of the endoscope based on the relationship between the field of view of the endoscope and the tip position of the surgical instrument, and if the tip position of the surgical instrument is outside the field of view of the endoscope, changes the operation of the surgical instrument corresponding to the command received by the operating device to the operation when the tip of the surgical instrument is within the field of view of the endoscope.

[0123] (Item 2) The surgical system described in item 1, wherein the control device cancels the process of changing the operation of the surgical instrument corresponding to the command received by the operating device when the tip position of the surgical instrument changes from being outside the field of view of the endoscope to being within the field of view.

[0124] (Item 3) the command includes a command to move the surgical instrument; 3. The surgical system according to claim 1, wherein the control device controls the second manipulator so that the surgical instrument does not move in response to the movement command when the tip position of the surgical instrument is outside the field of view of the endoscope.

[0125] (Item 4) the command includes a command to move the surgical instrument; the surgical instrument is configured to move in response to the movement command with a predetermined scaling; The surgical system described in any one of items 1 to 3, wherein the control device controls the second manipulator to move the surgical instrument with a scaling smaller than the predetermined scaling in response to the movement command when the tip position of the surgical instrument is outside the field of view of the endoscope.

[0126] (Item 5) the surgical instrument includes an electrosurgical instrument for supplying electrical energy to a tip thereof; the command includes a power-on command to supply the electrical energy, The surgical system according to any one of items 1 to 4, wherein the control device controls the supply of electricity to the electrosurgical instrument so as not to supply the electrical energy in response to the power supply command when the tip position of the surgical instrument is outside the field of view of the endoscope.

[0127] (Item 6) the surgical instrument includes openable and closable jaws; the commands include opening and closing commands to open or close the jaws of the surgical instrument; The surgical system described in any one of items 1 to 5, wherein the control device controls the operation of the jaw of the surgical instrument so that the jaw is not opened or closed in response to the opening / closing command when the tip position of the surgical instrument is outside the field of view of the endoscope.

[0128] (Item 7) A mode setting device for setting an operation change mode is further provided, A surgical system described in any one of items 1 to 6, wherein when the operation change mode is set by the mode setting device, the control device changes the operation of the surgical instrument corresponding to the command when the tip position of the surgical instrument is outside the field of view of the endoscope.

[0129] (Item 8) Item 8. The surgical system of item 7, wherein if the operation change mode is not set by the mode setting device, the control device operates the surgical instrument without changing the operation of the surgical instrument corresponding to the command when the tip position of the surgical instrument is outside the field of view of the endoscope.

[0130] (Item 9) Further provided is an operation site input device that receives input of the operation site, A surgical system described in any one of items 1 to 8, wherein when the first surgical section is set by the surgical section input device, the control device changes the operation of the surgical instrument corresponding to the command when the tip position of the surgical instrument is outside the field of view of the endoscope.

[0131] (Item 10) Item 10. The surgical system of item 9, wherein when a second surgical section different from the first surgical section is set by the surgical section input device, the control device operates the surgical instrument without changing the operation of the surgical instrument corresponding to the command when the tip position of the surgical instrument is outside the field of view of the endoscope.

[0132] (Item 11) Further, a display device is provided for displaying an image captured by the endoscope, Item 11. The surgical system described in item 10, wherein the control device, when the tip position of the surgical instrument is outside the field of view of the endoscope, displays a first indication on the display unit indicating that the surgical instrument is outside the field of view.

[0133] (Item 12) Item 12. The surgical system according to item 11, wherein the control device, when the tip position of the surgical instrument is outside the field of view of the endoscope, displays a second display indicating that pulling the endoscope toward the user will bring the tip position of the surgical instrument within the field of view.

[0134] (Item 13) 13. The surgical system according to any one of items 1 to 12, wherein the operating device is disposed on the second manipulator.

[0135] (Item 14) 14. The surgical system according to any one of items 1 to 13, wherein the operating device is disposed independently of the first manipulator and the second manipulator.

[0136] (Item 15) a step of determining whether or not the tip position of a surgical instrument supported by a second manipulator is outside the field of view of the endoscope based on the relationship between the field of view of the endoscope supported by a first manipulator and the tip position of the surgical instrument supported by the second manipulator; a step of changing the operation of the surgical instrument corresponding to a command received by an operating device for receiving commands to operate the surgical instrument when the tip position of the surgical instrument is outside the field of view of the endoscope, to the operation when the tip of the surgical instrument is within the field of view of the endoscope. [Explanation of symbols]

[0137] 2 Remote control device 4. Medical instruments (surgical instruments) 6 Endoscopy 8a Display section (display device) 8b Image processing device (control device) 21 Operating handle (operating device) 22d Incision pedal (operating device) 22e Coagulation pedal (operating device) 23 Touch panel (mode setting device) 24 Monitor (display device) 31 Control unit (control device) 33 Input device (operating site input device) 60c Arm (first manipulator) 60a, 60b, 60d Arm (second manipulator) 80 Arm operation unit (operation device) 100 Surgical system (surgical system) MK sign (1st display) PU Pull indicator (second indicator)

Claims

1. a first manipulator that supports an endoscope; a second manipulator supporting a surgical instrument; an operating device that receives a command to operate the surgical instrument; a mode setting device for setting an operation change mode; a control device including one or more processors and controlling the operation of the surgical instrument so that the operating device operates in accordance with the command received by the operating device; The control device determining whether the tip position of the surgical instrument is outside the field of view of the endoscope based on the relationship between the field of view of the endoscope and the tip position of the surgical instrument; When the operation change mode is set by the mode setting device and the tip position of the surgical instrument is outside the field of view of the endoscope, the operation of the surgical instrument corresponding to the command received by the operating device is changed to an operation when the tip of the surgical instrument is within the field of view of the endoscope; A surgical system that operates the surgical instrument without changing the operation of the surgical instrument corresponding to the command when the operation change mode is not set by the mode setting device and the tip position of the surgical instrument is outside the field of view of the endoscope.

2. The surgical system according to claim 1, wherein the control device cancels a process for changing the operation of the surgical instrument corresponding to the command received by the operating device when the tip position of the surgical instrument changes from being outside the field of view of the endoscope to being within the field of view.

3. the command includes a command to move the surgical instrument; The surgical system of claim 1, wherein the control device controls the second manipulator so that the surgical instrument does not move in response to the movement command when the operation change mode is set and the tip position of the surgical instrument is outside the field of view of the endoscope.

4. the command includes a command to move the surgical instrument; the surgical instrument is configured to move in response to the movement command with a predetermined scaling; The surgical system of claim 1, wherein the control device controls the second manipulator to move the surgical instrument at a scaling smaller than the specified scaling in response to the movement command when the operation change mode is set and the tip position of the surgical instrument is outside the field of view of the endoscope.

5. the surgical instrument includes an electrosurgical instrument that supplies electrical energy to a tip thereof; the command includes a power-on command to supply the electrical energy, The surgical system according to claim 1, wherein the control device controls the supply of electricity to the electrosurgical instrument so as not to supply the electrical energy in response to the power supply command when the operation change mode is set and the tip position of the surgical instrument is outside the field of view of the endoscope.

6. the surgical instrument includes openable and closable jaws; the commands include opening and closing commands to open or close the jaws of the surgical instrument; The surgical system according to claim 1, wherein the control device controls the operation of the jaw of the surgical instrument so that the jaw is not opened or closed in response to the opening / closing command when the operation change mode is set and the tip position of the surgical instrument is outside the field of view of the endoscope.

7. The surgical system according to claim 1 , wherein the mode setting device is disposed on the operating device.

8. The surgical system according to claim 7 , wherein the operation device includes a touch panel as the mode setting device for setting the operation change mode.

9. Further, a display device is provided for displaying an image captured by the endoscope, The surgical system according to claim 1 , wherein the control device displays a first indication on the display device indicating that the surgical instrument is outside the field of view when the tip position of the surgical instrument is outside the field of view of the endoscope.

10. The surgical system according to claim 9, wherein the control device, when the tip position of the surgical instrument is outside the field of view of the endoscope, displays a second indication indicating that the tip position of the surgical instrument will come within the field of view if the endoscope is pulled back.

11. The surgical system according to claim 1 , wherein the operating device is disposed independently of the first manipulator and the second manipulator.

12. a medical manipulator including the first manipulator, the second manipulator, and an operation site input device that receives input from the operation site; The surgical system according to claim 1 , wherein the surgical site input device includes a touch panel display.

13. A method for operating a surgical system executed by a control device included in the surgical system, comprising: a step in which the control device receives an input of an operation change mode via a mode setting device; a step in which the control device determines whether or not the tip position of the surgical instrument is outside the field of view of the endoscope based on the relationship between the field of view of the endoscope supported by the first manipulator and the tip position of the surgical instrument supported by the second manipulator; and a step of receiving an input of the operation change mode via the mode setting device and, when the tip position of the surgical instrument is outside the field of view of the endoscope, changing, by the control device, the operation of the surgical instrument corresponding to the command received by an operation device that receives a command to operate the surgical instrument, to the operation when the tip of the surgical instrument is within the field of view of the endoscope; A method for operating a surgical system, in which when the input of the operation change mode is not received via the mode setting device and the tip position of the surgical instrument is outside the field of view of the endoscope, the control device operates the surgical instrument without changing the operation of the surgical instrument corresponding to the command.

Images