Surgical support system, method for operating surgical support robot, and program

Abstract

Provided are a surgery support system, an operation method of a surgery support robot, and a program capable of suppressing a work burden in setting work for avoiding contact with an avoidance target in a case where the surgery support robot is operated.SOLUTION: The surgery support system 500 includes a surgery support robot 100, a remote control device 200, marker members 91, 92, 93, and 94 attached to a bed 501 and a screen 502, an imaging unit 43, and a first control device 310. Based on the marker members 91, 92, 93, and 94 detected based on the captured image, the first control device 310 causes the surgical robot 100 to execute an avoidance operation including avoiding contact with the bed 501 and the screen 502, and canceling the contact when the surgical robot 100 contacts the bed 501 and the screen 502.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a surgery assistance system, a method for operating a surgery assistance robot, and a program. [Background technology]

[0002] Conventionally, surgical support systems equipped with surgical support robots have been known. Patent Document 1 discloses a medical system equipped with a robot system that performs treatment on a patient as a surgical support robot. This robot system includes a robot arm having a medical device connected to its end. In addition, in the medical system disclosed in Patent Document 1, in order to avoid a collision with an object, the movement of the medical device connected to the robot arm is controlled by setting a collision area in association with the object. In this medical system, the end of the robot arm is manually moved to a position adjacent to the object, and the collision area is set based on the position of the robot arm moved to be adjacent to the object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2023-539372 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, when setting a collision area to avoid a collision between a surgical robot and an object, the tip of the robot arm of the surgical robot is manually moved to a position adjacent to the object. Therefore, each time a setting for contact avoidance is made, the tip of the robot arm of the surgical robot must be moved, which places a burden on the operator. Therefore, it is desirable to reduce the workload involved in the setting work for avoiding contact with an object when operating a surgical robot.

[0005] This disclosure aims to provide a surgical support system, a method for operating a surgical support robot, and a program that can reduce the workload involved in setting up a surgical support robot to avoid contact with an object to be avoided when operating the surgical support robot. [Means for solving the problem]

[0006] A surgical support system according to a first aspect of the present disclosure includes a surgical support robot including a robotic arm to which a surgical instrument is attached; an operating device that receives operations on the surgical instrument; a marker element attached to an avoidance target which includes at least one of a patient on whom surgery is to be performed using the surgical support robot and an obstacle placed around the patient; an imaging unit that images the marker element; and a control unit that detects the marker element based on an image of the marker element captured by the imaging unit, and causes the surgical support robot to perform an avoidance operation based on the detected marker element, the avoidance operation including at least one of avoiding contact with the avoidance target and canceling contact if the surgical support robot comes into contact with the avoidance target.

[0007] A surgical assistance system according to a first aspect of the present disclosure includes a control unit that detects a marker member based on an image captured by an imaging unit, and causes a surgical assistance robot to perform an avoidance operation including at least one of avoiding contact with an avoidance target and canceling contact if the marker member comes into contact with the avoidance target, based on the detected marker member. This allows the surgical assistance robot to perform an avoidance operation with respect to the avoidance target based on the marker member detected using the image captured by the imaging unit. Therefore, unlike a case in which the end of the robot arm is moved to a position adjacent to the avoidance target, the avoidance operation can be performed based on the captured image without moving the robot arm each time settings are made to avoid contact with the avoidance target. As a result, when operating the surgical assistance robot, the workload involved in setting up the operation to avoid contact with the avoidance target can be reduced.

[0008] A method for operating a surgical support robot according to a second aspect of the present disclosure includes capturing an image of a marker element attached to an object to be avoided, the object including at least one of a patient on whom surgery is to be performed using a surgical support robot including a robotic arm to which a surgical instrument is attached and an obstacle disposed around the patient; detecting the marker element based on the captured image of the marker element; and performing an avoidance operation by the surgical support robot, based on the detected marker element, the avoidance operation including at least one of avoiding contact with the object to be avoided and canceling the contact if the object to be avoided comes into contact.

[0009] A method for operating a surgical support robot according to a second aspect of the present disclosure includes detecting a marker member based on a captured image in which the marker member is captured, as described above. The method for operating a surgical support robot according to the second aspect of the present disclosure also includes, as described above, performing an avoidance operation by the surgical support robot, including at least one of avoiding contact with an avoidance target and canceling contact if the avoidance target comes into contact, based on the detected marker member. This allows the surgical support robot to perform an avoidance operation with respect to the avoidance target based on the marker member detected using the captured image. Therefore, unlike a case in which the end of the robot arm is moved to a position adjacent to the avoidance target, the avoidance operation can be performed based on the captured image without moving the robot arm each time settings are made to avoid contact with the avoidance target. As a result, a method for operating a surgical support robot can be provided that can reduce the workload involved in setting up the operation to avoid contact with the avoidance target when operating the surgical support robot.

[0010] A program according to a third aspect of the present disclosure causes a computer to perform the following steps: capture an image of a marker member attached to an object to be avoided, which includes at least one of a patient on whom surgery is to be performed using a surgical support robot including a robotic arm to which a surgical instrument is attached and an obstacle placed around the patient; detect the marker member based on the captured image of the marker member; and perform an avoidance operation by the surgical support robot, which includes at least one of avoiding contact with the object to be avoided and canceling the contact if the object to be avoided comes into contact, based on the detected marker member.

[0011] A program according to a third aspect of the present disclosure causes a computer to detect a marker member based on a captured image in which the marker member is captured, as described above. The program according to the third aspect of the present disclosure also causes a computer to execute, based on the detected marker member, an avoidance operation by a surgical support robot, including at least one of avoiding contact with an avoidance target and canceling contact if the avoidance target comes into contact with the avoidance target. This allows the surgical support robot to execute an avoidance operation with respect to the avoidance target based on the marker member detected using the captured image. Therefore, unlike a case in which the end of the robot arm is moved to a position adjacent to the avoidance target, the avoidance operation can be executed based on the captured image without moving the robot arm each time settings are made to avoid contact with the avoidance target. As a result, a program can be provided that can reduce the workload involved in setting up the operation to avoid contact with the avoidance target when operating a surgical support robot. [Effects of the Invention]

[0012] According to the present disclosure, when operating a surgical support robot, the workload involved in setting up the robot to avoid contact with an object to be avoided can be reduced. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram showing the configuration of a surgery support system according to a first embodiment. [Figure 2] FIG. 2 is a diagram showing a display unit of the medical cart according to the first embodiment. [Figure 3] 1 is a diagram showing the configuration of a medical cart according to a first embodiment. FIG. [Figure 4] 1 is a diagram showing a configuration of a robot arm according to a first embodiment. FIG. [Figure 5] FIG. [Figure 6] FIG. 2 is a perspective view showing the configuration of an arm operating unit according to the first embodiment. [Figure 7] FIG. 10 is a diagram for explaining translational movement of a robot arm. [Figure 8] FIG. 10 is a diagram for explaining the rotational movement of the robot arm. [Figure 9] FIG. 1 is a diagram showing an endoscope. [Figure 10] FIG. 10 shows a pivot position setting tool. [Figure 11] FIG. 2 is a diagram showing an operation unit according to the first embodiment. [Figure 12] FIG. 2 is a diagram showing a wrist portion for a right hand according to the first embodiment. [Figure 13] FIG. 2 is a diagram showing a wrist portion for a left hand according to the first embodiment. [Figure 14] 1 is a perspective view showing a foot pedal according to a first embodiment. FIG. [Figure 15] FIG. 2 is a control block diagram of the surgery support robot according to the first embodiment. [Figure 16] FIG. 2 is a control block diagram of the robot arm according to the first embodiment. [Figure 17] FIG. 2 is a control block diagram of the positioner and medical cart according to the first embodiment. [Figure 18] FIG. 2 is a control block diagram of an operation unit according to the first embodiment. [Figure 19] 4 is an example of an image captured by an imaging unit. [Figure 20] FIG. 10 is a perspective view for explaining an avoidance target area. [Figure 21] FIG. 10 is a diagram showing an example of instruction information in an endoscopic image. [Figure 22] FIG. 2 is a control flow diagram for explaining an operation method of the surgery support robot according to the first embodiment. [Figure 23] FIG. 10 is a diagram showing the configuration of a surgery support system according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] [First embodiment] A first embodiment of the present disclosure will be described below with reference to the drawings.

[0015] (Configuration of surgical support system) The configuration of a surgery support system 500 according to the first embodiment will be described. As shown in Fig. 1, the surgery support system 500 includes a surgery support robot 100, a remote control device 200, a vision unit 300, and an image processing unit 400. The remote control device 200 is an example of an operation device.

[0016] In this specification, the longitudinal direction of the surgical instrument 1 is defined as the Z direction, as shown in Figure 4. The distal end side of the surgical instrument 1 is defined as the Z1 side, and the proximal end side of the surgical instrument 1 is defined as the Z2 side. The direction perpendicular to the Z direction is defined as the X direction. The direction perpendicular to the Z direction and the X direction is defined as the Y direction.

[0017] As shown in FIG. 1, a surgical support robot 100 is placed in an operating room. A remote control device 200 is placed at a location remote from the surgical support robot 100. The remote control device 200 receives operations on a surgical instrument 1. Specifically, an operator such as a doctor inputs commands to the remote control device 200 to cause the surgical support robot 100 to perform a desired operation. The remote control device 200 transmits the input commands to the surgical support robot 100. The surgical support robot 100 operates based on the received commands. The surgical support robot 100 is placed in an operating room, which is a sterilized sterile field.

[0018] (Configuration of surgical support robot) As shown in Figure 1, the surgical support robot 100 includes a medical cart 10, a cart positioner operating unit 20, a positioner 30, an arm base 40, multiple robot arms 50, and an arm operating unit 60 provided on each robot arm 50.

[0019] 3, the cart positioner operating unit 20 is supported by a cart positioner operating support unit 21 at the rear of the medical cart 10, and the medical cart 10 or the positioner 30 is moved by operating the cart positioner operating unit 20. The cart positioner operating unit 20 includes an input device 22 and an operating handle 23. The input device 22 receives operations to move and change the posture of the positioner 30, the arm base 40, and the multiple robot arms 50, mainly to prepare for surgery before the procedure.

[0020] As shown in Fig. 3, the input device 22 of the medical cart 10 includes a display unit 22a, a joystick 22b, an enable switch 22c, an error reset button 22d, and a speaker 22e. The display unit 22a is, for example, a liquid crystal panel. As shown in Fig. 2, the display unit 22a displays numbers corresponding to the multiple robot arms 50. The display unit 22a also displays the type of surgical instrument 1 attached to each of the multiple robot arms 50. The display unit 22a displays a check mark CM indicating that a pivot position PP, which will be described later, has been set.

[0021] 3, the joystick 22b is disposed near the display unit 22a of the input device 22 of the medical cart 10. By selecting an operation mode displayed on the display unit 22a and operating the joystick 22b, the positioner 30 is moved three-dimensionally.

[0022] The enable switch 22c is disposed near the joystick 22b. The enable switch 22c permits or prohibits movement of the positioner 30. When the enable switch 22c is pressed down to permit movement of the positioner 30, the positioner 30 is moved by operating the joystick 22b.

[0023] The error reset button 22d resets an error in the surgery support system 500. The error may be, for example, a deviation abnormality error. The speaker 22e is provided as a pair. The pair of speakers 22e is provided near the location of the positioner 30 on the medical cart 10.

[0024] The operating handle 23 is disposed near the display unit 22a. The operating handle 23 has a throttle 23a that is gripped and rotated by an operator such as a nurse or technician to control the movement of the medical cart 10. Specifically, the operating handle 23 is disposed below the input device 22. The medical cart 10 moves forward when the throttle 23a is rotated from the front side to the back side. The medical cart 10 moves backward when the throttle 23a is rotated from the back side to the front side. The speed of the medical cart 10 is changed according to the amount of rotation of the throttle 23a. The operating handle 23 is configured to be rotatable left and right, indicated by the R direction, and the medical cart 10 rotates as the operating handle 23 is rotated.

[0025] An enable switch 23b that permits or prohibits movement of the medical cart 10 is disposed on the operating handle 23. When the enable switch 23b is pressed down to permit movement of the medical cart 10, the medical cart 10 is moved by operating the throttle 23a of the operating handle 23.

[0026] 1, the positioner 30 is, for example, a seven-axis articulated robot. The positioner 30 is placed on a medical cart 10. The positioner 30 adjusts the position of the arm base 40. The positioner 30 moves the position of the arm base 40 three-dimensionally.

[0027] The positioner 30 includes a base portion 31 and a plurality of link portions 32 connected to the base portion 31. The plurality of link portions 32 are connected to each other by joints 33.

[0028] The arm base 40 is attached to the tip of the positioner 30. The base ends of the multiple robot arms 50 are attached to the arm base 40. The multiple robot arms 50 can be folded for storage. The arm base 40 and the multiple robot arms 50 are covered with a sterile drape when in use. The robot arms 50 also support a surgical instrument 1.

[0029] 15, a status indicator 41 and an arm status indicator 42 are arranged on the arm base 40. The status indicator 41 displays the status of the surgery assistance system 500. The arm status indicator 42 displays the status of the robot arm 50.

[0030] As shown in FIG. 1 , an imaging unit 43 is disposed on the arm base 40. The imaging unit 43 captures images of the marker members 91, 92, 93, and 94. The imaging unit 43 captures a captured image 71 as a three-dimensional image shown in FIG. 19 . The imaging unit 43 is, for example, a stereo camera. The imaging unit 43 has an imaging element such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The imaging unit 43 transmits the captured image 71 to the first control device 310. The imaging unit 43 also serves as an imaging unit for positioning the surgical support robot 100. When the operator rolls in the surgical support robot 100 to move it close to the patient P placed on the bed 501, the captured image 71 captured by the imaging unit 43 is displayed in real time on the display unit 22a shown in FIG. 3 .

[0031] A plurality of robot arms 50 are provided. Specifically, four robot arms 50a, 50b, 50c, and 50d are provided. The robot arms 50a, 50b, 50c, and 50d have the same configuration as each other.

[0032] As shown in FIG. 4, the robot arm 50 includes an arm section 51, a first link section 52, a second link section 53, and a translational movement mechanism section 54. The robot arm 50 has joints JT1, JT2, JT3, JT4, JT5, JT6, JT7, and JT8. The joints JT1, JT2, JT3, JT4, JT5, JT6, and JT7 have A1, A2, A3, A4, A5, A6, and A7 axes as rotation axes, respectively. JT8 has A8 axis as a linear axis. The arm section 51 includes a base section 51a and a link section 51b.

[0033] The arm unit 51 is a seven-axis articulated robot arm. The first link unit 52 is located at the tip of the arm unit 51. The arm operating unit 60, which will be described later, is attached to the second link unit 53. The translational movement mechanism 54 is located between the first link unit 52 and the second link unit 53. A holder 55 that holds the surgical instrument 1 is located on the second link unit 53. The translational movement mechanism 54 translates the holder 55, to which the surgical instrument 1 is attached, between a first position and a second position. The first position is the end position on the Z2 side of the range of movement of the holder 55 by the translational movement mechanism 54 along the A8 axis. The second position is the end position on the Z1 side of the range of movement of the holder 55 by the translational movement mechanism 54 along the A8 axis.

[0034] A surgical instrument 1 is attached to the tip of each of the multiple robot arms 50. The surgical instrument 1 includes, for example, an interchangeable instrument 2, an endoscope 3 shown in FIG. 9 for capturing an endoscopic image 72 that is an image of the surgical site, and a pivot position setting instrument 4 shown in FIG. 10 for setting a pivot position PP. The instrument 2 includes a driven unit 2a, an end effector 2b, and a shaft 2c shown in FIG. 5. The end effector 2b is connected to the tip of the shaft 2c.

[0035] 1, an endoscope 3 is attached to the tip of one of the multiple robot arms 50, for example, robot arm 50c, and instruments 2 are attached to the tips of the remaining robot arms 50a, 50b, and 50d. Of the four robot arms 50 arranged adjacent to each other, it is desirable that the endoscope 3 be attached to one of the two robot arms 50b and 50c arranged in the middle.

[0036] (Instrument configuration) 5, an end effector 2b having, for example, jaw members 2f and 2g is attached to the tip of the instrument 2. Examples of the end effector 2b include instruments with joints, such as scissors, graspers, needle holders, microdissectors, stable appliers, tackers, suction and irrigation tools, snare wires, and clip appliers. Instruments without joints, such as cutting blades, cauterizing probes, irrigators, catheters, and suction orifices, are attached to the tip of the instrument 2.

[0037] The instrument 2 includes a first support member 2d and a second support member 2e. The first support member 2d is attached to a shaft 2c. The second support member 2e is supported by the first support member 2d so as to be rotatable about the A10 axis, and supports the end effector 2b so as to be rotatable about the A11 axis that intersects with the A10 axis. The shaft 2c rotates about the A9 axis.

[0038] As shown in Figure 6, the instrument 2 is engaged with the holder 55 via a drape adapter 55a. That is, the holder 55 to which the instrument 2 is attached is a broad concept that includes both a case in which the instrument 2 is attached directly to the holder 55 and a case in which the instrument 2 is attached to the holder 55 via the drape adapter 55a. The drape adapter 55a is attached to the holder 55 when attaching a drape in preparation for surgery, and is not removed during surgery. When replacing the instrument 2, the instrument 2 is removed from the drape adapter 55a, and another instrument 2 is attached.

[0039] (Arm operation unit configuration) 6, the arm operating unit 60 is attached to the robot arm 50 and operates the robot arm 50. Specifically, the arm operating unit 60 is attached to the second link unit 53.

[0040] The arm operating unit 60 includes an enable switch 61 , a joystick 62 , a linear switch 63 , a mode switching button 64 , a mode indicator 65 , a pivot button 66 , and an adjustment button 67 .

[0041] When the enable switch 61 is pressed, it allows or disallows movement of the robot arm 50 using the joystick 62 and the linear switch 63. When the enable switch 61 is pressed while the arm operating unit 60 is being held by an operator such as a nurse or assistant, movement of the surgical instrument 1 by the robot arm 50 is permitted.

[0042] The joystick 62 is an operating tool for controlling the movement of the surgical instrument 1 by the robot arm 50. The joystick 62 controls the movement direction and movement speed of the robot arm 50. The robot arm 50 moves according to the direction and angle at which the joystick 62 is tilted.

[0043] The linear switch 63 is a switch for moving the surgical instrument 1 in the Z direction, which is the longitudinal direction of the surgical instrument 1. The linear switch 63 includes a linear switch 63a for moving the surgical instrument 1 in the direction of inserting it into the patient P, and a linear switch 63b for moving the surgical instrument 1 in the direction away from the patient P. Both the linear switch 63a and the linear switch 63b are push button switches.

[0044] The mode switching button 64 is a push button switch for switching between a translational movement mode and a rotational movement mode of the surgical instrument 1. As shown in FIG. 7, in the translational movement mode of the robot arm 50, the robot arm 50 is moved so that the tip 1a of the surgical instrument 1 moves on the XY plane. As shown in FIG. 8, in the rotational movement mode of the robot arm 50, when the pivot position PP is not stored in the memory unit 351, the robot arm 50 is rotated around the center of the end effector 2b of the instrument 2 (as the surgical instrument 1) on the A11 axis or the tip of the end effector 2b as a fulcrum. When the pivot position PP is stored in the memory unit 351, the robot arm 50 is moved so that the surgical instrument 1 is rotated around the pivot position PP as a fulcrum. Note that the surgical instrument 1 is rotated with the shaft 1c of the surgical instrument 1 inserted into the trocar T. The mode switching button 64 is located on the Z-direction surface of the arm operating unit 60.

[0045] The mode indicator 65 displays the switched mode. When the mode indicator 65 is lit, it indicates the rotational movement mode, and when it is off, it indicates the translational movement mode. The mode indicator 65 also serves as a pivot position indicator that indicates that the pivot position PP has been set. The mode indicator 65 is located on the surface of the arm operation unit 60 on the Z direction side.

[0046] The pivot button 66 is a push button switch for setting a pivot position PP that serves as a fulcrum for the movement of the surgical instrument 1 attached to the robot arm 50.

[0047] The adjustment button 67 is a button for optimizing the position of the robot arm 50. After setting the pivot position PP for the robot arm 50 to which the endoscope 3 is attached, pressing the adjustment button 67 optimizes the positions of the other robot arms 50 and the arm base 40. The adjustment button 67 is a button different from the enable switch 61.

[0048] (remote control device) As shown in Fig. 1, the remote control device 200 is placed, for example, inside or outside an operating room. The remote control device 200 includes an operation unit 110, a foot pedal 120, a touch panel 130, a monitor 140, a support arm 150, a support bar 160, and an error reset button 161. The operation unit 110 constitutes an operation handle that allows an operator, such as a doctor, to input commands. The monitor 140 is an example of an operation device display unit.

[0049] (Operation unit) As shown in FIG. 11 , the operating unit 110 is a handle for operating the surgical instrument 1. The operating unit 110 also receives operations on the surgical instrument 1. When viewed from an operator such as a doctor, the operating unit 110 includes an operating unit 110L located on the left side and operated with the operator's left hand, and an operating unit 110R located on the right side and operated with the operator's right hand. The operating unit 110 includes an arm unit 111 and a wrist unit 112. The operating unit 110R includes an arm unit 111R and a wrist unit 112R. The operating unit 110L also includes an arm unit 111L and a wrist unit 112L.

[0050] The arm unit 111 has joints JT21, JT22, and JT23 shown in Figure 11, and JT24, JT25, JT26, and JT27 shown in Figures 12 and 13. The rotation axes of the joints JT21, JT22, JT23, JT24, JT25, JT26, and JT27 are defined as A21, A22, A23, A24, A25, A26, and A27 axes, respectively.

[0051] (Arm part) As shown in FIG. 11 , the arm 111R has link portions 111a, 111b, and 111c. The upper end of link portion 111a is attached to the remote control device 200 so as to be rotatable around a vertical axis A21. The upper end of link portion 111b is attached to the lower end of link portion 111a so as to be rotatable around a horizontal axis A22. One end of link portion 111c is attached to the lower end of link portion 111b so as to be rotatable around a horizontal axis A23. The wrist portion 112 is attached to the other end of link portion 111c so as to be rotatable around an A24 axis. Link portion 111a is connected to the remote control device 200 by a joint JT21. Link portions 111a and 111b are connected by a joint JT22. Link portions 111b and 111c are connected by a joint JT23. The arm portion 111 supports the wrist portion 112. The arm portion 111L has the same configuration as the arm portion 111R.

[0052] Wrist section 112 includes wrist section 112R, which is operated by the operator's right hand as shown in Fig. 12, and wrist section 112L, which is operated by the operator's left hand as shown in Fig. 13. Fig. 12 shows the reference position of operation section 110R, and Fig. 13 shows the reference position of operation section 110L. Wrist section 112R and wrist section 112L have the same configuration.

[0053] The wrist unit 112 includes link units 112a, 112b, 112c, and a grip support member 112d that is operated by an operator such as a doctor. The base end of link unit 112a is connected to the tip end of the arm unit 111 and rotates around the A24 axis. The base end of link unit 112b is connected to the tip end of link unit 112a and rotates around the A25 axis. The base end of link unit 112c is connected to the tip end of link unit 112b and the grip support member 112d is connected to the tip end of link unit 112c and rotates around the A26 axis relative to link unit 112b. The grip support member 112d rotates around the A27 axis relative to link unit 112c. The link units 112a, 112b, and 112c each have an L-shape.

[0054] The wrist section 112 includes a pair of grip members 112e that are opened and closed by the operator. The grip members 112e are made of elongated, plate-like lever members, and the proximal ends of each of the pair of grip members 112e are rotatably connected to the proximal end of the grip support member 112d. Cylindrical finger insertion sections 112f are disposed on the grip members 112e. The operator inserts their fingers into the pair of finger insertion sections 112f to operate the wrist section 112. The base ends of each of the pair of grip members 112e are connected to the grip support member 112d, and the opening angle between the jaw members 2f and 2g is changed by increasing or decreasing the angle between the pair of grip members 112e. A magnet is disposed on one of the grip members 112e, and a Hall sensor is disposed on the grip support member 112d. When the operator opens or closes the grip member 112e, the magnet and Hall sensor function as an angle detection sensor, and the Hall sensor outputs the opening angle. As an angle detection sensor, a Hall sensor may be disposed on the grip member 112e and a magnet may be disposed on the grip support member 112d. Alternatively, a magnet or a Hall sensor may be disposed on both of the grip members 112e.

[0055] As shown in FIG. 1 , the monitor 140 is a scope-type display device for displaying an endoscopic image 72 captured by the endoscope 3. That is, the monitor 140 displays the endoscopic image 72 captured by the endoscope 3. The monitor 140 also has an alarm unit 141 disposed thereon. The alarm unit 141 emits an error sound. The support arm 150 supports the monitor 140 so that the height of the monitor 140 is adjusted to the height of the face of an operator such as a doctor. The touch panel 130 is disposed on a support bar 160. The surgical support robot 100 can be operated by the remote control device 200 by detecting the operator's head with a sensor disposed near the monitor 140. The operator operates the operation unit 110 and the foot pedal 120 while visually checking the affected area using the endoscopic image 72 displayed on the monitor 140. This inputs commands to the remote control device 200. The commands input to the remote control device 200 are transmitted to the surgical support robot 100.

[0056] (foot pedal) As shown in FIG. 14 , a plurality of foot pedals 120 are provided to perform functions related to the surgical instrument 1. The plurality of foot pedals 120 are arranged on a base 121. The foot pedals 120 include a switching pedal 122, a clutch pedal 123, a camera pedal 124, an incision pedal 125, a coagulation pedal 126, and a foot detector 127. The switching pedal 122, the clutch pedal 123, the camera pedal 124, the incision pedal 125, and the coagulation pedal 126 are operated by the operator's feet. The incision pedals 125 include a incision pedal 125R for the right robot arm 50 and a incision pedal 125L for the left robot arm 50. The coagulation pedals 126 include a coagulation pedal 126R for the right robot arm 50 and a coagulation pedal 126L for the left robot arm 50.

[0057] The switching pedal 122 switches the robot arm 50 operated by the operation unit 110. The clutch pedal 123 performs a clutch operation that temporarily disconnects the operational connection between the robot arm 50 and the operation unit 110. While the clutch pedal 123 is depressed by the operator, the operation by the operation unit 110 is not transmitted to the robot arm 50. Furthermore, while the operator is depressing the camera pedal 124, the operation unit 110 can operate the robot arm 50 to which the endoscope 3 is attached. While the operator is depressing the incision pedal 125 or the coagulation pedal 126, the electrosurgical device is activated.

[0058] Foot detector 127 detects the feet of the operator operating foot pedal 120. Foot detector 127 is provided for each of switch pedal 122, clutch pedal 123, camera pedal 124, incision pedal 125L and coagulation pedal 126L, and incision pedal 125R and coagulation pedal 126R, and detects the feet in a hover state located above each foot pedal 120. Foot detector 127 is disposed on base 121. Note that the function of foot pedal 120, including camera pedal 124, is not limited to a pedal that is stepped on by the operator's foot as in this embodiment, and may be operated by the operator's hand by providing an input device such as a hand switch in operation unit 110, for example.

[0059] (Vision unit and image processing unit) 1, the vision unit 300 and the image processing unit 400 are placed on a cart 210. The image processing unit 400 processes an endoscopic image 72 captured by the endoscope 3. A display unit 220 is disposed on the cart 210. The endoscopic image 72 captured by the endoscope 3 is displayed on the display unit 220.

[0060] (Control system configuration) As shown in Fig. 15, the surgery support system 500 includes a first control device 310, an arm control device 320, a positioner control device 330, an operation control device 340, and a second control device 350. The surgery support system 500 also includes a storage unit 311 connected to the first control device 310 and a storage unit 351 connected to the second control device 350. The first control device 310 is configured by a computer including a CPU (Central Processing Unit) that performs calculations. The first control device 310 also performs processing according to a program. The program that causes the first control device 310 to execute processing is stored in the storage unit 311, for example. The first control device 310 is an example of a control device.

[0061] The first control device 310 is disposed inside the medical cart 10 so as to communicate with the arm control device 320 and the positioner control device 330, and controls the entire surgery support system 500. Specifically, the first control device 310 communicates with and controls each of the arm control device 320, the positioner control device 330, and the operation control device 340. The first control device 310, the arm control device 320, the positioner control device 330, and the operation control device 340 are connected via a LAN or the like. The first control device 310 is disposed inside the medical cart 10.

[0062] An arm control unit 320 is provided for each of the plurality of robot arms 50. That is, a plurality of arm control units 320 corresponding to the number of the plurality of robot arms 50 are provided inside the medical cart 10.

[0063] As shown in Fig. 15, the input device 22 is connected to the first control device 310 via a LAN or the like. The status indicator 41, arm status indicator 42, operating handle 23, throttle 23a, joystick 22b, and positioner control section 330 are serially connected via a wiring 360 over a communication network that allows them to share information with one another. Note that Fig. 15 shows the status indicator 41, arm status indicator 42, and the like as if they were all connected to one wiring 360, but in reality, a wiring 360 is provided for each of the status indicator 41, arm status indicator 42, operating handle 23, throttle 23a, and joystick 22b.

[0064] As shown in FIG. 16, the arm 51 is provided with a plurality of servo motors SM1, an encoder EN1, and a reducer corresponding to each of the joints JT1, JT2, JT3, JT4, JT5, JT6, and JT7. The encoder EN1 detects the rotation angle of the servo motor SM1. The reducer decelerates the rotation of the servo motor SM1 to increase the torque. Inside the medical cart 10, a servo control unit SC1 for controlling the servo motor SM1 is disposed adjacent to the arm control unit 320. The servo control unit SC1 is electrically connected to the encoder EN1 for detecting the rotation angle of the servo motor SM1.

[0065] The second link section 53 is provided with a servo motor SM2 for rotating a driven member disposed in the driven unit 2a of the surgical instrument 1, an encoder EN2, and a reducer. The encoder EN2 detects the rotation angle of the servo motor SM2. The reducer reduces the rotation speed of the servo motor SM2 to increase the torque. The medical cart 10 is also provided with a servo control section SC2 for controlling the servo motor SM2 that drives the surgical instrument 1. The servo control section SC2 is electrically connected to an encoder EN2 for detecting the rotation angle of the servo motor SM2. Note that multiple servo motors SM2, encoders EN2, and servo control sections SC2 are provided.

[0066] The translational movement mechanism 54 is provided with a servo motor SM3 for translating the surgical instrument 1, an encoder EN3, and a reducer. The encoder EN3 detects the rotation angle of the servo motor SM3. The reducer decelerates the rotation of the servo motor SM3 to increase the torque. The medical cart 10 also has a servo control unit SC3 for controlling the servo motor SM3 for translating the surgical instrument 1. The encoder EN3 for detecting the rotation angle of the servo motor SM3 is electrically connected to the servo control unit SC3.

[0067] The first control device 310 generates command values ​​that command the positions of the servo motors SM1, SM2, and SM3 based on the operation received by the remote operation device 200, and drives the servo motors SM1, SM2, and SM3 based on the command values. The first control device 310 then detects a deviation abnormality error when the difference between the command values ​​and the positions of the servo motors SM1, SM2, and SM3 detected by the sensors exceeds an allowable range.

[0068] 17, the positioner 30 is provided with a plurality of servo motors SM4, an encoder EN4, and a reducer so as to correspond to a plurality of joints 33 of the positioner 30. The encoder EN4 is configured to detect the rotation angle of the servo motor SM4. The reducer is configured to reduce the rotation speed of the servo motor SM4 to increase the torque.

[0069] The medical cart 10 is equipped with wheels, including front wheels as drive wheels and rear wheels steered by the operating handle 23. The rear wheels are located closer to the operating handle 23 than the front wheels. The medical cart 10 also includes a servo motor SM5 that drives each of the front wheels of the medical cart 10, an encoder EN5, a reducer, and a brake BRK. The reducer is configured to reduce the rotation speed of the servo motor SM5 and increase the torque. The operating handle 23 of the medical cart 10 is also provided with a potentiometer P1, as shown in FIG. 3, and the servo motor SM5 of the front wheels is driven based on the rotation angle detected by the potentiometer P1 in response to the twist of the throttle 23a. The rear wheels of the medical cart 10 are dual-wheel type, and are steered based on the left and right rotation of the operating handle 23. 3 is disposed on the rotation shaft of the operating handle 23 of the medical cart 10, and a servomotor SM6, an encoder EN6, and a reducer are disposed on the rear wheels of the medical cart 10. The reducer is configured to reduce the rotation speed of the servomotor SM6 and increase the torque. The servomotor SM6 is driven based on the rotation angle detected by the potentiometer P2 in response to the left and right rotation of the operating handle 23. In other words, steering of the rear wheels by the left and right rotation of the operating handle 23 is configured to be power-assisted by the servomotor SM6.

[0070] The front wheels of the medical cart 10 are driven to move forward and backward, and the rear wheels are steered by rotating the operating handle 23 of the medical cart 10, causing the medical cart 10 to rotate left and right.

[0071] As shown in FIG. 17, the medical cart 10 is provided with a servo control unit SC4 for controlling a servo motor SM4 that moves the positioner 30. An encoder EN4 for detecting the rotation angle of the servo motor SM4 is electrically connected to the servo control unit SC4. The medical cart 10 is also provided with a servo control unit SC5 for controlling a servo motor SM5 that drives the front wheels of the medical cart 10. An encoder EN5 for detecting the rotation angle of the servo motor SM5 is electrically connected to the servo control unit SC5. The medical cart 10 is also provided with a servo control unit SC6 for controlling a servo motor SM6 that power-assists the steering of the rear wheels of the medical cart 10. An encoder EN6 for detecting the rotation angle of the servo motor SM6 is electrically connected to the servo control unit SC6.

[0072] As shown in FIGS. 16 and 17 , brakes BRK are mounted on the joints JT1, JT2, JT3, JT4, JT5, JT6, and JT7 of the arm unit 51 and on the joint 33 of the positioner 30. Brakes BRK are also mounted on the front wheels of the medical cart 10, the arm base 40, and the translational movement mechanism 54. Control signals are transmitted unidirectionally from the arm control unit 320 to the brakes BRK mounted on the joints JT1, JT2, JT3, JT4, JT5, JT6, and JT7 of the arm unit 51 and on the translational movement mechanism 54. The control signals are signals that turn the brakes BRK on and off. The signal that turns the brakes BRK on includes a signal that keeps the brakes BRK engaged. The same applies to control signals sent from the positioner control unit 330 to the brakes BRK mounted on the joints 33 of the positioner 30 and on the arm base 40. At startup, all brakes BRK on the arm base 40, arm unit 51, and translational movement mechanism 54 are released, and the servo motor SM is driven to resist gravity, thereby maintaining the posture of the robot arm 50 and the posture of the arm base 40. When an error occurs in the surgery support system 500, the brakes BRK on the arm base 40, arm unit 51, and translational movement mechanism 54 are turned on. When the error in the surgery support system 500 is resolved, the brakes BRK on the arm base 40, arm unit 51, and translational movement mechanism 54 are turned off. A shutdown operation of the surgery support system 500 turns on the brakes BRK on the arm base 40, arm unit 51, and translational movement mechanism 54. In addition, the brakes BRK on the front wheels of the medical cart 10 are always turned on, and are released only while the enable switch 23b of the medical cart 10 is pressed down. Furthermore, the brakes BRK of the joints 33 of the positioner 30 are always on, and the brakes BRK are released only while the enable switch 22c of the medical cart 10 is pressed.

[0073] As shown in FIG. 18, servo motors SM7a, SM7b, SM7c, SM7d, SM7e, SM7f, and SM7g are disposed at joints JT21, JT22, JT23, JT24, JT25, JT26, and JT27 of the operation unit 110, respectively. Servo motor SM7a rotates link portion 111a around the A21 axis. Servo motor SM7b rotates link portion 111b around the A22 axis. Servo motor SM7c rotates link portion 111c around the A23 axis. Servo motor SM7d rotates link portion 112a around the A24 axis. Servo motor SM7e rotates link portion 112b around the A25 axis. Servo motor SM7f rotates link portion 112c around the A26 axis. Servo motor SM7g rotates grip support member 112d around axis A27. Servo control units SC7a, SC7b, SC7c, SC7d, SC7e, SC7f, and SC7g are provided to control the servo motors. Encoders EN7a, EN7b, EN7c, EN7d, EN7e, EN7f, and EN7g are electrically connected to the servo control units to detect the rotation angles of the servo motors. The servo motors, servo control units, and encoders are provided in operation unit 110L and operation unit 110R, respectively.

[0074] The first control device 310 controls each servo motor via the operation control section 340 to generate a torque that cancels out the gravitational torque generated in the rotation shaft of each servo motor according to the attitude of the operation unit 110. This enables the operator to operate the operation unit 110 with a relatively small force.

[0075] As shown in FIG. 15 , the first control device 310 controls the robot arm 50 based on an operation received by the arm operation unit 60. For example, the first control device 310 controls the robot arm 50 based on an operation received by the joystick 62 of the arm operation unit 60. Specifically, the arm control unit 320 outputs an input signal input from the joystick 62 to the first control device 310. The first control device 310 generates a position command based on the received input signal and a rotation angle detected by the encoder EN1, and outputs the position command to the servo control unit SC1 via the arm control unit 320. The servo control unit SC1 generates a current command based on the position command input from the arm control unit 320 and the rotation angle detected by the encoder EN1, and outputs the current command to the servo motor SM1. As a result, the robot arm 50 moves in accordance with the operation command input to the joystick 62.

[0076] The first control device 310 controls the robot arm 50 based on an input signal from the linear switch 63 of the arm operation unit 60. Specifically, the arm control unit 320 outputs the input signal input from the linear switch 63 to the first control device 310. The first control device 310 generates a position command based on the received input signal and the rotation angle detected by the encoder EN1 or EN3, and outputs the position command to the servo control unit SC1 or SC3 via the arm control unit 320. The servo control unit SC1 or SC3 generates a current command based on the position command input from the arm control unit 320 and the rotation angle detected by the encoder EN1 or EN3, and outputs the current command to the servo motor SM1 or SM3. As a result, the robot arm 50 moves in accordance with the operation command input to the linear switch 63.

[0077] The positioner control unit 330 is disposed in the medical cart 10. The positioner control unit 330 controls the positioner 30 and the medical cart 10. A servo motor SM4, an encoder EN4, and a reducer are disposed in the positioner 30 so as to correspond to the multiple joints 33 of the positioner 30. A servo control unit SC4 that controls the servo motor SM4 of the positioner 30 is disposed in the medical cart 10. The medical cart 10 is disposed with servo motors SM5 and SM6 that drive the multiple front wheels of the medical cart 10, encoders EN5 and EN6, reducers, servo control units SC5 and SC6, and a brake BRK.

[0078] The operation control unit 340 is disposed in the main body of the remote operation device 200. The operation control unit 340 controls the operation unit 110. As shown in FIG. 15 , the operation control unit 340 is disposed to correspond to each of the operation unit 110L for the left hand and the operation unit 110R for the right hand. The operation unit 110 is provided with a servo motor SM, an encoder EN, and a reducer to correspond to the multiple joints JT21 to JT27 of the operation unit 110. The servo control unit SC that controls the servo motor SM of the operation unit 110 is disposed in the main body of the remote operation device 200 adjacent to the operation control unit 340.

[0079] 15, the vision unit 300 and the image processing unit 400 are connected to a first control device 310 via a LAN or the like. The display unit 220 is connected to the vision unit 300.

[0080] (Avoidance action) As shown in Fig. 1, in the first embodiment, the surgery support system 500 avoids contact between the surgery support robot 100 and a bed 501 and a screen 502, which are objects to be avoided. The first control device 310 causes the surgery support robot 100 to execute an avoidance operation for the objects to be avoided. The bed 501 and the screen 502 are obstacles located around a patient P on whom surgery is to be performed using the surgery support robot 100. The bed 501 and the screen 502 are examples of objects to be avoided.

[0081] The bed 501 is a bed on which the patient P lies. The screen 502 has a cloth-like curtain that separates the head of the patient P lying on the bed 501 from the surgical field, and a rod-like support member that supports the cloth-like curtain. The screen 502 is attached to the bed 501 and is positioned above the chest or neck of the patient P so as to separate the head from the torso. The screen 502 is, for example, an anesthesia screen and is also called a discharge rack.

[0082] The surgery assistance system 500 includes a plurality of marker members 91, 92, 93, and 94 attached to the patient P and / or to the bed 501 and / or the screen 502, which are obstacles. In the first embodiment, the marker members 91 and 92 are attached to the bed 501. The marker members 93 and 94 are attached to the screen 502. For example, the marker members 91 and 92 are respectively arranged at two of the four corners of the tabletop on which the patient P lies, which are located on the head side. The marker members 93 and 94 are arranged at a vertically upper portion of the screen 502. The marker members 93 and 94 are arranged spaced apart from each other at one end and the other end in the left-right direction of the screen 502. For example, the marker members 91, 92, 93, and 94 are arranged at positions visible from the vertically upper side where the imaging unit 43 is arranged. That is, the marker members 91, 92, 93, and 94 are arranged at positions included in the imaging range of the imaging unit 43. The distance between the imaging unit 43 and each of the marker members 91, 92, 93, and 94 is, for example, about 1 m to 2 m.

[0083] 19 , in the first embodiment, the first control device 310 acquires a captured image 71 in which marker members 91, 92, 93, and 94 are captured by the imaging unit 43. The first control device 310 detects the marker members 91, 92, 93, and 94 based on the acquired captured image 71. Then, in the first embodiment, the first control device 310 causes the surgery support robot 100 to perform an avoidance operation to avoid contact with the bed 501 and the screen 502 based on the detected marker members 91, 92, 93, and 94.

[0084] Each of the marker members 91, 92, 93, and 94 includes a planar graphic. Specifically, the marker members 91, 92, 93, and 94 include, as planar graphics, code graphics indicating the identification information of the corresponding avoidance targets, the bed 501 and the screen 502. For example, the marker members 91, 92, 93, and 94 are AR markers including rectangular planar graphics. That is, the marker members 91 and 92 are AR markers including code graphics indicating the identification information of the bed 501, and the marker members 93 and 94 are AR markers including code graphics indicating the identification information of the screen 502. The AR markers serve as landmarks for displaying augmented reality, which allows digital content to be overlaid on real space. The AR markers are formed, for example, by combining white and black graphics. The marker members 91, 92, 93, and 94 are detachable from the patient P and / or the bed 501 and the screen 502, which are obstacles. For example, the marker members 91, 92, 93, and 94 have an adhesive layer and are attached by an operator to at least one of the patient P and the obstacles, namely, the bed 501 and the screen 502. Each of the marker members 91, 92, 93, and 94 has a size of, for example, about 20 mm to 30 mm.

[0085] 20 , the first control device 310 detects each of the marker members 91, 92, 93, and 94, and sets the avoidance target region 81 and the avoidance target region 82 from the identification information included in each of the marker members 91, 92, 93, and 94. Specifically, the first control device 310 acquires the three-dimensional positions of the marker members 91, 92, 93, and 94 based on the captured image 71 as a three-dimensional image. At this time, the required accuracy for reading the positions of the marker members 91, 92, 93, and 94 is, for example, approximately ±10 mm. The first control device 310 then acquires the positions of the marker members 91, 92, 93, and 94 in the robot coordinate system, which is a coordinate space for controlling the operation of the robot arm 50.

[0086] For example, the first control device 310 acquires the attitude of the robot arm 50 and the attitude of the arm base 40 in the robot coordinate system based on signals from the encoders EN1, EN2, and EN3 of the robot arm 50 and the encoder EN4 of the positioner 30. Then, the first control device 310 acquires the positions of the marker members 91, 92, 93, and 94 in the robot coordinate system based on the preset position of the imaging unit 43 in the arm base 40 and the positions of the marker members 91, 92, 93, and 94 detected in the captured image 71, which is a three-dimensional image captured by the imaging unit 43.

[0087] The first control device 310 then sets an avoidance target area 81 corresponding to the bed 501 based on the detected marker members 91 and 92, and sets an avoidance target area 82 corresponding to the screen 502 based on the detected marker members 93 and 94. For example, information indicating which of the avoidance targets each of the marker members 91, 92, 93, and 94 corresponds to is set in advance and stored in the storage unit 311. The storage unit 311 stores the marker members 91 and 92 and the avoidance target area 81 corresponding to the bed 501 in association with each other, and stores the marker members 93 and 94 and the avoidance target area 82 corresponding to the screen 502 in association with each other. The identification information is information for identifying which of the marker members 91, 92, 93, and 94 it is. The first control device 310 acquires the positions of the detected marker members 91, 92, 93, and 94 from the identification information indicated by the marker members 91, 92, 93, and 94, and thereby sets, in the robot coordinate system, avoidance target regions 81 and 82 associated with the marker members 91, 92, 93, and 94, respectively. The avoidance target regions 81 and 82 are set as three-dimensional spatial regions having a rectangular parallelepiped shape corresponding to the size and shape of the bed 501 and the screen 502, respectively. That is, the first control device 310 detects the marker members 91 and 92, and sets the avoidance target region 81 corresponding to the size and shape of the bed 501 based on the identification information indicated by the marker members 91 and 92. Similarly, the first control device 310 detects the marker members 93 and 94, and sets the avoidance target region 82 corresponding to the size and shape of the screen 502 based on the identification information indicated by the marker members 93 and 94.

[0088] In the first embodiment, the first control device 310, in the avoidance operation, stops the operation of the surgery support robot 100 outside the set avoidance target areas 81 and 82, thereby avoiding contact with the bed 501 and the screen 502, which are the avoidance targets. Specifically, when surgery is being performed on a patient P using the surgery support robot 100, the first control device 310 stops the operation of the surgery support robot 100 outside the set avoidance target areas 81 and 82 when an operation to enter at least one of the robot arm 50 and the surgical instrument 1 into the set avoidance target areas 81 and 82 is received by the remote operation device 200.

[0089] For example, when the first control device 310 operates the robot arm 50 based on an operation received by the remote operation device 200, it determines whether either the surgical instrument 1 or the robot arm 50 will enter the set avoidance target area 81 or the set avoidance target area 82. If it is determined that at least one of the robot arm 50 and the surgical instrument 1 will enter either the avoidance target area 81 or the avoidance target area 82, the first control device 310 stops the operation of the surgery support robot 100. For example, the first control device 310 stops the operation of the surgery support robot 100 and the operation of the surgical instrument 1 and the robot arm 50 by turning on the brakes BRK mounted on the arm base 40, the arm unit 51, and the translational movement mechanism unit 54.

[0090] <Instruction information output> Here, even when an avoidance operation is executed to avoid contact with the bed 501 and the screen 502, which are the avoidance targets, by stopping the operation of the surgical support robot 100 outside the set avoidance target area 81 or 82, the surgical support robot 100 may come into contact with the bed 501 or the screen 502, which are the avoidance targets. That is, even when an avoidance operation is executed to stop the operation of the surgical support robot 100 outside the set avoidance target area 81 or 82, the surgical support robot 100 may enter the avoidance target area 81 or 82. In contrast to this, in the first embodiment, the first control device 310 causes the surgical support robot 100 to execute an operation to eliminate contact when it comes into contact with the bed 501 and the screen 502, based on the detected marker members 91, 92, 93, and 94, during the avoidance operation.

[0091] 21 , when at least one of the robot arm 50 and the surgical instrument 1 enters either of the set avoidance target regions 81 and 82 during the avoidance operation, the first control device 310 determines that contact with the bed 501 and the screen 502 has occurred. In this case, the first control device 310 outputs entry information 73 as an operation to resolve the contact. In the first embodiment, the monitor 140 of the remote operation device 200 displays the entry information 73 to the operator operating the remote operation device 200. When surgery is being performed on a patient P using the surgery support robot 100 and at least one of the robot arm 50 and the surgical instrument 1 enters either of the set avoidance target regions 81 and 82, the first control device 310 displays the entry information 73, including instruction information instructing the direction in which to retreat from the avoidance target regions 81 and 82, on the monitor 140 together with the endoscopic image 72.

[0092] For example, when the robot arm 50 or the surgical instrument 1 enters the avoidance target areas 81 and 82, the first control device 310 acquires a retreat direction in which the robot arm 50 should be moved to eliminate the entry, based on the shapes of the avoidance target areas 81 and 82 and the postures of the robot arm 50 and the arm base 40 at the time of entry. Then, the first control device 310 displays the acquired entry information 73, including instruction information instructing the retreat direction, on the monitor 140, superimposed on the endoscopic image 72. The entry information 73 including instruction information includes, for example, text information indicating the retreat direction. Note that the entry information 73 including instruction information may be displayed on the display unit 220 of the vision unit 300 together with the endoscopic image 72. The entry information 73 may also be notified to the operator as audio information.

[0093] (Operation method of surgical support robot) Next, the control process of the operation method of the surgery support robot 100 according to the first embodiment will be described with reference to Fig. 22. The control from step S1 to step S6 is executed by the first control device 310.

[0094] First, in step S1, images of the marker members 91, 92, 93, and 94 are captured. Specifically, a captured image 71 in which the marker members 91, 92, 93, and 94 are captured is obtained by the imaging unit 43. The marker members 91, 92, 93, and 94 are attached in advance by the operator to the patient P and at least one of the bed 501 and the screen 502 as obstacles during advance preparation.

[0095] Next, in step S2, the marker members 91, 92, 93, and 94 are detected based on the captured image 71. Specifically, by detecting the marker members 91, 92, 93, and 94 from the captured image 71 based on the captured image 71 as a three-dimensional image, the relative positions of the marker members 91, 92, 93, and 94 in three-dimensional space with respect to the imaging unit 43 are detected.

[0096] Next, in step S3, regions to be avoided 81 and 82 are set. Specifically, the positions of the marker members 91, 92, 93, and 94 in the robot coordinate system are set based on the positions in three-dimensional space of the detected marker members 91, 92, 93, and 94. Then, based on the set positions of the marker members 91, 92, 93, and 94 and the identification information of the marker members 91, 92, 93, and 94, the regions to be avoided 81 and 82 indicating the positions of the targets to be avoided in the robot coordinate system are set so as to correspond to the identification information.

[0097] Next, in step S4, the operation unit 110 starts accepting operations for the surgical instrument 1. Specifically, with the endoscopic image 72 displayed on the monitor 140 of the remote control device 200, operations for the operation unit 110 to operate the surgical instrument 1 and the robot arm 50 are accepted.

[0098] Next, in step S5, it is determined whether or not entry into the avoidance target areas 81 and 82 has been detected. For example, when an operation to enter at least one of the surgical instrument 1 and the robot arm 50 into either of the set avoidance target areas 81 and 82 is received, it is determined that entry into the avoidance target areas 81 and 82 has been detected. Also, when at least one of the surgical instrument 1 and the robot arm 50 has entered either of the avoidance target areas 81 and 82, it is determined that entry into the avoidance target areas 81 and 82 has been detected. If it is determined that entry into the avoidance target areas 81 and 82 has been detected, the process proceeds to step S6. If it is not determined that entry into the avoidance target areas 81 and 82 has been detected, the process returns to step S4. Whether or not entry into the avoidance target areas 81 and 82 has been detected is determined based on the signals from each of the encoders EN1, EN2, EN3, and EN4, the attitude of the robot arm 50 and the attitude of the arm base 40 in the robot coordinate system, and the set avoidance target areas 81 and 82.

[0099] In step S5, the surgical support robot 100 performs an avoidance operation, which includes at least one of avoiding contact with the bed 501 and the screen 502, which are the avoidance targets, and resolving the contact, based on the detected marker members 91, 92, 93, and 94. For example, when an operation to enter the avoidance target areas 81 and 82 is received, the operation of the surgical support robot 100 is stopped as an operation to avoid contact. In this case, information indicating that entry has been detected may be output. Furthermore, when at least one of the surgical instrument 1 and the robot arm 50 enters the avoidance target areas 81 and 82, it is determined that contact with the avoidance targets has occurred, and the surgical support robot 100 is caused to perform an operation to resolve the contact. For example, as an operation to resolve the contact, entry information 73 including instruction information instructing the retreat direction is displayed on the monitor 140 together with the endoscopic image 72.

[0100] Note that steps S4 and S5 may be repeatedly performed over a period in which operations are being received on the operation unit 110 of the remote control device 200. For example, after a roll-in operation is performed to move the surgical support robot 100 to a position where surgery is to be performed on the patient P, the processing from capturing images of the marker members 91, 92, 93, and 94 in step S1 to setting the avoidance target regions 81 and 82 in step S3 may be performed during a preparation period before the start of surgery, and steps S4 and S5 may be repeatedly performed during the surgery on the patient P. Also, steps S1 to S6 may be repeatedly performed. For example, during the surgery on the patient P, the captured image 71 may be re-acquired at predetermined control intervals, thereby updating the set avoidance target regions 81 and 82 in real time.

[0101] [Effects of the first embodiment] The surgery support system 500 includes a first control device 310 as a control unit that detects the marker members 91, 92, 93, and 94 based on a captured image 71 in which the marker members 91, 92, 93, and 94 are captured by the imaging unit 43, and causes the surgery support robot 100 to perform an avoidance operation including at least one of avoiding contact with the bed 501 and the screen 502 as avoidance targets and canceling contact if the bed 501 and the screen 502 are contacted, based on the detected marker members 91, 92, 93, and 94. This allows the surgery support robot 100 to perform an avoidance operation against the bed 501 and the screen 502 based on the marker members 91, 92, 93, and 94 detected using the captured image 71 by the imaging unit 43. Therefore, unlike the case where the end of the robot arm 50 is moved to a position adjacent to the bed 501 and the screen 502, the avoidance operation can be performed based on the captured image 71 without moving the robot arm 50 every time settings are made to avoid contact with the bed 501 and the screen 502. As a result, when operating the surgery support robot 100, the workload involved in setting work to avoid contact with the bed 501 and the screen 502 can be reduced.

[0102] The first control device 310 as a control unit sets avoidance target areas 81 and 82 by detecting the marker members 91, 92, 93, and 94. In the avoidance operation, the first control device 310 performs at least one of stopping the operation of the surgical support robot 100 outside the set avoidance target areas 81 and 82 and operating the surgical support robot 100 while avoiding entry into the avoidance target areas 81 and 82, thereby avoiding contact with the bed 501 and the screen 502, which are the avoidance targets. This effectively prevents the surgical support robot 100 from entering the set avoidance target areas 81 and 82. Therefore, the workload for avoiding the bed 501 and the screen 502 can be reduced, and by effectively preventing the surgical support robot 100 from entering the avoidance target areas 81 and 82, the bed 501 and the screen 502 can be easily avoided.

[0103] When an operation to cause at least one of the robot arm 50 and the surgical instrument 1 to enter the set avoidance target areas 81 and 82 is received by the remote operation device 200 as an operation device during surgery on a patient P using the surgical support robot 100, the first control device 310 as a control unit at least one of stopping the operation of the surgical support robot 100 outside the avoidance target areas 81 and 82 and operating the surgical support robot 100 while avoiding entry into the avoidance target areas 81 and 82. As a result, even when an operation to cause at least one of the robot arm 50 and the surgical instrument 1 to enter the avoidance target areas 81 and 82 is received by the remote operation device 200 during surgery, it is possible to automatically prevent at least one of the robot arm 50 and the surgical instrument 1 from entering the avoidance target areas 81 and 82. Therefore, it is possible to easily prevent at least one of the robot arm 50 and the surgical instrument 1 from coming into the avoidance target areas 81 and 82 during surgery.

[0104] The marker members 91, 92, 93, and 94 are attached to at least one of the avoidance targets: a bed 501 on which the patient P lies, a screen 502 separating the patient P's head from the surgical field, and medical equipment used during surgery. The first control device 310, which serves as a control unit, controls the surgery support robot 100 to perform an avoidance operation for at least one of the bed 501, the screen 502, and the medical equipment based on the detected marker members 91, 92, 93, and 94. The types, shapes, and positions of the avoidance targets, such as the bed 501, the screen 502, and the medical equipment, vary depending on the type of surgery or the patient P's body shape and posture. In contrast, in the first embodiment, the first control device 310 controls the surgery support robot 100 to perform an avoidance operation for at least one of the bed 501, the screen 502, and the medical equipment based on the detected marker members 91, 92, 93, and 94. As a result, even when the types, shapes, positions, etc. of the objects to be avoided, including the bed 501 and the screen 502, are different, the bed 501 and the screen 502 can be easily avoided by detecting the marker members 91, 92, 93, and 94. Therefore, the workload for avoiding the bed 501 and the screen 502 can be effectively reduced.

[0105] The first control device 310 as a control unit sets avoidance target areas 81 and 82 by detecting marker members 91, 92, 93, and 94. During the avoidance operation, the first control device 310 outputs entry information 73 when at least one of the robot arm 50 and the surgical instrument 1 enters the set avoidance target areas 81 and 82. As a result, since the entry information 73 is output by the first control device 310, even when at least one of the robot arm 50 and the surgical instrument 1 enters the avoidance target areas 81 and 82, it is possible to easily recognize that at least one of the robot arm 50 and the surgical instrument 1 has entered the avoidance target areas 81 and 82 by recognizing the output entry information 73. Therefore, it is possible to easily resolve the state in which at least one of the robot arm 50 and the surgical instrument 1 has entered the avoidance target areas 81 and 82.

[0106] The surgery support system 500 includes a monitor 140 as an operation device display unit that displays entry information 73 to an operator operating a remote operation device 200 as an operation device. When at least one of the robot arm 50 and the surgical instrument 1 enters set avoidance target areas 81 and 82 during surgery on a patient P using the surgery support robot 100, a first control device 310 as a control unit displays the entry information 73 including instruction information instructing a direction to retreat from the avoidance target areas 81 and 82 on the monitor 140. As a result, when at least one of the robot arm 50 and the surgical instrument 1 enters the avoidance target areas 81 and 82, the operator operating the remote operation device 200 can visually recognize the entry information 73 including instruction information displayed on the monitor 140, and thus easily recognize in which direction the robot arm 50 should be operated to eliminate the state of entering the avoidance target areas 81 and 82. Therefore, the state in which at least one of the robot arm 50 and the surgical instrument 1 has entered the avoidance target areas 81 and 82 can be resolved more easily.

[0107] The monitor 140, which serves as an operation device display unit, displays an endoscopic image 72 captured by the endoscope 3. The first control device 310, which serves as a control unit, displays entry information 73 including instruction information on the monitor 140 along with the endoscopic image 72 when surgery is being performed on a patient P using the surgical support robot 100. This allows the operator operating the remote operation device 200, which serves as an operation device, to easily visually recognize the entry information 73 including instruction information displayed on the monitor 140 along with the endoscopic image 72 when at least one of the robot arm 50 and the surgical instrument 1 enters the avoidance target areas 81 and 82 while performing surgery while visually recognizing the endoscopic image 72 displayed on the monitor 140. Therefore, by visually recognizing the entry information 73 including the displayed instruction information, the operator can easily recognize in which direction the robot arm 50 should be operated to eliminate the state of having entered the avoidance target areas 81 and 82. As a result, even while performing surgery while visually recognizing the endoscopic image 72, it is possible to easily eliminate the state in which at least one of the robot arm 50 and the surgical instrument 1 has entered the avoidance target areas 81 and 82.

[0108] The marker elements 91, 92, 93, and 94 include planar figures. The first control device 310 as a control unit sets the avoidance target regions 81 and 82 by detecting the marker elements 91, 92, 93, and 94 that include planar figures. This makes it possible to easily obtain the three-dimensional arrangement of the marker elements 91, 92, 93, and 94 based on the size and shape of the planar figures included in the marker elements 91, 92, 93, and 94, unlike when the marker elements 91, 92, 93, and 94 are elements that represent points. Therefore, since the marker elements 91, 92, 93, and 94 include planar figures, the avoidance target regions 81 and 82 can be easily set.

[0109] The marker members 91, 92, 93, and 94 include, as planar figures, code graphics indicating identification information of the bed 501 and the screen 502 as avoidance targets. The first control device 310 serving as a control unit detects the marker members 91, 92, 93, and 94 including the code graphics, and sets avoidance target regions 81 and 82 corresponding to the size and shape of the bed 501 and the screen 502 based on the identification information indicated by the code graphics. This makes it possible to easily set the avoidance target regions 81 and 82 corresponding to the size and shape of the bed 501 and the screen 502 by detecting the marker members 91, 92, 93, and 94. Therefore, it is possible to easily set the avoidance target regions 81 and 82 corresponding to the actual bed 501 and screen 502, and therefore it is possible to more easily and accurately prevent contact between the bed 501 and the screen 502 and the surgery support robot 100.

[0110] The imaging unit 43 captures the captured image 71 as a three-dimensional image. This allows the positions of the marker members 91, 92, 93, and 94 to be detected more accurately based on the captured image 71 than when the captured image 71 is a two-dimensional image. Therefore, based on the detected marker members 91, 92, 93, and 94, it is possible to more accurately prevent the surgery support robot 100 from entering the avoidance target regions 81 and 82.

[0111] [Second embodiment] Next, a surgery support system 600 according to a second embodiment will be described with reference to Fig. 23. In the surgery support system 600 of the second embodiment, an imaging unit 610 is arranged separately from a surgery support robot 700. Note that the same components as those in the first embodiment are denoted by the same reference numerals, and description thereof will be omitted.

[0112] 23, in the second embodiment, a surgery support system 600 includes a surgery support robot 700. The surgery support robot 700 according to the second embodiment has the same configuration as the surgery support robot 100 according to the first embodiment, except that the imaging unit 43 is not provided. The surgery support system 600 according to the second embodiment includes an imaging unit 610 and an imaging unit moving mechanism 620.

[0113] The imaging unit 610 captures images of the marker members 91, 92, 93, and 94, similar to the imaging unit 43 of the first embodiment. The imaging unit 610 is a stereo camera that captures a captured image 71 as a three-dimensional image, similar to the imaging unit 43. The imaging unit 610 transmits the captured image 71, which captures the marker members 91, 92, 93, and 94, to the first control device 310. In the second embodiment, the imaging unit 610 is disposed separately from the surgery support robot 700. For example, the imaging unit 610 is disposed on the bed 501. The imaging unit 610 is attached to the bed 501 via an attachment member 611, for example. The imaging unit 610 is disposed so as to capture images of the marker members 91, 92, 93, and 94 from above the bed 501.

[0114] Furthermore, in the second embodiment, the imaging unit moving mechanism 620 changes at least one of the imaging direction and position of the imaging unit 610. The imaging unit moving mechanism 620 includes, for example, a drive source such as a motor, and changes the imaging direction of the imaging unit 610. For example, based on an input operation received by the remote control device 200 or the like, the first control device 310 controls the operation of the imaging unit moving mechanism 620, thereby changing the imaging direction of the imaging unit 610. Note that the imaging unit moving mechanism 620 may be configured to include a slide mechanism or the like, thereby changing the position of the imaging unit 610.

[0115] Furthermore, in the second embodiment, the imaging unit 610 is disposed separately from the surgical support robot 700, and therefore it is necessary to detect the position of the imaging unit 610 in the robot coordinate system. Therefore, in the second embodiment, a marker member 790 is disposed on the surgical support robot 700 separately from the marker members 91, 92, 93, and 94 attached to the bed 501 and the screen 502, which are to be avoided. The first control device 310 controls the imaging unit moving mechanism 620 to change the shooting direction of the imaging unit 610, thereby causing the imaging unit 610 to capture an image of the marker member 790 disposed on the surgical support robot 700. The first control device 310 acquires an image of the marker member 790 captured by the imaging unit 610 and detects the marker member 790 based on the acquired image, thereby acquiring the relative positional relationship between the imaging unit 610 disposed on the bed 501 and the surgical support robot 700. As a result, the first control device 310 acquires the position of the imaging unit 610 in the robot coordinate system. Then, similarly to the first embodiment, the first control device 310 detects the marker members 91, 92, 93, and 94 based on the captured image 71 captured by the imaging unit 610, thereby setting avoidance target areas 81 and 82 in the robot coordinate system. The control process of the avoidance operation by the surgery support robot 700 is the same as in the first embodiment. The other configurations of the second embodiment are the same as in the first embodiment.

[0116] [Effects of the second embodiment] The surgery support system 600 includes an imaging unit moving mechanism 620 that changes at least one of the imaging direction and position of the imaging unit 610. This allows the imaging unit moving mechanism 620 to change at least one of the imaging direction and position of the imaging unit 610, making it possible to easily capture images of the marker members 91, 92, 93, and 94 from an imaging direction or position that makes them easy to detect. Therefore, by changing at least one of the imaging direction and position of the imaging unit 610 using the imaging unit moving mechanism 620, the marker members 91, 92, 93, and 94 can be easily detected based on the captured image 71. As a result, the surgery support robot 700 can be easily prevented from entering the avoidance target areas 81 and 82 based on the detected marker members 91, 92, 93, and 94.

[0117] The imaging unit 610 is disposed separately from the surgical support robot 700. This allows the marker members 91, 92, 93, and 94 to be captured from a position separate from the surgical support robot 700, thereby preventing the surgical support robot 700 itself from interfering with the capture of the marker members 91, 92, 93, and 94. Furthermore, since the imaging unit 610 is disposed separately from the surgical support robot 700, vibrations caused by the operation of the surgical support robot 700 are prevented from being transmitted to the imaging unit 610. This allows the captured image 71, in which the marker members 91, 92, 93, and 94 are easily detected, to be captured more easily. This makes it easier to detect the marker members 91, 92, 93, and 94 based on the captured image 71. As a result, the surgical support robot 700 can be more easily prevented from entering the avoidance target areas 81 and 82 based on the detected marker members 91, 92, 93, and 94. Other advantages of the second embodiment are similar to those of the first embodiment.

[0118] [Variations] It should be noted that the embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present disclosure is defined by the claims, not by the description of the above embodiments, and further includes all modifications and variations within the meaning and scope of the claims.

[0119] In the above-described first and second embodiments, the marker members 91, 92, 93, and 94 are arranged on the bed 501 and the screen 502 as obstacles to be avoided that are arranged around the patient P, but the present disclosure is not limited to this. In the present disclosure, the marker members may be arranged on medical equipment used during surgery as obstacles to be avoided. For example, the marker members may be arranged on medical equipment such as an anesthesia machine or an X-ray machine. The marker members may also be arranged on the patient on whom surgery is to be performed. For example, by arranging the marker members around the operative field of the patient, contact of a surgical instrument or a robot arm with parts of the patient other than the operative field may be avoided.

[0120] In the first and second embodiments, an example was described in which the operation of the surgical support robot 100 was stopped outside the set avoidance target areas 81 and 82 during the avoidance operation. However, the present disclosure is not limited thereto. In the present disclosure, when an operation to enter at least one of the robot arm and the surgical instrument into the set avoidance target area is received from the operating device during surgery on a patient using the surgical support robot, the control unit may at least one of stop the operation of the surgical support robot outside the avoidance target area and operate the surgical support robot while avoiding entry into the avoidance target area. That is, during the avoidance operation, the surgical support robot may be operated while avoiding entry into the avoidance target area, thereby avoiding contact with the avoidance target. For example, the surgical instrument may be moved while restricting the operating range of the robot arm to avoid entry into the avoidance target area. Furthermore, if it is possible to operate the surgical support robot while avoiding entry into the avoidance target area, the operation of the surgical support robot may not be stopped, but if it is difficult to operate the surgical support robot while avoiding entry into the avoidance target area, the operation may be stopped. Furthermore, when an operation to enter at least one of the robot arm and the surgical instrument into a set avoidance area is received by the operating device during surgery, the surgical support robot may operate in accordance with the received operation up to the point just before the avoidance area, and may stop its operation at a position where it enters the avoidance area. Furthermore, when the robot is about to enter the avoidance area or has already entered the avoidance area, the operating speed may be reduced rather than stopped. Furthermore, during the avoidance operation, contact may be avoided by outputting notification information notifying the robot of the entry into the avoidance area. In this case, the notification information may be displayed on the display unit, or may be output as audio.

[0121] In the first embodiment, an example was described in which the imaging unit 43 was disposed on the arm base 40 of the surgical support robot 100, and in the second embodiment, an example was described in which the imaging unit 610 was disposed on the bed 501 separately from the surgical support robot 700. However, the present disclosure is not limited to this. In the present disclosure, the imaging unit may be disposed in a position other than the arm base of the surgical support robot. For example, the imaging unit may be disposed on the robot arm, a medical cart, or a positioner. The imaging unit may also be disposed on the ceiling or wall of the operating room. An endoscope attached to the robot arm may also serve as the imaging unit that captures an image of the marker member. In this case, for example, the avoidance target area may be set in advance before surgery by capturing an image of the marker member with the endoscope during roll-in before the start of surgery.

[0122] In the above first and second embodiments, an example was shown in which the marker members 91, 92, 93, and 94 are AR markers that include, as planar figures, code figures that indicate identification information of the bed 501 and screen 502 that are to be avoided, but the present disclosure is not limited to this. In the present disclosure, the marker members may be members that are recognized as points in space without including planar figures. Furthermore, the marker members may include planar figures that are simply patterns other than AR markers. Furthermore, the number of marker members may be one instead of multiple.

[0123] In the first and second embodiments described above, the imaging units 43 and 610 are stereo cameras that capture the captured image 71 as a three-dimensional image, but the present disclosure is not limited to this. In the present disclosure, the imaging unit may be a time-of-flight (ToF) camera. Furthermore, the imaging unit may be a single two-dimensional camera that captures a single two-dimensional image. Even in this case, by disposing a marker member that includes a planar image, such as an AR marker, it becomes possible to detect the three-dimensional arrangement of the marker member based on the distortion of the size and shape of the planar image. Furthermore, the imaging unit may include multiple two-dimensional cameras to capture the captured image as a three-dimensional image.

[0124] In the first and second embodiments, examples have been described in which the avoidance target areas 81 and 82 are set corresponding to the size and shape of the bed 501 and screen 502, which are the avoidance targets. However, the present disclosure is not limited to this. In the present disclosure, the avoidance target area may be set based on the arrangement of marker elements, regardless of the size and shape of the avoidance target. For example, the avoidance target area may be set by detecting the marker elements to detect the plane on which the marker elements are arranged and setting the plane on which the marker elements are arranged as the boundary surface of the avoidance target area. Alternatively, by arranging multiple marker elements, a space with each of the multiple marker elements as a vertex may be set as the avoidance target area. Alternatively, the avoidance target area corresponding to the avoidance target may be set by identifying the avoidance target to which a marker element is attached from the captured image through image authentication. For example, the shape of the avoidance target area may be extracted by extracting the shape of an area including the marker elements in the captured image.

[0125] In the first and second embodiments, the entry information 73, including instruction information for instructing the evacuation direction, is superimposed on the endoscopic image 72 and displayed on the monitor 140 serving as the display unit of the operation device. However, the present disclosure is not limited to this. In the present disclosure, rather than instructing the evacuation direction, entry information indicating that an entry has occurred may be displayed or audibly notified. The entry information may be displayed separately from the endoscopic image. The entry information may also be displayed on a display unit separate from the monitor 140. For example, the entry information may be displayed on a display unit of the vision unit or a display unit disposed on the medical cart. An image of the marker member may also be displayed on the monitor of the remote control device serving as the operation device, on a display unit of the vision unit, or on a display unit disposed on the medical cart. The set avoidance target area may also be displayed so that it can be visually recognized.

[0126] In the first and second embodiments, an example was described in which an avoidance operation was performed when the robot arm 50 of the surgical support robots 100 and 700 was operated based on the operation of the remote control device 200 as the operation device, but the present disclosure is not limited to this. In the present disclosure, an avoidance operation may be performed when each part of the surgical support robot is moved based on the operation of the cart positioner operation unit or the arm operation unit. For example, an avoidance operation may be performed based on a detected marker member before surgery on a patient is performed, such as during roll-in.

[0127] Furthermore, in the above first and second embodiments, an example has been shown in which the first control device 310 as a control unit performs the control processes of detecting the marker members 91, 92, 93, and 94 from the captured image 71, setting the avoidance target areas 81 and 82, and performing the avoidance operation including avoiding and resolving contact, but the present disclosure is not limited to this. In the present disclosure, one or more of the control processes of detecting the marker members, setting the avoidance target areas, and performing the avoidance operation may be performed by a control unit that is arranged separately from the control device that controls the operation of the robot arm.

[0128] Furthermore, in the above-described first and second embodiments, an example in which four robot arms 50 are provided is shown, but the present disclosure is not limited to this. In the present disclosure, the number of robot arms may be any other number as long as there is at least one or more.

[0129] Furthermore, in the above first and second embodiments, an example was shown in which the arm unit 51 and the positioner 30 were configured as a seven-axis articulated robot, but the present disclosure is not limited to this. In the present disclosure, the arm unit and the positioner may be configured as an articulated robot with an axis configuration other than a seven-axis articulated robot. An axis configuration other than a seven-axis articulated robot is, for example, a six-axis or eight-axis configuration. Furthermore, the surgical support robot may have a robot arm with a horizontal articulated mechanism, a linear motion mechanism, or a parallel link mechanism.

[0130] In the above first and second embodiments, examples have been shown in which the surgical support robots 100 and 700 include the medical cart 10, the positioner 30, and the arm base 40, but the present disclosure is not limited to this. In the present disclosure, the medical cart, the positioner, and the arm base are not necessarily required, and the surgical support robot may be composed of only a robot arm.

[0131] The functions of the elements disclosed herein can be performed using circuits or processing circuits, including general-purpose processors, special-purpose processors, integrated circuits, application-specific integrated circuits (ASICs), conventional circuits, and / or combinations thereof, configured or programmed to perform the disclosed functions. A processor is considered a processing circuit or circuit because it includes transistors and other circuitry. In this disclosure, a circuit, unit, or means is hardware that performs the recited functions or hardware that is programmed to perform the recited functions. The hardware may be hardware disclosed herein or other known hardware that is programmed or configured to perform the recited functions. Where the hardware is a processor, which is considered a type of circuit, the circuit, means, or unit is a combination of hardware and software, and the software is used to configure the hardware and / or processor.

[0132] [Aspect] It will be appreciated by those skilled in the art that the exemplary embodiments described above are examples of the following aspects.

[0133] (Aspect 1) a surgical robot including a robotic arm to which a surgical instrument is attached; an operating device that accepts an operation on the surgical instrument; A marker member attached to an object to be avoided, which includes at least one of a patient on whom surgery is to be performed using the surgical support robot and an obstacle disposed around the patient; an imaging unit that images the marker member; a control unit that detects the marker member based on an image of the marker member captured by the imaging unit, and causes the surgical support robot to execute an avoidance operation including at least one of avoiding contact with the avoidance target and canceling contact if the marker member comes into contact with the avoidance target based on the detected marker member.

[0134] (Aspect 2) The control unit By detecting the marker member, an avoidance target area is set, The surgical support system of aspect 1, wherein the avoidance operation avoids contact with the target to be avoided by performing at least one of stopping the operation of the surgical support robot outside the set area to be avoided and operating the surgical support robot while avoiding entry into the area to be avoided.

[0135] (Aspect 3) The surgical support system of aspect 2, wherein when an operation to enter at least one of the robot arm and the surgical instrument into the set avoidance target area is accepted by the operation device while surgery is being performed on the patient using the surgical support robot, the control unit at least one of stops the operation of the surgical support robot outside the avoidance target area and operates the surgical support robot while avoiding entry into the avoidance target area.

[0136] (Aspect 4) The marker member is attached to the object to be avoided, which includes at least one of a bed on which the patient lies, a screen separating the patient's head from a surgical field, and medical equipment used during surgery; The surgical support system according to any one of aspects 1 to 3, wherein the control unit causes the surgical support robot to perform the avoidance operation against the avoidance target, which includes at least one of the bed, the screen, and the medical equipment, based on the detected marker member.

[0137] (Aspect 5) The control unit By detecting the marker member, an avoidance target area is set, A surgical support system according to any one of aspects 1 to 4, which outputs entry information when at least one of the robot arm and the surgical instrument enters the set avoidance target area during the avoidance operation.

[0138] (Aspect 6) an operation device display unit that displays the entry information to an operator who operates the operation device; The control unit of the surgical support system described in aspect 5, when at least one of the robot arm and the surgical instrument enters the set avoidance target area while surgery is being performed on the patient using the surgical support robot, displays the entry information on the operation device display unit, including instruction information instructing the direction of escape from the avoidance target area.

[0139] (Aspect 7) the operation device display unit displays an endoscopic image captured by an endoscope, The surgical support system of aspect 6, wherein the control unit displays the instruction information on the operation device display unit together with the endoscopic image when surgery is being performed on the patient using the surgical support robot.

[0140] (Aspect 8) the marker member includes a planar graphic; The surgery support system according to aspect 2 or 3, wherein the control unit sets the avoidance target region by detecting the marker member including the planar figure.

[0141] (Aspect 9) the marker member includes a code graphic indicating identification information of the object to be avoided as the planar graphic, The control unit of the surgical support system described in aspect 8 sets the avoidance target area corresponding to the size and shape of the avoidance target based on the identification information indicated by the code graphic by detecting the marker element including the code graphic.

[0142] (Aspect 10) The surgery support system according to any one of aspects 1 to 9, wherein the imaging unit captures the captured image as a three-dimensional image.

[0143] (Aspect 11) The surgery support system according to any one of aspects 1 to 10, further comprising an imaging unit moving mechanism that changes at least one of the imaging direction and position of the imaging unit.

[0144] (Aspect 12) The surgical support system according to any one of aspects 1 to 11, wherein the imaging unit is arranged separately from the surgical support robot.

[0145] (Aspect 13) capturing an image of a marker member attached to an object to be avoided, the object including at least one of a patient on whom surgery is to be performed using a surgical support robot including a robot arm to which a surgical instrument is attached, and an obstacle disposed around the patient; Detecting the marker member based on a captured image of the marker member; A method for operating a surgical support robot, comprising: executing, by the surgical support robot, an avoidance operation that includes at least one of avoiding contact with the object to be avoided based on the detected marker member and canceling the contact if the object to be avoided is contacted.

[0146] (Aspect 14) capturing an image of a marker member attached to an object to be avoided, the object including at least one of a patient on whom surgery is to be performed using a surgical support robot including a robot arm to which a surgical instrument is attached, and an obstacle disposed around the patient; Detecting the marker member based on a captured image of the marker member; A program that causes a computer to execute, by the surgical support robot, an avoidance operation that includes at least one of avoiding contact with the avoidance target based on the detected marker member and canceling the contact if the avoidance target is contacted. [Explanation of symbols]

[0147] 1 surgical instruments 3 Endoscopy 43, 610 Imaging unit 50 Robot Arm 71 Captured images 72 Endoscopic images 73 Approach information 81, 82 Areas to be avoided 91, 92, 93, 94 Marker parts 100, 700 Surgical support robot 140 Monitor (operating device display) 200 Remote control device (operation device) 310 First control device (control unit) 500, 600 Surgical Support System 501 Bed (obstacle, avoidance target) 502 Screen (obstacles, avoidance targets) 620 Imaging unit movement mechanism

Claims

1. a surgical robot including a robotic arm to which a surgical instrument is attached; an operating device that accepts an operation on the surgical instrument; A marker member attached to an object to be avoided, which includes at least one of a patient on whom surgery is to be performed using the surgical support robot and an obstacle disposed around the patient; an imaging unit that images the marker member; a control unit that detects the marker member based on an image of the marker member captured by the imaging unit, and causes the surgical support robot to execute an avoidance operation including at least one of avoiding contact with the avoidance target and canceling contact if the marker member comes into contact with the avoidance target based on the detected marker member.

2. The control unit By detecting the marker member, an avoidance target area is set, 2. The surgical support system according to claim 1, wherein, in the avoidance operation, contact with the avoidance target is avoided by performing at least one of stopping the operation of the surgical support robot outside the set avoidance target area and operating the surgical support robot while avoiding entry into the avoidance target area.

3. 3. The surgical support system according to claim 2, wherein when an operation to enter at least one of the robot arm and the surgical instrument into the set avoidance target area is accepted by the operation device while surgery is being performed on the patient using the surgical support robot, the control unit at least one of stopping operation of the surgical support robot outside the avoidance target area and operating the surgical support robot while avoiding entry into the avoidance target area.

4. The marker member is attached to the object to be avoided, which includes at least one of a bed on which the patient lies, a screen separating the patient's head from a surgical field, and medical equipment used during surgery; The surgical support system according to any one of claims 1 to 3, wherein the control unit causes the surgical support robot to perform the avoidance operation against the avoidance target including at least one of the bed, the screen, and the medical equipment based on the detected marker member.

5. The control unit By detecting the marker member, an avoidance target area is set, The surgical assistance system according to any one of claims 1 to 3, wherein, during the avoidance operation, entry information is output when at least one of the robot arm and the surgical instrument enters the set avoidance target area.

6. an operation device display unit that displays the entry information to an operator who operates the operation device; 6. The surgical support system according to claim 5, wherein when at least one of the robot arm and the surgical instrument enters the set avoidance target area while surgery is being performed on the patient using the surgical support robot, the control unit causes the operation device display unit to display the entry information including instruction information instructing a direction to retreat from the avoidance target area.

7. the operation device display unit displays an endoscopic image captured by an endoscope, The surgery support system according to claim 6, wherein the control unit causes the operation device display unit to display the instruction information together with the endoscopic image while surgery is being performed on the patient using the surgery support robot.

8. the marker member includes a planar graphic; The surgery support system according to claim 2 , wherein the control unit sets the avoidance target region by detecting the marker member including the planar figure.

9. the marker member includes a code graphic indicating identification information of the object to be avoided as the planar graphic, 9. The surgical assistance system according to claim 8, wherein the control unit detects the marker member including the code graphic, and sets the avoidance target area corresponding to the size and shape of the avoidance target based on the identification information indicated by the code graphic.

10. 4. The surgery support system according to claim 1, wherein the imaging unit captures the captured image as a three-dimensional image.

11. The surgery support system according to any one of claims 1 to 3, further comprising an imaging unit moving mechanism that changes at least one of the imaging direction and position of the imaging unit.

12. The surgery support system according to any one of claims 1 to 3, wherein the imaging unit is arranged separately from the surgery support robot.

13. capturing an image of a marker member attached to an object to be avoided, the object including at least one of a patient on whom surgery is to be performed using a surgical support robot including a robot arm to which a surgical instrument is attached, and an obstacle disposed around the patient; Detecting the marker member based on a captured image of the marker member; A method for operating a surgical support robot, comprising: executing, by the surgical support robot, an avoidance operation that includes at least one of avoiding contact with the object to be avoided based on the detected marker member and canceling the contact if the object to be avoided is contacted.

14. capturing an image of a marker member attached to an object to be avoided, the object including at least one of a patient on whom surgery is to be performed using a surgical support robot including a robot arm to which a surgical instrument is attached, and an obstacle disposed around the patient; Detecting the marker member based on a captured image of the marker member; A program that causes a computer to execute, by the surgical support robot, an avoidance operation that includes at least one of avoiding contact with the avoidance target based on the detected marker member and canceling the contact if the avoidance target is contacted.

Images