Surgery support system, method for controlling surgery support system, and program

Abstract

To provide a surgery support system capable of reliably setting a pivot position serving as a fulcrum of movement of a surgical instrument.SOLUTION: The surgical support system 500 includes the surgical robot 100 including the robot arm 50 to which the surgical instrument 1 is attached, the marker imager 70 that images the marker member M detachably attached to at least one of the patient P on whom the surgery is performed using the surgical robot 100 and the trocar T inserted into the body surface S of the patient P, and the second control device 350 that detects the marker member M based on the captured image in which the marker member M is imaged and sets the pivot position PP serving as the fulcrum of the movement of the surgical instrument 1 based on the detected marker member M.SELECTED DRAWING: Figure 19

Description

[Technical Field]

[0001] The present disclosure relates to a surgery assistance system, a control method for a surgery assistance system, and a program. [Background technology]

[0002] Conventionally, surgical assistance systems equipped with a robotic arm to which a surgical instrument is attached have been known. The surgical assistance system disclosed in Patent Document 1 includes a robotic arm to which a surgical instrument is attached and a processing device that controls the operation of the robotic arm. Furthermore, in Patent Document 1, a surgical instrument is inserted into a patient's body through a port placed on the patient's body surface. Furthermore, the surgical assistance system disclosed in Patent Document 1 also includes an imaging system that images the port. An image of the port captured by the imaging system is input to the processing device. Then, based on the image of the port captured by the imaging system, the processing device defines the coordinates of the port as a fulcrum for movement of the surgical instrument. As a result, the surgical instrument is moved using the position of the port as a fulcrum. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6290372 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, the coordinates of the port are defined as the fulcrum for the movement of the surgical instrument based on an image of the port captured by the imaging system. In this case, it is conceivable that the port may be in a blind spot for other medical devices, and the imaging system may not be able to capture an image of the port. In this case, it becomes impossible to set the fulcrum for the movement of the surgical instrument. Therefore, it is desirable to be able to reliably set the fulcrum for the movement of the surgical instrument.

[0005] The present disclosure provides a surgery assistance system, a control method for a surgery assistance system, and a program that can reliably set a pivot position that serves as a fulcrum for the movement of a surgical instrument. [Means for solving the problem]

[0006] A surgical support system according to a first aspect of the present disclosure includes a surgical support robot including a robotic arm to which a surgical instrument is attached; a marker imaging unit that images a marker member that is detachably attached to at least one of a patient on whom surgery is to be performed using the surgical support robot and a trocar inserted into the surface of the patient's body; and a control unit that detects the marker member based on the captured image of the marker member and sets a pivot position that serves as a fulcrum for movement of the surgical instrument based on the detected marker member.

[0007] A surgical assistance system according to a first aspect of the present disclosure includes a marker imaging unit that images a marker member detachably attached to at least one of a patient and a trocar, and a control unit that detects the marker member based on the captured image of the marker member and sets a pivot position that serves as a fulcrum for movement of a surgical instrument based on the detected marker member. Because the marker member is detachable from the patient and the trocar, the marker member can be attached to a position that does not expose other medical devices or other blind spots. Therefore, the marker imaging unit can reliably image the marker member, and therefore the pivot position that serves as a fulcrum for movement of the surgical instrument can be reliably set based on the marker member detected from the captured image.

[0008] A control method for a surgical support system according to a second aspect of the present disclosure includes capturing an image of a marker element detachably attached to at least one of a patient on whom surgery is to be performed using a surgical support robot including a robotic arm to which a surgical instrument is attached and a trocar inserted into the surface of the patient's body, detecting the marker element based on the captured image of the marker element, and setting, by a control unit, a pivot position that serves as a fulcrum for movement of the surgical instrument based on the detected marker element.

[0009] A control method for a surgery assistance system according to a second aspect of the present disclosure includes capturing an image of a marker member detachably attached to at least one of a patient and a trocar, detecting the marker member based on the captured image of the marker member, and having a control unit set a pivot position serving as a fulcrum for movement of a surgical instrument based on the detected marker member. Because the marker member is detachable from the patient and the trocar, the marker member can be attached to a position that does not expose blind spots of other medical devices. Therefore, the marker capturing unit can reliably capture an image of the marker member, thereby providing a control method for a surgery assistance system that can reliably set a pivot position serving as a fulcrum for movement of a surgical instrument based on the marker member detected from the captured image.

[0010] A program according to a third aspect of the present disclosure performs the following steps: capturing an image of a marker member detachably attached to at least one of a patient undergoing surgery using a surgical support robot including a robotic arm to which a surgical instrument is attached and a trocar inserted into the surface of the patient's body; detecting the marker member based on the captured image of the marker member; and setting a pivot position that serves as a fulcrum for movement of the surgical instrument based on the detected marker member.

[0011] A program according to a third aspect of the present disclosure includes capturing an image of a marker member detachably attached to at least one of a patient and a trocar, detecting the marker member based on the captured image of the marker member, and having a control unit set a pivot position serving as a fulcrum for movement of a surgical instrument based on the detected marker member. Because the marker member is detachable from the patient and the trocar, the marker member can be attached to a position that does not expose other medical devices or other blind spots. Therefore, the marker capturing unit can reliably capture an image of the marker member, providing a program that can reliably set a pivot position serving as a fulcrum for movement of a surgical instrument based on the marker member detected from the captured image. [Effects of the Invention]

[0012] According to the present disclosure, a pivot position that serves as a fulcrum for the movement of a surgical instrument can be reliably set. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram showing the configuration of a surgery support system according to a first embodiment. [Figure 2] FIG. 2 is a diagram showing a display unit of the medical cart according to the first embodiment. [Figure 3] 1 is a diagram showing the configuration of a medical cart according to a first embodiment. FIG. [Figure 4] 1 is a diagram showing a configuration of a robot arm according to a first embodiment. FIG. [Figure 5] FIG. [Figure 6] FIG. 2 is a perspective view showing the configuration of an arm operating unit according to the first embodiment. [Figure 7] FIG. 10 is a diagram for explaining translational movement of a robot arm. [Figure 8] FIG. 10 is a diagram for explaining the rotational movement of the robot arm. [Figure 9] FIG. 1 is a diagram showing an endoscope. [Figure 10]FIG. 10 shows a pivot position setting tool. [Figure 11] FIG. 2 is a diagram showing an operation unit according to the first embodiment. [Figure 12] FIG. 2 is a diagram showing a wrist portion for a right hand according to the first embodiment. [Figure 13] FIG. 2 is a diagram showing a wrist portion for a left hand according to the first embodiment. [Figure 14] 1 is a perspective view showing a foot pedal according to a first embodiment. FIG. [Figure 15] FIG. 1 is a control block diagram of a surgery assistance system according to a first embodiment. [Figure 16] FIG. 2 is a control block diagram of the robot arm according to the first embodiment. [Figure 17] FIG. 2 is a control block diagram of the positioner and medical cart according to the first embodiment. [Figure 18] FIG. 2 is a control block diagram of an operation unit according to the first embodiment. [Figure 19] 10A and 10B are diagrams for explaining setting of a pivot position using a marker member attached to a patient. [Figure 20] 10A and 10B are diagrams for explaining setting of a pivot position using a marker member attached to a trocar. [Figure 21] FIG. 10 is a diagram showing an AR marker as a marker member. [Figure 22] 10 is a diagram showing a state in which an image of a marker member is captured by a marker capturing unit arranged on an arm base. FIG. [Figure 23] FIG. 10 is a diagram showing a patient, a trocar, and a marker member displayed on a display unit. [Figure 24] 10A and 10B are diagrams for explaining a method of setting a pivot position from a marker member attached to a patient. [Figure 25] FIG. 10 illustrates pivot positions set on the patient's abdominal wall. [Figure 26] FIG. 2 is a flow chart for explaining a control method of the surgery assistance system according to the first embodiment. [Figure 27] FIG. 10 is a diagram showing a marker imaging unit disposed on a robot arm. [Figure 28] FIG. 10 is a diagram showing two marker members arranged for one robot arm. [Figure 29] FIG. 10 is a diagram showing a marker imaging unit placed on an operating table. [Figure 30] FIG. 10 is a diagram showing an endoscope as a marker imaging unit. DETAILED DESCRIPTION OF THE INVENTION

[0014] [First embodiment] (Configuration of surgical support system) The following describes the configuration of a surgery assistance system 500 according to the first embodiment. As shown in Fig. 1, the surgery assistance system 500 includes a surgery assistance robot 100, a remote control device 200, a vision unit 300, and an image processing unit 400.

[0015] In this specification, the longitudinal direction of the surgical instrument 1 is defined as the Z direction, as shown in Figure 4. The distal end side of the surgical instrument 1 is defined as the Z1 side, and the proximal end side of the surgical instrument 1 is defined as the Z2 side. The direction perpendicular to the Z direction is defined as the X direction. The direction perpendicular to the Z direction and the X direction is defined as the Y direction.

[0016] As shown in FIG. 1, a surgical support robot 100 is placed in an operating room. A remote control device 200 is placed at a location remote from the surgical support robot 100. The remote control device 200 receives operations on a surgical instrument 1. Specifically, an operator such as a doctor inputs commands to the remote control device 200 to cause the surgical support robot 100 to perform a desired operation. The remote control device 200 transmits the input commands to the surgical support robot 100. The surgical support robot 100 operates based on the received commands. The surgical support robot 100 is placed in an operating room, which is a sterilized sterile field.

[0017] (Configuration of surgical support robot) As shown in Figure 1, the surgical support robot 100 includes a medical cart 10, a cart positioner operating unit 20, a positioner 30, an arm base 40, multiple robot arms 50, and an arm operating unit 60 provided on each robot arm 50.

[0018] 3, the cart positioner operating unit 20 is supported by a cart positioner operating support unit 21 at the rear of the medical cart 10, and the medical cart 10 or the positioner 30 is moved by operating the cart positioner operating unit 20. The cart positioner operating unit 20 includes an input device 22 and an operating handle 23. The input device 22 receives operations to move and change the posture of the positioner 30, the arm base 40, and the multiple robot arms 50, mainly to prepare for surgery before the procedure.

[0019] As shown in Fig. 3, the input device 22 includes a display unit 22a, a joystick 22b, an enable switch 22c, an error reset button 22d, and a speaker 22e. The display unit 22a is, for example, a liquid crystal panel. As shown in Fig. 2, the display unit 22a displays numbers corresponding to the multiple robot arms 50. The display unit 22a also displays the type of surgical instrument 1 attached to each of the multiple robot arms 50. The display unit 22a displays a check mark CM indicating that a pivot position PP, which will be described later, has been set.

[0020] 3, the joystick 22b is disposed near the display unit 22a of the input device 22. By selecting an operation mode displayed on the display unit 22a and operating the joystick 22b, the positioner 30 is moved three-dimensionally.

[0021] The enable switch 22c is disposed near the joystick 22b. The enable switch 22c permits or prohibits movement of the positioner 30. When the enable switch 22c is pressed down to permit movement of the positioner 30, the positioner 30 is moved by operating the joystick 22b.

[0022] The error reset button 22d resets an error in the surgery support system 500. The error may be, for example, a deviation abnormality error. The speaker 22e is provided as a pair. The pair of speakers 22e is provided near the location of the positioner 30 on the medical cart 10.

[0023] The operating handle 23 is disposed near the display unit 22a. The operating handle 23 has a throttle 23a that is gripped and rotated by an operator such as a nurse or technician to control the movement of the medical cart 10. Specifically, the operating handle 23 is disposed below the input device 22. The medical cart 10 moves forward when the throttle 23a is rotated from the front side to the back side. The medical cart 10 moves backward when the throttle 23a is rotated from the back side to the front side. The speed of the medical cart 10 is changed according to the amount of rotation of the throttle 23a. The operating handle 23 is configured to be rotatable left and right, indicated by the R direction, and the medical cart 10 rotates as the operating handle 23 is rotated.

[0024] An enable switch 23b that permits or prohibits movement of the medical cart 10 is disposed on the operating handle 23. When the enable switch 23b is pressed down to permit movement of the medical cart 10, the medical cart 10 is moved by operating the throttle 23a of the operating handle 23.

[0025] 1, the positioner 30 is, for example, a seven-axis articulated robot. The positioner 30 is placed on a medical cart 10. The positioner 30 adjusts the position of the arm base 40. The positioner 30 moves the position of the arm base 40 three-dimensionally.

[0026] The positioner 30 includes a base portion 31 and a plurality of link portions 32 connected to the base portion 31. The plurality of link portions 32 are connected to each other by joints 33.

[0027] The arm base 40 is attached to the tip of the positioner 30. The base ends of the multiple robot arms 50 are attached to the arm base 40. The multiple robot arms 50 can be folded and stored. The arm base 40 and the multiple robot arms 50 are covered with a sterile drape when in use. The robot arms 50 support a surgical instrument 1. A marker imaging unit 70, which will be described later, is disposed on the arm base 40. The marker imaging unit 70 is an example of an imaging unit for alignment.

[0028] 15, a status indicator 41 and an arm status indicator 42 are arranged on the arm base 40. The status indicator 41 displays the status of the surgery assistance system 500. The arm status indicator 42 displays the status of the robot arm 50.

[0029] A plurality of robot arms 50 are provided. Specifically, four robot arms 50a, 50b, 50c, and 50d are provided. The robot arms 50a, 50b, 50c, and 50d have the same configuration as each other.

[0030] As shown in FIG. 4, the robot arm 50 includes an arm section 51, a first link section 52, a second link section 53, and a translational movement mechanism section 54. The robot arm 50 has joints JT1, JT2, JT3, JT4, JT5, JT6, JT7, and JT8. The joints JT1, JT2, JT3, JT4, JT5, JT6, and JT7 have A1, A2, A3, A4, A5, A6, and A7 axes as rotation axes, respectively. JT8 has A8 axis as a linear motion axis. The arm section 51 includes a base section 51a and a link section 51b.

[0031] The arm unit 51 is a seven-axis articulated robot arm. The first link unit 52 is located at the tip of the arm unit 51. The arm operating unit 60, which will be described later, is attached to the second link unit 53. The translational movement mechanism 54 is located between the first link unit 52 and the second link unit 53. A holder 55 that holds the surgical instrument 1 is located on the second link unit 53. The translational movement mechanism 54 translates the holder 55, to which the surgical instrument 1 is attached, between a first position and a second position. The first position is the end position on the Z2 side of the range of movement of the holder 55 by the translational movement mechanism 54 along the A8 axis. The second position is the end position on the Z1 side of the range of movement of the holder 55 by the translational movement mechanism 54 along the A8 axis.

[0032] A surgical instrument 1 is attached to the tip of each of the multiple robot arms 50. The surgical instrument 1 includes, for example, an interchangeable instrument 2, an endoscope 3 shown in FIG. 9 for capturing an image of the surgical site, and a pivot position setting instrument 4 shown in FIG. 10 for setting a pivot position PP. The instrument 2 includes a driven unit 2a, an end effector 2b, a wrist joint 2c shown in FIG. 5, and a shaft 2d. The end effector 2b is connected to the tip of the shaft 2d via the wrist joint 2c.

[0033] 1, an endoscope 3 is attached to the tip of one of the multiple robot arms 50, for example, robot arm 50c, and instruments 2 are attached to the tips of the remaining robot arms 50a, 50b, and 50d. Of the four robot arms 50 arranged adjacent to each other, it is desirable that the endoscope 3 be attached to one of the two robot arms 50b and 50c arranged in the middle.

[0034] (Instrument configuration) 5, an end effector 2b having, for example, jaw members 2g and 2h is attached to the tip of the instrument 2. As the end effector 2b, scissors, graspers, needle holders, microdissectors, stable appliers, tackers, suction and cleaning tools, snare wires, clip appliers, and the like can be used.

[0035] The instrument 2 includes a first support member 2e and a second support member 2f. The first support member 2e is attached to a shaft 2d. The second support member 2f is supported by the first support member 2e so as to be rotatable about the A10 axis, and supports the end effector 2b so as to be rotatable about the A11 axis that intersects with the A10 axis. The shaft 2d rotates about the A9 axis. The wrist joint 2c is provided between the second support member 2f and the first support member 2e, with the A10 axis as its rotation axis.

[0036] (Arm operation unit configuration) 6, the arm operating unit 60 is attached to the robot arm 50 and operates the robot arm 50. Specifically, the arm operating unit 60 is attached to the second link unit 53.

[0037] The arm operating unit 60 includes an enable switch 61 , a joystick 62 , a linear switch 63 , a mode switching button 64 , a mode indicator 65 , a pivot button 66 , and an adjustment button 67 .

[0038] When the enable switch 61 is pressed, it allows or disallows movement of the robot arm 50 using the joystick 62 and the linear switch 63. When the enable switch 61 is pressed while the arm operating unit 60 is being held by an operator such as a nurse or assistant, movement of the surgical instrument 1 by the robot arm 50 is permitted.

[0039] The joystick 62 is an operating tool for controlling the movement of the surgical instrument 1 by the robot arm 50. The joystick 62 controls the movement direction and movement speed of the robot arm 50. The robot arm 50 moves according to the direction and angle at which the joystick 62 is tilted.

[0040] The linear switch 63 is a switch for moving the surgical instrument 1 in the Z direction, which is the longitudinal direction of the surgical instrument 1. The linear switch 63 includes a linear switch 63a for moving the surgical instrument 1 in the direction of inserting it into the patient P, and a linear switch 63b for moving the surgical instrument 1 in the direction away from the patient P. Both the linear switch 63a and the linear switch 63b are push button switches.

[0041] The mode switching button 64 is a push button switch for switching between a translational movement mode and a rotational movement mode of the surgical instrument 1. As shown in FIG. 7, in the translational movement mode of the robot arm 50, the robot arm 50 is moved so that the tip 1a of the surgical instrument 1 moves on the XY plane. As shown in FIG. 8, in the rotational movement mode of the robot arm 50, when the pivot position PP is not stored in the memory unit 351, the robot arm 50 is rotated around the center of the end effector 2b of the instrument 2 (as the surgical instrument 1) on the A11 axis or the tip of the end effector 2b as a fulcrum. When the pivot position PP is stored in the memory unit 351, the robot arm 50 is moved so that the surgical instrument 1 is rotated around the pivot position PP as a fulcrum. Note that the surgical instrument 1 is rotated with the shaft 1c of the surgical instrument 1 inserted into the trocar T. The mode switching button 64 is located on the Z-direction surface of the arm operating unit 60.

[0042] The mode indicator 65 displays the switched mode. When the mode indicator 65 is lit, it indicates the rotational movement mode, and when it is off, it indicates the translational movement mode. The mode indicator 65 also serves as a pivot position indicator that indicates that the pivot position PP has been set. The mode indicator 65 is located on the surface of the arm operation unit 60 on the Z direction side.

[0043] The pivot button 66 is a push button switch for manually setting a pivot position PP, which serves as a fulcrum for the movement of the surgical instrument 1 attached to the robot arm 50, without using a marker member M, which will be described later.

[0044] The adjustment button 67 is a button for optimizing the position of the robot arm 50. After setting the pivot position PP for the robot arm 50 to which the endoscope 3 is attached, pressing the adjustment button 67 optimizes the positions of the other robot arms 50 and the arm base 40. The adjustment button 67 is a button different from the enable switch 61.

[0045] (remote control device) 1, the remote control device 200 is placed, for example, inside or outside an operating room. The remote control device 200 includes an operation unit 110, a foot pedal 120, a touch panel 130, a monitor 140, a support arm 150, a support bar 160, and an error reset button 161. The operation unit 110 constitutes an operation handle that allows an operator, such as a doctor, to input commands.

[0046] (Operation unit) As shown in FIG. 11 , the operating unit 110 is a handle for operating the surgical instrument 1. The operating unit 110 also receives operations on the surgical instrument 1. When viewed from an operator such as a doctor, the operating unit 110 includes an operating unit 110L located on the left side and operated with the operator's left hand, and an operating unit 110R located on the right side and operated with the operator's right hand. The operating unit 110 includes an arm unit 111 and a wrist unit 112. The operating unit 110R includes an arm unit 111R and a wrist unit 112R. The operating unit 110L also includes an arm unit 111L and a wrist unit 112L.

[0047] The arm unit 111 has joints JT21, JT22, and JT23 shown in Figure 11, and JT24, JT25, JT26, and JT27 shown in Figures 12 and 13. The rotation axes of the joints JT21, JT22, JT23, JT24, JT25, JT26, and JT27 are defined as A21, A22, A23, A24, A25, A26, and A27 axes, respectively.

[0048] (Arm part) As shown in FIG. 11 , the arm 111R has link portions 111a, 111b, and 111c. The upper end of link portion 111a is attached to the remote control device 200 so as to be rotatable around a vertical axis A21. The upper end of link portion 111b is attached to the lower end of link portion 111a so as to be rotatable around a horizontal axis A22. One end of link portion 111c is attached to the lower end of link portion 111b so as to be rotatable around a horizontal axis A23. The wrist portion 112 is attached to the other end of link portion 111c so as to be rotatable around an A24 axis. Link portion 111a is connected to the remote control device 200 by a joint JT21. Link portions 111a and 111b are connected by a joint JT22. Link portions 111b and 111c are connected by a joint JT23. The arm portion 111 supports the wrist portion 112. The arm portion 111L has the same configuration as the arm portion 111R.

[0049] Wrist section 112 includes wrist section 112R, which is operated by the operator's right hand as shown in Fig. 12, and wrist section 112L, which is operated by the operator's left hand as shown in Fig. 13. Fig. 12 shows the reference position of operation section 110R, and Fig. 13 shows the reference position of operation section 110L. Wrist section 112R and wrist section 112L have the same configuration.

[0050] The wrist unit 112 includes link units 112a, 112b, 112c, and a grip support member 112d that is operated by an operator such as a doctor. The base end of link unit 112a is connected to the tip end of the arm unit 111 and rotates around the A24 axis. The base end of link unit 112b is connected to the tip end of link unit 112a and rotates around the A25 axis. The base end of link unit 112c is connected to the tip end of link unit 112b and the grip support member 112d is connected to the tip end of link unit 112c and rotates around the A26 axis relative to link unit 112b. The grip support member 112d rotates around the A27 axis relative to link unit 112c. The link units 112a, 112b, and 112c each have an L-shape.

[0051] The wrist section 112 includes a pair of grip members 112e that can be opened and closed by the operator. The grip members 112e are made of elongated, plate-like lever members, and the proximal ends of each of the pair of grip members 112e are rotatably connected to the proximal end of the grip support member 112d. Cylindrical finger insertion sections 112f are provided on the grip members 112e. The operator inserts their fingers into the pair of finger insertion sections 112f to operate the wrist section 112. The base ends of each of the pair of grip members 112e are connected to the grip support member 112d, and the opening angle between the jaw members 2g and 2h is changed by increasing or decreasing the angle between the pair of grip members 112e. A magnet is provided on one of the grip members 112e, and a Hall sensor is provided on the grip support member 112d. When the operator opens or closes the grip member 112e, the magnet and Hall sensor function as an angle detection sensor, and the Hall sensor outputs the opening angle. As an angle detection sensor, a Hall sensor may be disposed on the grip member 112e and a magnet may be disposed on the grip support member 112d. Alternatively, a magnet or a Hall sensor may be disposed on both of the grip members 112e.

[0052] As shown in FIG. 1 , the monitor 140 is a scope-type display device for displaying an image captured by the endoscope 3. The monitor 140 also has an alarm unit 141. The alarm unit 141 issues an error sound. The support arm 150 supports the monitor 140 so that the height of the monitor 140 is at the same height as the face of an operator such as a doctor. The touch panel 130 is mounted on a support bar 160. The surgical support robot 100 can be operated by the remote control device 200 by detecting the operator's head with a sensor provided near the monitor 140. The operator operates the operation unit 110 and the foot pedal 120 while visually checking the affected area on the monitor 140. This inputs commands to the remote control device 200. The commands input to the remote control device 200 are transmitted to the surgical support robot 100.

[0053] (foot pedal) As shown in FIG. 14 , a plurality of foot pedals 120 are provided to perform functions related to the surgical instrument 1. The plurality of foot pedals 120 are arranged on a base 121. The foot pedals 120 include a switching pedal 122, a clutch pedal 123, a camera pedal 124, an incision pedal 125, a coagulation pedal 126, and a foot detector 127. The switching pedal 122, the clutch pedal 123, the camera pedal 124, the incision pedal 125, and the coagulation pedal 126 are operated by the operator's feet. The incision pedals 125 include a incision pedal 125R for the right robot arm 50 and a incision pedal 125L for the left robot arm 50. The coagulation pedals 126 include a coagulation pedal 126R for the right robot arm 50 and a coagulation pedal 126L for the left robot arm 50.

[0054] The switching pedal 122 switches the robot arm 50 operated by the operation unit 110. The clutch pedal 123 performs a clutch operation that temporarily disconnects the operational connection between the robot arm 50 and the operation unit 110. While the clutch pedal 123 is depressed by the operator, the operation by the operation unit 110 is not transmitted to the robot arm 50. Furthermore, while the operator is depressing the camera pedal 124, the operation unit 110 can operate the robot arm 50 to which the endoscope 3 is attached. While the operator is depressing the incision pedal 125 or the coagulation pedal 126, the electrosurgical device is activated.

[0055] (Vision unit and image processing unit) 1, the vision unit 300 and the image processing unit 400 are placed on a cart 210. The image processing unit 400 processes images captured by the endoscope 3. A display unit 220 is disposed on the cart 210. The image captured by the endoscope 3 is displayed on the display unit 220.

[0056] (Control system configuration) 15, the surgery assistance system 500 includes a first control device 310, an arm control device 320, a positioner control device 330, an operation control device 340, and a second control device 350. The surgery assistance system 500 also includes a storage unit 311 connected to the first control device 310 and a storage unit 351 connected to the second control device 350. The second control device 350 is an example of a control unit. The storage unit 311 stores a program 311a executed by the first control device 310. The storage unit 351 stores a program 351a executed by the second control device 350.

[0057] The first control device 310 is disposed inside the medical cart 10 so as to communicate with the arm control device 320 and the positioner control device 330, and controls the entire surgery support system 500. Specifically, the first control device 310 communicates with and controls each of the arm control device 320, the positioner control device 330, and the operation control device 340. The first control device 310, the arm control device 320, the positioner control device 330, and the operation control device 340 are connected via a LAN or the like. The first control device 310 is disposed inside the medical cart 10.

[0058] An arm control unit 320 is provided for each of the plurality of robot arms 50. That is, a plurality of arm control units 320 corresponding to the number of the plurality of robot arms 50 are provided inside the medical cart 10.

[0059] As shown in Fig. 15, the input device 22 is connected to the first control device 310 via a LAN or the like. The status indicator 41, arm status indicator 42, operating handle 23, throttle 23a, joystick 22b, and positioner control section 330 are serially connected via wiring 360 through a communication network that allows them to share information with each other. Note that Fig. 15 shows the status indicator 41, arm status indicator 42, and the like as if they were all connected to one wiring 360, but in reality, a wiring 360 is provided for each of the status indicator 41, arm status indicator 42, operating handle 23, throttle 23a, joystick 22b, stabilizer 24, and electric cylinder 25.

[0060] As shown in FIG. 16, the arm 51 is provided with a plurality of servo motors SM1, an encoder EN1, and a reducer corresponding to each of the joints JT1, JT2, JT3, JT4, JT5, JT6, and JT7. The encoder EN1 detects the rotation angle of the servo motor SM1. The reducer decelerates the rotation of the servo motor SM1 to increase the torque. Inside the medical cart 10, a servo control unit SC1 for controlling the servo motor SM1 is disposed adjacent to the arm control unit 320. The servo control unit SC1 is electrically connected to the encoder EN1 for detecting the rotation angle of the servo motor SM1.

[0061] The second link section 53 is provided with a servo motor SM2 for rotating a driven member disposed in the driven unit 2a of the surgical instrument 1, an encoder EN2, and a reducer. The encoder EN2 detects the rotation angle of the servo motor SM2. The reducer reduces the rotation speed of the servo motor SM2 to increase the torque. The medical cart 10 is also provided with a servo control section SC2 for controlling the servo motor SM2 that drives the surgical instrument 1. The servo control section SC2 is electrically connected to an encoder EN2 for detecting the rotation angle of the servo motor SM2. Note that multiple servo motors SM2, encoders EN2, and servo control sections SC2 are provided.

[0062] The translational movement mechanism 54 is provided with a servo motor SM3 for translating the surgical instrument 1, an encoder EN3, and a reducer. The encoder EN3 detects the rotation angle of the servo motor SM3. The reducer decelerates the rotation of the servo motor SM3 to increase the torque. The medical cart 10 also has a servo control unit SC3 for controlling the servo motor SM3 for translating the surgical instrument 1. The encoder EN3 for detecting the rotation angle of the servo motor SM3 is electrically connected to the servo control unit SC3.

[0063] The first control device 310 generates command values ​​that command the positions of the servo motors SM1, SM2, and SM3 based on the operation received by the remote operation device 200, and drives the servo motors SM1, SM2, and SM3 based on the command values. The first control device 310 then detects a deviation abnormality error when the difference between the command values ​​and the positions of the servo motors SM1, SM2, and SM3 detected by the sensors exceeds an allowable range.

[0064] 17, the positioner 30 is provided with a plurality of servo motors SM4, an encoder EN4, and a reducer so as to correspond to a plurality of joints 33 of the positioner 30. The encoder EN4 is configured to detect the rotation angle of the servo motor SM4. The reducer is configured to reduce the rotation speed of the servo motor SM4 to increase the torque.

[0065] The medical cart 10 is equipped with wheels, including front wheels as drive wheels and rear wheels steered by the operating handle 23. The rear wheels are located closer to the operating handle 23 than the front wheels. The medical cart 10 also includes a servo motor SM5 that drives each of the front wheels of the medical cart 10, an encoder EN5, a reducer, and a brake BRK. The reducer is configured to reduce the rotation of the servo motor SM5 and increase the torque. The operating handle 23 is also provided with a potentiometer P1, as shown in FIG. 3, and the servo motor SM5 of the front wheels is driven based on the rotation angle detected by the potentiometer P1 in response to the twist of the throttle 23a. The rear wheels of the medical cart 10 are dual-wheel type, and are steered based on the left and right rotation of the operating handle 23. 3 is disposed on the rotation shaft of the operating handle 23, and a servomotor SM6, an encoder EN6, and a reducer are disposed on the rear wheels of the medical cart 10. The reducer is configured to reduce the rotation speed of the servomotor SM6 and increase the torque. The servomotor SM6 is driven based on the rotation angle detected by the potentiometer P2 in response to the left and right rotation of the operating handle 23. In other words, steering of the rear wheels by the left and right rotation of the operating handle 23 is configured to be power-assisted by the servomotor SM6.

[0066] The medical cart 10 moves forward and backward by driving the front wheels, and the rear wheels are steered by turning the operating handle 23, causing the medical cart 10 to rotate left and right.

[0067] As shown in FIG. 17, the medical cart 10 is provided with a servo control unit SC4 for controlling a servo motor SM4 that moves the positioner 30. An encoder EN4 for detecting the rotation angle of the servo motor SM4 is electrically connected to the servo control unit SC4. The medical cart 10 is also provided with a servo control unit SC5 for controlling a servo motor SM5 that drives the front wheels of the medical cart 10. An encoder EN5 for detecting the rotation angle of the servo motor SM5 is electrically connected to the servo control unit SC5. The medical cart 10 is also provided with a servo control unit SC6 for controlling a servo motor SM6 that power-assists the steering of the rear wheels of the medical cart 10. An encoder EN6 for detecting the rotation angle of the servo motor SM6 is electrically connected to the servo control unit SC6.

[0068] As shown in FIGS. 16 and 17 , brakes BRK are mounted on the joints JT1, JT2, JT3, JT4, JT5, JT6, and JT7 of the arm unit 51 and on the joint 33 of the positioner 30. Brakes BRK are also mounted on the front wheels of the medical cart 10, the arm base 40, and the translational movement mechanism 54. Control signals are transmitted unidirectionally from the arm control unit 320 to the brakes BRK mounted on the joints JT1, JT2, JT3, JT4, JT5, JT6, and JT7 of the arm unit 51 and on the translational movement mechanism 54. The control signals are signals that turn the brakes BRK on and off. The signal that turns the brakes BRK on includes a signal that keeps the brakes BRK engaged. The same applies to control signals sent from the positioner control unit 330 to the brakes BRK mounted on the joints 33 of the positioner 30 and on the arm base 40. At startup, all brakes BRK on the arm base 40, arm unit 51, and translational movement mechanism 54 are released, and the servo motor SM is driven to resist gravity, thereby maintaining the posture of the robot arm 50 and the posture of the arm base 40. When an error occurs in the surgery support system 500, the brakes BRK on the arm base 40, arm unit 51, and translational movement mechanism 54 are turned on. When the error in the surgery support system 500 is resolved, the brakes BRK on the arm base 40, arm unit 51, and translational movement mechanism 54 are turned off. A shutdown operation of the surgery support system 500 turns on the brakes BRK on the arm base 40, arm unit 51, and translational movement mechanism 54. In addition, the brakes BRK on the front wheels of the medical cart 10 are always turned on, and are released only while the enable switch 23b is pressed down. Furthermore, the brake BRK of each joint 33 of the positioner 30 is always on, and the brake BRK is released only while the enable switch 22c is pressed.

[0069] As shown in FIG. 18, servo motors SM7a, SM7b, SM7c, SM7d, SM7e, SM7f, and SM7g are disposed at joints JT21, JT22, JT23, JT24, JT25, JT26, and JT27 of the operation unit 110, respectively. Servo motor SM7a rotates link portion 111a around the A21 axis. Servo motor SM7b rotates link portion 111b around the A22 axis. Servo motor SM7c rotates link portion 111c around the A23 axis. Servo motor SM7d rotates link portion 112a around the A24 axis. Servo motor SM7e rotates link portion 112b around the A25 axis. Servo motor SM7f rotates link portion 112c around the A26 axis. Servo motor SM7g rotates grip support member 112d around axis A27. Servo control units SC7a, SC7b, SC7c, SC7d, SC7e, SC7f, and SC7g are provided to control the servo motors. Encoders EN7a, EN7b, EN7c, EN7d, EN7e, EN7f, and EN7g are electrically connected to the servo control units to detect the rotation angles of the servo motors. The servo motors, servo control units, and encoders are provided in operation unit 110L and operation unit 110R, respectively.

[0070] The first control device 310 controls each servo motor via the operation control section 340 to generate a torque that cancels out the gravitational torque generated in the rotation shaft of each servo motor according to the attitude of the operation unit 110. This enables the operator to operate the operation unit 110 with a relatively small force.

[0071] Furthermore, when the operator performs an operation to rotate the grip support member 112d of the operation unit 110 around the A27 axis shown in Figures 12 and 13, the shaft 2d of the instrument 2 rotates around the A9 axis shown in Figure 5. Furthermore, when the operator performs an operation to rotate the joints JT24, JT25, and JT26 of the operation unit 110 shown in Figures 12 and 13, the end effector 2b bends around the A10 axis or A11 axis shown in Figure 5.

[0072] Operation control unit 340 is arranged on the main body of remote control device 200. Operation control unit 340 controls operation unit 110. As shown in Fig. 15, operation control unit 340 is arranged to correspond to operation unit 110L for the left hand and operation unit 110R for the right hand.

[0073] 15, the vision unit 300 and the image processing unit 400 are connected to a first control device 310 via a LAN or the like. The display unit 220 is connected to the vision unit 300.

[0074] (Pivot position setting) Next, setting of the pivot position PP using the marker imaging unit 70 will be described. In the first embodiment, a marker member M is attached to at least one of a patient P on whom surgery is performed using the surgical support robot 100 and a trocar T inserted into the body surface S of the patient P. The distance between the marker imaging unit 70 and the marker member M is, for example, approximately 1 m. In FIG. 19, the marker member M is attached to the body surface S of the patient P, and in FIG. 20, the marker member M is attached to the trocar T. The marker member M is detachable from at least one of the patient P and the trocar T. The marker member M is placed on at least one of the patient P and the trocar T by the operator. For example, an adhesive layer is formed on the marker member M, and the marker member M is attached to at least one of the patient P and the trocar T. The marker member M may be attached to both the body surface S of the patient P and the trocar T. The marker member M may be attached to a position that is easy for the marker imaging unit 70 to capture an image, based on the operator's judgment, for example.

[0075] In the first embodiment, as shown in FIG. 21 , the marker element M is a rectangular AR marker. The AR marker serves as a landmark for displaying augmented reality, which allows digital content to be displayed superimposed on real space. The AR marker is formed, for example, by combining white and black shapes. Information regarding the direction and distance from the marker element M at which a pivot position is to be set is associated with the marker element M. This associated information is stored in advance in at least one of the storage unit 311 and the storage unit 351. The length of one side of the marker element M is, for example, 20 mm or more and 30 mm or less.

[0076] Furthermore, in the first embodiment, as described above, a plurality of robot arms 50 to which surgical instruments 1 are attached are disposed. For example, four robot arms 50a, 50b, 50c, and 50d are disposed. A plurality of AR markers are disposed corresponding to the plurality of robot arms 50. In FIGS. 19 and 20, four AR markers are affixed corresponding to the four robot arms 50a, 50b, 50c, and 50d. The plurality of AR markers are linked with at least one of information about the corresponding robot arm 50 and information about the type of surgical instrument 1 attached to the robot arm 50. For example, information indicating which of the four robot arms 50a, 50b, 50c, and 50d each of the four AR markers corresponds to is stored in advance in at least one of the storage unit 311 and the storage unit 351. In addition, information as to whether the surgical tool 1 attached to each of the four robot arms 50a, 50b, 50c, and 50d is an instrument 2 or an endoscope 3 is stored in advance in at least one of the memory unit 311 and the memory unit 351.

[0077] The marker imaging unit 70 is disposed in at least one of the robot arm 50, the operating table B on which the patient P is placed, and the arm base 40 to which the robot arm 50 is attached. In the first embodiment, as shown in FIG. 22 , the marker imaging unit 70 is disposed in the arm base 40 to which the robot arm 50 is attached. The marker imaging unit 70 attached to the arm base 40 also serves as an imaging unit for positioning the surgical support robot 100. Specifically, when the operator moves the surgical support robot 100 to the vicinity of the patient P placed on the operating table B, as shown in FIG. 23 , an image of the patient P captured by the marker imaging unit 70 is displayed in real time on the display unit 22 a. As shown in FIG. 23 , a first mark MK1 and a second mark MK2 are displayed on the display unit 22 a. The first mark MK1 is circular, and the second mark MK2 is cross-shaped. The display unit 22a displays the body surface S of the patient P photographed by the marker imaging unit 70 so as to overlap the first mark portion MK1 and the second mark portion MK2. While visually checking the display unit 22a, the operator moves the surgical support robot 100 so that the trocar T, into which the endoscope 3 is inserted, is positioned inside the first mark portion MK1.

[0078] In the first embodiment, the marker imaging unit 70 captures an image of a marker element M detachably attached to at least one of a patient P and a trocar T inserted into a body surface S of the patient P. The marker imaging unit 70 also includes a two-dimensional camera. The captured image of the marker element M captured by the marker imaging unit 70 is transmitted to at least one of the first control device 310 and the second control device 350. An example in which the image of the marker element M is transmitted to the second control device 350 will be described below. In this case, information linked to the marker element M is stored in the storage unit 351.

[0079] In the first embodiment, the second control device 350 detects the marker element M based on a captured image of the marker element M. The second control device 350 then sets a pivot position PP, which serves as a fulcrum for the movement of the surgical instrument 1, based on the detected marker element M. Specifically, the second control device 350 reads information about the shape of the marker element M from the captured image of the marker element M and reads out information associated with the read marker element M stored in the storage unit 351. As described above, the information associated with the marker element M includes information about which robot arm 50 the read marker element M corresponds to, which of the instrument 2 and the endoscope 3 is attached to the corresponding robot arm 50, and, as shown in FIG. 24 , the direction and distance from the marker element M at which the pivot position PP is to be set. The second control device 350 also calculates the position and orientation of the marker element M from the captured image of the marker element M. The position and orientation of the marker element M are calculated by having the marker image capturing unit 70 capture images of the marker element M arranged in various positions and orientations in advance, and calibrating the marker image capturing unit 70 based on the captured images. For example, the marker element M is captured from multiple angles by the marker image capturing unit 70 to identify the lens distortion and focal length of the marker image capturing unit 70. The second control device 350 then calculates the position and orientation of the marker element M based on the known side length of the actually captured marker element M and the calibration results. The second control device 350 then sets the pivot position PP at coordinates separated by the read distance and in the direction read from the storage unit 351 based on the position and orientation of the marker element M in the captured image. Note that, in reality, the coordinates of the marker element M detected from the captured image are in the coordinate system of the marker image capturing unit 70. Therefore, the coordinates of the marker element M are converted from the coordinate system of the marker image capturing unit 70 to the coordinate system of the surgical support robot 100, and the pivot position PP is then set based on the direction and distance read from the storage unit 351.The second control device 350 may detect the coordinates of the marker member M from the captured image, and set the pivot position to a coordinate spaced from the detected coordinates by the read distance in the direction read from the storage unit 351, and then further moved a predetermined distance into the abdominal wall or pleura of the patient P. For example, as shown in Fig. 25, the pivot position PP is set to the center position in the thickness direction of the abdominal wall of the portion of the abdominal wall where the trocar T of the patient P is inserted. The accuracy of detecting the position of the marker member M is, for example, about ±10 mm.

[0080] Furthermore, in the first embodiment, the marker imaging unit 70 captures images of the marker element M in real time. The second control device 350 changes the pivot position PP based on the captured images of the marker element M captured in real time. Specifically, the marker imaging unit 70 captures moving images of the marker element M. The second control device 350 detects the marker element M in the captured images at predetermined time intervals. The second control device 350 then sets a new pivot position PP based on the newly detected marker element M. Note that the second control device 350 may change the pivot position PP when the error between the pivot position PP based on the newly detected marker element M and the currently set pivot position PP is equal to or greater than a predetermined threshold.

[0081] (Method for controlling a surgical assistance system) A control method for the surgery assistance system 500 will be described.

[0082] First, as a preliminary preparation, the operator attaches a marker member M to at least one of the patient P and the trocar T. If multiple robot arms 50 are arranged on the surgery support robot 100, the operator attaches the marker members M in a number corresponding to the number of the multiple robot arms 50.

[0083] As shown in Figure 26, in step S1, the operator operates the operation handle 23 of the cart positioner operation unit 20, thereby starting the movement of the surgical support robot 100. The surgical support robot 100 is moved to the vicinity of the patient P placed on the operating table B. The movement of the surgical support robot 100 is controlled by the first control device 310.

[0084] In step S2, the marker imaging unit 70 starts imaging. Note that the imaging by the marker imaging unit 70 is performed continuously. That is, the marker imaging unit 70 captures a moving image. Then, as shown in FIG. 22, when the surgery support robot 100 is moved near the patient P placed on the operating table B, the marker imaging unit 70 captures an image of the marker member M detachably attached to at least one of the patient P and the trocar T inserted into the body surface S of the patient P, as shown in FIG. 23.

[0085] In step S3, the second control device 350 detects the marker element M based on the captured image of the marker element M. For example, the second control device 350 recognizes the shape of the marker element M from the captured image of the marker element M. Note that the detection of the marker element M may be performed automatically by the second control device 350 when the marker imaging unit 70 captures an image of the marker element M, or the second control device 350 may start the operation of recognizing the marker element M from the captured image based on an input operation by the operator.

[0086] In step S4, the second control device 350 sets a pivot position PP, which serves as a fulcrum for the movement of the surgical instrument 1, based on the detected marker member M. For example, the second control device 350 reads out the shape of the marker member M recognized in step S3 and information associated with the marker member M that is stored in advance in the storage unit 351. Then, based on the read-out information, the second control device 350 sets a pivot position PP for the robot arm 50 that corresponds to the detected marker member M.

[0087] In step S5, the second control device 350 determines whether pivot positions PP have been set for all of the robot arms 50. If the answer is no in step S5, the operations of steps S3 and S4 are repeated. Note that, since the marker member M contains information indicating which robot arm 50 it corresponds to, it is also possible to perform detection of the marker member M for all of the robot arms 50 in step S3, and then set pivot positions PP for all of the robot arms 50 simultaneously in step S4.

[0088] In step S6, the second control device 350 changes the pivot position PP based on the captured image of the marker member M captured in real time. Note that the operation of step S6 is performed while the surgery is being performed by the surgery assistance system 500.

[0089] [Effects of the first embodiment] The surgery assistance system 500 includes a marker imaging unit 70 that captures an image of a marker element M attached to at least one of a patient P and a trocar T, and a second control device 350 that detects the marker element M based on the captured image of the marker element M and sets a pivot position PP that serves as a fulcrum for movement of the surgical instrument 1 based on the detected marker element M. As a result, the marker element M is detachable from the patient P and the trocar T, so that the marker element M can be attached to a position that does not expose other medical equipment or the like. Therefore, the marker imaging unit 70 can reliably capture an image of the marker element M, and therefore the pivot position PP that serves as a fulcrum for movement of the surgical instrument 1 can be reliably set based on the marker element M detected from the captured image.

[0090] Furthermore, when setting the pivot position PP using the pivot button 66, if the operator's skill level is low, the pivot position PP may not be set correctly. As described above, the second control device 350 sets the pivot position PP based on the detected marker member M, allowing the pivot position PP to be set accurately and in a short time. If the patient P's body shifts after the pivot position PP is set, the trocar T and the surgical instrument 1 may come into contact with each other with relatively high pressure. In this case, when the surgical instrument 1 is moved, the friction between the trocar T and the surgical instrument 1 causes the surgical instrument 1 and the robot arm 50 holding the surgical instrument 1 to vibrate. By accurately setting the pivot position PP as described above, the vibration of the surgical instrument 1 and the robot arm 50 holding the surgical instrument 1 can be suppressed.

[0091] The marker member M includes a rectangular AR marker, the marker imaging unit 70 includes a two-dimensional camera, and the second control device 350 sets the pivot position PP based on an image of the AR marker captured by the two-dimensional camera. This makes it possible to easily set the pivot position PP based on the image of the AR marker captured by the two-dimensional camera, for example, by linking the AR marker with information such as the direction and distance the pivot position PP should be shifted from the AR marker.

[0092] A plurality of robot arms 50 to which surgical instruments 1 are attached are arranged, and a plurality of AR markers are arranged corresponding to the plurality of robot arms 50, and the plurality of AR markers are linked to at least one of information about the corresponding robot arm 50 and information about the type of surgical instrument 1 attached to the robot arm 50. This allows the second control device 350 to recognize which robot arm 50 and which type of surgical instrument 1 the captured AR marker corresponds to, and therefore allows the pivot position PP to be appropriately set for the robot arm 50 and surgical instrument 1 linked to the captured AR marker.

[0093] The marker imaging unit 70 is disposed on the arm base 40 to which the robot arm 50 is attached. This prevents the marker imaging unit 70 from interfering with other medical devices as the robot arm 50 moves, unlike when the marker imaging unit 70 is disposed on the robot arm 50.

[0094] The surgery support system 500 includes a marker imaging unit 70 as an imaging unit for alignment of the surgery support robot 100 attached to the arm base 40. When the surgery support robot 100 is moved near the patient P placed on the operating table B, an image of the patient P captured by the marker imaging unit 70 as an imaging unit for alignment is displayed in real time on the display unit 22a. In other words, the marker imaging unit 70 also serves as an imaging unit for alignment. This prevents the configuration of the surgery support system 500 from becoming complicated, unlike when the marker imaging unit 70 is provided separately from the imaging unit for alignment.

[0095] The marker imaging unit 70 captures an image of the marker member M in real time, and the second control device 350 changes the pivot position PP based on the captured image of the marker member M captured in real time. As a result, the pivot position PP is changed in real time, so even if the patient P shifts from the operating table B, the surgical support robot 100 can be operated with the pivot position PP appropriately set. Furthermore, since fluctuations in the pivot position PP due to the patient P's breathing can be detected in real time, the surgical support robot 100 can be operated with the pivot position PP appropriately set, even if the pivot position PP shifts due to the patient P's breathing.

[0096] [Second embodiment] The configuration of a surgery assistance system 500 according to the second embodiment will be described.

[0097] As shown in FIG. 27 , in the second embodiment, the marker imaging unit 71 is disposed on the robot arm 50. The marker imaging unit 71 is attached to the robot arm 50 when setting the pivot position PP, and is detached from the robot arm 50 after setting the pivot position PP. For example, in step S1 of the control method for the surgery assistance system 500 of the first embodiment, the marker imaging unit 71 is attached to one of the four robot arms 50 before the surgery assistance robot 100 is moved near the patient P placed on the operating table B. For example, the marker imaging unit 71 is attached to the second link unit 53 of the robot arm 50. The marker imaging unit 71 is detachable from the second link unit 53. The marker imaging unit 71 may be attached to a portion of the robot arm 50 other than the second link unit 53. Other configurations of the second embodiment are similar to those of the first embodiment.

[0098] [Effects of the second embodiment] The marker imaging unit 70 is disposed on the robot arm 50. As a result, even if the marker member M is disposed in a position that is in a blind spot of other medical equipment, the marker imaging unit 70 can be moved by the robot arm 50, so that the marker member M can be imaged.

[0099] The marker imaging unit 70 is attached to the robot arm 50 when setting the pivot position PP, and after setting the pivot position PP, is detached from the robot arm 50. As a result, since the marker imaging unit 70 is detached from the robot arm 50 after setting the pivot position PP, it is possible to prevent the marker imaging unit 70 from interfering with other medical devices during surgery, etc.

[0100] [Variations] It should be noted that the embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present disclosure is defined by the claims, not by the description of the above embodiments, and further includes all modifications and variations within the meaning and scope of the claims.

[0101] In the above first and second embodiments, an example was shown in which the marker member M was disposed on either the patient P or the trocar T, but the present disclosure is not limited to this. For example, the marker member M may be disposed on both the patient P and the trocar T.

[0102] In the first and second embodiments, the marker member M is an AR marker, but the present disclosure is not limited to this. For example, if the marker imaging unit is a three-dimensional camera, the marker member M may be a simple pattern other than an AR marker.

[0103] In the above first and second embodiments, an example has been shown in which a plurality of marker members M are arranged to correspond to a plurality of robot arms 50, but the present disclosure is not limited to this. For example, if only one robot arm 50 is arranged, only one marker member M may be arranged. In this case, the marker member M does not need to include information regarding which robot arm 50 the marker member M corresponds to.

[0104] Although the first and second embodiments illustrate examples in which the marker imaging unit 70 is a two-dimensional camera, the present disclosure is not limited thereto. For example, the marker imaging unit 70 may be a three-dimensional camera. Examples of the three-dimensional camera include a stereo camera and a ToF (Time of Flight) camera. The second control device 350 sets the pivot position PP based on the captured image of the marker element M captured by the three-dimensional camera. That is, the second control device 350 calculates the distance to the marker element M, the position, and the orientation of the marker element M based on the captured image of the marker element M captured by the three-dimensional camera. The second control device 350 then sets the pivot position PP based on the calculated distance to the marker element M, the position, and the orientation of the marker element M, and the information read from the memory unit 351. In this way, since the marker imaging unit 70 is a three-dimensional camera, the three-dimensional camera can detect the three-dimensional shape of the subject, and therefore the pivot position PP, which serves as a fulcrum for the movement of the surgical instrument 1, can be appropriately set based on the captured image of the marker element M captured by the three-dimensional camera.

[0105] In the above-described first and second embodiments, an example has been described in which one marker member M is disposed for one robot arm 50. However, the present disclosure is not limited to this. For example, as shown in FIG. 28, a plurality of marker members M may be disposed for one robot arm 50. In FIG. 28, two marker members M are disposed for one robot arm 50. The second control device 350 then sets the pivot position PP based on the detected plurality of marker members M. For example, the second control device 350 sets the pivot position PP at the midpoint between the two marker members M. By disposing a plurality of marker members M for one robot arm 50 in this way, even in cases where the pivot position PP cannot be set accurately based on one marker member M, the pivot position PP can be set accurately based on the plurality of marker members M.

[0106] In the first embodiment, the marker imaging unit 70 is disposed on the arm base 40, and in the second embodiment, the marker imaging unit 71 is disposed on the robot arm 50. However, the present disclosure is not limited to this. For example, as shown in FIG. 29 , the marker imaging unit 72 may be disposed on the operating table B on which the patient P is placed. For example, the marker imaging unit 72 is disposed at a relatively high position so that a plurality of marker members M are within the field of view. In this way, by disposing the marker imaging unit 72 on the operating table B on which the patient P is placed, the marker imaging unit 72 moves together with the operating table B. Therefore, even if the operating table B is moved, the marker members M can be imaged by the marker imaging unit 72. Furthermore, the marker imaging unit may be disposed at a plurality of positions among the arm base 40, the robot arm 50, and the operating table B.

[0107] 30, the endoscope 3 disposed on the robot arm 50 may also function as a marker imaging unit 70 that images the marker members M. In this case, the endoscope 3 images the multiple marker members M, and the second control device 350 sets pivot positions PP for the multiple robot arms 50 based on the images of the multiple marker members M. In this way, the endoscope 3 also functions as the marker imaging unit 70 that images the marker members M, which makes it possible to prevent the configuration of the surgery assistance system 500 from becoming complicated, unlike when the marker imaging unit 70 is disposed separately from the endoscope 3.

[0108] Furthermore, in the above-described first and second embodiments, an example in which four robot arms 50 are arranged has been shown, but the present disclosure is not limited to this. In the present disclosure, the number of robot arms 50 may be other than four.

[0109] In the first and second embodiments, the arm unit 51 and the positioner 30 are configured as a seven-axis articulated robot, but the present disclosure is not limited to this. For example, the arm unit 51 and the positioner 30 may be configured as an articulated robot with an axis configuration other than a seven-axis articulated robot. An example of an articulated robot with an axis configuration other than a seven-axis articulated robot is a six-axis or eight-axis robot.

[0110] In the first and second embodiments, the surgical support robot 100 includes the medical cart 10, the positioner 30, and the arm base 40. However, the present disclosure is not limited to this. For example, the medical cart 10, the positioner 30, and the arm base 40 are not necessarily required, and the surgical support robot 100 may be configured with only the robot arm 50.

[0111] The functions of the elements disclosed herein can be performed using circuits or processing circuits, including general-purpose processors, special-purpose processors, integrated circuits, application-specific integrated circuits (ASICs), conventional circuits, and / or combinations thereof, configured or programmed to perform the disclosed functions. A processor is considered a processing circuit or circuit because it includes transistors and other circuitry. In this disclosure, a circuit, unit, or means is hardware that performs the recited functions or hardware that is programmed to perform the recited functions. The hardware may be hardware disclosed herein or other known hardware that is programmed or configured to perform the recited functions. Where the hardware is a processor, which is considered a type of circuit, the circuit, means, or unit is a combination of hardware and software, and the software is used to configure the hardware and / or processor.

[0112] [Aspect] It will be appreciated by those skilled in the art that the exemplary embodiments described above are examples of the following aspects.

[0113] (Aspect 1) a surgical robot including a robotic arm to which a surgical instrument is attached; a marker imaging unit that images a marker member detachably attached to at least one of a patient on whom surgery is performed using the surgical support robot and a trocar inserted into the body surface of the patient; a control unit that detects the marker member based on an image of the marker member, and sets a pivot position that serves as a fulcrum for movement of the surgical instrument based on the detected marker member.

[0114] (Aspect 2) The marker member includes a rectangular shaped AR marker, the marker imaging unit includes a two-dimensional camera, The surgery support system according to aspect 1, wherein the control unit sets the pivot position based on an image of the AR marker captured by the two-dimensional camera.

[0115] (Aspect 3) The robot arm to which the surgical instrument is attached is provided in plurality, a plurality of the AR markers are arranged corresponding to the plurality of robot arms; A surgical support system as described in aspect 2, wherein the multiple AR markers are linked to at least one of information about the corresponding robot arm and information about the type of surgical instrument attached to the robot arm.

[0116] (Aspect 4) the marker imaging unit includes a three-dimensional camera, The surgical support system according to any one of aspects 1 to 3, wherein the control unit sets the pivot position based on an image of the marker member captured by the three-dimensional camera.

[0117] (Aspect 5) a plurality of the marker members are arranged for one of the robot arms, The surgery support system according to any one of aspects 1 to 4, wherein the control unit sets the pivot position based on the detected plurality of marker members.

[0118] (Aspect 6) A surgical support system according to any one of aspects 1 to 5, wherein the marker imaging unit is arranged in at least one of the robot arm, the operating table on which the patient is placed, and the arm base to which the robot arm is attached.

[0119] (Aspect 7) The marker imaging unit When setting the pivot position, the robot arm is attached A surgical assistance system as described in aspect 6, which is detached from the robot arm after the pivot position is set.

[0120] (Aspect 8) an imaging unit for positioning the surgical support robot attached to the arm base; When the surgical support robot is moved to the vicinity of the patient placed on the operating table, an image of the patient captured by the positioning imaging unit is displayed on a display unit in real time, A surgical support system according to aspect 7, wherein the alignment imaging unit also serves as the marker imaging unit that images the marker member.

[0121] (Aspect 9) the surgical instrument includes an endoscope; A surgical support system according to aspect 7 or 8, wherein the endoscope also serves as the marker imaging unit that images the marker member.

[0122] (Aspect 10) the marker imaging unit images the marker member in real time; The surgical support system according to any one of aspects 1 to 9, wherein the control unit changes the pivot position based on an image of the marker member captured in real time.

[0123] (Aspect 11) Taking an image of a marker member detachably attached to at least one of a patient undergoing surgery using a surgical support robot including a robot arm to which a surgical instrument is attached and a trocar inserted into a body surface of the patient; Detecting the marker member based on a captured image of the marker member; A control method for a surgical assistance system, comprising: a control unit setting a pivot position that serves as a fulcrum for movement of the surgical instrument based on the detected marker member.

[0124] (Aspect 12) Taking an image of a marker member detachably attached to at least one of a patient undergoing surgery using a surgical support robot including a robot arm to which a surgical instrument is attached and a trocar inserted into a body surface of the patient; Detecting the marker member based on a captured image of the marker member; and setting a pivot position that serves as a fulcrum for movement of the surgical instrument based on the detected marker member. [Explanation of symbols]

[0125] 1 surgical instruments 3 Endoscopy 22a Display section 40 Arm Base 50 Robot Arm 70 Marker imaging unit (imaging unit for positioning) 71 Marker imaging unit 72 Marker imaging unit 100 Surgical Assist Robot 350 Second control device (control unit) 500 Surgical Support System B Operating table M marker material P patient PP pivot position S body surface T trocar

Claims

1. a surgical robot including a robotic arm to which a surgical instrument is attached; a marker imaging unit that images a marker member detachably attached to at least one of a patient on whom surgery is performed using the surgical support robot and a trocar inserted into the body surface of the patient; a control unit that detects the marker member based on an image of the marker member, and sets a pivot position that serves as a fulcrum for movement of the surgical instrument based on the detected marker member.

2. the marker member includes a rectangular AR marker; the marker imaging unit includes a two-dimensional camera, The surgery assistance system according to claim 1 , wherein the control unit sets the pivot position based on an image of the AR marker captured by the two-dimensional camera.

3. The robot arm to which the surgical instrument is attached is provided in plurality, a plurality of the AR markers are arranged corresponding to the plurality of robot arms; 3. The surgical assistance system according to claim 2, wherein the plurality of AR markers are linked to at least one of information about the corresponding robot arm and information about the type of the surgical instrument attached to the robot arm.

4. the marker imaging unit includes a three-dimensional camera, The surgery assistance system according to claim 1 , wherein the control unit sets the pivot position based on an image of the marker member captured by the three-dimensional camera.

5. a plurality of the marker members are arranged for one of the robot arms, The surgery assistance system according to claim 1 , wherein the control unit sets the pivot position based on the detected plurality of marker members.

6. 2. The surgical assistance system according to claim 1, wherein the marker imaging unit is arranged in at least one of the robot arm, an operating table on which the patient is placed, and an arm base to which the robot arm is attached.

7. The marker imaging unit When setting the pivot position, the robot arm is attached The surgical assistance system according to claim 6 , which is detached from the robot arm after the pivot position is set.

8. an imaging unit for positioning the surgical support robot attached to the arm base; When the surgical support robot is moved to the vicinity of the patient placed on the operating table, an image of the patient captured by the positioning imaging unit is displayed on a display unit in real time, The surgery assistance system according to claim 7 , wherein the image capturing unit for alignment also serves as the marker image capturing unit for capturing an image of the marker member.

9. the surgical instrument includes an endoscope; The surgery assistance system according to claim 7 , wherein the endoscope also serves as the marker imaging unit that images the marker member.

10. the marker imaging unit images the marker member in real time; The surgery assistance system according to claim 1 , wherein the control unit changes the pivot position based on an image of the marker member captured in real time.

11. Taking an image of a marker member detachably attached to at least one of a patient undergoing surgery using a surgical support robot including a robot arm to which a surgical instrument is attached and a trocar inserted into a body surface of the patient; Detecting the marker member based on a captured image of the marker member; A control method for a surgical assistance system, comprising: a control unit setting a pivot position that serves as a fulcrum for movement of the surgical instrument based on the detected marker member.

12. Taking an image of a marker member detachably attached to at least one of a patient undergoing surgery using a surgical support robot including a robot arm to which a surgical instrument is attached and a trocar inserted into a body surface of the patient; Detecting the marker member based on a captured image of the marker member; and setting a pivot position that serves as a fulcrum for movement of the surgical instrument based on the detected marker member.

Images