Robot

Abstract

To provide a robot that handles surgical instruments.SOLUTION: A robot 1 for handing over surgical instruments to a surgeon comprises a hand part 2, an arm part 3 to which the hand part 2 is connected, and a control part 7. The hand part 2 comprises: a jamming hand 20 having a bag body 202 in which granules are stored; a suction part 24 for sucking a gaseous matter from inside the bag body 202; and a camera 23 for imaging an instrument. The control part 7 controls the suction part 24, the camera 23 and the arm part 3.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a robot capable of delivering instruments during surgery. [Background technology]

[0002] A surgical support robot is known that responds to a surgeon's voice instruction by selecting and picking up a specific forceps from a group of forceps having various tip shapes and presenting it to the surgeon (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-122473 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned conventional technique, the forceps are held between two parallel, plate-like fingers, which can make it difficult to hold surgical instruments with complex shapes.

[0005] The present invention has been made in view of the above-mentioned problems, and has as its object to provide a robot that is compatible with surgical instruments. [Means for solving the problem]

[0006] The robot of the present invention is a robot that has a hand unit, an arm unit to which the hand unit is connected, and a control unit, and that hands over surgical instruments to a surgeon, wherein the hand unit has a jamming hand that has a bag body inside which powder or granular material is stored, a suction unit that sucks gas from inside the bag body, and a camera that photographs the instruments, and the control unit controls the suction unit, the camera, and the arm unit. [Effects of the Invention]

[0007] According to the present invention, a robot compatible with surgical instruments can be provided. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a perspective view of a robot arm according to an embodiment. [Figure 2] FIG. 1 is a side view of a robot arm according to an embodiment. [Figure 3] 2 is a front view of the robot arm according to the embodiment (a view of the robot arm shown in FIG. 2 as seen from the right side of FIG. 2). [Figure 4] FIG. 10 is a diagram showing the operating range of a fourth joint section according to the embodiment. [Figure 5] 10A and 10B are diagrams illustrating actuation states of a second joint portion and a third joint according to the embodiment. [Figure 6] 10A and 10B are diagrams illustrating actuation states of a second joint portion and a third joint according to the embodiment. [Figure 7] 10A and 10B are diagrams illustrating a state in which an instrument is held by a bag body according to an embodiment. [Figure 8] 1 is a block diagram showing a schematic configuration of a control device according to an embodiment; [Figure 9] FIG. 10 is a diagram showing an overview of acquiring the grip position of an instrument using machine learning according to an embodiment. [Figure 10] 10A and 10B are diagrams showing gripping positions of the instrument according to the embodiment. [Figure 11] 10A and 10B are diagrams illustrating the positional relationship between the hand unit and the instrument when the hand unit grips the instrument according to the embodiment. [Figure 12] 10 is a flowchart of grip control according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Specific embodiments of the present invention will be described below with reference to the drawings. The dimensions, materials, shapes, relative positions, etc. of the components described in the present embodiments are not intended to limit the technical scope of the invention unless otherwise specified.

[0010] In the robot according to the present invention, a surgical instrument is grasped by a hand unit. The hand unit includes a jamming hand having a bag body, and powder or granular material is contained inside the bag body. The hand unit also includes a suction unit, which can solidify the powder or granular material inside the bag by sucking gas from inside the bag with the suction unit. The bag is placed around the instrument, and the powder or granular material solidifies, allowing the bag to grasp the instrument. The instrument may include tools, instruments, devices, etc. used in surgery. The control unit can acquire an image of the instrument by photographing the instrument with a camera. Then, based on the image of the instrument, the control unit controls the arm unit and suction unit to grasp the instrument. The hand unit is moved to a grasping position by the arm unit controlled by the control unit.

[0011] The hand unit may also include a fixing unit that fixes the camera and the suction unit to the jamming hand. Fixing the camera to the hand unit allows the state of the instrument as seen from the hand unit to be photographed, making it easier for the hand unit to grasp the instrument. Furthermore, the instrument can be photographed at the same angle as the angle of the hand unit. Furthermore, fixing the suction unit to the hand unit shortens the distance from the suction unit to the bag body, thereby shortening the time required to suck gas from the bag body. Therefore, the instrument can be grasped more quickly. Furthermore, the suction unit can be made smaller due to the increased suction efficiency. Furthermore, since the functions related to photography and grasping are separated and integrated into the arm unit, the hand unit can be easily detached from the arm unit and maintenance such as cleaning and repair of the hand unit can be improved.

[0012] The camera may include a depth camera, and may be fixed to the fixed part with the optical axis of the depth camera directed substantially parallel to the central axis of the jamming hand. The depth camera is a camera capable of acquiring three-dimensional information. By providing a depth camera, the three-dimensional position of the instrument can be grasped more accurately. Furthermore, by directing the optical axis of the depth camera substantially parallel to the central axis of the jamming hand and fixing it to the fixed part, the arrangement of the instrument as viewed from a shooting position where the jamming hand directly faces the instrument base can be grasped without complex position control on the arm side.

[0013] The control unit may cause the depth camera to photograph the instrument before grasping the instrument and store the photographed image in a storage unit. By storing the image of the instrument in the storage unit in advance, the position of the instrument can be determined based on the stored image. Furthermore, since real-time image processing is not required, the processing load can be reduced. Note that the photograph of the instrument may be taken before grasping the instrument each time the instrument is grasped, at a predetermined timing, or only once when the robot is started.

[0014] The control unit may generate a learning model that uses the image of the instrument and the identification information of the instrument as input data and the horizontal grip position of the instrument as output data, and that uses the image of the instrument, the identification information of the instrument, and the horizontal grip position of the instrument as training data. The identification information of the instrument is information that can identify the instrument, and may be the name of the instrument, the product number of the instrument, an abbreviation of the instrument, or any symbol or number. By generating a learning model in advance, the horizontal grip position of the instrument can be easily identified by inputting the image of the instrument and the identification information of the instrument.

[0015] The horizontal gripping position of the instrument in the training data is other than the position where the surgeon grips it. The instrument may be held at a position other than the position where it comes into contact with the patient. If the hand unit grips the instrument at a position where the surgeon would normally grip it, the surgeon may need to change hands after receiving it from the robot. Therefore, by holding the instrument at a position other than the position where the surgeon would normally grip it with the hand unit, the surgeon can avoid having to take the trouble of changing hands.

[0016] The control unit may determine the vertical target position of the hand unit when the hand unit grasps the instrument so that the distance from a first predetermined position of the instrument to a second predetermined position of the hand unit is a predetermined distance. The first predetermined position is a reference position for the instrument. The first predetermined position may be the lowest or highest position of the instrument. The second predetermined position is a reference position for the hand unit. The second predetermined position is a position other than the bag body. The predetermined distance is such that the instrument does not come into contact with the hand unit other than the bag body. In this way, by setting the distance from the first predetermined position to the second predetermined position as a predetermined distance, it is possible to prevent the instrument from coming into contact with the hand unit and applying excessive force to the hand unit. Furthermore, even if the vertical height of each instrument differs, the hand unit can appropriately grasp the instrument.

[0017] The arm unit may include a shoulder unit, an elbow unit, and a wrist unit, each of which has multiple joints. The hand unit and the arm unit are connected at the wrist unit. The movable angle of the shoulder unit may be set to 90 degrees or less, and the movable angle of the elbow unit may be set to 180 degrees or less. The movable angle may be limited by a control unit or may be physically limited by, for example, a stopper. By limiting the movable angle of the shoulder unit and the elbow unit in this manner, the robot moves in a manner similar to that of a human. This allows a person near the robot to intuitively grasp the range of movement of the robot, thereby preventing the robot from coming into contact with the person.

[0018] The arm may have a plurality of joints and may be provided with a plurality of stepping motors for bending and extending the plurality of joints. By using stepping motors, even if the robot comes into contact with a person, step-out will occur, thereby increasing safety.

[0019] The arm may include a base that is fixed to a surgical bed. By fixing the base to the surgical bed, a separate table or the like for supporting the robot is not required. Also, the robot can be installed in a small space. The base may be fixed to the surgical bed via a bracket or the like.

[0020] <Embodiment> Specific embodiments of the present invention will be described below with reference to the drawings. The dimensions, materials, shapes, relative positions, etc. of the components described in the present embodiments are not intended to limit the technical scope of the invention unless otherwise specified.

[0021] In this embodiment, an example in which a robot according to the present disclosure is applied to a robot arm 1 will be described. FIG. 1 is a perspective view of the robot arm 1 according to the embodiment. FIG. 2 is a side view of the robot arm 1 according to the embodiment. FIG. 3 is a front view of the robot arm 1 according to the embodiment (a view of the robot arm 1 shown in FIG. 2 as seen from the right side of FIG. 2). The front of the robot arm 1 faces an instrument stand 61, which will be described later, and faces downward and to the right in FIG. 1. The robot arm 1 includes a hand unit 2, an arm unit 3, and a base unit 4. The hand unit 2 is attached to one end of the arm unit 3. The other end of the arm unit 3 is attached to the base unit 4. The hand unit 2 includes a jamming hand 20 having a cup unit 201 and a bag body 202 provided inside the cup unit 201. The detailed configuration of the hand unit 2 will be described later. The robot arm 1 receives power from, for example, a commercial power source.

[0022] In the following, the vertical direction is defined as the upward direction of the robot arm 1, and the vertical direction is defined as the upward direction of the robot arm 1. The downward direction, i.e., the direction of gravity, is defined as the downward direction of the robot arm 1. In the following description, an XYZ Cartesian coordinate system is set up, and the position of each member is described with reference to this XYZ Cartesian coordinate system. The up-down direction of the robot arm 1 is defined as the Z-axis direction, the direction in which the robot arm 1 faces forward in its initial state is defined as the X-axis direction, and the direction perpendicular to the X-axis and Z-axis directions is defined as the Y-axis direction. The front of the robot arm 1 is the lower right side in FIG. 1, and is the direction in which an instrument stand 61 and an instrument 62, which will be described later, are arranged as viewed from the robot arm 1. The XY plane is a horizontal plane.

[0023] (Arm mechanism) The arm unit 3 includes a first arm link unit 31, a second arm link unit 32, a third arm link unit 33, and a connecting member 34. One end of the hand unit 2 is connected to one end of the first arm link unit 31 at a first joint unit 30A. The other end of the first arm link unit 31 is connected to one end of the second arm link unit 32 at a second joint unit 30B. The other end of the second arm link unit 32 is connected to one end of the third arm link unit 33 at a third joint unit 30C. The other end of the third arm link unit 33 is connected to one end of the connecting member 34 at a fourth joint unit 30D. The other end of the connecting member 34 is fixed vertically to the base unit 4. The base unit 4 is configured to be fixable to a rail 51 provided on, for example, a general-purpose surgical bed 5 via a bracket 52. The first joint 30A is an example of a wrist, the second joint 30B and the third joint 30C are examples of an elbow, and the fourth joint 30D is an example of a shoulder. The first joint 30A, the second joint 30B, the third joint 30C, and the fourth joint 30D are examples of a movable part.

[0024] The first arm link unit 31 is provided with a first motor 311 for rotating the hand unit 2 relative to the first arm link unit 31, and a second motor 312 for rotating the first arm link unit 31 relative to the second arm link unit 32. The first motor 311 is a motor for bending and extending the first joint unit 30A, and the second motor 312 is a motor for bending and extending the second joint unit 30B. The third arm link unit 33 is provided with a third motor 331 for rotating the second arm link unit 32 relative to the third arm link unit 33, and a fourth motor 332 for rotating the third arm link unit 33 relative to the connecting member 34. The third motor 331 is a motor for bending and extending the third joint unit 30C, and the fourth motor 332 is a motor for bending and extending the fourth joint unit 30D. The connecting member 34 is provided with a fifth motor 341 for rotating the third arm link portion 33 relative to the connecting member 34 around the central axis of the connecting member 34. The central axis of the connecting member 34, i.e., the rotation axis of the fifth motor 341, is disposed in the Z-axis direction.

[0025] The first joint unit 30A has a first rotation shaft 301A, and when the first motor 311 is operated, the hand unit 2 rotates relative to the first arm link unit 31 around the first rotation shaft 301A. The first joint unit 30A also has a mechanism that rotates the hand unit 2 relative to the first arm link unit 31 around an axis in the Z-axis direction in the state shown by the solid line in FIG. 5. The second joint unit 30B has a second rotation shaft 301B, and when the second motor 312 is operated, the first arm link unit 31 rotates relative to the second arm link unit 32 around the second rotation shaft 301B. The third joint unit 30C has a third rotation shaft 301C, and when the third motor 331 is operated, the second arm link unit 32 rotates relative to the third arm link unit 33 around the third rotation shaft 301C. The fourth joint portion 30D is provided with a fourth rotation shaft 301D, and when the fourth motor 332 is operated, the third arm link portion 33 rotates relatively to the connecting member 34 around the fourth rotation shaft 301D.

[0026] The first rotating shaft 301A, the second rotating shaft 301B, the third rotating shaft 301C, and the fourth rotating shaft 30 The first rotation shaft 301A, the second rotation shaft 301B, the third rotation shaft 301C, and the fourth rotation shaft 301D are arranged parallel to each other and perpendicular to the rotation shaft of the fifth motor 341. That is, the first rotation shaft 301A, the second rotation shaft 301B, the third rotation shaft 301C, and the fourth rotation shaft 301D are arranged in the horizontal direction (the Y-axis direction in FIGS. 1 to 3).

[0027] FIG. 4 is a diagram showing the operating range of the fourth joint unit 30D according to the embodiment. The fourth joint unit 30D is configured so that the relative rotation angle A1 of the third arm link unit 33 with respect to the connecting member 34 is, for example, 90 degrees or less from the Z-axis direction. That is, the fourth joint unit 30D is configured so that the angle by which the third arm link unit 33 can rotate relative to the connecting member 34 around the fourth rotation axis 301D is, for example, 90 degrees or less. In this way, the movable angle of the fourth joint unit 30D is limited. As shown in FIG. 4, the third arm link unit 33 can bend and extend in a range from the Z-axis direction to the X-axis direction. Furthermore, as shown by the dashed line in FIG. 4, when the third arm link unit 33 is positioned in the Z-axis direction, the rotation direction of the third arm link unit 33 is limited to one direction (clockwise in FIG. 4). Therefore, in FIG. 4, the third arm link unit 33 does not extend counterclockwise from a state parallel to the Z-axis direction.

[0028] FIG. 5 is a diagram illustrating the actuation states of the second joint portion 30B and the third joint portion 30C according to the embodiment. In the example shown in FIG. 5, the second joint portion 30B and the third joint portion 30C are actuated so that the relative rotation angle A2 of the first arm link portion 31 with respect to the third arm link portion 33 is, for example, 90 degrees from the central axis of the third arm link portion 33 (from the Z-axis direction in FIG. 5). However, the first arm link portion 31 can also be rotated relative to the third arm link portion 33 by actuating only one of the second joint portion 30B and the third joint portion 30C. Furthermore, as indicated by the dashed line in FIG. 5, when the first arm link portion 31 is disposed in the Z-axis direction, the rotation direction of the first arm link portion 31 and the second arm link portion 32 is limited to one direction (clockwise in FIG. 5). 5, the first arm link portion 31 and the second arm link portion 32 do not extend counterclockwise from a state in which they are positioned on the central axis of the third arm link portion 33. In this way, the movable angles of the second joint portion 30B and the third joint portion 30C are limited.

[0029] Furthermore, the rotation angle A3 of the hand unit 2 relative to the first arm link unit 31 may be limited to a range in which, for example, a bracket 22, a camera 23, and a pump 24, which will be described later, do not come into contact with the first arm link unit 31. Furthermore, the first joint unit 30A may be configured so that the hand unit 2 does not extend counterclockwise from a state in which it is positioned on the central axis of the first arm link unit 31.

[0030] FIG. 6 is a diagram illustrating the operating states of the second joint portion 30B and the third joint portion 30C according to the embodiment. In the example illustrated in FIG. 6, the second joint portion 30B and the third joint portion 30C are bent further than in the example illustrated in FIG. 5. The second joint portion 30B and the third joint portion 30C are configured so that the relative rotation angle A2 of the first arm link portion 31 with respect to the third arm link portion 33 is 180 degrees or less. Note that the relative rotation angle A2 of the first arm link portion 31 with respect to the third arm link portion 33 may be limited to a range in which, for example, the bracket 22, the camera 23, and the pump 24 described below do not come into contact with the third arm link portion 33. In this way, the movable angles of the second joint portion 30B and the third joint portion 30C are limited.

[0031] The first motor 311, the second motor 312, the third motor 331, the fourth motor 332, and the fifth motor 341 are each a stepping motor. The first motor 311, the second motor 312, the third motor 331, and the fourth motor 332 each have a linear motion mechanism that converts rotational motion into linear motion, and the first joint 30A, the second joint 30B, the third joint 30C, and the fourth joint 30D are flexed and extended by the advancement and retreat of this linear motion mechanism.

[0032] (Hand part) Next, the configuration of the hand unit 2 will be described. The hand unit 2 is configured to include a jamming hand 20. The jamming hand 20 has a cup unit 201 and a bag body 202. The cup unit 201 has a cylindrical tube portion 201A formed on the first joint unit 30A side and an expanded diameter portion 201B formed closer to the tip of the hand unit 2 than the tube portion 201A. The tube portion 201A and the expanded diameter portion 201B are formed to be hollow. The central axis of the hand unit 2 coincides with the central axis of the tube portion 201A and the central axis of the expanded diameter portion 201B. The expanded diameter portion 201B expands in diameter from the first joint unit 30A side toward the tip of the hand unit 2, and is formed so that the cross section perpendicular to the central axis of the cup unit 201 is circular. The expanded diameter portion 201B opens at the tip of the hand unit 2 and contains a part of the bag body 202 inside. The tip side of the hand unit 2 means the side of the hand unit 2 opposite to the first joint unit 30A.

[0033] The bag 202 is made of a flexible material. Examples of materials that can be used for the bag 202 include biocompatible materials such as latex, synthetic rubber, or thin-film soft plastic. The bag 202 has an opening for introducing and discharging air, and this opening is fixed to the inside of the tubular portion 201A. The bag 202 may be formed, for example, in the shape of a balloon. The bag 202 is filled with powder or granular material. The powder or granular material may be, for example, granular ion exchange resin or plastic pellets, but is not limited to these. The particle size and filling amount of the powder or granular material may be determined depending on the instrument 62 to be grasped. The size of the bag 202 and the amount of protrusion from the cup portion 201 may also be determined depending on the instrument 62.

[0034] A camera 23 and two pumps 24 are fixed to the tubular portion 201A via a bracket 22. The bracket 22 is an example of a fixing portion, and the two pumps 24 are an example of a suction portion. One end of a tube 241 is connected to each of the two pumps 24 so as to exhaust air from the bag body 202. The other end of the tube 241 is connected to an opening of the bag body 202. The two pumps 24 are activated by supplying power to suck air from the bag body 202 and release the sucked air into the atmosphere. The camera 23 is a depth camera (which may be a 3D camera) capable of acquiring information about a three-dimensional position, and is, for example, a stereo camera including two CCD image sensors or two CMOS image sensors. As another example, the camera 23 may be a ToF (Time of Flight) camera or a structured light camera. The two optical axes of the camera 23 are arranged parallel to the central axis of the tube portion 201A and point toward the tip side of the hand portion 2. The angle of view of the camera 23 is set so that the entire instrument table 61 can be photographed.

[0035] The hand unit 2 can grasp the instrument 62 by operating the pump 24 while the bag 202 is in contact with the instrument 62 placed on the upper surface 61A of the instrument stand 61. FIG. 7 is a diagram showing the state in which the instrument 62 is grasped by the bag 202 according to the embodiment. When air is sucked from the bag 202 by the pump 24, the shape of the bag 202 deforms to conform to the shape of the instrument 62, and the powder and granular material inside solidifies. This allows the bag 202 to grasp the instrument 62. On the other hand, when the pump 24 is stopped, air is introduced into the bag 202. This causes the bag 202 to expand, loosening the powder and granular material and releasing the instrument 62. Note that the instrument 62 is not limited as long as it can be grasped by the hand unit 2. Examples of the instrument 62 include, but are not limited to, laparoscopic forceps, laparoscopic needle holders, scissors, vascular forceps, Pean forceps, needle holders, and tweezers.

[0036] (Control device) Next, the configuration of the control device 7 will be described with reference to FIG. 8. FIG. 8 is a block diagram showing a schematic configuration of the control device 7 according to the embodiment. The control device 7 is a small computer such as a personal computer (PC). The control device 7 has an arithmetic processing unit and a memory. The control device 7 controls the hand unit of the robot arm 1. 2 and the arm unit 3.

[0037] The control device 7 has a control unit 71, a storage unit 72, an input unit 73, a display 74, and a communication unit 75. The control unit 71 is a functional unit formed by executing a predetermined control program in a computer included in the control device 7. The control unit 71 can be realized by a hardware processor such as a CPU. The control unit 71 may also be configured to include RAM, ROM, cache memory, etc. The control unit 71 is configured to include an arm control unit 711, a hand control unit 712, an image acquisition unit 713, a grip position derivation unit 714, and a learning unit 715.

[0038] The storage unit 72 is a means for storing information, and is configured with a storage medium such as a RAM or a flash memory. The storage unit 72 stores programs executed by the control unit 71, data used by the programs, etc. The storage unit 72 also stores a trained model (a grip position derivation model 721).

[0039] The input unit 73 is a means for accepting input operations performed by the user, and is, for example, a touch panel, a mouse, a keyboard, a microphone, or a foot switch. The display 74 is a means for presenting information to the user, and is, for example, an LCD (Liquid Crystal Display) or an EL (Electroluminescence) panel. The display 74 may be configured as a single touch panel display. The communication unit 75 is a communication interface for connecting the control device 7 to the robot arm 1. The communication unit 75 may be configured to include, for example, a network interface board, a wireless communication interface for wireless communication, or a USB (Universal Serial Bus) interface.

[0040] The arm control unit 711 controls the first motor 311, second motor 312, third motor 331, fourth motor 332, and fifth motor 341 of the arm unit 3 by supplying drive signals based on an image acquired by an image acquisition unit 713 (described below) and the grip position of the instrument 62 derived by a grip position derivation unit 714. The arm control unit 711 controls each motor to operate the arm unit 3, thereby moving, for example, the hand unit 2 to a predetermined target position suitable for gripping the instrument 62. Details will be described later.

[0041] The hand control unit 712 is configured to supply a drive signal for driving the pump 24 provided in the hand unit 2 to the pump 24. Details will be described later.

[0042] The image acquisition unit 713 acquires an image of the instrument 62 to be grasped by the hand unit 2 from the camera 23. The image acquisition unit 713 captures an image of the instrument 62 with the camera 23 at a predetermined timing. The predetermined timing may be, for example, when the control device 7 is started, when grasp control begins, each time the instrument 62 is grasped, or when a predetermined operation is performed by the user. The predetermined operation may be detected by a sensor. The predetermined operation by the user may be, for example, the user pressing a foot switch with their foot. The image acquisition unit 713 transmits information to the arm control unit 711 so that the camera 23 moves to a position (hereinafter also referred to as the imaging position) where the entire upper surface 61A of the instrument table 61 is captured by the camera 23. The imaging position is a position where the entire upper surface 61A of the instrument table 61 is included in the image captured by the camera 23, and is, for example, a position vertically higher than the standby position before the operation of grasping the instrument 62 is started. The imaging position may be a position determined in advance by the user.

[0043] The gripping position derivation unit 714 derives a position (hereinafter also referred to as a target position) to which the hand unit 2 is moved in order to grip the instrument 62, based on the identification information of the instrument 62, the image acquired by the image acquisition unit 713, and the gripping position derivation model 721 stored in the storage unit 72. The target position is a position in three-dimensional space. The target position is the distance from the standby position in the X-axis direction. The standby position may be expressed by the amount of movement in the Y-axis direction, the amount of movement in the Z-axis direction, and the amount of movement in the Y-axis direction. The standby position is the position of the hand unit 2 when control by the control unit 71 is not being performed.

[0044] FIG. 9 is a diagram illustrating an overview of acquiring the grip position of the instrument 62 using machine learning according to an embodiment. The learning unit 715 generates a grip position derivation model 721, which is a machine learning model for identifying the grip position of the instrument 62. The grip position derivation model 721 is a model that uses an image and identification information of the instrument 62 as input data and the grip position of the instrument 62 as output data. When generating the grip position derivation model 721, the learning unit 715 acquires an image of the actual instrument 62 using the camera 23. At this time, the learning unit 715 detects the instrument 62 from the image and surrounds the instrument 62 with a rectangular bounding box. The user assigns tags or metadata to the instrument 62 surrounded by the bounding box by performing an annotation task. At this time, the user inputs the identification information of the instrument 62 and the grip position of the instrument 62 via the input unit 73 of the control device 7. The grip position of the instrument 62 may be input as an area of ​​a certain size. The gripping position of the instrument 62 here refers to the gripping position that the hand unit 2 aims to grasp when gripping the instrument 62, and is, for example, a position other than the position where the surgeon grasps the instrument 62. As another example, the gripping position of the instrument 62 may be a position other than the position where it comes into contact with the patient. The identification information of the instrument 62 is information that can identify the instrument 62, and may be the name of the instrument 62, the product number of the instrument 62, an abbreviation of the instrument 62, or any symbol or number corresponding to the instrument 62.

[0045] 10 is a diagram showing the gripping position R1 of the instrument 62 according to the embodiment. The gripping position R1 of the instrument 62 is set so as to avoid the position R2 where the surgeon grips the instrument and the position R3 where the instrument comes into contact with the patient. Alternatively, the gripping position of the instrument 62 may be set to a position that avoids the position R2 where the surgeon grips the instrument and the position R3 where the instrument comes into contact with the patient and is closest to the center of gravity of the instrument 62. This can improve stability when the instrument 62 is gripped. The center of gravity of the instrument 62 may be stored in advance in the memory unit 72.

[0046] As another example, the gripping position R1 of the instrument 62 may be the position where the diagonals of a bounding box surrounding the instrument 62 intersect (i.e., the center of the rectangle). In other words, most instruments 62 are used by the surgeon grasping the vicinity of the edge. Therefore, the center of the bounding box surrounding the instrument 62 is likely to correspond to a position other than the position where the surgeon grasps the instrument. This allows for a simple designation of the gripping position. However, if the center of the bounding box is offset from the center of gravity of the instrument 62, the instrument 62 may become unstable when the hand unit 2 lifts it. For such instruments 62, a label for the gripping position R1 may be individually added in an annotation process, and the learning model may be regenerated. In this case, the gripping position R1 may be set to a position closer to the center of gravity of the instrument 62. In other words, the gripping position R1 may be set to a position other than the position where the surgeon grasps the instrument and other than the position where the instrument comes into contact with the patient, and the position closest to the center of gravity may be set as the gripping position R1. In this way, the learning unit 715 constructs a machine learning model (grip position derivation model 721) using as training data the image, identification information, and grip position of the instrument 62. The grip position, which is output data, is a position on a horizontal plane.

[0047] The grasping position derivation unit 714 acquires an overall image of the upper surface 61A of the instrument table 61 from the storage unit 72. This overall image is an image captured by the image acquisition unit 713 at the capturing position. The grasping position derivation unit 714 also specifies the identification information of the instrument 62 to be grasped. The instrument 62 to be grasped may be input by the user. As another example, the instruments 62 to be grasped may be selected in a predetermined order. For example, each time the user presses the foot switch, the instruments 62 may be selected in a predetermined order. The predetermined order may be, for example, the order used in surgery. It is to be noted that the input is not limited to the foot switch, and voice input or gesture input may also be used, for example.

[0048] The grip position derivation unit 714 inputs the identification information of the instrument 62 to be grasped and the entire image acquired from the storage unit 72 to the grip position derivation model 721. As a result, the grip position derivation unit 714 obtains the horizontal grip position of the instrument 62 output from the grip position derivation model 721. The grip position derivation unit 714 also specifies the vertical position of the instrument 62 to be grasped in the horizontal position of the instrument 62 from the image. At this time, for example, the position of the upper end, lower end, or vertical center of the target instrument 62 near the grip position may be specified.

[0049] Fig. 11 is a diagram showing the positional relationship between the hand unit 2 and the instrument 62 when the hand unit 2 according to the embodiment grips the instrument 62. Fig. 11 is a diagram showing the hand unit 2 and the instrument 62 as viewed from the horizontal direction. In this embodiment, to make it easier to grip the instrument 62, a spacer 63 is placed between the upper surface 61A of the instrument stand 61 and the instrument 62, and the instrument 62 is positioned away from the instrument stand 61. By using such a spacer 63, the bag body 202 can easily reach the side and bottom surfaces of the instrument 62, making it easier to grip the instrument 62.

[0050] The grip position derivation unit 714 acquires the highest position in the vertical direction of the instrument 62 near the horizontal grip position R1 output from the grip position derivation model 721. Note that the whole image is Since the image is captured by a depth camera, the highest position can be obtained using a known algorithm. The vicinity of the gripping position R1 here is set to an area that can prevent contact between the cup portion 201 and the instrument 62. In other words, the highest position of the instrument 62 is obtained within the range where the cup portion 201 can come into contact so that the cup portion 201 does not come into contact with the instrument 62 when the hand portion 2 grips the instrument 62 at the gripping position.

[0051] In the example shown in FIG. 11, the highest position C2 of the instrument 62 near the gripping position R1 is assumed to be on the central axis L1 of the cup portion 201, which passes through the center C1 of the tip 201C of the cup portion 201. The gripping position deriving unit 714 determines a position a predetermined distance B1 above the highest position C2 in the Z-axis direction as the target position in the Z-axis direction of the center C1 of the tip 201C of the cup portion 201. The highest position C2 is an example of a first predetermined position, and the center C1 of the tip 201C of the cup portion 201 is an example of a second predetermined position. The predetermined distance B1 is a distance at which the cup portion 201 and the instrument 62 do not come into contact with each other and at which the bag body 202 comes into contact with the instrument 62. Meanwhile, the target position in the horizontal direction is set to the center position of the gripping position R1. In this way, the gripping position deriving unit 714 derives the target position of the center C1 of the tip 201C of the cup portion 201. In this embodiment, the target position is determined based on the center C1 of the tip 201C of the cup portion 201, but the reference for the target position is not limited to the center C1 of the tip 201C of the cup portion 201, and may be a predetermined position in the hand portion 2. In this case, the deformable bag body 202 does not serve as the reference for the target position. In the following, the target position of the center C1 of the tip 201C of the cup portion 201 is also referred to as the target position of the hand portion 2.

[0052] 11, the target position of the hand unit 2 is set a predetermined distance B1 above the highest vertical position C2 of the instrument 62 near the gripping position R1, but the reference position on the instrument 62 side is not limited to the highest position C2. For example, the lowest vertical position of the instrument 62 may be used as the reference. In this case, the predetermined distance B1 may be learned for each instrument 62. As another example, the vertical position of the upper surface 61A of the instrument stand 61 may be used as the reference in the vertical direction of the instrument 62.

[0053] The arm control unit 711 controls the first motor 311, the second motor 312, the third motor 331, the fourth motor 332, and the fifth motor 341 by supplying a drive signal based on the target position derived by the grip position derivation unit 714. When the hand unit 2 moves to the target position, the hand control unit 712 activates the pump 24. The hand unit 2 grips the instrument 62. When the hand unit 2 grips the instrument 62, the arm control unit 711 controls the first motor 311, the second motor 312, the third motor 331, the fourth motor 332, and the fifth motor 341 to move the instrument 62 to a position where it is released (hereinafter also referred to as the release position). The release position of the instrument 62 may be stored in the memory unit 72, or may be instructed by the surgeon using voice or gestures. Then, when the hand unit 2 moves to the release position of the instrument 62, the hand control unit 712 stops the pump 24, thereby releasing the instrument 62.

[0054] (Flow of Control) FIG. 12 is a flowchart of gripping control according to the embodiment. This routine is executed by the control device 7. Note that the description will be made on the assumption that the gripping position derivation model 721 has already been generated. In step S101, the arm control unit 711 determines whether or not a request to put out an instrument has been made. The arm control unit 711 determines that a request to put out an instrument has been made when, for example, a predetermined input has been made to the input unit 73. The predetermined input is, for example, input by a foot switch, voice, or gesture. The request to put out an instrument may also include identification information of the instrument 62 to be gripped. If the arm control unit 711 makes a positive determination in step S101, the process proceeds to step S102, and if a negative determination has been made, this routine ends.

[0055] In step S102, the arm control unit 711 controls the first motor 311, the second motor 312, the third motor 331, the fourth motor 332, and the fifth motor 341 to move the hand unit 2 to the imaging position. The imaging position is a position where the entire upper surface 61A of the instrument stand 61 can be imaged from above. The upper surface 61A of the instrument stand 61 is the surface on which the instrument 62 is placed.

[0056] When the arm control unit 711 completes the processing of step S102, in step S103, the image acquisition unit 713 controls the camera 23 to capture an image of the entire upper surface 61A of the instrument table 61. The image acquisition unit 713 stores the image obtained by capturing the image in the memory unit 72.

[0057] When the image acquisition unit 713 acquires the image, in step S104, the grip position derivation unit 714 inputs the identification information of the instrument 62 to be grasped acquired in step S101 and the image stored in the storage unit 72 in step S103 as input data to the grip position derivation model 721. As a result, the grip position derivation model 721 outputs the horizontal grip position of the instrument 62. Then, in step S105, the grip position derivation unit 714 identifies the horizontal grip position of the instrument 62 based on the output data from the grip position derivation model 721.

[0058] Furthermore, in step S106, the grip position deriving unit 714 identifies the highest vertical position of the instrument 62 based on the image stored in the storage unit 72 in step S103. A known technique can be used for this identification.

[0059] In step S107, the gripping position deriving unit 714 determines a target position of the hand unit 2 based on the horizontal position identified in step S105 and the vertical position identified in step S106. At this time, the gripping position deriving unit 714 determines the target position of the hand unit 2 so that the center C1 of the tip 201C of the cup unit 201 is a predetermined distance B1 above the vertical position identified in step S106 and the horizontal position of the center C1 of the tip 201C of the cup unit 201 is the horizontal position identified in step S105.

[0060] When the grip position derivation unit 714 determines the target position, in step S108, the arm control unit 711 controls the first motor 311, the second motor 312, the third motor 331, the fourth motor 332, and the fifth motor 341 to move the hand unit 2 to the target position. At this time, the angle of each joint may be adjusted so that the central axis of the cup unit 201 is closer to the vertical direction, for example. The logic for determining the actuation amounts of the first motor 311, the second motor 312, the third motor 331, the fourth motor 332, and the fifth motor 341 relative to the target position is stored in the storage unit 72.

[0061] When the hand unit 2 moves to the target position, in step S109, the hand control unit 712 activates the pump 24. As a result, the instrument 62 is gripped.

[0062] When a predetermined time has elapsed since the pump 24 started operating, in step S110, the arm control unit 711 controls the first motor 311, the second motor 312, the third motor 331, the fourth motor 332, and the fifth motor 341 to move the hand unit 2 to the release position. The predetermined time is set as the time it takes for the instrument 62 to become grippable by discharging air from the bag body 202. The release position is, for example, a predetermined position where the hand unit 2 releases the instrument 62, making it easy for the surgeon to receive the instrument.

[0063] When the hand unit 2 moves to the release position, in step S111, the hand control unit 712 stops the pump 24. This causes air to flow into the bag body 202, releasing the instrument 62. Thereafter, in step S112, the arm control unit 711 returns the arm unit 3 to the standby position. The standby position is the position in which the robot arm 1 waits until a request to release the instrument 62 is made. This moves the hand unit 2 to the standby position. The arm control unit 711 controls the first motor 311, the second motor 312, the third motor 331, the fourth motor 332, and the fifth motor 341 to place the arm unit 3 in the standby position.

[0064] As described above, with the robot arm 1 according to this embodiment, the bag body 202 of the hand unit 2 deforms according to the shape of the instrument 62, allowing a single hand unit 2 to grasp instruments 62 of various shapes. Furthermore, the first motor 311, the second motor 312, the third motor 331, the fourth motor 332, and the fifth motor 341 of the arm unit 3 are stepping motors. Therefore, even if the robot arm 1 comes into contact with a person, it will lose step-up, resulting in high safety. Furthermore, because the range of motion of each joint is set to be the same as the range of motion of a human joint, a person can intuitively imagine the range of motion of the robot arm 1. This prevents the robot arm 1 from coming into contact with a person due to unexpected movements. Furthermore, because the grasping position of the instrument 62 can be learned by machine learning, the instrument 62 can be grasped in a way that makes it easy for the surgeon to receive the instrument 62. Furthermore, because the entire top surface 61A of the instrument table 61 is photographed by the camera 23 before grasping the instrument 62, the instrument 62 can be recognized based on the image acquired at that time, eliminating the need for real-time image processing. Therefore, processing can be performed using a general, inexpensive, small computer such as a personal computer. Furthermore, since the robot arm 1 can be attached to a rail 51 on a surgical bed 5 via a bracket 52, it can be installed in an existing operating room without introducing additional equipment. Furthermore, since the robot arm 1 can deliver instruments, it can assist in the instrument delivery process. Furthermore, since the pump 24 is located adjacent to the cup portion 201, air can be quickly expelled from the bag body 202. This reduces the time required to grasp the instrument 62. Furthermore, since the camera 23 is located in the hand portion 2, the shooting position can be freely set using the arm portion 3. Furthermore, since the robot according to the present disclosure can be miniaturized while incorporating the elements necessary for the instrument delivery operation (the suction unit, the gripping unit, and the camera unit), it can be used to assist in the instrument delivery operation simply by bringing it into an existing operating room together with a personal computer for control. [Explanation of symbols]

[0065] 1···Robot arm, 2···Hand portion, 7···Control device, Jamming hand···20, 201···Cup portion, 202···Bag body, 22···Bracket, 23···Camera, 24···Pump, 3···Arm portion, 4···Base portion, 62···Instrument

Claims

1. A hand part, an arm portion to which the hand portion is connected; A control unit; A robot that delivers surgical instruments to a surgeon, The hand unit includes: a jamming hand having a bag body for accommodating powder and granular material therein; a suction section that sucks gas from inside the bag; a camera for photographing the instrument; Equipped with The control unit controls the suction unit, the camera, and the arm unit. robot.

2. The hand unit includes a fixing unit that fixes the camera and the suction unit to the jamming hand. The robot of claim 1 .

3. The camera includes a depth camera, and is fixed to the fixed part with the optical axis of the depth camera directed to be approximately parallel to the central axis of the jamming hand. The robot according to claim 2.

4. The control unit before grasping the instrument, causing the depth camera to photograph the instrument; storing the image obtained by photographing in a storage unit; The robot according to claim 3.

5. The control unit generates a learning model that uses the image of the instrument and the identification information of the instrument as input data and the horizontal grip position of the instrument as output data, and uses the image of the instrument, the identification information of the instrument, and the horizontal grip position of the instrument as training data. The robot of claim 1 .

6. The horizontal grip position of the instrument in the teacher data is a position other than the position where the surgeon grips the instrument and a position other than the position where the instrument contacts the patient. The robot according to claim 5.

7. the control unit determines a target position in the vertical direction of the hand unit when the hand unit grips the instrument so that a distance from a first predetermined position of the instrument to a second predetermined position of the hand unit is a predetermined distance. The robot of claim 1 .

8. the arm portion includes a shoulder portion, an elbow portion, and a wrist portion, which are multiple joint portions; the hand unit and the arm unit are connected at the wrist unit, The movable angle of the shoulder is set to 90 degrees or less, The movable angle of the elbow is set to 180 degrees or less. The robot of claim 1 .

9. The arm portion It has a plurality of joints, a plurality of stepping motors for bending and extending the plurality of joints; The robot of claim 1 .

10. The arm portion includes a base portion that is fixed to a surgical bed. The robot of claim 1 .

Images