Robot device and method for controlling same

Abstract

Provided is a robot device comprising: a robot hand to which a plurality of finger parts are provided; an arm part that drives the robot hand; a plurality of gripping parts that follow the plurality of finger parts and can grip an object; at least one identification part on at least one of the plurality of gripping parts; and a control unit that uses position information regarding the identification part and drives the robot hand via the arm part to position the plurality of gripping parts with respect to the object. Thus, positional alignment of the object and the robot hand can be efficiently performed.

Description

Robot device and control method thereof

[0001] The present invention relates to a robot device and a method for controlling a robot device.

[0002] A robot device is in use that has a robot hand (end effector) for grasping an object and an arm that drives the robot hand, and that can detect the amount of positional deviation of the robot hand relative to the arm from an image of the robot hand (see, for example, Patent Document 1). In such technology, it is desirable to efficiently align the object and the robot hand.

[0003] JP 2017-124468 A

[0004] According to a first aspect of the present invention, there is provided a robot device comprising: a robot hand provided with a plurality of finger portions; an arm portion that drives the robot hand; a plurality of gripping portions that can grip an object in response to the plurality of finger portions; at least one identification portion that is provided on at least one of the plurality of gripping portions; and a control portion that uses position information of the identification portion to drive the robot hand via the arm portion and position the plurality of gripping portions relative to the object.

[0005] According to a second aspect, there is provided a control method for a robot device including a robot hand provided with a plurality of finger portions and an arm portion that drives the robot hand, the control method including: causing a plurality of gripping portions capable of gripping an object to follow the plurality of finger portions; storing position information of at least one identification portion possessed by at least one of the plurality of gripping portions; and using the position information of the identification portion to drive the robot hand via the arm portion and position the plurality of gripping portions relative to the object.

[0006] 1A is a perspective view showing a robot device according to a first embodiment, (B) is a perspective view showing a modified finger, and (C) is a diagram showing the main parts of a modified robot hand. (A) is a side view showing the robot hand of FIG. 1A, (B) is a front view of the fingers of the robot hand as viewed in the -Y direction, and (C) is a diagram showing an example of an image within the field of view of an imaging device. (A) is a diagram showing an example of a control device for the robot device and the robot hand. (A) is a plan view showing a tool fixed to the finger, (B) is an image captured in the state shown in FIG. 4A, (C) is an image captured in a state where the gripping center of the tool is aligned with the center of the workpiece, and (D) is an image captured in a state where the workpiece is gripped by the tool. (A) is an image captured in a state where the angle between the center line of the workpiece and the center line of the tool is large, (B) is an image captured in a state after the robot hand has been rotated relative to the workpiece, (C) is an image captured in a state where the gripping center of the tool is aligned with the center of the workpiece, and (D) is a plan view showing another example of a marking member. 10A is a flowchart showing an example of a method for detecting a mark member, (B) is a flowchart showing an example of a method for gripping a workpiece with a robot hand, and (C) is a flowchart showing a modified example of the gripping method. (A) and (B) are side views of a robot device, respectively, and (C) and (D) are images captured in the cases of FIGS. 7A and 7B, respectively. (A) is a side view showing a main portion of a robot hand when gripping another workpiece, (B) is a diagram showing an image obtained by an imaging device in the state of FIG. 8A, and (C) is a cross-sectional view showing a case where workpieces are gripped at two different heights. (A) is a plan view showing a state in which multiple workpieces are gripped with a tool, (B) is a side view showing the tool in the case of FIG. 9A, (C) is a cross-sectional view showing an example of a tip portion of a tool configured not to grip multiple workpieces, (D) is a side view showing the tip portion of the tool in FIG. 9C, and (E) is an enlarged view showing another example of the tip portion of the tool. (B) is a perspective view showing a robot device of a second embodiment. (C) is a diagram showing an example of a screen captured by the imaging device of FIG. 10. 12A is a side view showing a modified robot hand, FIG. 12B is a front view of the finger in FIG. 12A viewed in the -Y direction, and FIG. 12C is a diagram showing an example of an image within the field of view of the imaging device in FIG. 12A.

[0007] First Embodiment A first embodiment will be described below with reference to FIGS. 1A to 6C. FIG. 1A shows an articulated robotic device 2 according to this embodiment. Note that any robotic device, such as a horizontal articulated robotic device (SCARA type), may be used as the robotic device 2. In FIG. 1A, as an example, the X and Y axes are taken as being parallel to and orthogonal to the installation surface of the robotic device 2, and the Z axis is taken as being perpendicular to the installation surface. The installation surface is a substantially horizontal plane, and the positive direction of the Z axis is defined as the direction above the installation surface (the direction opposite to the vertical direction). The coordinate system consisting of positions on the X, Y, and Z axes is referred to as the robot coordinate system (X, Y, Z).

[0008] 1A, the robot device 2 includes a robot main body 4, a robot hand 6 joined to the robot main body 4, a control device 10 that controls the operation of the robot device 2, and a control device 12 that controls the operation of the robot hand 6. The robot main body 4 includes a base 14, a rotating unit 16 connected to the base 14, and a first robot arm 18A, a second robot arm 18B, and a third robot arm 18C that are connected to the rotating unit 16 in a manner that allows them to move in that order. The robot hand 6 is joined to a tip 20A of the third robot arm 18C. The robot hand 6 can also be called an end effector.

[0009] As an example, the rotation angle θ1 of the rotating unit 16 around an axis parallel to the Z axis can be controlled, the swing angle φ1 of the first robot arm 18A can be controlled, the swing angle φ2 of the second robot arm 18B can be controlled, and the swing angle φ3 of the third robot arm 18C and the swing angle θ2 of the tip end 20A (robot hand 6) around an axis parallel to the Z axis can be controlled. The third robot arm 18C also has a rotating unit 20C that changes the rotation angle θ2 of the tip end 20A and a tilting drive unit 20B that changes the swing angle φ3. The rotating unit 20C and the tilting drive unit 20B each have a drive motor and an encoder that detects the rotation angle. The rotating unit 16 and the other robot arms 18A and 18B also have drive motors and encoders. Therefore, the robot hand 6 joined to the third robot arm 18C can be controlled in four degrees of freedom (including angle), including at least the position in the X direction (direction along the X axis; same applies below), Y direction, Z direction, and the rotation angle around the Z axis.

[0010] The origin 58A of the robot coordinate system (X, Y, Z) is, for example, the center of the base unit 14, but the origin 58A can be set or changed to any position. As an example, the robot hand 6 is driven by the rotating unit 16 and the robot arms 18A to 18C so as to move parallel to the Z axis. Note that, because the robot hand 6 has at least five encoders, it is also possible to control the position of the robot hand 6 with three degrees of freedom and the angle with two degrees of freedom. Furthermore, instead of the robot device 2, the robot hand 6 may be joined to a robot device (not shown) that is capable of controlling the position with three degrees of freedom and the angle with three degrees of freedom.

[0011] The robot device 2 also includes an imaging device 22 and a lighting device (not shown). As an example, the imaging device 22 is attached to the robot hand 6. However, the imaging device 22 may be provided at a location other than the robot hand 6, such as the side of the robot arm 18C, a measurement position on the installation surface of the robot device 2, or a wall or ceiling of the room in which the robot device 2 is installed. As an example, the imaging device 22 is a camera that captures ordinary two-dimensional images. The focal position is set, for example, near an area including the tips of the tools 8b and 8c (described below) fixed in the robot hand 6, and the focal depth is set relatively deep. The imaging device 22 may also include an autofocus lens. When the focal position of the imaging device 22 is near an area including the tips of the tools 8b and 8c, when the tips of the tools 8b and 8c are in contact with or close to the placement surface FL of the workpiece 48, the workpiece 48 will also be located at approximately the focal position of the imaging device 22. However, a stereo camera or the like may also be used as the imaging device 22.

[0012] The robot hand 6 of this embodiment also has two fingers 38A, 38B that can be driven to open and close. Members for gripping an object (hereinafter referred to as tools) 8b, 8c are fixed to the inner surfaces of the fingers 38A, 38B, respectively, with bolts B1 or the like (see FIG. 2A ). As an example, the tools 8b, 8c (gripping portions) are supported so as to protrude forward of the robot hand 6 and have tips that protrude downward, allowing an object to be gripped between the tips. The tips of the tools 8b, 8c may have any shape depending on the object. While the tools 8b, 8c are made of metal, some or all of the tools 8b, 8c may be made of synthetic resin or the like. By closing the fingers 38A, 38B, the tools 8b, 8c close in response to the movement of the fingers 38A, 38B, and an object can be gripped or supported between the tips of the movable portions 8b, 8c. Furthermore, by opening the fingers 38A and 38B, the tools 8b and 8c open in accordance with this movement, and the tools 8b and 8c can be removed from the object.

[0013] Note that one tool 8b may be fixed, and only the other tool 8c may be movable. That is, the tools 8b and 8c may be capable of opening and closing relative to each other. In this case, the finger 38A to which the fixed tool 8b is fixed may also be fixed. Furthermore, three or more tools may be used instead of the tools 8b and 8c. In this case, only at least one of the three or more tools may be movable, and the other tools may be fixed. When three or more tools are held by the robot hand 6, the number of fingers 38A, etc. may be the same as the number of the tools. Furthermore, when some of the three or more tools are fixed, the fingers 38A, etc. supporting the fixed tools may be fixed.

[0014] Furthermore, instead of fixing the tools 8b and 8c to the fingers 38A and 38B, fingers 38C and 38D whose tips 38Cb and 38Db are machined to resemble tools may be used, as shown in FIG. 1B. Hereinafter, the objects to be gripped by the tools 8b and 8c (objects to be gripped) are assumed to be multiple workpieces 48 placed on a mounting surface FL that is substantially parallel to the horizontal plane. As an example, the workpieces 48 are bolts having a shaft 48a and a head 48b. The shapes and sizes of the objects to be gripped are arbitrary, as long as they have a portion that can be gripped by the tools 8b and 8c. Furthermore, even if the workpieces are long, string-like or cable-like, they can be gripped by the tools 8b and 8c as long as they are within the diameter of the cross-sectional shape.

[0015] 2A shows the robot hand 6 in FIG. 1A, and illustrates a state in which the robot hand 6 is joined to the lower end of the tip 20A of the robot arm 18C so that the opening and closing directions of the tools 8b and 8c are parallel to the X axis (X direction). In FIG. 2A, the robot hand 6 includes a hand main body 28 joined to the lower end of the tip 20A, a connecting portion 36 connected to an actuator 32 of the hand main body 28, a pair of fingers 38A and 38B that are opened and closed in the X direction by the actuator 32, and tools 8b and 8c fixed to the fingers 38A and 38B. The tools 8b and 8c are opened and closed in the X direction by the fingers 38A and 38B. The fingers 38A and 38B are made of, for example, metal.

[0016] The hand main body 28 also includes a frame 30 having a flat joint (joint) joined to the tip end 20A, and an actuator 32 connected to the frame 30 via a linear slider 34 so as to be relatively movable in the Z direction. The imaging device 22 is supported on the +Y-direction side of the frame 30 via a support member 50. The joint of the frame 30 is joined to the tip end 20A mechanically (by screws, a movable claw mechanism, a permanent magnet, etc.) or electrically (by an electromagnet, etc.). Furthermore, the joint is provided with connections for electrical wiring, compressed air piping, and / or vacuum suction piping, as needed. The joint of the frame 30 is provided with a plurality of positioning portions (not shown), such as protrusions or recesses, so that the position and rotation angle relative to the tip end 20A are predetermined.

[0017] The imaging device 22 is also positioned slightly diagonally upward (in the +Y and +Z directions) with respect to the tools 8b and 8c. As shown in FIG. 2C , the center of the image 22G captured within the field of view of the imaging device 22 is defined as an origin 58B. The x-axis passes through the origin 58B and is parallel to the longitudinal direction of the image 22G, and the y-axis is parallel to the lateral direction of the image 22G. The coordinate system (x, y) is referred to as the coordinate system of the imaging device 22. Note that the position of the origin 58B is arbitrary. In the state shown in FIG. 2C , the x-axis and y-axis are parallel to the X-axis and Y-axis of the robot coordinate system (X, Y, Z), respectively. However, the position and angle of the coordinate system (x, y) of the imaging device 22 relative to the robot coordinate system (X, Y, Z) change as the robot hand 6 is driven by the robot arms 18A to 18C.

[0018] Furthermore, the position of the imaging device 22 is set so that the tips of the images 8bI and 8cI of the tools 8b and 8c are located on the -y direction side of the origin 58B in the image 22G. Also, because the magnification of the imaging device 22 is known, when the Z positions of the tools 8b and 8c in the robot coordinate system (X, Y, Z) are set to a certain position, the control device 10 can obtain the spacing (approximate values) in the X and Y directions on the robot coordinate system (X, Y, Z) corresponding to that spacing by dividing the spacing (distance) in the x and y directions on the image 22G of the imaging device 22 by that magnification.

[0019] Two expandable tension coil springs (hereinafter referred to as elastic members) 44A, 44B are connected in a direction parallel to the Z axis between the joint (upper plate portion) of the frame 30 and both X-direction side surfaces of the connecting portion 36. Hereinafter, the elastic members 44A, 44B are collectively referred to as elastic member 44. A suspension mechanism 46 (buffer mechanism) is configured, including the linear slider 34 and the elastic member 44, to support the movable portion (a portion including the actuating portion 32, fingers 38A, 38B, and tools 8b, 8c) relative to the frame 30 so as to be elastically displaceable in the Z direction. A stopper 52 for limiting the displacement of the actuating portion 32 (movable portion) relative to the frame 30 in the Z direction within a predetermined range is configured by a U-shaped limiting member 52a provided on the frame 30 and a pin 52b provided on the actuating portion 32 so as to be displaceable in the Z direction between the limiting member 52a. When the tools 8b and 8c are not gripping the processing object, the stopper 52 is set so that the pin 52b is positioned in the middle of the restricting member 52a.

[0020] The suspension mechanism 46 may have any configuration, such as a mechanism that uses, for example, a leaf spring instead of a tension coil spring as the elastic member 44. In this embodiment, the above-mentioned movable part is supported by the suspension mechanism 46. As another configuration, for example, a suspension mechanism or the like may be used that fixes the operating part 32 to the frame 30 and supports the tools 8 b, 8 c so that they can be elastically displaced in the Z direction relative to the operating part 32.

[0021] Furthermore, a proximity sensor 56 is provided above the actuating unit 32 inside the frame 30 for non-contact detection of the distance between the actuating unit 32 and the movable unit 32. The detection result of the proximity sensor 56 can detect the relative displacement of the movable unit (the actuating unit 32, the tools 8b, 8c, etc.) with respect to the frame 30, and this detection result can be used to detect whether the tips of the tools 8b, 8c have contacted the mounting surface FL of the workpiece 48. If the mounting surface FL has a certain degree of conductivity, a contact sensor or the like that detects conductivity between the tools 8b, 8c and the mounting surface FL may be used instead of the proximity sensor 56. Alternatively, a proximity sensor that detects whether the tips of the tools 8b, 8c have approached the mounting surface FL may be provided on the side of the fingers 38A, 38B, for example, and this proximity sensor may be used to detect when the tips of the tools 8b, 8c have approached the mounting surface FL.

[0022] As another example, when the robot device 2 starts operating, for example, if the proximity sensor 56 detects that the tip of the tool 8 b, 8 c has come into contact with the placement surface FL of the workpiece 48, the value of the position on the Z axis of the robot coordinate system (X, Y, Z) (hereinafter also referred to as the Z position) may be preset to 0. This preset information is supplied to, for example, a coordinate calculation unit 11C (see FIG. 3 ), which will be described later. Thereafter, the control devices 10 and 12 may recognize the distance in the Z direction between the tip of the tool 8 b, 8 c and the placement surface FL based on the value of the Z position of the robot coordinate system (X, Y, Z).

[0023] Furthermore, by detaching the frame 30 from the tip 20A and joining the frame 30 of a robot hand equipped with tools 8b, 8c of a different shape or fingers 38A, 38B of a different shape to the tip 20A, it is possible to replace the robot hand 6 joined to the tip 20A of the robot arm 18C. Furthermore, in the robot hand 6 joined to the tip 20A, the tools 8b, 8c fixed to the fingers 38A, 38B can be replaced with tools of a different shape depending on the shape of the object to be processed, etc.

[0024] 2(B) is a front view of FIG. 2(A) as viewed in the -Y direction, and in FIG. 2(B), the actuating unit 32 includes a finger driving unit 13C (see FIG. 3) including a rotary motor, a conversion unit (not shown) that reduces the rotational speed of the finger driving unit 13C and converts it into linear motion to open and close the fingers 38A, 38B, and a gap detection unit 13B (see FIG. 3) that detects the position and gap between the fingers 38A, 38B from the rotation angle of the finger driving unit 13C. The actuating unit 32 causes the pair of fingers 38A, 38B to open and close the tools 8b, 8c in the opening and closing direction (here, the X direction).

[0025] In this embodiment, the actuator 32 drives the fingers 38A and 38B to change the spacing between the tools 8b and 8c in the opening and closing direction. The actuator 32 may also operate the fingers 38A and 38B so that the tools 8b and 8c move independently in the X direction. When not gripping or supporting an object, the −X direction finger 38A (tool 8b) waits at the outermost (−X direction) position (hereinafter also referred to as the standby position) within its movable range in the X direction, and the +X direction finger 38B (tool 8c) waits at the outermost (+X direction) position (hereinafter also referred to as the standby position) within its movable range in the X direction. When gripping an object with the tools 8b and 8c, the fingers 38A and 38B move from the standby position to close the tools 8b and 8c, and when separating the tools 8b and 8c from the object, the fingers 38A and 38B move toward the standby position.

[0026] 3 shows the control device 10 of the robot device 2 and the control device 12 of the robot hand 6. In Fig. 3, the control device 10 has a control information input / output unit 11A that inputs and outputs control information (such as the type of tool to be used and the position of the object to be processed) to and from an operator, a main control unit 11B that transmits and receives control information to and from the control device 12 of the robot hand 6 and controls the operation of the entire device, and a coordinate calculation unit 11C that processes the detection results of at least five-axis encoders that detect the movements of the robot arms 18A to 18C, etc., and calculates the coordinates and rotation angles of the tips of the tools 8b and 8c held by the robot hand 6 at the tip of the third robot arm 18C.

[0027] The control device 10 further includes an image processing unit 11D that processes image signals from an imaging device 22 attached to the robot hand 6 to determine, for example, the positional relationship between the tip of a tool 8b or 8c held by the robot hand 6 and an object; an arm control unit 11E that controls the movement of the rotation unit 16 and the robot arms 18A-18C based on the processing results of the coordinate calculation unit 11C and the image processing unit 11D; and a memory unit 11F that stores various data. The image processing unit 11D may be provided in the control device 12 on the robot hand 6 side. The memory unit 11F may include, for example, a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), or a non-volatile memory such as a USB (universal serial bus) memory. The memory unit 11F stores programs and various setting values ​​for the control device 10.

[0028] The control device 12 also includes a hand control unit 13A that exchanges control information between the main control unit 11B and the arm control unit 11E of the control device 10 and controls the overall operation of the robot hand 6, a finger drive unit 13C that opens and closes the fingers 38A and 38B, and a spacing detection unit 13B that detects the position and spacing of the fingers 38A and 38B. The control device 12 also includes a finger gripping force detection unit 13D that detects the reaction force (force of the fingers 38A and 38B) from the tools 8b and 8c to the fingers 38A and 38B as a finger gripping force FF, a contact detection unit 13E that processes the detection signal of the proximity sensor 56 in FIG. 2 to detect whether the tools 8b and 8c held by the robot hand 6 have contacted the mounting surface FL, and a memory unit 13F. The memory unit 13F has a configuration similar to that of the memory unit 11F and stores programs and various setting values ​​of the control device 12.

[0029] As an example, the finger grip force detection unit 13D detects the finger grip force FF from the current flowing through the rotation motor of the finger drive unit 13C, and the hand control unit 13A controls the operation of the fingers 38A, 38B (finger drive unit 13C) so that the detected finger grip force FF is equal to or less than a preset level (standard value). This prevents damage to the tools 8b, 8c, the fingers 38A, 38B, and / or the target object. Note that instead of the finger grip force detection unit 13D, a sensor capable of detecting the finger grip force FF, such as a strain gauge or pressure sensor provided on the fingers 38A, 38B, can be used. The control information input to the hand control unit 13A includes, for example, an instruction to start gripping an object by the tools 8b and 8c and an instruction to release the grip, and the control information output from the hand control unit 13A to the main control unit 11B includes, for example, information indicating that the tools 8b and 8c have completed gripping the object, and information indicating that the tools 8b and 8c have completed releasing their grip on the object (completion of detachment of the tools from the object).

[0030] Next, an example of the basic operation of the robot device 2 of this embodiment will be described. First, FIG. 4A shows the fingers 38A, 38B, tools 8b, 8c, and a workpiece 48 to be grasped, as viewed from the side of the actuation unit 32 of the robot hand 6 in FIG. 2A. The workpiece 48 has a small-diameter shaft 48a and a large-diameter head 48b. In the robot device 2, as an example, the robot hand 6 (tools 8b, 8c) is moved in the X and Y directions with the tips of the tools 8b, 8c in contact with the mounting surface FL of the workpiece 48 (or set at a position slightly higher than the mounting surface FL) to place the shaft 48a of the workpiece 48 between the tips of the tools 8b, 8c. Then, the fingers 38A, 38B are closed to grasp the shaft 48a of the workpiece 48 between the tools 8b, 8c.

[0031] In FIG. 4A , small, flat-plate marking members 62A and 62B, made of metal or the like, on which marks MA and MB are formed are fixed to the tips of tools 8b and 8c (portions that can be captured in the image 22G of the imaging device 22). As an example, the midpoint of the line segment connecting the centers of marks MA and MB is considered to be the gripping center 58C of tools 8b and 8c. The gripping center 58C is set slightly inward from the center (midpoint) of the tips of tools 8b and 8c. The marking members 62A and 62B are fixed to the top surfaces of tools 8b and 8c, for example, by adhesive or glue, so as not to interfere with the opening and closing of tools 8b and 8c. In this embodiment, marks MA and MB have the same circular shape, but marks MA and MB may have different shapes. Furthermore, if the shape of a portion of tools 8b and 8c is distinctive, that portion may be used instead of marks MA and MB or marking members 62A and 62B.

[0032] The marks MA and MB can be any mark or marker, such as a symbol other than a circle, such as a cross or polygon, an alphabet or number, or a symbol containing predetermined information, such as a barcode. The marks MA and MB can also be identification codes such as April Tags or QR Codes (registered trademark) (two-dimensional barcodes). Such identification or recognition codes can be used to recognize and manage whether the tools 8b and 8c being used for the object to be grasped (workpiece) have the correct shape. The April Tag is a visual mark (marker) for applications that detect the position and orientation of a three-dimensional object. The April Tag is also an example of an AR (Augmented Reality) marker.

[0033] In this embodiment, the tools 8b and 8c are substantially line-symmetrical with respect to a center line 8cl in a plane substantially parallel to a plane including the Y-axis and the Z-axis, and the marks MA and MB are fixed at positions substantially line-symmetrical with respect to the tools 8b and 8c with respect to the center line 8cl. Furthermore, since the tools 8b and 8c are inclined with respect to a plane perpendicular to the optical axis of the imaging device 22 (see FIG. 2A), the marks MA and MB of the mark members 62A and 62B fixed to the tools 8b and 8c are formed on the mark members 62A and 62B in shapes that can be easily identified in an image captured by the imaging device 22.

[0034] 4B shows an image 22G obtained by capturing an image of the tools 8b, 8c and the workpiece 48 in the state shown in FIG. 4A using the imaging device 22 shown in FIG. 4B. In FIG. 4B, the images 8bI, 8cI of the tools 8b, 8c, the images 62AI, 62BI of the mark members 62A, 62B, the images MAI, MBI of the marks MA, MB, and the image 48I of the workpiece 48 are displayed in the image 22G. The information of the image 22G is supplied to the image processing unit 11D shown in FIG. 3, and the image processing unit 11D regards the center of the images MAI, MBI of the two marks as the image 58CI of the gripping center 58C of the tool 8. Furthermore, the image processing unit 11D regards the center of the image of the shaft portion 48a in the image 48I of the workpiece 48 as the image 48cI of the center of the workpiece 48, as an example. Furthermore, the image processing unit 11D calculates the distance δx in the x direction and the distance δy in the y direction on the image 22G between the image 48cI of the center of the workpiece 48 and the image 58CI of the gripping center of the tools 8b and 8c, and then divides these distances (distances) by the magnification of the imaging device 22 to calculate distances converted into distances in the X direction and Y direction on the robot coordinate system (X, Y, Z), and supplies the calculated distances to the arm control unit 11E via the main control unit 11B.

[0035] In response to this, the arm control unit 11E drives the rotating unit 16 and the robot arms 18A-18C to offset the gap, so that the image of the center of the workpiece 48 substantially coincides with the image 58CI of the gripping center of the tools 8b and 8c, as shown in image 22G of the imaging device 22 in FIG. 4C. If the image of the center of the workpiece 48 and the image 58CI of the gripping center of the tools 8b and 8c are significantly misaligned in this state, the robot arms 18A-18C, etc., can be driven again so that the image of the center of the workpiece 48 and the image 58CI of the gripping center coincide with each other. Then, by closing the tools 8b and 8c, the shaft of the workpiece 48 is gripped between the tools 8b and 8c so that the image 48I of the workpiece is sandwiched between the tool images 8bI and 8cI in image 22G in FIG. 4D. Furthermore, as shown in FIG. 4C , even if the center line of the workpiece 48 (a straight line passing through the center of the shaft portion 48a) and the center line of the tools 8b and 8c are inclined to some extent, closing the tools 8b and 8c makes the center line of the workpiece 48 and the center line of the tools 8b and 8c substantially parallel, allowing the workpiece 48 to be gripped between the tools 8b and 8c. Thereafter, the robot arms 18A-18C, etc., are driven to transport the workpiece 48 gripped by the tools 8b and 8c of the robot hand 6 to a workpiece storage box (not shown), etc., and the fingers 38A and 38B (tools 8b and 8c) are opened to remove the workpiece 48, thereby completing the movement of the workpiece 48. The gripping position of the workpiece 48 does not necessarily have to be the center 48c of the workpiece. The tools 8b and 8c may be positioned so that the appropriate gripping position for each individual workpiece coincides with the gripping center of the tools 8b and 8c. For example, the center of gravity of the workpiece may coincide with the gripping center of the tools 8b and 8c.

[0036] Next, an example of a control method for the robot apparatus 2 according to this embodiment will be described with reference to the flowcharts of FIGS. 6A to 6C. First, as a preparation step, in step 102 of FIG. 6A, the tools 8b and 8c are fixed (mounted) to the fingers 38A and 38B of the robot hand 6, as shown in FIG. 4A. Furthermore, in step 104, marking members 62A and 62B carrying marks MA and MB are attached to the tips of the tools 8b and 8c. Note that step 104 may be performed before step 102. Then, in step 106, as shown in FIG. 4A (the Z positions of the tips of the tools 8b and 8c may be arbitrary), the imaging device 22 captures images of the marks MA and MB on the marking members 62A and 62B at the tips of the tools 8b and 8c.

[0037] Furthermore, in step 108, the image processing unit 11D calculates the position (xm, ym) of the center of the image of the marks MA and MB (the image of the gripping center 58C or the gripping position) on the coordinate system (x, y) of the imaging device 22 from the obtained image 22G, and stores the obtained position (xm, ym) in the memory unit 11F. Note that if the relative positions of the tools 8b and 8c with respect to the imaging device 22 hardly change, subsequent imaging of the marks MA and MB by the imaging device 22 can be omitted, and the stored position (xm, ym) can be used as the gripping position. However, if there is a possibility that the relative positions of the tools 8b and 8c with respect to the imaging device 22 may change due to vibration, short-term changes over time, or the like, steps 106 and 108 may be executed repeatedly.

[0038] Thereafter, when the robot device 2 transports the multiple workpieces 48 shown in FIG. 1A to a workpiece storage box (not shown) or the like, in step 112 of FIG. 6B, the tips of the tools 8b and 8c held by the robot hand 6 are brought into contact with the placement surface FL of the workpieces 48, as shown in FIG. 2A. At this time, the contact of the tips of the tools 8b and 8c with the placement surface FL can be detected from the detection signal of the proximity sensor 56. Note that if the Z position of the tools 8b and 8c is preset to 0 in advance when the tips of the tools 8b and 8c contact the placement surface FL, the robot hand 6 may be simply driven via the robot arms 18A-18C, etc., so that the Z position becomes 0 on the robot coordinate system (X, Y, Z). Note that the contact of the tips of the tools 8b and 8c with the placement surface FL may be determined, for example, by image processing of an image captured by an imaging device (not shown) capturing an image of the vicinity of the placement surface FL from the side of the robot hand 6. Furthermore, the operator may visually determine that the tips of the tools 8 b and 8 c have come into contact with the mounting surface FL, and set the Z position at that time to 0. Also, in step 112, the height of the tips of the tools 8 b and 8 c may be set to a position higher than the mounting surface FL by a predetermined value δZ (for example, a height at which they will not come into contact with the workpiece 48).

[0039] Then, in step 114, with the tips of the tools 8b and 8c raised by a predetermined value δZ above the mounting surface FL via the robot arms 18A to 18C, etc. (if the heights of the tools 8b and 8c were set to that predetermined value in step 112, the heights are left unchanged), the robot hand 6 is driven in the X and Y directions to a position where the workpiece 48 is within the image 22G of the imaging device 22, and an image of the workpiece 48 is captured by the imaging device 22. Then, in step 116, the image processing unit 11D calculates the distances δx and δy in the x and y directions on the image 22G from the center of the workpiece image 48I to the gripping center image 58CI (gripping center), as shown in FIG. 4(B), and calculates the distance (amount of positional deviation) on the robot coordinate system (X, Y, Z) from this distance.

[0040] Furthermore, in step 118, the robot hand 6 is moved by the robot arms 18A-18C, etc. so as to offset the positional deviation, and then the tips of the tools 8b, 8c are brought into contact with the mounting surface FL or brought close to the mounting surface FL. Then, in step 120, the workpiece 48 is gripped by the tools 8b, 8c of the robot hand 6. Then, in step 122, the robot arms 18A-18C, etc. are driven to move the robot hand 6, and the workpiece 48 gripped by the tools 8b, 8c of the robot hand 6 is transported to a storage box or the like, and the tools 8b, 8c are detached from the workpiece 48. Thereafter, in step 124, if there are remaining workpieces 48, the process proceeds to step 112, and the next workpiece 48 is transported from the mounting surface FL to a storage box or the like. When there are no remaining workpieces 48 in step 124, the transport process ends.

[0041] If the imaging results of step 114 indicate a large angle between the workpiece 48 and the tools 8b and 8c, the rotation angle correction process of FIG. 6C may be performed. In step 128 of FIG. 6C, the image processing unit 11D calculates the angle Δθ between the image 48c1I of the workpiece center line and the image 8c1I of the tool center line on the image 22G of the imaging device 22, as shown in FIG. 5A. The angle θ is an angle around an axis approximately parallel to the Z axis. Then, in step 130, if the angle Δθ is smaller than a predetermined angle (e.g., 30 degrees), the workpiece 48 can be gripped by the tools 8b and 8c. Therefore, the process proceeds to step 116 without correcting the rotation angle, and the workpiece 48 is gripped by the tools 8b and 8c.

[0042] On the other hand, if the angle θ is equal to or greater than the predetermined angle in step 130, the process proceeds to step 132, where the robot arms 18A-18C are driven to rotate the robot hand 6 (tools 8b, 8c) around an axis parallel to the Z axis so as to offset the angle θ. As a result, as shown in FIG. 5B, the image 48cI of the center line of the workpiece 48 becomes parallel to the images of the center lines of the tools 8b, 8c. After this, the process proceeds to step 116. If the workpiece has a substantially circular shape that can be gripped without angle correction, no rotation angle correction is necessary, and the rotation angle correction step shown in FIG. 6C can be omitted. Then, in FIG. 5B, the image processing unit 11D calculates the distances δx and δy in the x and y directions from the center 48cI of the workpiece image 48I to the gripping center image 58CI, and from this distance, calculates the distance (positional deviation) in the robot coordinate system (X, Y, Z). Furthermore, in step 118, the robot hand 6 is moved so as to offset the positional deviation, and in step 120, the tools 8b and 8c are closed as shown by the tool images 8bI and 8cI in Figure 5(C), thereby enabling the workpiece 48 to be grasped by the tools 8b and 8c.

[0043] As described above, according to this embodiment, the tools 8b and 8c are fixed so as to be opened and closed by the fingers 38A and 38B of the robot hand 6. Furthermore, the images of the marks MA and MB on the marking members 62A and 62B attached to the tools 8b and 8c are detected, and an image 58CI of the tool's gripping center 58C is obtained from the detection results. The robot hand 6 is then driven to align the image 58CI with the center of the image 48I of the workpiece 48, thereby efficiently and accurately aligning the robot hand 6 (tool) with the workpiece 48. The tool's gripping center 58C may be aligned with any position suitable for gripping the workpiece 48. As a result, the workpiece 48 can be efficiently and accurately gripped by the tools 8b and 8c of the robot hand 6.

[0044] Furthermore, when the angle between the workpiece 48 and the tools 8b, 8c is equal to or greater than a predetermined angle, the workpiece 48 and the tools 8b, 8c can be efficiently aligned by rotating the robot hand 6 (tool) to offset that angle. Furthermore, because the tips of the tools 8b, 8c are brought into contact with the mounting surface FL of the workpiece 48 to align the tools 8b, 8c with respect to the workpiece 48, alignment in the Z direction can be performed automatically. Therefore, the tools 8b, 8c can be efficiently aligned with respect to the workpiece 48 using two-dimensional images captured by a normal imaging device 22.

[0045] As described above, the robot device 2 of this embodiment includes the robot hand 6 provided with a plurality of fingers 38A, 38B, the rotating unit 16 and robot arms 18A to 18C (arm unit) that drive the robot hand 6, a plurality of tools 8b, 8c (gripping units) that are capable of gripping the workpiece 48 in response to the plurality of fingers 38A, 38B, mark members 62A, 62B (identification units) that the plurality of tools 8b, 8c have, and a main control unit 11B (control unit) that uses position information of the mark members 62A, 62B to drive the robot hand 6 via the arm unit and position the plurality of tools 8b, 8c relative to the workpiece 48.

[0046] Furthermore, the control method for the robot device 2 is a control method for the robot device 2 that includes a robot hand 6 provided with a plurality of fingers 38A, 38B, a rotating unit 16 that drives the robot hand 6, and robot arms 18A to 18C (arm units), and includes steps 104 and 112 of causing a plurality of tools 8b, 8c (gripping units) that can grip a workpiece 48 to follow the plurality of fingers 38A, 38B, a step 108 of storing position information of mark members 62A, 62B that the plurality of tools 8b, 8c have, and steps 114, 116, and 118 of driving the robot hand 6 via the arm unit using the position information of the mark members 62A, 62B to position the plurality of tools 8b, 8c relative to the workpiece 48.

[0047] According to this embodiment, the positions (gripping centers) of the tools 8b and 8c are determined from the position information of the mark members 62A and 62B carried by the tools 8b and 8c, and therefore even if the shapes of the tools 8b and 8c or the fingers 38A and 38B are difficult to distinguish, or even if it is difficult to distinguish the shapes of the tools 8b and 8c or the fingers 38A and 38B from the shape of the workpiece 48, it is possible to efficiently and accurately align the workpiece 48 with the robot hand 6. Furthermore, since the tools 8b and 8c having shapes appropriate for the processing object can be fixed to the fingers 38A and 38B and the tools 8b and 8c can be opened and closed by the fingers 38A and 38B, the processing object can be gripped and moved efficiently or without damaging the processing object using tools having shapes appropriate for the processing object.

[0048] In the above embodiment, the workpiece 48 gripped by the tools 8b and 8c is moved. However, various operations such as pressing down, twisting, pulling out, peeling, or attaching the workpiece gripped or supported by the tools 8b and 8c held by the robot hand 6 of the above embodiment are also possible. Furthermore, while the two fingers 38A and 38B are movable in the above embodiment, for example, only one tool 8b may be movable by one finger 38A, while the other tool 8c may be fixed. In this case, the other finger 38B may be fixed. The robot hand 6 then operates only one finger 38A to open and close the tools 8b and 8c. In this case, the marking member 62A may be attached only to the tool 8b, and the fixed tool 8c may not be provided with the marking member 62B. In this case, the position of the gripping center of the tools 8b and 8c may be determined based on the detection result of the mark MA of one marking member 62A.

[0049] Furthermore, in the above-described embodiment, the mark members 62A, 62B (marks MA, MB) are detected by capturing an image using the imaging device 22, but the positions of the mark members 62A, 62B may be detected using any other detection method (for example, a method using a gap sensor, etc.). Furthermore, in the above-described embodiment, the workpiece is gripped by opening and closing the tools 8b, 8c with the fingers 38A, 38B of the robot hand 6, but the workpiece may also be gripped directly with the fingers 38A, 38B of the robot hand 6. In this case, the mark members 62A, 62B (marks MA, MB) may be attached to the fingers 38A, 38B, and the positions of the gripping centers of the fingers 38A, 38B may be determined from detection information of the mark members 62A, 62B. The positions may then be used to position the fingers 38A, 38B relative to the workpiece, and the workpiece may then be gripped by the fingers 38A, 38B.

[0050] Next, the above-described embodiment can be modified as follows. First, in the above-described embodiment, the marking members 62A and 62B are attached to the upper portions (end portions) of the tools 8b and 8c. Alternatively, as shown in FIG. 5D , marking members 62C and 62D with an L-shaped cross section may be fixed to the outer side surfaces of the tools 8b and 8c, and marks MA and MB may be formed on the marking members 62C and 62D. In this configuration, even with the marking members 62C and 62D, the tools 8b and 8c can more easily grip the workpiece.

[0051] Next, in the above-described embodiment, the proximity sensor 56 detects that the tips of the tools 8b and 8c held by the robot hand 6 have contacted the workpiece 48's mounting surface FL, and in this state, the workpiece 48 and the tool tips are aligned in the Z direction. Alternatively, as shown in FIG. 7A, the robot arms 18A-18C of the robot device 2 are driven to bring the tips of the tools 8b and 8c fixed to the fingers 38A and 38B into contact with the workpiece 48's mounting surface FL, thereby moving the tools 8b and 8c to a position where they sandwich the shafts of the workpiece 48. In this state, the imaging device 22 provided on the robot hand 6 captures images of the tools 8b and 8c, thereby obtaining an image 22G shown in FIG. 7C. The image processing unit 11D processes the image 22G in FIG. 7C to determine the x- and y-direction positions of the workpiece image 48I that passes through the grip center image 58CI within the images 8bI and 8cI of the tools 8b and 8c. In this state, the Z position of the robot coordinate system (X, Y, Z) is set to 0.

[0052] In this case, as shown in FIGS. 7A and 7B, when the height of the imaging device 22 changes, the position of the image 48I of the workpiece 48 on the image 22G moves in the y direction due to the parallax of the imaging device 22. Therefore, as shown in FIG. 7B, the robot arms 18A-18C, etc. are driven to raise the position of the robot hand 6 (imaging device 22 and tools 8b, 8c) in the +Z direction by a predetermined distance Za from the state shown in FIG. 7A. In this state, the imaging device 22 captures images of the workpiece 48 and tools 8b, 8c, thereby obtaining the image 22G shown in FIG. 7D. The image processing unit 11D processes the image 22G in FIG. 7D to determine the amount of movement Δy in the y direction of the workpiece image 48I relative to the state shown in FIG. 7C. The Z position Za and movement amount Δy at this time are stored.

[0053] Thereafter, conversely, with the Z position of the robot hand 6 (tools 8b, 8c) set to Za, the robot hand 6 is positioned so that the center of the workpiece image 48I is shifted by Δy in the y direction from the centers of the tool images 8bI, 8cI on the image 22G. Thereafter, by lowering the Z position of the robot hand 6 to 0, the shaft of the workpiece 48 can be positioned between the tools 8b, 8c, as shown in FIG. 7A, and the workpiece 48 can be grasped by the tools 8b, 8c.

[0054] Next, as shown in FIG. 8A, a workpiece 48A having a truncated conical shank 48Aa and a disk-shaped head 48Ab is assumed to be gripped by tools 8Bb and 8Bc with flat tips, with the shank 48Aa standing on a mounting surface FL. At this time, the tools 8Bb and 8Bc, to which marking members 62A and 62B are attached, are moved to both sides of the shank 48Aa of the workpiece 48A. In this case, as shown in FIG. 8B, in the image 22G captured by the imaging device 22, the head image 48AbI of the workpiece image 48AI partially covers the mark images MAI and MBI of the marking members 62A and 62B. This may make it difficult to accurately calculate the gripping center between the tools 8Bb and 8Bc from the mark images MAI and MBI. Therefore, as an example, the base color of the mark members 62A, 62B can be set to a color different from the mounting surface FL of the workpiece 48A, and the images 62AaI, 62BaI of the outer edge portions of the mark members 62A, 62B can be used as positional references, and the position of the gripping center of the tools 8Bb, 8Bc can be determined from the position of the images 62AaI, 62BaI of the edge portions.

[0055] Furthermore, the robot device 2 of the above-described embodiment is equipped with a finger grip force detector 13D and a gap detector 13B that detects the gap between the fingers 38A and 38B. For example, in FIG. 4A , when the fingers 38A and 38B close the tools 8b and 8c with nothing between them, the finger grip force increases when the tools 8b and 8c come into contact. This allows the hand control unit 13A to recognize that the tools 8b and 8c have come into contact. The gap between the fingers 38A and 38B in this state is stored as an initial value. When the fingers 38A and 38B are subsequently opened to open the tools 8b and 8c, the gap between the tools 8b and 8c can be calculated by subtracting the initial value from the gap between the fingers 38A and 38B.

[0056] 8(C), it is possible to detect the distance d between the tools 8Bb and 8Bc when the tools 8Bb and 8Bc grip the shanks 48Aa of the workpiece 48A at a height Z1 from the mounting surface FL, and the distance d' between the tools 8Bb and 8Bc when the tools 8Bb and 8Bc grip the shanks 48Aa at a height Z2 from the mounting surface FL. From the distances d, d', etc., the hand control unit 13A or the main control unit 11B can recognize the diameter (size) and / or its distribution (shape), etc. of the shanks 48Aa of the workpieces gripped by the tools 8Bb and 8Bc.

[0057] Furthermore, by using the detection value of the gap detection unit 13B, it is also possible to measure the length of the workpiece 48 or object gripped by the tools 8Bb, 8Bc, etc. In this case, in order to measure the length more accurately, the detection value of the gap detection unit 13B may be calibrated by gripping a workpiece of known length in advance, such as a gauge block, with the tools 8Bb, 8Bc, etc. In this way, the method of gripping the workpiece 48 with the tools 8Bb, 8Bc, etc. of this embodiment provides a function for measuring the length of an object such as the workpiece 48. Furthermore, by using the detection value of the gap detection unit 13B, it is also possible to determine whether the workpiece is in a gripped state, in which the workpiece is gripped by the tools 8Bb, 8Bc, etc., or in a non-gripped state, in which the workpiece is not gripped by the tools 8Bb, 8Bc, etc.

[0058] Furthermore, the spacing between the mark images MAI, MBI of the marking members 62A, 62B captured by the imaging device 22 when the tip of the tool is gripping a workpiece or when the tip of the tool is closed may be used. In this case, as an example, a reference value is stored in advance, which is an intermediate value between the spacing between the mark images when one workpiece is gripped by the tool and the spacing between the mark images when the tip of the tool is closed. The main control unit 11B can then determine that the tool is in a non-gripping state, in which the workpiece is not being gripped, when the spacing between the mark images is narrower than the reference value, and that the tool is in a gripping state, in which the workpiece is being gripped, when the spacing between the mark images is wider than the reference value.

[0059] Next, as shown in FIG. 9(A), assume that there are multiple workpieces 48 between the tools 8Cb and 8Cc held by the robot hand 6 in FIG. 1(A). In this case, as shown by the arrows, when the tools 8Cb and 8Cc are closed, there is a risk that two workpieces 48, for example, at positions P1 and P2, may be gripped between the tools 8Cb and 8Cc. FIG. 9(B) is a side view of the two cases in FIG. 9(A). From FIG. 9(B), it can be seen that the two workpieces 48 face in opposite directions and are stacked between the tools 8Cb and 8Cc in the height direction relative to the mounting surface. In this case, even if the tool 8C is moved by the robot hand 6, it may be difficult to accurately place the two workpieces 48 in a storage box (not shown) or the like.

[0060] To prevent the tools from accidentally gripping multiple workpieces 48, as shown in FIG. 9C, for example, a pin B2 may be provided on one of the tips of the tools 8Cb and 8Cc, and a guide through-hole 8Cf for passing the pin B2 may be provided on the other. FIG. 9D is a side view of FIG. 9C. In this case, when the tools 8Cb and 8Cc are closed, the pin B2 fits into the through-hole 8Cf, preventing another workpiece 48 from being pinched between the tools 8Cb and 8Cc by the pin B2. In other words, because the shape of the tips of the tools 8Cb and 8Cc does not allow them to grip multiple workpieces 48, only one workpiece 48 can be gripped between the tools 8Cb and 8Cc.

[0061] Furthermore, as shown by arrow E1 in Fig. 9(E), the tips of the tools 8Db and 8Dc may be formed into a conical or triangular plate shape with a gradually narrowing width. In this case, the distance between the tools 8Db and 8Dc when one workpiece 48 is held by each of the tools 8Db and 8Dc is de1. In contrast, as shown by arrow E2 in Fig. 9(E), when two workpieces 48 are held between the tools 8Db and 8Dc, the distance de2 between the tools 8Db and 8Dc is larger than the distance de1. Therefore, the hand control unit 13F and the like can recognize that multiple workpieces 48 are held between the tools 8Db and 8Dc.

[0062] Furthermore, as shown by arrow E3 in FIG. 9E, notches 8Ed and 8Ee may be provided at the tips of tools 8Eb and 8Ec so that a single workpiece 48 can be accommodated. Even when using tool 8E, gripping multiple workpieces 48 can be prevented. As shown in FIG. 1C, for example, the actuator 32 may be composed of a frame 32b and an actuator main body 32a that opens and closes fingers 38A and 38B within the frame 32b. Furthermore, a load sensor 32d (e.g., a load cell, pressure sensor, or weight meter such as a strain gauge) may be provided between the frame 32b and the bottom surface of the actuator main body 32a, and a preload may be applied to the load sensor 32d from the top of the frame 32b using an elastic member 32c, such as a compression coil spring, to the actuator main body 32a. In this case, since the load detected by the load sensor 32d varies depending on the weight of the workpiece held by the tools 8b and 8c, the weight of the workpiece held by the tools 8b and 8c may be obtained from the detected value of the load sensor 32d. Note that the load sensor 32d may be provided, for example, at the tip of the tools 8b and 8c.

[0063] Second Embodiment A second embodiment will be described with reference to Figures 10 and 11. In Figures 10 and 11, parts corresponding to those in Figures 1(A), 4(A), and 4(B) are designated by the same reference numerals, and detailed descriptions thereof will be omitted. Figure 10 shows a robot device 2A according to this embodiment. In Figure 10, the robot device 2A includes a robot main body 4, a robot hand 6 joined to the robot main body 4, a control device 10 that controls the operation of the robot device 2A, and a control device 12 that controls the operation of the robot hand 6.

[0064] The robot hand 6 has a configuration similar to that of the robot hand 6 of FIG. 1A , except that the imaging device 22 is not provided on the side of the robot hand 6 of FIG. 10 . Specifically, tools 8b and 8c are fixed to fingers 38A and 38B of the robot hand 6, and the tools 8b and 8c are opened and closed by the fingers 38A and 38B, thereby gripping a workpiece 48 (here, a bolt) with the tools 8b and 8c. Furthermore, imaging devices 22A and 22B are installed on the ceiling or sidewall of the room in which the robot device 2A is installed, for capturing images of an area including the tip of the tool 8 held by the robot hand 6. While two imaging devices 22A and 22B are installed in FIG. 10 , three or more imaging devices may be installed. Furthermore, for example, if there is a position that can overlook the entire range in which the tips of the tools 8b and 8c of the robot hand 6 move, only one imaging device may be installed at that position.

[0065] As an example, the imaging devices 22A and 22B are ordinary cameras, and the focal positions of the imaging devices 22A and 22B are set, for example, near the placement surface FL of the workpiece 48, with a relatively wide focal depth. Note that a stereo camera may be used as the imaging devices 22A and 22B. The image processing unit 11D (see FIG. 3) of the control device 10 of this embodiment processes the image signals of the imaging devices 22A and 22B to calculate the amount of misalignment between the tools 8b and 8c and the workpiece 48. The remaining configuration is the same as that of the first embodiment. Marking members 62A and 62B are fixed to the upper surfaces of the tips of the tools 8b and 8c held by the robot hand 6, and marks MA and MB are formed on the marking members 62A and 62B.

[0066] In the robot apparatus 2A of this embodiment, when the tools 8b and 8c held by the robot hand 6 grip a workpiece 48 on a mounting surface FL, an image captured by an imaging device (here, imaging device 22B) capturing images of the workpiece 48 to be gripped and the tips of the tools 8b and 8c is used. FIG. 11 shows an example of an image 22BG captured by the imaging device 22B. In FIG. 11, the image 22BG includes an image 6I of the robot hand 6, tool images 8bI and 8cI, images 62AI and 62BI of mark members 62A and 62B at the tips of the images 8bI and 8cI, an image 48I of the workpiece at position P1 to be gripped, and an image 48I of another workpiece. At this time, as an example, the tips of the tools 8b and 8c are in contact with the mounting surface FL of the workpiece 48.

[0067] In this embodiment, the image processing unit 11D also determines the positions of the mark images MAI and MBI in the mark member images 62AI and 62BI on the coordinate system (x, y) of the imaging device 22B, and then determines the position of the grip center image 58CI of the tools 8b and 8c from these positions. Furthermore, the image processing unit 11D determines the position of the center 48cI of the image 48I of the workpiece to be gripped, and determines the distances δx and δy in the x and y directions between this center 48cI and the grip center image 58CI. Furthermore, the distances δx and δy are converted to distances in the robot coordinate system (X, Y, Z) by dividing them by the known magnification of the imaging device 22B. Then, by driving the robot hand 6 using the robot arms 18A-18C, etc., to offset the distance, the tools 8b and 8c can be moved to sandwich the workpiece 48, and the workpiece 48 can be gripped by the tools 8b and 8c in this state.

[0068] In this embodiment, the marks MA and MB on the marking members 62A and 62B on the tools 8b and 8c are detected, and the gripping centers of the tools 8b and 8c are determined from the detection results. This allows for efficient and highly accurate alignment of the workpiece 48 with the robot hand 6A (tools 8b and 8c). Furthermore, if the rotation angle between the workpiece image 48I and the tool 8 image is large, the robot hand 6A may be rotated to correct the rotation angle. Furthermore, in this embodiment, calibration between the coordinate system (x, y) of the imaging devices 22A and 22B and the robot coordinate system (X, Y, Z) may be performed in advance. In this case, calibration may be performed so that the x-axis and y-axis of the coordinate system (x, y) are parallel to the x-axis and y-axis of the robot coordinate system, respectively.

[0069] Next, in the above-described embodiment, tools 8b and 8c are fixed to fingers 38A and 38B. However, as shown in the modified robot hand 6A in FIG. 12(A), fingers 38E and 38F may be used to open and close movable portions 9b and 9c of tweezers 9 (tweezers-type tool) to grip a workpiece 48. Note that in FIGS. 12(A) to 12(C), parts corresponding to those in FIGS. 1 and 2(A) to 2(C) are designated by the same or similar reference numerals, and detailed descriptions thereof will be omitted. The tweezers 9 may be commercially available, custom-made, self-made, or a modified version of a commercially available product. The tweezers 9 have a pair of flexible, elongated, flat, movable portions (also referred to as gripping portions, contact portions, or opening / closing portions) 9b and 9c that are manufactured to be openable and closable about a fulcrum 9a. Then, by pressing inward with fingers 38E, 38F a point of force approximately midway between the fulcrum 9a of the tweezers 9 and the tips (points of action) of the movable parts 9b, 9c, the movable parts 9b, 9c close, and a workpiece 48 can be grasped or supported between the tips. As an example, the fulcrum 9a of the tweezers 9 is held by a fixed part 42 of the robot hand 6, which will be described later.

[0070] Fig. 12(A) shows a robot hand 6A of this modified example, in which the robot hand 6A is joined to the lower end of the tip portion 20A of a robot arm 18C so that the opening and closing directions of the movable portions 9b and 9c of the tweezers 9 held by the robot hand 6A are in the X direction. In Fig. 12(A), the robot hand 6A includes a hand main body 28 joined to the lower end of the tip portion 20A, a tweezers holding portion 26 connected to an actuating portion 32 of the hand main body 28 to hold the tweezers 9, and a pair of fingers 38E and 38F that are opened and closed in the X direction by the actuating portion 32. The movable portions 9b and 9c of the tweezers 9 held by the tweezers holding portion 26 are opened and closed in the X direction by the fingers 38E and 38F. In other words, the movable portions 9b and 9c of the tweezers 9 are opened and closed following the movement of the fingers 38E and 38F.

[0071] The hand main body 28 has the same configuration as in the first embodiment, with the imaging device 22 fixed to the frame 30 of the hand main body 28 via a support member 50. The tweezers holding unit 26 includes a connecting unit 38 connected to the -Y-direction side of the actuating unit 32 via a connecting unit 36, a rotating unit 40 fixed to the connecting unit 38 so as to be rotatable within a predetermined range (e.g., several degrees) around an axis parallel to the Z axis, and a fixed unit 42 fixed to the bottom surface of the rotating unit 40. The rotating unit 40 is provided with a stopper 54 for defining the rotation angle. The fixed unit 42 includes a fixing plate fixed to the bottom surface of the rotating unit 40 and a presser plate that clamps the fulcrum 9a of the tweezers 9 against the fixing plate. The presser plate is fixed to the fixing plate with multiple bolts B1, thereby stably holding the fulcrum 9a of the tweezers 9 on the fixing unit 42. At this time, the tips of the movable parts 9b and 9c of the tweezers 9 pass between the fingers 38E and 38F and protrude in the +Y direction of the fingers 38E and 38F. Note that the method of holding the tweezers 9 is arbitrary.

[0072] The imaging device 22 is positioned slightly diagonally above (in the +Y and +Z directions) the movable parts 9b and 9c of the tweezers 9. As shown in FIG. 12C , the images 9bI and 9cI of the movable parts 9b and 9c of the tweezers 9 are positioned so that they are located on the −y side of the origin 58B in the image 22G of the imaging device 22. Even in this modification, a suspension mechanism 46 (a buffer mechanism) is configured that includes the linear slider 34 and the elastic member 44 and supports the movable part (including the actuating part 32, fingers 38E and 38F, tweezers holding part 26, and tweezers 9) relative to the frame 30 so that the movable part can be elastically displaced in the Z direction. The detection result of the proximity sensor 56 can detect the relative displacement of the movable part (such as the actuating part 32 and tweezers 9) relative to the frame 30. Based on this detection result, it can be determined whether the tip of the tweezers 9 has contacted the mounting surface FL of the workpiece 48.

[0073] The connecting portion 36 and the connecting portion 38 are detachably connected, for example, by a permanent magnet or the like provided inside. The connecting portion 38 of the tweezers holding portion 26 is provided with a groove 38Aa. The tweezers holding portion 26 can be installed (stored) on a tool stand (not shown) by aligning the groove 38Aa with the base of the tool stand and installing the second connecting portion 38 on the base. In this case, multiple tweezers holding portions 26, each with a different shape of tweezers fixed to the fixing portion 42, may be prepared and installed on the tool stand. Then, by simply moving the robot hand 6 to detach and connect the connecting portion 36 and the connecting portion 40, the tweezers (tweezers holding portion 26) held by the robot hand 6 can be easily replaced with tweezers appropriate for the object.

[0074] 12(B) is a front view of Fig. 12(A) as viewed in the -Y direction, and in Fig. 12(B), a pair of fingers 38E, 38F opens and closes the movable parts 9b, 9c of the tweezers 9 in an opening / closing direction (here, the X direction) by the actuation unit 32. Furthermore, hemispherical protrusions 38Ea, 38Fa, for example, are provided on the inside of the fingers 38E, 38F, and the protrusions 38Ea, 38F come into contact with the force points (or points between the fulcrum 9a and the action points 9d, 9e) of the movable parts 9b, 9c of the tweezers 9 to push the movable parts 9b, 9c in the +X direction and the -X direction, thereby closing the movable parts 9b, 9c and allowing the tips of the movable parts 9b, 9c to grasp, pinch, or support an object. Thereafter, by moving the protrusions 38Ea, 38Fa in the −X direction and the +X direction, the movable parts 9b, 9c open, and the tweezers 9 can be removed from the object. Note that the actuating unit 32 may actuate the fingers 38E, 38F so that the fingers 38E, 38F move in the X direction independently of each other.

[0075] Furthermore, the protrusions 38Ea, 38Fa of the fingers 38E, 38F are formed, for example, from a material with low frictional resistance (such as Teflon (registered trademark)). This allows the tweezers 9 held by the tweezers holding portion 26 to move (slide) slightly in the Z and Y directions relative to the fingers 38E, 38F. In this embodiment, the suspension mechanism 46 displaces the actuating unit 32 and the tool including the tweezers holding portion 26 as a unit, so the protrusions 38Aa, 38Ba may be made of a material with high frictional resistance. It is not necessary to provide the protrusions 38Ea, 38F on the inside of the fingers 38E, 38F.

[0076] In this modification, marking members 62A and 62B, on which marks MA and MB similar to those in the example of FIG. 4A are formed, are attached to the tips of the movable parts 9b and 9c of the tweezers 9. Images of the marking members 62A and 62B are captured by the imaging device 22, and the gripping positions of the centers of the movable parts 9b and 9c can be determined using the captured images. Images 62AI and 62BI of the marking members and images MAI and MBI of the marks are formed in the image 22G of FIG. 12C. For example, the center positions of the images MAI and MBI (or positions shifted by a predetermined amount) can be regarded as the image of the gripping position. Then, by positioning the robot hand 6A so that the gripping position is aligned with the center of the workpiece 48 and closing the fingers 38E and 38F, the movable parts 9b and 9c of the tweezers 9 can grip the workpiece 48 efficiently and accurately. In this modified example, if, for example, part of the shape of the movable parts 9b and 9c of the tweezers 9 is distinctive, that distinctive part can be used instead of the marks MA and MB or the mark members 62A and 62B.

[0077] Next, the robot hands 6, 6A in the above-described embodiments can be connected to the robot arms of any robot device configuration, such as a horizontally articulated type (SCARA type), a parallel link type, or an orthogonal type, in addition to the vertically articulated type robot device 2. Furthermore, the robot hands 6, 6A are not limited to the above-described robot devices, but can also be applied to various robot devices (for example, assembly robots, human-collaborative robots, etc.) that have joint structures similar to other robot arms.

[0078] The present specification also describes the following aspects of the invention. 1) A robot device comprising: a robot hand provided with a plurality of finger portions; an arm portion that drives the robot hand; a plurality of gripping portions capable of gripping an object in response to the plurality of finger portions; at least one identification portion provided in at least one of the plurality of gripping portions; and a control portion that drives the robot hand via the arm portion using position information of the identification portion to position the plurality of gripping portions with respect to the object. 2) The robot device described in 1, wherein the plurality of gripping portions are connected to the plurality of finger portions so as to protrude in a direction intersecting a direction in which the plurality of finger portions open and close. 3) The robot device described in 1, wherein the plurality of gripping portions are a plurality of movable portions that are supported so as to protrude in a direction intersecting a direction in which the plurality of finger portions open and close, and are opened and closed by the opening and closing motion of the plurality of finger portions. 4) The robot device according to any one of 1 to 3, further comprising an imaging device that images the identifier, wherein the control unit determines position information of the identifier from an imaging result of the imaging device and determines position information related to gripping positions of the plurality of grippers from the position information. 5) The robot device according to 4, wherein an image of at least a part of the identifier and an image of at least a part of the object can be captured within the same field of view of the imaging device, and the control unit determines difference information between gripping positions of the plurality of grippers and a gripped position of the object using position information of at least a part of the image of the identifier and at least a part of the image of the object. 6) The robot device according to 4 or 5, wherein the plurality of grippers have two identifiers and can capture images of the two identifiers within the same field of view of the imaging device, and the control unit determines distance information between the two identifiers from position information of the images of the two identifiers, and uses the distance information to distinguish between a gripping state in which the object is gripped between the plurality of grippers and a non-gripped state in which the object is not gripped between the plurality of grippers.

[0079] 7) The robot device according to any one of claims 4 to 6, wherein the plurality of gripping units have two of the identifiers and are capable of capturing images of the two identifiers within the same field of view of the imaging device, and the control unit calculates distance information between the two identifiers from position information of the images of the two identifiers, and calculates the size of the gripped position of the object gripped between the plurality of gripping units using the distance information. 8) The robot device according to any one of claims 1 to 7, wherein the plurality of gripping units have a shape that can grip only one of the object. 9) The robot device according to any one of claims 1 to 8, wherein the plurality of finger units and the plurality of gripping units each have two units. 10) The robot device according to any one of claims 1 to 9, further comprising a weighing scale that calculates weight information of the object gripped by the plurality of gripping units. 11) The robot device according to any one of claims 1 to 10, wherein the identifier is a member on which an identification mark is formed. 12) The robot device according to claim 11, wherein the color of the member on which the identification mark is formed is different from the color of a placement surface on which the object is placed. 13) The robot device according to any one of 1 to 10, wherein the identifier is shaped as a part of the gripper. 14) A method for controlling a robot device including a robot hand provided with a plurality of finger portions and an arm portion that drives the robot hand, the method including: causing a plurality of grippers capable of gripping an object to follow the plurality of finger portions; storing position information of at least one identifier provided on at least one of the plurality of grippers; and driving the robot hand via the arm portion using the position information of the identifier to position the plurality of grippers with respect to the object. 15) The method for controlling a robot device according to 14, wherein the plurality of grippers are connected to the plurality of finger portions so as to protrude in a direction intersecting a direction in which the plurality of finger portions open and close. 16) The method for controlling a robot device according to 14, wherein the plurality of grippers are movable portions that are supported so as to protrude in a direction intersecting a direction in which the plurality of finger portions open and close, and are opened and closed by the opening and closing motion of the plurality of finger portions.

[0080] 17) The method for controlling a robot device according to any one of 14 to 16, comprising capturing an image of the identification unit with an imaging device, determining position information of the identification unit from the imaging result of the imaging device, and determining position information related to gripping positions of the plurality of gripping units from the position information. 18) The method for controlling a robot device according to 17, wherein capturing an image of the identification unit with the imaging device comprises capturing an image of at least a part of the identification unit and an image of at least a part of the object within the same field of view of the imaging device, and comprising determining difference information between the gripping positions of the plurality of gripping units and a gripped position of the object using position information of the image of at least a part of the identification unit and the image of at least a part of the object. 19) The method for controlling a robot device according to any one of 17 to 18, wherein the plurality of gripping units have two identifiers, and capturing images of the identifiers with the imaging device includes capturing images of the two identifiers within a same field of view of the imaging device, determining distance information between the two identifiers from positional information of the images of the two identifiers, and using the distance information to distinguish between a gripped state in which the object is gripped between the plurality of gripping units and a non-gripped state in which the object is not gripped between the plurality of gripping units. 20) The method for controlling a robot device according to any one of 17 to 19, wherein the plurality of gripping units have two identifiers, and capturing images of the identifiers with the imaging device includes capturing images of the two identifiers within a same field of view of the imaging device, determining distance information between the two identifiers from positional information of the images of the two identifiers, and using the distance information to determine a size of a gripped position of the object gripped between the plurality of gripping units. 21) A method for controlling a robot device according to any one of claims 14 to 20, wherein the plurality of gripping units are shaped so as to be able to grip only one of the objects. 22) A method for controlling a robot device according to any one of claims 14 to 21, wherein the plurality of finger units and the plurality of gripping units each comprise two units. 23) A method for controlling a robot device according to any one of claims 14 to 22, including determining weight information of the object gripped by the plurality of gripping units. 24) A method for controlling a robot device according to any one of claims 14 to 23, wherein the identification unit is a member on which an identification mark is formed.25) The method for controlling a robot device according to claim 24, wherein the color of the component on which the identification mark is formed is different from the color of the placement surface on which the object is placed. 26) The method for controlling a robot device according to any one of claims 14 to 23, wherein the identification part is shaped as a part of the gripping part.

[0081] 2, 2A...Robot device, 6, 6A...Robot hand, 8b, 8c...Tool, 9...Tweezers, 10...Control device of robot device, 12...Control device of robot hand, 18A to 18C...Robot arm, 16...Rotating unit, 22, 22A, 22B...Imaging device, 28...Hand main body, 32...Actuating unit, 38A, 38B...Fingers, 46...Suspension mechanism, 48...Work

Claims

1. A robot device comprising: a robot hand provided with a plurality of finger portions; an arm portion that drives the robot hand; a plurality of gripping portions capable of gripping an object in response to the plurality of finger portions; at least one identification portion possessed by at least one of the plurality of gripping portions; and a control portion that uses position information of the identification portion to drive the robot hand via the arm portion and position the plurality of gripping portions relative to the object.

2. A robot device according to claim 1, wherein the plurality of gripping portions are connected to the plurality of finger portions so as to protrude in a direction intersecting the direction in which the plurality of finger portions open and close.

3. A robot device as described in claim 1, wherein the multiple gripping portions are supported so as to protrude in a direction intersecting the direction in which the multiple finger portions open and close, and are multiple movable portions that are opened and closed by the opening and closing movements of the multiple finger portions.

4. A robot device as described in claim 1, further comprising an imaging device that images the identification part, wherein the control unit determines position information of the identification part from the imaging results of the imaging device, and determines position information regarding the gripping positions of the multiple gripping parts from the position information.

5. The robot device according to claim 4, wherein an image of at least a portion of the identification unit and an image of at least a portion of the object can be captured within the same field of view of the imaging device, and the control unit uses positional information of the image of at least a portion of the identification unit and the image of at least a portion of the object to determine differential information between the grasping positions of the multiple gripping units and the grasped position of the object.

6. A robot device as described in claim 4, wherein the multiple gripping parts have two of the identification parts, and images of the two identification parts can be captured within the same field of view of the imaging device, and the control part determines distance information of the two identification parts from position information of the images of the two identification parts, and uses the distance information to distinguish between a gripping state in which the object is gripped between the multiple gripping parts and a non-gripping state in which the object is not gripped between the multiple gripping parts.

7. A robot device as described in claim 4, wherein the multiple gripping sections have two of the identification sections, and images of the two identification sections can be captured within the same field of view of the imaging device, and the control section determines distance information between the two identification sections from positional information of the images of the two identification sections, and uses the distance information to determine the size of the gripped position of the object gripped between the multiple gripping sections.

8. The robot device according to claim 1, wherein each of the plurality of gripping parts is shaped so as to be able to grip only one of the objects.

9. The robot device according to claim 1, wherein the number of said finger portions and the number of said gripping portions are each two.

10. The robot device according to claim 1, further comprising a weight scale for determining weight information of the object grasped by the plurality of grasping portions.

11. The robot device according to any one of claims 1 to 10, wherein the identification part is a member on which an identification mark is formed.

12. The robot device according to claim 11, wherein the color of the member on which the identification mark is formed is different from the color of the placement surface on which the object is placed.

13. The robot device according to any one of claims 1 to 10, wherein the identification part is shaped as a part of the grip part.

14. A method for controlling a robot device having a robot hand provided with a plurality of finger portions and an arm portion that drives the robot hand, the method comprising: causing a plurality of gripping portions capable of gripping an object to follow the plurality of finger portions; storing position information of at least one identification portion possessed by at least one of the plurality of gripping portions; and driving the robot hand via the arm portion using the position information of the identification portion to position the plurality of gripping portions relative to the object.

15. A method for controlling a robot device as set forth in claim 14, wherein the plurality of gripping portions are connected to the plurality of finger portions so as to protrude in a direction intersecting the direction in which the plurality of finger portions open and close.

16. A method for controlling a robot device as described in claim 14, wherein the multiple gripping portions are supported so as to protrude in a direction intersecting the direction in which the multiple finger portions open and close, and are multiple movable portions that are opened and closed by the opening and closing movements of the multiple finger portions.

17. A method for controlling a robot device as described in claim 14, comprising: capturing an image of the identification part with an imaging device; determining position information of the identification part from the imaging results of the imaging device; and determining position information regarding the gripping positions of the multiple gripping parts from the position information.

18. A method for controlling a robot device as described in claim 17, wherein capturing an image of the identification part with the imaging device includes capturing an image of at least a portion of the identification part and an image of at least a portion of the object within the same field of view of the imaging device, and using positional information of the image of at least a portion of the identification part and the image of at least a portion of the object, determining difference information between the grasping positions of the multiple gripping parts and the grasped position of the object.

19. A method for controlling a robot device as described in claim 17, wherein the plurality of gripping parts have two of the identification parts, and capturing an image of the identification parts with the imaging device includes capturing images of the two identification parts within the same field of view of the imaging device, and includes determining distance information of the two identification parts from positional information of the images of the two identification parts, and using the distance information to distinguish between a gripping state in which the object is gripped between the plurality of gripping parts and a non-gripping state in which the object is not gripped between the plurality of gripping parts.

20. A method for controlling a robot device as described in claim 17, wherein the multiple gripping parts have two of the identification parts, and capturing an image of the identification parts with the imaging device includes capturing images of the two identification parts within the same field of view of the imaging device, and includes determining distance information between the two identification parts from positional information of the images of the two identification parts, and using the distance information to determine the size of the gripped position of the object gripped between the multiple gripping parts.

21. The method for controlling a robot device according to claim 14, wherein the plurality of gripping parts are shaped so as to be able to grip only one of the objects.

22. The method for controlling a robot device according to claim 14, wherein the number of said finger portions and the number of said gripping portions are each two.

23. The method for controlling a robot device according to claim 14, further comprising determining weight information of the object being gripped by the multiple grippers.

24. A method for controlling a robot device according to any one of claims 14 to 23, wherein the identification section is a member on which an identification mark is formed.

25. A method for controlling a robot device according to claim 24, wherein the color of the member on which the identification mark is formed is different from the color of the placement surface on which the object is placed.

26. A method for controlling a robot device according to any one of claims 14 to 23, wherein the identification part is shaped as a part of the grip part.

Images