Robot device, robot device control method, article manufacturing method, program, recording medium

Abstract

To achieve saving of time when performing movement control of a position of an object on the basis of an imaged image and a target image.SOLUTION: A robot device comprises: an end effector (100) having an imaging part for performing imaging, a body part, and a move part which can move an object with respect to the body part; a robot which movably supports a position of the end effector (100); and a control part which controls the imaging part, the end effector (100) and the robot. The control part can execute a first mode for moving a position of the object by controlling the robot and moving the position of the end effector (100) and a second mode for moving a position of an object (W) by controlling the move part when movement control for moving the position of the object (W) is executed on the basis of an image imaged by the imaging part and a target image.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present invention relates to a robot device, a control method for a robot device, a manufacturing method for an article, a program, and a recording medium. [Background technology]

[0002] In recent years, a robot device having a robot arm has been proposed that captures an image with a camera and corrects the position and posture of the robot arm using the image (see Patent Document 1). That is, in such a robot device, a reference target image is set, and the robot arm is controlled by so-called visual servo so that the captured image matches the set target image. This makes it possible to align the relative position between an end effector such as a robot hand mounted on the tip of the robot arm and a target object by correcting the position and posture of the robot arm. This makes it unnecessary to accurately position the target object in advance or to precisely teach the robot arm in advance, which makes it possible to reduce the cost of the robot device and shorten the teaching time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-89188 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the case of controlling a robot arm by visual servoing as in Patent Document 1, for example, the robot arm has many control axes, such as a six-axis articulated robot, and these many control axes are driven simultaneously to move the end effector toward the target direction. However, since many control axes are feedback-controlled simultaneously, there is a problem that it takes a long time for alignment to the target position to converge.

[0005] Therefore, an object of the present invention is to provide a robot device, a control method for a robot device, a manufacturing method for an article, a program, and a recording medium that are capable of shortening the time required when controlling the movement of an object's position based on a captured image and a target image. [Means for solving the problem]

[0006] One aspect of the present invention is a robot device comprising: an end effector having an imaging unit that captures images, a main body unit, and a moving unit that can move an object relative to the main body unit; a robot that movably supports a position of the end effector; and a control unit that controls the imaging unit, the end effector, and the robot, wherein the control unit, when executing movement control to move a position of the object based on an image captured by the imaging unit and a target image, is capable of executing a first mode in which the position of the object is moved by controlling the robot to move the position of the end effector, and a second mode in which the control unit is controlled to move the position of the object.

[0007] One aspect of the present invention is a control method for a robot device including an imaging unit that captures images, an end effector having a main body and a moving unit capable of moving an object relative to the main body, a robot that movably supports a position of the end effector, and a control unit that controls the imaging unit, the end effector, and the robot, characterized in that the control unit, when executing movement control to move the position of the object based on an image captured by the imaging unit and a target image, comprises a first step of controlling the robot to move the position of the end effector to move the position of the object, and a second step of controlling the moving unit to move the position of the object when executing the movement control. Effect of the Invention

[0008] According to the present invention, it is possible to reduce the time required for controlling the movement of the position of an object based on a captured image and a target image. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a robot system according to a first embodiment. [Diagram 2] FIG. 1 is a perspective view showing a robot hand according to a first embodiment. [Diagram 3] FIG. 2 is a bottom view showing the robot hand with a workpiece gripped by the fingers positioned within the field of view of the camera. [Figure 4] FIG. 2 is a bottom view showing the robot hand in a state where the workpiece gripped by the fingers is outside the field of view of the camera. [Diagram 5] FIG. 2 is a side view showing the configuration of the fingers of the robot hand. [Figure 6] 1A is a schematic diagram showing a state where a workpiece is gripped by the finger portions, FIG. 1B is a schematic diagram showing a state where a workpiece is gripped when the second finger portion is moved first, and FIG. 1C is a schematic diagram showing a state where a workpiece is gripped when the first finger portion is moved first. [Figure 7] FIG. 1 is a schematic diagram showing a configuration of a robot system according to a first embodiment and each coordinate system. [Figure 8] 13 is a flowchart showing workpiece movement control by visual servo control. [Figure 9] 1A and 1B are diagrams illustrating extraction of target image feature amounts in a target image used in visual servo control. [Figure 10] 5 is a flowchart showing workpiece movement control according to the first embodiment. [Figure 11] (a) is a diagram showing a captured image, (b) is a diagram showing a target image, (c) is a diagram showing the amount of movement of a workpiece under visual servo control, and (d) is a diagram showing a state in which the workpiece has been moved to a target position by moving the robot arm under visual servo control. [Figure 12]FIG. 1A is a diagram showing the movement direction of a workpiece and the movement amount in each direction in the workpiece movement control according to the first embodiment. FIG. 1B is a diagram showing a state in which the workpiece is moved in the X direction by the fingers of a robot hand in the workpiece movement control according to the first embodiment. FIG. 1C is a diagram showing a state in which the workpiece is moved in the X direction and Y direction by the fingers of a robot hand and a robot arm in the workpiece movement control according to the first embodiment. [Figure 13] 13A is a diagram showing the movement direction of a workpiece and the movement amount in each direction in the workpiece movement control according to the second embodiment, FIG. 13B is a diagram showing a state in which the workpiece is moved in the X direction by the fingers of a robot hand in the workpiece movement control according to the second embodiment, and FIG. 13C is a diagram showing a state in which the workpiece is moved in the X and Y directions by the fingers of a robot hand in the workpiece movement control according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] First Embodiment A first embodiment for carrying out the present invention will be described below with reference to FIGS. 1 to 12. FIG.

[0011] [Outline of the robot system] First, a schematic configuration of a robot system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the robot system according to the first embodiment.

[0012] 1, a robot system 1 installed in a factory or the like includes a robot device 10 that grasps and moves a pin W, which is a workpiece, from a pin placement table (not shown) on which a metallic pin W, for example, is placed, and inserts the pin W into a hole 402H in a workpiece 402 to be assembled. The robot device 10 includes a base 2, a robot arm 10A as a robot or robot main body supported by the base 2, and a robot hand 100 as an end effector attached to the tip of the robot arm 10A. The robot device 10 also includes a camera 120 as an imaging unit, a control device 200 that controls the robot arm 10A and the robot hand 100 (movement unit 130 described later), and the control device 200 is configured to be connectable to a controller 300.

[0013] The pin stand (not shown) is formed with a plurality of holes into which the pins W are inserted, and the pins W supplied as pre-operation parts are placed in these holes. The work 402 to be assembled is also formed with holes 402H into which the pins W as the work are inserted and fitted for assembly. That is, the robot device 10, which will be described later in detail, executes assembly control to perform the operation of assembling the pins W to the work 402 to be assembled, and the completed work 402 to be assembled is manufactured as an article by assembling a plurality of pins W to the work 402 to be assembled.

[0014] The robot arm 10A is a so-called six-axis articulated manipulator, and includes a pedestal 2, which is a base portion fixed to a workbench or a surface plate, and multiple links 11, 12, 13, 14, 15, and 16 that transmit displacement and force. The robot arm 10A also includes multiple joints J1, J2, J3, J4, J5, and J6 that connect the links 11 to 16 so that they can be turned or rotated. The link 16 disposed at the tip of the robot arm 10A is configured as a flange-shaped portion to which the robot hand 100 is attached.

[0015] The control device 200 is configured by a computer using a microprocessor element and the like, and can control the robot device 10. As shown in Fig. 1, the computer constituting the control device 200 is configured by, for example, a CPU 201 as a control unit, a ROM 202 storing programs for controlling each unit, a RAM 203, a communication interface (I / F in the figure) 204, and the like. Among these, the RAM 203 is used for temporarily storing data such as teaching points and control commands obtained by operating the controller 300. The programs for controlling each unit can be installed in the ROM 202 from a recording medium, and may be read from the recording medium into the ROM 202 via the communication interface 204.

[0016] The controller 300 may be an operating device such as a teaching pendant (TP), but may also be another computer device (PC or server) capable of editing a robot program. The controller 300 can be connected to the control device 200 via a wired or wireless communication connection means, and has a user interface function such as robot operation and status display. The CPU 201 receives, for example, teaching point data input by the controller 300 from the communication interface 204. In addition, the CPU 201 can generate trajectories of each axis (each link) of the robot device 10 based on the teaching point data input from the controller 300, and transmit the trajectories to the robot device 10 as control target values ​​via the communication interface 204. This allows the robot device 10 to operate the pin W, which is an operation target, by the robot hand 100 attached to the tip of the robot arm 10A.

[0017] [Structure of the robot hand] Next, the detailed configuration of the robot hand 100 will be described with reference to Figs. 2, 3, 4, 5, and 6. Fig. 2 is a perspective view showing the robot hand according to the first embodiment. Fig. 3 is a bottom view showing the robot hand in a state where the workpiece held by the fingers is located within the field of view of the camera. Fig. 4 is a bottom view showing the robot hand in a state where the workpiece held by the fingers is located outside the field of view of the camera. Fig. 5 is a side view showing the configuration of the fingers of the robot hand. Fig. 6(a) is a schematic diagram showing a state where the workpiece is held by the fingers. Fig. 6(b) is a schematic diagram showing a state where the workpiece is held when the second finger is moved first. Fig. 6(c) is a schematic diagram showing a state where the workpiece is held when the first finger is moved first.

[0018] As shown in FIG. 2, the robot hand 100 is detachably attached to the tip of the robot arm 10A (see FIG. 1). The robot hand 100 is roughly configured to include a base 101 as a main body, a camera 120, a finger portion 140 as a tool and a gripping portion, and a moving portion 130 that moves and drives the finger portion 140. The base 101 has an attachment portion 102 that is attached to a link 16 (see FIG. 1) of the robot arm 10A, and a camera 120 capable of capturing images is fixed and supported on the base 101. The camera 120 is moved so that the position of the robot hand 100 is a position in the sky at a predetermined distance away from the workpiece 402 to be assembled by controlling the robot arm 10A, and the position of the camera 120 is controlled so that the focal position P is a hole 402H of the workpiece 402 to be assembled. This enables the camera 120 to capture an image of the hole 402H, which is an image-capturing target, a work target, and a contact position.

[0019] In addition, the base 101 supports a first finger portion 141 and a second finger portion 142 that constitute a finger portion 140 capable of performing an operation on a pin W, which is a workpiece, in a freely movable manner. The fingertip portion 141b of the first finger portion 141 and the fingertip portion 142b of the second finger portion are configured to be able to abut against the pin W from each direction, and are able to support the pin W by pinching and holding the pin W between them. The fingertip portion 141b of the first finger portion 141 and the fingertip portion 142b of the second finger portion are located between the base 101 and the focal position P of the camera 120 while being within the field of view of the camera 120, as will be described in detail later. This allows the camera 120 to capture an image of an area farther away than the first finger portion 141 and the second finger portion 142 and perform various controls based on the image.

[0020] Next, the moving unit 130 that drives the finger unit 140 to move will be described. As described above, the finger unit 140 has a first finger unit 141 and a second finger unit 142 as shown in Figs. 2 and 3, and is configured to be able to grip a pin W that is a workpiece. The first finger unit 141 is fixed to a first slider 143 supported by the base unit 101, and the second finger unit 142 is fixed to a second slider 144 supported by the base unit 101. The moving unit 130 includes a first guide unit 170 and a second guide unit 180 that movably guide the first slider 143 and the second slider 144, a first drive unit 150 that drives the first slider 143, and a second drive unit 160 that drives the second slider 144.

[0021] In detail, the first guide part 170 as a sliding part has a first linear guide 171, and movable parts 172 and 173 slidably supported thereon. The second guide part 180 as a sliding part has a second linear guide 181, and movable parts 182 and 183 slidably supported thereon. The first slider 143 is fixed to the movable part 173 and the movable part 183, and the second slider 144 is fixed to the movable part 172 and the movable part 182.

[0022] On the other hand, the first driving unit 150 is roughly configured to include a motor 151, a driving pulley 152, a belt 153, a driven pulley 154, a ball screw shaft 155, a ball nut 156, and an angle detection sensor 159. That is, the motor 151 outputs a driving rotation, and the driving pulley 152 fixed to the output shaft of the motor 151 is rotated. A belt 153 is stretched between the driving pulley 152 and the driven pulley 154, and the rotation of the driving pulley 152 is transmitted to the driven pulley 154. The driven pulley 154 is fixed to one end of a ball screw shaft 155, and by rotating the ball screw shaft 155, a ball nut 156 fixed to the first slider 143 is slid in the axial direction via a ball (not shown). Therefore, the first slider 143 slides linearly in the axial direction of the ball screw shaft 155 (in the direction along the first linear guide 171 and the second linear guide 181) while being guided by the first guide portion 170 and the second guide portion 180. Therefore, the first finger portion 141 is driven to move in the axial direction of the ball screw shaft 155 according to the rotation direction of the motor 151. For example, when the motor 151 is rotated forward, the first finger portion 141 moves to one side toward the right in the figure, and when the motor 151 is rotated backward, the first finger portion 141 moves to the other side toward the left in the figure. In addition, the rotation angle of the motor 151 is detected by an angle detection sensor 159 constituted by, for example, an encoder, and is output to the control device 200. Based on this, the CPU 201 determines the position of the first finger portion 141 by calculating it, and also controls the position of the first finger portion 141 by issuing a command to drive the motor 151 based on the position.

[0023] Similarly, the second driving unit 160 is roughly configured to include a motor 161, a driving pulley 162, a belt 163, a driven pulley 164, a ball screw shaft 165, a ball nut 166, and an angle detection sensor 169. That is, the motor 161 outputs a driving rotation, and the driving pulley 162 fixed to the output shaft of the motor 161 is rotated. A belt 163 is stretched between the driving pulley 162 and the driven pulley 164, and the rotation of the driving pulley 162 is transmitted to the driven pulley 164. The driven pulley 164 is fixed to one end of a ball screw shaft 165, and by rotating the ball screw shaft 165, a ball nut 166 fixed to the second slider 144 is slid in the axial direction via a ball (not shown). Therefore, the second slider 144 linearly slides in the axial direction of the ball screw shaft 165 while being guided by the first guide portion 170 and the second guide portion 180. Therefore, the second finger portion 142 is driven to move in the axial direction of the ball screw shaft 165 according to the rotation direction of the motor 161. For example, when the motor 161 is rotated forward, the second finger portion 142 moves to one side toward the left in the figure, and when the motor 161 is rotated backward, the second finger portion 142 moves to the other side toward the right in the figure. In addition, the rotation angle of the motor 161 is detected by an angle detection sensor 169 constituted by, for example, an encoder, and is output to the control device 200. Based on this, the CPU 201 determines the position of the second finger portion 142 by calculating it, and also controls the position of the second finger portion 142 by instructing the motor 161 to drive based on the position.

[0024] Since the moving portion 130 is configured as described above, the first finger portion 141 and the second finger portion 142 can move independently on the same first linear motion guide 171 and second linear motion guide 181. This reduces deviations in the relative positions of the first finger portion 141 and the second finger portion 142 other than in the movement direction when the first finger portion 141 and the second finger portion 142 move while gripping the pin W, making it easier to control the movement of the fingers while gripping the pin W.

[0025] Furthermore, by placing the second linear guide 181 on the opposite side of the first linear guide 171 from the gripping position of the pin W, the moment force generated when placing the pin W8 can be supported by the two linear guides. This makes it possible to improve the durability of the first linear guide 171 and the second linear guide 181.

[0026] When the pin W is to be gripped between the first finger portion 141 and the second finger portion 142, first, the rotation angle of the motor 151 is detected by the angle detection sensor 159, and the rotation angle of the motor 161 is detected by the angle detection sensor 169. Then, based on these values, the CPU 201 controls the motor 151 and the motor 161 so that the first finger portion 141 and the second finger portion 142 are spaced apart from each other by the pin W and are positioned at the pin W. As a result, the pin W is gripped between the first finger portion 141 and the second finger portion 142.

[0027] Moreover, the first finger portion 141 and the second finger portion 142 can move in the same direction along the first linear guide 171 and the second linear guide 181 while maintaining a distance therebetween, that is, they can move while gripping the pin W. That is, the first finger portion 141 and the second finger portion 142 can be moved from the position shown in Fig. 3 to the position shown in Fig. 4. The state shown in Fig. 4 is a state in which the first finger portion 141, the second finger portion 142, and the pin W are in a retracted position outside the field of view of the camera 120 (hereinafter simply referred to as "outside the field of view").

[0028] Here, an operation in which the finger portion 140 gripping the pin W is moved by the moving unit 130 will be described. For example, as shown in FIG. 6(a), the first finger portion 141 abuts against one side of the pin W (in the direction of the arrow X1), and the second finger portion 142 abuts against the other side of the pin W (opposite the direction of the arrow X1), thereby pinching and gripping the pin W. From this state, the moving unit 130 slides the first finger portion 141 and the second finger portion 142 in the first guide portion 170 and the second guide portion 180 in the direction of the arrow X1, which is one side of the pin W. As described above, the first finger portion 141 and the second finger portion 142 can be independently driven to move. In this case, even if an instruction is given to move the first finger portion 141 and the second finger portion 142 simultaneously, there is a risk of an error in the transmission timing of the instruction, a drive error of the first drive unit 150 and the second drive unit 160, etc. occurring. Then, as shown in FIG. 6(b), there is a possibility that the second finger portion 142 moves first as indicated by an arrow M1, which may cause a gap d between the pin W and the second finger portion 142, causing the pin W to fall. Therefore, as shown in FIG. 6(c), when the pin W is moved to one side (the direction of the arrow X) by the finger portion 140 moved by the moving portion 130, the first finger portion 141 abutting on one side of the pin W is driven before the second finger portion 142, as indicated by an arrow M1. This causes the first finger portion 141 (or the second finger portion 142) to be slightly elastically deformed and bent, but the pin W can be maintained sandwiched between the first finger portion 141 and the second finger portion 142 without causing a gap d between the first finger portion 141 and the second finger portion 142.

[0029] Next, the shapes of the first finger portion 141 and the second finger portion 142 will be described with reference to Fig. 5. The second finger portion 142 has a second root portion 142a fixed to the second slider 144, a second fingertip portion 142b that contacts and grips the pin W, and a second connecting portion 142c that connects the second root portion 142a and the second fingertip portion 142b. Similarly, the first finger portion 141 has a first root portion 141a fixed to the first slider 143, a first fingertip portion 141b that contacts and grips the pin W, and a first connecting portion 141c that connects the first root portion 141a and the first fingertip portion 141b. In addition, since the shape of the first finger portion 141 and the shape of the second finger portion 142 are substantially the same, the shape of the second finger portion 142 will be described and a description of the first finger portion 141 will be omitted.

[0030] 5 with respect to the imaging direction (i.e., the optical axis direction) of camera 120, and is linearly slid as described above at a position that does not intersect with the imaging direction. Second connecting portion 142c is formed in a bent shape so that second fingertip portion 142b faces the imaging direction, that is, is formed so that second fingertip portion 142b can enter inside the field of view of camera 120 (hereinafter simply referred to as "within the field of view"). That is, second fingertip portion 142b moves linearly parallel to second root portion 142a by the sliding movement of second root portion 142a, and therefore moves from outside the field of view of camera 120 through the field of view to outside the field of view.

[0031] Since the first finger portion 141 and the second finger portion 142 are shaped as described above, the camera 120 can capture an image of the first fingertip portion 141b and the second fingertip portion 142b and the pin W being held. The image captured by the camera 120 does not capture the first connecting portion 141c, the second connecting portion 142c, the first root portion 141a, or the second root portion 142a other than the first fingertip portion 141b and the second fingertip portion 142b. This makes it possible to minimize the area where the first finger portion 141 and the second finger portion 142 overlap with the work position (the hole portion 402H of the workpiece 402 to be assembled, the hole portion 401H of the pin rest 401, the pin W placed on the pin rest 401, etc.) when captured by the camera 120. In addition, such a bent shape makes it possible to achieve a compact structure that can capture images of the first fingertip 141b and the second fingertip 142b without interfering with the camera 120 even when the first finger 141 and the second finger 142 are moved. This makes it possible to reduce the size of the robot hand 100.

[0032] [Visual servo control details] Next, visual servo control will be described with reference to Fig. 7, Fig. 8, and Fig. 9. Fig. 7 is a schematic diagram showing the configuration of a robot system according to a first embodiment and each coordinate system. Fig. 8 is a flowchart showing the movement control of a workpiece by visual servo control. Fig. 9 is a diagram for explaining extraction of target image feature amounts in a target image used in visual servo control.

[0033] In this embodiment, the term "visual servo" refers to an operation of moving the robot arm 10A based on a captured image and a target image to align an object to a target position. In addition, the term "visual servo control" refers to a series of controls for performing visual servo operations. That is, the visual servo control is a method for controlling the movement of the position and posture of the robot arm 10A, and is a control method in which a change in the position of a target object is measured as visual information and used as information for feedback control. In addition, in the visual servo control in this embodiment, an image feature included in an image of an object on a current image is extracted, and a control is performed in which a difference between the image feature on a target image is fed back. Specifically, the pin W held by the robot hand 100 is aligned with a hole 402H of a workpiece 402 to be assembled, which is an image capture target, or the position is corrected.

[0034] (Robot Systems and Coordinate Systems) Here, first, the robot system 1 for performing visual servo control and the coordinate system of each part in the robot system 1 will be described with reference to FIG. 7. As shown in FIG. 7, as described above, the robot system 1 (see FIG. 1) includes the robot device 10, the control device 200, and the controller 300 as an input device. The control device 200 is connected to a display 500 as a display unit. The robot device 10 also includes a servo control unit 210 that controls servos (not shown) in the joints J1 to J6 of the robot arm 10A. The camera 120 attached to the robot hand 100 is an example of an imaging unit, and is, for example, a digital camera. The camera 120 is a two-dimensional camera that can capture an image of a subject and obtain two-dimensional image information. The camera 120 is not limited to a two-dimensional camera, and may be, for example, a three-dimensional camera.

[0035] As described above, in this embodiment, the robot arm 10A is a vertically multi-jointed robot arm. The base end (fixed end) of the robot arm 10A is installed on the base 600. The robot hand 100 is attached to the tip (free end) of the robot arm 10A. The robot arm 10A has a base 2, a plurality of links 11 to 16, and a plurality of joints J1 to J6. The plurality of links 11 to 16 are connected in series in this order via the plurality of joints J1 to J6. From the base end side (link 11 side) of the robot arm 10A to the tip end side (link 16 side), the joints are designated as a first joint J1, a second joint J2, a third joint J3, a fourth joint J4, a fifth joint J5, and a sixth joint J6. The link 11, which is the base end of the robot arm 10A, is fixed to the base 2. The base 2 is fixed to the upper surface of the base 600. Each link 11 to 16 is rotationally driven (or may be extended and contracted) on the control axis of each joint J1 to J6, so that the robot arm 10A can adjust the robot hand 100 to any position in the three axial directions and to any posture in the three axial directions.

[0036] The robot hand 100 is attached to a link 16 which is the tip of the robot arm 10A. In other words, the link 16 is a support configured to support an end effector such as the robot hand 100. In other words, the position of the link 16 is the position of the robot hand 100. However, by driving the sixth joint J6, the posture of the link 16 and the robot hand 100 is changed around the control axis. In the visual servo described below, it is sufficient to only control the position of the robot hand 100 by driving and controlling the first joint J1 to the fifth joint J5, that is, the sixth joint J6 does not have to be driven by the visual servo.

[0037] The posture of the robot arm 10A can be expressed in a coordinate system. 0 , the coordinate system T of the robot hand 100 (end effector) e , the coordinate system T of the camera 120 c That is, the coordinate system T in FIG. 0is a coordinate system set on the stand 600 to which the robot arm 10A is fixed. e is a coordinate system set in the robot hand 100. e represents TCP (Tool Center Position). For example, in the coordinate system T e is set in the robot hand 100. The coordinate system T 0 and coordinate system T e is expressed in three-axis orthogonal coordinates consisting of the X, Y and Z axes.

[0038] Coordinate system T c is a coordinate system set at the center of the camera 120, and the coordinate system T o and coordinate system T e Similarly, the coordinate system T is expressed by a three-axis orthogonal coordinate system consisting of the X, Y and Z axes, and the optical axis direction of the camera 120 is set to be the Z axis direction. Note that in this embodiment, the camera 120 is fixed to the robot hand 100, so the coordinate system T c and coordinate system T c However, the camera is fixed to the ceiling of the factory where the robot system 1 is installed or to a predetermined position of the robot arm 10A, and the coordinate system T e and coordinate system T c may be in different coordinate systems.

[0039] The servo control unit 210 drives and controls servos (motors) (not shown) of the joints J1 to J6. The servo control unit 210 is disposed, for example, inside the pedestal 2. The servo control unit 210 is not limited to being disposed inside the pedestal 2, and may be disposed anywhere. For example, the servo control unit 210 may be disposed inside the housing of the control device 200. That is, the servo control unit 210 may be a part of the configuration of the control device 200. The servo control unit 210 drives and controls the servos of the joints J1 to J6 based on the command values ​​corresponding to the joints J1 to J6 acquired from the control device 200 so that the angles or torques of the joints J1 to J6 follow the command values. That is, the servo control unit 210 is configured to be able to perform position control or torque control of each joint of the robot arm 10A.

[0040] (Visual servo control) Next, the workpiece movement control by visual servo control will be described with reference to FIG. 8. The CPU 201 executes a program to execute the workpiece movement control by visual servo control shown in FIG. 8. The workpiece movement control by visual servo control shown in FIG. 8 may be executed as a part of the workpiece movement control shown in FIG. 10, which will be described later in detail, or may be executed selectively separately from the workpiece movement control shown in FIG. 10. That is, the workpiece movement control by visual servo control shown in FIG. 8 may be executed as a first mode, and the workpiece movement control including the movement of the finger parts shown in FIG. 10, which will be described later, may be executed as a second mode. The selection of these modes may be made manually by an operator, or may be automatically switched depending on the shape of the workpiece, the state of the robot device, and the like.

[0041] 8, when the movement control of the workpiece by visual servo control is started, first, the CPU 201 acquires the teaching data recorded in the RAM 203 or the like (S11). Here, the teaching data is a target image captured by the camera 120 when teaching the robot arm 10A an operation.

[0042] Next, the CPU 201 extracts target image feature amounts from the target image (S12). That is, as shown in Fig. 9, the CPU 201 extracts, for example, edges f111 and f121 of the workpieces W1 and W2 that are closest to each other, and predetermined portions f112 and f122 of the workpieces W1 and W2, respectively, from the target image. The predetermined portions are, for example, the end points of the edges f111 and f121 of the workpieces W1 and W2, respectively. The edges f111 and f121 and the predetermined portions f112 and f121 can be extracted by any known method.

[0043] For example, a line segment detection method using a Hough transform can be used to extract the edges f111 and f121. The predetermined portions f112 and f122 may be extracted based on the extraction results of the edges f111 and f121. More preferably, the edges f111 and f121 and the predetermined portions f112 and f122 are extracted using different methods. For example, a line segment detection method using a Hough transform is used to extract the edges f111 and f121, and template matching is used to extract the predetermined portions f112 and f122. Then, even if an incorrect image feature is extracted due to a failure in image processing, the error can be detected by comparing the extraction results of the edges f111 and f112 and the extraction results of the predetermined portions f121 and f122.

[0044] When template matching is used to extract the predetermined regions f112, f122, in step S12, values ​​input by the operator or values ​​recorded in ROM 202 or the like are registered as the positions and sizes of the predetermined regions f112, f122. The corresponding range of the target image may then be registered as a template. The registered template is temporarily stored in RAM 203 or the like.

[0045] The coordinate system Tc' is a coordinate system on the target image in which the perpendicular bisector of the edge f111 as an image feature is the X' axis and the direction along the edge f111 is the Y' axis. The target image feature amount is calculated using coordinates on the coordinate system Tc'. For example, the target image feature amount is composed of three values: a distance f131 of the intersection of the edges f111, f121 with the X' axis, a difference f132 in the Y' axis direction between the positions of the predetermined parts f112, f122, and an angle f133 formed by the edges f111, f121. In other words, the target image feature amount is calculated by a three-dimensional vector consisting of the distance f131, the position difference f132, and the angle f133 as the target image feature amount F g =[f131 f132 f133] T The subscript T denotes the transpose of a vector or matrix.

[0046] Next, the CPU 201 acquires a currently captured image (hereinafter, referred to as a "current image") from the camera 120 (S13). The current image is temporarily recorded in the RAM 203, for example.

[0047] Next, the CPU 201 extracts a current image feature from the current image in the same manner as in step S12, in which the target image feature is extracted from the target image (S14). Hereinafter, the components of the target image feature are represented as fg131, fg132, and fg133, respectively, and the components of the current image feature are represented as fc131, fc132, and fc133, respectively. In other words, the target image feature F g =[fg131 fg132 fg133] T , the current image feature F c =[fc131 fc132 fc133] T Let us assume that.

[0048] Next, the CPU 201 calculates the control amount q of the control axis of each joint J1 to J6 of the robot from the target image feature amount Fg and the current image feature amount Fc. v The control variable q is calculated and the robot arm 10A is operated by visual servo (S15). v To calculate the feature difference F between the current image feature Fc and the target image feature Fg, e (F is in bold) is calculated according to the following formula (1).

number

[0049] Next, CPU201 calculates the image Jacobian J img (J is in bold) matrix and the robot Jacobian J r (J is in bold) Calculate the image Jacobian J img is a 3-row, 3-column matrix that associates the small displacement amount of the coordinate system Te set in the robot hand 100 with the small displacement amount of the current image feature amount Fc. ris a 3-row, 6-column matrix that associates the minute displacement amount of each joint J1 to J6 of the robot arm 10A with the minute displacement amount of the coordinate system Te set in the robot hand 100. The image Jacobian J img and the robot Jacobian J r Define.

number

[0050] where x e (x in bold) is the three-degree-of-freedom position vector x of the coordinate system Te in the coordinate system To e =[X e Y e α e ] T and α e is the rotation angle around the Z axis of the coordinate system To. q (q is in bold) is the joint angle vector q=[q 1 …q 6 ] T It is.

[0051] Next, the CPU 201 calculates the control amount q of each of the joints J1 to J6 of the robot arm 10A. v (q is in bold) is calculated. Control quantity q v is calculated, for example, according to the following formula (3).

number

[0052] In equation (3), λ (λ is in bold) is a feedback gain expressed as a three-dimensional vector, and the subscript - indicates an inverse matrix, and + indicates a pseudo-inverse matrix. Note that the feature difference F e to the control quantity q v The method of calculating the control amount q is not limited to the above-mentioned method, and other known methods may be freely used. vThen, the CPU 201 performs visual servo operation by operating the robot arm 10A via the servo control unit 210 based on the angle command value.

[0053] Then, the CPU 201 judges whether the position correction operation by visual servo is completed or not (S16). e If the characteristic amount difference F is equal to or smaller than the predetermined value, the correction operation is deemed to be complete and the operation is terminated (Yes in S16). e If it is not equal to or less than the predetermined value (No in S16), the process returns to step S13 and repeats the above process. Here, the predetermined value is a value that can be determined with sufficient accuracy to ensure that the subsequent assembly work can be performed under force control (torque control). In addition, when determining that it is equal to or less than the predetermined value, the same value is used to calculate the feature amount difference F e It is possible to perform a judgment for all values ​​included in the condition, or to perform judgment for each different value and continue the correction operation until all values ​​satisfy the condition. In this way, it is possible to operate the robot arm 10A so that the current image and the target image match.

[0054] [Workpiece movement control according to the first embodiment] Next, the movement control of the pin W as the workpiece according to the first embodiment will be described with reference to Figs. 10, 11, and 12. Fig. 10 is a flowchart showing the movement control of the workpiece according to the first embodiment. Fig. 11(a) is a diagram showing a captured image. Fig. 11(b) is a diagram showing a target image. Fig. 11(c) is a diagram showing the movement amount of the workpiece in visual servo control. Fig. 11(d) is a diagram showing a state in which the robot arm is moved by visual servo control to move the workpiece to the target position. Fig. 12(a) is a diagram showing the movement direction of the workpiece and the movement amount in each direction in the movement control of the workpiece according to the first embodiment. Fig. 12(b) is a diagram showing a state in which the workpiece is moved in the X direction by the fingers of the robot hand in the movement control of the workpiece according to the first embodiment. Fig. 12(c) is a diagram showing a state in which the workpiece is moved in the X direction and the Y direction by the fingers of the robot hand and the robot arm in the movement control of the workpiece according to the first embodiment.

[0055] In the movement control of the pin W according to the first embodiment, a control is described in which the pin W is moved from a state where the pin W is held by the finger portion 140 to a hole portion 402H of the workpiece 402 to be assembled as a work position, so as to be aligned with the hole portion 402H of the workpiece 402 to be assembled. That is, before this movement control, a control is executed in which the pin W is held by the finger portion 140 from a pin placement table (not shown), the robot hand 100 is moved by the robot arm 10A, and the pin W is roughly positioned above the hole portion 402H of the workpiece 402 to be assembled. After this movement control, a control is executed in which the robot arm 10A is torque-controlled to move the robot hand 100, so that the pin W is inserted into the hole portion 402H of the workpiece 402 to be assembled. By making the robot device 10 perform the work so as to repeat these controls, a plurality of pins W are assembled into the workpiece 402 to be assembled, and the product is manufactured. That is, in the following description of the movement control of the pins W, only the operation of aligning one pin W directly above one hole 402H will be described.

[0056] As shown in FIG. 10, when the CPU 201 of the control device 200 starts the movement control of the pin W held by the first finger portion 141 and the second finger portion 142, first, the CPU 201 reads a target image 902 stored in the ROM 202 or the like (S21). As shown in FIG. 11(b), the target image 902 is an image including a target feature amount 912 as a target position of the pin W. In addition, the CPU 201 captures an image of the current pin W by the camera 120 and acquires the image as a current image 901 (S22). As shown in FIG. 11(a), the current image 901 is an image including a current feature amount 911 as a current position of the pin W held by the robot hand 100. Note that the finger portions 140 are omitted in the current image 901 shown in FIG. 11(a). In addition, in the target image 902 shown in FIG. 11B, the target feature amount 912 may be the feature amount (image) of the hole 402H of the workpiece 402, instead of the feature amount (image) of the pin W as the target position.

[0057] Next, the CPU 201 extracts features (target feature 912 and current feature 911) from the target image 902 and the current image 901, respectively, in the same manner as in the above-mentioned visual servo control (S23). Then, the CPU 201 calculates a control amount as a target movement amount required for moving the position of the robot arm 10A or for moving the position of the finger portion 140 of the robot hand 100, from the difference between the target feature 912 and the current feature 911 (S24). That is, as shown in Fig. 11(c), in a composite image 903 obtained by combining the current image 901 and the target image 902, a control amount 920 as a target movement amount, which is the difference between the target feature 912 and the current feature 911, is calculated.

[0058] Here, as shown in FIG. 10(d), when the pin W currently located at the position of the feature amount 911 is moved to the position of the target feature amount 912 by the control amount 920, the pin W is moved only by the operation of the robot arm 10A (only the movement of the position of the robot hand 100). In this case, for example, since the robot arm 10A is a six-axis articulated robot, linear movement must be realized while controlling the multiple control axes in accurate synchronization. However, in order to accurately control the multiple control axes so that, for example, an error of six axes is reduced, it is necessary to slow down the driving speed of the robot arm 10A. Furthermore, since, for example, an error of six axes occurs even if it is slight, there is a problem that it takes time to converge the alignment by feedback control in visual servo control when aligning the pin W to the position of the target feature amount 912.

[0059] Therefore, in the movement control of the pin W according to this embodiment, a first mode (first step) and a second mode (second step) are executable as follows. That is, the first mode is a mode in which the position of the pin W (target object) is moved by controlling the robot arm 10A to move the position of the robot hand 100. Also, the second mode is a mode in which the movement unit 130 of the robot hand 100 is controlled to move the position of the pin W (target object) on a straight line by the above-mentioned first guide unit 170 and second guide unit 180. In the movement control of the pin W according to this embodiment, based on the control amount 920 calculated as described above (i.e., based on the target image and the current image), it is determined whether to execute only the first mode, execute only the second mode, or execute both the first mode and the second mode, and transition to the execution of each of them.

[0060] Roughly speaking, if the control amount 920 (i.e., the difference between the current position of the pin W and the target position) is a small value equal to or less than a specified value, the visual servo of the robot arm 10A does not take much time, so only the first mode is executed. If the control amount 920 is a value greater than the specified value and the movement direction of the pin W in the control amount 920 includes a component of the movement direction of the finger unit 140 by the moving unit 130, both the second mode and the first mode are executed while giving priority to the second mode. That is, the necessary control amount (movement amount of the pin W) is divided into the control amount of the finger unit 140 of the robot hand 100 and other control amounts, and the operation is allocated to the robot hand 100 with fewer control axes, thereby reducing the operation of the robot arm 10A. Note that in this case, if the movement direction of the pin W in the control amount 920 is only the movement direction of the finger unit 140 by the moving unit 130, then inevitably only the second mode is executed.

[0061] In detail, first, the CPU 201 converts the control amount 920 obtained as described above into a control amount in an arbitrary coordinate reference. In general, the control amount is managed in the coordinate reference of the robot, and the starting point is often set at a TCP (Tool Center Point). Then, as shown in FIG. 12(a), the CPU 201 divides the control amount 920 into a first control amount 920x and a second control amount 920y viewed from an arbitrary coordinate reference. That is, the first control amount 920x as the first movement amount is a control amount of a movement direction (e.g., X direction) as a first direction in which the finger unit 140 can move. The second control amount 920y as the second movement amount is a control amount of a direction other than the first direction and a second direction (e.g., Y direction) different from the first direction.

[0062] Next, it is determined whether or not the first control amount 920x of the movement direction of the finger 140 divided as described above can be realized by the movement (movement) of the finger 140 (the first finger 141 and the second finger 142), that is, whether or not the second mode is executable (S26). For example, when the movable amount from the current position of the pin W to the movable range end of the finger 140 is smaller than the first control amount 920x, it is determined that the first control amount 920x cannot be realized by the movement of the finger 140 (No in S26). In other words, if the first control amount 920x is larger than the movable amount of the finger 140, it is determined that the first control amount 920x cannot be realized by the movement of the finger 140. In this case, the CPU 201 determines that the second mode is not executable and selects execution of only the first mode, that is, performs visual servoing by only driving the robot arm 10A (S27). As a result, the pin W is moved by visual servoing by the robot arm 10A together with the robot hand 100, and is positioned at the target position (see Fig. 10(c) and Fig. 10(d)). Then, the visual servoing is continued until the difference between the target image (target feature amount 912) and the current image (current feature amount 911) becomes equal to or less than a certain value (No in S28), and when the difference between the target image and the current image becomes equal to or less than the certain value (Yes in S28), the movement control of the pin W is terminated.

[0063] On the other hand, when the CPU 201 determines that the first control amount 920x can be realized by the movement of the finger unit 140 (Yes in S26), it determines whether or not the first control amount 920x (control amount of the finger unit) that controls the finger unit 140 is equal to or less than a specified value (S29). When the first control amount 920x of the finger unit is equal to or less than the specified value, that is, the movement amount of the pin W is small, it does not take much time to perform visual servo of the robot arm 10A, so the process proceeds to step S27. In this case, the CPU 201 also selects execution of only the first mode, that is, performs visual servo only by driving the robot arm 10A (S27). As a result, the pin W is moved by visual servo by the robot arm 10A until the difference between the target image and the current image becomes equal to or less than a certain value (No in S28), and when the difference between the target image and the current image becomes equal to or less than a certain value (Yes in S28), the movement control of the pin W is terminated.

[0064] Then, when the CPU 201 determines that the first control amount 920x can be realized by the movement of the finger 140 (Yes in S26) and further determines that the first control amount 920x is not equal to or less than the specified value (No in S29), the process proceeds to step S30. Then, as shown in FIG. 11(a), the CPU 201 executes the second mode for the first control amount 920x in the movement direction of the finger 140, which is one of the control amounts obtained by dividing the control amount 920. That is, as shown in FIG. 11(b), the moving unit 130 of the robot hand 100 moves the finger 140 by the first control amount 920x, and the position of the pin W moves in the first direction by the first control amount 920x. At this time, when the pin W is moved by the finger 140, the first finger 141 is driven before the second finger 142 as described above (see FIG. 6(c)). In the second mode, visual servoing is not performed on the movement of the finger portion 140, but a drive command corresponding to the first control amount 920x is given to the moving portion 130 to move the position of the pin W. Note that visual servoing may be performed on the movement of the finger portion 140.

[0065] Further, the CPU 201 executes the first mode for the remaining second control amount 920y obtained by dividing the control amount 920. That is, as shown in FIG. 11(c), the finger unit 140 is moved by the second control amount 920y together with the robot hand 100 by the robot arm 10A, and the position of the pin W is moved in the second direction by the second control amount 920y. Note that even in the first mode at this time, visual servo is not performed for the movement of the robot arm 10A, but a drive command corresponding to the second control amount 920y is given to the servo control unit 210 to move the position of the pin W. Of course, visual servo may be performed for the movement of the robot arm 10A. Further, although the control amount of the robot arm 10A is not mentioned here, for example, when the control amount of the robot arm 10A is equal to or less than a preset value, only the finger unit 140 may be operated by the moving unit 130, and the operation of the robot arm 10A may be omitted.

[0066] Then, it is determined whether the difference between the target image and the current image is equal to or less than a certain value (S28). Here, since the movement of the pin W should be completed for the control amount 920, the difference between the target image and the current image should be equal to or less than a certain value (Yes in S28). If the difference between the target image and the current image is not equal to or less than a certain value (No in S28), the pin W is moved by the robot arm 10A using visual servo until the difference between the target image and the current image is equal to or less than a certain value. Then, the movement control of the pin W is terminated.

[0067] [Summary of the first embodiment] As described above, in the first embodiment, the position of the pin W is controlled and moved based on the captured current image and the target image. In this case, a first mode in which the pin W is moved by driving the robot arm 10A and a second mode in which the pin W is moved by driving the finger unit 140 by the moving unit 130 can be executed. As a result, the second mode can bear part or all of the burden that occurs when accurately driving the multiple control axes of the robot arm 10A, and the time required for movement control to move the position of the pin W can be shortened.

[0068] In addition, a control amount 920 for moving the position of the pin W is calculated based on the current image and the target image, and the control amount 920 is divided and calculated into a first control amount 920x in a first direction in which the pin W can be moved by the moving unit 130, and a second control amount 920y in a second direction. This makes it possible to determine whether the first control amount 920x can be achieved by the movement of the finger unit 140.

[0069] Also, for example, the workpiece 402 is provided with a plurality of holes 402H (see FIG. 1) into which the pins W are to be attached, each of which is a known position. Then, for example, after the above-mentioned pin W movement control is performed for the first pin W, the second pin W may be moved to the hole 402H in a known position relative to the hole 402H into which the first pin W is inserted. In this case, in step S24, a target movement amount for moving the position of the pin W (i.e., a movement direction and movement amount to the hole 402H into which the second pin W is to be inserted) can be calculated based on the current image, the target image, and the target position that is a known position relative to the target image. In particular, the hole 402H into which the second pin W is to be inserted may be arranged in line with the movement direction of the finger 140 relative to the hole 402H into which the first pin W is inserted. In this case, by executing the second mode, the second pin W can be easily aligned, and time can be reduced.

[0070] In the above-described movement control of the pin W, visual servoing by the robot arm 10A is not performed in step S30. However, the present invention is not limited to this. For example, after only the movement of the finger unit 140 by the moving unit 130 is performed in step 30, the process may proceed to step 27 to perform visual servoing by the robot arm 10A. In this case, it is possible to calculate the remaining control amount by generating a new target image excluding the first control amount 920x of the finger unit 140, and to perform visual servoing by the robot arm 10A for the remaining control amount.

[0071] In addition, in the above-described movement control of the pin W, if the first control amount 920x is larger than the movable amount of the finger portion 140 in step S26, it is determined that the second mode is not executable. However, this is not limited to the above, and it may be determined to execute the second mode even if the first control amount 920x is larger than the movable amount of the finger portion 140. That is, it may be possible to move the pin W by the movable amount of the finger portion 140 in the first control amount 920x by executing the second mode, and to move the pin W by the remaining control amount by executing the first mode. In particular, in this case, if the divided first control amount 920x (first movement amount) is larger than the second control amount 920y (second movement amount), it is possible to reduce the control amount (movement amount) in the first mode and shorten the time by determining to execute the second mode.

[0072] <Second embodiment> Next, a second embodiment, which is a partial modification of the first embodiment, will be described with reference to Fig. 13. Fig. 13(a) is a diagram showing the movement direction of a workpiece and the movement amount in each direction in the workpiece movement control according to the second embodiment. Fig. 13(b) is a diagram showing a state in which a workpiece is moved in the X direction by the fingers of a robot hand in the workpiece movement control according to the second embodiment. Fig. 13(c) is a diagram showing a state in which a workpiece is moved in the X direction and Y direction by the fingers of a robot hand in the workpiece movement control according to the second embodiment.

[0073] In the second embodiment, compared to the first embodiment, the moving part moves the finger part in two directions (two coordinate directions). In other words, the second mode allows the position of the pin W to be moved on a plane including the X direction and the Y direction.

[0074] 13(a), in the robot hand 100 according to the second embodiment, a finger base portion 1143 is attached by a moving unit (not shown) so as to be movable in the X direction, which is a first direction. Also, the first finger portion 1141 and the second finger portion 1142 are attached by a moving unit (not shown) so as to be movable in the Y direction relative to the finger base portion 1143. Therefore, in the robot hand 100 according to the second embodiment, the pin W held by the first finger portion 1141 and the second finger portion 1142 is configured to be movable on a plane in the X direction and the Y direction, that is, in the second mode, the pin W can be moved on a plane.

[0075] In the robot hand 100 thus configured, when performing the movement control of the pin W (see FIG. 10), as shown in FIG. 13(a), first, the CPU 201 calculates the control amount 920 based on the target image and the current image (S24). Next, the control amount is divided into a first control amount 920x in the X direction and a second control amount 920y in the Y direction (S25), where both directions are the movement directions of the finger parts. Therefore, it is determined that the control amount of the movement direction of the finger parts can be realized by the movement of the finger parts, and that the control amount of the finger parts is equal to or less than a specified value, and therefore the second mode is executable (Yes in S29). In this case, as shown in FIG. 13(b), the pin W is moved by the first control amount 920x by moving the finger base part 1143. Also, as shown in FIG. 13(c), the pin W is moved by the second control amount 920y by moving the first finger part 1141 and the second finger part 1142 (S30). That is, in the second embodiment, it is not necessary for the robot arm 10A to perform the operation for the remaining control amount, that is, the first mode is not executed and the pin W is moved only in the second mode. In other words, by eliminating the need to move the pin W by the robot arm 10A, the entire burden that occurs when accurately driving the multiple control axes of the robot arm 10A can be borne by the second mode, and the time required for movement control to move the position of the pin W can be shortened.

[0076] Other configurations, operations, and effects of the second embodiment are similar to those of the first embodiment, and therefore description thereof will be omitted.

[0077] <Possibilities for other embodiments> In the above-described first and second embodiments, the position of the pin W as the workpiece is moved and controlled, but the target is not limited to the workpiece. For example, before gripping the pin as the workpiece, the tip of the finger portion (first finger portion and second finger portion) is the target and its movement is controlled. In this case, the moving portion of the end effector moves the position of the finger portion as the second mode, and the position of the finger portion is moved by the robot together with the end effector as the first mode. Furthermore, if the end effector is equipped with a tool such as a screwdriver or tweezers, the tip of the tool may be the target and its position may be moved and controlled. In this case, the moving portion of the end effector moves the position of the tool as the second mode, and the position of the tool is moved by the robot together with the end effector as the first mode.

[0078] In the first and second embodiments, the movement amount (control amount) of the pin is calculated based on a current image of the pin W and a target image as a target position of the pin W. However, the present invention is not limited to this, and the movement amount (control amount) of the pin may be calculated based on, for example, a current image of the hole 402H of the workpiece 402 to be assembled, a target image as a target position of the hole 402H, and the position of the pin W (finger) in the end effector. In addition, the present image and the target image are described as using images in the field of view of the camera 120 fixed and supported on the robot hand 100 as the end effector, but the present invention is not limited to this. That is, the camera may be supported at a predetermined position of the robot arm 10A, or may be supported on the ceiling of a factory or the like in which the robot system 1 is installed. That is, the coordinate system may be converted based on the positions of these cameras to calculate the movement direction of the target object, and the movement amount (control amount) may be calculated.

[0079] In the first embodiment, the robot hand 100 is described as sliding the first finger portion 141 and the second finger portion 142, but the present invention is not limited to this configuration. For example, the robot hand 100 may have three or more fingers, or the fingers may move on an arc instead of a straight line. In addition, the robot hand 100 is described as having the first guide portion 170 and the second guide portion 180 so that the first finger portion 141 and the second finger portion 142 slide, but the present invention is not limited to this, and may have one guide portion or three or more guide portions. Of course, the presence of multiple guide portions stabilizes the movement of the first finger portion 141 and the second finger portion 142, and the gripping of the workpiece is also stabilized, so it is preferable to have two or more guide portions. In addition, the robot hand 100 is described as driving the first finger portion 141 and the second finger portion 142 with the motor 151 and the motor 161, but the present invention is not limited to this, and may be driven by, for example, a solenoid or hydraulic pressure. Also, in the second embodiment, the number of fingers may be three or more.

[0080] In the first and second embodiments, the robot arm 10A is driven and controlled in the first mode, and the moving unit 130 is driven and controlled in the second mode. However, the present invention is not limited to this, and the first joint J1 to the fifth joint J5 of the robot arm 10A may be driven and controlled in the first mode, and the moving unit 130 and the sixth joint J6 may be driven and controlled in the second mode. In this case, in the second mode, the X direction, which is the moving direction of the finger unit 140, can be rotated around the sixth joint J6, and in particular, even with the structure of the robot hand 100 as in the first embodiment, the finger unit 140 (pin W) can be moved on a plane.

[0081] In addition, the present disclosure is not limited to the above-described embodiments, and the embodiments can be modified in many ways within the technical concept of the present disclosure. For example, at least two of the above-described embodiments and modifications may be combined. In addition, the effects described in the present embodiment are merely a list of the most preferable effects resulting from the embodiments of the present disclosure, and the effects of the embodiments of the present disclosure are not limited to those described in the present embodiment.

[0082] In the above-mentioned first and second embodiments, the robot body is a vertically articulated robot, but the present invention is not limited to this. The robot body may be, for example, a horizontally articulated robot, a parallel link robot, or an orthogonal robot. The above-mentioned embodiments are applicable to a machine that can automatically perform the actions of expanding and contracting, bending and stretching, moving up and down, moving left and right, or turning, or a combination of these actions, based on information in a storage device provided in a control device.

[0083] The present disclosure can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions.

[0084] <Disclosure of the Present Embodiment> [Configuration 1] An imaging unit that captures an image; An end effector having a main body and a moving part capable of moving an object relative to the main body; a robot that movably supports the position of the end effector; a control unit that controls the imaging unit, the end effector, and the robot, The control unit is When performing movement control to move the position of the object based on the image captured by the imaging unit and a target image, A first mode in which the robot is controlled to move the position of the end effector, thereby moving the position of the object, and a second mode in which the moving unit is controlled to move the position of the object, are executable. A robot device comprising: [Configuration 2] The control unit is Calculating a target movement amount for moving the position of the object based on the image captured by the imaging unit and a target image; The target movement amount is calculated by dividing the target movement amount into a first movement amount in a first direction in which the moving unit can move and a second movement amount in a second direction different from the first direction; when it is determined to execute the second mode based on the first movement amount, the position of the object is moved by the first movement amount in the second mode and moved by the second movement amount in the first mode. 2. The robot device according to configuration 1. [Configuration 3] The control unit is calculating a target movement amount for moving the position of the object based on the image captured by the imaging unit, a target image, and a target position that is a known position with respect to the target image; calculating a first movement amount in a first direction in which the moving unit can move based on the target movement amount, and calculating a second movement amount in a second direction different from the first direction based on the target movement amount and the first movement amount; when it is determined to execute the second mode based on the first movement amount, the position of the object is moved by the first movement amount in the second mode and moved by the second movement amount in the first mode. 2. The robot device according to configuration 1. [Configuration 4] the control unit determines to execute the second mode when the first movement amount is greater than the second movement amount. 4. The robot device according to configuration 2 or 3. [Configuration 5] The control unit determines to execute the second mode when the first movement amount is greater than a preset specified value. 5. The robot device according to any one of configurations 2 to 4. [Configuration 6] when determining that the second mode is impracticable based on the first movement amount, the control unit moves the position of the object in the first mode. 6. The robot device according to any one of configurations 2 to 5. [Configuration 7] when the position of the object is moved in the first mode, the control unit executes visual servoing based on the image captured by the imaging unit and a target image. 7. The robot device according to any one of configurations 1 to 6. [Configuration 8] the control unit executes visual servoing until a difference between an image captured by the imaging unit and a target image becomes equal to or smaller than a certain value. 8. The robot device according to configuration 7. [Configuration 9] The control unit is capable of calculating a coordinate system of the end effector, The moving unit is capable of moving the object in at least one coordinate direction in a coordinate system of the end effector. 9. The robot device according to any one of configurations 1 to 8. [Configuration 10] The moving unit is capable of moving the object in a straight line relative to the main body unit. 10. The robot device according to configuration 9. [Configuration 11] The moving unit is capable of moving the object on a plane relative to the main body unit. 10. The robot device according to configuration 9. [Configuration 12] The end effector has a gripping portion that grips and supports a workpiece, The object is a workpiece, The moving unit is capable of moving the gripping unit. 12. The robot device according to configuration 10 or 11. [Configuration 13] The gripping portion includes a first finger portion capable of contacting one side of the workpiece and a second finger portion capable of contacting the other side of the workpiece, and the first finger portion and the second finger portion sandwich and grip the workpiece. 13. The robot device according to configuration 12. [Configuration 14] The moving unit has a slide unit that supports the first finger unit and the second finger unit so as to be slidable, a first driving unit that drives the first finger unit to slide on the slide unit, and a second driving unit that drives the second finger unit to slide on the slide unit. 14. The robot device according to configuration 13. [Configuration 15] When the control unit slides the first finger unit and the second finger unit on the slide unit, when the workpiece is moved to one side, the control unit drives the first finger unit before the second finger unit, and when the workpiece is moved to the other side, the control unit drives the second finger unit before the first finger unit. 15. The robot device according to configuration 14. [Configuration 16] The imaging unit is supported by the main body. 16. The robot device according to any one of configurations 1 to 15. [Method 17] A control method for a robot device including an imaging unit that captures images, an end effector having a main body and a moving unit that can move an object relative to the main body, a robot that movably supports a position of the end effector, and a control unit that controls the imaging unit, the end effector, and the robot, comprising: The control unit: a first step of controlling the robot to move a position of the end effector to move the position of the object when executing movement control for moving a position of the object based on an image captured by the imaging unit and a target image, and a second step of controlling the moving unit to move the position of the object when executing the movement control. A method for controlling a robot device comprising: [Method 18] 17. A method for manufacturing an article, comprising the steps of: manufacturing an article by using a robot device according to any one of configurations 1 to 16. [Configuration 19] A program for causing a computer to execute the control method for a robot device according to Method 17. [Configuration 20] A computer-readable recording medium storing the program according to configuration 19. [Explanation of symbols]

[0085] 10...Robot device / 10A...Robot arm (robot) / 100...Robot hand (end effector) / 101...Base (main body) / 120...Camera (imaging unit) / 130...Moving unit / 140...Finger (gripping unit) / 141...First finger / 142...Second finger / 150...First driving unit / 160...Second driving unit / 170...First guide (slide unit) / 180...Second guide (slide unit) / 201...CPU (control unit) / 901...Current image (captured image) / 902...Target image / 920...Control amount (target movement amount) / 920x...First control amount (first movement amount) / 920y...Second control amount (second movement amount) / W...Pin (work, object)

Claims

1. An imaging unit that captures an image; An end effector having a main body and a moving part capable of moving an object relative to the main body; a robot that movably supports the position of the end effector; a control unit that controls the imaging unit, the end effector, and the robot, The control unit is When performing movement control to move the position of the object based on the image captured by the imaging unit and a target image, A first mode in which the robot is controlled to move the position of the end effector, thereby moving the position of the object, and a second mode in which the moving unit is controlled to move the position of the object, are executable. A robot device comprising:

2. The control unit is Calculating a target movement amount for moving the position of the object based on the image captured by the imaging unit and a target image; The target movement amount is calculated by dividing the target movement amount into a first movement amount in a first direction in which the moving unit can move and a second movement amount in a second direction different from the first direction; when it is determined to execute the second mode based on the first movement amount, the position of the object is moved by the first movement amount in the second mode and moved by the second movement amount in the first mode.

2. The robot device according to claim 1 .

3. The control unit is calculating a target movement amount for moving the position of the object based on the image captured by the imaging unit, a target image, and a target position that is a known position with respect to the target image; calculating a first movement amount in a first direction in which the moving unit can move based on the target movement amount, and calculating a second movement amount in a second direction different from the first direction based on the target movement amount and the first movement amount; when it is determined to execute the second mode based on the first movement amount, the position of the object is moved by the first movement amount in the second mode and moved by the second movement amount in the first mode.

2. The robot device according to claim 1 .

4. the control unit determines to execute the second mode when the first movement amount is greater than the second movement amount.

3. The robot device according to claim 2.

5. the control unit determines to execute the second mode when the first movement amount is greater than a preset specified value.

3. The robot device according to claim 2.

6. when determining that the second mode is impracticable based on the first movement amount, the control unit moves the position of the object in the first mode.

3. The robot device according to claim 2.

7. when the position of the object is moved in the first mode, the control unit executes visual servoing based on the image captured by the imaging unit and a target image.

2. The robot device according to claim 1 .

8. the control unit executes visual servoing until a difference between an image captured by the imaging unit and a target image becomes equal to or smaller than a certain value.

8. The robot device according to claim 7.

9. The control unit is capable of calculating a coordinate system of the end effector, The moving unit is capable of moving the object in at least one coordinate direction in a coordinate system of the end effector.

2. The robot device according to claim 1 .

10. The moving unit is capable of moving the object in a straight line relative to the main body unit.

10. The robot device according to claim 9.

11. The moving unit is capable of moving the object on a plane relative to the main body unit.

10. The robot device according to claim 9.

12. The end effector has a gripping portion that grips and supports a workpiece, The object is a workpiece, The moving unit is capable of moving the gripping unit.

11. The robot device according to claim 10.

13. The gripping portion includes a first finger portion capable of contacting one side of the workpiece and a second finger portion capable of contacting the other side of the workpiece, and the first finger portion and the second finger portion sandwich and grip the workpiece.

13. The robot device according to claim 12.

14. The moving unit has a slide unit that supports the first finger unit and the second finger unit so as to be slidable, a first driving unit that drives the first finger unit to slide on the slide unit, and a second driving unit that drives the second finger unit to slide on the slide unit.

14. The robot device according to claim 13.

15. When the control unit causes the first finger unit and the second finger unit to slide on the slide unit, when a workpiece is moved to one side, the control unit drives the first finger unit prior to the second finger unit, and when a workpiece is moved to the other side, the control unit drives the second finger unit prior to the first finger unit.

15. The robot device according to claim 14.

16. The imaging unit is supported by the main body.

2. The robot device according to claim 1 .

17. A method for controlling a robot device including an imaging unit that captures images, an end effector having a main body and a moving unit capable of moving an object relative to the main body, a robot that movably supports a position of the end effector, and a control unit that controls the imaging unit, the end effector, and the robot, comprising: The control unit: When performing movement control to move the position of the object based on the image captured by the imaging unit and a target image, a first step of controlling the robot to move a position of the end effector, thereby moving a position of the object; and a second step of controlling the moving unit to move the position of the object when the movement control is executed. A method for controlling a robot device comprising:

18. A method for manufacturing an article, comprising the steps of: manufacturing an article by using the robot device according to claim 1;

19. A program for causing a computer to execute the control method for a robot device according to claim 17.

20. A computer-readable recording medium storing the program according to claim 19.

Images