Control device, control method, control program, handling system

Abstract

It enables precise grasping movements without the use of vision sensors. [Solution] The control device (10) controls the posture of the robot arm (11) so that the grid pattern captured by the imaging unit (12c) and a predetermined grid pattern are similar, thereby adjusting the direction of the robot hand (12) to face the object to be grasped (W).

Description

Technical Field

[0001] The present invention relates to a control device, a control method, and a control program for controlling a robot arm. It also relates to a handling system including a robot arm and a control device.

Background Art

[0002] Handling systems that use a robot arm equipped with a robot hand to handle various items are widely used. The items to be handled (hereinafter also referred to as "workpieces") cover a wide range, from mechanical parts such as bolts to food products such as hamburgers.

[0003] When handling workpieces with complex shapes or soft workpieces, an accurate gripping operation is required. This is because if the gripping operation for a workpiece with a complex shape is inaccurate, there is a high possibility of gripping failure, and if the gripping operation for a soft workpiece is inaccurate, there is a high possibility of damaging the workpiece.

[0004] In order to achieve an accurate gripping operation, a configuration is often adopted in which the position and orientation of the robot hand with respect to the workpiece are monitored, and the position and orientation of the robot hand are corrected based on the monitoring results repeatedly during the gripping operation. Usually, a plurality of vision sensors are used for monitoring the position and orientation of the robot hand with respect to the workpiece.

[0005] Also, in order to achieve an accurate gripping operation, it is also important to accurately identify the shape of the workpiece. For example, Patent Document 1 discloses a technique for identifying the shape of a workpiece by applying a phase shift method to an image including the workpiece as a subject onto which a striped pattern is projected.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

[0007] A handling system that repeatedly monitors the position and orientation of a robot hand relative to a workpiece, and corrects the position and orientation of the robot hand based on the monitoring results, during a gripping operation has the following problems. Firstly, vision sensors are expensive. Therefore, it is difficult to reduce the cost of such a handling system. Secondly, the process of analyzing images obtained by vision sensors to determine the position and orientation of the workpiece is time-consuming. Therefore, it is difficult to speed up the gripping operation in such a handling system. Note that the technology described in Patent Document 1 is a technology for determining the shape of a workpiece, and therefore does not directly contribute to solving these problems.

[0008] One aspect of the present invention has been made in view of the above-mentioned problems, and its object is to achieve accurate gripping operations in a handling system without using a vision sensor. [Means for solving the problem]

[0009] A control device according to one aspect of the present invention is a control device for controlling the posture of a robot arm to which a robot hand is attached, wherein the robot hand has a projection unit that projects a grid pattern onto an object to be grasped, and an imaging unit that images the grid pattern projected onto the object to be grasped, the projection unit is configured to project a predetermined grid pattern onto a plane directly facing the robot hand, and the control device performs a direction adjustment process to adjust the direction of the robot hand so that it faces the object to be grasped by controlling the posture of the robot arm so that the grid pattern imaged by the imaging unit and the predetermined grid pattern are similar.

[0010] A control method according to one aspect of the present invention is a control method for controlling the posture of a robot arm to which a robot hand is attached, wherein the robot hand has a projection unit that projects a grid pattern onto an object to be grasped, and an imaging unit that images the grid pattern projected onto the object to be grasped, the projection unit is configured to project a predetermined grid pattern onto a plane directly facing the robot hand, and the control method is characterized in that it includes a direction adjustment process that adjusts the direction of the robot hand so that it faces the object to be grasped by controlling the posture of the robot arm so that the grid pattern imaged by the imaging unit and the predetermined grid pattern are similar. [Effects of the Invention]

[0011] According to one aspect of the present invention, accurate grasping operations can be achieved without using a vision sensor. [Brief explanation of the drawing]

[0012] [Figure 1] This is a side view showing the configuration of the handling system according to the embodiment. [Figure 2] Figure 1 is a side view showing the positional relationship between the workpiece and the robot hand in the handling system. [Figure 3] Figure 1 shows images captured by the imaging unit of the robot hand in the handling system shown in Figure 1. (a) is an image captured before the direction adjustment operation is performed, and (b) is an image captured after the direction adjustment operation is performed. [Figure 4] Figure 1 shows images captured by the imaging unit of the robot hand in the handling system shown in Figure 1. (a) is an image captured before the position adjustment operation is performed, and (b) is an image captured after the position adjustment operation is performed. [Modes for carrying out the invention]

[0013] (Handling system configuration) The configuration of the handling system 1 according to one embodiment of the present invention will be described with reference to Figure 1. Figure 1 is a side view showing the configuration of the handling system 1. In Figure 1, a plan view of the robot hand 12 included in the handling system 1, viewed from the finger side, is also shown.

[0014] The handling system 1 is a system for handling a workpiece W and includes a robot arm 11, a robot hand 12 attached to the robot arm 11, and a control device 10 for controlling the posture of the robot arm 11. The workpiece W is not particularly limited, but it is assumed to have at least one surface. Hereinafter, this surface will be referred to as the principal surface S of the workpiece W. A straight line passing through a representative point (e.g., the center point) of this surface and perpendicular to this surface will be referred to as the principal normal N of the workpiece W. Since the workpiece W is an object grasped by the robot hand 12, it can also be referred to as the object to be grasped.

[0015] The robot arm 11 is configured to set the position and direction of the robot hand 12 by changing its posture. In this embodiment, a vertical articulated robot is used as the robot arm 11. In this case, changing the posture of the robot arm 11 is equivalent to changing the joint angle of each joint.

[0016] The robot hand 12 is configured to grasp the workpiece W by changing its posture. In this embodiment, a three-fingered robot hand having fingers 12a1 to 12a3 is used as the robot hand 12. For example, the first finger 12a1 is a fixed finger, and the second finger 12a2 and the third finger 12a3 are movable fingers. In this case, changing the posture of the robot hand 12 is equivalent to changing the joint angles of the movable fingers 12a2 to 12a3.

[0017] The robot hand 12 further has a projection unit 12b and an imaging unit 12c. The projection unit 12b is configured to project a grid pattern and a frame line onto the workpiece W. The projection unit 12b is arranged on the finger-side surface of the housing of the robot hand 12 such that the optical axis of the light output from the projection unit 12b is parallel to the central axis CA of the robot hand 12. The projection unit 12b is constituted by, for example, a laser light source and a DOE (Diffractive Optical Element, diffractive optical element) arranged on the optical path of the laser light output from the laser light source. The imaging unit 12c is configured to image the workpiece W onto which the grid pattern is projected. The imaging unit 12c is arranged on the finger-side surface of the housing of the robot hand 12 such that the optical axis of the light input to the imaging unit 12c is parallel to the central axis CA of the robot hand 12. The imaging unit 12c is constituted by, for example, an image sensor and a lens group arranged on the optical path of the light input to the image sensor.

[0018] The projection unit 12b is configured to project a predetermined grid pattern onto a plane facing the robot hand 12. Here, when a certain plane faces the robot hand 12, it means that the central axis CA of the robot hand 12 is perpendicular to that plane, that is, the optical axes of the projection unit 12b and the imaging unit 12c are perpendicular to that plane.

[0019] In the present embodiment, it is assumed that the projection unit 12b is configured to project a square grid pattern onto a plane facing the robot hand 12. However, the present invention is not limited to this. For example, the projection unit 12b may be configured to project a face-centered lattice grid pattern onto a plane facing the robot hand 12, or may be configured to project a hexagonal grid pattern onto a plane facing the robot hand 12.

[0020] Further, the projection unit 12b is configured to project a frame line approximating the outer edge of the workpiece W arranged on this plane onto a plane facing the robot hand 12.

[0021] In this embodiment, the projection unit 12b projects, onto a plane facing the robot hand 12, a first frame line (hereinafter also referred to as the "inner frame line") that approximates the outer edge of the workpiece W arranged on this plane from the inside, and a second frame line (hereinafter also referred to as the "outer frame line") that approximates the outer edge of the workpiece W arranged on this plane from the outside. However, the present invention is not limited to this. For example, the projection unit 12b may be configured to project only the inner frame line onto the plane facing the robot hand 12, or may be configured to project only the outer frame line onto the plane facing the robot hand 12.

[0022] The control device 10 is configured to realize the handling of the workpiece W by controlling the robot arm 11 and the robot hand 12. In this embodiment, a general-purpose computer including a memory 10a and a processor 10b is used as the control device 10. In this case, the processor 10b executes the control program stored in the memory 10a, thereby realizing the control of the robot arm 11 and the robot hand 12.

[0023] (Features of the handling system) The features of the handling system 1 will be described with reference to FIG. 2. FIG. 2 is a side view showing the positional relationship between the workpiece W and the robot hand 12 in the handling system 1.

[0024] The feature of the handling system 1 lies in performing a direction adjustment operation and a position adjustment operation before performing the gripping operation.

[0025] The direction adjustment operation refers to an operation in which the control device 10 controls the posture of the robot arm 11 to adjust the direction of the robot hand 12 so as to face the workpiece W. The grid pattern projected by the projection unit 12b and imaged by the imaging unit 12c is used for this direction adjustment operation as will be described later. By performing the direction adjustment operation, as shown in (a) of FIG. 2, the central axis CA of the robot hand 12 becomes parallel to the principal normal N of the workpiece W.

[0026] The position adjustment operation refers to the operation in which the control device 10 controls the posture of the robot arm 11 to adjust the position of the robot hand 12 so that it faces the workpiece W directly. The frame lines projected by the projection unit 12b and captured by the imaging unit 12c are used in this position adjustment operation, as will be described later. By performing the position adjustment operation after the direction adjustment operation, the central axis CA of the robot hand 12 coincides with (is parallel to and overlaps with) the principal normal N of the workpiece W, as shown in Figure 2(b).

[0027] In the handling system 1, after performing direction adjustment and position adjustment operations, a gripping operation is performed to grasp the workpiece W. This ensures that the direction and position of the robot hand 12 are directly facing the workpiece W before the gripping operation can begin. Therefore, the robot hand 12 can properly grasp the workpiece W without monitoring the direction and position of the robot hand 12 relative to the workpiece W during the gripping operation and without correcting the trajectory of the robot hand 12. Even if the workpiece W is a soft item such as food products (e.g., hamburgers), the workpiece W can be grasped without damaging or destroying it.

[0028] When the direction and position of the robot hand 12 relative to the workpiece W are monitored and the trajectory of the robot hand 12 is corrected during the gripping operation, it is necessary to repeatedly perform processes such as imaging, image analysis, and trajectory correction during the gripping operation, which can lead to a decrease in the speed of the gripping operation. In addition, in this case, it is necessary to add a vision sensor to the configuration for imaging the workpiece W and the robot hand 12 during the gripping operation, which can lead to an increase in the manufacturing and operating costs of the handling system 1. In contrast, in the handling system 1 according to this embodiment, it is not necessary to monitor the direction and position of the robot hand 12 relative to the workpiece W and correct the trajectory of the robot hand 12 during the gripping operation, thus reducing the possibility of these problems occurring.

[0029] (Direction adjustment operation) The direction adjustment operation performed in the handling system 1 will be explained with reference to Figure 3. In Figure 3, (a) represents an image captured by the imaging unit 12c before the direction adjustment operation is performed, and (b) represents an image captured by the imaging unit 12c after the direction adjustment operation is performed.

[0030] During the direction adjustment operation, the control device 10 performs the direction adjustment process by referring to the grid pattern included as a subject in the image captured by the imaging unit 12c.

[0031] When the robot hand 12 is not facing the workpiece W, the grid pattern projected onto the main surface S of the workpiece W in the image captured by the imaging unit 12c is distorted and does not resemble the predetermined grid pattern (see Figure 3(a)). On the other hand, when the robot hand 12 is facing the workpiece W, the grid pattern projected onto the main surface S of the workpiece W in the image captured by the imaging unit 12c is not distorted and resembles the predetermined grid pattern (see Figure 3(b)).

[0032] The control device 10 utilizes this property to perform direction adjustment processing. Specifically, it controls the posture of the robot arm 11 so that the grid pattern projected onto the main surface S of the workpiece W in the image captured by the imaging unit 12c is similar to a predetermined grid pattern. This allows the direction of the robot hand 12 to be adjusted to face the workpiece W directly.

[0033] The control device 10 can determine the direction of the robot hand 12 facing the workpiece W by, for example, "searching". As an example, the control device 10 repeatedly performs the process of acquiring an image captured by the imaging unit 12c and the process of randomly changing the direction of the robot hand 12. The control device 10 then identifies the direction of the robot hand 12 when an image is captured in which the grid pattern projected onto the main surface S of the workpiece W is similar to a predetermined grid pattern as the direction when facing the workpiece W.

[0034] The control device 10 may also control the posture of the robot arm 11 so that the grid pattern projected onto the main surface S of the workpiece W in the image captured by the imaging unit 12c is similar to a predetermined grid pattern, and the similarity ratio between the grid pattern projected onto the main surface S of the workpiece W and the predetermined grid pattern is a predetermined value. This allows the distance from the plane containing the main surface S of the workpiece W to the robot hand 12 to be set to a predetermined distance.

[0035] (Position adjustment operation) The position adjustment operation performed in the handling system 1 will be explained with reference to Figure 4. In Figure 4, (a) represents an image captured by the imaging unit 12c before the position adjustment operation is performed, and (b) represents an image captured by the imaging unit 12c after the position adjustment operation is performed.

[0036] During the position adjustment operation, the control device 10 performs a position adjustment process by referring to the inner and outer frame lines included as subjects in the image captured by the imaging unit 12c.

[0037] When the robot hand 12 is not directly facing the workpiece W, the outer edge of the workpiece W is not contained between the inner and outer frame lines in the image captured by the imaging unit 12c (see Figure 4(a)). On the other hand, when the robot hand 12 is directly facing the workpiece W, the outer edge of the workpiece W is contained between the outer and inner frame lines in the image captured by the imaging unit 12c (see Figure 4(b)). That is, the inner frame line is located inside the outer edge of the workpiece W, and the outer frame line is located outside the outer edge of the workpiece W.

[0038] The control device 10 utilizes this property to perform direction adjustment processing. Specifically, it controls the posture of the robot arm 11 so that, in the image captured by the imaging unit 12c, the inner frame line is located inside the outer edge of the workpiece W, and the outer frame line is located outside the outer edge of the workpiece W. This allows the position of the robot hand 12 to be adjusted to face the workpiece W directly.

[0039] The control device 10 can determine the position of the robot hand 12 facing the workpiece W by, for example, "searching". As an example, the control device 10 repeatedly performs the process of acquiring an image captured by the imaging unit 12c and the process of randomly changing the position of the robot hand 12. The control device 10 then identifies the position of the robot hand 12 when an image is captured in which the inner frame line is located inside the outer edge of the workpiece W and the outer frame line is located outside the outer edge of the workpiece W as the direction in which the robot hand 12 will face the workpiece W.

[0040] (Example of implementation using software) The functions of the control device 10 can be realized by a control program that causes the computer to function as the control device 10. In this case, the control device 10 includes at least one processor and at least one memory as hardware for executing the control program. The functions of the control device 10 described in each of the embodiments are realized by the execution of the control program stored in at least one memory by at least one processor.

[0041] The control program described above may be recorded on one or more computer-readable recording media, rather than on a temporary basis. This recording media may or may not be provided by the control device 10. In the latter case, the control program may be supplied to the control device 10 via any wired or wireless transmission medium.

[0042] Furthermore, some or all of the functions of the control device 10 can also be implemented by logic circuits. For example, an integrated circuit in which logic circuits that implement some or all of the functions of the control device 10 are formed is also included in the scope of this disclosure.

[0043] (summary) A control device according to Embodiment 1 of the present invention is a control device for controlling the posture of a robot arm to which a robot hand is attached, wherein the robot hand has a projection unit that projects a grid pattern onto an object to be grasped, and an imaging unit that images the grid pattern projected onto the object to be grasped, the projection unit is configured to project a predetermined grid pattern onto a plane directly facing the robot hand, and the control device performs a direction adjustment process to adjust the direction of the robot hand so that it faces the object to be grasped by controlling the posture of the robot arm so that the grid pattern imaged by the imaging unit and the predetermined grid pattern are similar.

[0044] A control device according to aspect 2 of the present invention is characterized in that, in the control device according to aspect 1, in the direction adjustment process, the control device controls the posture of the robot arm so that the grid pattern captured by the imaging unit and the predetermined grid pattern are similar, and the similarity ratio between the grid pattern captured by the imaging unit and the predetermined grid pattern is a predetermined value.

[0045] The control device according to aspect 3 of the present invention is characterized in that, in the control device according to aspect 1 or 2, the predetermined grid pattern is a square grid pattern.

[0046] A control device according to aspect 4 of the present invention is a control device according to any one of aspects 1 to 3, wherein the projection unit is configured to project a frame line approximating the outer edge of the object to be grasped, which is placed on a plane facing the robot hand, and the control device, after performing the direction adjustment process, further performs a position adjustment process to adjust the position of the robot hand so that it faces the object to be grasped, by controlling the posture of the robot arm so that the frame line captured by the imaging unit approximates the outer edge of the object to be grasped, which is captured by the imaging unit.

[0047] A control device according to aspect 5 of the present invention is a control device according to aspect 4, wherein the projection unit is configured to project a first frame line that approximates the outer edge of the object to be grasped, which is placed on the plane, from the inside, onto a plane directly facing the robot hand, and a second frame line that approximates the outer edge of the object to be grasped, which is placed on the plane, from the outside, and in the position adjustment process, the control device controls the posture of the robot arm such that the first frame line captured by the imaging unit is located inside the outer edge of the object to be grasped, which is captured by the imaging unit, and the second frame line captured by the imaging unit is located outside the outer edge of the object to be grasped, which is captured by the imaging unit.

[0048] A control method according to aspect 6 of the present invention is a control method for controlling the posture of a robot arm to which a robot hand is attached, wherein the robot hand has a projection unit that projects a grid pattern onto an object to be grasped, and an imaging unit that images the grid pattern projected onto the object to be grasped, the projection unit is configured to project a predetermined grid pattern onto a plane directly facing the robot hand, and the control method is characterized in that it includes a direction adjustment process that adjusts the direction of the robot hand so that it faces the object to be grasped by controlling the posture of the robot arm so that the grid pattern imaged by the imaging unit and the predetermined grid pattern are similar.

[0049] A control program according to aspect 7 of the present invention is a control program for operating a computer as a control device according to any one of aspects 1 to 5, characterized in that it causes the computer to perform the direction adjustment process.

[0050] A handling system according to aspect 8 of the present invention is characterized by including a control device according to any one of aspects 1 to 5 and the robot hand.

[0051] (Additional notes) This disclosure is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of this disclosure. [Explanation of symbols]

[0052] 1: Handling system, 10: Control device, 11: Robot arm, 12: Robot hand, 12b: Projection unit, 12c: Imaging unit

Claims

1. A control device for controlling the posture of a robot arm to which a robot hand is attached, The robot hand has a projection unit that projects a grid pattern onto the object to be grasped, and an imaging unit that captures the grid pattern projected onto the object to be grasped. The projection unit is configured to project a predetermined grid pattern onto a plane directly facing the robot hand. The control device performs a direction adjustment process to adjust the direction of the robot hand so that it faces the object to be grasped, by controlling the posture of the robot arm so that the grid pattern captured by the imaging unit and the predetermined grid pattern are similar. A control device characterized by the following features.

2. In the direction adjustment process, the control device controls the posture of the robot arm so that the grid pattern captured by the imaging unit and the predetermined grid pattern are similar, and the similarity ratio between the grid pattern captured by the imaging unit and the predetermined grid pattern is a predetermined value. The control device according to feature 1.

3. The predetermined grid pattern is a square grid pattern. The control device according to claim 1 or 2.

4. The projection unit is configured to project a frame line approximating the outer edge of the object to be grasped, which is placed on a plane directly facing the robot hand, After performing the direction adjustment process, the control device further performs a position adjustment process to adjust the position of the robot hand so that it faces the object to be grasped, by controlling the posture of the robot arm so that the frame line captured by the imaging unit approximates the outer edge of the object to be grasped captured by the imaging unit. The control device according to claim 1 or 2.

5. The projection unit is configured to project a first frame line that approximates the outer edge of the object to be grasped, placed on the plane facing the robot hand, from the inside, and a second frame line that approximates the outer edge of the object to be grasped, placed on the plane, from the outside. In the position adjustment process, the control device controls the posture of the robot arm such that the first frame line captured by the imaging unit is located inside the outer edge of the object to be grasped as captured by the imaging unit, and the second frame line captured by the imaging unit is located outside the outer edge of the object to be grasped as captured by the imaging unit. The control device according to feature 4.

6. A control method for controlling the posture of a robot arm to which a robot hand is attached, The robot hand has a projection unit that projects a grid pattern onto the object to be grasped, and an imaging unit that captures the grid pattern projected onto the object to be grasped. The projection unit is configured to project a predetermined grid pattern onto a plane directly facing the robot hand. The control method includes a direction adjustment process that adjusts the direction of the robot hand so that it faces the object to be grasped by controlling the posture of the robot arm so that the grid pattern captured by the imaging unit and the predetermined grid pattern are similar. A control method characterized by the following:

7. A control program for operating a computer as a control device according to claim 1 or 2, wherein the control program causes the computer to perform the direction adjustment process.

8. The control device according to claim 1 or 2 and the robot hand, A handling system characterized by the following.

Images