TAX DEVICE, TAX PROCEDURE, TAX PROGRAM AND HANDLING SYSTEM

Abstract

To achieve a goal of achieving an accurate gripping process without using a vision sensor, a control device (10) sets a direction of a robot hand (12) so that the robot hand (12) is directly facing a gripping target (W) by controlling a position of a robot arm (11) such that a grid pattern detected by an imaging section (12c) and a predetermined grid pattern are similar.

Description

Technical field

[0001] The present invention relates to a control device, a control method, and a control program, each for controlling a robot arm. Furthermore, the present invention relates to a handling system comprising a robot arm and a control device. background

[0002] Handling systems are widely used that are designed to handle various items using robotic arms to which robotic hands are attached. The items to be handled (hereinafter also referred to as "workpiece") range widely from mechanical components such as bolts to food items such as hamburger patties.

[0003] When handling a workpiece with a complex shape or a soft workpiece, a precise gripping process is essential. This is because an imprecise gripping process on a workpiece with a complex shape is highly likely to lead to grip failure, and an imprecise gripping process on a soft workpiece is highly likely to result in damage to the workpiece.

[0004] To achieve precise gripping, a configuration is often used during the gripping process to repeatedly monitor the position and orientation of a robot hand relative to a workpiece and to correct the position and orientation of the robot hand based on the monitoring results. Monitoring the position and orientation of the robot hand relative to the workpiece is typically performed using a variety of vision sensors.

[0005] To achieve precise gripping, it is also important to accurately identify the shape of the workpiece. For example, document JP 2019-058993A discloses a technology for identifying the shape of a workpiece by applying a phase-shift method to an image containing a workpiece as the object, onto which a stripe pattern is projected. Summary of the invention; Technical task

[0006] The handling system, which repeatedly monitors the position and orientation of a robot hand relative to a workpiece during the gripping process and corrects the position and orientation of the robot hand based on the monitoring results, presents the following problems. First, the vision sensor is expensive. This makes it difficult to reduce the cost of such a handling system. Second, the process of analyzing an image obtained using the vision sensor to identify the position and orientation of the workpiece takes time. This makes it difficult to speed up the gripping process with such a handling system. The technique disclosed in document JP 2019-058993A is a technique for identifying the shape of a workpiece and therefore does not directly contribute to solving these problems.

[0007] One aspect of the present invention was made with regard to the aforementioned problem, and it is an objective of it to achieve a precise gripping process without the use of a vision sensor in a handling system. Solution to the task

[0008] A control device according to one aspect of the present invention is a control device that controls the position of a robot arm to which a robot hand is attached, wherein the robot hand has a projection section that projects a grid pattern onto a grasping target and an imaging section that images the grid pattern projected onto the grasping target, wherein the projection section is configured to project a predetermined grid pattern onto a plane that faces directly towards the robot hand, wherein the control device performs a direction-setting process to set a direction of the robot hand so that the robot hand faces directly towards the grasping target by controlling the position of the robot arm so that the grid pattern detected by the imaging section and the predetermined grid pattern are similar.

[0009] A control method according to one aspect of the present invention is a control method for controlling a position of a robot arm to which a robot hand is attached, wherein the robot hand has a projection section that projects a grid pattern onto a grasping target and an imaging section that maps the grid pattern projected onto the grasping target, wherein the projection section is configured to project a predetermined grid pattern onto a plane that faces directly towards the robot hand, wherein the control method includes a direction setting process for setting a direction of the robot hand such that the robot hand faces directly towards the grasping target by controlling the position of the robot arm so that the grid pattern detected by the imaging section and the predetermined grid pattern are similar. Advantageous effects of the invention

[0010] According to one aspect of the present invention, it is possible to achieve a precise gripping process without using a vision sensor. Brief description of the drawings Fig. Figure 1 is a side view showing a configuration of a handling system according to one embodiment. Fig. 2 is a side view showing a positional relationship between a workpiece and a robot hand in the Fig. 1 represents the handling system shown. Fig. 3 is a view that represents an image of a section of the robot hand in which it is shown. Fig. The handling system shown in section 1 is recorded. Reference number 3001 of Fig. Figure 3 shows an image that is captured before a direction adjustment operation is performed, and the reference numeral 3002 of Fig. Figure 3 shows an image that is captured after the direction adjustment process has been performed. Fig. 4 is a view that represents an image of a section of the robot hand in which it is shown. Fig. The handling system shown in section 1 is recorded. Reference number 4001 of Fig. Figure 4 shows an image that is captured before a position adjustment operation is performed, and the reference numeral 4002 of Fig. Figure 4 shows an image that is captured after the position adjustment process has been performed. Description of embodiments (configuration of the handling system)

[0011] With reference to Fig. 1 The following description will discuss a configuration of a handling system 1 according to an embodiment of the present invention. Fig. Figure 1 is a side view showing a configuration of handling system 1. Fig. Figure 1 also shows a top view, which, viewed from one side of a finger, represents a robot hand 12 contained in the handling system 1.

[0012] The handling system 1, which is a system for handling a workpiece W, comprises a robot arm 11, a robot hand 12 attached to the robot arm 11, and a control device 10 configured to control the position of the robot arm 11. The workpiece W is not particularly constrained but has at least one surface. This surface is referred to below as the "principal surface S" of the workpiece W. A straight line passing through a representative point (e.g., the center point) of this surface and perpendicular to this surface is referred to as the "principal normal line N" of the workpiece W. The workpiece W, which is an object to be grasped by the robot hand 12, can also be expressed as the "grasping target".

[0013] The robot arm 11 is configured to change its position in order to adjust the position and direction of the robot hand 12. In the present embodiment, a vertical articulated robot is used as the robot arm 11. In this case, changing the position of the robot arm 11 is equivalent to changing the joint angles of the joints.

[0014] The robot hand 12 is configured to change its position in order to grasp the workpiece W. In the present embodiment, a three-fingered robot hand with fingers 12a1 to 12a3 is used as the robot hand 12. As an example, the first finger 12a1 is a fixed finger, whereas the second finger 12a2 and the third finger 12a3 are movable fingers. In this case, changing the position of the robot hand 12 is equivalent to changing the joint angles of the movable fingers 12a2 to 12a3.

[0015] The robot hand 12 further comprises a projection section 12b and an imaging section 12c. The projection section 12b is configured to project a grid pattern and a frame line onto the workpiece W and is located on a surface on a finger side of a housing of the robot hand 12 such that an optical axis of the light emitted by the projection section 12b is parallel to a central axis CA of the robot hand 12. The projection section 12b consists, for example, of a laser light source and a diffractive optical element (DOE) arranged on an optical path of the laser light emitted by the laser light source.Imaging section 12c is configured to image the workpiece W onto which the grid pattern is projected and is located on the surface on the finger side of the housing of the robot hand 12 such that an optical axis of the light input into imaging section 12c is parallel to the central axis CA of the robot hand 12. Imaging section 12c consists, for example, of an image sensor and a lens group arranged along an optical path of the light input into the image sensor.

[0016] The projection section 12b is configured to project a predetermined grid pattern onto a plane directly facing the robot hand 12. Here, the phrase "plane directly facing the robot hand 12" means that the central axis CA of the robot hand 12 is perpendicular to the plane, i.e., the optical axes of the projection section 12b and the imaging section 12c are perpendicular to the plane.

[0017] In the present embodiment, the projection section 12b is configured to project a grid pattern in a square grid onto the plane directly facing the robot hand 12. It should be noted that the present invention is not limited to this. For example, the projection section 12b can be configured to project a grid pattern in a face-centered grid onto the plane directly facing the robot hand 12, or it can be configured to project a grid pattern in a hexagonal grid onto the plane directly facing the robot hand 12.

[0018] Furthermore, the projection section 12b is set up to project a frame line onto the plane that is directly facing the robot hand 12, which approaches an outer edge of the workpiece W that is arranged on this plane.

[0019] In the present embodiment, the projection section 12b is configured to project a first frame line (hereinafter also referred to as the "inner frame line"), which approaches the outer edge of the workpiece W arranged on this plane from the inside, and a second frame line (hereinafter also referred to as the "outer frame line"), which approaches the outer edge of the workpiece W arranged on this plane from the outside. It should be noted that the present invention is not limited to this. For example, the projection section 12b can be configured to project only the inner frame line onto the plane directly facing the robot hand 12, or it can be configured to project only the outer frame line onto the plane directly facing the robot hand 12.

[0020] The control device 10 is configured to handle the workpiece W by controlling the robot arm 11 and the robot hand 12. In the present embodiment, a general-purpose computer with a memory 10a and a processor 10b is used as the control device 10. In this case, the processor 10b executes a control program stored in the memory 10a, thus controlling the robot arm 11 and the robot hand 12. (Features of the handling system)

[0021] With reference to Fig. 2 will discuss the following description of the features of handling system 1. Fig. Figure 2 is a side view showing a positional relationship between the workpiece W and the robot hand 12 in the handling system 1.

[0022] Features of handling system 1 include performing a direction setting operation and a position setting operation before performing a gripping operation.

[0023] The direction setting operation refers to the process of controlling the position of the robot arm 11 by the control device 10 to adjust the direction of the robot hand 12 so that the robot hand 12 faces directly towards the workpiece W. The grid pattern projected by projection section 12b and captured by figure section 12c is used for this direction setting operation, as described later. Performing the direction setting operation ensures that the central axis CA of the robot hand 12 is parallel to the principal normal line N of the workpiece W, as indicated by reference numeral 2001 of Fig. 2 displayed.

[0024] The position setting operation refers to the process of controlling the position of the robot arm 11 by the control device 10 to adjust the position of the robot hand 12 so that the robot hand 12 faces directly towards the workpiece W. The frame line projected by projection section 12b and captured by figure section 12c is used for this position setting operation, as described later. Performing the position setting operation after the direction setting operation has been carried out allows the central axis CA of the robot hand 12 to coincide with (be parallel to and overlap with) the principal normal line N of the workpiece W, as indicated by reference numeral 2002 of Fig. 2 displayed.

[0025] The handling system 1 performs a gripping operation on the workpiece W after the direction and position adjustment processes have been completed. This allows the gripping operation to be started from a state where it is ensured that the direction and position of the robot hand 12 are those that would be achieved if the robot hand 12 were directly facing the workpiece W. This enables the workpiece W to be gripped appropriately using the robot hand 12 without having to correct its trajectory, while the direction and position of the robot hand 12 relative to the workpiece W are monitored during the gripping operation. Even if the workpiece W is a soft item such as food (for example, a hamburger patty), it can be gripped without being damaged or broken.

[0026] In a case where the trajectory of the robot hand 12 is corrected while its direction and position relative to the workpiece W are monitored during the gripping process, it is necessary to repeatedly perform processing such as imaging, image analysis, and trajectory correction during the gripping process. This can lead to a reduced gripping speed. Furthermore, in this case, it is necessary to add a vision sensor to the configuration to image the workpiece W and the robot hand 12 during the gripping process. This can lead to increased manufacturing and operating costs for the handling system 1.In contrast, with the handling system 1 according to the present embodiment, it is not necessary to correct the trajectory of the robot hand 12 while monitoring the direction and position of the robot hand 12 in relation to the workpiece W during the gripping process, and thus it is possible to reduce the possibility of these problems occurring. (Direction adjustment process)

[0027] With reference to Fig. Section 3 will discuss the direction adjustment process carried out with handling system 1. Fig. Reference 3001 indicates an image captured by figure section 12c before the direction adjustment operation is performed, and reference 3002 indicates an image captured by figure section 12c after the direction adjustment operation has been performed.

[0028] During the direction adjustment process, the control device 10 performs a direction adjustment process with reference to the grid pattern that is contained as an object in the image captured by section 12c of the figure.

[0029] In a case where the direction of the robot hand 12 is not a direction that is assumed when the robot hand 12 is directly facing the workpiece W, the grid pattern projected onto the main surface S of the workpiece W in the image captured by figure section 12c is distorted and not similar to a predetermined grid pattern (see reference numeral 3001 of Fig. 3) In contrast, in a case where the direction of the robot hand 12 is a direction adopted when the robot hand 12 is directly facing the workpiece W, the grid pattern projected onto the main surface S of the workpiece W in the image captured by figure section 12c is not distorted and is similar to a predetermined grid pattern (see reference numeral 3002 of Fig. 3).

[0030] The control device 10 performs the direction setting process using this method. That is, the position of the robot arm 11 is controlled such that the grid pattern projected onto the main surface S of the workpiece W in the image captured by section 12c is similar to the predetermined grid pattern. This makes it possible to adjust the direction of the robot hand 12 so that the robot hand 12 faces directly towards the workpiece W.

[0031] It should be noted that the control device 10 can determine the direction of the robot hand 12, which is facing directly towards the workpiece W, for example by "searching". As an example, the control device 10 repeatedly performs a process to obtain an image captured by section 12c of the figure and a process to randomly change the direction of the robot hand 12. The control device 10 then identifies as the direction that the robot hand 12 will take when facing directly towards the workpiece W the direction of the robot hand 12 at a time when an image is captured in which the grid pattern projected onto the main surface S of the workpiece W is similar to the predetermined grid pattern.

[0032] It should be noted that the control device 10 can control the position of the robot arm 11 such that in the image captured by section 12c, the grid pattern projected onto the main surface S of the workpiece W is similar to the predetermined grid pattern and a similarity ratio between the grid pattern projected onto the main surface S of the workpiece W and the predetermined grid pattern becomes a predetermined value.

[0033] This makes it possible to set a predetermined distance from a plane containing the main surface S of the workpiece W to the robot hand 12. (Positioning process)

[0034] With reference to Fig. Section 4 will discuss the position setting process carried out with handling system 1. Fig. Reference 4001 indicates an image captured by figure section 12c before the position adjustment operation is performed, and reference 4002 indicates an image captured by figure section 12c after the position adjustment operation has been performed.

[0035] During the position setting process, the control device 10 performs a position setting process with reference to the inner frame line and the outer frame line, which are contained as objects in the image captured by figure section 12c.

[0036] In a case where the position of the robot hand 12 is not a position that is assumed when the robot hand 12 is directly facing the workpiece W, the outer edge of the workpiece W does not fall between the inner frame line and the outer frame line in the image captured by figure section 12c (see reference numeral 4001 of Fig. 4) In contrast, in a case where the position of the robot hand 12 is a position assumed when the robot hand 12 is directly facing the workpiece W, the outer edge of the workpiece W falls between the outer frame line and the inner frame line in the image captured by section 12c of the figure (see reference numeral 4002 of Fig. 4) That is, the inner frame line is located inside the outer edge of the workpiece W, and the outer frame line is located outside the outer edge of the workpiece W.

[0037] The control device 10 performs the position setting process using this method. That is, the position of the robot arm 11 is controlled so that, in the image captured by section 12c of the figure, the inner frame line is located within the outer edge of the workpiece W and the outer frame line is located outside the outer edge of the workpiece W. This makes it possible to set the position of the robot hand so that the robot hand is directly facing the workpiece W.

[0038] It should be noted that the control device 10 can determine the position of the robot hand 12, which is directly facing the workpiece W, for example by "searching". As an example, the control device 10 repeatedly performs a process to obtain an image captured by section 12c of the figure and a process to randomly change the position of the robot hand 12. The control device 10 then identifies as the position assumed when the robot hand 12 is directly facing the workpiece W the position of the robot hand 12 at a time when an image is captured in which the inner frame line is inside the outer edge of the workpiece W and the outer frame line is outside the outer edge of the workpiece W. (Software implementation example)

[0039] Functions of the control device 10 can be implemented by a control program that causes a computer to function as the control device 10. In this case, the control device 10 comprises at least one processor and at least one memory as hardware for executing the control program. The functions of the control device 10 described in the embodiments above are implemented by executing the control program stored in the at least one memory by the at least one processor.

[0040] The control program can be stored on one or more non-volatile, computer-readable storage media. These storage media may or may not be included in the control device 10. In the latter case, the control program can be supplied to the control device 10 via any wired or wireless transmission medium.

[0041] Furthermore, some or all of the functions of the control device 10 can be implemented by a logic circuit. For example, the present disclosure includes an integrated circuit in which a logic circuit is formed that achieves some or all of the functions of the control device 10.

[0042] Aspects of the present invention can also be expressed as follows: A control device according to aspect 1 of the present invention is a control device that controls the position of a robot arm to which a robot hand is attached, wherein the robot hand has a projection section that projects a grid pattern onto a grasping target and an imaging section that performs an image acquisition of the grid pattern projected onto the grasping target, wherein the projection section is configured to project a predetermined grid pattern onto a plane that is directly facing the robot hand, wherein the control device performs a direction setting process to set a direction of the robot hand so that the robot hand is directly facing the grasping target, by controlling the position of the robot arm so that the grid pattern captured by the imaging section and the predetermined grid pattern are similar.

[0043] A control device according to aspect 2 of the present invention is arranged such that in the control device according to aspect 1, the control device controls the position of the robot arm in the direction setting process, so that the grid pattern detected by the illustration section and the predetermined grid pattern are similar and a similarity ratio between the grid pattern detected by the illustration section and the predetermined grid pattern becomes a predetermined value.

[0044] A control device according to aspect 3 of the present invention is arranged such that in the control device according to aspect 1 or 2 the predetermined grid pattern is a grid pattern in a square grid.

[0045] A control device according to aspect 4 of the present invention is configured such that in the control device according to one of aspects 1 to 3 the projection section is configured to project a frame line onto a plane directly facing the robot hand, the frame line approaching an outer edge of the gripping target arranged on the plane, wherein, after performing the direction setting process, the control device further performs a position setting process to set a position of the robot hand so that the robot hand is directly facing the gripping target, by controlling the position of the robot arm, such that the frame line captured by the projection section approaches the outer edge of the gripping target captured by the projection section.

[0046] A control device according to aspect 5 of the present invention is configured such that in the control device according to aspect 4 the projection section is configured to project a first frame line onto the plane directly facing the robot hand, approaching the outer edge of the gripping target arranged on the plane from the inside, and a second frame line approaching the outer edge of the gripping target arranged on the plane from the outside, wherein in the position setting process the control device controls the position of the robot arm, such that the first frame line captured by the projection section is located inside the outer edge of the gripping target captured by the projection section, and the second frame line captured by the projection section is located outside the outer edge of the gripping target captured by the projection section.

[0047] A control method according to aspect 6 of the present invention is a control method for controlling a position of a robot arm to which a robot hand is attached, wherein the robot hand has a projection section that projects a grid pattern onto a grasping target and an imaging section that performs image acquisition of the grid pattern projected onto the grasping target, wherein the projection section is configured to project a predetermined grid pattern onto a plane that is directly facing the robot hand, wherein the control method comprises a direction setting process for setting a direction of the robot hand so that the robot hand is directly facing the grasping target, by controlling the position of the robot arm so that the grid pattern captured by the imaging section and the predetermined grid pattern are similar.

[0048] A control program according to aspect 7 of the present invention is a control program for causing a computer to function as the control device according to any one of aspects 1 to 5, wherein the control program causes the computer to perform the direction setting process.

[0049] A handling system according to aspect 8 of the present invention comprises: the control device according to one of aspects 1 to 5; and the robot hand. (Additional comments)

[0050] The present disclosure is not limited to the embodiments described above, but can be modified by a person skilled in the art within the scope of the claims. The present disclosure also encompasses, within its technical scope, any embodiment derived as required by combining technical means disclosed in different embodiments. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2019 - 058 993 A [0005, 0006]

Claims

[1] Control device that controls the position of a robot arm to which a robot hand is attached, wherein the robot hand has a projection section that projects a grid pattern onto a grasping target and an imaging section that performs an image capture of the grid pattern projected onto the grasping target, wherein the projection section is set up to project a predetermined grid pattern onto a plane that is directly facing the robot hand, wherein the control device performs a direction setting process to set a direction of the robot hand so that the robot hand is directly facing the grasping target, by controlling the position of the robot arm so that the grid pattern captured by the figure section and the predetermined grid pattern are similar. [2] Control device according to claim 1, wherein in the direction setting process the control device controls the position of the robot arm, such that the grid pattern detected by the illustration section and the predetermined grid pattern are similar and a similarity ratio between the grid pattern detected by the illustration section and the predetermined grid pattern becomes a predetermined value. [3] Control device according to claim 1 or 2, wherein the predetermined grid pattern is a grid pattern in a square grid. [4] Control device according to one of claims 1 to 3, wherein the projection section is configured to project a frame line onto a plane directly facing the robot hand, the frame line approaching an outer edge of the grasping target arranged on the plane, wherein, after performing the direction setting process, the control device further performs a position setting process to set a position of the robot hand so that the robot hand is directly facing the grasping target, by controlling the position of the robot arm, such that the frame line captured by the projection section approaches the outer edge of the grasping target captured by the projection section. [5] Control device according to claim 4, wherein the projection section is configured to project a first frame line approaching the outer edge of the grasping target arranged on the plane from the inside and a second frame line approaching the outer edge of the grasping target arranged on the plane from the outside, wherein in the position setting process the control device controls the position of the robot arm such that the first frame line captured by the projection section is located inside the outer edge of the grasping target captured by the projection section and the second frame line captured by the projection section is located outside the outer edge of the grasping target captured by the projection section. [6] Control method for controlling the position of a robot arm to which a robot hand is attached, wherein the robot hand has a projection section that projects a grid pattern onto a grasping target and an imaging section that performs an image capture of the grid pattern projected onto the grasping target, wherein the projection section is set up to project a predetermined grid pattern onto a plane that is directly facing the robot hand, wherein the control method includes a direction setting process for setting a direction of the robot hand so that the robot hand is directly facing the grasping target, by controlling the position of the robot arm so that the grid pattern captured by the figure section and the predetermined grid pattern are similar. [7] Control program for causing a computer to act as the control device according to any one of claims 1 to 5, wherein the control program causes the computer to perform the direction setting process. [8] Handling system comprising: - the control device according to any one of claims 1 to 5; and - the robot hand.

Images