Method and gripping device for a robot for gripping an object

Abstract

The invention relates to a gripping device (1) for a robot (3) for gripping an object (5), comprising a main body (7) and a plurality of fingers (9) that are movable with respect to the main body (7), and comprising a first camera (11) and a second camera (12), which are configured such that they form a first stereo camera (21) for three-dimensionally capturing the object (5), characterised in that the gripping device (1) comprises a third camera (13), wherein the second camera (12) and the third camera (13) are configured such that they form a second stereo camera (22) for three-dimensionally capturing the object (5).

Description

[0001] Method and gripping device for a robot for gripping an object

[0002] The invention relates to a gripping device for a robot for gripping an object. Furthermore, the invention relates to a method for detecting an object using a gripping device for a robot for gripping the object.

[0003] With the increasing automation of industrial operations, particularly in production and logistics, as well as in healthcare and other sectors, a growing number of robots are being deployed to perform, facilitate, and optimize work. Such robots are used for a variety of purposes and can incorporate various interchangeable tools on the so-called end effector to fulfill their purpose. In addition to welding fixtures, various gripping devices can be used as tools to grasp objects and, in particular, to change their spatial position and / or orientation. A camera can be used to precisely localize the object and ultimately optimize gripping.

[0004] A well-known challenge in robot-assisted gripping of an object in the state of the art is the so-called occlusion problem. Occlusion refers to the partial or complete blocking of a camera's view, particularly by components of the robot itself, in particular by the gripping device or a robot arm segment. Thus, occlusion results in more difficult capture and ultimately suboptimal gripping of the object. As the gripping device or camera approaches the object closer, the occlusion can become more severe. Some attempts are made to solve the occlusion problem by increasing the number of cameras, i.e., by using a camera array, in order to achieve more comprehensive capture of the object. However, any individual camera in a camera array can also be subject to occlusion. In addition, multiple cameras generally pose another challenge: increased costs.

[0005] Against this background, the task is to provide a cost-effective solution that enables improved detection and thus optimized gripping of an object using a gripping device.

[0006] This object is achieved by a gripping device for a robot for gripping an object, having a base body and a plurality of fingers movable relative to the base body, and having a first camera and a second camera which are configured such that they form a first stereo camera for three-dimensionally capturing the object, wherein the gripping device has a third camera, wherein the second camera and the third camera are configured such that they form a second stereo camera for three-dimensionally capturing the object.

[0007] The gripping device has a base body and a plurality of fingers that can be moved relative to the base body. The fingers can be directly connected to the base body so that they can rotate and have a plurality of finger segments that are connected to one another in an articulated manner. This has the particular advantage of imitating a human hand. The gripping device can therefore be used in particular in humanoid robots. The gripping device further comprises a first and a second camera. The first and second cameras form a first stereo camera. The first stereo camera can be understood as a pair of human eyes. Using the two cameras of the first stereo camera, the object can be captured from different perspectives and in three dimensions. According to the invention, the gripping device has a third camera. The second camera and the third camera form a second stereo camera.The second stereo camera can have the same properties and advantages as the first stereo camera. Overall, the gripping device according to the invention results in the advantage of increased robustness against occlusion of the object, for example by the fingers. At the same time, the gripping device according to the invention provides a cost-effective solution, as it prevents the use of an increased number of cameras by allowing at least one camera, the second camera according to the invention, to be used redundantly—as part of both the first and the second stereo camera. The gripping device according to the invention also has the advantage that either only the first or only the second stereo camera can be used, or the two stereo cameras can be used alternately, thus enabling further cost savings.

[0008] A stereo camera is understood to be a capture device that has two individual, particularly digital, cameras for capturing two-dimensional image data and is configured to enable three-dimensional capture of objects in the overlapping capture range of the two cameras based on this image data. According to an advantageous embodiment of the invention, the cameras each have a lens, wherein the lenses of the cameras are arranged in the same plane or in planes parallel to one another. Optionally, at least one lens is designed to be movable, particularly translationally in the sense of a telescopic lens. Thus, the lenses of the respective first and second cameras can be arranged alternately coplanar and non-coplanar. The mobility of a lens offers the advantage of changing a capture range and thus improved focus adjustment.Preferably, both lenses are designed to be movable.

[0009] In an advantageous embodiment of the invention, it is provided that the second camera is arranged between the first camera and the third camera. The second camera can be arranged equidistant from the first and third cameras. This results in the advantage that the second camera occupies a central position between the first and third cameras. Furthermore, the gripping device can be configured such that the second camera is movable along a straight line formed between the first and third cameras. This results in the advantage that a first camera distance of the first stereo camera and a second camera distance of the second stereo camera are adjustable and, in particular, can be unequal. Furthermore, with three cameras, three stereo cameras, each with a different camera distance, can be formed.

[0010] In particular, a third camera distance can be created between the first camera and the third camera, with the third camera distance being greater than the first and second camera distances. This results in the overall advantage of different ranges for the individual stereo cameras.

[0011] A preferred embodiment of the invention provides a fourth camera, wherein the second and fourth cameras are configured such that they form a third stereo camera for three-dimensional capturing of the object. As a result, the second camera can be designed as part of three stereo cameras. Furthermore, three stereo cameras can be realized using four instead of six individual cameras. In other words, two individual cameras and the associated costs can be saved. In particular, the four cameras can be arranged in a star shape, i.e. the second camera can be arranged as a central camera and the first, third and fourth cameras can be arranged as satellite or planetary cameras. In particular, the first, third and fourth cameras can be arranged along a circular circumference, in particular evenly spaced, i.e. equidistant. In addition, up to six stereo cameras can be formed with four cameras.According to an advantageous embodiment of the invention, a fifth camera is provided, wherein the third camera and the fifth camera are configured such that they form a fourth stereo camera for three-dimensional capture of the object. In this respect, a further stereo camera can be provided without requiring two additional individual cameras. Rather, the third camera can be used to provide both the second stereo camera and the fourth stereo camera. Thus, several individual cameras, in this case the second and third cameras, are used as part of several stereo cameras. With five cameras, ten stereo cameras can be formed.

[0012] In a preferred embodiment of the invention, the first camera, the second camera, and the third camera, as well as optionally the fourth camera and / or the fifth camera, are arranged on the base body. The cameras are preferably arranged on a side of the base body facing the fingers and the object. The base body can be flat or trough-shaped, or have a flat or trough-shaped section. In particular, the cameras can be arranged in a central section of the base body. The cameras can also be arranged—at least partially—in an edge region of the base body, which offers the advantage of as many different detection areas and viewing angles as possible.

[0013] The gripping device preferably comprises more than five cameras, for example, N. This makes it possible to create additional stereo cameras and thus further increase robustness against object occlusion. With N cameras, N(N-1) / 2 stereo cameras can be created.

[0014] A further embodiment of the invention provides a finger camera arranged on a finger. This is advantageous in that a finger camera has a different viewing angle than the cameras on the base body. In particular, during a movement of the finger—and thus of the finger camera—relative to the base body, multiple images of the object can be taken, and then, based on the multiple images, a 3D reconstruction of the surface of the object, and thus of the position and orientation of the object, can be performed. If a finger camera is present and is provided as part of a stereo camera, it is advantageous if the gripping device has a device for detecting the position of the finger, in particular of the finger camera, relative to the base body. The detected position of the finger is preferably fed to the respective stereo camera, of which the finger camera is a part.Based on the detected position, the stereo camera can calculate the position relative to the other camera of the stereo camera and, on this basis, enable three-dimensional detection.

[0015] According to an advantageous embodiment of the invention, a light source is provided for illuminating the object. The light source can preferably be switched on and off as needed. Using a light source, the object, its surface, position, and orientation can be detected more precisely, since shading on the object in the case of uneven sections, protrusions, undercuts, etc. can be at least partially eliminated. The brightness of the light source is preferably adjustable, for example, continuously.

[0016] A further subject of the invention is a method for detecting an object with a gripping device for a robot for gripping the object, which has a base body and a plurality of fingers movable relative to the base body, wherein a first camera and a second camera are configured such that they capture the object three-dimensionally as a first stereo camera, wherein the second camera and a third camera are configured such that they capture the object three-dimensionally as a second stereo camera.

[0017] The same advantages and technical effects apply to the method according to the invention as have already been described in the context of the explanation of the advantages and technical effects of the gripping device according to the invention and its embodiments.

[0018] According to an advantageous embodiment of the invention, a point cloud representing the object is generated based on image data acquired by the first stereo camera and the second stereo camera. Based on the generated point cloud, the object can be reconstructed three-dimensionally, particularly with less occlusion.

[0019] Alternatively or additionally, the advantageous features or embodiments explained in connection with the gripping device according to the invention can also be used, alone or in combination, in the method according to the invention.

[0020] Further details and advantages of the invention will be explained below with reference to the exemplary embodiment shown in the drawing. Herein:

[0021] Fig. 1 shows an embodiment of a gripping device according to the invention for a robot for gripping an object; Fig. 2 shows a first image taken by a first stereo camera and a second image taken by a second stereo camera; and

[0022] Fig. 3 shows schematically three possible camera arrangements.

[0023] Fig. 1 shows an embodiment of a gripping device 1 according to the invention for a robot 3 for gripping an object 5. The gripping device 1 has a base body 7 and two fingers 9 movable relative to the base body 7. The two fingers 9 each have four finger segments 9', wherein the most distal finger segment 9'd of the two fingers 9 is designed as a contact and clamping element for gripping the object 5. The gripping device 1 further comprises a first camera 11 and a second camera 12, which are configured such that they form a first stereo camera 21 for three-dimensional capture of the object 5. In addition, the gripping device 1 has a third camera 13, wherein the second camera 12 and the third camera 13 are configured such that they form a second stereo camera 22 for three-dimensional capture of the object 5.

[0024] The three cameras 11, 12, 13 each have a lens, with the lenses of the cameras 11, 12, 13 being arranged in the same plane. The second camera 12 is arranged centrally between the first camera 11 and the third camera 13. All three cameras 11, 12, 13 are arranged between the fingers 9, on the base body 7, and in a common housing 10. They face directly toward the object 5.

[0025] The cameras 11, 12, 13 are arranged in a row. A light source (not shown in Fig. 1) for illuminating the object 5 can be arranged in the same row.

[0026] Fig. 2 schematically shows a first image 2T taken by the first stereo camera 21 and a second image 22' taken by the second stereo camera 22. Since the first stereo camera 21 and the second stereo camera 22 have different viewing angles relative to the object 5, the object 5 is each captured from a different perspective. As can be seen from the two image recordings 21', 22', depending on the viewing angle of the stereo camera 21, 22 in question, other areas of the object are obscured by the distalmost finger segments 9'd (occlusion). Consequently, a single stereo camera 21, 22 cannot capture the object 5, its surface, and orientation completely. Using two stereo cameras 21, 22, the object 5 can be captured in more detail. With an increasing number of additional stereo cameras, this can be captured more optimally.

[0027] Based on the first and second image recordings 2T, 22', the gripping device 1

[0028] - before grasping the object 5 - adopt an optimal alignment, especially of the fingers 9.

[0029] Fig. 3 shows three possible camera arrangements. All three camera arrangements represent alternative designs to the camera arrangement shown in Fig. 1, in which the three cameras 11, 12, and 13 are arranged in series.

[0030] Fig. 3a shows a triangular camera arrangement comprising the first, second, and third cameras 11, 12, 13. In the triangular camera arrangement according to Fig. 3a, the first camera 11 forms the first stereo camera 21 with the second camera 12. Furthermore, the second camera 12 forms the second stereo camera 22 with the third camera 13. Thus, the second camera 12 is used redundantly, i.e., it is part of the two stereo cameras 21, 22, thereby saving costs.

[0031] Fig. 3b further shows a quadrangular camera arrangement comprising the first, second, third and fourth cameras 11, 12, 13, 14. In the quadrangular camera arrangement according to Fig. 3b, the first camera 11 forms the first stereo camera 21 with the second camera 12. Furthermore, the second camera 12 forms the second stereo camera 22 with the third camera 13. Furthermore, the second camera 12 forms a third stereo camera 23 with the fourth camera 14. Thus, the second camera 12 is used multiple times, i.e., it is part of the three stereo cameras 21, 22, 23, thereby saving costs.

[0032] In addition, Fig. 3c shows a star-shaped camera arrangement, with the second camera 12 serving as the central camera and the first, third, and fourth cameras 11, 13, 14 serving as planetary cameras. The first, third, and fourth cameras 11, 13, 14 shown in Fig. 3c are evenly spaced with respect to a circumferential direction of a circle formed around the central camera 12. The second camera 12 forms

[0033] - with the first camera 11 a first stereo camera 21,

[0034] - with the third camera 13 a second stereo camera 22, and

[0035] - with the fourth camera 14 a third stereo camera.

[0036] In addition, the first camera 11 and the third camera 13 form a fourth stereo camera. The third camera 13 forms a fifth stereo camera with the fourth camera 14. Finally, the fourth camera 14 and the first camera 11 form a sixth stereo camera.

[0037] List of reference symbols

[0038] I Gripping device

[0039] 3 robots

[0040] 5 Object

[0041] 7 basic bodies

[0042] 9 fingers

[0043] 9' finger segments

[0044] 9'd Distal finger segments

[0045] 10 housings

[0046] II First Camera

[0047] 12 Second Camera

[0048] 13 Third Camera

[0049] 14 Fourth Camera

[0050] 21 First stereo camera

[0051] 21 ' First image capture

[0052] 22 Second stereo camera

[0053] 22' Second image capture

Claims

Patent claims 1. Gripping device (1) for a robot (3) for gripping an object (5), with a base body (7) and a plurality of fingers (9) movable relative to the base body (7), and with a first camera (11) and a second camera (12) which are configured such that they form a first stereo camera (21) for three-dimensionally capturing the object (5), characterized in that the gripping device (1) has a third camera (13), wherein the second camera (12) and the third camera (13) are configured such that they form a second stereo camera (22) for three-dimensionally capturing the object (5).

2. Gripping device (1) according to one of the preceding claims, characterized in that the cameras (11, 12, 13) each have a lens, wherein the lenses of the cameras (11, 12, 13) are arranged in the same plane or in planes parallel to one another.

3. Gripping device (1) according to one of the preceding claims, characterized in that the second camera (12) is arranged between the first camera (11) and the third camera (13).

4. Gripping device (1) according to one of the preceding claims, characterized by a fourth camera (14), wherein the second camera (12) and the fourth camera (14) are configured to form a third stereo camera (23) for three-dimensionally capturing the object (5).

5. Gripping device (1) according to one of the preceding claims, characterized by a fifth camera (15), wherein the third camera (13) and the fifth camera (15) are configured to form a fourth stereo camera (24) for three-dimensionally capturing the object (5).

6. Gripping device (1) according to one of the preceding claims, characterized in that the first camera (11), the second camera (12) and the third camera (13), as well as optionally the fourth camera (14) and / or the fifth camera (15), the base body (7).

7. Gripping device (1) according to claim 6, characterized by a finger camera arranged on a finger (9).

8. Gripping device (1) according to one of the preceding claims, characterized by a light source for illuminating the object (5).

9. Method for detecting an object (5) with a gripping device (1) for a robot (3) for gripping the object (5), which has a base body (7) and a plurality of fingers (9) movable relative to the base body (7), wherein a first camera (11) and a second camera (12) are configured such that they capture the object (5) three-dimensionally as a first stereo camera (21), characterized in that the second camera (12) and a third camera (13) are configured such that they capture the object (5) three-dimensionally as a second stereo camera (22).

10. The method according to claim 9, characterized in that a point cloud representing the object (5) is generated on the basis of image data acquired by the first stereo camera (21) and the second stereo camera (22).

Images