Method for determining a gripping pose for a robot

By using 2D environmental information to update gripping poses, the method addresses inefficiencies in existing 3D recapture methods, enabling faster and more reliable object handling in robot operations.

WO2026022139A1PCT designated stage Publication Date: 2026-01-29ROBERT BOSCH GMBH
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
PCT/EP2025/070977
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2024-07-23
Filing Date
2025-07-22
Publication Date
2026-01-29

AI Technical Summary

Technical Problem

Existing methods for determining gripping poses for robots require frequent recapture of 3D environmental information, which is time-consuming, and fail to account for changes in the work environment due to shifting objects, potentially leading to damage or inefficiency in pick-and-place operations.

Method used

A method that utilizes 2D environmental information to detect changes in the work environment, eliminating the need for frequent 3D recapture by comparing current and initial 2D images to update the gripping pose dataset, ensuring valid poses are used for robot movements.

Benefits of technology

This approach significantly reduces the time required for object picking operations by eliminating the need for repeated 3D data capture, allowing faster and more efficient grasping and movement of objects while ensuring pose validity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure EP2025070977_29012026_PF_FP_ABST
    Figure EP2025070977_29012026_PF_FP_ABST
Patent Text Reader

Abstract

The invention relates to a method for determining a gripping pose for a robot, said method being intended for use in the process of determining a movement sequence for the robot (100) in order to grip and / or move an object (140) in a work environment by means of an end effector (108) of the robot, having the steps of: checking (300) whether checking criteria of a specified checking criteria set (302) are met; and if the checking criteria are met, carrying out a gripping pose selection process (330), the gripping pose selection process having the steps of: providing (330) initial 2D environment information (332), which has been acquired from the work environment by means of a first sensor (124), the initial 2D environment information (332) having been captured as part of or in addition to 3D environment information (318); providing (350) current 2D environment information (352) acquired after acquiring the initial 2D environment information; determining (354), on the basis of the current 2D environment information and the initial 2D environment information, changes (356) in the work environment; determining (362), on the basis of a current and / or an initial pose data set (342) and the changes (356), an updated pose data set (364); and providing (366) the updated pose data set (364) in order to determine a gripping pose to be used and the movement sequence for the robot.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Method for determining a gripping pose for a robot

[0002] Description

[0003] The present invention relates to a method for determining a gripping pose for a robot, for use in determining a movement sequence for the robot to grasp and / or move an object in a working environment by means of an end effector of the robot, a computing unit and a computer program for carrying it out, and a robot.

[0004] Background of the invention

[0005] Robots, often also referred to as kinematics, can be used in production facilities, logistics, and other systems to, for example, transport or assemble parts. Typical types of such robots include Cartesian robots, SCARA robots, and articulated robots.

[0006] Disclosure of the invention

[0007] According to the invention, a method for determining a gripping pose for a robot, a computing unit and a computer program for its execution, as well as a robot with the features of the independent claims, are proposed. Advantageous embodiments are the subject of the dependent claims and the following description.

[0008] The invention relates generally to robots that can be used, for example, in production plants, logistics, or other facilities. Types of such robots, often also referred to as kinematics, include, for example, Cartesian robots, SCARA robots, and articulated robots. Such robots can be used to grasp and / or move objects (i.e., items or parts). A typical application is to remove an object from a box containing, for example, a large number of (identical or different) objects and, for example, place it somewhere or possibly assemble it. For this purpose, such a robot has, for example, a gripper or, more generally, an end effector. Such robots can also be referred to as manipulators or manipulation devices.

[0009] In order for the robot to grasp and / or move an object in a work environment, e.g. in a box, or in or on other carriers (e.g. a container or a pallet) using the end effector, the robot, in particular its end effector, must perform a movement sequence to move the end effector to the position of the object and then, e.g. after the object has been grasped, to move the end effector away from the position of the object again, e.g. to a desired storage position.

[0010] For a successful gripping operation, the (preferably) exact pose for the gripper – hereinafter also referred to as the gripping pose – must first be determined. Environmental information, especially 3D environmental information such as a point cloud of the work environment (scene), particularly of a support structure such as a box, can be acquired and provided for this purpose. Sensors such as lidar sensors (laser scanners), cameras, or other depth sensors can be used for this. The environmental information then also includes, in particular, one or more objects located in the work environment, especially in or on a support structure.

[0011] Based on the 3D environment information, especially the point cloud, gripping poses can be determined for each—or at least some—of the multiple objects. This can, for example, involve segmenting the point cloud to find suitable gripping poses.

[0012] In many use cases, there is a large number of objects, for example in a box, from which objects are to be taken one by one. For example, it might be intended that objects are taken from a box until the box is empty.

[0013] A crucial performance metric for such bin-picking applications is the number of objects that can be picked from the bin (or generally from a given quantity) per hour (or more generally per unit of time) ("picks-per-hour"). Very often, the cycle times of downstream processes are shorter than those of the picking process, which means that the downstream processes have to wait.

[0014] As has been shown, various factors play a role in such a "pick-and-place" or object picking operation. These include, for example, data acquisition, i.e., the generation of the point cloud or 3D environment information, the determination of the gripping poses (i.e., the evaluation of the point cloud or 3D environment information), the selection of the most suitable gripping pose (gripping strategy), the planning of the collision-free path for picking up and, if necessary, placing the object, as well as the robot's travel speed.

[0015] A list of possible gripping poses will subsequently be referred to generally as a pose data set. The pose data set contains information about several gripping poses, whereby, based on each gripping pose, one of several objects can be gripped and / or moved by the end effector.

[0016] It has also been shown that data acquisition, including the creation of a 3D image of the scene (i.e., the 3D environment information), accounts for a significant portion of the cycle time. Typically, several possible gripping poses are detected when evaluating a point cloud or 3D environment information. Therefore, it would initially seem conceivable that after capturing an object, further gripping poses should be available without re-acquiring the scene, i.e., without re-acquiring the 3D environment information.

[0017] However, when performing a grab, it cannot be guaranteed that neighboring objects have not shifted. Consequently, the already detected or determined gripping poses may become invalid. A further grab without information about the effects of the previous grab on the already detected gripping poses could therefore potentially lead to damage to objects, grippers, end effectors, or robots. Therefore, a new point cloud would need to be acquired with the 3D sensor after each grab; that is, 3D environmental information would need to be recorded.

[0018] Against this background, a method is now proposed that eliminates the need to capture 3D environmental information after each data acquisition, requiring only the acquisition of 2D environmental information. A comparison of the time required to create a point cloud with the exposure time for capturing, for example, a 2D color image shows that the former is orders of magnitude longer. Data acquisition with a high-resolution 3D sensor typically takes between one and two seconds, while a 2D color image can be captured in milliseconds.

[0019] First, it is checked whether the test criteria of a predefined set of test criteria are met. If the test criteria are met, a gripping pose selection process is carried out. The predefined set of test criteria includes, for example, at least one of the following test criteria.

[0020] One test criterion could be that a current pose record contains information about at least one gripping pose, where the current pose record corresponds to a pose record following the last robot movement sequence. In other words, at least one gripping pose should still be present; otherwise, it would not be possible to easily select another gripping pose from it. Another test criterion could be that no new set of objects (e.g., a new box of objects) has been provided from which an object can be grasped and / or moved using the robot's end effector. In this case, there would be no initial pose record at all. It is advantageous if both test criteria are present.

[0021] If the test criteria are met, the gripping pose selection process is carried out. This involves providing initial 2D environmental information, acquired from the work environment by a first sensor. This initial 2D environmental information is acquired as part of, or in addition to, 3D environmental information (e.g., a point cloud), which is acquired from the work environment by the first or a second sensor. Depending on the type of 3D sensor available, it may also be possible to acquire 2D environmental information. Alternatively, a separate 2D sensor, such as a (possibly simple) camera, can be used.

[0022] Furthermore, a current and / or an initial pose dataset can be provided. A pose dataset contains information about multiple gripping poses, whereby, based on each gripping pose, one of several objects can be grasped and / or moved by the end effector. Such an initial pose dataset is created, for example, whenever a new set of objects is received or when no existing gripping pose can be used. In particular, a (new) initial pose dataset can be created if no test criterion of the predefined set of test criteria is met. For this, (new) initial 2D environment information and new 3D environment information must also be captured or provided. This can then be referred to as the gripping pose capture process.

[0023] A current pose record can, for example, always be the last used pose record, possibly without a last used gripping pose.

[0024] The gripping pose selection process then further includes providing current 2D environmental information, which was acquired from the work environment by the first sensor after the initial 2D environmental information has been acquired. Specifically, the current 2D environmental information is acquired after at least one object has been gripped and / or moved by the robot's end effector following the acquisition of the initial 2D environmental information. Therefore, no 3D environmental information is acquired here, only 2D environmental information, such as a 2D image. As mentioned, this requires significantly less time.

[0025] Based on the current and initial 2D environment information, changes to the work environment are then determined, specifically changes in the positions and / or orientations of objects within the work environment. This can, for example, affect objects that have shifted or moved within a box, because, for instance, the end effector touched and moved other objects besides the one it was gripping during the last action.

[0026] Based on the current and / or initial pose data set and any changes, an updated pose data set is then determined. This updated pose data set, compared to the initial pose data set, includes only those gripping poses that are still valid, meaning they can be approached by the end effector. The updated pose data set is then used to determine the gripping pose to be used and the motion sequence for the robot.

[0027] Based on the updated pose dataset, the specific gripping pose to be used can then be determined, which in turn determines the movement sequence. This, in turn, allows for the determination of control information for moving the robot, which can then be provided or used to move the robot.

[0028] In summary, the proposed approach allows objects to be captured more quickly, as it eliminates the need to re-capture 3D environment information after each capture (or attempted capture), requiring only 2D environment information. This significantly reduces the time required. Furthermore, it has been shown that 2D environment information is sufficient only for detecting changes in the scene, while capture poses from the initial capture pose dataset can still be used.

[0029] In one embodiment, determining changes to the working environment based on the current and initial 2D environment information involves calculating a difference between the current and initial 2D environment information to obtain this difference information. Both the initial and current 2D environment information can, for example, each comprise an image. The changes are then determined based only on portions of the difference information where a value of a difference parameter lies within a predefined threshold range. This allows for simple and fast data processing to determine changes. The difference information can, for example, be generated as a grayscale image, where the difference parameter is then a grayscale intensity. Similarly, the difference information can be generated as a color image, where the difference parameter is then a color intensity.

[0030] In one embodiment, determining the updated poses dataset based on the current and / or initial poses dataset and the changes involves identifying, based on the changes, selected grid cells in a grid corresponding to the initial 2D environment information where a change has occurred. Then, grip poses are removed from the current or initial grip poses dataset based on the selected grid cells to obtain the updated poses dataset. This allows for simple and fast data processing to determine changes.

[0031] For more details and a concrete example of implementation, please refer to the character description.

[0032] A computing unit according to the invention (i.e., generally a system for data processing), e.g., a control unit or a control unit of a robot, or a central server or other computing system, is, in particular in terms of programming, equipped to carry out a method according to the invention.

[0033] The invention also relates to a robot configured to receive control information as described above. Furthermore, or alternatively, the robot comprises a computing unit according to the invention.

[0034] Furthermore, the robot includes, in particular, a control unit and a drive unit for moving the robot. In addition, the robot may have at least one sensor for capturing environmental information, e.g., a camera and / or a lidar sensor.

[0035] Implementing a method according to the invention in the form of a computer program or computer program product with program code for carrying out all method steps is also advantageous, as this incurs particularly low costs, especially if an executing control unit is already available for other tasks. Finally, a machine-readable storage medium is provided with a computer program stored on it as described above. Suitable storage media or data carriers for providing the computer program are, in particular, magnetic, optical, and electrical storage media, such as hard drives, flash memory, EEPROMs, DVDs, etc. Downloading a program via computer networks (Internet, intranet, etc.) is also possible. Such a download can be wired or wireless (e.g., via a WLAN network, a 3G, 4G, 5G, or 6G connection, etc.).

[0036] Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawing.

[0037] The invention is schematically illustrated in the drawing using an exemplary embodiment and is described below with reference to the drawing.

[0038] Brief description of the drawings

[0039] Figure 1 schematically shows a robot to illustrate the invention.

[0040] Figure 2 schematically shows a container to illustrate the invention.

[0041] Figure 3 schematically shows a process flow in one embodiment.

[0042] Figures 4a, 4b, 4c schematically show objects in different situations to illustrate the invention.

[0043] embodiment(s) of the invention

[0044] Figure 1 schematically illustrates a robot 100 to explain the invention. By way of example, the robot 100 has stand and arm components 102, 104, 106, so-called axes, which are each movably and movably connected by means of joints 112, 114.

[0045] Furthermore, the robot 100 has an end effector 108, e.g., a gripper. The end effector 108 is movably and reversibly connected to the arm component 106 by means of a joint 116.

[0046] Furthermore, the robot 100 has a drive system 120, shown only schematically here, as well as a computing unit 122 designed, for example, as a control or regulation unit. This allows the drive system 120 to be controlled, for example, using control information, in order to move the robot according to a desired sequence of movements. This can include, for example, moving the axes relative to each other by means of the joints, but also rotating the axes themselves, provided that appropriate drives are present.

[0047] It should be noted that the robot 100 is only used here as an example for illustrative purposes. A robot for grasping and / or moving objects in containers can also be designed differently, for example, using an end effector that can only move linearly along several different rails.

[0048] Furthermore, a box 132 is shown in a work environment 130, containing, for example, an object 140. The robot 100 can now be controlled, for example, in such a way that it grasps and / or moves the object 140 using the end effector 108, in particular also taking it out of the box and, for example, placing it somewhere else.

[0049] Furthermore, a first sensor 124 is shown as an example, which is, for instance, a 3D sensor, specifically a lidar sensor. This first sensor 124 can, for example, capture both 3D environmental information, such as a point cloud, and 2D environmental information, such as an image. However, a second sensor 126 is also shown, purely as an example. It is also possible that the first sensor 124 can only capture 2D environmental information, while the second sensor 126 can only capture 3D environmental information. In this case, the first and second sensors, and the environmental information they capture, should be registered with each other.

[0050] The first sensor 124 and, if applicable, the second sensor 126 can be arranged in the work environment in a suitable manner, e.g., on a ceiling and thus separately from the robot 100. However, the first sensor 124 and, if applicable, the second sensor 126 could also be part of the robot and, for example, be arranged on the arm component 106 or the end effector 108.

[0051] Using the first sensor 124 and, if applicable, the second sensor 126, the working environment 130, and in particular the box 132 and its interior, including the object 140, can now be detected. Based on the environmental information and sensor data obtained in this way, a movement sequence for the robot can be created to grasp and / or move the object 140 using the end effector 108, as will be explained in more detail below.

[0052] Figure 2 shows a box 232, comparable to box 132 in Figure 1, containing various objects 240, 241, 242, 243, and 244. A typical task for a robot is to remove as many objects as possible from the box. It is advantageous to first remove the easiest or safest object to grasp and so on, while adhering to certain guidelines, such as not covering forbidden areas with the end effector. It can also be seen that the objects can differ from one another, for example, being cuboid or cylindrical, and that the objects can also be randomly arranged.

[0053] An example movement sequence is shown for grasping object 240 and removing it from the box. This includes an approach path, or first part of the movement sequence 251, by which the end effector moves towards object 240 in order to grasp it. Additionally, a retraction path, or second part of the movement sequence 252, is shown, along which the end effector can be moved with object 240 out of the box.

[0054] Together, the first partial movement sequence 251 and the second partial movement sequence 252 form a complete movement sequence. It can be seen that the movement sequence is such that the end effector must be in a specific pose – a grasping pose – in order to properly grasp the object 240.

[0055] It can also be seen that, on the one hand, several gripping poses can be defined for the various objects, but also that when one object is gripped, other objects can be moved, for example. This means that some of the previously defined gripping poses may no longer be valid.

[0056] Figure 3 schematically illustrates the sequence of a process in one embodiment. Reference is also made to Figure 4, which shows a container or box and objects, and which will be used to explain various steps in more detail.

[0057] In step 300, it is first checked whether the test criteria of a predefined set of test criteria 302 are met. This is the case, for example, if a current pose record contains information about at least one gripping pose, i.e., if there are still entries in the current list of gripping poses, and if, in addition, no new set of objects has been provided, i.e., if no new box with objects is available.

[0058] If the test criteria are not met, a gripping pose detection process 310 is performed; if, however, the test criteria are met, a gripping pose selection process 330 is performed. Since the gripping pose detection process is generally also performed initially, i.e., when a new box containing objects is provided for the first time (the test criteria are then also not met), this should be explained first.

[0059] In step 312, initial 2D environment information 314 (possibly new), e.g., as a 2D image, is provided, and in step 316, 3D environment information 318 (possibly new) is provided. As mentioned, a single sensor or two different sensors can be used for this purpose.

[0060] Figure 4a shows an example of a 2D image depicting several objects in a box 432. An object 440, to be grasped later, is specifically marked. In step 320, an initial (possibly new) pose data set 322 is determined based on at least the 3D environment information. This can, for example, involve segmenting the objects, based on which the grasping poses and thus the initial pose data set (i.e., a list of grasping poses) are then determined.

[0061] In contrast, during the gripping pose selection process 330, initial 2D environment information 334 is provided in step 332. This initial 2D environment information 332 refers in particular to the initial 2D environment information obtained during the last gripping pose detection process, for example, the 2D environment information 314.

[0062] Furthermore, for example, in step 340, an initial pose record 342 is provided. This initial pose record 342 is specifically the initial pose record obtained during the last executed gripping pose recording process, i.e., pose record 322. As mentioned, a current pose record can also be provided additionally or alternatively.

[0063] Both the initial 2D environment information 332 and the initial pose data set 342 can, for example, be stored or cached in a memory, so that they can be accessed in the following steps.

[0064] In step 350, current 2D environmental information 352, e.g., as a 2D image, is provided. This information was acquired from the working environment by the first sensor after the initial 2D environmental information 332 has been acquired. For this purpose, the first sensor can be controlled, e.g., to take a picture.

[0065] Figure 4b shows an example of a 2D image depicting several objects in a box 432. In contrast to the 2D image in Figure 4a, the object 440 is no longer present; it was, for example, picked up and removed from the box 432 in a previous movement or grasping cycle. In step 354, based on the current 2D environment information 352 and the initial 2D environment information 342, changes 356 to the work environment, in particular changes to the positions and / or orientations of objects in the work environment, specifically, for example, in the box, are determined.

[0066] For example, in step 358, a difference is calculated between the current 2D environment information and the initial 2D environment information to obtain difference information 360; this could, for example, be a grayscale image. The changes 356 are then determined based only on parts of the difference information where a value of a difference quantity lies within a predefined threshold range.

[0067] Figure 4c shows an example of a difference image, which corresponds to a difference between the 2D images from Figures 4a and 4b.

[0068] To analyze for differences, the two 2D images can first be converted into, for example, an 8-bit grayscale image to accelerate further data processing. Detecting changes in pixel grayscale is usually sufficient to identify regions in the image that do not match between the two images. The two images can then be subtracted from each other (i.e., the difference is calculated).

[0069] In the resulting image (difference image), those pixels whose absolute gray value lies within adjustable or predefined threshold pairs (minimum, maximum; this corresponds to a threshold range) can now be further analyzed, meaning they differ between the two images. The threshold pairs allow the algorithm's sensitivity to differences between the two images to be adjusted. If gray values ​​are insufficient to reliably detect changes in the 2D images, 2D color images can also be used, and their color channels can be examined for changes between the two images. To increase the algorithm's robustness, the proximity relationships of the detected pixels can be analyzed in a further step. Adjacent pixels can be grouped into regions. If the area of ​​such a region exceeds an adjustable minimum size, it is marked as a change.

[0070] In step 362, an updated poses record 364 is determined based on the initial poses record 342 and / or the current poses record, as well as based on the changes 356. For this purpose, for example, selected grid cells in a grid corresponding to the initial 2D environment information, in which a change has occurred, are determined based on the changes. Then, gripping poses can be removed from the initial or current gripping poses record based on the selected grid cells to obtain the updated poses record.

[0071] A grid with an adjustable number of grid points or grid cells is overlaid on the 2D image (difference image). All grid surfaces that overlap with changes can be marked with a change flag.

[0072] Figure 4c shows such a grid with 450.

[0073] In a further step, those gripping poses can then be filtered from the current pose data set used for the last capture (or from the initial pose data set) that have (xy) coordinates lying within the areas with a change flag. The z-coordinate (for a possible orientation of the coordinate axes, see Figure 2) does not need to be considered here, since in the 2D image all gripping poses are projected onto the corresponding 2D surfaces and therefore all gripping poses within this surface become invalid if this surface has the change flag.

[0074] Since the 2D image provides (x,y) pixel coordinates, but the (x,y,z) coordinates of the gripping poses are points in 3D space, the corresponding points in 3D space can now be extracted for the (x,y) pixel coordinates. Because the 2D image and the 3D point cloud are registered to each other (so-called "ordered point cloud"), the following procedure can be used.

[0075] If a regular grid with a known number of grid points has been overlaid on the 2D image, the (x,y) pixel coordinates of each surface, e.g., the upper left and lower right corners, can be determined using the change flag. The point with the corresponding pixel coordinate can then be selected in the most recently acquired point cloud (i.e., the initial 3D environment information). If the point cloud contains a valid spatial point at that location, then its (x,y) spatial coordinate is the corresponding (x,y) coordinate in 3D space to the (x,y) pixel coordinate. If no valid spatial point exists at that location in the point cloud, e.g., because the 3D sensor could not triangulate a spatial point there, then the next valid spatial point can be searched for in the point cloud, e.g., spirally around this point, and its (x,y) spatial coordinate can then be assigned to the corresponding (x,y) pixel coordinate.

[0076] Now all gripping poses can be filtered out from the list whose (x,y)-space coordinate lies within the extracted (x,y)-space coordinates between the upper left and lower right points of the surfaces with the change flag.

[0077] In step 366, the updated pose data set is then made available for determining a gripping pose to use and the movement sequence for the robot.

[0078] In step 370, the gripping pose to be used is determined based on the updated pose data set, allowing the robot's movement sequence to be defined, as already explained in Figure 2. Similarly, in the case of the gripping pose detection process, a gripping pose can be determined, but this time not based on the updated pose data set 364, but rather on the (new) initial pose data set 322. In both cases, the most promising gripping pose from the list for a successful pick-up can be selected, and the path for picking up and, if necessary, placing the object can be planned.

Claims

Claims 1. Method for determining a gripping pose for a robot, for use in determining a movement sequence for the robot (100) to grasp and / or move an object (140) in a work environment by means of an end effector (108) of the robot, comprising: Check (300) whether test criteria of a given set of test criteria (302) are met; and if the test criteria are met, perform a gripping pose selection operation (330), wherein the gripping pose selection operation comprises: Providing (330) initial 2D environment information (332) acquired from the work environment by means of a first sensor (124), wherein the initial 2D environment information (332) has been acquired as part of or in addition to 3D environment information (318), wherein the 3D environment information (318) has been acquired from the work environment by means of the first or a second sensor ...J; Providing (350) current 2D environment information (352) that has been acquired from the work environment by means of the first sensor (124) and after acquiring the initial 2D environment information, wherein the current 2D environment information has been acquired in particular after at least one object has been grasped and / or moved by means of the end effector of the robot after acquiring the initial 2D environment information; Determine (354), based on the current 2D environment information and the initial 2D environment information, changes (356) in the work environment, in particular changes in the positions and / or orientations of objects in the work environment; Determine (362), based on a current and / or an initial poses record (342) and the changes (356), an updated poses record (364); and Providing (366) the updated pose data set (364) for determining a gripping pose to use and the movement sequence for the robot.

2. Method according to claim 1, wherein the specified set of test criteria comprises at least one of the following test criteria: the current pose data set comprises information about at least one gripping pose, wherein the current pose data set corresponds to a pose data set after a last performed movement sequence for the robot; no new set of objects has been provided from which an object is to be gripped and / or moved by means of the end effector of the robot.

3. Method according to claim 1 or 2, wherein the initial and the current 2D environment information each comprise an image.

4. Method according to any of the preceding claims, wherein determining (354), based on the current 2D environment information and the initial 2D environment information, comprises: Calculating a difference between the current 2D environment information and the initial 2D environment information to obtain difference information; and Determining the changes based only on parts of the difference information where a value of a difference quantity lies within a specified threshold range.

5. Method according to claims 3 and 4, wherein the difference information is formed as a grayscale image, and wherein the difference quantity is a grayscale intensity.

6. Method according to claims 3 and 4, wherein the difference information is formed as a color image, and wherein the difference magnitude is an intensity of a color.

7. Method according to any one of claims 3 to 6, wherein determining, based on the initial poses data set and the changes, the updated poses data set comprises: Determine, based on the changes, selected grid cells in a grid corresponding to the initial 2D environment information where a change has occurred; and Remove gripping poses from the current or initial gripping poses dataset based on the selected grid cells to obtain the updated poses dataset.

8. A method according to any of the preceding claims, comprising, if the test criteria are not met: performing a gripping pose detection process, wherein the gripping pose detection process comprises: Providing (312, 316) new initial 2D environment information (314) and new 3D environment information (318); and Determine (320) a new initial pose record (322).

9. A method according to any of the foregoing claims, further comprising: Determine, based on the updated pose data set, the gripping pose to use, and Determining the sequence of movements, taking into account the gripping pose to be used.

10. The method of claim 9, further comprising: Determine, based on the movement sequence, control information for moving the robot, and Providing control information and / or moving the robot based on the control information.

11. Computing unit comprising means for carrying out the method according to one of the preceding claims.

12. Robot configured to receive control information determined by a method according to claim 10, and / or with a computing unit according to claim 11, and with a drive system and a control or regulation unit for controlling the drive system, with one or more, in particular several selectable, end effectors for grasping and / or moving a Object, and preferably with at least one sensor, in particular a camera, for capturing environmental information of a working environment.

13. Computer program comprising commands which, when the program is executed by a computer, cause it to execute the method according to claims 1 to 10.

14. Computer-readable storage medium on which the computer program according to claim 13 is stored.

Citation Information

Patent Citations

  • Device and method for removing loosely stored objects by a robot

    DE102013012068A1

  • Object receiving device and method for receiving an object

    DE102013018222A1

  • Device and method for picking up randomly distributed objects with a robot

    DE102014016033A1

  • ADAPTIVE PLANNING OF ACCESS FOR PICKING UP CONTAINERS

    DE102021107568A1