Method for determining a movement sequence for a robot

Abstract

The invention relates to a method for determining a movement sequence (250) for a robot in order to grip and / or move an object in a working environment by means of an end effector of the robot, comprising: providing (206) environment information (202, 204) which has been detected from the working environment by means of at least one sensor; determining (210), based on the environment information, a pose for one or each of a plurality of objects in the working environment in order to obtain an object pose data set (228); determining (230), based on the object pose data set, one or more poses for one or each of a plurality of end effectors, in each case for the one or at least one of the plurality of objects of the object pose data set, in order to obtain an end effector pose data set (236); determining (240), based on the end effector pose data set, the movement sequence (250) for the robot in order to grip and / or move the object in the working environment by means of the one or more end effectors of the robot; and providing (252) the movement sequence (250), and in particular causing the robot to carry out the movement sequence.

Description

[0001] Method for determining a movement sequence for a robot

[0002] Description

[0003] The present invention relates to a method for determining a movement sequence for a robot in order to grasp and / or move an object in a working environment by means of an end effector of the robot, a computing unit and a computer program for its execution, and a robot.

[0004] Background of the invention

[0005] Robots, often also referred to as kinematics, can be used in production facilities, logistics, and other systems to, for example, transport or assemble parts. Typical types of such robots include Cartesian robots, SCARA robots, and articulated robots.

[0006] Disclosure of the invention

[0007] According to the invention, a method for determining a movement sequence for a robot, a computing unit and a computer program for its execution, as well as a mobile device with the features of the independent claims, are proposed. Advantageous embodiments are the subject of the dependent claims and the following description.

[0008] The invention relates generally to robots that can be used, for example, in production plants, logistics, or other facilities. Types of such robots, often also referred to as kinematics, include, for example, Cartesian robots, SCARA robots, and articulated robots. Such robots can be used to grasp and / or move objects (i.e., items or parts). A typical application is to remove an object from a box containing, for example, a large number of (identical or different) objects and, for example, place it somewhere or possibly assemble it. For this purpose, such a robot has, for example, a gripper or, more generally, an end effector. Such robots can also be referred to as manipulators or manipulation devices.

[0009] In order for the robot to grasp and / or move an object in a work environment, e.g. in a box, using the end effector, the robot, in particular its end effector, must perform a movement sequence to move the end effector to the position of the object and then, e.g. after the object has been grasped, to move the end effector away from the position of the object again, e.g. to a desired storage position.

[0010] The movements or tasks that the robot is to perform can be programmed, for example, by an application engineer (technician or engineer). All positions that the robot is to move to in the specific application must then be specified by the application engineer or programmer.

[0011] If the points in the application are static, i.e., points that do not change during the application's runtime, the applicator can precisely move the robot to them using a handheld control device and transfer the position to the program.

[0012] Dynamic points, meaning points that change during application runtime, can be defined using sensors, especially 3D sensors. Optical sensors are particularly suitable for this purpose. Dynamic points are often gripping points on moving objects. The sensor can capture the environment or workspace, for example, as a point cloud. This captured environmental information (data) can then be analyzed, for example, using a software component, to determine or locate the object in 3D space, including its position and orientation (generally and in combination also referred to as pose).

[0013] Especially in modern systems, 3D sensors can be used to pick parts from boxes or pallets, or to lift them from conveyor belts. This technology significantly simplifies the mechanical aspects of the system, as precise mechanical locking and pre-singling of objects are no longer necessary. This can make the entire system more cost-effective, since standardized 3D sensors and software components can be used. Adaptations for new products or objects, or the reuse of components, are possible with minimal effort. Particularly in intralogistics, parts handling, and especially order picking, there is a very large, sometimes unmanageable, variety of different objects with varying surfaces, shapes, packaging materials, and other physical properties.

[0014] Against this background, a way is proposed to meet these requirements, specifically through adaptive software that uses intelligent strategies to decide which objects should be removed from a box, how, and when. In particular, it is proposed to generically structure and optimize the complex movement sequence of the robot with its end effector for grasping or moving an object—for example, removing an object from a box—in order to simplify and simultaneously optimize the selection, the end effector (gripper), and the gripping of the objects to be grasped. The individual steps can be implemented as modules in the software, so that different modules are used sequentially during application, in which the various steps are executed, at least as far as applicable.

[0015] This process begins by providing environmental information captured from the work environment by at least one sensor. This includes, in particular, a camera (especially a 2D camera) and / or a Time-of-Flight (TOF) sensor, another depth sensor, or another 3D sensor. The environmental information then comprises, in particular, 2D environmental information and / or 3D environmental information. While the 2D environmental information can be images, such as color images, grayscale images, or black-and-white images, the 3D environmental information can include point clouds or depth maps, as well as potentially 3D images. The sensors can be, for example, mounted on the robot or be part of the robot; alternatively, such sensors can be provided and used separately to obtain the best possible environmental information about the work environment, such as...to be able to detect a box containing objects.

[0016] Based on the environmental information, a pose is then determined for one or more of the objects in the workspace. Depending on whether the environmental information includes 2D and / or 3D data, for example, only 3D environmental information might be processed initially, followed by only 2D environmental information. It is also conceivable that 2D and 3D environmental information are processed (together). This can also be done differently for different parts of the environmental information.

[0017] Based on the object pose data set, one or more end effector poses are then determined for one or at least one of the multiple objects in the object pose data set to obtain an end effector pose data set. It should be noted that the robot can also have multiple end effectors, from which a suitable one can be selected, for example, depending on the type of object to be grasped. In the case of multiple end effectors, one or more poses can be determined for each of them.

[0018] Based on the end-effector pose data set, the motion sequence for the robot is then determined to grasp and / or move an object in a work environment using one or more of the robot's end effectors. The resulting motion sequence is then deployed; specifically, the robot is instructed to execute it. For example, control information for moving the robot can be determined based on the motion sequence and then deployed, or used to move the robot.

[0019] This approach is particularly advantageous for the automated picking of objects at order picking workstations in intralogistics. The automatic selection of an end effector and the automatic generation of various grips based on the end effector's geometry increase both the success rate of picking objects and the success rate of completely emptying boxes. At order picking workstations, a wide variety of objects are removed from boxes. Product portfolios often comprise more than 100,000 different items.

[0020] As mentioned, the environmental information can include 2D and / or 3D environmental information, which can then be processed in various ways. It should be noted that the 2D and / or 3D environmental information, or corresponding 2D and 3D images, are recalibrated with respect to each other, meaning that their relative poses, especially in relation to the respective sensor, are known.

[0021] In one embodiment, processing only 3D environment information includes determining, based on the 3D environment information, the position and / or orientation for one or each of the multiple objects; and / or assigning, based on the 3D environment information, one or at least one of the multiple objects to an object class, wherein the object pose record includes information about the assigned object class.

[0022] In one embodiment, processing only 2D environment information includes identifying, based on the 2D environment information, a part, e.g., an edge, of one or at least one of the multiple objects; and / or assigning, based on the 2D environment information, one or at least one of the multiple objects to an object class, wherein the object pose record includes information about the assigned object class.

[0023] In one embodiment, the processing of 2D and 3D environment information includes classifying and / or segmenting, based on the 2D environment information, one or at least one of the multiple objects; and determining, based on the 3D environment information, the pose for one or at least one of the multiple classified and / or segmented objects.

[0024] Separate processing of 3D and 2D environment information allows for efficient and fast processing of each specific type of information. Combined processing of 3D and 2D environment information, on the other hand, allows for more accurate information, such as more precise poses.

[0025] In one embodiment, at least one of the following steps is provided to obtain the object-pose data set. For example, based on at least one object selection criterion, one or more selected objects are determined from the one or more objects. The object-pose data set then only includes poses for the one or more selected objects. This allows, for example, poses of objects that lie outside a valid or predefined range, or that exhibit, for example, excessive tilt, to be disregarded. Alternatively, a sequence can be assigned to each of the multiple objects in the object-pose data set, according to at least one object ordering criterion. Here, for example, higher-ordered poses can be sorted to the front so that they are given priority later, as such objects are, for example, easier to grasp.

[0026] In one embodiment, to obtain the end-effector pose data set, at least one of the following steps is provided. For example, based on at least one end-effector selection criterion, one or more selected poses are determined for one or at least one of the multiple end-effectors, specifically from the one or more poses of the respective end-effector. The end-effector pose data set then comprises only the one or more selected poses for the respective end-effector. Similar to the poses of objects, poses for an end-effector that are difficult to approach can be disregarded. Alternatively, a sequence can be assigned to each of the multiple poses for one or at least one of the multiple end-effectors in the end-effector pose data set, according to at least one end-effector ordering criterion. Here, for example,Poses that are higher up are sorted to the front so that they can be processed preferentially later, as such poses are easier to reach for an end effector, for example.

[0027] In one embodiment, determining the motion sequence includes determining an axis configuration data set, particularly based on inverse kinematics. This axis configuration data set comprises an axis configuration of the robot for one or at least one of the multiple poses of one or at least one of the multiple end effectors. Preferably, a potential motion sequence is also determined for one or at least one of the multiple axis configurations. In the case of a potential motion sequence, this is used as the final motion sequence, or in the case of multiple potential motion sequences, one of them is determined as the motion sequence to be used. The multiple potential motion sequences can apply to different poses for one end effector, or to different end effectors, each with one or more different poses. The selection of the final motion sequence thus potentially includes...also a choice of end effector.

[0028] The motion sequence can, in particular, comprise a first sub-sequence and a second sub-sequence. The first sub-sequence involves a movement of one or more end effectors toward a position of the object, and the second sub-sequence involves a movement of one or more end effectors away from the object's position. This also applies accordingly to potential motion sequences.

[0029] A computing unit according to the invention (i.e., generally a system for data processing), e.g., a control unit or a control unit of a robot, or a central server or other computing system, is, in particular in terms of programming, equipped to carry out a method according to the invention.

[0030] The invention also relates to a robot configured to receive control information as described above. In addition, or alternatively, the robot comprises a computing unit according to the invention. Furthermore, the robot comprises, in particular, a control unit and a drive unit for moving the robot. The robot may also have at least one sensor for acquiring environmental information, e.g., a camera and / or a 3D sensor.

[0031] Implementing a method according to the invention in the form of a computer program or computer program product with program code for carrying out all method steps is also advantageous, as this incurs particularly low costs, especially if an executing control unit is already available for other tasks. Finally, a machine-readable storage medium is provided with a computer program stored on it as described above. Suitable storage media or data carriers for providing the computer program are, in particular, magnetic, optical, and electrical storage media, such as hard drives, flash memory, EEPROMs, DVDs, etc. Downloading a program via computer networks (Internet, intranet, etc.) is also possible. Such a download can be wired or wireless (e.g., via a WLAN network, a 3G, 4G, 5G, or 6G connection, etc.).

[0032] Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawing.

[0033] The invention is schematically illustrated in the drawing using an exemplary embodiment and is described below with reference to the drawing.

[0034] Brief description of the drawings

[0035] Figure 1 schematically shows a robot to illustrate the invention.

[0036] Figure 2 schematically shows a process flow in one embodiment.

[0037] Figures 3 to 6 schematically show objects and robots to illustrate the invention.

[0038] embodiment(s) of the invention

[0039] Figure 1 schematically illustrates a robot 100 to explain the invention. By way of example, the robot 100 has stand and arm components 102, 104, 106, so-called axes, each of which is movably and reversibly connected by means of joints 112, 114. Furthermore, the robot 100 has an end effector 108, e.g., a gripper. The end effector 108 is movably and reversibly connected to the arm component 106 by means of a joint 116.

[0040] Furthermore, the robot 100 has a drive system 120, shown only schematically here, as well as a computing unit 122 designed, for example, as a control or regulation unit. This allows the drive system 120 to be controlled, for example, using control information, in order to move the robot according to a desired sequence of movements. This can include, for example, moving the axes relative to each other by means of the joints, but also rotating the axes themselves, provided that appropriate drives are available.

[0041] Furthermore, a box 132 is shown in a work environment 130, containing, for example, an object 140. The robot 100 can now be controlled, for example, in such a way that it grasps and / or moves the object 140 using the end effector 108.

[0042] Furthermore, an example sensor 124 is shown, which could be, for example, a camera or a 3D sensor. Both types of sensors can also be used. Likewise, several identical sensors can be used. The sensor 124 can, for example, be arranged in a suitable manner in the work environment, e.g., on a ceiling and thus separately from the robot 100. However, the sensor 124 could also be part of the robot and, for example, be arranged on the arm component 106 or the end effector 108.

[0043] The sensor 124 can now detect the work environment 130, and in particular the box 132 and its interior, including the object 140. Based on this environmental information, a movement sequence can be created for the robot to grasp and / or move the object 140 using the end effector 108. This will be explained in more detail below.

[0044] Figure 2 schematically illustrates the sequence of steps in one embodiment of a method. It explains various steps which, when implemented in software (e.g., on the computing unit 122 according to Figure 1 or elsewhere), can be provided as modules, as will be explained in more detail below.

[0045] Reference is also made to figures 3 to 6, which show objects and / or robots, and which are used to explain various steps in more detail.

[0046] In step 200, or module 200, environmental information is captured from the work environment using at least one sensor. Specifically, this could be, for example, a scene capture of the box or the work environment in which objects are to be detected. The sensor captures, for example, one or more 3D and / or 2D images—generally referred to as 3D environmental information (202) and 2D environmental information (204)—which are processed in subsequent steps or modules. The 2D image might be, for example, an RGB image or a grayscale image. The 3D image might be, for example, a point cloud or depth image. Typically, the 2D and 3D images are calibrated to each other, meaning they are related to one another.

[0047] This environmental information, thus captured, is then made available for further use or processing in step 206.

[0048] In step 210, based on the environment information, a pose is determined for one or each of several objects in the work environment in order to obtain an object pose record 228.

[0049] Step 210 can comprise three different steps or modules: processing only 3D environment information in step or module 212, processing only 2D environment information in step or module 214, and processing 2D and 3D environment information in step or module 216.

[0050] Step or module 212 can include determining the pose for one or each of the multiple objects based on the 3D environment information, and assigning one or at least one of the multiple objects to an object class based on the 3D environment information, with the object pose record then including information about the assigned object class.

[0051] Specifically, 3D images can be processed and instantiated in image processing workflows. Such workflows can range from simple surface segmentation to complex algorithms using neural networks that classify and segment complex objects. The result is, for example, a vector containing object poses. These poses can also be referred to as vision poses, as they are derived from visual environmental information. These poses represent all localized objects. Optionally, algorithm-specific parameters, such as surface information, can be assigned to the poses and / or the objects in the object-pose dataset.

[0052] Step or module 214 may include identifying a part of one or at least one of the multiple objects based on the 2D environment information, or assigning one or at least one of the multiple objects to an object class based on the 2D environment information, with the object pose record including information about the assigned object class.

[0053] Specifically, 2D images can be processed and instantiated in image processing workflows. Such workflows might include barcode recognition, edge detection of objects, or a classifier that recognizes different object classes. Algorithms (with and without AI) can also be used to extract 6D poses. This can be achieved, for example, using shape-based CAD matchers or AI-based methods, such as generating a synthetic depth image that enables 3D point cloud processing.

[0054] Step or module 216 may include classifying and / or segmenting one or at least one of the multiple objects based on the 2D environment information, and determining the pose for one or at least one of the multiple classified and / or segmented objects based on the 3D environment information.

[0055] Specifically, in hybrid image processing, combined images or data can be instantiated from 2D and 3D images. For example, objects are classified and segmented in the 2D image and then cut out accordingly in the 3D point cloud or depth image. This approach allows, for instance, the segmentation of complex objects in the 3D image. The result is, for example, a vector containing object poses. These poses can also be referred to as vision poses, as they are derived from visual environment information. These poses represent, in particular, all localized objects. Optionally, algorithm-specific parameters, such as area information, can be assigned to the poses and / or the objects in the object-pose dataset.

[0056] Using steps or modules 212, 214, 216 - depending on the situation, for example, only some of them may be used - a preliminary object-pose data set 218 is then obtained, which includes poses and, if applicable, the other information mentioned about the recognized objects.

[0057] To obtain the (final) object pose record 228, further steps or modules 222 and / or 224 may be required. Step or module 222 includes, for example, determining one or more selected objects based on at least one object selection criterion, and is taken from the one or more objects (i.e., from the preliminary object pose record 218), whereby the (final) object pose record 228 then only contains poses for the one or more selected objects.

[0058] Specifically, various processes and / or filters can be instantiated here. For example, poses from steps or modules 212, 214, and 216 that lie outside a valid or otherwise predefined range can be filtered out. For instance, all poses of objects exhibiting a strong tilt relative to the bottom of the box can be filtered out. Figure 3 shows a box 332, which can correspond, for example, to box 132 in Figure 1. Box 332 contains four objects 340, 341, 342, and 343, which can have any pose. 350 represents a normal to a bottom 334 of box 332 (i.e., a box bottom), and 352 represents a normal 352 to a surface of object 340. This normal 352 can represent the pose of object 340.

[0059] The number 354 now denotes a difference or difference vector between the normals 350 and 352, which is a measure of the tilt of object 340 relative to the box bottom 332. In this way, poses with a very strong tilt relative to the box bottom, such as object 343, can be filtered out. Suitable threshold values ​​can be specified here.

[0060] Step or module 224 includes, for example, assigning a sequence to each of the multiple objects in the object-pose data record according to at least one object ordering criterion. This could simply be reordering the entries in a list corresponding to the object-pose data record. A concrete example is sorting the poses according to their height within the box.

[0061] In the example in Figure 3, object 340 would be the one in first place, followed by objects 341 and 342. Object 343 would already be sorted out at this point due to the tilting.

[0062] The result is, for example, a vector containing the remaining and possibly rearranged poses of objects, i.e., the (final) object-pose data set 228.

[0063] In step or module 230, based on the object-pose data set 228, one or more poses are determined for one or each of several end effectors, specifically for one or at least one of the objects in the object-pose data set 228, in order to obtain an end effector pose data set 236. Specifically, in this step or module, various processes can be instantiated that serve to convert object poses (vision poses) into end effector poses (gripping poses). Depending on the image processing algorithm used and the available end effectors (grippers), different end effector poses can be generated.

[0064] For example, a cylinder fitting algorithm outputs a vision pose that is axially aligned in the z-direction.

[0065] Figure 4 shows an object 440 on the left, which has the shape of a cylinder or at least approximates a cylinder. Starting from a Cartesian coordinate system shown in the lower left, the z-axis in the indicated pose 452 of object 440 is aligned along an axis of the cylinder.

[0066] The required gripping poses, however, lie on the cylinder's surface. Using a vision-pose parameter set that includes the radius of the detected cylinder for the cylinder-fit algorithm, the gripping positions can be calculated, for example.

[0067] Figure 4 on the right shows an example of an end effector or gripper 408 (or part thereof) next to the object 440. The radius can be used, for example, to select a suitable tool for gripping the cylinder. In other words, if multiple end effectors are available, the one that can grip the object with the specified radius of the cylinder's surface can be selected (possibly even later). The generated gripping poses include information on the vision pose, the generated gripping pose (pose of the end effector), the end effector itself, and, for example, cost information describing the grip's quality. The grip refers to how precisely the end effector can grasp the object, i.e., how accurately the end effector can be positioned.For example, handles located on the top of the cylinder's surface have a higher quality than those tilted at the side. Compared to pose 452 of the object, pose 460 of the end effector 408 is modified again; it can be obtained, for example, by translation and rotation. The cylinder is axially symmetric; therefore, any number of handles orthogonal to the surface can be calculated using the known radius. With the cylinder length information (transmitted by the vision), a translational shift is also conceivable to generate further handles on the surface.

[0068] Specifically, further processes and filters can also be instantiated in this module. For example, processes can be used to generate secondary gripping positions using the selected end effector or gripper from the previous step, thus expanding the number of possible poses of the end effector.

[0069] For example, if it is a rotationally symmetrical gripper, correspondingly rotationally symmetrical handles can be generated, which ultimately increases the success rate of the gripping action.

[0070] Figure 5 shows an object 540 with an end effector 508, which can correspond to the object 440 with end effector 408 according to Figure 4. The 562 indicates a rotation of the end effector 508 about its axis. Depending on the degree of rotation, e.g., in specific angular increments, several new poses can be created. Processes that generate the gripping poses generate them, for example, based on the gripping poses that are passed when the module is called.

[0071] Furthermore, for example, a step or module 232 may be provided in which, based on at least one end-effector selection criterion, one or more selected poses are determined for one or at least one of the several end-effectors, specifically from the one or more poses of that one or more end-effectors. The end-effector pose record 236 then comprises only the one or more selected poses for the one or at least one of the several end-effectors. This may also, for example, include a step or module 234 in which an order is assigned to each of the several poses for the one or at least one of the several end-effectors in the end-effector pose record 236 according to at least one end-effector ordering criterion.

[0072] To filter the generated gripping poses, filter processes can be instantiated to, for example, remove gripping poses with excessively high costs or excessive tilt relative to the box bottom. Processes can also be implemented to sort the gripping poses or evaluate or filter them according to specific criteria such as height, gripping area, or localization accuracy.

[0073] The result is a vector containing the remaining gripper poses and their costs, i.e., the end effector pose dataset 236.

[0074] In a step or module 240, based on the end effector pose data set 236, the motion sequence 250 for the robot is determined to grasp and / or move the object in the work environment using one or more end effectors of the robot.

[0075] This can, for example, include a step or module 242 in which an axis configuration data set 244 is determined, in particular based on inverse kinematics. The axis configuration data set includes an axis configuration of the robot for one or at least one of the several poses of one or at least one of the several end effectors.

[0076] Specifically, for example, the corresponding specific axis configurations of the robot used for each gripping pose can be calculated using inverse kinematics. This allows for consideration of the possible movements, and in particular rotations of the joints as mentioned in Figure 1.

[0077] Gripping poses that the robot cannot physically reach have no solution in the inverse kinematic calculation and are therefore automatically omitted. The result is a vector containing the valid robot axis configurations at the gripping points, i.e., axis configuration dataset 244. The vector elements include, for example, the robot axis configuration of all movable axes at the gripping point as well as the corresponding gripping pose.

[0078] This can also include a step or module 246 in which a potential motion sequence 248 is determined for one or at least one of the several axis configurations 244. In the case of (only) one potential motion sequence, this will be used as motion sequence 250; in the case of several potential motion sequences, one of them will be determined as motion sequence 250, e.g., according to a quality criterion that determines how or which object can be grasped most quickly.

[0079] Specifically, processes or filters can be instantiated to, for example, filter out unwanted robot axis configurations. Specifically, axis configurations where a particular axis is positioned by more than + / - 90° (e.g., relative to a standard position) should be excluded. This could be, for example, axis 106 as shown in Figure 1. Here, it is evident that the same pose of the end effector, as shown, could also be achieved with axis 106 rotated by 180°; however, this may be undesirable for various reasons, such as increasing the load on the robot or restricting subsequent movements.

[0080] The motion sequence 250 can comprise a first partial motion sequence and a second partial motion sequence, wherein the first partial motion sequence comprises a movement of one or one of the several end effectors towards a position of the object, and wherein the second partial motion sequence comprises a movement of one or one of the several end effectors away from the position of the object.

[0081] For each available robot axis configuration at the gripping point, a collision-free movement using the end effector into the box to pick up the object and out of the box can be calculated, provided that physical reach is possible from the given starting position. Specifically, processes for calculating the approach path (first part of the movement sequence) can be instantiated in a module. Such processes serve to determine segments of the approach path or the entire collision-free approach path to the specified gripping point. Information about the robot used, the gripper, and interfering contours in the system and on the robot should be taken into account to ensure a collision-free path.

[0082] Accordingly, processes for calculating the retraction path (second part of the motion sequence) can be instantiated, for example, in a separate module. Such processes serve to determine segments of the retraction path or the entire collision-free retraction path to the specified gripping point. Information about the robot, gripper, and interfering contours in the system and on the robot should be taken into account to ensure a collision-free path.

[0083] Figure 6 shows a box 632, comparable to box 332 in Figure 3, containing various objects 640, 641, 642, 643, and 644 as examples. The goal is to grasp object 640 and remove it from the box. For this purpose, an approach path, or first movement sequence 651, is shown, by means of which the end effector moves to object 640 in order to grasp it. Additionally, a retraction path, or second movement sequence 652, is shown, along which the end effector can be moved with object 640 out of the box.

[0084] The motion sequence 250 thus created is then provided in step 252, so that the robot can be instructed to grasp the object based on it.

Claims

Claims 1. Method for determining a motion sequence (250) for a robot (100) to grasp and / or move an object (140) in a work environment (130) by means of an end effector (108) of the robot, comprising: Providing (206) environmental information (202, 204) that has been acquired from the working environment by means of at least one sensor (124); Determine (210), based on the environment information, a pose for one or each of several objects in the work environment to obtain an object pose record (228); Determine (230), based on the object poses record, one or more poses for one or each of several end effectors, for each of the one or at least one of the several objects of the object poses record, in order to obtain an end effector poses record (236); Determine (240), based on the end effector pose data set, the motion sequence (250) for the robot to grasp and / or move the object in the work environment using one or more of the robot's end effectors; and Providing (252) the movement sequence (250), and in particular causing the robot to execute the movement sequence.

2. The method of claim 1, wherein the environment information comprises 3D environment information (202), in particular one or more 3D images, depth images or point clouds, and / or 2D environment information (204), in particular one or more 2D images, and wherein determining (210) the object pose data set comprises at least one of the following steps: Processing (212) only 3D environment information; Processing (214) only 2D environment information; Processing (216) 2D and 3D environment information.

3. The method of claim 2, wherein the processing (212) comprises only 3D environment information: Determine, based on the 3D environment information, the pose for one or each of the multiple objects; and / or Assigning, based on the 3D environment information, one or at least one of the multiple objects to an object class, wherein the object pose record includes information about the assigned object class; wherein processing (214) of 2D environment information only includes: identifying, based on the 2D environment information, a part of the one or at least one of the multiple objects; and / or assigning, based on the 2D environment information, one or at least one of the multiple objects to an object class, wherein the object pose record includes information about the assigned object class; and wherein processing (216) of 2D and 3D environment information includes: Classifying and / or segmenting, based on the 2D environment information, of one or at least one of the multiple objects; and Determine, based on the 3D environment information, the pose for one or at least one of the several classified and / or segmented objects.

4. A method according to any of the preceding claims, further comprising, in order to obtain the object pose data set (228), at least one of the following steps: Determine (222), based on at least one object selection criterion, one or more selected objects from the one or more objects, wherein the object poses record includes only poses for the one or more selected objects; Assigning (224) a sequence to each of the multiple objects in the object poses record according to at least one object ordering criterion.

5. A method according to any of the preceding claims, further comprising, in order to obtain the end effector pose data set (236), at least one of the following steps: Determine (232), based on at least one end-effector selection criterion, one or more selected poses for the one or at least one of the multiple end-effectors, from the one or the multiple poses of the one of the multiple end-effectors, wherein the end-effector poses data set includes only the one or the multiple selected poses for the one or the at least one of the multiple end-effectors; Assigning (234) an order to each of the multiple poses for the one or at least one of the multiple end effectors in the end effector poses record according to at least one end effector ordering criterion.

6. Method according to one of the preceding claims, wherein determining (240) the sequence of movements (246) comprises: Determining (242) an axis configuration data set (244), in particular based on inverse kinematics, wherein the axis configuration data set comprises an axis configuration of the robot for one or at least one of the several poses of one or at least one of the several end effectors.

7. The method of claim 6, further comprising: Determine (246) for one or at least one of the multiple axis configurations of a potential motion sequence; and in the case of one potential motion sequence: use the potential motion sequence as the motion sequence, or in the case of multiple potential motion sequences: determine one of the multiple potential motion sequences as the motion sequence.

8. A method according to any of the preceding claims, wherein the motion sequence (250) comprises a first partial motion sequence (651) and a second partial motion sequence (652), wherein the first partial motion sequence involves a movement of one or one of the multiple end effectors towards comprising a position of the object, and wherein the second part of the motion sequence comprises a movement of one or one of the multiple end effectors away from the position of the object.

9. A method according to any of the foregoing claims, further comprising: Determine, based on the movement sequence, control information for moving the robot, and Providing control information and / or moving the robot based on the control information.

10. Computing unit comprising means for carrying out the method according to any of the preceding claims.

11. Robot configured to receive control information determined by a method according to claim 9, and / or with a computing unit according to claim 10, and with a drive system and a control or regulation unit for controlling the drive system, with one or more, in particular several selectable, end effectors for grasping and / or moving an object, and preferably with at least one sensor, in particular a camera and / or a 3D sensor, for capturing environmental information of a working environment.

12. Computer program comprising instructions which, when the program is executed by a computer, cause it to execute the method according to claims 1 to 9.

13. Computer-readable storage medium on which the computer program according to claim 12 is stored.

Images