Control device and method for controlling a robot device for packing and / or unpacking a storage container

Abstract

Various aspects relate to a control device and a method for controlling a robot device for removing at least one object from a storage container, the control device comprising: a memory device which stores, for each storage container of a plurality of storage containers, respectively associated first container information which indicates which plurality of objects are arranged in the storage container and in which arrangement the plurality of objects are located in the storage container; and a processor which is configured to: receive identification data which represent an identification of a storage container; determine the first container information associated with the storage container using the identification data; receive sensor data which represent an image of the plurality of objects arranged in the storage container;Using object recognition and the sensor data, determine second container information indicating which objects are arranged in the storage container; generate control instructions for controlling the robot device to pick up at least one of the plurality of objects based on the first container information associated with the storage container and the second container information.

Description

Technical field

[0001] Various aspects of this disclosure generally relate to a control apparatus and method for controlling a robotic device to fully or partially unpack or remove objects from a storage container. background

[0002] In warehouses (also called depots), objects (e.g. goods) can be stored in storage bins for safekeeping and then removed from the storage bins at a later time (e.g. when the objects are to be shipped). A robot can be used to retrieve the objects from a storage bin. In order for the robot to retrieve the objects, the objects must be correctly detected and identified in order to determine how the robot can reach the objects, in which order the robot can retrieve the objects, and how it can grip each object. This problem is also known as the bin picking problem. Brief description of the revelation

[0003] According to various embodiments, a control device and a method for controlling a robotic device for fully or partially unpacking or removing objects from a storage container are provided, which enable improved detection and grasping of objects in a storage container. For example, a classification rate with which objects are correctly classified (i.e., recognized) is increased. According to various aspects, the detection and grasping of objects is improved by capturing information regarding the objects in a storage container during packing of the storage container and using it during unpacking or grasping. Short description of the characters

[0004] In the drawings, reference characters generally refer to the same parts throughout the several views. The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings, in which: Fig. 1 shows an exemplary warehouse system according to different aspects. Fig. 2 shows an exemplary unpacking station according to different aspects. Fig. 3 and Fig. 4 each show a flow diagram for removing at least one object from a storage container according to various aspects. Fig. 5 shows an exemplary stacking of multiple objects in a storage container according to various aspects. Fig. 6 shows an exemplary image of multiple objects arranged in a storage container according to various aspects. Fig. 7 shows a flowchart of a method for controlling a robotic device to remove at least one object from a storage container according to various aspects. Fig. 8 shows a flowchart of a method for controlling a robotic device for packing a storage container according to various aspects. Detailed description

[0005] The following detailed description refers to the accompanying drawings which, by way of illustration, show specific details and embodiments in which the invention may be practiced.

[0006] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be considered preferred or advantageous over other embodiments.

[0007] The terms "at least one" and "one or more" may be understood to mean a numerical quantity greater than or equal to one (e.g., one, two, three, four, [...], etc.). The term "a plurality" may be understood to mean a numerical quantity greater than or equal to two (e.g., two, three, four, five, [...], etc.).

[0008] The terms "multiple" and "plurality" expressly refer to a set greater than one. Accordingly, all expressions that expressly refer to the foregoing words (e.g., a plurality of elements, multiple elements) refer to a set of elements, expressly referring to more than one of those elements. The terms "group (of)," "set (of)," "collection (of)," "series (of)," "sequence (of)," "grouping (of)," etc., and similar expressions in the description and claims, refer to a set equal to or greater than one, i.e., one or more.

[0009] The phrase "at least one of" with respect to a group of elements may be used herein to mean at least one element from the group comprising the elements. For example, the phrase "at least one of" with respect to a group of elements may be used herein to mean a selection of: one of the listed elements, a plurality of one of the listed elements, a plurality of individual listed elements, or a plurality of a multiple of individual listed elements.

[0010] The term "data" as used herein can be understood to include information in any suitable analog or digital form, e.g., in the form of a file, a portion of a file, a set of files, a signal or stream, a portion of a signal or stream, a set of signals or streams, and the like. In addition, the term "data" can also be used to refer to information, e.g., in the form of a pointer. However, the term "data" is not limited to the aforementioned examples and can take various forms and represent any information as understood in the art.

[0011] The term "processor," as used herein, can be understood as any type of entity that allows the processing of data or signals. The data or signals can, for example, be processed according to at least one (i.e., one or more than one) specific function performed by the processor. A processor can include or be formed from an analog circuit, a digital circuit, a mixed-signal circuit, a logic circuit, a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a programmable gate array (FPGA), an integrated circuit, or any combination thereof. Any other type of implementation of the respective functions, which are described in more detail below, can also be understood as a processor or logic circuit.It is understood that one or more of the method steps described in detail herein may be executed (e.g., realized) by a processor through one or more specific functions performed by the processor. The processor may therefore be configured to perform one of the methods described herein or its components for information processing.

[0012] The distinctions between software- and hardware-implemented computing can be blurred. A processor, security system, computing system, and / or other aspects described herein may be implemented in software, hardware, and / or as a hybrid implementation with both software and hardware.

[0013] The term "memory" is understood herein to mean a computer-readable medium in which data or information can be stored for retrieval. Memory used in the embodiments may be volatile memory, for example, DRAM (Dynamic Random Access Memory), or non-volatile memory, for example, PROM (Programmable Read-Only Memory), EPROM (Erasable PROM), EEPROM (Electrically Erasable PROM), or flash memory, such as a floating-gate memory device, a charge-trapping memory device, MRAM (Magnetoresistive Random Access Memory), or PCRAM (Phase Change Random Access Memory). Memory may be flash memory, solid-state memory, magnetic tape, a hard disk drive, an optical drive, etc., or any combination thereof. Registers, shift registers, processor registers, data buffers, etc. are also included in the term "memory."The term “software” refers to all types of executable instructions, including firmware.

[0014] The term "system" (e.g., a computing system, a warehouse system, etc.) as further discussed herein may be understood as a set of interacting elements, where the elements may be, by way of example and not limitation, one or more mechanical components, one or more electrical components, one or more instructions (e.g., encoded in storage media), and / or one or more processors, and the like.

[0015] The term "actuator" can be understood as a component capable of influencing a mechanism in response to being driven. The actuator can convert instructions issued by a control device (called activation) into mechanical movements. The actuator, e.g., an electromechanical transducer, can be configured to convert electrical energy into mechanical energy in response to its control.

[0016] Unless explicitly stated, the term "transmit" includes both direct (point-to-point) and indirect transmission (via one or more intermediate points). Similarly, the term "receive" includes both direct and indirect reception. Furthermore, the terms "send," "receive," "communicate," and similar terms encompass both physical transmission (e.g., the transmission of radio signals) and logical transmission (e.g., the transmission of digital data over a logical software-level connection).For example, a processor or controller may send or receive data over a software-level connection with another processor or controller in the form of radio signals, with the physical transmission and reception being handled by radio-layer components such as RF transceivers and antennas, and the logical transmission and reception over the software-level connection being performed by the processors or controllers. The term "communicate" encompasses both sending and receiving, i.e., unidirectional or bidirectional communication in one or both directions, i.e., inbound and outbound.

[0017] The term “calculate” includes both “direct” calculations using a mathematical expression / formula / relationship and “indirect” calculations using lookup or hash tables and other array indexing or search operations.

[0018] A model (e.g., a machine learning-based model (also referred to as a machine learning model)) may, for example, comprise or be a reinforcement learning model (e.g., using Q-learning, temporal difference (TD), deep adversarial networks, etc.) and / or a classification model (e.g., a linear classifier (e.g., a logistic regression classifier or a naive Bayes classifier), a support vector machine, a decision tree, a boosted tree classifier, a random forest classifier, a neural network, or a nearest neighbor model). A neural network can be or comprise any type of neural network, such as a convolutional neural network (CNN), a variational autoencoder network (VAE), a sparse autoencoder network (AAN), or a multi-level neural network (MNE).sparse autoencoder network (SAE), a recurrent neural network (RNN), a deconvolutional neural network (DNN), a generative adversarial network (GAN), a forward-thinking neural network, a sum-product neural network, a transformer-based network, etc.

[0019] A "control device," as used herein, can be understood as any type of entity (e.g., implementing logic) that allows the processing of data or signals. The control device can, for example, comprise one or more processors.

[0020] The term “image” as used herein may be any type of digital image data that can represent a pictorial representation, such as a digital RGB image, a digital RGB-D image, a binary image, a 3D image, a point cloud, a time series, a semantic segmentation image, etc.

[0021] The expression that an element, a parameter, etc. "represents" another element, another parameter, etc. can be understood to mean that these are linked to each other, e.g. the element and / or the parameter is a (e.g. unique, e.g. one-to-one) function of the other element and / or parameter.

[0022] Fig. 1 shows an exemplary warehouse system 100 according to various aspects. The warehouse system 100 may have a packing station 102. At the packing station, a plurality of objects 104 may be packed into a storage container 106. In some embodiments, the plurality of objects 104 may be manually packed into the storage container 106 by a person. In other aspects, the packing station 102 may have a packing station robot device configured to pack objects into a storage container 106. This packing process typically results in a topological arrangement of the objects in the storage container, e.g., by arranging them next to one another or one above the other. The objects may be the same (e.g., containing identical makes) or different (e.g., containing different make versions or make colors); the objects may furthermore be stackable (e.g., rectangular boxes) or individual or irregular (e.g.,blister packs, bulk goods).

[0023] The storage container 106 can then be transported from the packing station 102 to a storage area 108 (also referred to as a storage area), where the storage container 106 is kept (e.g., stored). If the multiple objects 104 are needed, for example, because they are to be further processed and / or shipped, the storage container 106 can be transported from the storage area 108 to an unpacking station 110. Transporting a storage container as described herein can be done manually (e.g., by a person) and / or at least partially automated (e.g., by means of a transport robot).

[0024] The unpacking station 110 may include a robotic device 114 configured to unpack the storage container 106. For example, the warehouse system 100 may include a transport system 112 (e.g., a conveyor belt), and the robotic device 114 may be configured to remove the plurality of objects 104 from the storage container 106 and place them on the transport system 112.

[0025] Fig. 2 shows an exemplary embodiment of the unpacking station 110 according to various aspects.

[0026] The robotic device 114 may include a robotic arm 206 (with one or more connecting elements). The robotic device 114 may include one or more robotic limbs 204. A robotic limb refers to movable parts of the robotic device 114 whose actuation enables physical interaction with the environment, e.g., to perform a task, e.g., to perform one or more skills. The one or more robotic limbs 204 may provide joints between portions of the robotic arm 206. A robotic limb may provide rotational motion and / or translational motion.

[0027] The robot device 114 may include a mount (e.g., a base) 202 to which the robot arm 206 may be attached.

[0028] The robot device 114 may include an end effector 208. The end effector 208 may be configured to enable picking up an object. For this purpose, the end effector may include one or more tools. For example, the end effector 208 may include a gripping tool and / or a suction device (e.g., a suction head).

[0029] Clearly, the robot arm 206 with the end effector 208 can grasp and move objects similar to a human arm.

[0030] It is understood that the robotic device 114 is exemplary, and that the unpacking station 110 may also include any other type of robotic device capable of removing objects from a storage container. Likewise, it is understood that the unpacking station 110 may also include more than one robotic device (e.g., multiple robotic devices with different end effectors).

[0031] The unpacking station 110 may include one or more data acquisition devices. Each data acquisition device may be configured to acquire and provide sensor data. For example, the unpacking station 110 may include a first data acquisition device 210. The first data acquisition device 210 may be configured to acquire first sensor data representing an image of the plurality of objects 104 arranged in the storage container. The first data acquisition device 210 may include one or more sensors configured to acquire the first sensor data. For example, the first data acquisition device 210 may include a camera (e.g., standard cameras, digital cameras, video cameras, SLR cameras, infrared cameras, stereo cameras, etc.), a charge-coupled device (CCD), a radar sensor, a LIDAR (light detection and ranging) sensor, and / or an ultrasonic sensor.

[0032] Sensor data representing an image of the plurality of objects 104 arranged in the storage container 106 may include, for example, an RGB image, an RGB-D image, or a depth image (also referred to as a D-image). A depth image described herein may be any type of image that includes depth information, such as a point cloud provided by a LIDAR sensor, a radar sensor, and / or an ultrasonic sensor.

[0033] The unpacking station 110 (e.g., the robotic device 114) may include one or more additional data acquisition devices that may be configured to acquire sensor data characterizing a state of the robotic device 114. For example, the one or more additional data acquisition devices may include an imaging sensor, such as a camera (e.g., a standard camera, a digital camera, an infrared camera, a stereo camera, etc.), a radar sensor, a LIDAR sensor, a position sensor, a speed sensor, an ultrasonic sensor, an acceleration sensor, a pressure sensor, etc., for acquiring sensor data characterizing the robotic device 114.

[0034] The unpacking station 110 may include a control device 212. The control device 212 may be configured to control the robot device 114 to unpack the storage container 106. To this end, the control device 212 may, for example, control a movement of the robot arm 206 (e.g., by controlling one or more robot limbs using associated actuators) and a picking up (e.g., gripping or sucking) of an object using the end effector 208. The control device 212 may include one or more processors 214. The control device 212 may include a storage device (memory for short) 216. The memory 216 may store code and data (e.g., comprising a robot control model) on the basis of which the processor 214 controls the robot device 114.

[0035] A processor described herein may include respective hardware-based processing devices. For example, a processor may include a microprocessor, preprocessors (e.g., an image preprocessor), graphics processors, a central processing unit (CPU), support circuits, digital signal processors, integrated circuits, memory, or other types of devices suitable for executing applications and for image processing and analysis. In various aspects, a processor may include any type of single-core or multi-core processor, mobile device microcontroller, central processing unit, etc. These types of processors may each include multiple processing units with local memory and instruction sets.

[0036] The first data acquisition device 210 may be configured to transmit the acquired first sensor data to the control device 212. The control device 212 may be configured to control the robot device 114 using the first sensor data acquired by the first data acquisition device 210.

[0037] Various aspects relate to controlling the robot device 114 to unpack the storage container 106 (also referred to as a storage box). Unpacking the storage container 106 may include removing the multiple objects 104 one after the other from the storage container 106. An object may be removed by picking up the object, then moving it, and then placing it at a position outside the storage container 106. Picking up an object may depend on one or more tools of the end effector 208. If the end effector 208 has a gripping tool, picking up an object may involve gripping the object. If the end effector 208 has a suction tool, picking up an object may involve sucking the object.This applies accordingly to packing the storage container 106 at the packing station 102, provided that one (or more than one) packing station robot device is used there. The packing station robot device can be configured as described above for the robot device 114. Packing the storage container 106 can include the multiple objects 104 being placed one after the other in the storage container 106. In this case, an object can be picked up, subsequently moved, and then placed in the storage container 106.

[0038] The transmission of data (e.g., sensor data, control instructions, etc.) described herein can be carried out via a network that connects the components to one another in a wired and / or wireless manner. In the case of a wireless network, each device described herein (e.g., the control device 212, the first data acquisition device 210, and / or the robot device 114, etc.) can each have a corresponding communication interface (e.g., comprising a transmitter and receiver). In various aspects, a communication interface can be configured to transmit and receive data according to multiple radio communication technologies.

[0039] Various processes for unpacking and packing the storage container 106 are explained below. It is understood that the control device 212 (e.g., using the processor 214) may be configured to implement these processes.

[0040] When packing the storage container 106, initial container information for the storage container 106 can be generated. The initial container information can then be stored in the memory 216 together with identification information for identifying the storage container 106. The identification of a storage container can comprise any suitable identification principle, such as a barcode, a quick response (QR) code, a transponder, etc.

[0041] The first container information can, for example, indicate which of the multiple objects 104 are arranged in the storage container 106. For example, the first container information can indicate an object type (e.g., indicated by an object identification, object ID for short) for each object of the multiple objects. The first container information can further indicate the arrangement in which the multiple objects 104 are located. The arrangement can have a temporal arrangement and / or a spatial arrangement. The temporal arrangement can have an order in which the multiple objects 104 were arranged in the storage container 106. The spatial arrangement can be the topological position of the multiple objects 104. The first container information can therefore indicate the order in which the multiple objects 104 were arranged in the storage container 106 and / or their topological position (e.g.,absolute position in the storage container and / or relative position to one or more other objects). If the order of the multiple objects 104 is discussed below, it is understood that this can be the temporal arrangement. If the respective position and / or pose of the multiple objects 104 is discussed below, it is understood that this can be the spatial arrangement.

[0042] In some embodiments, the packing station 102 can be a manual packing station at which a person packs the plurality of objects 104 one after the other into the storage container 106. In this case, the person can scan a respective object of the plurality of objects 104 using a scanning device (e.g., a barcode of the object) and then place it in the storage container 106. Clearly, the control device 212 can receive object identification data assigned to one of the plurality of objects 104 one after the other, which object identification data represents an identification of the object. The order in which the object identification data of the plurality of objects is received can then correspond to the order in which the plurality of objects 104 are / were arranged in the storage container 106. The first container information can then include the object identification data of the plurality of objects 104 in addition to the order.

[0043] Optionally, the manual packing station can have a packing station data acquisition device. The packing station data acquisition device can be configured to acquire packing station sensor data representing an image of the one or more objects arranged in the storage container. For example, the packing station data acquisition device can be used to acquire respective packing station sensor data after a respective object of the plurality of objects 104 has been arranged in the storage container 106. The packing station data acquisition device can be configured as described above for the first data acquisition device. This enables the control device 212 to determine position data and / or pose data for each object of the plurality of objects 104 based on the packing station sensor data. The position data can indicate a position of the respective object in the storage container 106.The pose data may indicate a pose of the respective object in the storage container 106. In this case, the first container information may include, in addition to which multiple objects 104 are arranged in the storage container 106 and in what order, the position data and / or pose data for each of the multiple objects 104. Clearly, the arrangement of the multiple objects may include the order in which the multiple objects were arranged in the storage container 106 (as a temporal arrangement) and the position data and / or pose data of the multiple objects (as a spatial arrangement).

[0044] Therefore, when reference is made herein to the first container information comprising a respective position and / or pose of the plurality of objects 104, it is understood that the packing station in this case may comprise the packing station data acquisition device.

[0045] The pose data can clearly indicate how (e.g., oriented) the respective object is arranged in the storage container 106 (e.g., which side of the object is facing upwards).

[0046] In other embodiments, the packing station 102 may be an at least partially automated packing station. In this case, the packing station 102 may comprise the packing station robot device in addition to the packing station data acquisition device. The packing control instructions for controlling the packing station robot device during packing of the storage container 106 may specify, for each object, the position and / or pose in which the object is / was arranged in the storage container 106. Consequently, in this case, the first container information may comprise the respective position and / or pose for each object of the plurality of objects 104.

[0047] To generate control instructions for packing a storage container, various robot control models can be used, which, for example, reduce (e.g., minimize) a packing time (i.e., a time required for packing). According to various aspects, the control device 212 can be configured to generate the packing control instructions for controlling the packing station robot device for packing the storage container 106 (e.g., using a robot control model) such that not only the (predicted) packing time, but also the (predicted) unpacking time is reduced (e.g., minimized). In this case, the control device 212 can store a packing plan (which also represents an unpacking plan) as part of the first container information in the memory 216. In this way, an effort (e.g.,Computational effort and / or energy expenditure) can be reduced, since the sequence for removing the objects does not have to be determined again during unpacking.

[0048] The control device 212 (e.g., the one or more processors 214) may be configured to use the respective position and / or pose of the plurality of objects 104 to determine stacking information that indicates how the plurality of objects 104 are stacked on top of one another in the storage container. The stacking information may be determined using the dimensions of the respective object. The dimensions (e.g., width, height, depth, shape, extent, etc.) of an object may, for example, be represented by the first container information (since these indicate which plurality of objects 104 are arranged in the storage container 106). Optionally, the dimensions may also be determined from the packing station sensor data.

[0049] An exemplary stacking of five objects A, B, C, D, E is shown for illustration in Fig. 5. According to various aspects, the first container information may include stacking information. The stacking information may, for example, include a doubly linked list, a graph, and / or a tree structure. Illustratively, the stacking information may indicate how the plurality of objects 104 are stacked on top of one another in the storage container 106.

[0050] Fig. 3 shows a flowchart 300 for unpacking the storage container 106 according to various aspects.

[0051] At 302, identification data representing an identification of the storage container can be received. As described above, the identification data can be, for example, a barcode, a QR code, a transponder, etc. The identification of the storage container 106 can be performed in any way that enables a unique identification of a storage container from a plurality of storage containers.

[0052] In 304, the first container information (e.g., stored in the memory 216) for the (identified) storage container 106 can be determined.

[0053] In 306, the control device 212 may receive sensor data from the first data acquisition device 210 representing an image of the plurality of objects 104 arranged in the storage container 106.

[0054] In 308, the control device 212 may perform object recognition to recognize the plurality of objects 104 arranged in the storage container 106. To this end, the control device 212 may determine second container information that indicates (e.g., includes a prediction of) which objects are arranged in the storage container 106. Fig. 6 shows second container information as an exemplary image 600 of a plurality of objects arranged in a storage container 602. Furthermore, the image includes numbers indicating an exemplary determined order in which the plurality of objects are to be removed from the storage container. This exemplary image illustrates the importance of the order in which the objects are removed from the storage container. If, for example, object 2 were to be removed before object 1, the overlaps between these objects could result in both items being grabbed at the same time or object 1 being moved out of the storage container (if, for example, the overlap between object 2 and object 1 is not correctly recognized). This applies accordingly to objects 3 and 4, which are both arranged below object 2.

[0055] The object recognition can be any type of object recognition by means of which objects in a storage container can be recognized. For example, the object recognition can include (e.g., semantic) object segmentation. The control device 212 can be configured to implement a model (e.g., a machine learning model) for object recognition.

[0056] In 310, the control device 212 may compare the first container information with the second container information and, based thereon, generate control instructions for controlling the robot device 114.

[0057] If the control instructions were generated exclusively using the second container information, some of the multiple objects 104 could be incorrectly recognized. However, if the first container information is additionally used, which explicitly indicates which multiple objects 104 are arranged in the storage container 106 (and optionally also their respective position and / or pose), the recognition rate can be significantly increased.

[0058] The increased detection rate can also reduce the number of human interventions and increase the throughput of the warehouse system.

[0059] Since the first container information already indicates which of the multiple objects 104 are arranged in the storage container 106, each object of the multiple objects 104 can be assigned to exactly one object of the second container information. If, for example, two or more classes (representing the object type) are determined for an object when classifying the sensor data (to determine the second container information), the correct class can be selected using the first container information.

[0060] The first container information may also include data that cannot be detected by means of object recognition for determining the second container information, such as a respective material (e.g., whether a material is glass or plastic) of the plurality of objects 104.

[0061] Correct object recognition is required to determine correct gripping data according to which the robot device 114 grips the objects.

[0062] In some aspects, the controller 212 may determine the control instructions using the stacking information. For example, the controller 212 may determine, using the respective object type of the multiple objects 104 and the stacking information, whether removing one or more objects may damage one or more other objects (e.g., because one or more objects are easily breakable). Consequently, in this case, damage to objects during unpacking of a storage container may be reduced. With reference to the example image of Fig. 6, without the stack information, it might be possible to remove objects 8 and 9 because they do not overlap with other objects. However, the stack information might indicate that removing object 8 and / or object 9 could cause the stack of multiple objects to collapse.

[0063] According to various aspects, the controller 212 may be further configured to compare the packing station sensor data representing one or more images of the objects arranged in the storage container at the packing station 102 with the sensor data representing one or more images of the objects arranged in the storage container at the unpacking station 110 to determine whether one or more objects of the plurality of objects 104 are damaged.

[0064] Fig. 4 shows a flowchart 400 for removing at least one object from a storage container using position data according to various aspects.

[0065] In 402, the storage container 106 is identified as described above. Subsequently, the control device 212 can determine the first container information for the (identified) storage container 106.

[0066] At 404, the controller 212 may determine an order in which the plurality of objects 104 should be removed from the storage container 106. For example, the order in which the plurality of objects 104 should be removed from the storage container 106 may be opposite to the order in which the plurality of objects 104 were arranged in the storage container 106.

[0067] At 406, object information of the next object to be removed according to the order in which the plurality of objects 104 are to be removed from the storage container 106 may be determined. The first container information may include and / or represent the object information. In this example, the object information may include at least an object type of the respective object and a position of the respective object (and optionally further a pose of the respective object).

[0068] The control device 212 can receive sensor data from the first data acquisition device, which represents an image of the one or more objects arranged (not yet removed) in the storage container 106 at that time (i.e., before removal of the next object to be removed).

[0069] The control device 212 can be configured to determine a search space for searching for the object in the image using the position of the next object to be extracted. The search space can comprise a section of the image. In 408, the next object to be extracted can be searched for in the search space (optionally using the object's pose). For this purpose, the object recognition described herein can be applied to the sensor data.

[0070] If an object is searched for in a search space (as a subspace of the entire storage container 106), the number of false detections can be reduced (since only the objects in this search space are considered). Using the search space also allows for the differentiation of objects that have the same object type (but are located at different positions in the storage container 106).

[0071] If the next object to be removed is found in the search space (“Yes” in 410), the controller 212 generates (in 412) the control instructions for removing the object.

[0072] If the next object to be retrieved is not found in the search space ("No" in 410), the controller 212 may determine another search space that is larger than the search space (e.g., the entire storage container 106) and / or may optionally search for a different pose. In 414, the next object to be retrieved may be searched in the other search space (optionally using the object's pose).

[0073] If the next object to be retrieved is found in the other search space ("Yes" in 416), the controller 212 generates (in 412) the control instructions for retrieving the object. If the next object to be retrieved is not found in the search space ("No" in 414), the controller 212 may proceed with the next object to be retrieved (and optionally store the acquired information regarding the unfound object in the memory 216).

[0074] By using the search space, the computational effort required to find an object can be reduced (since the entire image does not have to be searched for each object).

[0075] According to various aspects, the transport of the storage container 106 from the packing station 102 (via the storage area 108) to the unpacking station 110 is taken into account.

[0076] For this purpose, the first container information may further comprise transport data representing how the storage container 106 was transported from the packing station 102 to the unpacking station 110.

[0077] For example, the storage container 106 may have been moved by one or more transport robots, and each transport robot may collect acceleration data while transporting the storage container 106. The transport data may include this acceleration data from each transport robot.

[0078] For example, the storage container 106 may include an acceleration sensor that detects acceleration data of the storage container 106, and the transport data may include this acceleration data.

[0079] The control device 212 (e.g., the one or more processors 214) may be configured to implement a motion model. The motion model may be configured to output adjusted position data and / or adjusted pose data for each object of the plurality of objects 104 in response to an input of the object information (e.g., object type, object shape, position, pose, etc.) of each object of the plurality of objects 104 and the transport data (and optionally further the stack information) into the motion model. Clearly, the motion model can be used to predict how the plurality of objects 104 moved as a result of the transport in the storage container 106.

[0080] These adapted position data and / or adapted pose data can then serve, for example, as a basis for determining the search space in order to search for the object to be extracted in this search space (in 408).

[0081] Optionally, the motion model can be trained online using information about the actual position and / or pose of a given object in the storage container 106. Thus, the difference between the position and / or pose of an object in the storage container 106 at the packing station and the position and / or pose of the object upon removal at the unpacking station 110 provides information about the movement of the object as a result of transport. It can also be concluded, for example, that if many objects have moved significantly, the transport process should be revised.

[0082] Fig. 7 shows a flowchart of a method for controlling a robotic device to remove at least one object from a storage container according to various aspects.

[0083] The method 700 may include (in 702) receiving identification data representing an identification of a storage container to be unpacked.

[0084] The method 700 may include (in 704) determining first container information associated with the storage container (e.g., stored in a storage device) using the identification data. The first container information may indicate which multiple objects are arranged in the storage container and the order in which the multiple objects were arranged in the storage container.

[0085] The method 700 may include (in 706) receiving (e.g., from a sensor, such as an imaging device) sensor data representing an image of the plurality of objects disposed in the storage container.

[0086] The method 700 may include (in 708) determining second container information by means of object recognition using the sensor data. The second container information may indicate (e.g., include a prediction of) which objects are located in the storage container.

[0087] The method 700 may include (in 710) generating control instructions for controlling the robotic device to pick up at least one object of the plurality of objects based on the first container information and the second container information.

[0088] Fig. 8 shows a flow diagram of a method 800 for controlling a robotic device to pack a storage container according to various aspects.

[0089] The method 800 may include (at 802) determining a packing plan for packing a plurality of objects into the storage container. The packing plan may specify an order in which the plurality of objects are to be packed into the storage container and a respective position (and optionally further pose) for each of the plurality of objects. The packing plan may be determined using a prediction of a packing time required to pack the plurality of objects into the storage container and a prediction of an unpacking time required to unpack the plurality of objects from the storage container.

[0090] The method 800 may include (in 804) generating control instructions for controlling the robotic device to pack the storage container according to the packing plan.

[0091] Various examples are provided below that describe one or more aspects of the controller 212 and the methods (e.g., the method 700 and / or the method 800). It is understood that aspects described with respect to the controller 212 may also apply to the methods, and vice versa.

[0092] Example 1 is a control device for controlling a robot device for removing at least one object from a storage container, the control device comprising: a storage device which stores, for each storage container of a plurality of storage containers, respectively associated first container information which indicates which plurality of objects are arranged in the storage container and in which (e.g.temporal and / or spatial) arrangement in which the plurality of objects are located in the storage container; and a processor which is configured to: receive identification data representing an identification of a storage container; use the identification data to determine the first container information associated with the storage container; receive sensor data representing an image of the plurality of objects arranged in the storage container; use the sensor data to determine second container information by means of object recognition, which indicates (e.g., has a prediction about) which objects are arranged in the storage container; generate control instructions for controlling the robot device to pick up at least one object of the plurality of objects based on the first container information associated with the storage container and the second container information.

[0093] Example 2 is configured according to Example 1, wherein the processor is configured to determine the first container information associated with a respective storage container of the plurality of storage containers by the processor iteratively: receiving, for each object of the plurality of objects, object identification data associated with the object, which represents an identification of the object, before the object is arranged in the storage container; storing the first container information associated with the object in the storage device, wherein the first container information comprises the object identification data;and wherein the processor is configured to store the first container information in the storage device, wherein the arrangement in which the plurality of objects are located in the storage container has an order (as a temporal arrangement) that corresponds to the order in which the processor receives the object identification data of the plurality of objects.;

[0094] Example 3 is configured according to Example 2, wherein the processor is further configured to iteratively: receive, for each object of the plurality of objects, packing station sensor data associated with the object (e.g., from a packing station sensor, such as a packing station imaging unit) representing an image of the one or more objects arranged in the storage container after the object has been arranged in the storage container; determine, using the packing station sensor data, position data associated with the object that indicates a position of the object in the storage container; wherein the arrangement comprises the position data (as a spatial arrangement); wherein the processor is configured to generate the control instructions using the position data.

[0095] Example 4 is configured according to Example 3, wherein the processor is further configured to iteratively: for each object of the plurality of objects, determine, using the packing station sensor data, pose data associated with the object that indicates a pose (e.g., orientation) of the object in the storage container; wherein the arrangement comprises the pose data (as a spatial arrangement); wherein the processor is configured to generate the control instructions using the pose data.

[0096] Example 5 is configured according to example 3 or 4, wherein the processor is configured to: determine, using the arrangement (e.g., the order in which the plurality of objects were arranged in the storage container and the position data of the plurality of objects (and optionally further the pose data of the plurality of objects)), stacking information indicating how the plurality of objects are stacked on top of each other in the storage container; and generate the control instructions using the stacking information.

[0097] Example 6 is configured according to any one of Examples 1 to 5, wherein the processor is configured to: determine a packing plan for packing the plurality of objects into the storage container, wherein the packing plan specifies an order in which the plurality of objects are to be packed into the storage container and a respective position (and optionally further pose) for each object of the plurality of objects; wherein, in determining the packing plan, a prediction of a packing time required to pack the plurality of objects into the storage container and a prediction of an unpacking time required to unpack the plurality of objects from the storage container are taken into account.

[0098] Example 7 is configured according to Example 6, wherein the processor is configured to: store the packing plan in the storage device; and generate the control instructions for controlling the robot device using the packing plan.

[0099] Example 8 is configured according to any one of Examples 1 to 7, wherein the processor is configured to determine whether one or more than one object of the plurality of objects is damaged based on the first container information associated with the storage container and the second container information.

[0100] Example 9 is configured according to any one of Examples 1 to 8, wherein the second container information comprises a plurality of predicted objects; wherein the processor is configured to assign an object of the plurality of objects to each predicted object of the plurality of predicted objects.

[0101] Example 10 is configured according to Example 9, wherein the arrangement further comprises, for each object of the plurality of objects, respective position data (as a spatial arrangement) indicating a position of the object in the storage container; wherein the second container information associated with the respective storage container further comprises, for each predicted object of the plurality of predicted objects, respective predicted position data indicating a predicted position of the predicted object in the storage container; wherein the processor is configured to assign the object of the plurality of objects to each predicted object of the plurality of predicted objects using a comparison of the position data of the plurality of objects with the predicted position data of the plurality of predicted objects.

[0102] Example 11 is configured according to Example 10, wherein the arrangement further comprises, for each object of the plurality of objects, respective pose data (as a spatial arrangement) indicating a pose of the object in the storage container; wherein the second container information associated with the respective storage container further comprises, for each predicted object of the plurality of predicted objects, respective predicted pose data indicating a predicted pose of the predicted object in the storage container; wherein the processor is configured to assign the object of the plurality of objects to each predicted object of the plurality of predicted objects using a comparison of the pose data of the plurality of objects with the predicted pose data of the plurality of predicted objects.

[0103] Example 12 is configured according to example 10 or 11, wherein the storage device further stores transport data representing how the storage container was transported from a packing station, at which the plurality of objects were packed into the storage container, to an unpacking station comprising the robot device; wherein the processor is configured to compare the position data of the plurality of objects with the predicted position data of the plurality of predicted objects by determining, for each object of the plurality of objects, adjusted position data (which takes into account a change in position of the object as a result of the transport) using the position data and the transport data, and comparing the adjusted position data of the plurality of objects with the predicted position data of the plurality of predicted objects.

[0104] Example 13 is configured according to examples 11 and 12, wherein the processor is configured to compare the pose data of the plurality of objects with the predicted pose data of the plurality of predicted objects by the processor determining, for each object of the plurality of objects, adjusted pose data (which takes into account a change in pose of the object as a result of the transport) using the pose data and the transport data, and comparing the adjusted pose data of the plurality of objects with the predicted pose data of the plurality of predicted objects.

[0105] Example 14 is configured according to any one of Examples 1 to 13, wherein the processor is configured to generate the control instructions using the arrangement in which the plurality of objects are located in the storage container.

[0106] Example 15 is configured according to Example 14, wherein the control instructions specify an order in which the plurality of objects are to be removed from the storage container, the order being opposite to an order specified by the arrangement in which the plurality of objects were arranged in the storage container.

[0107] Example 16 is configured according to any one of Examples 1 to 15, wherein the processor is configured to control the robotic device to deposit the (picked up) at least one object on a conveyor belt and / or in another storage container.

[0108] Example 17 is a system (e.g., a warehouse system) comprising: a control device according to any one of Examples 1 to 16; and an unpacking station for unpacking the storage container, wherein the unpacking station comprises a sensor for acquiring the sensor data (e.g., an imaging unit) and the robot device.

[0109] Example 18 is configured according to Example 17, further comprising: a packing station for packing objects into a storage container, the packing station comprising a packing station sensor for detecting one or more objects arranged in the storage container.

[0110] Example 19 is configured according to example 17 or 18, wherein the packing station further comprises a packing station robot device configured to pick up an object and place the object in the storage container.

[0111] Example 20 is a method for controlling a robot device for removing at least one object from a storage container, the method comprising: receiving identification data representing an identification of a storage container to be unpacked; determining first container information associated with the storage container (e.g., stored in a memory device) using the identification data, wherein the first container information indicates which of the plurality of objects are arranged in the storage container and in which (e.g., temporal and / or spatial) arrangement the plurality of objects are located in the storage container; receiving (e.g.,from a sensor, such as an imaging unit) of sensor data representing an image of the plurality of objects arranged in the storage container; determining second container information by means of object recognition using the sensor data, wherein the second container information indicates (e.g., comprises a prediction of) which objects are arranged in the storage container; generating control instructions for controlling the robot device to pick up at least one object of the plurality of objects based on the first container information and the second container information.

[0112] Example 21 the method according to example 20, further comprising: determining the first container information respectively associated with a respective storage container of the plurality of storage containers by iteratively receiving for each object of the plurality of objects: object identification data associated with the object, which represent an identification of the object, before the object is arranged in the storage container, wherein the first container information comprises the object identification data; wherein the arrangement in which the plurality of objects are located in the storage container has an order (as a temporal arrangement) that corresponds to the order in which the object identification data of the plurality of objects are received.

[0113] Example 22 is configured according to Example 21, wherein determining the first container information respectively associated with the respective storage container of the plurality of storage containers further comprises: for each object of the plurality of objects iteratively: receiving (e.g., from a packing station sensor, such as a packing station imaging unit) packing station sensor data associated with the object, which represents an image of the one or more objects arranged in the storage container after the object has been arranged in the storage container; using the packing station sensor data, determining position data associated with the object, which indicates a position of the object in the storage container, wherein the arrangement comprises the position data (as a spatial arrangement); wherein the control instructions are generated using the position data.

[0114] Example 23 is configured according to Example 22, wherein determining the first container information respectively associated with the respective storage container of the plurality of storage containers further comprises: for each object of the plurality of objects iteratively: using the packing station sensor data, determining pose data associated with the object, which indicate a pose (e.g., orientation) of the object in the storage container, wherein the arrangement comprises the pose data (as a spatial arrangement); wherein the control instructions are generated using the pose data.

[0115] Example 24 is the method of example 22 or 23, further comprising: using the arrangement (e.g., the order in which the plurality of objects were arranged in the storage container, the position data of the plurality of objects (and optionally further the pose data of the plurality of objects)), determining stacking information indicating how the plurality of objects are stacked on top of each other in the storage container; wherein the control instructions are generated using the stacking information.

[0116] Example 25 is the method of any one of Examples 20 to 24, further comprising: determining a packing plan for packing the plurality of objects into the storage container, the packing plan specifying an order in which the plurality of objects are to be packed into the storage container and a respective position (and optionally further pose) for each object of the plurality of objects; wherein determining the packing plan takes into account a prediction of a packing time required to pack the plurality of objects into the storage container and a prediction of an unpacking time required to unpack the plurality of objects from the storage container; and optionally packing the plurality of objects into the storage container according to the packing plan.

[0117] Example 26 is configured according to Example 25, wherein the control instructions for controlling the robot device are generated using the packing plan.

[0118] Example 27 is the method of any one of Examples 20 to 26, further comprising: determining, based on the first container information associated with the storage container and the second container information, whether one or more than one object of the plurality of objects is damaged.

[0119] Example 28 is configured according to any one of Examples 20 to 27, wherein the second container information comprises a plurality of predicted objects; wherein the method comprises assigning an object of the plurality of objects to each predicted object of the plurality of predicted objects.

[0120] Example 29 is configured according to Example 28, wherein the arrangement further comprises, for each object of the plurality of objects, respective position data (as a spatial arrangement) indicating a position of the object in the storage container; wherein the second container information associated with the respective storage container further comprises, for each predicted object of the plurality of predicted objects, respective predicted position data indicating a predicted position of the predicted object in the storage container; wherein the method comprises assigning the object of the plurality of objects to each predicted object of the plurality of predicted objects using a comparison of the position data of the plurality of objects with the predicted position data of the plurality of predicted objects.

[0121] Example 30 is configured according to Example 29, wherein the arrangement further comprises, for each object of the plurality of objects, respective pose data (as a spatial arrangement) indicating a pose of the object in the storage container; wherein the second container information associated with the respective storage container further comprises, for each predicted object of the plurality of predicted objects, respective predicted pose data indicating a predicted pose of the predicted object in the storage container; wherein the method comprises assigning the object of the plurality of objects to each predicted object of the plurality of predicted objects using a comparison of the pose data of the plurality of objects with the predicted pose data of the plurality of predicted objects.

[0122] Example 31 is the method according to example 29 or 30, further comprising: receiving transport data representing how the storage container was transported from a packing station at which the plurality of objects were packed into the storage container to an unpacking station comprising the robotic device; wherein comparing the position data of the plurality of objects with the predicted position data of the plurality of predicted objects comprises: for each object of the plurality of objects, determining adjusted position data (taking into account a change in position of the object as a result of the transport) using the position data and the transport data, and comparing the adjusted position data of the plurality of objects with the predicted position data of the plurality of predicted objects.

[0123] Example 32 is configured according to examples 30 and 31, wherein comparing the pose data of the plurality of objects with the predicted pose data of the plurality of predicted objects comprises: for each object of the plurality of objects, determining adjusted pose data (taking into account a change in position of the object due to transportation) using the pose data and the transportation data, and comparing the adjusted pose data of the plurality of objects with the predicted pose data of the plurality of predicted objects.

[0124] Example 33 is configured according to any one of Examples 20 to 32, wherein the control instructions are generated using the arrangement in which the plurality of objects are located in the storage container.

[0125] Example 34 is configured according to Example 33, wherein the control instructions specify an order in which the plurality of objects are to be removed from the storage container, the order being opposite to an order specified by the arrangement in which the plurality of objects were arranged in the storage container.

[0126] Example 35 is the method according to any one of examples 20 to 34, further comprising: controlling the robot device to deposit the (picked up) at least one object on a conveyor belt and / or in another storage container.

[0127] Example 36 is a method for controlling a robotic device for removing at least one object from a storage container, the method comprising: receiving identification data representing an identification of a storage container to be unpacked; determining first container information associated with the storage container (e.g., stored in a memory device) using the identification data, wherein the first container information indicates which of the plurality of objects are arranged in the storage container and represents the order in which the plurality of objects are to be removed from the storage container; iteratively for a respective object of the plurality of objects according to the order in which the plurality of objects are to be removed from the storage container: receiving (e.g.,from a sensor, such as an imaging unit) of sensor data representing an image of the one or more objects arranged in the storage container (not yet removed); determining, using the sensor data, whether the image shows the respective object; if it is determined that the image shows the respective object, generating control instructions for controlling the robot device to remove the respective object from the storage container.

[0128] Example 37 is the method of Example 36, further comprising: if it is determined that the image does not show the respective object, proceeding with the next object of the plurality of objects according to the order.

[0129] Example 38 is configured according to example 36 or 37, wherein the first container information comprises, for each object of the plurality of objects, respective position data indicating a position of the object in the storage container; wherein determining whether the image shows the respective object comprises: determining, using the position data of the respective object, a search space for searching the respective object in the image; and determining whether the image shows the respective object in the search space.

[0130] Example 39 is configured according to Example 38, wherein determining whether the image shows the respective object further comprises: if it is determined that the image does not show the respective object in the search space, determining an adjusted search space that includes a larger portion of the image than the search space; and determining whether the image shows the respective object in the adjusted search space.

[0131] Example 40 is configured according to example 38 or 39, wherein the first bin information comprises, for each object of the plurality of objects, respective pose data indicating a pose of the object in the storage bin; and wherein the search space is determined using the pose data.

[0132] Example 41 is the method of any one of Examples 38 to 40, further comprising: receiving transport data representing how the storage container was transported from a packing station at which the plurality of objects were packed into the storage container to an unpacking station comprising the robotic device; wherein the search space is determined using the transport data.

[0133] Example 42 is a method for controlling a robotic device for packing a storage container, the method comprising: determining a packing plan for packing a plurality of objects into the storage container, wherein the packing plan specifies an order in which the plurality of objects are to be packed into the storage container and, for each object of the plurality of objects, a respective position (and optionally further pose); and generating control instructions for controlling the robotic device to pack the storage container according to the packing plan; wherein the packing plan is determined using a prediction of a packing time required to pack the plurality of objects into the storage container and a prediction of an unpacking time required to unpack the plurality of objects from the storage container

[0134] Example 43 is configured according to Example 42, wherein the packing plan is determined such that the prediction of the packing time required to pack the plurality of objects into the storage container and the unpacking time required to unpack the plurality of objects from the storage container are reduced (e.g., minimized).

[0135] Example 44 is the method according to example 42 or 43, further comprising: controlling the robotic device to pack the plurality of objects sequentially into the storage container according to the packing plan; after a respective object of the plurality of objects has been packed into the storage container, acquiring (e.g., by means of a packing station sensor, such as a packing station imaging unit) packing station sensor data associated with the object, which represents an image of the one or more objects arranged in the storage container at that time.

[0136] Example 45 is a (e.g., non-transitory) computer-readable medium (e.g., a computer program product, a non-transitory storage medium, a non-transitory storage medium, or a non-volatile storage medium) storing instructions that, when executed by a processor, cause the processor to control an apparatus to perform a method according to any one of Examples 20 to 44.

[0137] Example 46 is one or more means for performing the method according to any one of Examples 20 to 44.

[0138] In the foregoing description and the accompanying figures, the components of electronic devices are sometimes depicted as separate elements. In this regard, it is understood that discrete elements may be combined or integrated into a single element. This includes combining two or more circuits into a single circuit, mounting two or more circuits on a common chip or chassis to form an integrated element, executing discrete software components on a common processor core, etc.Conversely, it is understood that a single element can be divided into two or more discrete elements, such as dividing a single circuit into two or more separate circuits, dividing a chip or chassis into discrete elements originally intended on it, dividing a software component into two or more sections and executing them on a separate processor core, etc.

[0139] It is understood that the implementations of the methods described herein are demonstrative in nature and can therefore be implemented in a corresponding device. Likewise, it is understood that implementations of the device described herein can be implemented as a corresponding method. It is therefore understood that a device corresponding to a method described herein can include one or more components configured to perform each aspect of the corresponding method.

[0140] While the invention has been particularly shown and described with reference to specific embodiments, it is understood that various changes in design may be made without departing from the scope of the invention as defined by the appended claims. The scope of the invention is thus determined by the appended claims, and all changes which come within the meaning and range of equivalence of the claims are therefore intended to be embraced.

Claims

[1] Control device (212) for controlling a robot device (114) for removing at least one object from a storage container (106), the control device (212) comprising: a storage device (216) which stores, for each storage container (106) of a plurality of storage containers, respective associated first container information which indicates which plurality of objects (104) are arranged in the storage container (106) and in which arrangement the plurality of objects (104) are located in the storage container (106); and a processor (214) configured to: • to receive identification data representing an identification of a storage container (106); • to determine the first container information associated with the storage container (106) using the identification data; • receive sensor data representing an image of the plurality of objects (104) arranged in the storage container (106); • to determine second container information by means of object recognition using the sensor data, which indicates which objects are arranged in the storage container (106); • generate control instructions for controlling the robot device (114) to pick up at least one object of the plurality of objects (104) based on the first container information associated with the storage container (106) and the second container information. [2] Control device (212) according to claim 1, wherein the processor (214) is configured to determine the first container information associated with a respective storage container (106) of the plurality of storage containers by the processor (214) iteratively for each object of the plurality of objects (104): • receives object identification data associated with the object, which represents an identification of the object, before the object is placed in the storage container (106); • stores the first container information associated with the object in the storage device (216), wherein the first container information comprises the object identification data; and wherein the processor (214) is configured to store the first container information in the storage device (216), wherein the arrangement in which the plurality of objects (104) are located in the storage container (106) has an order that corresponds to the order in which the processor (214) receives the object identification data of the plurality of objects (104). [3] The control device (212) of claim 2, wherein the processor (214) is further configured to iteratively perform for each object of the plurality of objects (104): • receive packing station sensor data associated with the object, representing an image of the one or more objects disposed in the storage container (106) after the object has been disposed in the storage container (106); • using the packing station sensor data to determine position data associated with the object, which indicate a position of the object in the storage container (106); • wherein the arrangement comprises the position data; wherein the processor (214) is configured to generate the control instructions using the position data. [4] Control device (212) according to claim 3, wherein the processor (214) is arranged: • using the arrangement comprising the position data of the plurality of objects, to determine stacking information indicating how the plurality of objects are stacked on top of one another in the storage container (106); and • generate the control instructions using the stack information. [5] Control device (212) according to one of claims 1 to 4, wherein the processor (214) is arranged: • to determine a packing plan for packing the plurality of objects (104) into the storage container (106), the packing plan specifying an order in which the plurality of objects (104) are to be packed into the storage container (106) and a respective position for each object of the plurality of objects (104); • wherein, when determining the packing plan, a prediction of a packing time required to pack the plurality of objects (104) into the storage container (106) and a prediction of an unpacking time required to unpack the plurality of objects (104) from the storage container (106) are taken into account. [6] Control device (212) according to one of claims 1 to 5, • wherein second container information comprises a plurality of predicted objects; • wherein the processor (214) is configured to assign one object of the plurality of objects (104) to each predicted object of the plurality of predicted objects. [7] Control device (212) according to claim 6, • wherein the arrangement has, for each object of the plurality of objects (104), respective position data indicating a position of the object in the storage container (106); • wherein the second container information further comprises, for each predicted object of the plurality of predicted objects, respective predicted position data indicating a predicted position of the predicted object in the storage container (106); • wherein the processor (214) is configured to assign the object of the plurality of objects (104) to each predicted object of the plurality of predicted objects using a comparison of the position data of the plurality of objects with the predicted position data of the plurality of predicted objects. [8] System (100), comprising: • a control device (212) according to one of claims 1 to 7; and • an unpacking station (110) for unpacking the storage container (106), wherein the unpacking station (110) has a sensor (210) for detecting the sensor data and the robot device (114). [9] A method (700) for controlling a robot device (114) for removing at least one object from a storage container (106), the method comprising: • Receiving (702) identification data representing an identification of a storage container (106) to be unpacked; • Determining (704) first container information associated with the storage container (106) using the identification data, wherein the first container information indicates which plurality of objects are arranged in the storage container (106) and in which arrangement the plurality of objects are located in the storage container (106); • Receiving (706) sensor data representing an image of the plurality of objects arranged in the storage container (106); • Determining (708) second container information by means of object recognition using the sensor data, wherein the second container information indicates which objects are arranged in the storage container (106); • Generating (710) control instructions for controlling the robot device (114) to pick up at least one object of the plurality of objects based on the first container information and the second container information. [10] A non-transitory computer-readable medium storing instructions that, when executed by a processor (214), cause the processor (214) to control an apparatus to perform a method according to claim 9.

Images