Method for controlling a robot to manipulate, in particular pick up, an object

Abstract

Method (300) for controlling a robot (100) for manipulating, in particular picking up, an object, comprising: Obtaining (301) a picture showing the object; Generating (302) from the image, a manipulation quality image in which, for each pixel representing a location on the surface of the object, the pixel value of the pixel provides an estimate of how well the object can be manipulated at that location; Register (303) descriptors of locations of the object that, according to user input, should be used during manipulation, and / or of locations that, according to user input, should be avoided during manipulation; Mapping (304) the image onto a descriptor image; Generating (305) by comparing the registered descriptors of positions with the descriptor image, a manipulation preference image, in which, For each pixel representing a location on the object's surface, the pixel value provides an estimate of whether the object should be manipulated at that location; Selecting (306) a location to manipulate the object, taking into account the pixel values ​​of the manipulation quality image and the pixel values ​​of the manipulation preference image; and Control (307) the robot (100) to manipulate the object at the selected location.

Description

State of the art

[0001] The present disclosure relates to methods for controlling a robot to manipulate, in particular pick up, an object.

[0002] There are several approaches to identifying the most promising gripping points on objects when a robot picks them up, for example, to remove an object from a container. These approaches typically work with RGBD images (i.e., images with color and depth information) of the scene, although in some cases either RGB or depth images are sufficient. Furthermore, most of these approaches rely on AI methods, such as using neural networks to learn a mapping between input data and promising gripping points. Therefore, it is crucial that these AI methods can reliably identify grippable areas on the surfaces of objects.However, a major challenge for AI methods in this context is learning how to grasp an object when a specific method is required or when certain easily damaged areas need to be avoided. While this task is often trivial from a human perspective, it is difficult to predefine general criteria that a machine learning model can consider.

[0003] It is an object of the invention to provide approaches that enable the automatic picking up of an object by a robot, wherein the robot takes into account which parts of the object should be avoided or preferred during picking. This object is achieved according to the claim set.

[0004] DE 10 2021 202 570 A1 concerns a method for locating the positions of objects from camera images of the objects.

[0005] DE 10 2021 103 726 A1 concerns a measurement parameter optimization procedure.

[0006] DE 10 2020 214 301 A1 relates to a method for controlling a robot to pick up an object in different positions.

[0007] DE 10 2019 206 444 A1 concerns machine learning for object recognition using a robot-guided camera.

[0008] DE 10 2014 223 167 A1 relates to a method and a system for determining reach spaces on an object.

[0009] EP 2 263 837 B1 concerns the learning of a gripping process for a work object whose shape and 3D position are unknown to the robot. Disclosure of the invention

[0010] According to various embodiments, a method for controlling a robot to manipulate, in particular pick up, an object is provided, comprising: obtaining an image showing the object; generating, from the image, a manipulation quality image in which, for each pixel representing a location on the surface of the object, the pixel value of the pixel provides an estimate of how well the object can be manipulated at that location; registering descriptors of locations of the object that, according to user input, are to be used during manipulation and / or of locations that, according to user input, are to be avoided during manipulation; mapping the image onto a descriptor image; and generating, by comparing the registered descriptors of locations with the descriptor image, a manipulation preference image in which, for each pixel representing a location on the surface of the object, the pixel value of the pixel provides an estimate of how well the object can be manipulated at that location.whether the object should be manipulated at that location, selecting a location to manipulate the object taking into account the pixel values ​​of the manipulation quality image and the pixel values ​​of the manipulation preference image, and controlling the robot to manipulate the object at the selected location.

[0011] The method described above allows manipulation in areas that are not only easy to manipulate (with regard to their geometric or physical properties, e.g., no edges, not too rough), but should also be manipulated (according to user input and thus user preference), or avoids manipulation in areas that should be avoided from a user's point of view.

[0012] Manipulation can refer in particular to picking up (e.g., grasping or suction in the case of a suction gripper). The challenges of picking up described below, and the measures provided for in various embodiments, can arise in other activities such as turning a key, pressing a button, or pulling a lever, and can be successfully applied in these situations as well.

[0013] The detection of manipulation points is done pixel by pixel (e.g., for pixels of an input image, but resolution changes can also be made when creating the manipulation quality image and the manipulation preference image; for example, these two images can have a reduced resolution).

[0014] The following are various examples of implementation.

[0015] Exemplary embodiment 1 is a method for controlling a robot to manipulate, in particular pick up, an object as described above.

[0016] Embodiment 2 is a method according to embodiment 1, wherein the manipulation quality image is mapped to manipulation quality images by means of a first neural network trained to map images of objects and / or information derived from images of objects, and / or wherein the image is mapped to the descriptor image by means of a second neural network trained to map images of objects to descriptor images.

[0017] The use of neural networks enables the efficient execution of the respective tasks for generating the manipulation quality image or the manipulation preference image. The (input) image can contain color information (or grayscale values) and / or depth information.

[0018] Exemplary embodiment 3 is a method according to exemplary embodiment 1 or 2, comprising combining the manipulation quality image and the manipulation preference image and selecting the location to manipulate the object using the combination.

[0019] This creates a combined image (e.g., by combining pixels), which makes it possible to find a suitable place to manipulate, especially to pick up (grab or suck up) the object, by searching for a maximum or minimum pixel value in the combined image (e.g., depending on the encoding of the manipulation qualities or preferences or their combination).

[0020] Embodiment 4 is a method according to one of embodiments 1 to 3, wherein the estimation of whether the object should be manipulated at a given location is the degree of correspondence between the descriptor assigned to that location in the descriptor image and one of the registered descriptors. Since each location of the object is assigned a pixel in the (input) image, and this is mapped onto the descriptor image, each location of the object is correspondingly assigned a pixel in the descriptor image and thus also a descriptor (which is the pixel value of that pixel). Analogously, locations of the object are represented by pixels in the various images used and generated here, or rather, pixel values ​​are assigned to the locations.

[0021] This means the estimate also expresses an uncertainty as to whether a location should be avoided or used. This uncertainty can be accounted for by appropriately considering the pixel values ​​of the manipulation quality score when selecting the location for manipulation.

[0022] Exemplary embodiment 5 is a method according to one of the exemplary embodiments 1 to 4, comprising generating the manipulation quality image by forming, for each registered descriptor, a descriptor match image, wherein, for each pixel representing a location on the surface of the object, the pixel value of the pixel indicates how well the descriptor assigned to the location in the descriptor image matches the registered descriptor, and combining the descriptor match images to form the manipulation quality image.

[0023] For example, the descriptor-match images (e.g., "heatmaps") can be combined by taking the maximum pixel-by-pixel value across the descriptor-match images (or the minimum, depending on how the match is encoded in the pixel values). This allows for the efficient generation of the manipulation quality image.

[0024] Exemplary embodiment 6 is a method according to exemplary embodiment 5, comprising combining the manipulation quality image and the manipulation preference image by pixel-wise multiplication, pixel-wise maximum formation, pixel-wise minimum formation, excluding locations for which the manipulation quality image indicates a manipulation quality below a predetermined minimum quality, excluding locations for which the manipulation preference image indicates a match with a descriptor registered for a location to be used during manipulation according to the user input below a predetermined minimum match, and / or excluding locations for which the manipulation preference image indicates a match with a descriptor registered for a location to be avoided during manipulation according to the user input above a predetermined maximum match.

[0025] In this way, a combined image can be efficiently generated that indicates where the object can and should be manipulated.

[0026] Exemplary embodiment 7 is a robot control device configured to perform a method according to one of the exemplary embodiments 1 to 6.

[0027] Embodiment 8 is a computer program with instructions which, when executed by a processor, cause the processor to perform a method according to one of embodiments 1 to 6.

[0028] Exemplary embodiment 9 is a computer-readable medium that stores instructions which, when executed by a processor, cause the processor to perform a procedure according to one of Exemplary Embodiments 1 to 6.

[0029] In the drawings, similar reference numerals generally refer to the same parts in all the different views. The drawings are not necessarily to scale, with the emphasis generally placed on illustrating the principles of the invention. Various aspects are described in the following description with reference to the drawings. Fig. Figure 1 shows a robot. Fig. Figure 2 illustrates the determination of a location for receiving an object according to one embodiment. Fig. Figure 3 shows a flowchart illustrating a procedure for controlling a robot to manipulate, in particular pick up, an object.

[0030] The following detailed description refers to the accompanying drawings, which illustrate specific details and aspects of this disclosure in which the invention can be implemented. Other aspects may be used, and structural, logical, and electrical modifications may be made without deviating from the scope of the invention. The various aspects of this disclosure are not necessarily mutually exclusive, as some aspects of this disclosure may be combined with one or more other aspects of this disclosure to form new aspects.

[0031] Several examples are described in more detail below.

[0032] Fig. Figure 1 shows a robot 100.

[0033] The robot 100 includes a robot arm 101, for example, an industrial robot arm for handling or assembling a workpiece (or one or more other objects). The robot arm 101 includes manipulators 102, 103, 104 and a base (or support) 105, which supports the manipulators 102, 103, 104. The term "manipulator" refers to the movable components of the robot arm 101, the actuation of which enables physical interaction with the environment, for example, to perform a task. For control, the robot 100 includes a (robot) control unit 106, which is designed to implement the interaction with the environment according to a control program. The last component 104 (which is furthest from the support 105) of the manipulators 102, 103, 104 is also referred to as the end effector 104 and may include one or more tools, such as a welding torch, a gripping instrument, a painting device or the like.

[0034] The other manipulators 102, 103 (which are located closer to the support 105) can form a positioning device, so that, together with the end effector 104, the robot arm 101 is provided with the end effector 104 at its end. The robot arm 101 is a mechanical arm that can provide similar functions to a human arm (possibly with a tool at its end).

[0035] The robot arm 101 can include joint elements 107, 108, 109 that connect the manipulators 102, 103, 104 to each other and to the support 105. A joint element 107, 108, 109 can have one or more joints, each of which can provide rotatable (i.e., rotational) and / or translational (i.e., displacement) movement for associated manipulators relative to each other. The movement of the manipulators 102, 103, 104 can be initiated by means of actuators controlled by the control unit 106.

[0036] The term "actuator" can be understood as a component designed to effect a mechanism or process in response to its drive. The actuator can translate instructions (called activation) generated by the control unit 106 into mechanical movements. The actuator, e.g., an electromechanical transducer, can be designed to convert electrical energy into mechanical energy in response to its activation.

[0037] The term "controller" can be understood as any type of logic-implementing entity, which may include, for example, a circuit and / or a processor, capable of executing software, firmware, or a combination thereof stored on a storage medium, and capable of issuing instructions, e.g., to an actuator in this example. The controller can be configured, for example, by program code (e.g., software) to control the operation of a system, a robot in this example.

[0038] In the present example, the control unit 106 includes one or more processors 110 and a memory 111 that stores code and data based on which the processor 110 controls the robot arm 101. According to various embodiments, the control unit 106 controls the robot arm 101 based on a machine learning model 112 stored in the memory 111.

[0039] According to various embodiments, the machine learning model 112 is designed and trained to enable the robot 100 to recognize locations on an object 113 where the robot 100 can pick up the object 113 (or interact with it in other ways, e.g. by painting it).

[0040] The robot 100 can, for example, be equipped with one or more cameras 114 that enable it to take pictures of its workspace. The camera 114 is, for example, attached to the robot arm 101, so that the robot can take pictures of the object 113 from different perspectives by moving its robot arm 101 around.

[0041] According to various embodiments, the machine learning model 112 is a neural network 112, and the control unit 106 feeds the neural network 112 input data based on one or more digital images (color images, depth images, or both) of an object 113. The neural network 112 is configured to indicate locations (or areas) of the object 113 that are suitable for capturing the object 113. For example, the neural network can segment an input image showing the object 113 accordingly, e.g., assign each pixel a value (“manipulation quality value” or, in the present application, a “capture quality value”) indicating how well the pixel is suited for capture. The control unit 106 can then select a sufficiently large area as a capture location where these values ​​are sufficiently high (e.g., above a threshold, at their maximum on average, etc.).

[0042] According to various embodiments, the raw image data supplied by the cameras is preprocessed by the control unit 106 instead of being fed directly to the neural network 112 as input. The result of this preprocessing (possibly together with at least part of the raw image data) is used as input to the neural network 112.

[0043] One possibility for this is, for example, to calculate the standard deviation (or another measure of dispersion such as variance) of the surface normal vectors of objects shown in the digital images during preprocessing. The normal vector standard deviation is suitable for representing the local flatness of a surface and is therefore particularly relevant information for gripping quality (or for the quality of an object area for suction).

[0044] For example, one or more cameras, such as camera 114, provide raw image data (i.e., one or more images) containing, in particular, depth information about an object. From this depth information, normal vectors of the object's surface and their standard deviations (in different areas of the surface) are determined in a preprocessing step. This is fed to neural network 112 as input. The input to neural network 112 can also include parts of (or all of) the raw image data or image data generated from it after prior image enhancement preprocessing (e.g., noise reduction). Such prior image enhancement preprocessing can also be used to generate image data that is then used as the basis for determining the standard deviations. The neural network is trained on this data (e.g., using appropriate training inputs (including...).(normal vector standard deviations) and associated target outputs, i.e., ground truth information for supervised learning), to map the input to an output that identifies locations or areas of the object that are (e.g., particularly well) suited to capturing the object.

[0045] Various architectures can be used for the neural network 112. The neural network 112 can have a single input channel for the standard deviations of the normal vectors or a multi-channel input that includes the standard deviation of the normal vectors as data from one channel in addition to other data (e.g., color image data and / or depth image data). Examples include fully convolutional networks (e.g., UNet, ResNet), which assign a value (indicating the suitability of the respective location for capturing the displayed object) to each pixel of an input image in order to create an output image of the same size as the input image.

[0046] Using the above approach, for example, if normal vectors are taken into account as described above, a manipulation quality image can be generated from an input image taken by a camera 114, in which, for each pixel, the pixel value (which is e.g. in the interval [0, 1]) indicates how well an object can be captured at a location corresponding to the pixel (i.e., represented by a pixel in the input image whose position in the input image corresponds to the position of the pixel in the manipulation quality image).

[0047] However, selecting a location on an object based on the manipulation quality image (e.g., gripping at the location corresponding to a pixel with the highest pixel value in the manipulation quality image) can yield suboptimal results. This is because, for example, while the object can be picked up (or generally manipulated) well at that location, further processing may not be as effective if it is picked up at that location, e.g., because a part to be processed or a barcode is obscured by a gripper, or because the location is sensitive.

[0048] Therefore, according to various embodiments, a location for picking up an object is combined based on a combination of manipulation quality values, as provided by a manipulation quality image and indicating how well locations can be gripped, with values ​​(referred to herein as "manipulation preference values" or in the present application as "pickup preference values") that indicate whether locations should be gripped.

[0049] To generate a manipulation preference image with such manipulation preference values, according to various embodiments, a further neural network, e.g. as a further machine learning model 115 in the control unit 106, is used, which maps input images to descriptor space images.

[0050] An example of such a further neural network 115 is a dense object network. A dense object network (DON) maps an image to an arbitrarily dimensional (dimension D) descriptor space image. The dense object network is a neural network trained using self-supervised learning to output a descriptor space image for an input image. This allows images of known objects to be mapped to descriptor images containing descriptors that identify locations on the objects, independent of the image's perspective. A separate DON can be provided for each object type.

[0051] Such a neural network 115 (hereinafter referred to as DON as an example) makes it possible to generate a manipulation preference image from an input image as follows: (1) For each object type, a (reference) image of an object of that object type is mapped to a descriptor image by the respective DON 115. (2) The user selects pixels (e.g., by clicking) on ​​the image of the object where the object is to be captured, and the descriptors of the selected pixels (i.e., the descriptor value to which the pixels were mapped by DON 115) are registered. The user can also select pixels, for example, by selecting areas, e.g., by clicking on the vertices of a polygon or pixels within a corresponding convex hull. This is repeated until all areas or locations on the surface desired by the user have been selected (and their corresponding descriptors registered). Similarly, the user can also select locations to be avoided during capture. (3) (1) and (2) can be repeated for multiple reference images to capture all sides of an object. This provides a set of known (and registered) descriptors that correspond to preferred (or to be avoided) locations for recording. Using DON, these locations of an object recognized in the input image can be identified by searching for the registered descriptors in the input image. The process then proceeds as follows: (4) For the input image a. Does the DON 115 determine a descriptor image from the input image? b. The descriptor image for the input image is searched for the registered descriptors, and the manipulation preference image (for example, with pixel values ​​in the interval [0, 1]) is generated such that the pixel value of a pixel indicates how well the pixel's descriptor matches one of the registered descriptors, thus visually representing a "(descriptor match) heatmap" with respect to the match with the registered descriptors and therefore the locations selected by the user. This can be done, for example, by generating a separate heatmap for each registered descriptor and then calculating the manipulation preference image as the (pixel-wise) maximum across all these heatmaps.

[0052] Fig. Figure 2 illustrates the determination of a location for receiving an object according to an embodiment, for example carried out by the control device 106.

[0053] An input image 201 is fed to a first neural network 202 (corresponding to neural network 112), which generates a manipulation quality image 203 (in the present case, where the manipulation is a recording, a recording quality image).

[0054] The input image 201 is fed to a second neural network 204 (corresponding to neural network 115), which generates a descriptor image 205 for the input image 201. By comparing this with the registered descriptors 208, a heatmap 206, i.e., a manipulation preference image (in this case, where the manipulation is recording, a manipulation preference image), is generated from the descriptor image 205.

[0055] The manipulation quality image 203 and the manipulation preference image 206 are combined, here e.g. by pixel-wise multiplication of the pixel values ​​of the two images.

[0056] The control unit 106 then selects the point at which the robot arm 101 grasps the object 113 based on the combination 207 of the manipulation quality image with the manipulation preference image.

[0057] For example, a combined image 207 can be generated (e.g. by pixel-wise multiplication of the manipulation quality image with the manipulation preference image) and further processing can be carried out to determine a location for recording or a recording pose, e.g. by selecting a global maximum in the combined image.

[0058] The determination of image quality is thus complemented by the determination of image preferences using a neural network that generates descriptor images (e.g., a DON). A user can manually specify preferred areas of objects for a small number of (initial or reference) images, and the control unit 106 recognizes these areas based on their descriptors for new input images and can therefore (provided their image quality is also sufficiently high) prioritize them over other areas. Since the user only needs to mark a small number of areas or locations, the approach described above can also be easily used by inexperienced users.

[0059] The neural network 112, which determines the recording qualities, can be trained in a supervised manner from existing input data (e.g. RGB-D images) that are annotated so that they are labeled with locations on the surface of objects where the objects can be recorded.

[0060] Appropriate training data can be generated using a DON (Design of Objects) by automatically creating the target output of neural network 103. For this purpose, areas suitable for recording are marked on the surfaces of known (training) objects (this can also be done indirectly by marking areas that are unsuitable). By detecting these areas in the training input images, the corresponding areas in the training input images can be automatically marked. The detection of these areas in the training input images can, in turn, be achieved using descriptors.

[0061] The training of the DON to generate object and scene representations (in the form of descriptor images) can be self-monitored. For example, a method can be used where a static (i.e., stationary, e.g., permanently mounted) camera is used to capture the images, i.e., training images are taken that show a particular training scene from a single perspective. Training data for the machine learning model is then generated by augmenting the captured training images of the training scene. This is done in two (or more) ways, so that for each captured training image, the training data contains multiple augmented versions of the captured training image. The corresponding pixels can then be determined based on the augmentations used.Such augmentations (which enable this and can therefore be used according to various embodiments) include, for example, random rotations, perspective transformations, and affine transformations. The DON can then be trained using contrast loss with training image pairs, where each training image pair consists of the acquired training image and an augmentation of the training image, or two augmented versions of the training image. Descriptor values ​​generated by the DON for corresponding pixels are used as positive pairs, and descriptor values ​​generated by the machine learning model for non-corresponding pixels are used as negative pairs.

[0062] The DON's training is carried out, for example, as follows: (1) A set of object types is selected that the robot should be able to work with (i.e., that it should be able to pick up automatically). (2) Objects of the object types are randomly placed in a container in the work area of ​​robot 100. (3) An RGB-D image of the workspace is captured. (4) The RGB-D image is augmented and the DON is self-monitored and trained using the augmentations (and e.g. contrast loss).

[0063] In summary, a method is provided according to various embodiments, as described in Fig. 3 shown.

[0064] Fig. Figure 3 shows a flowchart 300 illustrating a procedure for controlling a robot to manipulate, in particular pick up, an object.

[0065] In 301, an image is obtained that shows the object.

[0066] In 302, a manipulation quality image is generated from the image, in which, for each pixel representing a location on the surface of the object, the pixel value of the pixel provides an estimate of how well the object can be manipulated at that location.

[0067] In 303, descriptors of parts of the object that, according to user input, are to be used during manipulation, and / or of parts that, according to user input, are to be avoided during manipulation, are registered.

[0068] In 304, the image is mapped onto a descriptor image.

[0069] In 305, by comparing the registered descriptors of locations with the descriptor image, a manipulation preference image is generated in which, for each pixel representing a location on the surface of the object, the pixel value of the pixel provides an estimate of whether the object should be manipulated at that location.

[0070] In 306, a location for manipulating the object is selected taking into account the pixel values ​​of the manipulation quality image and the pixel values ​​of the manipulation preference image.

[0071] In 307, the robot is controlled to manipulate the object at the selected location.

[0072] In other words, according to various implementations, annotations based on user preferences are combined with AI-guided prediction of manipulation points using a trained object representation model (e.g., object-centered, self-monitoring) that generates descriptors, such as a DON. The object representation model allows a human user to specify points that are likely to lead to greater success (during capture or further processing), and these points are automatically recognized for newly captured input images.

[0073] It should be noted that the processing of Fig. 3. The steps do not necessarily have to be performed in the order shown. For example, the manipulation preference image can also be generated before the manipulation quality image.

[0074] The procedure of Fig.3. This can be performed by one or more computers with one or more data processing units. The term "data processing unit" can be understood as any type of entity that enables the processing of data or signals. The data or signals can, for example, be processed according to at least one (i.e., one or more than one) special function performed by the data processing unit. A data processing unit can include or be composed of an analog circuit, a digital circuit, a logic circuit, a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an integrated circuit, a programmable gate array (FPGA), or any combination thereof.Any other method for implementing the respective functions described in more detail herein may also be understood as a data processing unit or logic circuit arrangement. One or more of the process steps described in detail herein may be executed (e.g., implemented) by a data processing unit through one or more special functions performed by the data processing unit.

[0075] Input data for machine learning models can include, for example, color and depth images. These can also be supplemented by sensor signals from other sensors such as radar, LiDAR, ultrasound, motion, thermal imaging, etc. For instance, an RGB and depth image is captured in a robot cell, and the image (or several such images) is (or are) used to generate candidate locations for grasping one or more objects and to create a descriptor-matching heatmap based on annotations according to user preference. The likelihood heatmap can then be used to discard candidate locations with low likelihood values: candidate locations whose descriptors are close to the registered descriptor values, i.e.,Candidate locations whose descriptors differ significantly from the registered descriptor values ​​are accepted. For example, if the heatmap value at a pixel position is below a certain threshold, a candidate location at that pixel position will be rejected.

[0076] From the remaining candidate locations, the robot can then (randomly) select a location and grasp (or suck up) the respective object at the selected location.

[0077] These embodiments can be used to train a machine learning system and control a robot, e.g., autonomously from robot manipulators, to accomplish various manipulation tasks under different scenarios. In particular, these embodiments are applicable to controlling and monitoring the execution of manipulation tasks, e.g., in assembly lines.

Claims

[1] Method (300) for controlling a robot (100) for manipulating, in particular picking up, an object, comprising: Obtaining (301) an image showing the object; generating (302) from the image a manipulation quality image in which, for each pixel representing a location on the surface of the object, the pixel value of the pixel indicates an estimate of how well the object can be manipulated at the location; Registering (303) descriptors of locations of the object which, according to a user input, are to be used during manipulation and / or of locations which, according to a user input, are to be avoided during manipulation; mapping (304) the image to a descriptor image; Generating (305), by comparing the registered descriptors of locations with the descriptor image, a manipulation preference image, in which, for each pixel representing a location on the surface of the object, the pixel value of the pixel indicates an estimate of whether the object should be manipulated at that location; Selecting (306) a location for manipulating the object taking into account the pixel values ​​of the manipulation quality image and the pixel values ​​of the manipulation preference image; and Controlling (307) the robot (100) to manipulate the object at the selected location. [2] The method of claim 1, wherein the manipulation quality image is mapped to the descriptor image by means of a first neural network trained to map images of objects and / or information derived from images of objects to manipulation quality images, and / or wherein the image is mapped to the descriptor image by means of a second neural network trained to map images of objects to descriptor images. [3] The method of claim 1 or 2, comprising combining the manipulation quality image and the manipulation preference image and selecting the location for manipulating the object using the combination. [4] A method according to any one of claims 1 to 3, wherein the estimate of whether the object is to be manipulated at the location is the degree of correspondence of the descriptor associated with the location in the descriptor image with one of the registered descriptors. [5] A method according to any one of claims 1 to 4, comprising generating the tamper quality image by forming, for each registered descriptor, a descriptor match image, wherein, for each pixel representing a location on the surface of the object, the pixel value of the pixel indicates how well the descriptor associated with the location in the descriptor image matches the registered descriptor, and combining the descriptor match images to form the tamper quality image. [6] The method of claim 5, comprising combining the manipulation quality image and the manipulation preference image by pixel-wise multiplication, pixel-wise maximum formation, pixel-wise minimum formation, excluding locations for which the manipulation quality image indicates a manipulation quality below a predetermined minimum quality, excluding locations for which the manipulation preference image indicates a match with a descriptor registered for a location to be used during manipulation according to the user input below a predetermined minimum match and / or excluding locations for which the manipulation preference image indicates a match with a descriptor registered for a location to be avoided during manipulation according to the user input above a predetermined maximum match. [7] Robot control device arranged to carry out a method according to one of claims 1 to 6. [8] A computer program comprising instructions which, when executed by a processor (110), cause the processor to perform a method according to any one of claims 1 to 6. [9] A computer-readable medium (111) storing instructions which, when executed by a processor, cause the processor (110) to perform a method according to any one of claims 1 to 6.

Images