Method for controlling robot for manipulating, in particular receiving, object

Abstract

To provide a method for controlling a robot for manipulating, in particular receiving, an object.SOLUTION: A method includes: acquiring an image which depicts an object; generating a manipulation-quality image from the image, in which, for each pixel which represents a point on the surface of the object, the pixel value of the pixel provides an assessment of how well the object may be manipulated at the point; registering descriptors of points of the object which should be used during the manipulation according to the user input and / or of points which should be avoided during the manipulation according to the user input; mapping the image onto a descriptor image; generating a manipulation-preference image that indicates an assessment on whether or not the object should be manipulated at the points with the pixel value of the pixel for each pixel that represents one or more points on the surface of the object by comparing the registered descriptors of points to the descriptor image; selecting a point for manipulating the object taking into account the pixel values of the manipulation-quality image and the pixel values of the manipulation-preference image; and controlling the robot to manipulate the object at the selected point.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] Background technology The present disclosure relates to a method for controlling a robot that manipulates an object, and in particular a robot that receives an object. [Background technology]

[0002] When accepting an object by a robot, there are several approaches to identify promising object locations for grasping, for example to remove the object from a container. These approaches typically work with RGBD images (i.e. images with color and depth information) of the scene, although in some cases either RGB or depth images are sufficient. Furthermore, many of these approaches are based on the use of AI methods, e.g. neural networks, that learn the correspondence between input data and promising grasping points. It is therefore crucial that these AI methods are able to reliably identify graspable areas on the surface of the object. However, a major challenge for AI methods is to learn how to grasp an object, if the object should be handled in a specific way or if certain areas that are vulnerable to damage should be avoided. This challenge is often trivial from a human point of view, but it is difficult to set in advance general criteria that can be taken into account by machine learning models. Summary of the Invention [Problem to be solved by the invention]

[0003] Therefore, what is desired is an approach that allows a robot to automatically accept an object while taking into account which parts of the object the robot should avoid or prioritize during acceptance. [Means for solving the problem]

[0004] Disclosure of the Invention Various embodiments provide a method for controlling a robot for manipulating an object, in particular a robot for accepting an object, comprising: acquiring an image showing the object; generating from the image a manipulation quality image for each pixel representing a location on a surface of the object, the pixel value of the pixel indicating an estimate of how well the object can be manipulated at that location; registering descriptors of locations of the object to be utilized during manipulation according to user input and / or locations of the object to be avoided during manipulation according to user input; mapping the image to a descriptor image; comparing the registered descriptors of the multiple locations with the descriptor image to generate a manipulation priority image for each pixel representing a location on the surface of the object, the pixel value of the pixel indicating an estimate of whether the object should be manipulated at that location or not; selecting a location for manipulating the object taking into account the pixel values ​​of the manipulation quality image and the pixel values ​​of the manipulation priority image; and controlling the robot to manipulate the object at the selected location.

[0005] The above-described method allows not only manipulation of locations that can be manipulated well (in terms of their geometric or physical characteristics, e.g., no edges, not overly rough surfaces, etc.), but also manipulation of locations that should be manipulated (according to user input and thus user preferences) or avoidance of manipulation of locations that should be avoided from the user's point of view.

[0006] Operation may in particular mean reception (e.g. grasping or suction in the case of a suction gripper). The tasks of reception described below and the measures dedicated in the various embodiments can be taken or applied with advantage in other actions such as turning a key, pressing a button or pulling a lever, just as in the case of reception.

[0007] In this case, detection of the operation location is performed on a pixel-by-pixel basis (e.g., on the pixels of the input image, but a resolution change can also be performed when creating the operation quality image and the operation priority image, e.g., both types of images can have a reduced resolution).

[0008] Various examples are given below.

[0009] Example 1 is the above-mentioned method for controlling a robot that manipulates an object, particularly a robot that receives an object.

[0010] Example 2 is a method as described in Example 1, wherein the operation quality image is mapped using a first neural network trained to map an image of an object and / or information derived from the image of the object to the operation quality image, and / or the image is mapped to a descriptor image using a second neural network trained to map an image of an object to a descriptor image.

[0011] The use of neural networks makes it possible to efficiently perform the respective tasks of generating a manipulation-quality image or a manipulation-priority image. The (input) image may for this purpose contain color information (or even grey values) and / or depth information.

[0012] Example 3 is a method as described in example 1 or 2, where the method includes combining an operation quality image and an operation priority image and using the combination to select a location for operating the object.

[0013] In this way, a combined image is generated (e.g. by pixel-by-pixel combination), and by searching for a location in the combined image (e.g. by searching for a maximum or minimum pixel value according to an encoding of the manipulation quality or manipulation priority or a combination of these), it is possible to find a location that is well suited for manipulating, and in particular for receiving (grasping or grabbing) an object.

[0014] Example 4 is a method according to any one of Examples 1 to 3, wherein the estimation of whether the object should be manipulated at the location is the degree of match between a descriptor associated with the location in the descriptor image and one of the registered descriptors. Each location of the object is associated with a pixel in the (input) image, which is mapped to the descriptor image, so that each location of the object is associated with a pixel in the descriptor image and thus with a descriptor (which is the pixel value of this pixel). Similarly, the location of the object is represented by a pixel in the various images used and generated herein, or is associated with a (pixel) value of the location.

[0015] In this way, the estimation also expresses the uncertainty about whether a certain location should be avoided or utilized, which can be taken into account by taking into account the pixel values ​​of the manipulation quality value accordingly when selecting the manipulation location.

[0016] Example 5 is a method according to any one of Examples 1 to 4, wherein the method includes generating an operation quality image by forming, for each pixel representing a location on a surface of the object, a descriptor match image indicating how well the pixel value of the pixel matches the descriptor associated with the location in the descriptor image with the registered descriptor, and combining the descriptor match image into the operation quality image.

[0017] For example, the descriptor match images (e.g., "heat maps") can be combined by taking the maximum across the descriptor match images for each pixel (or also the minimum, depending on the encoding of the match in the pixel values), allowing for efficient generation of an operational quality image.

[0018] Example 6 is a method according to Example 5, wherein the method includes combining an operation quality image and an operation priority image by pixel-wise multiplication, forming a maximum value per pixel, and forming a minimum value per pixel, excluding locations where the operation quality image exhibits an operation quality below a set minimum quality, excluding locations where the operation priority image exhibits a match with registered descriptors below a set minimum degree of match for locations that should be utilized during operation in accordance with user input, and / or excluding locations where the operation priority image exhibits a match with registered descriptors above a set maximum degree of match for locations that should be avoided during operation in accordance with user input.

[0019] In this manner, a combined image showing both where an object can be manipulated and where an object should be manipulated can be efficiently generated.

[0020] Example 7 is a robot controller configured to perform the method according to any one of examples 1 to 6.

[0021] Example 8 is a computer program comprising instructions for causing a processor to carry out the method according to any one of Examples 1 to 6 when executed by the processor.

[0022] Example 9 is a computer-readable medium having stored thereon instructions that, when executed by a processor, cause the processor to perform the method according to any one of Examples 1 to 6.

[0023] In the drawings, like reference characters generally refer to like parts throughout the various views. The drawings are not necessarily drawn to scale, emphasis instead being placed upon illustrating the principles of the invention as a whole. In the following description, various aspects are described with reference to the following drawings: [Brief description of the drawings]

[0024] [Figure 1] FIG. [Diagram 2] FIG. 1 illustrates the calculation of a location to receive an object, according to one embodiment. [Diagram 3] 1 is a flow chart illustrating a method for controlling a robot to manipulate an object, and in particular to receive an object. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] The following detailed description is made with reference to the accompanying drawings, which are shown for the purpose of illustrating specific details and aspects of the present disclosure in which the present invention can be practiced. Other aspects may be used, and structural, logical, and electrical changes may be made without departing from the scope of the present invention. Various aspects of the present disclosure are not necessarily mutually exclusive, since some aspects of the present disclosure may be combined with one or more other aspects of the present disclosure to form new aspects.

[0026] Various embodiments are described in detail below.

[0027] In FIG. 1, a robot 100 is shown.

[0028] The robot 100 comprises a robot arm 101, for example an industrial robot arm for handling or assembling a workpiece (or one or more other objects). The robot arm 101 comprises manipulators 102, 103, 104 and a base (or support) 105, on which the manipulators 102, 103, 104 are supported. The expression "manipulator" refers to a movable component of the robot arm 101, which can be manipulated to allow physical interaction with the surroundings, for example to perform a task. For control, the robot 100 comprises a (robot) controller 106, which is configured to realize the interaction with the environment according to a control program. The last component 104 of the manipulators 102, 103, 104 (farthest from the support 105), also called end effector 104, can include one or more tools, such as a welding burner, a gripper tool, a lacquering device, etc.

[0029] The other manipulators 102, 103 (located closer to the support 105) may form a positioning device which, together with the end effector 104, provides a robotic arm 101 terminated with an end effector 104. The robotic arm 101 is a mechanical arm that can provide functions similar to a human arm (possibly with a tool at its end).

[0030] The robotic arm 101 may include joint members 107, 108, 109 that couple the manipulators 102, 103, 104 to each other and to a support 105. The joint members 107, 108, 109 may include one or more joints, each of which may provide rotatable (i.e., rotational) and / or translational (i.e., translation) movement of the associated manipulators relative to each other. Movement of the manipulators 102, 103, 104 may be initiated by actuators controlled by a controller 106.

[0031] The expression "actuator" can be understood as a component configured as a response to a drive device that causes a mechanism or process. The actuator can realize an instruction (so-called actuation) formed by the control device 106 as a mechanical movement. The actuator, for example an electromechanical converter, can be configured to convert electrical energy into mechanical energy in response to the actuation.

[0032] The expression "controller" may include, for example, a circuit and / or a processor capable of executing software, firmware, or a combination thereof stored in a storage medium, and / or may be understood as any kind of entity implementing logic that can output instructions to the actuators in this embodiment. The controller may be configured, for example, by program code (e.g., software) to control, for example, the operation of the system, i.e., the robot.

[0033] In this example, the controller 106 includes one or more processors 110 and a memory 111 that stores code and data for the processor 110 to control the robotic arm 101. According to various embodiments, the controller 106 controls the robotic arm 101 based on a machine learning model 112 stored in the memory 111.

[0034] According to various embodiments, the machine learning model 112 is designed and trained to enable the robot 100 to identify locations on the object 113 where the robot 100 may accept the object 113 (or interact with the object in other ways, e.g., lacquer it).

[0035] The robot 100 may, for example, be equipped with one or more cameras 114 capable of taking images of its workspace. The cameras 114 are, for example, attached to the robot arm 101, so that the robot can generate images of the object 113 from different viewpoints by moving its robot arm 101.

[0036] According to various embodiments, the machine learning model 112 is a neural network 112, and the controller 106 provides the neural network 112 with input data based on one or more digital images (color images, depth images, or both) of the object 113, and the neural network 112 is configured to indicate locations (or regions) of the object 113 that are suitable for accepting the object 113. For example, the neural network can appropriately segment the input image showing the object 113, and can, for example, associate with each pixel a value ("handling quality value" or, as used herein, "acceptance quality value") indicating how well the pixel is suitable for acceptance. In this case, the controller 106 can select as acceptance locations those regions with sufficient size where these values ​​are sufficiently high (e.g., above a threshold, are on average maximum, etc.).

[0037] According to various embodiments, the raw image data sent from the camera is pre-processed by the controller 106, rather than being provided as a direct input to the neural network 112. The results of that pre-processing are used as input to the neural network 112 (possibly together with at least a portion of the raw image data).

[0038] One measure for this is, for example, the standard deviation (or another measure of scattering, e.g. variance) of the normal vectors of the surface of the object shown in the digital image during pre-processing. The standard deviation of the normal vectors is suitable for describing the local flatness of the surface and is therefore information that is particularly relevant for the gripping quality (or the quality for adsorbing an object region).

[0039] For example, one or more cameras, for example corresponding to camera 114, provide raw image data (i.e. one or more images) including depth information, particularly with respect to the object. From the depth information, in a pre-processing, normal vectors of the object's surface and their standard deviations (in various areas of the surface) are calculated. These are provided as inputs to the neural network 112. The inputs of the neural network 112 may include the raw image data or part (or all) of the image data generated from the raw image data following a previous image correction pre-processing (e.g. noise reduction). Such a previous image correction pre-processing can also be used to generate image data used as a basis for determining the standard deviation. The neural network is trained (e.g. with corresponding training inputs (including normal vector standard deviations) and corresponding target outputs, i.e. ground truth information for supervised learning) to map the inputs to outputs that identify places or areas of the object that are (e.g. particularly well) suitable for receiving the object.

[0040] Various architectures can be used for the neural network 112. The neural network 112 may have only a separate input channel for the standard deviation of the normal vectors, or may have a multi-channel input that includes the standard deviation of the normal vectors as one channel of data, in addition to another channel of data (e.g. color image data and / or depth image data). An example is a fully convolutional network (e.g. UNet, ResNet) that associates a value to each pixel of an input image, which value indicates the suitability of each location to receive the object shown, to generate an output image of the same size as the input image.

[0041] The above approach allows, for example, when taking into account normal vectors as described above, to generate from an input image captured by camera 114 an operation quality image that indicates how well an object can be received at a location corresponding to that pixel by a pixel value (e.g., in the interval [0,1]) (i.e., at a location represented by a pixel in the input image where the position of the pixel in the input image corresponds to the position of the pixel in the operation quality image).

[0042] However, selecting a location of the object based on the manipulation quality image (e.g., gripping at a location corresponding to the pixel with the highest pixel value in the manipulation quality image) may lead to suboptimal results, since although the object may indeed be well accepted at this location (or may be generally manipulable), if accepted at this location, it may not be well suited for further processing, e.g. because the location or barcode to be processed is obscured by the gripper or because this location is susceptible to damage.

[0043] Thus, according to various embodiments, a location for receiving an object is paired based on a combination of an operation quality value sent by the operation quality image regarding how well grasping is possible at that location and a value regarding whether grasping should be performed at that location (a value referred to in this specification as an "operation priority value" or a value referred to in this application as an "acceptance priority value").

[0044] To generate a manipulation priority image using such manipulation priority values, according to various embodiments, a separate neural network is used as a separate machine learning model 115 in the controller 106, for example, that maps the input image to a descriptor space image.

[0045] An example of such another neural network 115 is the Dense Object Network. A Dense Object Network (DON) maps an image into a descriptor space image of any dimension (dimension D). A Dense Object Network is a neural network trained using self-supervised learning and configured to output a descriptor space image for an input image of an image. In this way, an image of a known object can be mapped into a descriptor image, including descriptors that identify locations on the object independent of the viewpoint of the image. There may be a respective DON for each object type.

[0046] By using such a neural network 115 (hereinafter, referred to as DON as an example), it is possible to generate the following operation priority image for an input image. (1) For each object type, the respective DON 115 maps (reference) images of objects of the given object type to a descriptor image. (2) The user selects (e.g., by clicking) pixels on the image of the object where the object should be grasped, and the descriptors of the selected pixels (i.e., the descriptor values ​​to which the pixels are mapped by DON 115) are registered. For example, the user may also select pixels by selecting an area, e.g., by clicking on the vertices of a polygon or the vertices of pixels in the corresponding convex hull. This is repeated until all areas or points on the surface desired by the user have been marked (and their corresponding descriptors registered). Similarly, the user may also select points to be avoided upon acceptance. (3) (1) and (2) can be repeated for multiple reference images so that all sides of the object are detected. In this way, a set of descriptors corresponding to locations that are favorable for acceptance (or should be avoided) is known (and registered). DON can identify these locations of objects identified in the input image by searching the registered descriptors in the input image. Correspondingly, the following further proceeds: (4) For the input image, a. DON 115 computes a descriptor image from the input image; b. The registered descriptors are searched for in the descriptor image for the input image, and a manipulation priority image (e.g. having pixel values ​​in the interval [0,1]) is generated such that the pixel values ​​of the pixels indicate how well the descriptor of the pixel matches one of the registered descriptors, specifically in the form of a "(descriptor match) heat map" of matches with the registered descriptors and thus with the user selected portion. This can be done, for example, such that a respective heat map is generated for each registered descriptor, and the manipulation priority image is generated as the maximum (per pixel) value over all these heat maps.

[0047] FIG. 2 illustrates the calculation of a location to receive an object according to one embodiment, performed, for example, by controller 106.

[0048] An input image 201 is fed to a first neural network 202 (corresponding to neural network 112), which generates an action quality image 203 (an acceptance quality image in the present case where the action is acceptance).

[0049] The input image 201 is fed to a second neural network 204 (corresponding to neural network 115), which generates a descriptor image 205 for the input image 201. A heatmap 206, i.e., an action-preferred image (or an acceptance-preferred image in this case where the action is acceptance), is generated from the descriptor image 205 by comparison with registered descriptor images 208.

[0050] The action quality image 203 and the action preference image 206 are combined here, for example, by multiplying pixel values ​​of the two images pixel by pixel.

[0051] The control device 106 then selects a location for the robot arm 101 to grasp the object 113 based on a combination 207 of the operation quality image and the operation priority image.

[0052] For example, a combined image 207 may be generated (e.g., by pixel-wise multiplication of the operation quality image and the operation priority image) and further processed to determine acceptance locations or acceptance postures, e.g., by selecting global maxima in the combined image.

[0053] That is, the computation of the acceptance quality is supplemented by the computation of the acceptance priority using a neural network (e.g. DON) that generates a descriptor image. The user can manually address preferred object regions for a small number of (initial or reference) images, and the control device 106 can identify these regions based on the descriptors for new input images, thereby prioritizing them over other regions (as long as their acceptance quality is also high enough). Since the user only needs to mark a small number of regions or points, even inexperienced users can easily use the above approach.

[0054] The neural network 112 for computing the acceptance quality can be trained in a supervised manner from existing input data (e.g., RGB-D images) that have been annotated and therefore labeled to indicate the locations on the surface of the object that can accept the object.

[0055] Here too, the corresponding training data can be generated by automatically generating the target output of the neural network 103 by means of DON. For this purpose, areas that are well suited to receiving the surface of the known (training) object are marked (this can also be done indirectly by marking less suitable areas). By detecting such areas in the training input image, the areas in the training input image can be automatically marked. The detection of the areas in the training input image can also be done by means of a descriptor.

[0056] Training of the DON for generating object and scene representations (in the form of descriptor images) can be performed in a self-supervised manner. For example, for this purpose, a method is used in which a static (i.e. stationary, e.g. fixedly mounted) camera can be used for image capture, i.e. a training image showing each training scene from a single viewpoint can be captured. In this case, training data for the machine learning model is generated by augmenting the captured training image with the training scene. This is done in a two-time (or multiple-time) format, so that for each captured training image, the training data includes multiple augmented versions of the captured training image. It can then be calculated which pixels correspond to the augmentation used. Such augmentations (which allow the above and can therefore be used by various embodiments) include, for example, random rotations, viewpoint transformations and affine transformations. The DON can then be trained using training image pairs using contrast loss, where each training image pair has a captured training image and an augmentation of the training image or two augmented versions of the training image. Here, the descriptor values ​​generated by DON for pixels that correspond to each other are used as the positive pair, and the descriptor values ​​generated by the machine learning model for pixels that do not correspond to each other are used as the negative pair.

[0057] In this case, the training of DON is performed, for example, as follows: (1) A set of object types that the robot can or should work with (i.e., object types that the robot can or should automatically accept) is selected. (2) Objects of a given object type are randomly placed within a container in the working area of ​​the robot 100. (3) An RGB-D image of the working area is captured. (4) The RGB-D image is augmented and the DON is trained in a self-supervised manner using the augmentation (and, e.g., contrast loss).

[0058] In summary, a method according to various embodiments is provided, as shown in FIG.

[0059] FIG. 3 shows a flow chart 300 illustrating a method for controlling a robot to manipulate an object, and in particular to receive an object.

[0060] At 301, an image showing an object is captured.

[0061] At 302, a manipulation quality image is generated from the images that, for each pixel representing a location on the surface of the object, indicates an estimate of how well the object can be manipulated at this location given the pixel value of that pixel.

[0062] In 303, descriptors of parts of the object that are to be utilized during manipulation according to user input and / or descriptors of parts of the object that are to be avoided during manipulation according to user input are registered.

[0063] At 304, the image is mapped to a descriptor image.

[0064] In 305, by comparing the registered location descriptors with the descriptor image, a manipulation priority image is generated for each pixel representing a location on the surface of the object, where the pixel value of that pixel indicates an estimate of whether the object should be manipulated at that location.

[0065] At 306, a location for manipulating the object is selected taking into account pixel values ​​of the manipulation quality image and pixel values ​​of the manipulation preference image.

[0066] At 307, the robot is controlled to manipulate the object at the selected location.

[0067] In other words, according to various embodiments, annotation according to user preferences is combined with AI-driven prediction of manipulation locations using a trained (e.g., object-centric self-supervised) object representation model (a model that generates descriptors, e.g., DON). The object representation model allows the human user to handle locations that are expected to bring greater success (either upon acceptance or during further processing), and these locations are automatically identified for newly captured input images.

[0068] It should be noted here that the processes in Fig. 3 do not necessarily have to be performed in the order shown in the figure. For example, the operation priority image may be generated before the operation quality image.

[0069] The method of FIG. 3 can be performed by one or more computers with one or more data processing units. The term "data processing unit" can be understood as any type of entity capable of processing data or signals. The data or signals can be processed, for example, according to at least one (i.e., one or more) special functions performed by the data processing unit. The data processing unit can include or consist of analog circuits, digital circuits, logic circuits, microprocessors, microcontrollers, central processing units (CPUs), graphic processing units (GPUs), digital signal processors (DSPs), programmable gate devices (FPGAs) integrated circuits, or any combination thereof. Any other manner of implementing each function described in detail herein can also be understood as a data processing unit or logic circuit device. One or more of the individual method steps described herein can be performed (e.g., realized) by the data processing unit or by one or more special functions performed in the data processing unit.

[0070] As input data for the machine learning model, for example, color images and depth images are used. However, the input data can also be supplemented by sensor signals of other sensors, such as radar, LiDAR, ultrasound, motion, thermal images, etc. For example, RGB images and depth images are recorded in a robot cell, and this image (or several such images) is used to generate candidate locations for grasping one or more objects and to generate a descriptor match heat map based on annotations according to user preferences. In this case, the likelihood heat map can be used to discard candidate locations with low likelihood values. That is, candidate locations with descriptors in the vicinity of registered descriptor values, i.e., in the vicinity of the descriptors of the user selected locations, are accepted, while candidate locations with descriptors significantly different from the registered descriptor values ​​are discarded. For example, if the heat map value of a pixel location is below a threshold, the candidate location at that pixel location is discarded.

[0071] The robot can then (randomly) select one location from the remaining candidate locations and grasp (or attach) each object at the selected location.

[0072] The embodiments can be used for training machine learning systems and for autonomous control of robots, e.g., robotic manipulators, to accomplish various manipulation tasks in various scenarios, in particular, the embodiments are applicable to control and monitor the execution of manipulation tasks, e.g., in an assembly line.

[0073] Although specific embodiments have been illustrated and described herein, those skilled in the art will recognize that the illustrated and described specific embodiments may be replaced with various alternative and / or equivalent implementations without departing from the scope of protection of the present invention. This application is intended to cover any adaptations or variations of the specific embodiments described herein. Therefore, it is intended that the present invention be limited only by the claims and their equivalents.

Claims

1. A method for controlling a robot that manipulates objects, particularly a robot that receives objects, To obtain an image showing the aforementioned object, From the aforementioned image, for each pixel representing a single location on the surface of the object, an operation quality image is generated that shows an estimate of how well the object can be operated at that location based on the pixel value of that pixel, Registering descriptors for the parts of the object that are used during operation according to user input, and / or parts that should be avoided during operation according to user input, Mapping the aforementioned image to a descriptor image, By comparing registered descriptors for multiple locations with the descriptor image, an operation priority image is generated for each pixel representing a location on the surface of the object, indicating whether or not the object should be manipulated at that location based on the pixel value of that pixel. The location to operate the object is selected by considering the pixel values ​​of the operation quality image and the pixel values ​​of the operation priority image. Controlling the robot to manipulate the object at the selected location, A method that includes this.

2. The aforementioned operational quality image is mapped using a first neural network trained to map the image of the object and / or information derived from the image of the object to the operational quality image, and / or The aforementioned image is mapped to a descriptor image using a second neural network trained to map images of the object to descriptor images. The method according to claim 1.

3. Combining the aforementioned operation quality image and the aforementioned operation priority image, Select the location where the object will be operated using the said combination, The method according to claim 1, including the method described in claim 1.

4. The method according to claim 1, wherein the estimation of whether the object should be manipulated at the location is the degree of agreement between the descriptor associated with the location in the descriptor image and one of the registered descriptors.

5. The operation quality image is generated by, for each registered descriptor, forming a descriptor matching image for each pixel representing a single location on the surface of the object, which shows how well the descriptor associated with that location in the descriptor image matches the registered descriptor based on the pixel value of that pixel. The descriptor matching image is combined with the operation quality image, The method according to claim 1, including the method described in claim 1.

6. The operation quality image and the operation priority image are combined by pixel-by-pixel multiplication, pixel-by-pixel maximum value formation, and pixel-by-pixel minimum value formation. To exclude areas in the aforementioned operation quality image that show operation quality below the set minimum quality, The operation priority image excludes areas that should be used during operation according to user input and that show a match with a registered descriptor below the set minimum match degree, and / or excludes areas that should be avoided during operation according to user input and that show a match with a registered descriptor above the set maximum match degree. The method according to claim 5, including the method described in claim 5.

7. A robot control device configured to carry out the method described in any one of claims 1 to 6.

8. A computer program that, when executed by a processor, includes instructions causing the processor to perform the method according to any one of claims 1 to 6.

9. A computer-readable medium storing instructions for causing a processor to perform the method according to any one of claims 1 to 6 when executed by the processor.

Images