Method and computer system for selecting a robot tool

Abstract

A computer-implemented method (200) of selecting a tool from a collection of tools to be used by at least one industrial robot for performing a predefined robot task in a working environment, comprising: initializing (201) a machine-learning, ML, agent configured to control the industrial robot; initializing (202) a repeatable computer simulation of the working environment; starting (203) training of the ML agent in the computer simulation, wherein the ML agent controls the industrial robot in the working environment while it is equipped with a first tool in the collection, and wherein the ML agent is rewarded for completing the robot task or a portion thereof; monitoring (204), during said training, a time evolution of the ML agent's reward up to a predefined threshold, and recording an indicator relating to said time evolution; repeating (205) the preceding steps for each remaining tool in the collection; and selecting (206) one of the tools based on the recorded indicators.

Description

METHOD AND COMPUTER SYSTEM FOR SELECTING A ROBOT TOOLTECHNICAL FIELD

[0001] The present disclosure addresses a problem in the configuring and operating of industrial robots by proposing an automated method for selecting a tool for at least one industrial robot. The tool is selected from a collection of tools to be used by the industrial robot when performing a predefined robot task in a working environment.BACKGROUND

[0002] The problem of selecting a suitable end-effector tool to be used by an industrial robot in carrying out a robot task, such as a gripper tool for a pick-and- place application, has significant economic ramifications. A well-adapted tool may differ from its inferior alternatives with respect to the productivity rate (e.g., quantity of workpieces processed per unit time), incidence of damaged or dropped workpieces, percentage of system downtime, quality of the processed workpieces as perceived by the end user, etc. The selection problem can sometimes be complex to solve, especially in connection with robot tasks for which a wide range of potentially useful tools are on offer, or tasks made up of a long sequence of sub-tasks which have highly variable characteristics. In such cases, the operator has to either embark on a cumbersome evaluation of each tool, or apply some amount of intuition. Not all operators have accumulated the necessary experience to make intuitive decisions about robot tools. Even seasoned operators may have recourse to simulations or experiments for evaluating their initial tool selection and reconsidering the tool selection as needed.

[0003] In the prior art, there have been several attempts to automate the robottool selection, not only in the interest of saving manpower but also to reach a reasonable level of certainty that the chosen tool is optimal or near-optimal in an objective sense.

[0004] The robot tool selection may for example be entrusted to a machinelearning (ML) model which has previously been trained to output tool selections on the basis of images or other descriptions of workpieces to be picked and placed. The training may for example include assigning rewards to the ML model depending onhow suitable the selected tool is, and evolving the ML model iteratively with a view to maximize future rewards.

[0005] A more elaborate way of automating robot tool selection is known from the patent application published as US20230081119A1. The approach of that disclosure is to first select grasps, then identify end effectors that correspond to the selected grasps. The selection step is performed as a Markov Decision Process (MDP) with a reward function that balances pick success and a cost of changing end effectors. The selection step includes in particular feeding images of the objects to be picked to “grasp prediction models”, which can be provided as trained ML models. The approach according to US20230081119A1 maybe described as an ML-assisted direct grasp selection process, which is followed by a postprocessing step of identifying the corresponding end effector by conventional methods.SUMMARY

[0006] One objective of the present disclosure is to make available a method for selecting a tool from a collection of tools to be used by at least one industrial robot for performing a predefined robot task in a working environment. It is a particular objective to propose a tool-selection method which executes deterministically, and which produces tool-selection decisions on the basis of objective suitability criteria. It is a particular objective to propose a repeatable tool-selection method. A further objective is to make available a computer system with these characteristics.

[0007] At least some of these objectives are achieved by the invention, as defined by the independent claims. The dependent claims relate to advantageous embodiments.

[0008] In a first aspect of the present disclosure, there is provided a computer- implemented method of selecting a tool from a collection of tools to be used by at least one industrial robot for performing a predefined robot task in a working environment. The method comprises: initializing a machine-learning (ML) agent configured to control said at least one industrial robot; initializing a repeatable computer simulation of the working environment; starting training of the ML agent in the computer simulation, wherein the ML agent controls said at least one industrial robot in the working environment while the industrial robot is equipped with a first tool in the collection, and wherein the ML agent is rewarded for completing therobot task or a portion thereof; during said training, monitoring a time evolution of the ML agent’s reward up to a predefined threshold, and recording an indicator relating to said time evolution; repeating the preceding steps for each remaining tool in the collection; and selecting one of the tools based on the recorded indicators.

[0009] The inventors have realized that, for a given combination of a robot, environment and workpiece, an initialized ML agent will learn in finite time (convergence time) to control the robot such that it receives maximum rewards almost always. On this basis, the inventors propose to repeat the learning process while exchanging the end-effector tool and re-initializing the ML agent each time. Because all other circumstances remain constant, the time evolution of a particular learning process is expected to be indicative of the suitability of the tool used. In other words, the tool-selection decisions by the method of the first aspect are based on an evaluation of the time dynamics of a standardized training process which undergoes controlled perturbations corresponding to the tools used. The output of the training process is an ML agent trained to feed commands to the robot controller in such manner that rewards are maximized. The trained ML agent does not constitute the tool-selection decision, nor is the trained ML utilized for making the tool decisions. Rather, the present invention can be described as a way of extracting decisionrelevant information from a meta-observable of a repeatable process in an ML system.

[0010] In different embodiments, the indicator relating to said time evolution can include, for example, one of the following quantities:- a time at which the ML agent’s total reward reached the predefined threshold;- a time at which the ML agent’s average reward (e.g., a moving time average of the rewards) reached the predefined threshold;- a degree of monotonicity of (all or a part of) the monitored time evolution up to the threshold; and- an assessment whether the ML agent’s average reward remains above the threshold (after reaching the threshold).

[0011] In a second aspect of the present disclosure, there is provided a computer system configured to select a tool from a collection of tools to be used by at least one industrial robot for performing a predefined robot task in a working environment.The computer system comprises: an ML agent configured to control said at least one industrial robot; a simulator configured to provide a computer simulation of the working environment for training the ML agent to control said at least one industrial robot while the industrial robot is equipped with a tool in the collection; an interpreter configured to reward the ML agent for completing the robot task or a portion thereof in the computer simulation; and a controller. The controller is configured to perform the following actions for each tool in the collection: initialize the ML agent, initialize the simulator, start training the ML agent in the computer simulation, monitor a time evolution of the ML agent’s reward during said training at least until the reward reaches a predefined threshold, and record an indicator relating to said time evolution. The controller is further configured to select one of the tools based on the recorded indicators.

[0012] A computer system with these technical features generally shares the effects and advantages of the method according to the first aspect discussed above, and the computer system can be implemented with a corresponding degree of technical variation.

[0013] The present disclosure further relates to a computer program containing instructions for causing a computer system to carry out the above method. The computer program may be stored or distributed on a data carrier. As used herein, a “data carrier” may be a transitory data carrier, such as modulated electromagnetic or optical waves, or a non- transitory data carrier. Non-transitory data carriers include volatile and non-volatile memories, such as permanent and non-permanent storage media of magnetic, optical or solid-state type. Still within the scope of “data carrier”, such memories may be fixedly mounted or portable.

[0014] In the terminology of the present disclosure, an “industrial robot” is used in a broad sense, to cover in particular collaborative robots, hygiene robots, industrial robot tracks, industrial robot positioners.

[0015] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to “a / an / the element, apparatus, component, means, step, etc.” are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The stepsof any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Aspects and embodiments are now described, by way of example, with reference to the accompanying drawings, on which: figure 1 shows an industrial robot configured for a pick-and-place application; figure 2 is a flowchart of a method of selecting a tool from a collection of tools to be used by at least one industrial robot for performing a predefined robot task in a working environment; figure 3 shows a computer system configured to select a tool from a collection of tools to be used by at least one industrial robot for performing a predefined robot task in a working environment; and figure 4 shows how an ML agent’s reward evolves as a function of time (in number of backward passes) during training in three example cases.DETAILED DESCRIPTION

[0017] The aspects of the present disclosure will now be described more fully hereinafter with reference to the accompanying drawings, on which certain embodiments of the invention are shown. These aspects may, however, be embodied in many different forms and should not be construed as limiting; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and to fully convey the scope of all aspects of the invention to those skilled in the art. Like numbers refer to like elements throughout the description.

[0018] Figure 1 is a simplified block diagram generally showing a robot arm (or robot manipulator) no and a robot controller 120 communicatively coupled to the robot arm 110. The robot arm 110 and robot controller 120 can be said to constitute an industrial robot. The robot arm no is suitable for handling workpieces 130, in particular, for moving the workpieces 130 into or out of a container 132, both being located in a working environment 150 of the robot arm 110. Figure 1 further shows a camera 140 suspended above the work area which can be used, for example, for the purpose of estimating a position or orientation of a workpiece 130.

[0019] The robot arm no extends from a base 111 and carries a tool 118 at its distal end, shown as an end-effector tool in the example of figure 1. The tool 118 may for example be a gripper tool (as in figure 1), a suction cup, a fork, a magnet or a similar tool suitable for moving workpieces 130. In further examples, the tool 118 may be configured for processing the workpieces 130, such as by painting, coating, cleaning, cutting, honing, heating, refrigerating, welding or soldering. A reference point (not shown) on the tool 118 defines the tool center point (TCP) of the robot arm no. The robot arm no has a plurality of linear or rotary joints 116, each joint being equipped with at least one actuator 114 (e.g., brake, motor). The robot arm 110 is further provided with sensors 112, such as a position sensor (e.g., encoder, angular encoder, resolver), mechanical sensor (e.g., strain sensor, torque sensor), current sensor, thermometer, humidity sensor. The robot controller 120 is configured to control the actuators 114 in the robot arm 110, over a wireless interface (e.g., cellular, noncellular) or via a communication line 113 and associated wired interface 122. The robot controller 120 is further configured to sense a condition of the robot arm no based on signals from the sensors 112.

[0020] Figure 1 further depicts the inner workings of the robot controller 120 which, from a functional perspective, comprises an operator interface 122, the above- mentioned communication interface 128, processing circuitry 124 and a memory 126. The operator interface 122 may for example run the applicant’s software RobotStudio™ for facilitating interaction with a human operator. Further, the operator interface 122 maybe configured to accept input of machine-readable instructions or data, such as binary executable code (one or more robot programs), scripts (sequences of robot commands, generally in human-readable form), streams of robot commands, and the like. The operator interface 122 may in particular be configured to accept a stream of robot commands from a machine-learning (ML) agent.

[0021] The memory 126 in the robot controller 120 is suitable for storing data, such as executable code (one or more robot programs) 129, a model of the industrial robot, an operating system, a system configuration, a history of past tasks (for traceability, documentation and similar purposes), project-related data and the like. The interfaces 122, 128, processing circuitry 124 and memory 126 are interconnected, e.g., by a data bus, ethernet or the like. The robot controller 120 can be implemented locally or in a distributed fashion, including one or more remote or networked(‘cloud’) resources. The robot controller 120 maybe configured to generate (e.g., by means of power electronics) drive signals suitable for powering the actuators 114 as well as control signals. Alternatively, the drive and control signals can be unified into drive currents to be applied directly to the actuators 114. As a further alternative, the robot arm no is equipped with an independent power source, so that all it needs from the robot controller 120 in order to operate are information-carrying control signals that contain sensibly less electric power than is needed to power the robot arm no.

[0022] There will now be described a method 200 of selecting a tool 118 from a collection of tools to be used by at least one industrial robot for performing a predefined robot task in a working environment 150. As mentioned above, the tool may be connected to the industrial robot while the robot task is performed (e.g., like an end-effector tool) but could alternatively be a free tool which is temporarily gripped by the industrial tool to act on a workpiece during a portion of the robot task, as well as a mold, anvil, bracket, block or another type of fixed or semifixed tool which assists in shaping, cutting or stabilizing the workpiece, like the container 132 in figure 1 does. It is further noted that the definition of the robot task, which is normally unchanged over the execution of the method 200, may include the shape, size, surface texture (friction coefficient, ruggedness) of the workpiece to be handled.

[0023] A flowchart in figure 2 depicts an example sequence of the steps to be described below, as well as their logical (causal, conditional) relations.

[0024] It is envisioned that the method 200 may be executed by the robot controller 120. The method 200 could as well be executed by a processor that is independent of or spatially separate from the robot controller 120.

[0025] To illustrate, figure 3 shows a computer system 300 with a structure that makes it suitable for performing the method 200. In particular, the computer system 300 includes an ML agent 310 configured to control said at least one industrial robot no, 120; a simulator 320 configured to provide a computer simulation 322 of the working environment 150; an interpreter 330 configured to reward the ML agent 310 for completing the robot task or a portion thereof in the computer simulation; and a controller 340.

[0026] The ML agent 310 includes at least a memory 312 for storing a data structure representing the ML agent 310 at its current stage of training. If the ML agent has the format of an artificial neural network, the data structure may include values of configurable weights, the values of which determine a mathematical relation between inputs and outputs of the network. The ML agent 310 determines suitable actions A on the basis of a state S of the simulated robot system (i.e., the at least one industrial robot no, 120 while positioned in the working environment 150 and using a specific tool 118). The actions A may correspond to robot commands directed to the robot controller 120, which are fed to a virtual robot controller in the simulation 322.

[0027] The interpreter 330 is configured to determine, based on past and present observations of the simulation 322, the state S of the simulated robot system and to assign a reward R. The reward R may be calculated on the basis of a preconfigured reward function reflecting the success of the action A, or reflecting a behavior of the industrial robot no, 120 which is perceived as desirable, productive, meaningful etc. by a system owner. In particular, the reward function shall encourage (partial) outcomes which approach the (partial) completion of the predefined robot task. The reward function is preconfigured from the point of view of the training process, i.e., it normally cannot be changed by the ML agent 310 or the simulator 320, and it is normally to be kept unchanged while the different tools in the collection are evaluated so as not to introduce bias into the decision-making.

[0028] The simulator 320 is configured to simulate how the ML agent controls the industrial robot - including the robot arm no and robot controller 120 - in the working environment 150. The simulator 320 is therefore useful for training the ML agent 310 to control said at least one industrial robot under representative conditions. In accordance with the teachings of the present disclosure, rewards will be assigned to the ML agent 310 and the time evolution of the ML agent’s reward will be monitored for each of multiple test runs, while the industrial robot is equipped with different tools 118.

[0029] The controller 340 includes memory 342 and processing circuitry 344. In broad terms, the controller 340 is configured to control the computer system 300 to execute the tool-selection method 200.

[0030] Returning to figure 2, the method 200 begins with a step 201 where the entity executing the method 200 (e.g. robot controller 120 or external processor)initializes an ML agent configured to control said at least one industrial robot. The ML agent maybe adapted for supervised learning, unsupervised learning and / or reinforcement learning. The ML agent (or ML model) maybe provided in the form of an artificial neural network with configurable weights, the values of which determine a mathematical relation between inputs to the ML agent and related outputs. The initialization may include instantiating the ML agent from a class template. If an existing ML agent is utilized - which may be a more convenient option in second, third and further executions of step 201 - the initialization may include assigning a predefined set of initial values to the weights; this maybe likened to erase the memory of any earlier training of the ML agent 310. The initialization 201 does not necessarily include generating a new artificial neural network.

[0031] The execution flow of the method 200 continues to a step 202 of initializing a repeatable computer simulation 322 of the working environment 150. It is possible to implement the first step 201 and second step 202 independently of each other; the two steps may then be performed in any desired order, or (partially) overlapping in time. The computer simulation 322 shall be repeatable in the sense that it can be re-initialized, for thereby providing equivalent simulated training environments for each of the robot tools in the collection. The simulated training environments shall not differ to such an extent that bias or artefacts are introduced into the comparison of the robot tools. This said, neither a complete identity of the simulated training environments, nor a completely deterministic behavior of the simulator 320 is an essential requirement on implementations of the present invention.

[0032] Next, in a third step 203, training of the ML agent 310 in the computer simulation 322 is started. During the training, the ML agent 310 controls said at least one (simulated) industrial robot no, 120 in the (simulated) working environment 150 while the industrial robot is equipped with a (simulated) first tool in the collection of tools. During the training, further, the ML agent is rewarded for completing the robot task or a portion (e.g., a substep) thereof.

[0033] In different embodiments of the method 200, the ML agent 310 may exert various degrees of control over the industrial robot 110. Some options include: a) The (simulated / virtual) robot controller 120 executes a predefined robot program for all tools in the collection. The robot program has been created byan operator or generated (semi)automatically with the aid of another simulation. The robot program includes relatively higher-level (or result- oriented) instructions that define, inter alia, positions and orientations of workpieces 130 to be processed or handled, whereas the ML agent 310 provides relatively lower-level (or machine-oriented, or machine-level) instructions, such as fine positioning of the tool 118 relative to the workpiece 130, gripping speed, gripping pressure etc. b) The (simulated / virtual) robot controller 120 executes a first predefined robot program for some tool(s) in the collection, a second predefined robot program for some other tool(s) in the collection, etc. The ML agent 310 supplements the programs by providing relatively lower-level instructions, as described under option a). c) The robot controller 120 is operated based on instructions from the ML agent310 only. This is to say, the ML agent 310 generates a robot program for carrying out the predefined robot task in the course of the training.The ML agent 310 can provide this control in the form of robot programming code, movement instructions etc. The above options are applicable to robot controllers 120 with a variety of different control modes, including position-controlled, velocity- controlled and torque-controlled.

[0034] It is understood that an overreaching aim of the training is to improve the definition of the ML agent 310 by one or more gradual modifications (e.g., modifications of the values of weights of an artificial neural network) as measured by the yardstick of the rewards. This is to say, the definition of the ML agent 310 is improved generally in such manner that the rewards received by the ML agent 310 can be expected to improve with each modification. Each improvement of the definition of the ML agent 310 may correspond to a backpropagation (backward pass), as opposed to the mapping from input to output (forward pass).

[0035] As mentioned above, the rewards R to be assigned may be calculated on the basis of a preconfigured reward function, which depends on a present and / or past state S of the robot system, in particular depending on a change in state before and after executing an action A decided by the ML agent 310. The reward may be assigned for each action A (robot command) depending on the advantageous or disadvantage-ous effects the action A leads to in the working environment 150. Alternatively, the reward maybe assigned once the robot task is completed or once a recognizable stage of partial completion of the robot task is attained. Further, the reward function may also include a cost of taking a certain action A and / or a penalty for damage inflicted on the robot arm no, a workpiece 130 or the working environment 150.

[0036] For the purposes of the present disclosure, it is understood that the concept of a reward includes, as special cases, a return (discounted sum of future rewards) and a loss function. While reward or return are frequently used terms in reinforcement learning, loss function may be the preferred terminology in the fields of supervised and unsupervised learning. A loss function may represent a reward for an action weighted by a metric of the likelihood (e.g., commonness, frequency, incidence, prevalence) of the action. In the loss function, the reward may be replaced with a return.

[0037] During the training of the ML agent 310, the time evolution of the ML agent’s rewards is monitored and an indicator relating to said time evolution is recorded (step 204). The monitoring maybe carried out by the controller 340 on the basis of the signal R. The time evolution is monitored at least until a predefined threshold. In some embodiments, the monitoring goes on at least until a total reward reaches a first threshold. In other embodiments, the monitoring goes on at least until an average reward (e.g., a moving time average, an average over a clock-time interval, an average over the N latest actions decided by the ML agent 310 (A > 2)) reaches a second threshold. The second threshold may correspond to a maximum achievable average reward, or a percentage thereof, such as 95%, 90% or 80%. This is to say, an adequately trained ML agent which is ready for productive use (in particular, commercial use) with the robot application at hand is generally expected to receive the maximum achievable average reward. The maximum achievable average reward does not necessarily signify that the ML agent’s decided actions provide a perfect execution of the robot task; rather it means that the ML agent takes the best possible decision (given its state of knowledge and / or disregarding the influence of stochastic factors) at each point in time. The maximum achievable average reward can be configured by an operator in view of these and similar considerations.

[0038] Figure 4 shows three example time evolution curves 402, 404, 406 of an average reward assigned to the ML agent 310. The horizontal axis corresponds totime as measured, for example, as the number of backward passes, training episodes, training epochs, training steps etc. Alternatively, for a constant backward-pass frequency, time could be measured in seconds or other physical units. In other words, the time evolution monitored in step 204 can be described in other words as an evolution with respect to the number of backward passes. According to one of the realizations of the present inventors, the tool selection of the present method 200 is to be based on the relative rapidity or the relative steadiness of the time evolution for the respective tools in the collection. The vertical axis in figure 4 corresponds to the average reward normalized by the maximum achievable average reward Rmaxthe normalization implies that data points RltR2, R3, ... are plotted asin figure 4.

[0039] Step 204 includes, as mentioned, recording the indicator relating to said time evolution. The indicator may include one or more of the following:1. a time (convergence time) at which the ML agent’s total reward reached a predefined threshold,2. a time (convergence time) at which the ML agent’s average reward reached a predefined threshold,3. a degree of monotonicity of the monitored time evolution of the the ML agent’s(average) reward,4. an assessment whether the ML agent’s average reward remains above a predefined threshold.The suitability of the tool is reflected by each of the proposed items. Indeed, according to the inventors’ realization, error-free operation of the industrial robot no, 120 will be easier to achieve when a well-suited tool 118 is used, and so the necessary training time will be shorter. Further, a simpler training process will lead to a stable time evolution of the reward. The use of a well-suited tool will furthermore make the training process less stochastic, so that breakthroughs (increases in average reward) are more rarely neutralized by setbacks. Due to similar mechanisms, the reward for a robot no, 120 with a tool that is well adapted for the robot task will generally stay above the threshold stably once reached.

[0040] With respect to items i and 2, the convergence times are normally counted from the start of the training, e.g., as a number of backward passes.

[0041] With respect to item 3, the monotonicity could be graded as the percentage of time (percentage of the total number of backward passes) that the reward changes non-decreasingly in a monitored period. The monitored period may correspond to the entire time evolution from the start to up to the threshold, but it could also correspond to a portion of that time evolution. An ideally monotonic function is nondecreasing 100% of the time, that is, in the present application throughout the time evolution up to the threshold. To illustrate, looking at the interval [200, 1200] in figure 4, the curve 404 is decreasing at least in the subintervals [200, 300], [500, 530], [720, 780], [840, 870] and nondecreasing elsewhere; hence, it is nondecreasing about 79% of the time. The curve 406 is decreasing in the subintervals [210, 250], [400, 500], [600, 660], [770, 810], [1000, 1120] and nondecreasing elsewhere, hence about 64% of the time.

[0042] With respect to item 4, the assessment could be expressed - similar to the proposal for item 3 - as a percentage of time for which the reward stays above a predefined threshold after reaching the threshold for the first time. The curve 404 in figure 4 is considered as an example, and the threshold is assumed to be 0.8 (i.e., 80% of Rmax)- The curve 404 reaches this threshold for the first time at t = 950. After this point, in the interval [950, 1800], the curve 404 is above 0.8 except in the subintervals [1300, 1330], [1440, 1480] and [1750, 1800]. Hence, the percentage is 86%. The curve 406 is above 0.8 always except in [1080, 1160], corresponding to percentage of about 90% if the interval starting at the first crossing (t = 860) and extending up to t = 1700 is considered.

[0043] If item 3 is used on its own, because the monitored period may correspond to merely a portion of the time evolution from the start to up to the threshold, the threshold can be set arbitrarily. In particular, the threshold need not be related to the achievable average reward Rmax-

[0044] The method 200 then continues, in step 20 , by repeating the preceding steps 201-204 for each remaining tool 118 in the collection of tools, i.e., the second tool, third tool and so forth. When there are no more tools in the collection (N branch from box 205), it is expected that one indicator of the time evolution of the ML agent’s 310 reward has been recorded for each tool in the collection.

[0045] Next, in step 206, one of the tools is selected based on the recorded indicators. Preferably, the tool is selected by the device which executes the method 200, such as a robot controller 120 or another processor. Different selection rules may be applied depending on the content of the indicators. Some examples include those in Table 1.Continuing the running example discussed above, the selection rule of item 3 implies that curve 404 will be preferred over curve 406. Further, the selection rule of item 4 implies that curve 406 will be preferred over curve 404.

[0046] Alternatively, the tool is selected in step 206 based on a combination of the recorded indicators and input from an operator. For example, the operator interface 122 maybe used to display the recorded indicators to the operator and to receive the operator’s selection of a tool in reaction to the recorded indicators. To ensure the operator actually considers the guidance offered by the recorded indicators, the operator interface 122 may optionally be configured to request a confirmation from the operator if his or her selection deviates from what one of the above selection rules would provide.

[0047] Insome embodiments of the method 200, the indicator includes two or more of the items 1-4. Then, it is possible to define hierarchic combinations of selection rules, for example:- The indicator includes the time to reach a predefined threshold (item 1 or 2) and further includes the degree of monotonicity (item 3). Then, a combined selection rule to the effect that the tool with the shortest time shall be selected provided the time evolution is substantially monotonic, e.g., it is nondecreasing for at least 90% of the time. If the proviso is not fulfilled, the tool with the second shortest time shall be selected, and so forth.- The indicator includes the time to reach a predefined threshold (item 1 or 2) and further includes the assessment whether the average reward remains above the threshold (item 4). A combined selection rule based on these inputs maybe that the tool with the shortest time shall be selected provided the reward remains above the threshold substantially always, e.g., for at least an absolute percentage of the time, such as 80% or 90% of the time. If this is not fulfilled, then the tool with the second shortest time shall be selected, and so forth.Those skilled in the art will be able to propose further combined selection rules after studying the present disclosure.

[0048] In a further variation of the tool-selection method 200, the industrial robot no, 120 maybe reconfigured or reequipped in different ways depending on which tool 118 it carries. For example, the industrial robot no, 120 may undergo adaptations making it more suitable for the respective tools in the collection, possibly a dedicated adaptation for each tool. According to this variation, if for example certain tools are known to perform better when carried by a specific type of robot arm 110 and / or when the robot arm 110 is supported by a specific type of base 111 and / or when a specific conveyor or robot track is used, it is permissive to define the simulated robot arm 110 accordingly in the different training processes. The method 200 will then, conceptually speaking, provide a selection of a tool together with the adaptations associated with the tool. To ensure a fair comparison, however, it is understood that the simulated environments should remain equivalent or nearequivalent in all other respects; for example, the simulation of the working environment 150 in which the robot arm no is located should preferably repeat identically in each training process.

[0049] The aspects of the present disclosure have mainly been described above with reference to a few embodiments. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the invention, as defined by the appended patent claims.

Claims

CLAIMS1. A computer-implemented method (200) of selecting a tool (118) from a collection of tools to be used by at least one industrial robot (no, 120) for performing a predefined robot task in a working environment (150), the method comprising: initializing (201) a machine-learning, ML, agent configured to control said at least one industrial robot; initializing (202) a repeatable computer simulation of the working environment; starting (203) training of the ML agent in the computer simulation, wherein the ML agent controls said at least one industrial robot in the working environment while the industrial robot is equipped with a first tool in the collection, and wherein the ML agent is rewarded for completing the robot task or a portion thereof; monitoring (204), during said training, a time evolution of the ML agent’s reward up to a predefined threshold, and recording an indicator relating to said time evolution; repeating (205) the preceding steps (201-204) for each remaining tool in the collection; and selecting (206) one of the tools based on the recorded indicators.

2. The method (200) of claim 1, wherein the indicator includes a time at which the ML agent’s total reward reached the threshold, the method further comprising: identifying (206.1) the shortest recorded time at which the ML agent’s total reward reached the threshold, wherein the tool with the shortest recorded time is selected (206).

3. The method (200) of claim 1, wherein the indicator includes a time at which the ML agent’s average reward reached the threshold, the method further comprising: identifying (206.1) the shortest recorded time at which the ML agent’s average reward reached the threshold, wherein the tool with the shortest recorded time is selected (206).

4. The method (200) of claim 3, wherein the ML agent’s average reward corresponds to a moving time average.

5. The method (200) of claim 4, wherein the threshold corresponds to a maximum achievable average reward.

6. The method (200) of any of the preceding claims, wherein: the indicator includes a degree of monotonicity of the monitored time evolution up to the threshold or a degree of monotonicity of a portion thereof; and the tool with the relatively most monotonous time evolution up to the threshold is selected (206).

7. The method (200) of any of the preceding claims, wherein: the indicator includes a degree of monotonicity of the monitored time evolution up to the threshold; and the tool is selected (206) subject to a monotonicity requirement on the monitored time evolution up to the threshold.

8. The method (200) of any of the preceding claims, wherein: the indicator includes an assessment whether the ML agent’s average reward remains above the threshold; and the tool is selected (206) based on whether the ML agent’s average reward remains above the threshold.

9. The method (200) of claim 8, wherein the tool is selected (206) subject to a requirement that the ML agent’s average reward remain above the threshold.

10. The method (200) of any of the preceding claims, wherein each of the available tools is adapted to pick and place workpieces.

11. The method (200) of any of the preceding claims, wherein the ML agent is adapted for one or more of: supervised learning, unsupervised learning, reinforcement learning.

12. A computer system (300) configured to select a tool (118) from a collection of tools to be used by at least one industrial robot (no, 120) for performing a predefined robot task in a working environment, the computer system comprising: a machine-learning, ML, agent (310) configured to control said at least one industrial robot;a simulator (320) configured to provide a computer simulation (322) of the working environment for training the ML agent to control said at least one industrial robot while the industrial robot is equipped with a tool in the collection; an interpreter (330) configured to reward the ML agent for completing the robot task or a portion thereof in the computer simulation; and a controller (340) configured to, for each tool in the collection,- initialize the ML agent,- initialize the simulator,- start training the ML agent in the computer simulation, and- monitor during said training at time evolution of the ML agent’s reward up to a predefined threshold, and record an indicator relating to said time evolution, and further configured to select one of the tools based on the recorded indicators.

13. A computer program (349) comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method (200) of any of claims 1 to 11.

Images