Methods and computer systems for selecting robotic tools

By initializing a machine learning agent and training it in a computer simulation, and monitoring the evolution of reward time, the problems of automation and determinism in industrial robot tool selection are solved, thereby improving production efficiency and workpiece quality.

CN123125082APending Publication Date: 2026-11-17ABB (SCHWEIZ) AG
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202480090929.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-04-17
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

When selecting suitable end effector tools for industrial robots, existing technologies struggle to achieve automated, deterministic, and repeatable tool selection, especially when task characteristics are highly variable.

Method used

By initializing a machine learning (ML) agent, training it using repeatable computer simulations, monitoring the evolution of reward time and recording relevant indicators, and selecting tools based on these indicators.

Benefits of technology

It enables deterministic and repeatable tool selection, improves production efficiency, reduces workpiece damage and system downtime, and ensures workpiece quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123125082A_ABST
    Figure CN123125082A_ABST
Patent Text Reader

Abstract

A computer-implemented method (200) for selecting a tool from a set of tools used by at least one industrial robot to perform a predefined robotic task in a work environment, the method comprising: initializing (201) a machine learning (ML) agent configured to control the industrial robot; initializing (202) a repeatable computer simulation of the work environment; initiating (203) training (the ML agent) in the computer simulation, wherein the ML agent controls the industrial robot in the work environment while the industrial robot is equipped with a first tool from the set, and wherein the ML agent is rewarded for completing the robotic task or a portion thereof; during the training, monitoring (204) the temporal evolution of the ML agent's reward up to a predefined threshold and recording indicators associated with the temporal evolution; repeating (205) the foregoing steps for each remaining tool in the set; and selecting (206) one tool from the tools based on the recorded indicators.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure addresses problems in the configuration and operation of industrial robots by proposing an automated method for selecting tools for at least one industrial robot. The tools are selected from a set of tools used by the industrial robot when performing predefined robotic tasks in its working environment. Background Technology

[0002] The selection of appropriate end effector tools (such as grippers for pick-and-place applications) for industrial robots performing robotic tasks has significant economic implications. A well-fitting tool can differ from its inferior alternative in several ways: productivity (e.g., number of workpieces processed per unit time), the rate of damaged or dropped workpieces, the percentage of system downtime, and the perceived quality of processed workpieces by the end user. The selection problem can sometimes be difficult to solve, especially when a wide range of potentially useful tools are available, or when the task consists of a long list of subtasks with highly variable characteristics. In such cases, the operator must either undertake a tedious evaluation of each tool or apply a degree of intuition. Not all operators have accumulated enough experience to make intuitive decisions about robotic tools. Even experienced operators may need to evaluate their initial tool selection using simulations or experiments and reconsider their choices as needed.

[0003] In existing technologies, several attempts have been made to automate the selection of robotic tools, not only to save manpower, but also to achieve a reasonable level of certainty that the selected tools are objectively optimal or close to optimal.

[0004] Robot tool selection can be delegated, for example, to a machine learning (ML) model that has been previously trained to output tool selections based on images or other descriptions of the workpiece to be picked up and placed. Training can, for example, include assigning rewards to the ML model based on the suitability of the selected tool and iteratively evolving the ML model to maximize future rewards.

[0005] A more refined method for selecting automated robotic tools is known from patent application US20230081119A1. This disclosed method first selects a gripper, then identifies the end effector corresponding to the selected gripper. The selection step is performed as a Markov Decision Process (MDP) with a reward function that balances pickup success with the cost of altering the end effector. This selection step specifically includes feeding an image of the object to be picked up into a "grasping prediction model," which can be provided as a trained ML model. The method according to US20230081119A1 can be described as an ML-assisted direct gripping selection process followed by a post-processing step of identifying the corresponding end effector using conventional methods. Summary of the Invention

[0006] One object of this disclosure is to provide a method for selecting tools from a set of tools used by at least one industrial robot to perform predefined robotic tasks in a working environment. A specific object is to propose a tool selection method that is executed deterministically and generates tool selection decisions based on objective suitability criteria. A specific object is to propose a repeatable tool selection method. Another object is to provide a computer system having these characteristics.

[0007] At least some of these objectives are achieved by means of the invention as defined by the independent claims. The dependent claims relate to advantageous embodiments.

[0008] In a first aspect of this disclosure, a computer-implemented method is provided for selecting tools from a set of tools used by at least one industrial robot to perform predefined robotic tasks in a work environment. The method includes: initializing a machine learning (ML) agent configured to control at least one industrial robot; initializing a repeatable computer simulation of the work environment; initiating training of the ML agent in the computer simulation, wherein the ML agent controls at least one industrial robot in the work environment while the industrial robot is equipped with a first tool from the set, and wherein the ML agent receives a reward for completing a robotic task or a portion thereof; during said training, monitoring the temporal evolution of the ML agent's reward up to a predefined threshold and recording indicators associated with said temporal evolution; repeating the foregoing steps for each remaining tool in the set; and selecting one of these tools based on the recorded indicators.

[0009] The inventors have recognized that, for a given combination of robot, environment, and workpiece, an initialized ML agent will learn to control the robot within a finite time (convergence time) to almost always maximize its reward. Building on this, the inventors propose repeating the learning process each time the end effector tool is changed and the ML agent is reinitialized. Since all other conditions remain constant, the temporal evolution of a particular learning process is expected to indicate the suitability of the tool used. In other words, the tool selection decision made through the first aspect is based on an evaluation of the temporal dynamics of a standardized training process that undergoes controlled perturbations corresponding to the tool used. The output of the training process is the ML agent, trained to feed commands to the robot controller in a reward-maximizing manner. The trained ML agent neither constitutes nor is used to make tool selection decisions. Instead, the invention can be described as a way to extract decision-related information from the meta-observations of repeatable processes in an ML system.

[0010] In different embodiments, the indicator associated with the time evolution may include, for example, one of the following: - The time it takes for the total reward of the ML agent to reach a predefined threshold; - The time it takes for the average reward of the ML agent (e.g., the average time of movement of the reward) to reach a predefined threshold; - The degree of monotonicity of the monitored temporal evolution up to (all or part of) the threshold; and - An assessment of whether the average reward of the ML agent (after reaching the threshold) remains above the threshold.

[0011] In a second aspect of this disclosure, a computer system is provided configured to select tools from a set of tools used by at least one industrial robot to perform predefined robotic tasks in a working environment. The computer system includes: an ML agent configured to control the at least one industrial robot; a simulator configured to provide a computer simulation of the working environment for training the ML agent to control the at least one industrial robot when the industrial robot is equipped with tools from the set; an interpreter configured to reward the ML agent for completing a robotic task or a portion thereof in the computer simulation; and a controller. The controller is configured to perform the following actions for each tool in the set: initializing the ML agent; initializing the simulator; initiating training of the ML agent in the computer simulation; monitoring the temporal evolution of the ML agent's reward during the training period at least until the reward reaches a predefined threshold; and recording indicators associated with the temporal evolution. The controller is also configured to select one tool from the tools based on the recorded indicators.

[0012] Computer systems with these technical features typically share the effects and advantages of the methods discussed above according to the first aspect, and such computer systems can be implemented with a corresponding degree of technical variation.

[0013] This disclosure also relates to a computer program containing instructions for causing a computer system to perform the methods described above. The computer program may be stored on or distributed on a data carrier. As used herein, "data carrier" can be a transient data carrier, such as modulated electromagnetic waves or light waves, or a non-transient data carrier. Non-transient data carriers include volatile and non-volatile memories, such as permanent and non-permanent storage media of magnetic, optical, or solid-state types. Still within the scope of "data carrier," such memories can be fixedly mounted or portable.

[0014] In the terminology of this disclosure, "industrial robot" is used in a broad sense to specifically cover collaborative robots, hygienic robots, industrial robot tracks, and industrial robot positioners.

[0015] Generally, all terms used in the claims should be interpreted according to their ordinary meaning in the art, unless otherwise expressly defined herein. All references to “a”, “an”, “the” element, device, component, part, step, etc. should be openly interpreted as referring to at least one instance of said element, device, component, part, step, etc., unless otherwise expressly stated. The steps of any method disclosed herein need not be performed in the exact order disclosed unless expressly stated. Attached Figure Description

[0016] Aspects and embodiments will now be described by way of example with reference to the accompanying drawings, in which: Figure 1 An industrial robot configured for pick-and-place applications is shown; Figure 2 This is a flowchart of a method for selecting tools from a set of tools for use by at least one industrial robot to perform predefined robotic tasks in a working environment; Figure 3 A computer system is shown configured to select from a set of tools for use by at least one industrial robot to perform predefined robotic tasks in a working environment; and Figure 4 The diagram illustrates the evolution of the ML agent's reward over time (in terms of the number of backpasses) during training in three example cases. Detailed Implementation

[0017] Various aspects of this disclosure will now be described more fully below with reference to the accompanying drawings, which illustrate certain embodiments of the invention. However, these aspects may be embodied in many different forms and should not be construed as limiting; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of all aspects of the invention to those skilled in the art. Throughout this specification, the same reference numerals refer to the same elements.

[0018] Figure 1 This is a simplified block diagram, roughly showing a robot arm (or robot manipulator) 110 and a robot controller 120 communicatively coupled to the robot arm 110. The robot arm 110 and the robot controller 120 can be said to constitute an industrial robot. The robot arm 110 is suitable for handling workpieces 130, particularly for moving workpieces 130 into or out of a container 132, both of which are located within the working environment 150 of the robot arm 110. Figure 1 A camera 140 suspended above the work area is also shown, which can be used for purposes such as estimating the position or orientation of the workpiece 130.

[0019] Robotic arm 110 extends from base 111 and carries tool 118 at its distal end, which... Figure 1 The example shown is an end effector tool. Tool 118 could be, for example, a clamping tool (such as...). Figure 1 The tool 118 may be a suction cup, fork, magnet, or similar tool suitable for moving the workpiece 130. In other examples, the tool 118 may be configured to handle the workpiece 130, such as by spraying, coating, cleaning, cutting, honing, heating, cooling, welding, or brazing. A reference point (not shown) on the tool 118 defines the tool center point (TCP) of the robot arm 110. The robot arm 110 has multiple linear or rotary joints 116, each equipped with at least one actuator 114 (e.g., brake, motor). The robot arm 110 is also provided with sensors 112, such as position sensors (e.g., encoders, angle encoders, resolvers), mechanical sensors (e.g., strain sensors, torque sensors), current sensors, thermometers, and humidity sensors. The robot controller 120 is configured to control the actuators 114 in the robot arm 110 via a wireless interface (e.g., cellular, non-cellular) or via communication line 113 and associated wired interface 122. The robot controller 120 is also configured to sense the condition of the robot arm 110 based on signals from the sensors 112.

[0020] Figure 1The internal workings of the robot controller 120 are also depicted. Functionally, the robot controller includes an operator interface 122, the aforementioned communication interface 128, processing circuitry 124, and memory 126. The operator interface 122 can, for example, run the applicant's software RobotStudio™ to facilitate interaction with a human operator. Furthermore, the operator interface 122 can be configured to accept machine-readable instructions or data input, such as binary executable code (one or more robot programs), scripts (sequences of robot commands, typically in human-readable form), robot command streams, etc. In particular, the operator interface 122 can be configured to accept robot command streams from machine learning (ML) agents.

[0021] The memory 126 in the robot controller 120 is suitable for storing data such as executable code (one or more robot programs) 129, the model of the industrial robot, operating system, system configuration, history of past tasks (for traceability, documentation, and similar purposes), project-related data, etc. Interfaces 122, 128, processing circuitry 124, and memory 126 are interconnected, for example, via a data bus, Ethernet, etc. The robot controller 120 can be implemented locally or in a distributed manner, including one or more remote or networked (“cloud”) resources. The robot controller 120 can be configured to generate (e.g., via power electronics) drive signals and control signals suitable for powering the actuator 114. Alternatively, the drive and control signals can be unified into a drive current directly applied to the actuator 114. As another alternative, the robot arm 110 is equipped with an independent power supply, so that it only needs to obtain an information-carrying control signal from the robot controller 120 for operation, which contains significantly less electrical power than is required to power the robot arm 110.

[0022] A method 200 for selecting tools 118 from a set of tools used by at least one industrial robot to perform a predefined robotic task in a work environment 150 will now be described. As mentioned above, the tools can be attached to the industrial robot while performing the robotic task (e.g., like an end effector tool), but alternatively can be free tools that are temporarily gripped by the industrial tool to act on a workpiece during a portion of the robotic task, as well as another type of fixed or semi-fixed tool, such as a mold, anvil, support, block, or tool that assists in forming, cutting, or stabilizing a workpiece. Figure 1 As container 132 in the example does. It should also be noted that the definition of the robot task (which typically remains unchanged during the execution of method 200) may include the shape, size, and surface texture (coefficient of friction, roughness) of the workpiece to be processed.

[0023] Figure 2The flowchart in the diagram illustrates an example sequence of the steps that will be described below, as well as their logical (causal, conditional) relationships.

[0024] It is conceivable that method 200 can be executed by robot controller 120. Method 200 can also be executed by a processor independent of robot controller 120 or spatially separate from robot controller 120.

[0025] To illustrate, Figure 3 A computer system 300 is shown, having a structure that makes it suitable for performing method 200. Specifically, the computer system 300 includes an ML agent 310 configured to control the at least one industrial robot 110, 120; a simulator 320 configured to provide a computer simulation 322 of a working environment 150; an interpreter 330 configured to reward the ML agent 310 for completing a robot task or a portion thereof in the computer simulation; and a controller 340.

[0026] The ML agent 310 includes at least a memory 312 for storing data structures representing the ML agent 310 in its current training phase. If the ML agent has an artificial neural network format, the data structure may include values ​​of configurable weights that determine the mathematical relationship between the network's inputs and outputs. The ML agent 310 determines appropriate actions A based on the state S of the simulated robot system (i.e., the state of at least one industrial robot 110, 120 when it is located in the working environment 150 and using a specific tool 118). Action A may correspond to robot commands directed to the robot controller 120, which are fed to the virtual robot controller in the simulation 322.

[0027] Interpreter 330 is configured to determine the state S of the simulated robot system and assign a reward R based on past and current observations of simulation 322. The reward R can be calculated based on a pre-configured reward function that reflects the success of action A, or reflects the behavior of industrial robots 110, 120 as desired, productive, meaningful, etc., by the system owner. Specifically, the reward function should encourage results that approximate (partial) completion of a predefined robot task. The reward function is pre-configured from the perspective of the training process; that is, it generally cannot be changed by the ML agent 310 or simulator 320, and is generally invariant when evaluating different tools in the ensemble so as not to introduce decision bias.

[0028] Simulator 320 is configured to simulate how an ML agent controls an industrial robot—including a robotic arm 110 and a robot controller 120—in a working environment 150. Therefore, simulator 320 can be used to train ML agent 310 to control the at least one industrial robot under representative conditions. In accordance with the teachings of this disclosure, rewards are assigned to ML agent 310, and the temporal evolution of the ML agent's rewards is monitored for each of multiple test runs, while the industrial robot is equipped with different tools 118.

[0029] The controller 340 includes a memory 342 and a processing circuitry 344. In general, the controller 340 is configured to control the computer system 300 to execute the tool selection method 200.

[0030] return Figure 2 Method 200 begins Step 201 The entity executing method 200 (e.g., robot controller 120 or external processor) is initialized to control an ML agent of the at least one industrial robot. The ML agent can be adapted for supervised learning, unsupervised learning, and / or reinforcement learning. The ML agent (or ML model) can be provided in the form of an artificial neural network with configurable weights that determine the mathematical relationship between the ML agent's inputs and associated outputs. Initialization may include instantiating the ML agent from a class template. If an existing ML agent is used—which may be a more convenient option in the second, third, and subsequent executions of step 201—initialization may include assigning a predefined set of initial values ​​to the weights; this can be likened to erasing any previously trained memories of the ML agent 310. Initialization 201 does not necessarily involve generating a new artificial neural network.

[0031] The execution flow of method 200 continues to the initialization of the working environment 150 and the repeatable computer simulation 322. step 202 The first step 201 and the second step 202 can be implemented independently of each other; then these two steps can be performed in any desired order, or (partially) overlap in time. The computer simulation 322 should be repeatable, i.e., it can be reinitialized to provide an equivalent simulation training environment for each robot tool in the ensemble. Differences in the simulation training environments should not reach the point of introducing bias or artifacts into the comparison of the robot tools. That is, complete identity of the simulation training environment and completely deterministic behavior of the simulator 320 are not fundamental requirements for the implementation of this invention.

[0032] Next, in the third Step 203In the training process, the ML agent 310 is trained in a computer simulation 322. During training, the ML agent 310 controls at least one (simulated) industrial robot 110, 120 in a (simulated) working environment 150, while the industrial robot is equipped with a (simulated) first tool from a set of tools. Furthermore, during training, the ML agent receives rewards for completing robot tasks or parts thereof (e.g., sub-steps).

[0033] In different embodiments of method 200, the ML agent 310 can exert various degrees of control over the industrial robot 110. Some options include: a) The (simulated / virtual) robot controller 120 executes a predefined robot program for all tools in the set. This robot program has been created by the operator or generated automatically using another simulation. The robot program includes relatively high-level (or result-oriented) instructions that define, in particular, the position and orientation of the workpiece 130 to be processed or transported, while the ML agent 310 provides relatively low-level (or machine-oriented, or machine-level) instructions, such as the fine positioning of the tool 118 relative to the workpiece 130, gripping speed, gripping pressure, etc. b) The (simulated / virtual) robot controller 120 executes a first predefined robot program for one or more tools in the set, a second predefined robot program for another one or more tools in the set, and so on. The ML agent 310 supplements these programs by providing relatively low-level instructions, as described in option a). c) The robot controller 120 operates solely based on instructions from the ML agent 310. That is, the ML agent 310 generates robot programs during training to perform predefined robot tasks. The ML agent 310 can provide this control in the form of robot programming code, motion commands, etc. The above options are applicable to robot controllers 120 with various control modes, including position control, speed control, and torque control.

[0034] It should be understood that the overall goal of training is to improve the definition of the ML agent 310 through one or more progressive modifications (e.g., modifications to the weight values ​​of the artificial neural network), as measured by a metric of reward. That is, the definition of the ML agent 310 is typically improved in such a way that the reward received by the ML agent 310 can be expected to improve with each modification. Each improvement in the definition of the ML agent 310 can correspond to backpropagation (reverse propagation), the opposite of the mapping from input to output (forward propagation).

[0035] As described above, the reward R to be allocated can be calculated based on a pre-configured reward function that depends on the current and / or past states S of the robot system, particularly on the state changes before and after the execution of action A determined by the ML agent 310. The reward can be allocated for each action A (robot command) based on the beneficial or detrimental effects of action A in the working environment 150. Alternatively, the reward can be allocated once upon completion of the robot task or upon reaching a identifiable partial completion phase of the robot task. Furthermore, the reward function may also include the cost of taking a specific action A and / or penalties for damage to the robot arm 110, workpiece 130, or working environment 150.

[0036] For the purposes of this disclosure, it should be understood that the concept of reward, as a special case, includes both payoff (the sum of discounted future rewards) and loss function. Although award or Return It is a term frequently used in reinforcement learning, but loss function This is likely the preferred term in the fields of supervised and unsupervised learning. A loss function can represent the reward for an action, weighted by measures of action probability (e.g., commonality, frequency, occurrence, prevalence). In a loss function, the reward can be replaced by a payoff.

[0037] During the training of the ML agent 310, the temporal evolution of the ML agent's rewards is monitored and indicators associated with said temporal evolution are recorded. Step 204 Monitoring can be performed by controller 340 based on signal R. Monitoring of time evolution is performed at least until a predefined threshold is reached. In some embodiments, monitoring continues at least until the total reward reaches a first threshold. In other embodiments, monitoring continues at least until the average reward (e.g., average of movement time, average over clock time intervals, or the most recent N rewards determined by the ML agent 310) is reached. The average value of the actions reaches a second threshold. This second threshold can correspond to the maximum achievable average reward or a percentage thereof, such as 95%, 90%, or 80%. That is, a well-trained ML agent, ready for productive use (especially commercial use) in the robotic application at hand, is generally expected to receive the maximum achievable average reward. The maximum achievable average reward does not necessarily mean that the ML agent's decision action perfectly executes the robotic task; rather, it means that the ML agent takes the best possible decision at each time point (given its knowledge state and / or ignores the influence of random factors). The maximum achievable average reward can be configured by the operator with these and similar considerations in mind.

[0038] Figure 4Three example time evolution curves 402, 404, and 406 are shown for the average reward assigned to the ML agent 310. The horizontal axis corresponds to time, measured, for example, in terms of backpropagation count, training rounds, training epochs, training steps, etc. Alternatively, for a constant backpropagation frequency, time can be measured in seconds or other physical units. In other words, the time evolution monitored in step 204 can be described, in other words, as an evolution with respect to the number of backpropagations. According to the inventors' understanding, the tool selection of this method 200 should be based on the relative speed or relative stability of the time evolution of the corresponding tools in the set. Figure 4 The vertical axis in the equation corresponds to the maximum achievable average reward. Normalized average reward; Figure 4 In this context, normalization means that data points Drawn as .

[0039] As described above, step 204 includes recording an indicator related to the time evolution. This indicator may include one or more of the following: 1. The time it takes for the total reward of the ML agent to reach a predefined threshold (convergence time). 2. The time it takes for the average reward of the ML agent to reach a predefined threshold (convergence time). 3. The monotonicity of the temporal evolution of the (average) reward of the monitored ML agent. 4. Evaluation of whether the average reward of the ML agent remains above a predefined threshold. The suitability of the tool is reflected in each item of the proposed project. In fact, according to the inventors, when using a well-suited tool 118, it is easier to achieve error-free operation of industrial robots 110 and 120, and therefore the necessary training time will be shorter. Furthermore, a simpler training process will lead to a stable evolution of rewards over time. Using a well-suited tool will also reduce the randomness of the training process, making breakthroughs (increases in average reward) less likely to be offset by frustration. Due to similar mechanisms, for robots 110 and 120 with well-suited tools for the robotic task, rewards typically remain stably above the threshold after reaching it.

[0040] For Project 1 and Project 2, the convergence time is usually calculated from the start of training, for example, as the number of backpropagations.

[0041] Regarding Project 3, monotonicity can be graded as the percentage of time during which the reward changes non-decreasingly within the monitoring period (the percentage of total backpropagations). The monitoring period can correspond to the entire time evolution from the start until the threshold, or it can correspond to a portion of that time evolution. An ideal monotonic function is non-decreasing for 100% of the time, i.e., in this application, it is non-decreasing throughout the entire time evolution up to the threshold. To illustrate, observe... Figure 4 In the interval [200, 1200], curve 404 decreases at least in the subintervals [200, 300], [500, 530], [720, 780], and [840, 870], and is non-decreasing elsewhere; therefore, it is non-decreasing approximately 79% of the time. Curve 406 decreases in the subintervals [210, 250], [400, 500], [600, 660], [770, 810], and [1000, 1120], and is non-decreasing elsewhere, and is therefore non-decreasing approximately 64% of the time.

[0042] Regarding Project 4, similar to the suggestion in Project 3, the evaluation can be expressed as the percentage of time the reward remains above a predefined threshold after the initial threshold is reached. Figure 4 Taking curve 404 in the example, assuming the threshold is 0.8 (i.e., (80%). Curve 404 first reaches this threshold at t=950. After that point, in the interval [950, 1800], curve 404 is above 0.8, except in the sub-intervals [1300, 1330], [1440, 1480], and [1750, 1800]. Therefore, the percentage is 86%. Curve 406 is always above 0.8, except in [1080, 1160], which corresponds to approximately 90% if we consider the interval starting from the first crossover point (t=860) and extending to t=1700.

[0043] If Project 3 is used alone, the threshold can be set arbitrarily because the monitoring period can correspond to only a portion of the time evolution from the start to the threshold. Specifically, the threshold does not necessarily need to be related to the achievable average reward. Related.

[0044] Then method 200 repeats the aforementioned steps 201 to 204 for each remaining tool 118 in the tool set (i.e., the second tool, the third tool, etc.) while Step 205 Continue in the middle. When there are no more tools in the set (going N branches from box 205), it is expected that an indicator of the temporal evolution of the ML agent 310 reward has been recorded for each tool in the set.

[0045] Next, in Step 206In this process, one tool is selected based on the recorded indicators. Preferably, the tool is selected by the device performing method 200 (such as robot controller 120 or another processor). Different selection rules can be applied depending on the content of the indicators. Some examples include those selection rules in Table 1. Continuing with the running example discussed above, the selection rule for item 3 means that curve 404 will be superior to curve 406. Furthermore, the selection rule for item 4 means that curve 406 will be superior to curve 404.

[0046] Alternatively, in step 206, a tool is selected based on a combination of the recorded indicators and input from the operator. For example, operator interface 122 may be used to display the recorded indicators to the operator and receive the operator's selection of a tool in response to the recorded indicators. To ensure that the operator actually considers the guidance provided by the recorded indicators, operator interface 122 may optionally be configured to request confirmation from the operator if the operator's selection deviates from the result provided by one of the selection rules described above.

[0047] In some embodiments of method 200, the indicator includes two or more of items 1 to 4. A hierarchical combination of selection rules can then be defined, for example: - The indicators include the time to reach a predefined threshold (item 1 or 2) and also the degree of monotonicity (item 3). The selection rule, in essence, is: if the time evolution is essentially monotonic, for example, at least 90% of the time is non-decreasing, then the tool with the shortest time should be selected. If this condition is not met, then the tool with the second shortest time should be selected, and so on. - The indicators include the time it takes to reach a predefined threshold (item 1 or 2), and also an assessment of whether the average reward remains above the threshold (item 4). The selection rule based on this combination of inputs could be: if the reward is substantially always above the threshold, for example, at least an absolute percentage of time (such as 80% or 90%), then the tool with the shortest time should be selected. If this is not met, then the tool with the second shortest time should be selected, and so on. Those skilled in the art will be able to propose other selection rules after studying this disclosure.

[0048] In another variation of tool selection method 200, industrial robots 110, 120 can be reconfigured or refitted in different ways depending on the tool 118 they carry. For example, industrial robots 110, 120 can undergo adaptation to better suit the corresponding tool in the set, possibly a dedicated adaptation for each tool. According to this variation, if, for example, it is known that certain tools perform better when carried by a particular type of robot arm 110 and / or when the robot arm 110 is supported by a particular type of base 111 and / or when using a particular conveyor belt or robot track, then it is permissible to define the simulated robot arm 110 accordingly in different training processes. Method 200 then conceptually provides the selection of the tool and the adaptation associated with that tool. However, to ensure fair comparison, it should be understood that the simulation environment should remain equivalent or nearly equivalent in all other respects; for example, the simulation of the working environment 150 in which the robot arm 110 resides is preferably repeated identically in each training process.

[0049] Various aspects of this disclosure have been described above primarily with reference to several embodiments. However, as will be readily understood by those skilled in the art, other embodiments besides those disclosed above are also possible within the scope of the invention as defined by the appended claims.

Claims

1. A computer-implemented method (200) for selecting tools (118) from a set of tools, said set of tools being used by at least one industrial robot (110, 120) for performing a predefined robotic task in a working environment (150), said method comprising: Initialization (201) is configured to control a machine learning ML agent of the at least one industrial robot; Initialize (202) a repeatable computer simulation of the working environment; Training of the ML agent begins in the computer simulation (203), wherein the ML agent controls the at least one industrial robot in the working environment, the industrial robot is equipped with a first tool from the set, and wherein the ML agent is rewarded for completing the robot's task or a part thereof. During the training, (204) the temporal evolution of the ML agent's reward is monitored up to a predefined threshold, and indicators associated with the temporal evolution are recorded; Repeat steps (201-204) of the foregoing for each remaining tool in the set; and Select one of the tools described in (206) based on the recorded indicator.

2. The method (200) according to claim 1, wherein the indicator includes the time when the total reward of the ML agent reaches the threshold, the method further comprising: The identifier (206.1) represents the shortest recorded time for the total reward of the ML agent to reach the threshold. The tool with the shortest recording time was selected (206).

3. The method (200) according to claim 1, wherein the indicator includes the time when the average reward of the ML agent reaches the threshold, the method further comprising: The identifier (206.1) represents the shortest recorded time for the average reward of the ML agent to reach the threshold. The tool with the shortest recording time was selected (206).

4. The method (200) according to claim 3, wherein the average reward of the ML agent corresponds to the average movement time.

5. The method (200) of claim 4, wherein the threshold corresponds to the maximum achievable average reward.

6. The method (200) according to any one of the preceding claims, wherein: The indicator includes the degree of monotonicity of the monitored time evolution or a portion of the time evolution up to the threshold; and The tool with the most monotonic time evolution up to the threshold was selected (206).

7. The method (200) according to any one of the preceding claims, wherein: The indicator includes the degree of monotonicity of the monitored time evolution up to the threshold; and The tool is selected under the condition that the monitored time evolution up to the threshold satisfies the monotonicity requirement (206).

8. The method (200) according to any one of the preceding claims, wherein: The indicator includes an assessment of whether the average reward of the ML agent remains above the threshold; and The tool is selected based on whether the average reward of the ML agent remains above the threshold (206).

9. The method of claim 8 (200), wherein the tool is selected (206) under the condition that the average reward of the ML agent remains above the threshold.

10. The method (200) according to any one of the preceding claims, wherein each of the available tools is adapted to pick up and place the workpiece.

11. The method (200) according to any one of the preceding claims, wherein the ML agent is adapted to one or more of the following: supervised learning, unsupervised learning, and reinforcement learning.

12. A computer system (300) configured to select a tool (118) from a set of tools, said set of tools being used by at least one industrial robot (110, 120) to perform a predefined robotic task in a working environment, said computer system comprising: A machine learning (ML) agent (310) is configured to control the at least one industrial robot; A simulator (320) is configured to provide a computer simulation (322) of the working environment for training the ML agent to control the at least one industrial robot when the industrial robot is equipped with tools from the set. The interpreter (330) is configured to reward the ML agent for completing the robotic task or a part of the robotic task in the computer simulation; and The controller (340) is configured for each tool in the set: - Initialize the ML agent, - Initialize the simulator. - Begin training the ML agent in the computer simulation, and - During the training period, monitor the temporal evolution of the ML agent's reward until a predefined threshold is reached, and record indicators associated with the temporal evolution. It is also configured to select one of the tools based on the recorded indicators.

13. A computer program (349) comprising instructions that, when executed by a computer, cause the computer to perform the method (200) according to any one of claims 1 to 11.

Citation Information

Patent Citations

  • Automated Robotic Tool Selection

    US20230081119A1