Automatic configuration of robots in a multi-robot operating environment to optimize wear and other parameters

Abstract

A solution for a multi-robot configuration is co-optimized for wear, collisions, and optionally performance and / or energy expenditure over a set of heterogeneous parameters based on a set of tasks performed by the set of robots. The heterogeneous parameters may include two or more of the robots' respective base positions and orientations, the assignment of tasks to each robot, and the robots' respective target sequences and / or trajectories. This may be performed before runtime. The output may include, for each robot, a workcell layout, an ordered list or vector of targets, optionally a dwell duration at each target, and a path or trajectory between each pair of consecutive targets. The output may provide a complete and feasible solution to the problem, which can be used to control the robots without any modifications in the absence of timing variations. Genetic algorithms, such as differential evolution, may optionally be used in generating the population of candidate solutions.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This patent application claims priority to U.S. Patent Application No. 63 / 410,545, filed September 27, 2022, the entire disclosure of which is incorporated herein by reference for all purposes.

[0002] The present disclosure relates generally to robot configurations within a multi-robot operating environment or shared workspace, and to optimizing robot configurations within such environments or shared workspaces. [Background technology]

[0003] Conventional Technology Various applications may employ or desire to employ two or more robots in a multi-robot operating environment common to or otherwise shared by two or more robots. A multi-robot operating environment can take the form of a shared workspace. For example, two or more robots may be employed in performing tasks on or with one or more objects or workpieces in a common operating environment, such as threading bolts into chassis, where the robots may overlap in their range of motion.

[0004] Motion planning is a fundamental problem in robot control and robotics. A motion plan typically specifies a path a robot can follow from a start state to a goal state to complete a task without colliding with any obstacles in the operating environment or with a reduced likelihood of colliding with any obstacles in the operating environment. Challenges to motion planning involve the ability to perform motion planning at high speeds, possibly while accounting for environmental changes (e.g., changes in the position or orientation of obstacles in the environment). Challenges further include performing motion planning using relatively low-cost equipment, with relatively low energy consumption, and with a limited amount of storage (e.g., memory circuitry on a processor chip).

[0005] The operation of two or more robots in a multi-robot operating environment or shared workspace (a workspace is commonly referred to as a workcell) presents a particular class of problems: For example, motion plans should consider and avoid situations in which the robots or their robotic appendages may interfere with each other while performing a task.

[0006] One approach for operating multiple robots in a common workspace may be referred to as the task-level approach. Engineers can manually ensure that robots do not collide by defining portions of the workspace where the robots may collide (referred to herein as interference regions) and programming each robot so that only one robot is within the interference region of the workspace at any given time. For example, when a first robot begins to move into the interference region of the workspace, the first robot sets a flag. A controller (e.g., a programmed logic controller (PLC)) reads the flag and prevents other robots from moving into the interference region of the workspace until the first robot deasserts the flag upon exiting the interference region. While this approach is intuitive and easy to understand, it is typically difficult and time-consuming to implement and may not produce optimized results. This approach inevitably has low work throughput because the use of task-level collision avoidance typically leads to at least one of the robots being idle for a significant portion of the time, even if an idle robot is technically capable of performing useful work in the shared workspace.

[0007] In traditional approaches, a team of engineers typically divides the problem and optimizes smaller sub-problems (e.g., assigning tasks to robots, sequencing the tasks assigned to each robot, motion planning for each robot) independently of each other. This may involve iteratively simulating the motion to ensure that the robots / robot appendages do not collide with each other, which can take many hours of computational time and may not result in an optimal solution. Additionally, if a modification to the workspace results in a change in the trajectory of one of the robots / robot appendages, the entire workflow must be revalidated. Such an approach is naturally suboptimal and typically requires experts to go through a slow process of iteratively trying to find a combination of solutions that, when taken together, produce a good result. Summary of the Invention

[0008] Described herein are various methods and apparatus for generating a solution to a co-optimized multi-robot configuration, e.g., a co-optimized combination of workcell layout and task plan, over a set of heterogeneous parameters based on a given set of tasks to be performed by robots in a multi-robot operating environment, which can be performed offline or in a pre-runtime environment, providing a global optimizer for these types of problems.

[0009] The inputs may include a model of the multi-robot operating environment, a model of the robot, a limit on the total number of robots that may be employed, a set of tasks the robots will accomplish, and a limit on the total number of tasks that may be assigned to each robot (i.e., the target capabilities of the robots). The inputs may also optionally include one or more dwell durations for the robot or a portion thereof to dwell on a target, for example, to complete a task (e.g., screwing in a bolt or nut) or to avoid a collision. The inputs may also optionally include one or more of a set of bounds or constraints on one or more parameters or variables, or a time limit that limits the time provided to model or simulate a collision.

[0010] The output may include a complete solution to the problem, advantageously optimized with respect to at least the amount of wear the robot experiences as it performs the sequence of tasks, optimized with respect to collision assessment (e.g., probability or likelihood of collision and / or severity of collision), and optionally further optimized with respect to one or more other parameters, such as optimized with respect to energy expenditure, energy consumption, or energy efficiency, and / or optimized with respect to performance or latency (also referred to as time efficiency, e.g., time to complete the sequence of tasks). In particular, the solution may be optimized over a population of candidate solutions by an optimization engine that co-optimizes over a set of two or more heterogeneous parameters. The heterogeneous parameters may include, for example, two or more of the base positions and orientations of each of the robots, the assignment of tasks to each one of the robots, and the target sequences of each of the robots. In the absence of timing variations, the output may be used to control robots in a multi-robot environment without modification. Alternatively, one or more motion planners can be used during runtime to avoid collisions that may occur due to, for example, slight variations in timing (e.g., tightening a screw may take slightly longer or slightly shorter than other times).

[0011] The output may include, for example, a workcell layout and, for each robot, an ordered list or vector of targets (e.g., Robot1:{target 7, target 2, rest, target 9}), and optionally, the dwell duration at each target and the path or trajectory between each pair of consecutive targets (e.g., a collision-free path or trajectory).

[0012] The workcell layout may provide the base position and orientation of each robot base, for example in Cartesian coordinates, specified by a respective 6-tuple {X,Y,Z,r,p,y}, where X, Y, and Z each indicate a position along a respective axis in a Cartesian coordinate frame, r (i.e., roll) indicates the amount of rotation about a first one of the axes, p (e.g., pitch) indicates the amount of rotation about a second one of the axes, and y (e.g., yaw) indicates the amount of rotation about a third one of the axes.

[0013] The global optimizer may be based, for example, on a multivariate mixed integer optimization algorithm, such as the algorithm known as Differential Evolution (DE). Unless an algorithm is explicitly recited in the claims, the claims are not limited to DE algorithms.

[0014] The global optimizer optimizes the robot base placement (in Cartesian coordinates), the robot functional pose (i.e., in C-space), and the task plan for each robot (an ordered list of targets and rests for each robot). The primary optimization objective may be optimization based on the amount of wear the robot experiences and collision assessment, but additional optimization objectives may include energy expenditure, energy consumption or energy efficiency, performance or latency (also referred to as time efficiency, e.g., time to complete a sequence of tasks), efficient use of floor space, number of moves to complete a goal, ability to operate robots in parallel, minimizing robot wait time, robot availability, status conditions, and / or availability of robots suitable for performing particular types of tasks (e.g., availability of robots with particular types of end-of-arm tools or end effectors), etc. Thus, the structures and algorithms described herein facilitate the operation of two or more robots operating within a shared workspace or workcell, optimize such placement and operation, at least to some extent, for wear, and potentially prevent or at least reduce the risk of the robots or robotic appendages of the robots colliding with each other while operating to perform their respective tasks within the shared workspace. The structures and algorithms described herein can advantageously reduce the programming effort of a multi-robot workspace by performing, at least to some extent, optimized autonomous planning. Inputs may be limited to a description of the operating environment, the task(s) to be performed, and a geometric model of the robots. The structures and algorithms described herein can advantageously dynamically allocate tasks to be performed by the robots.

[0015] In the drawings, identical reference numbers identify similar elements or acts. The sizes and relative positions of elements in the drawings are not necessarily drawn to scale. For example, the shapes and angles of various elements are not drawn to scale, and some of these elements have been arbitrarily enlarged and positioned to improve the readability of the drawings. Furthermore, the particular shapes of the depicted elements are not intended to convey any information regarding the actual shape of the particular elements, but have been selected merely for ease of recognition in the drawings. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a schematic diagram of a shared workspace or multi-robot operating environment in which multiple robots operate to perform tasks and a configuration system performs optimizations to configure the robots, according to one illustrated implementation. [Figure 2] FIG. 2 is a functional block diagram of the configuration system of FIG. 1 , including one or more processors and one or more non-transitory processor-readable media that store processor-executable instructions, according to one illustrated implementation, and also showing multiple robots. [Figure 3] 1 illustrates a high-level robot configuration method of operation of a processor-based system for configuring multiple robots for a multi-robot operating environment in which multiple robots operate, according to at least one illustrated implementation. [Figure 4] 1 illustrates a high-level multi-robot environment simulation method of operation of a processor-based system for configuring multiple robots for a multi-robot operating environment in which the multiple robots operate, according to at least one illustrated implementation. [Figure 5] 1 illustrates a low-level multi-robot environment simulation method of operation of a processor-based system for configuring multiple robots for a multi-robot operating environment in which the multiple robots will operate, according to at least one illustrated implementation. [Figure 6]1 illustrates a low-level multi-robot optimization DE method of operation of a processor-based system for configuring multiple robots for a multi-robot operating environment in which multiple robots will operate, according to at least one illustrated implementation. [Figure 7] 1 illustrates a parameterized cost function that may be employed in conjunction with an implementation employing a differential evolution (DE) algorithm, according to at least one illustrated implementation. [Figure 8] 8 is a graph further illustrating the cost function of FIG. 7 according to at least one illustrated implementation. [Figure 9] 1 illustrates a data structure that may be employed by a processor-based system in representing candidate solutions in a format that allows for perturbations when performing the low-level multi-robot optimization DE method of FIG. 6 , for example, according to at least one illustrated implementation. [Figure 10] A low-level multi-robot DE candidate solution method of operation of a processor-based system for configuring multiple robots for a multi-robot operating environment in which multiple robots operate, according to at least one illustrated implementation, is shown, detailing the operation of a population generator. DETAILED DESCRIPTION OF THE INVENTION

[0017] In the following description, certain specific details are set forth to provide a thorough understanding of various disclosed embodiments. However, those skilled in the art will recognize that the embodiments can be practiced without one or more of these specific details, or with other methods, components, materials, etc. In other examples, well-known structures associated with computer systems, actuator systems, and / or communication networks have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments. In other examples, well-known computer vision methods and techniques for generating sensory data and volumetric representations of one or more objects, etc. have not been described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0018] Unless the context requires otherwise, throughout the following specification and claims, the word "comprise" and variations thereof, such as "comprises" and "comprising," are to be interpreted in their open and inclusive sense, such as "including but not limited to."

[0019] Throughout this specification, a reference to "one implementation" or "one embodiment" or "one embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one implementation or at least one embodiment. Thus, the appearances of the phrases "one implementation" or "one implementation" or "in one embodiment" or "in one embodiment" in various places throughout this specification do not necessarily all refer to the same implementation or embodiment. Furthermore, particular features, structures, or characteristics may be combined in any suitable manner in one or more implementations or embodiments.

[0020] As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the content clearly dictates otherwise. Also, it should be noted that the term "or" is generally used in its sense including "and / or" unless the content clearly dictates otherwise.

[0021] As used herein and in the appended claims, the terms optimizing, optimize, and optimized mean that an improved result is being prepared, produced, or created, or has been prepared, produced, or created. Such terms are used in their relative sense and do not imply that an absolute optimum value has been prepared, produced, or created.

[0022] As used herein and in the appended claims, the term "workspace" or "shared workspace" is used to refer to the current operating environment in which two or more robots operate, where one or more portions of the shared workspace are volumes in which the robots may collide with each other and are therefore sometimes referred to as interference regions. The operating environment may include obstacles and / or workpieces (i.e., items with which the robots interact or act).

[0023] As used herein and in the appended claims, the term "task" is used to refer to a robot task in which the robot transitions from Pose A to Pose B without colliding with obstacles in its environment. A task may possibly include grasping or ungrasping an item, moving or dropping an item, rotating an item, or retrieving or placing an item. The transition from Pose A to Pose B may optionally include transitions between one or more intermediate postures.

[0024] The headings provided herein and the Abstract of the Disclosure are for convenience only and do not interpret the scope or meaning of the embodiments.

[0025] FIG. 1 shows a robotic system 100 including multiple robots 102a, 102b, 102c (collectively 102) operating within a shared workspace 104 to perform tasks, according to one illustrated implementation.

[0026] The robot 102 can take any of a wide variety of forms. Typically, the robot 102 takes the form of or has one or more robotic appendages 103 (only one of which is referred to) and a base 105 (only one of which is referred to). The robot 102 may include one or more linkages having one or more joints and an actuator (e.g., an electric motor, a stepper motor, a solenoid, a pneumatic actuator, or a hydraulic actuator) coupled to the linkage and operable to move the linkage in response to a control signal or drive signal. A pneumatic actuator may include, for example, one or more pistons, cylinders, valves, a reservoir of gas, and / or a pressure source (e.g., a compressor, a blower). A hydraulic actuator may include, for example, one or more pistons, cylinders, valves, a reservoir of fluid (e.g., a low-compressibility hydraulic fluid), and / or a pressure source (e.g., a compressor, a blower). The robotic system 100 may employ other forms of robots 102, such as autonomous vehicles.

[0027] The shared workspace 104 typically represents a three-dimensional space in which the robots 102 can operate and move, although in certain limited implementations, the shared workspace 104 can represent a two-dimensional space. The shared workspace 104 is a volume or region in which at least portions of the robots 102 overlap in space and time or may collide if their motion is not controlled to avoid collisions. Note that the workspace 104 is a physical space or volume, and a physical space or volume is a position and orientation that can be conveniently represented, for example, via Cartesian coordinates relative to some frame of reference, e.g., the frame of reference represented by the orthogonal axes X, Y, and Z shown in FIG. 1 . Also, note that the frame of reference for the workspace 104 is distinct from the respective “configuration space” or “C-space” of any of the robots 102, which is typically represented by a set of joint positions, orientations, or configurations within the respective frame of reference of any of the robots 102.

[0028] As described herein, a robot 102a or a portion thereof can constitute an obstacle when considered from the perspective of another robot 102b (i.e., during motion planning for another robot 102b). The shared workspace 104 may further include other obstacles, such as portions of machinery (e.g., conveyors 106), posts, pillars, walls, ceilings, floors, tables, humans, and / or animals. The shared workspace 104 may further include one or more work items or workpieces, such as one or more parcels, packaging, fasteners, tools, items, or other objects, that the robot 102 manipulates as part of performing a task.

[0029] The robotic system 100 includes one or more processor-based multi-robot configuration optimization systems 108 (one is shown in FIG. 1). The multi-robot configuration optimization system(s) 108 receives a set of inputs 109 and generates as output 111 one or more solutions specifying a configuration of the robots 102, including a respective base position and orientation for each of the robots 102, a respective set of at least one defined pose, and a respective target sequence, which may be optimized to at least some extent with respect to the amount of wear the robots will experience when performing a sequence of tasks, and with respect to collision assessment (e.g., probability or likelihood of collision and / or severity of collision), and optionally further optimized with respect to one or more other parameters, such as energy expenditure, energy consumption, or energy efficiency, and / or performance or latency (also referred to as time efficiency, e.g., time to complete a sequence of tasks), efficient use of floor space, number of moves to complete a goal, ability to operate robots in parallel, minimization of robot wait time, robot availability, status conditions, and / or availability of robots suitable for performing a particular type of task (e.g., availability of robots having a particular type of end-of-arm tool or end effector), etc.

[0030] The multi-robot configuration optimization system(s) 108 may include a population generator 110, a multi-robot environment simulator 112, and a multi-robot optimization engine 114.

[0031] The population generator 110 generates a set of candidate solutions 116 based on the provided inputs 109. The candidate solutions 116 represent possible solutions to the configuration problem, i.e., how to configure the robots 102 in the workspace 104 to accomplish a set of tasks. Any given candidate solution 116 may or may not actually be feasible. That is, the initial candidate may be invalid (e.g., the robot is in an impossible location, there are unreachable targets, it has an infeasible task plan, or it would result in a collision). In some implementations, the population generator may attempt to find better candidate solutions.

[0032] The multi-robot environment simulator 112 models the multi-robot environment based on each candidate solution to determine certain attributes, such as the amount of wear the robots will incur or experience to complete the task, the probability or rate of collisions in completing the task, the level or other severity indicator of collisions, the feasibility or infeasibility of particular configurations specified by the candidate solutions, and optionally, the amount of energy expenditure or consumption required to complete the task and / or the amount of time required to complete the task. The multi-robot environment simulator 112 can reflect this in terms of costs generated via one or more cost functions.

[0033] The multi-robot optimization engine 114 evaluates candidate solutions based at least in part on their associated costs, and advantageously co-optimizes over two or more sets of heterogeneous parameters, such as over two or more of the robots' respective base positions and orientations, the assignment of tasks to each one of the robots, the robots' respective target sequences, and / or the respective trajectories or paths (e.g., collision-free paths) between successive targets. For ease of explanation, straight-line trajectories between successive targets are used, although the trajectories need not be straight-line trajectories.

[0034] The input 109 may include one or more static environmental models representing or characterizing the operating environment or workspace 104, e.g., representing floors, walls, ceilings, pillars, other obstacles, etc. The operating environment or workspace 104 may be represented by one or more models, e.g., geometric models (e.g., point clouds) representing floors, walls, ceilings, obstacles, and other objects in the operating environment, which may be represented, for example, in Cartesian coordinates.

[0035] The input 109 may include one or more robot models representing or characterizing each of the robots 102, specifying the geometry and kinematics, such as size or length, number of links, number of joints, joint types, range of motion, velocity limits, acceleration or jerk limits, etc. The robots 102 may be represented by one or more robot geometric models that define the geometry of a given robot 102a-102c, for example, in terms of joints, degrees of freedom, dimensions (e.g., linkage lengths), and / or in terms of the C-space of each of the robots 102a-102c.

[0036] The input 109 may include, for example, one or more sets of tasks to be performed, expressed as target objectives (e.g., positions or configurations). The tasks may be expressed, for example, in terms of end poses, configurations, or states, and / or intermediate poses, configurations, or states, of each robot 102a-102c. The poses, configurations, or states may be defined, for example, in terms of joint positions and joint angles / rotations (e.g., joint poses, joint coordinates) of each robot 102a-102c. The input 109 may optionally include one or more dwell durations specifying the amount of time a robot, or a portion thereof, should dwell on a given target to complete a task (e.g., tightening a screw or nut, picking and placing an object with the goal of sorting a pile of objects into two or more separate piles of each type of object by two or more robots operating within a common workspace).

[0037] Input 109 may optionally include limits on the number of robots that can be configured within workspace 104. Input 109 may optionally include limits on the number of tasks or targets that can be assigned to a given robot 102 a-102 c, referred to herein as task capacity, that can be configured within workspace 104, for example, to limit the complexity of the configuration problem to ensure that the configuration problem is solvable within some acceptable time period using available computational resources or to preclude certain solutions that are estimated to be too slow in light of apparent over-allocation of tasks or targets to a given robot 102 a-102 c. Input 109 may optionally include one or more bounds or constraints on variables or other parameters. Input 109 may optionally include a total number of iteration cycles or time limits for iterations that can be used in refining candidate solutions, for example, to ensure that the configuration problem is solvable using available computational resources or within some acceptable time period.

[0038] The robotic system 100 may optionally include one or more robot control systems 118 (only one shown in FIG. 1 ) communicatively coupled to control the robot 102. The robot control system(s) 118 may, for example, provide control signals (e.g., drive signals) to various actuators to move the robot 102 between various configurations to various designated targets in order to perform designated tasks.

[0039] The robotic system 100 may optionally include one or more motion planners 120 (only one shown in FIG. 1 ) communicatively coupled to control the robot 102. The motion planner(s) 120 generate or refine a motion plan for the robot 102, as described elsewhere herein, to account for slight deviations in time relative to the motion plan provided by the multi-robot optimization engine 114 or to account for the unexpected appearance of an obstacle (e.g., a human entering the operating environment or workspace 104). Any motion planner 120 may be operable to dynamically generate a motion plan for the robot 102 to perform a task in the operating environment. The motion planner 120, as well as other structure and / or operation, may be as described in U.S. Patent Application No. 62 / 865,431, filed June 24, 2019.

[0040] If included, the motion planner 120 is optionally communicatively coupled to receive as input, for example, sensory data provided by a perception subsystem (not shown). The sensory data describes static and / or dynamic objects in the workspace 104 that are not known a priori. The sensory data may be raw data, such as sensed via one or more sensors (e.g., cameras, stereo cameras, time-of-flight cameras, LIDAR), and / or converted by the perception subsystem into digital representations of obstacles, which can generate respective discretizations of representations of the environment in which the robot 102 operates to perform tasks for a variety of different scenarios.

[0041] 1 as lines between various structures, with arrows indicating the direction of inputs 109 and outputs 111 in some cases. The communication paths may take the form of, for example, one or more wired communication paths (e.g., electrical conductors, signal buses, or optical fibers) and / or one or more wireless communication paths (e.g., via RF or microwave radios and antennas, infrared transceivers). A communication channel may include, for example, one or more transmitters, receivers, transceivers, radios, routers, wired ports such as Ethernet ports, etc.

[0042] FIG. 2 illustrates a functional block diagram of the robotic system 100 of FIG. 1 according to at least one example implementation.

[0043] The robotic system 100 can include a robot configuration optimization system 108 and a robot 102. The optimization system 108 can be communicatively coupled to control the robot 102 directly or indirectly via an intervening robot control system 118 (FIG. 1).

[0044] Each robot 102a-102c may include a set of links, joints, end-of-arm tools or end effectors, and / or actuators 201a, 201b, 201c (three shown, collectively 201) operable to move the links about the joints. Each robot 102a-102c may include one or more motion controllers (e.g., motor controllers) 202 (only one shown) that receive control signals from, for example, the robot configuration optimization system 108 and provide drive signals to drive the actuators 201. A motion controller 202 may be dedicated to controlling a particular one of the actuators 201.

[0045] The robot configuration optimization system 108 is described in detail for purposes of illustration, and those skilled in the art will recognize that this description is exemplary and that variations can be made to the robot configuration optimization system 108 as described and illustrated.

[0046] The robot configuration optimization system 108 may include one or more processor(s) 222 and one or more associated non-transitory computer- or processor-readable storage media, such as a system memory 224a, a disk drive 224b, and / or memory or registers (not shown) of the processor(s) 222. The non-transitory computer- or processor-readable storage media 224a, 224b are communicatively coupled to the processor(s) 222 via one or more communication channels, such as a system bus 229. The system bus 229 may employ any known bus structure or architecture, including a memory bus with a memory controller, a peripheral bus, and / or a local bus. One or more of such components may also, or instead, communicate with each other via one or more other communication channels, such as one or more parallel cables, serial cables, or wireless network channels capable of high-speed communication, e.g., Universal Serial Bus (“USB”) 3.0, Peripheral Component Interconnect Express (PCIe), or Thunderbolt®.

[0047] The robot configuration optimization system 108 may also be communicatively coupled to one or more remote computer systems 212, e.g., a server computer, a desktop computer, a laptop computer, an ultra-portable computer, a tablet computer, a smartphone, a wearable computer, and / or sensors (not shown in FIG. 2 ), that are directly or indirectly communicatively coupled to the various components of the robot configuration optimization system 108, e.g., via a network interface (not shown) to a network 210. The remote computing system 212 (e.g., a server computer (e.g., a source of input)) may be used to program, configure, control, or otherwise interface with or provide input data (e.g., ambient models, robot models, tasks, target objectives, limits on the total number of robots, limits on tasks per robot, bounds or constraints on variables or other parameters, limits on iterations) to the robot configuration optimization system 108 and various components within the robotic system 100. Such connections may be through one or more communication channels, e.g., one or more wide area networks (WANs), e.g., Ethernet, or the Internet using the Internet Protocol. In some implementations, pre-runtime calculations (e.g., generating outputs) may be performed by a system separate from the robot 102, while runtime calculations may be performed by one or more optional intervening motion planners 120 (FIG. 1), which in some implementations may be onboard the robots 102a-102c.

[0048] As mentioned, the robot configuration optimization system 108 may include one or more processor(s) 222 (i.e., circuitry), non-transitory storage media 224a, 224b, and a system bus 229 that couples the various system components. The processor 222 may be any logic processing unit, such as one or more central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), programmable logic controllers (PLCs), etc. Non-limiting examples of commercially available computer systems include, but are not limited to, the Celeron, Core, Core 2, Itanium, and Xeon families of microprocessors offered by Intel® Corporation, USA; the K8, K10, Bulldozer, and Bobcat series of microprocessors offered by Advanced Micro Devices, USA; the A5, A6, and A7 series of microprocessors offered by Apple Computer, USA; the Snapdragon series of microprocessors offered by Qualcomm, Inc., USA; and the SPARC series of microprocessors offered by Oracle Corp., USA.The construction and operation of the various structures shown in FIG. 2 are based on International Patent Application No. PCT / US2017 / 036880, filed June 9, 2017, entitled "MOTION PLANNING FOR AUTONOMOUS VEHICLES AND RECONFIGURABLE MOTION PLANNING PROCESSORS," International Patent Application Publication No. WO2016 / 122840, filed January 5, 2016, entitled "SPECIALIZED ROBOT MOTION PLANNING HARDWARE AND METHODS OF MAKING AND USING SAME," and International Patent Application Publication No. WO2016 / 122840, filed January 12, 2018, entitled "APPARATUS, METHOD AND ARTICLE TO FACILITATE MOTION PLANNING OF AN AUTONOMOUS VEHICLE IN AN ENVIRONMENT HAVING DYNAMIC No. 62 / 616,783, filed June 24, 2019, entitled "MOTION PLANNING FOR MULTIPLE ROBOTS IN SHARED WORKSPACE," and / or U.S. patent application Ser. No. 62 / 865,431, filed June 24, 2019, entitled "MOTION PLANNING FOR MULTIPLE ROBOTS IN SHARED WORKSPACE."

[0049] The system memory 224a may include read only memory ("ROM") 226, random access memory ("RAM") 228, flash memory 230, and EEPROM (not shown). A basic input / output system ("BIOS") 232, which may form part of the ROM 226, contains the basic routines that help to transfer information between elements within the robotic system 100, such as during start-up.

[0050] The drive 224b may be, for example, a hard disk drive for reading from and writing to a magnetic disk, a solid-state (e.g., flash memory) drive for reading from and writing to a solid-state memory, and / or an optical disk drive for reading from and writing to a removable optical disk. The robot configuration optimization system 108 may also include any combination of such drives in various different embodiments. The drive 224b may communicate with the processor(s) 222 via a system bus 229. The drive(s) 224b may include an interface or controller (not shown) coupled between such a drive and the system bus 229, as known to those skilled in the art. The drive 224b and its associated computer-readable media provide non-volatile storage of computer- or processor-readable and / or executable instructions, data structures, program modules, and other data for the robot system 100. Those skilled in the art will appreciate that other types of computer-readable media capable of storing computer-accessible data may be used, such as a WORM drive, a RAID drive, a magnetic cassette, a digital video disk (“DVD”), a Bernoulli cartridge, RAM, ROM, a smart card, etc.

[0051] Executable instructions and data, such as an operating system 236, one or more application programs 238, other programs or modules 240, and program data 242, may be stored in system memory 224a. The application program 238 may include processor-executable instructions that cause the processor(s) 222 to one or more of: generate a population of candidate solutions; model the candidate solutions; generate or determine a cost associated with each candidate solution based at least in part on the modeling; and perform optimization on the population of C candidate solutions with respect to at least the amount of wear the robot experiences as it performs the sequence of tasks, and with respect to collision assessment (e.g., probability or likelihood of collision and / or severity of collision), optionally further optimized with respect to none or more of other parameters, such as none, energy expenditure, energy consumption or energy efficiency, efficient use of performance or latency (also referred to as time efficiency, e.g., time to complete a sequence of tasks), efficient use of floor space, number of moves to complete a goal, ability to operate robots in parallel, minimizing robot wait time, robot availability, status conditions, and / or availability of robots suitable for performing particular types of tasks (e.g., availability of robots having particular types of end-of-arm tools or end effectors). The optimization can be performed by an optimization engine that co-optimizes over a set of two or more heterogeneous parameters for two or more of the base positions and orientations of the robots, the assignment of tasks to each one of the robots, and the target sequences of the robots, and / or outputs that can be used to position and orient the robots and cause them to perform tasks within a multi-robot operating environment. Such operations can be performed as described herein (e.g., with reference to Figures 3 and 10) and in the references incorporated herein by reference.The processor-executable instructions, in at least some implementations, cause the processor(s) 222 to construct a motion plan (e.g., collision detection or evaluation, updating costs of edges in a motion planning graph based on the collision detection or evaluation, and performing path planning or evaluation). The application program 238 may further include one or more machine-readable and machine-executable instructions that cause the processor(s) 222 to perform other operations, for example, optionally processing sensory data (e.g., captured via sensors). The application program 238 may further include one or more machine-executable instructions that cause the processor(s) 222 to perform various other methods described herein and in the references incorporated herein by reference.

[0052] While shown in FIG. 2 as being stored in system memory 224a, operating system 236, application programs 238, other applications, programs / modules 240, and program data 242 may be stored on other non-transitory computer- or processor-readable media, such as drive(s) 224b.

[0053] Although not required, many of the implementations will be described in the general context of computer-executable instructions, such as program application modules, objects, or macros, stored on a computer- or processor-readable medium and executed by one or more computers or processors that can perform candidate solution generation, e.g., modeling candidate solutions via forward kinematics, detecting collisions in the model, determining time and other costs to implement, cost generation via a cost function, co-optimization over a set of heterogeneous parameters, generating trajectories or paths (e.g., collision-free paths), and / or other motion planning operations.

[0054] In various implementations, the operations may be performed entirely in hardware circuitry or as software stored in memory storage, such as system memory 224a, by one or more hardware processors 222, such as one or more microprocessors, digital signal processors (DSPs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), graphics processing unit (GPU) processors, program logic controllers (PLCs), electrically programmable read-only memories (EEPROMs), or a combination of hardware circuitry and software stored in memory storage.

[0055] The robot configuration optimization system 108 may optionally include one or more input / output components, for example, a monitor or touchscreen display 244, a keypad or keyboard 246, and / or a pointing device such as a computer mouse 248.

[0056] Those skilled in the art will appreciate that the illustrated implementation, as well as other implementations, can be practiced with other system architectures and configurations and / or other computing system architectures and configurations, including those of robots, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, personal computers (“PCs”), networked PCs, minicomputers, mainframe computers, etc. Implementations or embodiments, or portions thereof (e.g., at configuration time and run time), can be practiced in distributed computing environments where tasks or modules are performed by remote processing devices linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote memory storage devices or media. However, where and how certain types of information are stored is important to help improve robot configurations.

[0057] 3 illustrates a high-level robot configuration method 300 of operation of a processor-based system for configuring multiple robots for a multi-robot operating environment in which the multiple robots operate, according to at least one illustrated implementation. The processor-based system may include at least one processor and at least one non-transitory processor-readable medium that stores at least one of data and processor-executable instructions. When executed by the at least one processor, the processor-executable instructions cause the at least one processor to perform various operations or acts of the robot configuration method 300.

[0058] A number of robots can be configured to perform a set of tasks. The tasks can be specified as a task plan. The task plan can specify T tasks that need to be performed by R robots. The task plan can be modeled as a vector for each robot, where the vector is an ordered list of tasks that each robot should perform, e.g., {task 7, task 2, task 9}. The task vector can also optionally include a dwell duration that specifies the duration that the robot, or a portion thereof, should dwell in a given configuration or goal. The task vector can also specify a home pose and / or other "functional poses" not directly related to solving a task (e.g., an "out of the way" or storage pose). The poses can be specified in the robot's C-space.

[0059] The robot configuration method 300 may begin at 302, for example, in response to booting or powering up the system or its components, receiving information or data, or being called or invoked by a calling routine or program. The robot configuration method 300 may be performed at configuration time, which may occur before runtime, or before runtime. This advantageously allows some of the most computationally intensive work to be performed before runtime, when responsiveness is not a particular concern.

[0060] At 304, at least one component of the processor-based system receives inputs that characterize the multi-robot environment, the set of tasks to be performed, and provides various constraints or boundaries for the problem.

[0061] For example, the processor-based system can receive one or more models of a multi-robot operating environment. The model(s) can represent the physical environment in which the multiple robots operate, such as the floors, walls, and various objects within the environment.

[0062] Also, for example, the processor-based system may receive a respective model for each of multiple robots operating within a multi-robot operating environment, where the robot model may represent physical attributes of the robot, such as physical dimensions, range of motion, number of joints, number of links, link lengths, end effector type, velocity limits, acceleration limits, etc.

[0063] Also, for example, the processor-based system may receive a set of tasks or targets. The targets may represent various locations to which each robot, or portion thereof, must move in sequence or at specific times to complete the set of tasks. The targets may be represented, for example, within the configuration space (C-space) of each robot. Also, for example, the processor-based system may optionally receive one or more dwell durations for dwelling on one or more targets while at least one of the robots performs at least one task. The dwell durations may advantageously reflect an expected amount of time an end effector of the robot needs to remain on a given target to complete a respective task (e.g., threading a fastener into a threaded opening, inserting a part into a receptacle).

[0064] Also, for example, the processor-based system may optionally receive a set of bounds or constraints on variables. Various bounds and constraints may be applicable to optimization in a multi-robot environment. For example, the processor-based system may optionally receive a set of time intervals specifying time limits for modeling motion or simulating collisions. For example, the processor-based system may optionally receive a limit on the total number of robots allowed to operate in the multi-robot operating environment. For example, the processor-based system may optionally receive a maximum number of tasks or targets allowed per robot, which, again, may prevent problems from being submitted that are too complex to solve given the available computational resources or time.

[0065] At 306, the population generator generates a population of C candidate solutions. The population can include one or more candidate solutions. Each candidate solution in the population of C candidate solutions specifies, for each of the robots, a respective base position and orientation, a respective set of at least one defined pose, and a respective target sequence. The respective base position and orientation specifies a respective position and orientation of a base for each robot within the multi-robot operating environment. The respective set of at least one defined pose specifies at least a respective home pose and / or other functional pose (e.g., an out-of-the-way pose or a storage pose) for each robot within the multi-robot operating environment. The respective target sequence includes a respective ordered list of targets for each robot to move to complete a respective sequence of tasks. The population generator can be implemented as one or more processors executing processor-executable instructions.

[0066] The population generator can take the form of a pseudo-random population generator that can pseudo-randomly generate a population of C candidate solutions based on one or more input parameters. The pseudo-random population generator generates candidate solutions that may or may not actually be feasible solutions. Alternatively, the population generator can generate a population of C candidate solutions that have a lower probability of being an invalid candidate solution than a purely pseudo-randomly generated population of C candidate solutions. This can generate a population of candidate solutions that results in faster optimization without losing configuration space coverage. For example, the population generator can take into account the operating environment to avoid candidate solutions that would place the robot's base in an impossible location (e.g., occupied by a wall or other object), result in an infeasible task plan with one or more targets unreachable by the robot, or result in a collision. When a pseudo-random population generator is used, several techniques can be employed to improve the candidate solutions, such as those described below with reference to FIG. 6.

[0067] At 308, the optimization engine performs an optimization on the population of C candidate solutions, optimizing for at least the amount of wear the robot experiences when performing the sequence of tasks, optimizing for collision assessment (e.g., probability or likelihood of collision and / or severity of collision), and optionally optimizing for zero or more other parameters, such as energy expenditure, energy consumption or energy efficiency, performance or latency (also referred to as time efficiency, e.g., time to complete a sequence of tasks), efficient use of floor space, number of moves to complete a goal, ability to operate robots in parallel, minimization of robot wait time, robot availability, status conditions, and / or availability of robots suitable for performing particular types of tasks (e.g., availability of robots with particular types of end-of-arm tools or end effectors), etc. In particular, the optimization engine co-optimizes two or more of the base positions and orientations of the robots, the assignment of tasks to each one of the robots, and the target sequence of the robots, over a set of two or more heterogeneous parameters. The optimization engine may select an optimized candidate solution using, for example, a co-optimized combination of the respective optimized base positions and orientations for each of the robots' respective bases, the optimized task assignments, and the optimized motion plans. The optimization may relate to the amount of wear the robots experience as they perform a sequence of tasks. The optimization may further relate to collision assessment (e.g., the probability or likelihood of a collision and / or the severity of a collision). The optimization may additionally and optionally relate to zero, one, or more of one or more other parameters, such as energy expenditure, energy consumption or energy efficiency, performance or latency (also referred to as time efficiency, e.g., the time to complete a sequence of tasks), efficient use of floor space, the number of moves to complete a goal, the ability to operate robots in parallel, minimizing robot wait time, robot availability, status conditions, and / or the availability of robots suitable for performing a particular type of task (e.g., the availability of robots with a particular type of end-of-arm tool or end effector), etc.The optimization engine may be implemented as one or more processors executing processor-executable instructions. As described below with reference to Figure 4, the optimization engine may interact with the multi-robot simulator in performing the optimization, for example, by providing candidate solutions to be simulated and receiving costs generated or determined by the multi-robot simulator, where the costs characterize zero or more of the following: wear and collision probability or rate for each candidate solution, optionally the severity of collisions for each candidate solution, and optionally additional parameters, such as energy expenditure, energy consumption or energy efficiency, performance or latency (also referred to as time efficiency, e.g., time to complete a sequence of tasks), efficient use of floor space, number of moves to complete a goal, ability to operate robots in parallel, minimization of robot wait time, robot availability, status conditions, and / or availability of robots suitable for performing particular types of tasks (e.g., availability of robots with particular types of end-of-arm tools or end effectors), etc. The optimization engine may select one of the candidate solutions based at least in part on respective costs associated with the candidate solutions, each cost being based at least in part on an amount of wear experienced by the robot in completing the sequence of tasks and a collision value determined for each candidate solution.

[0068] At 310, the optimization engine provides output from the optimization. In particular, the optimization engine can provide as output one or more of a respective base position and orientation for each of the robots, a respective task assignment for each of the robots, a respective motion plan for each of the robots, and / or a set of collision-free paths for each of the robots with or without dwell durations at each target in the robot's C-space. The output may include an optimized task assignment that specifies, for each robot, a respective sequence of tasks to be performed in the form of an optimized ordered list of targets in the robot's C-space and one or more dwell durations at one or more of the targets. The output may additionally or alternatively include an optimized motion plan that specifies a set of collision-free paths, the set of collision-free paths specifying a respective collision-free path between each pair of consecutive targets in the ordered list of targets. In at least some implementations, the output is sufficient to drive the robot to perform the set of tasks. In at least some implementations, a motion planner may be employed to refine the motion plan.

[0069] The high-level robot configuration method 300 may end at 312, for example, until called again. Although the high-level robot configuration method 300 is described in terms of an ordered flow, various acts or operations will be performed simultaneously or in parallel in many implementations.

[0070] FIG. 4 illustrates a high-level multi-robot environment simulation method 400 of a processor-based system's operation for configuring multiple robots for a multi-robot operating environment in which the robots operate, according to at least one example implementation. The processor-based system may include at least one processor and at least one non-transitory processor-readable medium that stores at least one of data and processor-executable instructions. When executed by the at least one processor, the processor-executable instructions cause the at least one processor to perform various operations or acts of the multi-robot environment simulation method 400. The multi-robot environment simulation method 400 may be executed by a multi-robot environment simulator, which may be implemented by one or more separate processors dedicated to simulating movement in the multi-robot environment. Alternatively, the multi-robot environment simulator may be implemented by one or more processors that perform other operations, such as optimization.

[0071] The multi-robot environment simulation method 400 may begin at 402, for example, in response to system startup or power-up, receipt of information or data, or invocation or invocation by a calling routine or program (e.g., invocation by the high-level robot configuration method 300).

[0072] At 404, the multi-robot environment simulator receives candidate solutions, for example, from an optimization engine or population generator.

[0073] At 406, the multi-robot environment simulator models the robots in the multi-robot environment based on the particular candidate solution. The candidate solution may be submitted or provided to the multi-robot environment simulator by, for example, an optimization engine. A conventional modeling package may be used, for example, a modeling package that uses forward kinematics to model the motion of the robots based on the candidate solution.

[0074] At 408, the multi-robot environment simulator determines the amount of wear the robots will incur or experience to complete the sequence of tasks through modeling performed by the multi-robot environment simulator. The determining may include determining the amount of wear each robot will incur to complete the sequence of tasks. The determining may additionally or alternatively include determining the cumulative amount of wear incurred by the set of robots to complete the sequence of tasks by all of the robots. This may, for example, employ a model of wear that represents the respective amount of wear experienced by at least one robot, or portion thereof, with respect to one or more of the position, movement, velocity, acceleration, jerk, or torque of at least one robot, or portion thereof, in executing a movement and / or performing a set of tasks. Such a wear model may be based on empirical data collected over a relatively large number of operations of robots of the same or at least similar type as the type of robot used in the multi-robot environment.

[0075] Optionally, at 410, the multi-robot environment simulator determines the energy expenditure or energy consumption incurred by the robots to complete the sequence of tasks through modeling performed by the multi-robot environment simulator. The determining may include determining the energy expenditure or energy consumption incurred by each robot to complete the sequence of tasks. The determining may additionally or alternatively include determining a cumulative amount of energy expenditure or energy consumption incurred by the set of robots to complete the sequence of tasks by all of the robots. This may, for example, employ an energy expenditure or energy consumption model that represents a respective amount of energy expenditure or energy consumption that will be incurred by at least one robot with respect to one or more of position, movement, velocity, acceleration, jerk, or torque of at least one robot or part of a robotic appendage in executing a movement and / or performing a set of tasks. Such an energy expenditure or energy consumption model may be based on empirical data collected over a relatively large number of operations of robots of the same type or at least similar to the type of robot employed in the multi-robot environment.

[0076] Optionally, at 412, the multi-robot environment simulator determines respective times for completing the sequence of tasks through modeling performed by the multi-robot environment simulator. The determination may include determining a total time for completing the sequence of tasks by all of the robots, which may need to wait for each other. This may include, for example, determining respective times for completing each task for each of the robots.

[0077] At 414, the multi-robot environment simulator determines a collision value representing the rate or probability of a collision occurring in completing the sequence of tasks through modeling performed by the multi-robot environment simulator. The determination may include determining an overall collision value for the completion of the sequence of tasks by all of the robots. This may include, for example, determining a respective collision value for each of the robots.

[0078] At 416, the multi-robot environment simulator provides the determined values ​​(e.g., determined wear values, determined collision values, determined energy expenditure or consumption values, time to complete the sequence of tasks) to, for example, an optimization engine. In some implementations, the multi-robot environment simulator provides the determined wear values ​​and collision values, and optionally other values, as separate values ​​for processing by the optimization engine, as described elsewhere herein, to form values ​​that can be co-optimized across two or more heterogeneous parameters.

[0079] The multi-robot environment simulation method 400 may terminate at 418, for example, until called again. Although the multi-robot environment simulation method 400 is described with respect to an ordered flow, various acts or operations are performed simultaneously or in parallel in many implementations.

[0080] FIG. 5 illustrates a low-level multi-robot environment simulation method 500 of a processor-based system's operation for configuring multiple robots for a multi-robot operating environment in which the robots operate, according to at least one illustrated implementation. The processor-based system may include at least one processor and at least one non-transitory processor-readable medium that stores at least one of data and processor-executable instructions. When executed by the at least one processor, the processor-executable instructions cause the at least one processor to perform various operations or acts of the multi-robot environment simulation method 500. The multi-robot environment simulation method 500 may be executed by a multi-robot environment simulator, which may be implemented by one or more separate processors dedicated to simulating movement in the multi-robot environment. Alternatively, the multi-robot environment simulator may be implemented by one or more processors that perform other operations, such as optimization.

[0081] The multi-robot environment simulation method 500 may begin at 502, for example, in response to system startup or power-up, receipt of information or data, or invocation or invocation by a calling routine or program (e.g., invocation by the high-level robot configuration method 300).

[0082] At 504, the multi-robot environment simulator virtually executes each task in the sequence of tasks for each robot as specified by the candidate solution being evaluated or considered. The sequence of tasks may be specified as a respective ordered list of tasks. Each ordered list of tasks is equivalent to an ordered list of trajectories in the robot's C-space. The ordered list of tasks may include multiple trajectories between successive poses or configurations (e.g., joint configurations), one or more dwell durations in one or more poses, a home pose, and one or more other defined functional poses in the robot's C-space, such as a stowed or "out of the way" pose. To virtually execute each task, the processor-executable instructions, when executed by at least one processor, cause the processor to virtually simulate one or more of the multiple trajectories and one or more dwell time periods in the one or more poses, the home pose, or one or more other defined functional poses.

[0083] At 506, for each epoch of the plurality of epochs (e.g., 0.1 seconds or some selected input value), the multi-robot environment simulator samples the C-space position of a portion of at least one of the robots. The multi-robot environment simulator uses forward kinematics to identify potential collisions between one or more parts of each one of the robots and another part of each one of the robots, between each one of the robots and another one of the robots in the environment, and between each one of the robots and another object in the multi-robot operating environment that is not another robot.

[0084] The multi-robot environment simulation method 500 can end at 508, for example, until it is called again. Although the low-level multi-robot environment simulation method 500 is described with respect to an ordered flow, various acts or operations are performed simultaneously or in parallel in many implementations.

[0085] The described systems and methods may employ various approaches to refine or improve a candidate solution. For example, some approaches may start with a base solution and refine the base solution. Also, for example, some approaches may employ genetic algorithms or approaches, such as a differential evolution (DE) algorithm or similar techniques. A version of the DE algorithm is described with reference to FIG. 6 below.

[0086] FIG. 6 illustrates a low-level multi-robot optimization DE method 600 of operation of a processor-based system for configuring multiple robots for a multi-robot operating environment in which the multiple robots operate, according to at least one illustrated implementation. The processor-based system may include at least one processor and at least one non-transitory processor-readable medium that stores at least one of data and processor-executable instructions. When executed by the at least one processor, the processor-executable instructions cause the at least one processor to perform various operations or acts of the multi-robot optimization method 600. The multi-robot optimization method 600 may be performed by an optimization engine, which may be implemented by one or more separate processors dedicated to the optimization engine of the multi-robot environment. Alternatively, the multi-robot environment simulator may be implemented by one or more processors that perform other operations.

[0087] The multi-robot environment simulation method 600 may begin at 602, for example, in response to system startup or power-up, receipt of information or data, or invocation or invocation by a calling routine or program (e.g., invocation by the high-level robot configuration method 300).

[0088] At 604, a population generator generates a population of candidate solutions.

[0089] At 606, an outer loop counter I is initialized, for example, to zero. At 608, the outer loop counter I is incremented, for example, to I+1.

[0090] At 610, the candidate solutions are represented in a format that allows for perturbations. For example, the candidate solutions may be represented as a candidate solution vector. The candidate solution vector may, for example, include multiple real vector elements, e.g., one real vector element per task. The real vector elements may represent each combination of a respective one of the tasks, a priority for each one of the tasks, and one of the robots identified to perform each one of the tasks.

[0091] At 612, the optimizer engine perturbs the candidate solution I or perturbs the candidate solution I to generate a perturbed candidate solution I′. For example, the optimizer engine may modify real vector elements (i.e., real-valued values) of the candidate solution vector.

[0092] At 614, the multi-robot environment simulator models the perturbed candidate solution / '. For example, the multi-robot environment simulator can determine an amount of wear experienced to complete the sequence of tasks for the perturbed candidate solution / ' and a collision value representing the rate or probability of a collision occurring when completing the sequence of tasks for the perturbed candidate solution / '. Optionally, the multi-robot environment simulator can further determine values ​​for zero or more additional parameters (e.g., energy expenditure, energy consumption or energy efficiency, performance or latency) for the perturbed candidate solution / '.

[0093] In 616, the optimization engine or multi-robot environment simulator determines a cost value for the perturbed candidate solution I′. The determined cost value may be a function of a determined amount of wear experienced in completing the sequence of tasks for the perturbed candidate solution I′ and a determined collision value for the perturbed candidate solution I′. The determined cost value may also be a function of a determined energy expenditure, energy consumption, or energy efficiency for completing the sequence of tasks for the perturbed candidate solution I′, and / or a determined performance or latency (time efficiency for completing the sequence of tasks for the perturbed candidate solution I′). In DE, the cost function is a piecewise logarithmic function, but can be parameterized, for example, as represented by parameterization 700 in FIG. 7 and graph 800 in FIG. 8. In FIG. 7, the parameter “task_reachability” is equal to (number of reachable target objectives) / (total number of target objectives). Thus, if all targets are reachable, the value of the parameter “task_reachability” is equal to 1.

[0094] At 618, the optimization engine or multi-robot environment simulator determines whether the perturbed candidate solution I′ has a lower associated cost than the candidate solution I.

[0095] At 620, in response to determining that the perturbed candidate solution I′ has a lower associated cost than the candidate solution I, the optimization engine replaces each candidate solution I in the population of C candidate solutions with the perturbed candidate solution I′. In response to determining that the perturbed candidate solution I′ does not have a lower associated cost than the candidate solution I, the optimization engine leaves the population of C candidate solutions unmodified and passes control directly to 622.

[0096] At 622, the optimization engine determines whether an inner loop iteration termination condition has occurred. For example, the optimization engine determines whether an iteration limit has been reached and / or whether an iteration time limit has been reached when convergence has occurred. Convergence may be considered to have occurred, for example, when the standard deviation of costs over the current population of candidate solutions is less than the epsilon value.

[0097] If an exit condition occurs, control proceeds to 626, where the multi-robot environment simulation method 600 may end, for example, until called again. If an exit condition does not occur, control proceeds to 624, where the optimization engine or multi-robot environment simulator determines whether there are more candidate solutions in the perturbed population of candidate solutions. If there are more candidate solutions in the perturbed population of candidate solutions, control proceeds to 608, where the inner loop counter is incremented, and the next candidate solution is perturbed and analyzed. If there are no more candidate solutions in the perturbed population of candidate solutions, control proceeds to 606, where the inner loop counter is reinitialized, and possibly another pass through the updated population of candidate solutions may be performed.

[0098] Various operations of the multi-robot environment simulation method 600 can be repeated for multiple iterations to refine the population of candidate solutions until a termination condition (e.g., convergence) is met.

[0099] Although the low-level multi-robot optimized DE method 600 is described with respect to an ordered flow, various acts or operations are performed simultaneously or in parallel in many implementations.

[0100] FIG. 9 illustrates a data structure 900 that may be employed by a processor-based system in representing candidate solutions in a format that allows for perturbations, for example, when performing the low-level multi-robot optimization DE method 600 (FIG. 6) according to at least one illustrated implementation.

[0101] The candidate solution I can advantageously be represented as a vector of numbers, so that the system can perform one or more functions on the numbers to "perturb" the numbers, and thus the candidate solution, to generate a perturbed candidate solution I'. There are various ways to represent the candidate solution. One approach is described herein with reference to the data structure 900 shown in Figure 9. The system may employ this approach or other approaches to represent the candidate solution.

[0102] Candidate solutions can first be represented as a two-dimensional (2D) matrix, which is flattened into a one-dimensional (1D) vector. In the 2D matrix, rows correspond to tasks. Thus, for a problem with T tasks, there are T rows. In the 2D matrix, columns correspond to robots and priorities. If there are P priority levels and R robots, there are P*R columns. The example shown in FIG. 9 has three robots, three priority levels, and four tasks. In use, a particular problem may have a different number of robots, a different number of priority levels, and a different number of tasks than those shown in FIG. 9. The illustrated example has been simplified for ease of understanding.

[0103] Each task is assigned to a single robot, as shown in Figure 9 by each row having exactly one entry marked (i.e., marked with an asterisk *) indicating which robot will perform that task at which priority. A given robot can perform up to the maximum number of tasks specified in the input as the task capacity per robot. In particular, a given robot can only perform one task per priority level; i.e., there is only one asterisk (*) per column.

[0104] The system can flatten the 2D matrix into a 1D vector of length T, with one vector element per task. The vector elements correspond to the "entry number" of the 2D matrix, starting from 1. Thus, in the example shown in Figure 9, task 1 has a value of 2, task 2 has a value of 13 (the first row has 9 entries, and task 2's assignment (see the asterisk *) is in the fourth entry of row 2), task 3 has a value of 21, and task 4 has a value of 36. Notably, the vector elements are all integers.

[0105] The system can normalize the resulting integer vector <2,13,21,36> so that all values ​​are between 0 and 1. For example, the system can divide each value by the total number of entries in the matrix, e.g., P*R*T=36. The resulting normalized real-valued vector is <2 / 36, 13 / 36, 21 / 36, 36 / 36>, where the individual vector elements sum to 1.

[0106] To perturb the candidate real-valued vector V, the system may first take several (e.g., three) other randomly chosen candidates (e.g., A, B, and C) and represent those randomly chosen candidates (e.g., A, B, and C) with their respective normalized 1D vectors, normalized in the same manner as described immediately above. The system may then calculate the perturbed real-valued vector V'. For example, the system may calculate the perturbation vector V' as equal to A+mut*(CB).

[0107] The system can then calculate perturbation candidate Vp, for example, by randomly mixing the vector elements of vector V and perturbation vector V'. That is, for each element of perturbation candidate Vp, e.g., Vp[i], the system randomly selects either V[i] or V'[i]. The selection can be weighted by a parameter that can skew the selection, for example, so that the selection is not 50 / 50. The system can multiply perturbation candidate Vp by P*R*T to scale it back to a 1D vector representing the perturbed candidate. To convert from a 1D real-valued vector back to a 2D matrix, the system first rounds the real values ​​to integers. From there, the system can use those integer values ​​to mark up a blank 2D matrix.

[0108] FIG. 10 illustrates a low-level multi-robot DE candidate solution method 1000 of operation of a processor-based system for configuring multiple robots for a multi-robot operating environment in which the robots operate, according to at least one illustrated implementation, detailing the operation of a population generator. The processor-based system may include at least one processor and at least one non-transitory processor-readable medium that stores at least one of data and processor-executable instructions. When executed by the at least one processor, the processor-executable instructions cause the at least one processor to perform various operations or actions of the multi-robot DE candidate solution method 1000. The multi-robot DE candidate solution method 1000 may be executed by an optimization engine that may be implemented by one or more separate processors dedicated to the optimization engine of the multi-robot environment. Alternatively, the multi-robot environment simulator may be implemented by one or more processors that perform other operations.

[0109] The global optimizer may be based on a multivariate mixed integer optimization algorithm, such as the algorithm known as Differential Evolution (DE). Unless an algorithm is explicitly recited in a claim, the claims are not limited to such algorithms.

[0110] The global optimizer optimizes the robot base placement (e.g., in Cartesian coordinates), the robot's functional pose (e.g., in C-space), and the task plan (ordered list or vector of targets and dwell durations) for each robot. The primary optimization objective may be to minimize the amount of wear the robot experiences, but other objectives may include minimizing the risk or probability of collision, minimizing energy usage, optimizing the use of available floor space, minimizing overall time performance or latency, etc.

[0111] The optimizer system may comprise three components: an optimization engine, a multi-robot environment simulator, and a candidate solution generator (e.g., a seed generator).

[0112] As explained in more detail below, the candidate solution generator generates a population of C candidate solutions, and any given candidate may or may not be feasible.

[0113] As explained in more detail below, the optimization engine attempts to find a better candidate solution by, for example, perturbing one of the population of candidate solutions, say candidate P, and seeing if the resulting perturbed candidate P' has a lower cost. If so, perturbed candidate P' replaces candidate P in the population. The total number of candidates C remains unchanged.

[0114] To find the cost of a candidate solution, the multi-robot environment simulator simulates the candidate solutions. A given candidate solution includes an ordered list of tasks for each robot, which is equivalent to an ordered list of trajectories in C-space. Every epoch of time (e.g., 0.1 seconds or some selected input value), the multi-robot environment simulator samples each robot's position in C-space and uses forward kinematics to check whether the robot's virtual representation collides with itself, another robot, or any obstacles or objects in the operating environment. The multi-robot environment simulator determines the amount of wear the robot will experience if operated for each candidate solution. The multi-robot environment simulator determines the collision probability or rate (e.g., the percentage of epochs in which the robot has at least one collision with itself, another robot, or an obstacle or object in the operating environment). Note that generating a population of candidate solutions does not require generating a collision-free motion plan, but rather simply tracks the collision rate. While not requiring a collision-free path significantly speeds up processing, some implementations can instead generate a collision-free path at this stage. The cost of a candidate is a function of at least these two quantities (e.g., the amount of wear and the collision rate or probability). In DE, this cost function is piecewise logarithmic, which can be parameterized.

[0115] Optionally, the multi-robot environment simulator can determine values ​​for other parameters, such as the amount of energy that would be expanded or consumed by the robots if operated for each candidate solution, and / or how long it would take for the candidate solution to be fully implemented.

[0116] The overall workflow involves a cycle in which the optimizer generates candidates and the simulator evaluates their costs, and this process is repeated until some convergence criterion is met, such as the standard deviation of the costs being less than some epsilon value.

[0117] The candidate solution generator 1002 generates candidate solutions by performing various actions or acts.

[0118] At 1004, the candidate solution generator 1002 receives various variable bounds, fixed parameters, task or target goals, and population size. At 1006, the candidate solution generator 1002 generates base positions and orientations for each robot's base. At 1008, the candidate solution generator 1002 virtually places or positions the robot bases in the virtual multi-robot operating environment and provides it to a multi-robot simulator 1009, which models the candidate solution and provides feedback to the candidate solution generator 1002.

[0119] At 1010, the candidate solution generator 1002 generates uniformly random home poses and / or other functional poses for each robot.

[0120] At 1012, for each task or target goal, the candidate solution generator 1002 finds at 1012a a set of robots capable of completing the task or reaching the target goal, randomly selects one of those robots at 1012b, determines at 1012c whether the set is empty (indicating an infeasible task plan), and if the set is empty, returns to 1006 otherwise assigns the task or target goal to the selected robot at 1012d.

[0121] At 1014, for each target robot, the candidate solution generator 1002 determines whether the total number of tasks or target objectives assigned to the robot exceeds the limit or task or target capacity specified for the robot at 1014a, and if over-allocated (indicating an infeasible task plan), returns to 1006; otherwise, at 1014b, generates a random sequence of assigned tasks or target objectives.

[0122] At 1016, the candidate solution generator 1002 generates or defines candidate solutions, which may be vectors or other representations representing the base configuration and orientation of each robot, the home and / or other functional poses of each robot, and the target sequence of each robot. At 1018, the candidate solutions are included in a population of candidate solutions (CS).

[0123] At 1020, the candidate solution generator 1002 determines whether there are enough candidate solutions in the population of candidate solutions. If the number of candidate solutions in the population of candidate solutions is less than a specified number, control can return to 1006, where additional candidate solutions can be generated. Otherwise, control can pass to 1022, where the candidate solution generator 1002 returns the population of candidate solutions to the multi-robot optimization engine 1024 for further optimization.

[0124] Once an initial population of candidate solutions is generated, the population of candidate solutions may be refined, for example, by perturbing the candidate solutions.

[0125] Although the low-level multi-robot DE candidate solution method 1000 is described with respect to an ordered flow, various acts or operations are performed simultaneously or in parallel in many implementations.

[0126] The structures and algorithms described herein, in at least some implementations, can operate without cameras or other perception sensors. In at least some implementations, coordination between robots relies on a geometric model of the robots, the ability of the robots to communicate their respective motion plans, and a geometric model of the shared workspace. In other implementations, vision or other perception may optionally be employed, for example, to avoid humans or other dynamic obstacles that may enter or occupy portions of the shared workspace.

[0127] A wide variety of algorithms are used to solve motion planning problems. Each of these algorithms typically needs to be able to determine whether a given pose of the robot or movement from one pose to another will result in a collision with either the robot itself or an obstacle in the environment. Virtual collision evaluation or checking can be performed "in software," using a processor executing processor-executable instructions from a stored set of processor-executable instructions to execute the algorithm. Virtual collision evaluation or checking can be performed "in hardware," using a set of dedicated hardware circuits (e.g., collision-checking circuitry implemented in a field-programmable gate array (FPGA), application-specific integrated circuit (ASIC)). Such circuits can, for example, represent a volume swept by a robot / robot appendage or portion thereof during each movement or transition between two states (i.e., a swept volume). The circuits may, for example, generate a Boolean evaluation indicating whether a movement will collide with any obstacles, at least some of which represent volumes swept by other robots operating in a shared workspace when performing a movement or transition. [Example]

[0128] Example 1. A method operating in a processor-based system for configuring multiple robots for a multi-robot operating environment in which the multiple robots operate, the method comprising: generating a population of C candidate solutions via a population generator, each candidate solution in the population of C candidate solutions specifying, for each of the robots, a respective base position and orientation, a respective set of at least one defined pose, and a respective target sequence, wherein the respective base positions and orientations specify a respective position and orientation of a base for each robot within the multi-robot operating environment, the respective set of at least one defined pose specifies at least a respective home pose for each robot within the multi-robot operating environment, and the respective target sequences include a respective ordered list of targets for each robot to move to complete a respective sequence of tasks; performing an optimization with respect to the amount of wear that at least the robot experiences for a population of C candidate solutions by an optimization engine that co-optimizes over two or more sets of heterogeneous parameters for two or more of the base positions and orientations of each of the robots, the assignment of tasks to each one of the robots, and the target sequences of each of the robots; providing as output a respective base position and orientation for each of the robots, a respective task assignment for each of the robots, and a respective motion plan for each of the robots.

[0129] Example 2. The method of example 1, wherein providing as output a respective base position and orientation for each of the robots, a respective task assignment for each of the robots, and a respective motion plan for each of the robots includes: providing, for each robot, an optimized task assignment that specifies a respective sequence of tasks to be performed in the form of an optimized ordered list of targets in C-space for each robot and one or more dwell durations at one or more of the targets; and providing an optimized motion plan that specifies a set of collision-free paths, wherein the set of collision-free paths specifies a respective collision-free path between each pair of consecutive targets in the ordered list of targets.

[0130] Example 3. The method of example 1 or 2, further comprising determining, for each of the candidate solutions of the population C of candidate solutions, a respective time to complete the sequence of tasks and a respective collision value representing a rate or probability of a collision occurring in completing the sequence of tasks via modeling performed by a robot environment simulator.

[0131] Example 4. Co-optimizing over two or more sets of heterogeneous parameters. Performing optimization over a population of C candidate solutions with an optimization engine at least with respect to the amount of wear experienced by the robot. 4. The method of example 3, including selecting one of the candidate solutions via the optimization engine based at least in part on respective costs associated with the candidate solutions, the respective costs being based at least in part on an amount of wear the robot will incur to complete the sequence of tasks and a collision value determined for each candidate solution.

[0132] Example 5. Co-optimizing over two or more heterogeneous parameter sets. Performing optimization over a population of C candidate solutions with an optimization engine at least with respect to the amount of wear experienced by the robot. 4. The method of example 3, comprising selecting one of the candidate solutions via an optimization engine based at least in part on respective costs associated with the candidate solutions, the respective costs being based at least in part on each of: i) an amount of wear experienced by the robot in completing the sequence of tasks; ii) a time to complete the sequence of tasks; and iii) a collision value determined for each candidate solution.

[0133] Example 6. Co-optimizing over two or more sets of heterogeneous parameters. Performing optimization over a population of C candidate solutions with an optimization engine at least with respect to the amount of wear experienced by the robot. 4. The method of example 3, comprising selecting one of the candidate solutions via an optimization engine based at least in part on respective costs associated with the candidate solutions, the respective costs being based at least in part on each of: i) an amount of wear experienced by the robot in completing the sequence of tasks; ii) an amount of energy consumed to complete the sequence of tasks; and iii) a collision value determined for each candidate solution.

[0134] Example 7. The method of example 3, wherein performing optimization for a population of C candidate solutions by an optimization engine that co-optimizes across two or more sets of heterogeneous parameters, with respect to at least the amount of wear experienced by the robot, includes selecting one of the candidate solutions via the optimization engine based at least in part on respective costs associated with the candidate solutions, the respective costs being based at least in part on each of: i) the amount of wear experienced by the robot in completing a sequence of tasks; ii) the time to complete the sequence of tasks; iii) the amount of energy consumed to complete the sequence of tasks; and iv) a collision value determined for each candidate solution.

[0135] Example 8. Determining respective times to complete a sequence of tasks and respective collision values ​​representing a rate or probability of a collision occurring through modeling by an optimization engine includes: virtually executing each task of the sequence of tasks via a multi-robot environment simulator; For each epoch of a plurality of epochs, sampling a C-space position of at least a portion of one of the robots via the multi-robot environment simulator; and checking for collisions using forward kinematics to identify potential collisions between one or more parts of each one of the robots and another part of each one of the robots, between each one of the robots and another one of the robots in the environment, and between each one of the robots and another object in the multi-robot operating environment that is not another robot.

[0136] Example 9. The method of Example 8, wherein the ordered list of tasks is equivalent to an ordered list of trajectories in the robot's C-space, and includes a plurality of trajectories and one or more dwell durations in one or more postures, a home posture, and one or more other defined functional postures in the robot's C-space, and wherein virtually executing includes virtually executing one or more of the plurality of trajectories and one or more dwell durations in the one or more postures, the home posture, or one or more other defined functional postures.

[0137] Example 10. For each of several candidate solutions in a population of C candidate solutions, for at least one iteration: perturbing each candidate solution to generate a perturbed candidate solution; Modeling the perturbed candidate solution; determining whether the perturbed candidate solution has a lower associated cost than the respective candidate solution; 3. The method of claim 1 or 2, further comprising: in response to determining that the perturbed candidate solution has a lower associated cost than the respective candidate solution, replacing each candidate solution in the population of C candidate solutions with the perturbed candidate solution.

[0138] Example 11. The method of example 10, wherein the perturbing, modeling, determining, and replacing are repeated for multiple iterations until convergence occurs, an iteration limit, or an iteration time limit is reached.

[0139] Example 12. The method of example 10, wherein perturbing each candidate solution to generate a perturbed candidate solution includes perturbing a candidate solution vector, the candidate solution vector including a plurality of real vector elements, one real vector element for each task, the real vector element representing a respective combination of a respective one of the tasks, a priority for each one of the tasks, and one of the robots identified to perform each one of the tasks.

[0140] Example 13. The method of example 1 or 2, further comprising receiving input including at least one model of the multi-robot operating environment, a respective model of each of at least two of the robots operating in the multi-robot operating environment, a position, velocity, acceleration, jerk, or torque of the at least one robot as it performs a movement, and at least one model of wear for at least one of the robots representing a respective amount of wear experienced by the at least one robot with respect to one or more of the set of tasks.

[0141] Example 14. The method of example 1 or 2, further comprising receiving inputs including at least one model of the multi-robot operating environment, respective models of each of at least two of the robots operating within the multi-robot operating environment, and at least one model of wear of at least one of the robots representing a respective amount of wear experienced by the at least one robot with respect to one or more of: a position, velocity, acceleration, jerk, or torque of the at least one robot when performing a movement, a set of tasks, and at least one of a set of time intervals specifying one or more dwell durations for dwelling on one or more targets while at least one of the robots performs the at least one task, a set of boundaries or constraints on variables, or a time limit for simulating a collision.

[0142] Example 15. The method of example 1 or 2, wherein the population generator is a pseudo-random population generator, and generating the population of C candidate solutions via the population seed generator includes pseudo-randomly generating the population of C candidate solutions via the pseudo-random population generator.

[0143] Example 16. The method of example 1 or 2, wherein generating a population of C candidate solutions via a population generator includes generating a population of C candidate solutions that have a lower probability of being an invalid candidate solution than a population of C candidate solutions that are purely pseudo-randomly generated.

[0144] Example 17. The method described in Example 1 or 2, wherein performing optimization for a population of C candidate solutions with respect to at least the amount of wear experienced by the robot by an optimization engine that co-optimizes across a set of two or more heterogeneous parameters includes selecting optimized candidate solutions having co-optimized combinations of respective optimized base positions and orientations, optimized task assignments, and optimized motion plans for each respective base of each of the robots.

[0145] Example 18. The method of any one of Examples 1-17, further comprising configuring the robots based at least in part on one of a respective base position and orientation of each of the robots, a respective task assignment of each of the robots, and a respective motion plan of each of the robots specified by the output.

[0146] Example 19. A processor-based system for configuring multiple robots for a multi-robot operating environment in which the multiple robots operate, the processor-based system comprising: at least one processor; At least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions, the processor-executable instructions, when executed by the at least one processor, causing the processor to perform a method according to any one of Examples 1 to 18; and A processor-based system comprising:

[0147] Example 20. A processor-based system for configuring multiple robots for a multi-robot operating environment in which the multiple robots operate, the processor-based system comprising: at least one processor; at least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions; the processor-executable instructions, when executed by at least one processor, cause the processor to: generating a population of C candidate solutions via a population generator, each candidate solution in the population of C candidate solutions specifying, for each of the robots, a respective base position and orientation, a respective set of at least one defined pose, and a respective target sequence, wherein the respective base positions and orientations specify a respective position and orientation of a base for each robot within the multi-robot operating environment, the respective set of at least one defined pose specifies at least a respective home pose for each robot within the multi-robot operating environment, and the respective target sequences include a respective ordered list of targets for each robot to move to complete a respective sequence of tasks; performing an optimization with respect to the amount of wear that at least the robot experiences for a population of C candidate solutions by an optimization engine that co-optimizes over two or more sets of heterogeneous parameters for two or more of the base positions and orientations of each of the robots, the assignment of tasks to each one of the robots, and the target sequences of each of the robots; and providing as outputs a respective base position and orientation for each of the robots, a respective task assignment for each of the robots, and a respective motion plan for each of the robots.

[0148] Example 21. The processor-based system of example 20, wherein the processor-executable instructions, when executed by at least one processor, cause the processor to provide as output, for each robot, an optimized task assignment that specifies a respective sequence of tasks to be performed in the form of an optimized ordered list of targets in C-space for the respective robot and one or more dwell durations at one or more of the targets, and an optimized motion plan that specifies a set of collision-free paths, the set of collision-free paths specifying a respective collision-free path between each pair of consecutive targets in the ordered list of targets.

[0149] Example 22. A processor-based system as described in Example 20 or 21, wherein the processor-executable instructions, when executed by at least one processor, further cause the processor to determine, for each of the candidate solutions of the population C of candidate solutions, a respective time to complete the sequence of tasks and a respective collision value representing the rate or probability of a collision occurring when completing the sequence of tasks through modeling performed by the robot environment simulator.

[0150] Example 23. To perform optimization with an optimization engine that co-optimizes over two or more sets of heterogeneous parameters for a population of C candidate solutions with respect to at least the amount of wear experienced by a robot, the processor-executable instructions, when executed by at least one processor, cause the processor to: 23. The processor-based system of example 22, wherein one of the candidate solutions is selected via an optimization engine based at least in part on a respective cost associated with the candidate solution, the respective costs being based at least in part on an amount of wear the robot will incur to complete the sequence of tasks and a collision value determined for each candidate solution.

[0151] Example 24. To perform optimization with respect to at least the amount of wear experienced by a robot for a population of C candidate solutions by an optimization engine that co-optimizes over two or more sets of heterogeneous parameters, the processor-executable instructions, when executed by at least one processor, may cause the processor to: 23. The processor-based system of claim 22, wherein one of the candidate solutions is selected via an optimization engine based at least in part on respective costs associated with the candidate solutions, the respective costs being based at least in part on: i) the amount of wear the robot will incur to complete the sequence of tasks; ii) the time to complete the sequence of tasks; and iii) a collision value determined for each candidate solution.

[0152] Example 25. To perform optimization with an optimization engine that co-optimizes over two or more sets of heterogeneous parameters for a population of C candidate solutions with respect to at least the amount of wear experienced by a robot, the processor-executable instructions, when executed by at least one processor, cause the processor to: 23. The processor-based system of claim 22, wherein one of the candidate solutions is selected via an optimization engine based at least in part on respective costs associated with the candidate solutions, the respective costs being based at least in part on: i) the amount of wear the robot will incur to complete the sequence of tasks; ii) the amount of energy consumed to complete the sequence of tasks; and iii) a collision value determined for each candidate solution.

[0153] Example 26. To perform optimization with respect to at least the amount of wear experienced by a robot for a population of C candidate solutions by an optimization engine that co-optimizes over two or more sets of heterogeneous parameters, the processor-executable instructions, when executed by at least one processor, may cause the processor to: 23. The processor-based system of claim 22, wherein one of the candidate solutions is selected via an optimization engine based at least in part on respective costs associated with the candidate solution, the respective costs being based at least in part on each of: i) the amount of wear the robot will incur to complete the sequence of tasks; ii) the time to complete the sequence of tasks; iii) the amount of energy consumed to complete the sequence of tasks; and iv) a collision value determined for each candidate solution.

[0154] Example 27. For determining respective times for completing a sequence of tasks and respective collision values ​​representing a rate or probability of a collision occurring through modeling by an optimization engine, processor-executable instructions, when executed by at least one processor, may cause the processor to: virtually executing each task of the sequence of tasks via a multi-robot environment simulator; For each epoch of a plurality of epochs, sampling a C-space position of at least a portion of one of the robots via the multi-robot environment simulator; 27. The processor-based system of any one of Examples 22 to 26, wherein the processor-based system uses forward kinematics to check for collisions to identify potential collisions between one or more parts of each one of the robots and another part of each one of the robots, between each one of the robots and another one of the robots in the environment, and between each one of the robots and another object in the multi-robot operating environment that is not another robot.

[0155] Example 28. A processor-based system as described in Example 27, wherein the ordered list of tasks is equivalent to an ordered list of trajectories in the robot's C-space, and includes multiple trajectories and one or more dwell durations in one or more postures, a home posture, and one or more other defined functional postures in the robot's C-space, and to virtually perform each task, the processor-executable instructions, when executed by at least one processor, cause the processor to virtually perform one or more of the multiple trajectories and one or more dwell durations in the one or more postures, the home posture, or one or more other defined functional postures.

[0156] Example 29. The processor-executable instructions, when executed by at least one processor, cause the processor to: For each of some candidate solutions in the population of C candidate solutions, for at least one iteration, perturbing each candidate solution to generate a perturbed candidate solution; Modeling the perturbed candidate solution; determining whether the perturbed candidate solution has a lower associated cost than the respective candidate solution; 22. The processor-based system of claim 20 or 21, further comprising: replacing each candidate solution in the population of C candidate solutions with the perturbed candidate solution in response to determining that the perturbed candidate solution has a lower associated cost than each candidate solution.

[0157] Example 30. The processor-based system of Example 29, wherein the processor-executable instructions, when executed by at least one processor, cause the processor to further repeat the perturbing, modeling, determining, and replacing for multiple iterations until convergence occurs, an iteration count limit, or an iteration time limit is reached.

[0158] Example 31. The processor-based system of example 29, wherein the processor-executable instructions, when executed by at least one processor, cause the processor to perturb a candidate solution vector, the candidate solution vector including a plurality of real vector elements, one real vector element for each task, to perturb each candidate solution to generate a perturbed candidate solution, the candidate solution vector representing a respective combination of a priority for each one of the tasks and one of the robots identified to perform each one of the tasks.

[0159] Example 32. The processor-executable instructions, when executed by at least one processor, cause the processor to: 22. The processor-based system of claim 20 or 21, further receiving inputs including at least one model of the multi-robot operating environment, a respective model of each of at least two of the robots operating in the multi-robot operating environment, a position, velocity, acceleration, jerk, or torque of the at least one robot as it performs a movement, and at least one model of wear for at least one of the robots representing a respective amount of wear experienced by the at least one robot with respect to one or more of the set of tasks.

[0160] Example 33. The processor-executable instructions, when executed by at least one processor, cause the processor to: 22. The processor-based system of claim 20 or 21, further receiving inputs including at least one model of the multi-robot operating environment, respective models of each of at least two of the robots operating within the multi-robot operating environment, and at least one model of wear of at least one of the robots representing a respective amount of wear experienced by the at least one robot with respect to one or more of: a position, velocity, acceleration, jerk, or torque of the at least one robot when performing a movement, a set of tasks, and at least one of a set of time intervals specifying one or more dwell durations for dwelling on one or more targets while at least one of the robots performs the at least one task, a set of boundaries or constraints on variables, or a time limit for simulating a collision.

[0161] Example 34. The processor-based system of Example 20 or 21, wherein the population generator is a pseudo-random population generator, and wherein generating the population of C candidate solutions via the population seed generator includes pseudo-randomly generating the population of C candidate solutions via the pseudo-random population generator.

[0162] Example 35. The processor-based system of example 20 or 21, wherein generating a population of C candidate solutions via a population generator includes generating a population of C candidate solutions that have a lower probability of being an invalid candidate solution than a population of C candidate solutions that are purely pseudo-randomly generated.

[0163] Example 36: A processor-based system as described in Example 20 or 21, wherein the processor-executable instructions, when executed by at least one processor, for performing optimization on a population of C candidate solutions by an optimization engine that co-optimizes across two or more sets of heterogeneous parameters, cause the processor to select an optimized candidate solution involving a co-optimized combination of a respective optimized base position and orientation, an optimized task assignment, and an optimized motion plan for each respective base of each of the robots.

[0164] Example 37. A processor-based system described in any one of Examples 20 to 36, wherein the processor-executable instructions, when executed by at least one processor, cause the processor to configure the robots based on at least one of the respective base positions and orientations of each of the robots, the respective task assignments of each of the robots, and the respective motion plans of each of the robots specified by the output.

[0165] The foregoing detailed description describes various embodiments of devices and / or processes through the use of block diagrams, schematics, and examples. To the extent that such block diagrams, schematics, and examples include one or more functions and / or operations, those skilled in the art will appreciate that each function and / or operation within such block diagrams, flowcharts, or examples may be individually and / or collectively implemented by a wide range of hardware, software, firmware, or virtually any combination thereof. In one embodiment, the subject matter may be implemented via Boolean circuits, application-specific integrated circuits (ASICs), and / or FPGAs. However, those skilled in the art will recognize that the embodiments disclosed herein may be implemented, in whole or in part, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more controllers (e.g., microcontrollers), as one or more programs running on one or more processors (e.g., microprocessors), as firmware, or as virtually any combination thereof, in a variety of different implementations in standard integrated circuits, and that designing circuitry and / or writing code for the software and / or firmware is well within the skill of those skilled in the art in light of this disclosure.

[0166] Those skilled in the art will recognize that many of the methods or algorithms described herein may employ additional operations, omit some operations, and / or perform operations in an order other than that specified.

[0167] Additionally, those skilled in the art will appreciate that the mechanisms taught herein can be implemented in hardware, for example, in one or more FPGAs or ASICs.

[0168] The various embodiments described above can be combined to provide further embodiments. International Patent Application No. PCT / US2017 / 036880, filed June 9, 2017, entitled "MOTION PLANNING FOR AUTONOMOUS VEHICLES AND RECONFIGURABLE MOTION PLANNING PROCESSORS," International Patent Application Publication No. WO2016 / 122840, filed January 5, 2016, entitled "SPECIALIZED ROBOT MOTION PLANNING HARDWARE AND METHODS OF MAKING AND USING SAME," U.S. Patent Application No. 62 / 616,783, filed January 12, 2018, entitled "APPARATUS, METHOD AND ARTICLE TO FACILITATE MOTION PLANNING OF AN AUTONOMOUS VEHICLE IN AN ENVIRONMENT HAVING DYNAMIC OBJECTS," and U.S. Patent Application No. 62 / 616,783, filed February 6, 2018, entitled "MOTION PLANNING OF A U.S. patent application Ser. No. 62 / 626,939, entitled "ROBOT STORING A DISCRETIZED ENVIRONMENT ON ONE OR MORE PROCESSORS AND IMPROVED OPERATION OF SAME," filed June 3, 2019; U.S. patent application Ser. No. 62 / 856,548, entitled "APPARATUS, METHODS AND ARTICLES TO FACILITATE MOTION PLANNING IN ENVIRONMENTS HAVING DYNAMIC OBSTACLES," filed June 24, 2019; U.S. patent application Ser. No. 62 / 865,431, entitled "MOTION PLANNING FOR MULTIPLE ROBOTS IN SHARED WORKSPACE," filed June 24, 2019; U.S. patent application Ser. No. 62 / 964, entitled "CONFIGURATION OF ROBOTS IN MULTI-ROBOT OPERATIONAL ENVIRONMENT," filed January 22, 2020;All commonly-owned U.S. patent application publications, U.S. patent applications, foreign patents, and foreign patent applications referenced herein and / or listed in Application Data Sheets, including, but not limited to, US Pat. No. 405, and International Patent Application No. PCT / US2021 / 013610, published as WO2021 / 150439A1, are incorporated herein by reference in their entirety. These and other changes can be made to the embodiments in light of the above detailed description. Generally, in the following claims, the terms used should not be construed to limit the claims to the specific embodiments disclosed in the specification and claims, but rather to include all possible embodiments along with the full scope of equivalents to which such claims are entitled. Accordingly, the claims are not limited by the present disclosure.

Claims

1. A method for operating a processor-based system for configuring multiple robots for a multi-robot operating environment in which multiple robots operate, wherein the method is The method involves generating a set of C candidate solutions via a set generator, wherein each of the candidate solutions in the set of C candidate solutions specifies, for each of the robots, its respective base position and orientation, each set of at least one defined pose, and each target sequence, wherein the respective base position and orientation specify the respective base position and orientation of each robot in the multi-robot operating environment, each set of at least one defined pose specifies at least each home pose of each robot in the multi-robot operating environment, and each target sequence includes each ordered list of targets to which each robot moves to complete each sequence of tasks. An optimization engine co-optimizes the respective base positions and orientations of the robots, the assignment of the tasks to each of the robots, and two or more sets of heterogeneous parameters for two or more of the respective target sequences of the robots, thereby performing optimization with respect to at least the amount of wear the robots receive with respect to the set of C candidate solutions, A method comprising providing, as output, the respective base position and orientation for each of the robots, the respective task assignment for each of the robots, and the respective motion plan for each of the robots.

2. The method according to claim 1, wherein the output includes providing the respective base position and orientation for each of the robots, the respective task assignment for each of the robots, and the respective motion plan for each of the robots, the optimized task assignment for each of the robots specifies a sequence of tasks to be performed in the form of an optimized ordered list of targets in the C-space of each of the robots and one or more dwell durations on one or more of the targets, and the optimized motion plan specifies a set of collision-free paths, the set of collision-free paths specifying a collision-free path between each pair of consecutive targets in the ordered list of targets.

3. The method according to claim 1, further comprising determining, for each of the candidate solutions in the set C of candidate solutions, the respective time to complete the sequence of tasks and the respective collision value representing the rate or probability of collisions occurring when completing the sequence of tasks through modeling performed by a robot environment simulator.

4. Performing optimization on the set of C candidate solutions with respect to at least the amount of wear the robot experiences by an optimization engine that co-optimizes across two or more heterogeneous sets of parameters is: The method according to claim 3, comprising selecting one of the candidate solutions via the optimization engine based at least in part on the cost associated with each of the candidate solutions, wherein the cost of each is based at least in part on the amount of wear the robot receives when completing the sequence of tasks and the collision value determined for each of the candidate solutions.

5. Performing optimization on the set of C candidate solutions with respect to at least the amount of wear the robot experiences by an optimization engine that co-optimizes across two or more heterogeneous sets of parameters is: The method according to claim 3, comprising selecting one of the candidate solutions via the optimization engine based at least in part on the cost associated with each of the candidate solutions, wherein the cost of each is based at least in part on i) the amount of wear the robot will suffer as it completes the sequence of tasks, ii) the time required to complete the sequence of tasks, and iii) the respective collision value determined for each of the candidate solutions.

6. Performing optimization on the set of C candidate solutions with respect to at least the amount of wear the robot experiences by an optimization engine that co-optimizes across two or more heterogeneous sets of parameters is: The method according to claim 3, comprising selecting one of the candidate solutions via the optimization engine based at least in part on the cost associated with each of the candidate solutions, wherein the cost of each is based at least in part on i) the amount of wear the robot will experience in completing the sequence of tasks, ii) the amount of energy consumed to complete the sequence of tasks, and iii) the respective collision value determined for each of the candidate solutions.

7. Performing optimization on the set of C candidate solutions with respect to at least the amount of wear the robot experiences by an optimization engine that co-optimizes across two or more heterogeneous sets of parameters is: The method according to claim 3, comprising selecting one of the candidate solutions via the optimization engine based at least in part on the cost associated with each of the candidate solutions, wherein the cost of each is based at least in part on i) the amount of wear the robot will suffer as it completes the sequence of tasks, ii) the time to complete the sequence of tasks, iii) the amount of energy consumed to complete the sequence of tasks, and iv) the respective collision value determined for each of the candidate solutions.

8. Determining the time required to complete each of the aforementioned sequences of tasks, and the respective collision values ​​representing the rate or probability of collisions occurring through modeling by the optimization engine, Virtually executing each task in the sequence of tasks via a multi-robot environment simulator, For each epoch of multiple epochs, the C-space position of at least one portion of the robots is sampled via the multi-robot environment simulator, The method according to any one of claims 3 to 7, comprising checking for collisions using forward kinematics to identify potential collisions between one or more parts of each of the robots and one other part of each of the robots, between one of the robots and another of the robots in the environment, and between one of the robots and another object in the multi-robot operating environment that is not another robot.

9. The method of claim 8, wherein the ordered list of tasks is equivalent to the ordered list of trajectories in the C-space of the robot, and includes a plurality of trajectories and one or more dwell durations in one or more attitudes, home attitudes, and one or more other defined functional attitudes of the robot, and virtual execution includes virtual execution of the plurality of trajectories and one or more of the one or more dwell durations in one or more attitudes, home attitudes, or one or more other defined functional attitudes.

10. For each of the several candidate solutions within the set of C candidate solutions, for at least one iteration, Perturbing each of the aforementioned candidate solutions to generate perturbed candidate solutions, Modeling the perturbed candidate solution, The perturbed candidate solution is to determine whether it has a lower associated cost than each of the candidate solutions, The method according to claim 1 or 2, further comprising replacing each of the candidate solutions in the set of C candidate solutions with the perturbed candidate solution in response to a determination that the perturbed candidate solution has a lower associated cost than each of the candidate solutions.

11. The method according to claim 10, wherein the perturbing, modeling, determining, and replacing are repeated for multiple iterations until convergence occurs, the number of iterations is limited, or the iteration time is limited.

12. The method according to claim 10, wherein perturbing each of the candidate solutions to generate a perturbed candidate solution includes perturbing a candidate solution vector, the candidate solution vector comprising a plurality of real vector elements, one real vector element for each task, the real vector element representing each combination of one of the respective tasks, a priority for one of the respective tasks, and one of the robots identified to perform one of the respective tasks.

13. The method according to claim 1 or 2, further comprising receiving an input comprising at least one model of the multi-robot operating environment, at least two models of each of the respective robots operating in the multi-robot operating environment, and at least one model of wear for at least one of the robots, representing the position, velocity, acceleration, jerk, or torque of the at least one robot when performing motion, and one or more of a set of tasks.

14. The method according to claim 1 or 2, further comprising receiving an input comprising: at least one model of the multi-robot operating environment; each model of at least two of the robots operating within the multi-robot operating environment; and at least one model of wear of at least one of the robots, representing the respective wear experienced by the at least one robot with respect to one or more of the following: position, velocity, acceleration, jerk, or torque of the at least one robot when performing motion; a set of tasks; and one or more dwell durations for the at least one of the robots to dwell on one or more targets while performing at least one task; a set of boundaries or constraints on variables; or a set of time intervals specifying a time limit for simulating a collision.

15. The method according to claim 1 or 2, wherein the group generator is a pseudo-random group generator, and generating a group of C candidate solutions via a group seed generator includes generating the group of C candidate solutions pseudo-randomly via the pseudo-random group generator.

16. The method according to claim 1 or 2, wherein generating the set of C candidate solutions via a set generator comprises generating a set of C candidate solutions that has a lower probability of being invalid than a set of C candidate solutions generated purely pseudo-randomly.

17. The method according to claim 1 or 2, wherein an optimization engine that co-optimizes over two or more sets of heterogeneous parameters performs optimization on the set of C candidate solutions with respect to at least the amount of wear the robot receives, the optimization of an optimized candidate solution having a co-optimized combination of an optimized base position and orientation for each base of the robot, an optimized task assignment, and an optimized motion plan.

18. The method according to any one of claims 1 to 7, further comprising configuring the robots based at least in part on one of the respective base positions and orientations of each robot, the respective task assignments of each robot, and the respective motion plans of each robot specified by the outputs.

19. A processor-based system for configuring multiple robots for a multi-robot operating environment in which multiple robots operate, wherein the processor-based system is At least one processor, The system comprises at least one non-temporary processor-readable medium storing data and at least one of processor-executable instructions, wherein when the processor-executable instruction is executed by the at least one processor, the processor receives The method involves generating a set of C candidate solutions via a set generator, wherein each of the candidate solutions in the set of C candidate solutions specifies, for each of the robots, its respective base position and orientation, each set of at least one defined pose, and each target sequence, wherein the respective base position and orientation specify the respective base position and orientation of each robot in the multi-robot operating environment, each set of at least one defined pose specifies at least each home pose of each robot in the multi-robot operating environment, and each target sequence includes each ordered list of targets to which each robot moves to complete each sequence of tasks. An optimization engine co-optimizes the respective base positions and orientations of the robots, the assignment of the tasks to each of the robots, and two or more sets of heterogeneous parameters for two or more of the respective target sequences of the robots, thereby performing optimization with respect to at least the amount of wear the robots receive with respect to the set of C candidate solutions, A processor-based system that provides, as output, the respective base position and orientation for each of the robots, the respective task assignment for each of the robots, and the respective motion plan for each of the robots.

20. The processor-based system according to claim 19, wherein the processor-executable instruction, when executed by the at least one processor, causes the processor to provide, as output, an optimized task assignment specifying, for each robot, an optimized ordered list of targets in the C-space of each robot and a sequence of tasks to be performed in the form of one or more dwell durations on one or more of the targets, and an optimized motion plan specifying a set of collision-free paths, the set of collision-free paths specifying a collision-free path between each pair of consecutive targets in the ordered list of targets.

21. The processor-based system according to claim 19, wherein the processor-executable instruction, when executed by the at least one processor, causes the processor to further determine, for each of the candidate solutions of the set C of candidate solutions, the respective time to complete the sequence of tasks and the respective collision value representing the rate or probability of collisions occurring when completing the sequence of tasks through modeling performed by the robot environment simulator.

22. To perform optimization on the set of C candidate solutions with respect to at least the amount of wear the robot receives, by an optimization engine that co-optimizes across two or more sets of heterogeneous parameters, the processor executable instructions, when executed by at least one processor, cause the processor to: The processor-based system according to claim 21, wherein the optimization engine is used to select one of the candidate solutions, based at least in part on the cost associated with each of the candidate solutions, the cost of each of the candidate solutions being at least in part on the amount of wear the robot will experience when completing the sequence of tasks and the collision value determined for each of the candidate solutions.

23. To perform optimization on the set of C candidate solutions with respect to at least the amount of wear the robot receives, by an optimization engine that co-optimizes across two or more sets of heterogeneous parameters, the processor executable instructions, when executed by at least one processor, cause the processor to: The processor-based system according to claim 21, wherein the optimization engine is used to select one of the candidate solutions, based at least in part on the cost associated with each of the candidate solutions, the cost of each of the candidate solutions being at least in part on i) the amount of wear the robot will wear as it completes the sequence of tasks, ii) the time required to complete the sequence of tasks, and iii) the collision value determined for each of the candidate solutions.

24. To perform optimization on the set of C candidate solutions with respect to at least the amount of wear the robot receives, by an optimization engine that co-optimizes across two or more sets of heterogeneous parameters, the processor executable instructions, when executed by at least one processor, cause the processor to: The processor-based system according to claim 21, wherein the optimization engine is used to select one of the candidate solutions, based at least in part on the cost associated with each of the candidate solutions, the cost of each of the candidate solutions being at least in part on i) the amount of wear the robot will wear as it completes the sequence of tasks, ii) the amount of energy consumed to complete the sequence of tasks, and iii) the collision value determined for each of the candidate solutions.

25. To perform optimization on the set of C candidate solutions with respect to at least the amount of wear the robot receives, by an optimization engine that co-optimizes across two or more sets of heterogeneous parameters, the processor executable instructions, when executed by at least one processor, cause the processor to: The processor-based system according to claim 21, wherein the optimization engine is used to select one of the candidate solutions, based at least in part on the cost associated with each of the candidate solutions, the cost of each of the candidate solutions being at least in part on i) the amount of wear the robot will wear as it completes the sequence of tasks, ii) the time required to complete the sequence of tasks, iii) the amount of energy consumed to complete the sequence of tasks, and iv) the respective collision value determined for each of the candidate solutions.

26. To determine the time required to complete each sequence of tasks and the respective collision values ​​representing the rate or probability of collisions occurring through modeling by the optimization engine, the processor executable instructions, when executed by the at least one processor, cause the processor to: Virtually executing each task in the sequence of tasks via a multi-robot environment simulator, For each epoch of multiple epochs, the C-space position of at least one portion of the robots is sampled via the multi-robot environment simulator, A processor-based system according to any one of claims 21 to 25, which uses forward kinematics to check for collisions and to identify potential collisions between one or more parts of each of the robots and each other part of each of the robots, between each of the robots and another of the robots in the environment, and between each of the robots and another object in the multi-robot operating environment that is not another robot.

27. The processor-based system according to claim 26, wherein the ordered list of tasks is equivalent to the ordered list of trajectories in the C-space of the robot, and includes a plurality of trajectories and one or more dwell durations in one or more attitudes, home attitudes, and one or more other defined functional attitudes of the robot, and in order to virtually execute each task, the processor-executable instruction, when executed by the at least one processor, causes the processor to virtually execute one or more of the plurality of trajectories and one or more dwell durations in one or more attitudes, home attitudes, or one or more other defined functional attitudes.

28. When the aforementioned processor-executable instruction is executed by the at least one processor, the processor will: For each of the several candidate solutions within the set of C candidate solutions, for at least one iteration, Perturbing each of the aforementioned candidate solutions to generate perturbed candidate solutions, Modeling the perturbed candidate solution, The perturbed candidate solution is to determine whether it has a lower associated cost than each of the candidate solutions, The processor-based system according to claim 19 or 20, further comprising: replacing each of the C candidate solutions in the set of C candidate solutions with the perturbed candidate solution in response to a determination that the perturbed candidate solution has a lower association cost than each of the respective candidate solutions.

29. The processor-based system according to claim 28, wherein, once the processor-executable instruction is executed by the at least one processor, the processor causes the processor to further repeat the perturbing, modeling, determining, and replacing for a plurality of iterations until convergence occurs, a limit on the number of iterations is reached, or a limit on the iteration time is reached.

30. The processor-based system according to claim 28, in order to perturb each of the aforementioned candidate solutions and generate a perturbed candidate solution, the processor-executable instruction, when executed by the at least one processor, causes the processor to perturb a candidate solution vector, the candidate solution vector comprising a plurality of real vector elements, one real vector element for each task, the real vector element representing each combination of one of the tasks, a priority for one of the tasks, and one of the robots identified to perform one of the tasks.

31. When the aforementioned processor-executable instruction is executed by the at least one processor, the processor will: The processor-based system according to claim 19 or 20, further receiving inputs including at least one model of the multi-robot operating environment, at least two models of each of the robots operating in the multi-robot operating environment, and at least one model of wear for at least one of the robots, representing the position, velocity, acceleration, jerk, or torque of the at least one robot when performing motion, and one or more of a set of tasks.

32. When the aforementioned processor-executable instruction is executed by the at least one processor, the processor will: A processor-based system according to claim 19 or 20, further receiving an input comprising: at least one model of the multi-robot operating environment; each model of at least two of the respective robots operating within the multi-robot operating environment; and at least one model of wear of at least one of the robots, representing the respective wear experienced by the at least one robot with respect to one or more of the position, velocity, acceleration, jerk, or torque of the at least one robot when performing motion; a set of tasks; and one or more dwell durations for the at least one of the robots to dwell on one or more targets while performing at least one task; a set of boundaries or constraints on variables; or a set of time intervals specifying a time limit for simulating a collision.

33. The processor-based system according to claim 19 or 20, wherein the ensemble generator is a pseudo-random ensemble generator, and the generation of a ensemble of C candidate solutions via a ensemble seed generator includes the pseudo-random generation of the ensemble of C candidate solutions via the pseudo-random ensemble generator.

34. The processor-based system according to claim 20 or 21, wherein the generation of the set of C candidate solutions via a set generator includes the generation of a set of C candidate solutions that has a lower probability of being invalid than a set of C candidate solutions generated purely pseudo-randomly.

35. A processor-based system according to claim 19 or 20, wherein, in order to perform optimization on the set of C candidate solutions by an optimization engine that co-optimizes over two or more sets of heterogeneous parameters, the processor-executable instruction, when executed by the at least one processor, causes the processor to select an optimized candidate solution having a co-optimized combination of each optimized base position and orientation with respect to the base of each robot, an optimized task assignment, and an optimized motion plan.

36. When the aforementioned processor-executable instruction is executed by the at least one processor, the processor will: A processor-based system according to any one of claims 19 to 25, wherein the robot is configured based on at least one of the respective base position and orientation of each robot, the respective task assignment of each robot, and the respective motion plan of each robot specified by the output.

Images