Robot configuration in multi-robot operating environments

Abstract

In a multi-robot operational environment, solutions for multi-robot configuration are based, at least to some extent, on a given set of tasks to be performed by the robots to collaboratively optimize across a set of non-homogenous parameters. The non-homogenous parameters can include two or more of: respective base positions and orientations of the robots, task assignments to respective robots, respective target sequences and / or trajectories for the robots. This can be performed in advance of execution. For each robot, the output can include: a work cell layout, an ordered list or vector of targets, optionally a dwell duration at respective targets, and a path or trajectory between each pair of consecutive targets. The output can provide a complete, executable solution for the problem, which can be used to control the robots without any modification in the absence of timing variability. Genetic algorithms (e.g., differential evolution) can optionally be used to generate a population of candidate solutions.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to robot configuration in a multi-robot operating environment or shared workspace, and to optimization of robot configuration in such an environment or shared workspace. BACKGROUND

[0002] Related Art

[0003] Various applications employ or can desire to employ two or more robots in a multi-robot operating environment that is common or otherwise shared by the two or more robots. The multi-robot operating environment can take the form of a shared workspace. For example, two or more robots can be used to perform a task on one or more objects or workpieces in a common operating environment, such as screwing bolts to a chassis, where the motion range of the robots can overlap.

[0004] Motion planning is a fundamental problem in robot control and robotics. Motion planning specifies a path that a robot can follow from a start state to a goal state, typically to accomplish a task without colliding with any obstacles in the operating environment, or with a reduced likelihood of colliding with any obstacles in the operating environment. Challenges in motion planning involve the ability to perform motion planning quickly, while possibly taking into account changes in the environment (e.g., changing the position or orientation of obstacles in the environment). Challenges also include performing motion planning using relatively low-cost devices, with relatively low energy consumption and limited storage (e.g., memory circuitry (e.g., on a processor chip circuitry)).

[0005] Operation of two or more robots in a multi-robot operating environment or shared workspace (the workspace is often referred to as a workcell) presents a class of particular problems. For example, motion planning should take into account and avoid situations where the robots or robot appendages of the robots can interfere with each other during performance of a task.

[0006] One approach to operating multiple robots in a common workspace can be referred to as a task-level approach. By defining portions of the workspace in which the robots can collide with each other, referred to herein as interference zones, and programming the individual robots so that only one robot is in an interference zone of the workspace at any given point in time, an engineer can manually ensure that the robots do not collide. For example, when a first robot begins to move into an interference zone of the workspace, the first robot sets a flag. A controller (e.g., a programmable logic controller (PLC)) reads the flag and prevents other robots from moving into the interference zone of the workspace until the first robot cancels the flag when it exits the interference zone. This approach is intuitive and easy to understand, but is often difficult and time-consuming to implement, and can not produce optimal results. This approach necessarily has a low work throughput, because using task-level collision avoidance often results in at least one robot being idle for a significant amount of time, even though it would be technically possible for the idle robot to perform useful work in the shared workspace.

[0007] In a traditional approach, a team of engineers typically decomposes the problem and optimizes smaller sub-problems independently of each other (e.g., assigning tasks to robots, ordering the tasks assigned to each robot, motion planning for each robot). This can employ iterative simulated motion to ensure that the robots / robot attachments do not collide with each other, which can require hours of computation time and can not produce an optimal solution. Additionally, if a modification to the workspace causes a change in the trajectory of one of the robots / robot attachments, the entire workflow must be re-validated. Such an approach is certainly not optimal, and often requires experts to repeatedly try combinations of solutions that, when put together, produce good results, through a slow process. SUMMARY

[0008] Various methods and apparatuses are described herein that produce solutions for multi-robot configurations that are based on a collaborative optimization across a set of non-homogeneous parameters of a given set of tasks performed by the robots in a multi-robot operating environment, such as a combination of collaborative optimization of workcell layout and task planning. This can be performed in an offline or pre-operational environment, providing a global optimizer for these types of problems. The inventors are not aware of any existing global optimizer solutions for such problems.

[0009] The inputs can include a model of the multi-robot operating environment, a model of the robots, a limit on the total number of robots that can be employed, a set of tasks to be completed by the robots, and a limit on the total number of tasks that can be assigned to each robot (i.e., the target quantity of robots). The inputs can also optionally include one or more dwell durations in which a robot or a portion thereof stays at a target, e.g., to complete a task (e.g., to tighten a bolt or nut) or to avoid a collision. The inputs can also optionally include one or more of the following: a set of bounds or constraints on one or more parameters or variables or a time limit that limits the time provided for modeling or simulating collisions.

[0010] The output can include a complete solution to the problem that can have been advantageously optimized. In particular, the solution can have been optimized by an optimization engine that performs optimization on the population of C candidate solutions in coordination across two or more non-homogeneous parameters of the set. The non-homogeneous parameters can include, for example, two or more of the following: respective base positions and orientations of the robots, task assignments to respective ones of the robots, and respective target sequences of the robots. In the absence of any timing variability, the output can be used to control the robots in the multi-robot environment without any modification. Alternatively, one or more motion planners can be employed during runtime, e.g., to avoid collisions that can result from small changes in timing (e.g., sometimes it can take a little more or a little less time to tighten a screw than at other times).

[0011] The output can include, for example: a workcell layout, and for each robot: an ordered list or vector of targets (e.g., robot 1: {target 7, target 2, pause, target 9}), optionally a dwell duration at a respective target, and a path or trajectory (e.g., a collision-free path or trajectory) between each pair of consecutive targets.

[0012] The workcell layout can provide a base position and orientation (e.g., in Cartesian coordinates) for each robot. The base position and orientation of each robot base is specified with a respective 6-tuple {X, Y, Z, r, p, y}, where X, Y, and Z represent positions along respective axes of an orthogonal coordinate system, r (i.e., roll) represents an amount of rotation about a first one of the axes, p (e.g., pitch) represents an amount of rotation about a second one of the axes, and y (e.g., yaw) represents an amount of rotation about a third one of the axes.

[0013] The global optimizer can be based on, for example, a multivariate mixed-integer optimization algorithm, such as an algorithm referred to as differential evolution (DE). Unless a DE algorithm is explicitly recited in the claims, the claims are not limited to this algorithm.

[0014] A global optimizer optimizes robot base placement (in Cartesian coordinates), robot functional poses (i.e., in C-space), and task planning for each robot (a pause and ordered list of goals for the respective robot). The primary optimization goal can be latency (e.g., time to complete goals), but other goals can include efficient use of floor space, energy consumption or expenditure, number of moves to complete goals, ability to operate robots in parallel, minimization of robot wait times, availability of robots, state conditions, and / or availability of robots suitable for performing a particular type of task (e.g., availability of robots having a certain type of end-of-arm tool or end effector), etc. Thus, the structures and algorithms described herein facilitate operation of two or more robots operating in a shared workspace or workcell, optimize such placement and operation at least to some extent, and potentially prevent or at least reduce the risk of robots or robot accessories of the robots colliding with each other while performing respective tasks in the shared workspace. By performing autonomous planning that has been optimized to at least some extent, the structures and algorithms described herein can advantageously reduce the programming effort for a multi-robot workspace. Input can be limited to a description of the operating environment, the tasks to be performed, and geometric models of the robots. The structures and algorithms described herein can advantageously dynamically assign tasks to be performed by the robots. BRIEF DESCRIPTION OF DRAWINGS

[0015] In the drawings, like reference numerals refer to like elements or acts. The sizes and relative positions of elements in the drawings are not necessarily drawn to scale. For example, the shapes and relative sizes of some of the elements can have been exaggerated or skewed for clarity. Additionally, some of the depicted elements can have been arbitrarily moved, rotated, or otherwise manipulated to facilitate the reader's understanding of the figures. The particular shapes and relative sizes of the depicted elements are not intended to convey any information regarding the actual shapes, relative sizes, or actual proportions of the particular elements.

[0016] Figure 1 is a schematic diagram of a shared workspace or multi-robot operating environment in accordance with one illustrated embodiment, in which multiple robots operate to perform tasks, and a configuration system performs optimization to configure the robots.

[0017] Figure 2 is a schematic diagram of a shared workspace or multi-robot operating environment in accordance with one illustrated embodiment, in which multiple robots operate to perform tasks, and a configuration system performs optimization to configure the robots. Figure 1 is a functional block diagram of a configuration system of

[0018] Figure 3 shows a high-level robot configuration method in accordance with one illustrated embodiment, in which a processor-based system operates to configure multiple robots for a multi-robot operating environment in which the multiple robots will operate.

[0019] Figure 4 An advanced multi-robot environment simulation method is illustrated, based on a processor-based system operation according to an illustrated embodiment, for configuring multiple robots in a multi-robot operating environment in which multiple robots will operate.

[0020] Figure 5 A low-level multi-robot environment simulation method is illustrated, based on a processor-based system operation according to an illustrated embodiment, for configuring multiple robots in a multi-robot operating environment in which multiple robots will operate.

[0021] Figure 6 A processor-based system operation according to an illustrated embodiment is shown, which is a low-level multi-robot optimization (DE) method for configuring multiple robots in a multi-robot operating environment in which multiple robots will operate.

[0022] Figure 7 A parameterized cost function according to an illustrated embodiment is shown, which can be used with an embodiment using the differential evolution (DE) algorithm.

[0023] Figure 8 This is a further illustration based on an illustrated embodiment. Figure 7 The cost function curve.

[0024] Figure 9 A data structure according to an illustrated embodiment is shown, which can be used by a processor-based system when representing candidate solutions in a perturbation-allowed format (e.g., during execution). Figure 6 (This is used when optimizing low-level multi-robot DE methods).

[0025] Figure 10 A method for configuring low-level multi-robot DE candidate solutions for a multi-robot operating environment in which multiple robots will operate is illustrated according to a processor-based system operation according to an illustrated embodiment, which details the operation of a swarm generator. Detailed Implementation

[0026] In the following description, specific details are set forth to provide a full understanding of the various disclosed embodiments. However, those skilled in the art will recognize that embodiments can be practiced without one or more of these specific details, or using other methods, components, materials, etc. In other instances, well-known structures relating to computer systems, actuator systems, and / or communication networks have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments. In other instances, well-known computer vision methods and techniques for generating perceptual data and volumetric representations of one or more objects, etc., have not been described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0027] Unless the context clearly requires otherwise, throughout the description and the claims, the word "comprise," and variations such as "comprises" or "comprising," will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps.

[0028] Throughout this specification, unless the context clearly requires otherwise, the word "comprise," and variations such as "comprises" or "comprising," will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps.

[0029] The singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. It will be further understood that the terms "comprise," "comprises," "comprising," "includes," "including," and / or "has" when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0030] As used in this specification and claims, the term "optimize" means that an improved result is being prepared, generated, or produced, or has been prepared, generated, or produced. These terms are used in their relative sense and do not mean that an absolutely best value has been achieved.

[0031] As used in this specification and claims, the term "workspace" or "shared workspace" is used to refer to a current operating environment in which two or more robots operate, one or more portions of the shared workspace being volumes in which the robots can collide with each other and thus can be referred to as interference regions. The operating environment can include obstacles and / or workpieces (i.e., items with which the robots will interact or on which or with which the robots will act).

[0032] As used in this specification and claims, the term "task" is used to refer to a robot task in which a robot transitions from a pose A to a pose B without colliding with obstacles in its environment. The task can involve grasping or ungrasping an item, moving or setting down an item, rotating an item, or retrieving or placing an item. The transition from pose A to pose B can optionally include a transition between one or more intermediate poses.

[0033] The title and summary of the disclosure provided herein are provided merely for convenience and should not be used to construe the scope or meaning of any embodiment.

[0034] Figure 1A robotic system 100 is shown in accordance with one illustrated implementation that includes multiple robots 102a, 102b, 102c (collectively, 102) operating in a shared workspace 104 to perform a task.

[0035] Robots 102 can take any of a variety of forms. In general, robots 102 will take the form of or have one or more robot attachments 103 (only one labeled) and a base 105 (only one labeled). Robots 102 can include one or more linkages having one or more joints and coupled and operable actuators (e.g., electric motors, stepper motors, solenoids, pneumatic actuators, or hydraulic actuators) that move the linkages in response to control or drive signals. For example, pneumatic actuators can include one or more pistons, cylinders, valves, gas reservoirs, and / or pressure sources (e.g., compressors, blowers). For example, hydraulic actuators can include one or more pistons, cylinders, valves, fluid reservoirs (e.g., low-compressibility hydraulic fluid), and / or pressure sources (e.g., compressors, blowers). Robots 102 can take other forms (e.g., automated vehicles) in other implementations.

[0036] Shared workspace 104 generally represents a three-dimensional space in which robots 102 can operate and move, although in certain limited implementations, shared workspace 104 can represent a two-dimensional space. Shared workspace 104 is a volume or region in which at least portions of robots 102 can spatially and temporally overlap or collide if motion is not controlled to avoid collisions. It should be noted that workspace 104 is a physical space or volume whose position and orientation can be conveniently represented via Cartesian coordinates with respect to, for example, an orthogonal axis X, Y, Z reference frame as shown. Figure 1 It should also be noted that the reference frame of workspace 104 is different from the respective “configuration space” or “C-space” of any robot 102, which is generally represented by a set of joint positions, orientations, or configurations in the respective reference frame of any robot 102.

[0037] As described herein, robot 102a or portions thereof can constitute an obstacle when considered from the perspective of another robot 102b (i.e., when motion planning is performed for another robot 102b). Additionally, shared workspace 104 can include other obstacles, such as mechanical components (e.g., a conveyor 106), posts, pillars, walls, ceilings, floors, tables, people, and / or animals. Additionally, shared workspace 104 can include one or more work items or workpieces (e.g., one or more packages, packages, fasteners, tools, items, or other objects) that are manipulated by robots 102 as part of performing a task.

[0038] The robotic system 100 includes one or more processor-based multi-robot configuration optimization systems 108 (one of which is shown in FIG. 1). The multi-robot configuration optimization system 108 receives a set of inputs 109 and generates as output 111 one or more solutions specifying configurations of the robots 102 that can be optimized at least to some extent, including respective base positions and orientations, at least one defined pose for respective groups, and respective target sequences for each robot 102. Figure 1

[0039] The multi-robot configuration optimization system 108 can include a population generator 110, a multi-robot environment simulator 112, and a multi-robot optimization engine 114.

[0040] The population generator 110 generates a set of candidate solutions 116 based on the provided inputs 109. The candidate solutions 116 represent possible solutions to the configuration problem, i.e., how to configure the robots 102 in the workspace 104 to complete a set of tasks. Any given candidate solution 116 can or can not actually be feasible. That is, an initial candidate solution can be infeasible (e.g., a robot has an infeasible task plan in an impossible place, with targets that cannot be reached, or that would cause a collision). In some implementations, the population generator can attempt to find better candidate solutions.

[0041] The multi-robot environment simulator 112 models the multi-robot environment based on each candidate solution to determine certain properties (e.g., an amount of time required to complete the tasks, a probability of completing the tasks, or a collision rate, a feasibility or infeasibility of a particular configuration specified by the candidate solution). The multi-robot environment simulator 112 can reflect this in a cost, which is generated via one or more cost functions.

[0042] The multi-robot optimization engine 114 evaluates the candidate solutions based at least in part on the associated costs 118, and advantageously co-optimizes across two or more non-homogenous parameters of the set, e.g., co-optimizes across two or more of: respective base positions and orientations of the robots, task assignments to respective ones of the robots, respective target sequences for the robots, and / or respective trajectories or paths (e.g., collision-free paths) between consecutive targets. Straight-line trajectories between consecutive targets are used to simplify explanation, but the trajectories need not be straight-line trajectories.

[0043] ​The input 109 can include one or more static environment models representing or characterizing the operating environment or workspace 104 (e.g., representing the ground, walls, ceiling, pillars, other obstacles, etc.). The operating environment or workspace 104 can be represented by one or more models (e.g., geometric models (e.g., point clouds) representing the ground, walls, ceiling, obstacles, and other objects in the operating environment). This can be represented in Cartesian coordinates, for example.

[0044] The input 109 can include one or more robot models representing or characterizing each robot 102, e.g., specifying geometry and kinematics (e.g., size or length, number of links, number of joints, joint types, range of motion, velocity limits, limits on acceleration or jerk). The robots 102 can be represented by one or more robot geometry models defining the geometry of a given robot 102a-102c, e.g., in terms of joints, degrees of freedom, dimensions (e.g., lengths of linkages), and / or in terms of respective C-spaces of the robots 102a-102c.

[0045] The input 109 can include one or more sets of tasks to be performed (e.g., represented as target goals (e.g., positions or configurations)). The tasks can be represented in terms of end poses, end configurations, or end states and / or intermediate poses, intermediate configurations, or intermediate states of the respective robots 102a-102c, for example. The poses, configurations, or states can be defined in terms of joint positions and joint angles / rotations (e.g., joint poses, joint coordinates) of the respective robots 102a-102c, for example. The input 109 can optionally include one or more dwell durations specifying that a robot or portion thereof should dwell at a given goal to complete a task (e.g., tightening a screw or nut, picking and placing an object, sorting a pile of objects into two or more different object type piles of objects by two or more robots operating in a common workspace).

[0046] The inputs 109 can optionally include a limit on the number of robots that can be configured in the workspace 104. The inputs 109 can optionally include a limit on the number of tasks or goals (herein referred to as the task volume) that can be assigned to a given robot 102a-102c that can be configured in the workspace 104, e.g., to limit the complexity of the configuration problem to ensure that the configuration problem is solvable or solvable within a certain acceptable time period using available computing resources, or to preclude certain solutions that presume that the given robot 102a-102c is significantly over-assigned tasks or goals that are considered too slow. The inputs 109 can optionally include one or more bounds or constraints on variables or other parameters. The inputs 109 can optionally include a total number of iteration cycles or a time limit on iterations, which can be used to refine candidate solutions, e.g., to ensure that the configuration problem is solvable or solvable within a certain acceptable time period using available computing resources.

[0047] The robot system 100 can optionally include one or more robot control systems 118 (only one of which is shown in Figure 1 The robot control systems 118 can provide control signals (e.g., drive signals) to various actuators, for example, to move the robots 102 between various configurations to various specified targets in order to perform specified tasks.

[0048] The robot system 100 can optionally include one or more motion planners 120 (only one of which is shown in Figure 1 As described elsewhere herein, the motion planners 120 generate or refine motion plans for the robots 102, e.g., to account for small deviations in time relative to motion plans provided by the multi-robot optimization engine 114, or to account for unexpected appearance of obstacles (e.g., a person entering the operating environment or workspace 104). The optional motion planners 120 are operable to dynamically generate motion plans to cause the robots 102 to perform tasks in the operating environment. The motion planners 120, as well as other structures and / or operations, can employ those described in U.S. Patent Application No. 62 / 865,431, filed June 24, 2019.

[0049] Where included, the motion planner 120 is optionally communicatively coupled to receive, for example, perception data (provided by a perception subsystem (not shown)) as input. The perception data represents prior-unknown static and / or dynamic objects in the workspace 104. The perception data can be raw data sensed via one or more sensors (e.g., cameras, stereo cameras, time-of-flight cameras, LiDAR) and / or converted by the perception subsystem into digital representations of obstacles, which can generate a corresponding discretized representation of the environment in which the robot 102 will operate to perform tasks in various different scenarios.

[0050] Various communication paths in Figure 1 The diagram shows lines between various structures; in some cases, arrows indicate the direction of input 109 and output 111. Communication paths can take the form of, for example, one or more wired communication paths (e.g., electrical conductors, signal buses, or optical fibers) and / or one or more wireless communication paths (e.g., via RF or microwave radios and antennas, infrared transceivers). Communication channels can include, for example, one or more transmitters, receivers, transceivers, radio devices, routers, wired ports (e.g., Ethernet ports), etc.

[0051] Figure 2 The illustration shows at least one of the illustrated embodiments. Figure 1 The functional block representation of the robot system 100.

[0052] Robot system 100 may include robot configuration optimization system 108 and robot 102. Optimization system 108 may be communicatively coupled directly or via an intermediate robot control system 118. Figure 1 Indirectly control robot 102.

[0053] Each robot 102a-102c may include a set of links, joints, end-effectors, or end effectors and / or actuators 201a, 201b, 201c (three shown, collectively referred to as 201) operable to move the links about the joints. Each robot 102a-102c may include one or more motion controllers (e.g., motor controllers) 202 (only one shown), which receives control signals (e.g., from the robot configuration optimization system 108) and provides drive signals to drive the actuators 201. The motion controller 202 may be dedicated to controlling a specific one of the actuators 201.

[0054] For illustrative purposes, the robot configuration optimization system 108 will be described in detail. Those skilled in the art will recognize that this description is exemplary and variations may be made to the robot configuration optimization system 108 described and shown.

[0055] The robot configuration optimization system 108 can include one or more processors 222 and associated one or more non-transitory computer- or processor- readable storage media (e.g., system memory 224a, disk drives 224b, and / or memory or registers of the processors 222 (not shown)). The non-transitory computer- or processor-readable storage media 224a, 224b are communicatively coupled to the processors 222a via one or more communication channels (e.g., a system bus 229). The system bus 229 can take any known bus structures or architectures, including a memory bus having a memory controller, a peripheral bus, and / or a local bus. One or more such components can also or instead be communicatively coupled to one another via one or more other communication channels (e.g., one or more parallel, serial, or wireless network channels capable of high-speed communication, such as a Universal Serial Bus (“USB”) 3.0, Peripheral Component Interconnect Express (PCIe), or via a network interface (not shown) to one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or the Internet). ) communicate with one another.

[0056] The robot configuration optimization system 108 can also be communicatively coupled to one or more remote computer systems 212 (e.g., server computers, desktop computers, laptop computers, ultraportable computers, tablet computers, smartphones, wearable computers, and / or sensor Figure 2 (not shown in FIG. 1) that are directly communicatively coupled or indirectly communicatively coupled (e.g., via a network interface (not shown)) to various components of the robot configuration optimization system 108. The remote computing systems (e.g., server computers (e.g., input sources 212)) can be used to program input data, configuration input data, control input data, or otherwise interact with or provide input data (e.g., environment models, robot models, tasks, objectives, limits on the total number of robots, limits on the tasks for each robot, bounds or constraints on variables or other parameters, limits on iterations) to the robot configuration optimization system 108 and various components within the robot system 100. Such connections can be through one or more communication channels using an Internet protocol (e.g., one or more wide area networks (WANs) (e.g., Ethernet or the Internet)). In some implementations, run-ahead computations (e.g., generation of outputs) can be performed by a system separate from the robots 102, while run-time computations can be performed by an optional one or more intermediate motion planners 120 Figure 1 (not shown in FIG. 1) that are directly communicatively coupled or indirectly communicatively coupled (e.g., via a network interface (not shown)) to various components of the robot configuration optimization system 108. The remote computing systems (e.g., server computers (e.g., input sources 212)) can be used to program input data, configuration input data, control input data, or otherwise interact with or provide input data (e.g., environment models, robot models, tasks, objectives, limits on the total number of robots, limits on the tasks for each robot, bounds or constraints on variables or other parameters, limits on iterations) to the robot configuration optimization system 108 and various components within the robot system 100. Such connections can be through one or more communication channels using an Internet protocol (e.g., one or more wide area networks (WANs) (e.g., Ethernet or the Internet)). In some implementations, run-ahead computations (e.g., generation of outputs) can be performed by a system separate from the robots 102, while run-time computations can be performed by an optional one or more intermediate motion planners 120

[0057] As should be noted, the robot configuration optimization system 108 may include one or more processors 222 (i.e., circuitry), non-transitory storage media 224a, 224b, and a system bus 229 coupling various system components. The processor 222 may be any logic processing unit, such as one or more central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), etc. Non-limiting examples of commercially available computer systems include, but are not limited to, those in the United States. The company provides Celeron, Core, Core 2, Itanium, and Xeon series microprocessors; Advanced Micro Devices provides K8, K10, Bulldozer, and Bobcat series microprocessors; Apple Computer provides A5, A6, and A7 series microprocessors; Qualcomm provides Snapdragon series microprocessors; and Oracle provides SPARC series microprocessors. Figure 2 The construction and operation of the various structures shown can be implemented or adopted using international patent application No. PCT / US2017 / 036880, filed June 9, 2017, entitled "MOTION PLANNING FOR AUTONOMOUS VEHICLES AND RECONFIGURABLE MOTION PLANNING PROCESSORS"; international patent application publication No. WO 2016 / 122840, filed January 5, 2016, entitled "SPECIALIZED ROBOT MOTION PLANNING HARDWARE AND METHODS OF MAKING AND USINGSAME"; and international patent application No. WO 2016 / 122840, filed January 12, 2018, entitled "APPARATUS, METHOD AND ARTICLE TO FACILITATE MOTION PLANNING OF AN AUTONOMOUS VEHICLE IN AN ENVIRONMENT HAVING DYNAMIC". The structures, techniques, and algorithms described in or similar to those described in U.S. Patent Application No. 62 / 616,783, entitled “OBJECTS”, and / or U.S. Patent Application No. 62 / 865,431, filed June 24, 2019, entitled “MOTION PLANNING FOR MULTIPLE ROBOTS IN SHARED WORKSPACE”.

[0058] System memory 224a can include read-only memory ("ROM") 226, random access memory ("RAM") 228, FLASH memory (FLASH) 230, EEPROM (not shown). A basic input / output system ("BIOS") 232, which can constitute a portion of ROM 226, contains the basic routines that help transfer information between elements within robotic system 100, such as during start-up.

[0059] Drives 224b can be, for example, a hard disk drive for reading and writing a magnetic disk, a solid state (e.g., FLASH) drive for reading and writing solid state memory, and / or an optical disk drive for reading and writing a removable optical disk. Robotic configuration optimization system 108 can also include any combination of such drives in various different embodiments. Drives 224b can communicate with processor 222 via system bus 229. As those skilled in the relevant art will appreciate, drives 224b can include an interface or controller (not shown) coupled between the drives and system bus 229. Drives 224b and their associated computer-readable media provide nonvolatile storage for robotic system 100 of computer- or processor-readable and / or executable instructions, data structures, program modules, and other data. Those skilled in the relevant art will appreciate that other types of computer- readable media that can store data that is accessible by a computer, such as WORM drives, RAID drives, magnetic cassettes, digital video disks ("DVDs"), Bernoulli cartridges, RAM, ROM, smart cards, etc., can be used.

[0060] Executable instructions and data can be stored in system memory 224a (e.g., operating system 236, one or more application programs 238, other programs or modules 240, and program data 242). Application programs 238 can include processor-executable instructions that cause processor 222 to perform one or more of: generating a population of candidate solutions, modeling candidate solutions, producing or determining costs associated with respective candidate solutions based at least in part on the modeling, performing optimization on the population of C candidate solutions by an optimization engine that co-optimizes two or more of: respective base position and orientation of the robots, task assignment to respective ones of the robots, and respective target sequence of the robots, on two or more non-homogenous parameters grouped together; and / or providing outputs that can be used to position and orient the robots and cause the robots to perform tasks in a multi-robot operational environment. Such operations can be as described herein (e.g., with reference to FIGS. 1-10). Figure 3 and Figure 10and described in the references incorporated herein by reference. In at least some implementations, the processor-executable instructions cause the processor 222 to construct a motion plan (e.g., collision detection or evaluation, updating costs of edges in a motion planning graph based on the collision detection or evaluation, and performing a path search or evaluation). Additionally, the application 238 can include one or more machine-readable and machine-executable instructions that cause the processor 222 to perform other operations, such as optionally processing perception data (captured via sensors). Additionally, the application 238 can include one or more machine-executable instructions that cause the processor 222 to perform various other methods described herein and in the references incorporated herein by reference.

[0061] While shown in the Figure 2 system memory 224a, the operating system 236, the application 238, other applications, the program / modules 240, and the program data 242 can be stored on other non-transitory computer- or processor-readable media such as the drive 224b.

[0062] Although not required, many implementations will be described in the general context of computer-executable instructions, such as program application modules, objects, or macros stored on computer or processor-readable media, such as computer or processor memory or storage, that can be executed by one or more computers or processors to perform the processes described herein, such as the generation of candidate solutions, modeling of candidate solutions via forward kinematics, detection of collisions in the model, determination of execution time and other costs, generation of costs via a cost computation function, co-optimization across a set of non-homogeneous parameters, generation of trajectories or paths (e.g., collision-free paths), and / or other motion planning operations.

[0063] In various implementations, the operations can be performed entirely in hardware circuitry or as software stored in a memory storage (e.g., the system memory 224a) and executed by one or more hardware processors 222a (e.g., one or more microprocessors, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), graphics processing unit (GPU) processors, programmable logic controllers (PLCs), electrically programmable read-only memories (EEPROMs), or as a combination of hardware circuitry and software stored in a memory storage).

[0064] The robot configuration optimization system 108 can optionally include one or more input / output components (e.g., a monitor or touch screen display 244, a keypad or keyboard 246, and / or a pointing device such as a computer mouse 248).

[0065] Those skilled in the relevant art will appreciate that the illustrated embodiments, as well as other embodiments, can be practiced with other system configurations, including structures and arrangements of robots, hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, personal computers ("PCs"), networked PCs, minicomputers, mainframe computers, and the like. Embodiments or examples, or portions thereof (e.g., at configuration time and runtime) can be practiced in a distributed computing environment where tasks or modules are performed by remote processing devices, which are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote memory storage devices or media. However, the location and manner of storage of certain types of information is important to help improve robot configuration.

[0066] Figure 3 A high-level robot configuration method 300 is shown that is operated by a processor-based system to configure a plurality of robots for a multi-robot operating environment in which the plurality of robots will operate, according to one illustrated embodiment. The processor-based system can include at least one processor and at least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions. When executed by the at least one processor, the processor-executable instructions cause the at least one processor to perform various operations or actions of the robot configuration method 300.

[0067] The plurality of robots can be configured to perform a set of tasks. The tasks can be specified as a task plan. The task plan can specify a number T of tasks that need to be performed by a number R of robots. The task plan can be modeled as a vector for each robot, where the vector is an ordered list of tasks (e.g., {task 7, task 2, task 9}) to be performed by the respective robot. The task vector can also optionally include a dwell duration specifying how long the robot or portion thereof should dwell at a given configuration or goal. The task vector can also specify home poses and / or other "function poses" (e.g., "yield" or storage poses) that are not directly related to solving the tasks. The poses can be specified in C-space of the robot.

[0068] The robot configuration method 300 can begin at 302, for example, in response to a startup or power-up of the system or components thereof, receipt of information or data, or invocation or enablement by a calling routine or program. The robot configuration method 300 can be performed at configuration time or pre-runtime, which can occur prior to runtime. This advantageously allows some of the most computationally intensive work to be performed prior to runtime, when responsiveness is not a particular concern.

[0069] At 304, at least one component of the processor-based system receives input characterizing the multi-robot environment, the set of tasks to be performed, and various constraints or boundaries to provide for the problem.

[0070] For example, the processor-based system can receive one or more models of a multi-robot operating environment. The models can represent a physical environment in which multiple robots will operate (e.g., representing the ground, walls, and various objects in the environment).

[0071] For another example, the processor-based system can receive a respective model of each of a plurality of robots that will operate in the multi-robot operating environment. The robot models can represent physical properties of the robots (e.g., physical dimensions, range of motion, number of joints, number of links, link lengths, end effector types, velocity limits, acceleration limits, etc.).

[0072] For another example, the processor-based system can receive a set of tasks or goals. The goals can represent various locations that each robot or portion thereof must move to, in order to complete a set of tasks, in a sequence or at a particular time. For example, the goals can be represented in a configuration space (C-space) of the respective robots. For another example, the processor-based system can optionally receive one or more dwell durations at which at least one of the robots dwells at one or more of the goals while performing at least one of the tasks. The dwell durations can advantageously reflect an expected amount of time that an end effector of the robot needs to remain at a given goal to complete a corresponding task (e.g., to screw a fastener into a threaded opening, to insert a component into a receptacle).

[0073] For another example, the processor-based system can optionally receive a set of boundaries or constraints on variables. Various boundaries and constraints can be applicable to the optimization in the multi-robot environment. For example, the processor-based system can optionally receive a set of time intervals that specify a time limit on modeling motion or simulated collisions. For example, the processor-based system can optionally receive a limit on the total number of robots that are allowed to operate in the multi-robot operating environment. For example, the processor-based system can optionally receive a maximum number of tasks or goals that are allowed per robot, which can in turn prevent problems from being submitted that are too complex to solve given available computational resources or time.

[0074] At 306, a population generator generates a population of C candidate solutions. The population can include one or more candidate solutions. Each candidate solution in the population of C candidate solutions specifies, for each robot, a respective base position and orientation, at least one defined pose of a respective group, and a respective sequence of targets. The respective base position and orientation specifies a respective position and orientation of a base of the respective robot in the multi-robot operating environment. The at least one defined pose of the respective group specifies at least a respective home pose and / or other functional poses (e.g., a yielding or storage pose) of the respective robot in the multi-robot operating environment. The respective sequence of targets includes a respective ordered list of targets for the respective robot to move through to complete a respective sequence of tasks. The population generator can be implemented as one or more processors executing processor-executable instructions.

[0075] The population generator can take the form of a pseudo-random population generator that can generate the population of C candidate solutions pseudo-randomly based on one or more input parameters. The pseudo-random population generator generates candidate solutions that are likely to be both feasible and infeasible solutions. The population generator alternately generates populations of C candidate solutions that are less likely to be infeasible candidate solutions than purely pseudo-randomly generated populations of C candidate solutions. This can yield candidate solution populations that enable faster optimization without loss of configuration space coverage. For example, the population generator can take into account the operating environment to avoid candidate solutions that position a base of a robot in an impossible location (e.g., a location occupied by a wall or other object), that would result in an infeasible task plan with one or more targets that the robot cannot reach, or that would result in a collision. In the case of a pseudo-random population generator, a variety of techniques can be employed to improve the candidate solutions, such as the techniques described below with reference to Figure 6

[0076] At 308, an optimization engine performs optimization on the population of C candidate solutions. In particular, the optimization engine co-optimizes two or more of the following across the set of two or more non-homogeneous parameters: the respective base position and orientation of the robots, the assignment of tasks to the respective robots of the robots, and the respective sequence of targets of the robots. The optimization engine may, for example, select an optimized candidate solution that has a co-optimized combination of: a respective optimized base position and orientation of the respective base of each robot, an optimized assignment of tasks, and an optimized motion plan. The optimization engine can be implemented as one or more processors executing processor-executable instructions. As described below with reference to Figure 4 ​The optimization engine can interact with the multi-robot environment simulator in performing the optimization, e.g., providing candidate solutions to be simulated and receiving costs generated or determined by the multi-robot environment simulator, the costs characterizing efficiency (e.g., completion time) and probability or collision rate of the respective candidate solutions. The optimization engine can select one of the candidate solutions based at least in part on respective costs associated with the candidate solutions, the respective costs based at least in part on the time to complete the sequence of tasks and the collision values determined for the respective candidate solutions.

[0077] At 310, the optimization engine provides an output of the optimization. In particular, the optimization engine can provide one or more of the following as the output: respective base position and orientation of each robot, respective task assignment of each robot, respective motion plan of each robot, and / or a set of collision-free paths of each robot, with or without dwell durations at respective goals in the robot C-space. The output can include an optimized task assignment that specifies a respective sequence of tasks to be performed for each robot in the form of an optimized ordered list of goals in the respective robot's C-space and one or more dwell durations at one or more of the goals. The output can additionally or alternatively include an optimized motion plan that specifies a set of collision-free paths, the set of collision-free paths specifying a respective collision-free path between each pair of consecutive goals in the ordered list of goals. In at least some implementations, the output is sufficient to drive the robots to perform the set of tasks. In at least some implementations, a motion planner can be employed to refine the motion plan.

[0078] The high-level robot configuration method 300 can terminate at 312, e.g., until invoked again. While the high-level robot configuration method 300 is described in an ordered flow, in many implementations various actions or operations will be performed concurrently or in parallel.

[0079] Figure 4 A high-level multi-robot environment simulation method 400 is shown that is operated by a processor-based system to configure a plurality of robots for a multi-robot operating environment in which the plurality of robots will operate, according to one illustrated implementation. The processor-based system can include at least one processor and at least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions. When executed by the at least one processor, the processor-executable instructions cause the at least one processor to perform various operations or actions of the multi-robot environment simulation method 400. The multi-robot environment simulation method 400 can be performed by a multi-robot environment simulator, which can be implemented by one or more different processors that are dedicated to simulating movement in a multi-robot environment. Additionally, the multi-robot environment simulator can be implemented by one or more processors that perform other operations, e.g., performing an optimization.

[0080] The multi-robot environment simulation method 400 can begin, at 402, e.g., in response to a startup or power-up of the system, receipt of information or data, or invocation or enablement by a calling routine or program (e.g., enablement by the high-level robot configuration method 300).

[0081] At 404, the multi-robot environment simulator receives a candidate solution, e.g., from an optimization engine or colony generator.

[0082] At 406, the multi-robot environment simulator models the robots in the multi-robot environment based on the particular candidate solution. The candidate solution may, for example, be submitted or provided to the multi-robot environment simulator by the optimization engine. Conventional modeling packages can be employed, e.g., modeling packages that employ forward kinematics, to model the movement of the robots based on the candidate solution.

[0083] At 408, the multi-robot environment simulator determines, via the modeling performed by the multi-robot environment simulator, a respective time to complete the sequence of tasks. The determination can include determining a total time for all robots to complete the sequence of tasks, which can have to wait for one another. For example, this can include determining, for each robot, a respective time to complete the respective task.

[0084] At 410, the multi-robot environment simulator determines, via the modeling performed by the multi-robot environment simulator, a collision value representing a rate or probability of a collision occurring in completing the sequence of tasks. The determination can include determining a total collision value for all robots to complete the sequence of tasks. For example, this can include determining, for each robot, a respective collision value.

[0085] At 412, the multi-robot environment simulator provides the determined time to complete the sequence of tasks and the determined collision value to, e.g., the optimization engine. As described elsewhere herein, in some implementations, the multi-robot environment simulator provides the determined time and collision value as distinct values for processing by the optimization engine to form values that can be co-optimized across two or more non-homogenous parameters.

[0086] The multi-robot environment simulation method 400 can terminate, at 414, e.g., until invoked again. While the multi-robot environment simulation method 400 is described in an ordered flow, in many implementations, various acts or operations will be performed simultaneously or in parallel.

[0087] Figure 5A processor-based system is shown operating in accordance with one illustrated embodiment to configure a low-level multi-robot environment simulation method 500 for a plurality of robots for a multi-robot operating environment in which the plurality of robots will operate. The processor-based system can include at least one processor and at least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions. When executed by the at least one processor, the processor-executable instructions cause the at least one processor to perform various operations or actions of the multi-robot environment simulation method 500. The multi-robot environment simulation method 500 can be performed by a multi-robot environment simulator, which can be implemented by one or more distinct processors dedicated to simulating movement in a multi-robot environment. Additionally, the multi-robot environment simulator can be implemented by one or more processors that perform other operations, such as performing an optimization.

[0088] The multi-robot environment simulation method 500 can begin, at 502, in response to a startup or power-up of the system, a receipt of information or data, or an invocation or enablement by a calling routine or program, such as an enablement by the high-level robot configuration method 300.

[0089] At 504, the multi-robot environment simulator virtually executes each task in a task sequence for each robot, the tasks being specified by a candidate solution that is the subject of evaluation or consideration. The task sequence can be specified as an ordered list of respective tasks. The ordered list of tasks is equivalent to an ordered list of trajectories in the C-space of the robot. The ordered list of tasks can include a plurality of trajectories between successive poses or configurations (e.g., joint configurations) and one or more dwell durations at one or more poses, a home pose, and one or more other defined functional poses in the C-space of the robot (e.g., a stow or “yield” pose). To virtually execute each task, the processor-executable instructions, when executed by the at least one processor, cause the processor to virtually simulate the plurality of trajectories and one or more of the following: the one or more dwell durations at the one or more poses, the home pose, or the one or more other defined functional poses.

[0090] At 506, for each time period (e.g., 0.1 s or some selected input value) of a plurality of time periods, the multi-robot environment simulator samples C-space positions of a portion of at least one of the robots. The multi-robot environment simulator uses forward kinematics to identify possible collisions between one or more portions of a respective one of the robots and another portion of the respective one of the robots, possible collisions between the respective one of the robots and another robot in the environment, and possible collisions between the respective one of the robots and another object in the multi-robot operating environment that is not another robot.

[0091] The multi-robot environment simulation method 500 can terminate at 508, for example, until invoked again. While the low-level multi-robot environment simulation method 500 is described in an ordered flow, in many implementations, various actions or operations will be performed simultaneously or in parallel.

[0092] The described systems and methods can employ various methods to refine or improve candidate solutions. For example, some methods can start with a base solution and make improvements on the base solution. For another example, some methods can employ a genetic algorithm or method, such as a differential evolution (DE) algorithm or similar technique. Reference is made below to Figure 6 One version of a DE algorithm is described.

[0093] Figure 6 A low-level multi-robot optimization DE method 600 is shown that is operated by a processor-based system to configure a plurality of robots for a multi-robot operating environment in which the plurality of robots will operate, according to one illustrated implementation. The processor-based system can include at least one processor and at least one non-transitory processor-readable medium that stores at least one of data and processor-executable instructions. When executed by the at least one processor, the processor-executable instructions cause the at least one processor to perform various operations or actions of the multi-robot optimization method 600. The multi-robot environment simulation method 600 can be performed by an optimization engine that can be implemented by one or more different processors that are dedicated to the optimization engine of the multi-robot environment. In addition, the multi-robot environment simulator can be implemented by one or more processors that perform other operations.

[0094] The multi-robot environment simulation method 600 can start at 602, for example, in response to a start-up or power-up of the system, a receipt of information or data, or an invocation or enablement by a calling routine or program (e.g., enabled by the high-level robot configuration method 300).

[0095] At 604, a population generator generates a population of candidate solutions.

[0096] At 606, an outer loop counter I is initialized, for example, to zero. At 608, the outer loop counter I is incremented, for example, to I + 1.

[0097] At 610, the candidate solution is represented in a format that allows for perturbation. For example, the candidate solution can be represented as a candidate solution vector. The candidate solution vector may, for example, include a plurality of real number vector elements, for example, one real number vector element for each task. The real number vector elements can represent a respective one of the tasks, a priority of the respective one of the tasks, and a respective combination of one of the robots identified to perform the respective one of the tasks.

[0098] At 612, the optimization engine perturbs or causes the perturbation of the candidate solution I to produce a perturbed candidate solution I'. For example, the optimization engine can modify real number vector elements (i.e., real number values) of the candidate solution vector.

[0099] At 614, the multi-robot environment simulator models the perturbed candidate solution I', e.g., determines a time to complete the task sequence of the perturbed candidate solution I' and a collision value representing a rate or probability of collisions occurring when the task sequence of the perturbed candidate solution I' is completed.

[0100] At 616, the optimization engine or the multi-robot environment simulator determines a cost value for the perturbed candidate solution I'. The determined cost value can be a function of the determined time to complete the task sequence of the perturbed candidate solution I' and the determined collision value for the perturbed candidate solution I'. In DE, the cost function is a piecewise logarithmic function, but can be parameterized, e.g., by Figure 7 the parameterization 700 in Figure 8 and the curve 800 in Figure 7 In DE, the parameter "task_reachability" is equal to (number of reachable goal objects) / (total number of goal objects). Thus, if all goal objects are reachable, the value of the parameter "task_reachability" is equal to 1.

[0101] At 618, the optimization engine or the multi-robot environment simulator determines whether the perturbed candidate solution I' has a lower associated cost than the candidate solution I.

[0102] At 620, in response to determining that the perturbed candidate solution I' has a lower associated cost than the candidate solution I, the optimization engine replaces the respective candidate solution I in the population of C candidate solutions with the perturbed candidate solution I'. In response to determining that the perturbed candidate solution I' does not have a lower associated cost than the candidate solution I, the optimization engine does not modify the population of C candidate solutions, passing control directly to 622.

[0103] At 622, the optimization engine determines whether an inner loop iteration exit condition has occurred. For example, the optimization engine determines when convergence has occurred, whether a limit on the number of iterations has been reached, and / or whether a limit on the iteration time has been reached. For example, convergence can be considered to have occurred in the case where the cost standard deviation of the current population of candidate solutions is less than an epsilon value.

[0104] If an exit condition has occurred, control passes to 626, where the multi-robot environment simulation method 600 can terminate, for example, until invoked again. If an exit condition has not occurred, control passes to 624, where the optimization engine or multi-robot environment simulator determines whether there are more candidate solutions in the population of candidate solutions to be perturbed. If there are more candidate solutions in the population of candidate solutions to be perturbed, control returns to 608, where the inner loop counter is incremented and the next candidate solution is perturbed and analyzed. If there are no more candidate solutions in the population of candidate solutions to be perturbed, control returns to 606, where the inner loop counter is reinitialized, and another pass through the possibly updated population of candidate solutions can be performed.

[0105] The various actions of the multi-robot environment simulation method 600 can be repeated for multiple iterations, refining the population of candidate solutions until an exit condition (e.g., convergence) is met.

[0106] While the low-level multi-robot optimization DE method 600 is described in an ordered flow, in many implementations, various actions or operations will be performed simultaneously or in parallel.

[0107] Figure 9 A data structure 900 is shown, in accordance with at least one illustrated implementation, that can be employed by a processor-based system in representing a candidate solution in a format that allows for perturbation, for example, when performing the low-level multi-robot optimization DE method 600 (e.g., in 608). Figure 6 ) when performing the low-level multi-robot optimization DE method 600 (e.g., in 608).

[0108] A candidate solution I can advantageously be represented with a vector of numbers, such that the system can perform one or more functions on the numbers to “perturb” the numbers, and in turn, the candidate solution, to generate a perturbed candidate solution I’. There are various ways to represent a candidate solution. Reference is made herein to Figure 9 The data structure 900 shown describes one approach. The system can employ this approach or other approaches to represent a candidate solution.

[0109] A candidate solution can initially be represented as a two-dimensional (2D) matrix that will be flattened into a one-dimensional (1D) vector. In the 2D matrix, the rows correspond to tasks. For a problem with T tasks, there are thus T rows. In the 2D matrix, the columns correspond to robots and priorities. If there are P priorities and R robots, there are P * R columns. Figure 9 The example shown has 3 robots, 3 priorities, and 4 tasks. In use, a particular problem can have a different number of robots, a different number of priorities, and a different number of tasks than Figure 9 The example shown is simplified for ease of understanding.

[0110] Each task is assigned to a single robot, as Figure 9 As shown, each row has exactly one entry marked (i.e., marked with an asterisk *) indicating which robot performs the task at which priority. A given robot can perform at most a maximum number of tasks specified in the input as the task amount for each robot. Note that for a given robot, there is only one task per priority. That is, there is only one asterisk (*) per column.

[0111] The system can flatten the 2D matrix into a 1D vector of length T, where each task has a vector element. The vector element corresponds to the "entry number" of the 2D matrix starting at 1. Thus, for the example shown, the value for task 1 is 2, the value for task 2 is 13 (the first row has 9 entries, then the assignment for task 2 (see asterisk *) is in the 4th entry of the 2nd row), the value for task 3 is 21, and the value for task 4 is 36. Figure 9

[0112] The system can normalize the resulting integer vector <2, 13, 21, 36> so that all values are between 0 and 1. For example, the system can divide each value by the total number of entries in the matrix, e.g., P * R * T = 36. The resulting normalized real vector is <2 / 36, 13 / 36, 21 / 36, 36 / 36>, where the sum of the individual vector elements is 1.

[0113] To perturb the candidate real vector V, the system can first take a number (e.g., three) of other randomly selected candidates (e.g., A, B, and C) and represent those randomly selected candidates (e.g., A, B, and C) with corresponding normalized 1D vectors normalized in the same way as described above. The system can then compute a perturbed real vector V'. For example, the system can compute the perturbed vector V' as equal to A + mut * (C - B).

[0114] The system can then compute a perturbed candidate Vp, e.g., by randomly mixing the vector elements of the vector V and the perturbed vector V'. That is, for each element of the perturbed candidate Vp, say Vp[i], the system randomly selects V[i] or V'[i]. The selection can be weighted by a parameter, e.g., the parameter can skew the selection so that the selection is not 50 / 50. The system can multiply the perturbed candidate Vp by P * R * T to scale up to a 1D vector representing the perturbed candidate. To return from the 1D real vector to the 2D matrix, the system first rounds the real values to integers. From there, the system can use these integer values to mark a blank 2D matrix.

[0115] Figure 10 ​A method 1000 of processor-based system operations to configure low-level multi-robot DE candidate solutions for a plurality of robots for a multi-robot operating environment in which the plurality of robots will operate is shown, according to one illustrated implementation, which details the operations of the population generator. The processor-based system can include at least one processor and at least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions. When executed by the at least one processor, the processor-executable instructions cause the at least one processor to perform various operations or actions of the method 1000 of multi-robot DE candidate solutions. The method 1000 of multi-robot DE candidate solutions can be performed by an optimization engine, which can be implemented by one or more different processors dedicated to the optimization engine for the multi-robot environment. Additionally, a multi-robot environment simulator can be implemented by one or more processors that perform other operations.

[0116] The global optimizer can be based on a multivariate mixed integer optimization algorithm, such as an algorithm known as differential evolution (DE). Unless the algorithm is explicitly recited in the claims, the claims are not limited to this algorithm.

[0117] The global optimizer optimizes robot base placement (e.g., in Cartesian coordinates), robot functional poses (e.g., in C-space), and task plans (ordered list or vector of targets and dwell durations) for each robot. The primary optimization objective can be total latency, but other objectives can include optimized use of available floor space, minimal energy use, etc.

[0118] The optimizer system can include three components: namely, an optimization engine, a multi-robot environment simulator, and a candidate solution generator (e.g., a seed generator).

[0119] As described in further detail below, the candidate solution generator generates a population of C candidate solutions, any given candidate solution of which can or can not be feasible.

[0120] As described in further detail below, the optimization engine attempts to find better candidate solutions, e.g., by perturbing one of the candidate solutions in the population, say candidate P, and seeing if the resulting perturbed candidate P’ has a lower cost. If so, the perturbed candidate P’ replaces the candidate P in the population. The total number of candidates C is unchanged.

[0121] To find the cost of a candidate solution, the multi-robot environment simulator simulates the candidate solution. A given candidate solution includes an ordered list of tasks for each robot, which is equivalent to an ordered list of trajectories in C-space. The multi-robot environment simulator samples the position of each robot in C-space at each time period (e.g., 0.1 s or some selected input value) and employs forward kinematics to check whether the virtual representation of the robot collides with itself, with another robot, or with some obstacle or object in the operating environment. The multi-robot environment simulator determines how long it takes for the candidate solution to fully execute. The multi-robot environment simulator determines the probability of collision or the collision rate (e.g., the fraction of time periods during which the robot collides at least once with itself, with another robot, or with an obstacle or object in the operating environment). It should be noted that the generation of the population of candidate solutions does not need to produce collision-free motion plans, but simply tracks the collision rate. There is no need for collision-free paths, which greatly speeds up processing; however, some embodiments can instead produce collision-free paths at this stage. The candidate cost is at least a function of these two numbers (e.g., the run time and the collision rate). In DE, this cost function is a piecewise logarithmic value that can be parameterized.

[0122] The overall workflow involves a cycle in which the optimizer generates candidates and the simulator evaluates their cost. This process is repeated until some convergence criterion, such as a standard deviation of the cost that is less than some ε value.

[0123] The candidate solution generator 1002 generates candidate solutions by performing various operations or actions.

[0124] At 1004, the candidate solution generator 1002 receives various variable boundaries, fixed parameters, task or goal objectives, and population size. At 1006, the candidate solution generator 1002 generates a base position and orientation for the base of each robot. At 1008, the candidate solution generator 1002 virtually places or positions the robot base in a virtual multi-robot operating environment, which it provides to the multi-robot simulator 1009, which models the candidate solution, providing feedback to the candidate solution generator 1002.

[0125] At 1010, the candidate solution generator 1002 generates a uniform random home pose and / or other functional poses for each robot.

[0126] At 1012, for each task or goal, the candidate solution generator 1002 finds a set of robots that can complete the task or achieve the goal at 1012a, randomly selects one of these robots at 1012b, determines whether the set is empty (indicating an infeasible task plan) at 1012c, returns to 1006 if the set is empty, and assigns the task or goal to the selected robot at 1012d.

[0127] At 1014, for each target robot, the candidate solution generator 1002 determines whether the total number of tasks or goals assigned to the robot exceeds a limit or amount of tasks or goals specified for the robot at 1014a, returns to 1006 (indicating an infeasible task plan) if over-assignment occurs, and generates a random sequence of the assigned tasks or goals at 1014b.

[0128] At 1016, the candidate solution generator 1002 generates or defines a candidate solution, which can be a vector or other representation that indicates the base layout and orientation of each robot, the home pose and / or other functional poses of each robot, and the sequence of targets for each robot.

[0129] At 1018, the candidate solution generator 1002 determines whether there are enough candidate solutions in the population of candidate solutions. If the number of candidate solutions in the population of candidate solutions is less than a specified number, control can return to 1006 to generate additional candidate solutions. Otherwise, control can pass to 1022, where the candidate solution generator 1002 returns the population of candidate solutions to the multi-robot optimization engine 1024 for further optimization.

[0130] Once an initial population of candidate solutions is generated, the population of candidate solutions can be refined by, for example, perturbing the candidate solutions.

[0131] While the low-level multi-robot DE candidate solution method 1000 is described in an ordered flow, in many implementations, various actions or operations will be performed simultaneously or in parallel.

[0132] In at least some implementations, the structures and algorithms described herein can operate without cameras or other perception sensors. In at least some implementations, coordination between robots relies on geometric models of the robots, the ability of the robots to communicate their respective motion plans, and geometric models of the shared workspace. In other implementations, vision or other perception can optionally be employed, for example, to avoid people or other dynamic obstacles that can enter or occupy portions of the shared workspace.

[0133] A wide variety of algorithms are used to solve motion planning problems. Each of these algorithms generally requires an ability to determine whether a given pose of a robot or a motion from one pose to another will result in a collision with an obstacle, either the robot itself or in the environment. Virtual collision evaluation or checking can be performed “in software” using processors that execute processor-executable instructions from a stored set of processor-executable instructions to perform the algorithm. Virtual collision evaluation or checking can be performed “in hardware” using a set of special-purpose hardware circuits (e.g., collision checking circuits implemented in a field programmable gate array (FPGA), an application specific integrated circuit (ASIC)). Such circuits can represent, for example, a swept volume (i.e., a swept volume) swept by a robot / robot attachment or a portion thereof during a corresponding motion or transition between two states. The circuits can produce, for example, a Boolean evaluation indicating whether the motion will collide with any obstacles, where at least some of the obstacles represent swept volumes of other robots operating in the shared workspace performing motions or transitions.

[0134] Example

[0135] Example 1. A method of operating in a processor-based system to configure a plurality of robots for a multi-robot operating environment in which the plurality of robots are to operate, the method comprising:

[0136] generating, via a population generator, a population of C candidate solutions, each of the candidate solutions in the population of C candidate solutions specifying, for each of the robots: a respective base position and orientation, a respective set of at least one defined pose, and a respective sequence of tasks or goals, wherein the respective base position and orientation specifies a respective position and orientation of a base of the respective robot in the multi-robot operating environment, the respective set of at least one defined pose specifies at least a respective home pose of the respective robot in the multi-robot operating environment, and the respective sequence of tasks or goals comprises a respective ordered list of goals for the respective robot to move through to complete a respective sequence of tasks;

[0137] performing, by an optimization engine, optimization on the population of C candidate solutions, the optimization engine co-optimizing two or more of: the respective base position and orientation of the robots, the task assignment to respective ones of the robots, and the respective sequence of tasks or goals of the robots, across a set of two or more non-homogeneous parameters; and

[0138] providing, as output: the respective base position and orientation of each of the robots, the respective task assignment of each of the robots, and the respective motion plan of each of the robots.

[0139] Example 2. The method of example 1, wherein providing as output: respective base position and orientation of each of the robots, respective task assignment of each of the robots, and respective motion plan for each of the robots includes providing an optimized task assignment that specifies for each robot a respective sequence of tasks to be performed in the form of an optimized ordered list of goals in the C-space of the respective robot and one or more dwell durations at one or more of the goals, and providing an optimized motion plan that specifies a set of collision-free paths that specify respective collision-free paths between each pair of consecutive goals in the ordered list of goals.

[0140] Example 3. The method of any one of examples 1 or 2, further comprising:

[0141] determining, for each of the population of C candidate solutions, via modeling performed by the multi-robot environment simulator, a respective time to complete the sequence of tasks and a respective collision value representing a rate or probability of a collision occurring in completing the sequence of tasks.

[0142] Example 4. The method of example 3, wherein performing optimization by the optimization engine across the population of C candidate solutions by co-optimization across two or more non-homogeneous parameters includes:

[0143] selecting, via the optimization engine, one of the candidate solutions based at least in part on a respective cost associated with the candidate solution, the respective cost based at least in part on the time to complete the sequence of tasks and the collision value determined for the respective candidate solution.

[0144] Example 5. The method of example 3, wherein determining, via modeling by the optimization engine, a respective time to complete the sequence of tasks and a respective collision value representing a rate or probability of a collision occurring includes:

[0145] virtually performing each task in the sequence of tasks via the multi-robot environment simulator;

[0146] sampling, via the multi-robot environment simulator, a C-space position of a portion of at least one of the robots for each of a plurality of time periods; and

[0147] checking for collisions using forward kinematics to identify possible collisions between one or more portions of a respective one of the robots and another portion of the respective one of the robots, between the respective one of the robots and another one of the robots in the environment, and between the respective one of the robots and another object in the multi-robot operating environment that is not another robot.

[0148] Example 6. The method of example 5, wherein the ordered list of tasks is equivalent to an ordered list of trajectories in the C-space of the robot, and comprises a plurality of trajectories, one or more dwell durations at one or more poses, a home pose, and one or more other defined functional poses in the C-space of the robot, and the virtually executing comprises virtually executing: the plurality of trajectories and one or more of the one or more dwell durations at the one or more poses, the home pose, or the one or more other defined functional poses.

[0149] Example 7. The method of example 1 or 2, further comprising:

[0150] for each of a plurality of candidate solutions in the population of C candidate solutions:

[0151] perturbing the respective candidate solution to produce a perturbed candidate solution;

[0152] modeling the perturbed candidate solution;

[0153] determining whether the perturbed candidate solution has a lower associated cost than the respective candidate solution; and

[0154] in response to determining that the perturbed candidate solution has a lower associated cost than the respective candidate solution, replacing the respective candidate solution in the population of C candidate solutions with the perturbed candidate solution.

[0155] Example 8. The method of example 7, repeated with multiple iterations of the perturbing, modeling, determining, and replacing until convergence occurs, a limit on a number of iterations is reached, or a limit on an iteration time is reached.

[0156] Example 9. The method of example 7, wherein perturbing the respective candidate solution to produce a perturbed candidate solution comprises perturbing a candidate solution vector, the candidate solution vector comprising a plurality of real number vector elements, one for each task, the real number vector element representing a respective one of the tasks, a priority of the respective one of the tasks, and a respective combination of ones of the robots identified to perform the respective one of the tasks.

[0157] Example 10. The method of example 1 or 2, further comprising:

[0158] receiving input, the input comprising at least one model of a multi-robot operating environment, a respective model of each of at least two of the robots to operate in the multi-robot operating environment, and a set of tasks or objectives.

[0159] Example 11. The method of example 1 or 2, further comprising:

[0160] receiving input including at least one model of a multi-robot operating environment, a respective model of each of at least two of the robots that are to operate in the multi-robot operating environment, a set of tasks or objectives, and at least one of one or more dwell durations to dwell at one or more objectives while the at least one of the robots performs the at least one task, a set of boundaries or constraints on variables, or a set of time intervals that specify a simulated collision time limit.

[0161] Example 12, the method of example 11, wherein receiving input includes receiving input further including at least one of a limit on a maximum number of tasks or a maximum number of objectives allowed for each robot and a limit on a total number of robots allowed to operate in the multi-robot operating environment.

[0162] Example 13, the method of example 1 or 2, wherein the population generator is a pseudo-random population generator, and wherein generating, via the population generator, a population of C candidate solutions includes pseudo-randomly generating, via the pseudo-random population generator, the population of C candidate solutions.

[0163] Example 14, the method of example 1 or 2, wherein generating, via the population generator, a population of C candidate solutions includes generating a population of C candidate solutions that are less likely to be invalid candidate solutions than a population of C candidate solutions that are purely pseudo-randomly generated.

[0164] Example 15, the method of example 1 or 2, wherein performing optimization on the population of C candidate solutions by an optimization engine that optimizes collaboratively across the set of two or more non-homogenous parameters includes selecting an optimized candidate solution having a collaboratively optimized combination of: a respective optimized base position and orientation of a respective base of each of the robots, an optimized task assignment, and an optimized motion plan.

[0165] Example 16, a processor-based system to configure a plurality of robots for a multi-robot operating environment in which the plurality of robots are to operate, the processor-based system comprising:

[0166] at least one processor; and

[0167] at least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions that, when executed by the at least one processor, cause the processor to perform any of the methods of examples 1-15.

[0168] Example 17, a processor-based system to configure a plurality of robots for a multi-robot operating environment in which the plurality of robots are to operate, the processor-based system comprising:

[0169] at least one processor; and

[0170] at least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions that, when executed by the at least one processor, cause the processor to:

[0171] generate, via a population generator, a population of C candidate solutions, each of the candidate solutions in the population of C candidate solutions specifying, for each of the robots: a respective base position and orientation, a respective set of at least one defined pose, and a respective sequence of tasks or goals, wherein the respective base position and orientation specifies a respective position and orientation of a base of the respective robot in the multi-robot operating environment, the respective set of at least one defined pose specifies at least a respective home pose of the respective robot in the multi-robot operating environment, and the respective sequence of tasks or goals comprises a respective ordered list of goals for the respective robot to move through to complete the respective sequence of tasks;

[0172] perform, by an optimization engine, optimization on the population of C candidate solutions, the optimization engine co-optimizing two or more of: the respective base position and orientation of the robots, the assignment of tasks to respective ones of the robots, and the respective sequence of tasks or goals of the robots, across a set of two or more non-homogeneous parameters; and

[0173] provide, as output: the respective base position and orientation of each of the robots, the respective assignment of tasks to each of the robots, and the respective motion plan for each of the robots.

[0174] Example 18, the processor-based system of Example 17, wherein the processor- executable instructions, when executed by the at least one processor, cause the processor to provide, as output: an optimized assignment of tasks that specifies, for each robot, a respective sequence of tasks to be performed in the form of an optimized ordered list of goals in a C-space of the respective robot and one or more dwell durations at one or more of the goals, and an optimized motion plan that specifies a set of collision-free paths that specify respective collision-free paths between each pair of consecutive goals in the ordered list of goals.

[0175] Example 19, the processor-based system of any one of Examples 17 or 18, wherein the processor-executable instructions, when executed by the at least one processor, further cause the processor to:

[0176] via modeling performed by the robotic environment simulator, for each of the population of C candidate solutions, determine a respective time to complete the sequence of tasks and a respective collision value representing a rate or probability of a collision occurring in completing the sequence of tasks.

[0177] Example 20. The processor-based system of example 19, wherein to perform optimization of the population of C candidate solutions by an optimization engine that co-optimizes across two or more non-homogenous parameters, the processor-executable instructions, when executed by the at least one processor, cause the processor to:

[0178] select, via the optimization engine, one of the candidate solutions based at least in part on a respective cost associated with the candidate solution, the respective cost based at least in part on the time to complete the sequence of tasks and the collision value determined for the respective candidate solution.

[0179] Example 21. The processor-based system of example 19, wherein to determine, via modeling by the optimization engine, a respective time to complete the sequence of tasks and a respective collision value representing a rate or probability of a collision occurring, the processor-executable instructions, when executed by the at least one processor, cause the processor to:

[0180] virtually execute each task in the sequence of tasks via the multi-robot environment simulator;

[0181] for each of a plurality of epochs, sample C-space positions for a portion of at least one of the robots via the multi-robot environment simulator; and

[0182] check for collisions using forward kinematics to identify possible collisions between one or more portions of the respective one of the robots and another portion of the respective one of the robots, between the respective one of the robots and another one of the robots in the environment, and between the respective one of the robots and another object in the multi-robot operating environment that is not another robot.

[0183] Example 22. The processor-based system of example 21, wherein the ordered list of tasks is equivalent to an ordered list of trajectories in C-space of the robots, and includes a plurality of trajectories, one or more dwell durations at one or more poses, a home pose, and one or more other defined functional poses in C-space of the robots, and to virtually execute each task, the processor-executable instructions, when executed by the at least one processor, cause the processor to virtually execute: the plurality of trajectories and one or more of the one or more dwell durations at the one or more poses, the home pose, or the one or more other defined functional poses.

[0184] Example 23. The processor-based system of example 17 or 18, wherein the processor-executable instructions, when executed by the at least one processor, further cause the processor to:

[0185] for each of a plurality of candidate solutions in the population of C candidate solutions:

[0186] perturb the respective candidate solution to produce a perturbed candidate solution;

[0187] model the perturbed candidate solution;

[0188] determine whether the perturbed candidate solution has a lower associated cost than the respective candidate solution; and

[0189] in response to determining that the perturbed candidate solution has a lower associated cost than the respective candidate solution, replace the respective candidate solution in the population of C candidate solutions with the perturbed candidate solution.

[0190] Example 24. The processor-based system of example 23, wherein the processor-executable instructions, when executed by the at least one processor, further cause the processor to repeat the perturbing, modeling, determining, and replacing in multiple iterations until convergence occurs, a limit on a number of iterations is reached, or a limit on an iteration time is reached.

[0191] Example 25. The processor-based system of example 23, wherein to perturb the respective candidate solution to produce a perturbed candidate solution, the processor-executable instructions, when executed by the at least one processor, cause the processor to perturb a candidate solution vector, the candidate solution vector comprising a plurality of real number vector elements, one real number vector element for each task, the real number vector element representing a respective one of the tasks, a priority of the respective one of the tasks, and a respective combination of ones of the robots identified to perform the respective one of the tasks.

[0192] Example 26. The processor-based system of example 17 or 18, wherein the processor-executable instructions, when executed by the at least one processor, further cause the processor to:

[0193] receive input, the input comprising at least one model of a multi-robot operating environment, a respective model of each of at least two of the robots to operate in the multi-robot operating environment, and a set of tasks or objectives.

[0194] Example 27. The processor-based system of example 17 or 18, wherein the processor-executable instructions, when executed by the at least one processor, further cause the processor to:

[0195] receiving input including at least one model of a multi-robot operating environment, a respective model of each of at least two of the robots that are to operate in the multi-robot operating environment, a set of tasks or objectives, and at least one of one or more dwell durations to dwell at one or more objectives while the at least one of the robots performs the at least one task, a set of boundaries or constraints on variables, or a set of time intervals that specify a simulated collision time limit.

[0196] Example 28. The processor-based system of example 27, wherein the processor- executable instructions, when executed by the at least one processor, further cause the processor to receive as input at least one of a maximum number of tasks or a maximum number of objectives allowed for each robot and a limit on a total number of robots allowed to operate in the multi-robot operating environment.

[0197] Example 29. The processor-based system of example 17 or 18, wherein the population generator is a pseudo-random population generator, and wherein generating, via the population generator, the population of C candidate solutions comprises pseudo-randomly generating, via the pseudo-random population generator, the population of C candidate solutions.

[0198] Example 30. The processor-based system of example 17 or 18, wherein generating, via the population generator, the population of C candidate solutions comprises generating a population of C candidate solutions that are less likely to be invalid candidate solutions than a purely pseudo-randomly generated population of C candidate solutions.

[0199] Example 31. The processor-based system of example 17 or 18, wherein to perform optimization on the population of C candidate solutions by the optimization engine that co-optimizes across the set of two or more non-homogenous parameters, the processor-executable instructions, when executed by the at least one processor, cause the processor to select an optimized candidate solution that has a co-optimized combination of: a respective optimized base position and orientation of a respective base of each of the robots, an optimized task assignment, and an optimized motion plan.

[0200] The foregoing detailed description has set forth various embodiments of the devices and / or processes via the use of block diagrams, schematics, and examples. As will be appreciated by those skilled in the art, each block diagram, schematic, and example does not literally constitute an exclusive alternative to the other blocks, schematics, and examples but can be understood as one possible embodiment of a variety of potential alternatives. In one embodiment, the subject matter can be implemented via Boolean circuitry, application specific integrated circuits (ASICs), and / or FPGAs. However, those skilled in the art will recognize the entirety or a portion of the embodiments disclosed herein can be implemented in various different embodiments, such as, for example, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more controllers (e.g., microcontrollers), as one or more programs running on one or more processors (e.g., microprocessors), as firmware, or virtually any combination thereof, and that those skilled in the art will recognize designing the circuitry and / or writing the code for the software and / or firmware would be well within the skills of those of ordinary skill in the art, given the present disclosure.

[0201] Those skilled in the art will recognize that many of the methods or algorithms set forth herein can employ additional acts, may- omit certain acts, and / or may- perform acts in a different order than specified herein.

[0202] In addition, those skilled in the art will appreciate that the mechanisms taught herein can be implemented in hardware, for example, in one or more FPGAs or ASICs.

[0203] Combining the various embodiments described above can provide further embodiments. All commonly assigned U.S. patent application publications, U.S. patent applications, foreign patents, and foreign patent applications referred to in this specification and / or listed in the Application Data Sheet, including but not limited to International Patent Application No. PCT / US2017 / 036880 entitled “MOTION PLANNING FOR AUTONOMOUS VEHICLES AND RECONFIGURABLE MOTION PLANNING PROCESSORS” filed June 9, 2017; International Patent Application Publication No. WO 2016 / 122840 entitled “SPECIALIZED ROBOT MOTION PLANNING HARDWARE AND METHODS OF MAKING AND USING SAME” filed January 5, 2016; U.S. Patent Application No. 62 / 616,783 entitled “APPARATUS, METHOD AND ARTICLE TO FACILITATE MOTION PLANNING OF AN AUTONOMOUS VEHICLE IN AN ENVIRONMENT HAVING DYNAMIC OBJECTS” filed January 12, 2018; U.S. Patent Application No. 62 / 626,939 entitled “MOTION PLANNING OF A ROBOT STORING A DISCRETIZED ENVIRONMENT ON ONE OR MORE PROCESSORS AND IMPROVED OPERATION OF SAME” filed February 6, 2018; U.S. Patent Application No. 62 / 856,548 entitled “APPARATUS, METHODS AND ARTICLES TO FACILITATE MOTION PLANNING IN ENVIRONMENTS HAVING DYNAMIC OBSTACLES” filed June 3, 2019; U.S. Patent Application No. 62 / 865,431 entitled “MOTION PLANNING FOR MULTIPLE ROBOTS IN SHARED WORKSPACE” filed June 24, 2019; U.S. Patent Application No. 62 / 964,405 entitled “CONFIGURATION OF ROBOTS IN MULTI-ROBOT OPERATIONAL ENVIRONMENT” filed January 22, 2020, are incorporated by reference herein in their entirety.These and other changes can be made to the embodiments according to the detailed description above. In general, the terms used in the appended claims should not be construed to limit the claims of the application to the specific embodiments disclosed in the specification and claims, but should be construed to include all embodiments that would be protected under the claims. Accordingly, the claims are not limited to the above description and examples.

Claims

1. A method for configuring multiple robots in a processor-based system for a multi-robot operation environment in which multiple robots will operate, the method comprising: A population of C candidate solutions is generated via a population generator. Each candidate solution in the population of C candidate solutions specifies for each robot: a corresponding base position and orientation, at least one defined pose of a corresponding group, and a corresponding target sequence. The corresponding base position and orientation specifies the corresponding position and orientation of the base of the corresponding robot in the multi-robot operating environment. The at least one defined pose of the corresponding group specifies at least the corresponding in-situ pose of the corresponding robot in the multi-robot operating environment. The corresponding target sequence includes a corresponding ordered list of targets through which the corresponding robot moves to complete the corresponding task sequence. The optimization engine performs group optimization on the C candidate solutions, co-optimizing two or more of the following across two or more non-homogeneous parameters: the robot's corresponding base position and orientation, task assignment to the corresponding robots within the robot, and the robot's corresponding target sequence; and The outputs provided are: the corresponding base position and orientation of each robot, the corresponding task assignment of each robot, and the corresponding motion plan of each robot. The outputs include: the corresponding base position and orientation of each of the robots, the corresponding task assignment of each of the robots, and the corresponding motion planning of each of the robots, including providing an optimized task assignment that specifies for each robot a corresponding sequence of tasks to be executed in the form of an optimized ordered list of targets in the C-space of the corresponding robot and one or more dwell times at one or more of the targets, and providing an optimized motion planning that specifies a set of collision-free paths that specify the corresponding collision-free paths between each pair of consecutive targets in the ordered list of targets.

2. The method according to claim 1, further comprising: Modeling performed via a robot environment simulator determines, for each of the group of C candidate solutions, the corresponding time for completing the task sequence and the corresponding collision value representing the ratio or probability of collisions occurring in the task sequence.

3. The method of claim 2, wherein the optimization of the C candidate solutions by an optimization engine that performs collaborative optimization across two or more non-homogeneous parameters in a group comprises: One of the candidate solutions is selected via the optimization engine based at least in part on the corresponding cost associated with the candidate solution, the corresponding cost being based at least in part on the time to complete the task sequence and the collision value determined for the corresponding candidate solution.

4. The method of claim 2, wherein determining the corresponding time for completing the task sequence and the corresponding collision value representing the ratio or probability of a collision occurring, via modeling through the optimization engine, comprises: Each task in the task sequence is virtually executed via a multi-robot environment simulator; For each of the multiple periods, the C-space position of at least one part of the robot is sampled via the multi-robot environment simulator; as well as Use forward kinematics to check for collisions to identify possible collisions between one or more parts of a robot and another part of a robot, possible collisions between a robot and another robot in the environment, and possible collisions between a robot and another object in a multi-robot operating environment that is not another robot.

5. The method of claim 4, wherein the ordered list of tasks is equivalent to an ordered list of trajectories in the robot's C-space, and includes multiple trajectories, one or more dwell durations at one or more poses, in-situ poses, and one or more other defined functional poses in the robot's C-space, and virtual execution includes virtual execution of: multiple trajectories and one or more of the following: one or more dwell durations at the one or more poses, in-situ poses, or one or more other defined functional poses.

6. The method according to claim 1, further comprising: For each of the multiple candidate solutions in the group of C candidate solutions, for at least one iteration: Perturb the corresponding candidate solutions to generate perturbated candidate solutions; Model the perturbated candidate solutions; Determine whether the perturbed candidate solution has a lower associated cost than the corresponding candidate solution; as well as In response to determining that a perturbed candidate solution has a lower associated cost than the corresponding candidate solution, the corresponding candidate solution in the group of C candidate solutions is replaced with the perturbed candidate solution.

7. The method of claim 6, wherein the perturbation, modeling, determination and replacement are repeated iteratively multiple times until convergence, the iteration limit is reached, or the iteration time limit is reached.

8. The method of claim 6, wherein perturbing the corresponding candidate solution to generate a perturbed candidate solution comprises a perturbed candidate solution vector comprising a plurality of real vector elements, including one real vector element for each task, the real vector element representing a corresponding one in the task, the priority of the corresponding one in the task, and a corresponding combination of one of the robots identified as performing the corresponding one in the task.

9. The method according to claim 1, further comprising: The system receives input, which includes at least one model of a multi-robot operating environment, a corresponding model of each of at least two robots that will operate in the multi-robot operating environment, and a set of tasks.

10. The method according to claim 1, further comprising: Receive input, which includes at least one model of a multi-robot operating environment, a corresponding model of each of at least two robots that will operate in the multi-robot operating environment, a set of tasks, and at least one of the following: one or more dwell times at one or more targets while at least one of the robots is performing at least one task, a set of boundary or constraint variables, or a set of time intervals specifying simulation collision time limits.

11. The method of claim 10, wherein receiving input includes receiving input that further includes at least one of the following: a maximum number of tasks allowed for each robot and a limit on the total number of robots allowed to operate in a multi-robot operation environment.

12. The method of claim 1, wherein the population generator is a pseudo-random population generator, and wherein generating a population of C candidate solutions via a population seed generator includes generating a population of C candidate solutions via the pseudo-random population generator.

13. The method of claim 1, wherein generating a population of C candidate solutions via a population generator includes generating a population of C candidate solutions that has a lower probability of being invalid candidate solutions compared to a population of C candidate solutions generated purely by pseudo-randomness.

14. The method of claim 1, wherein performing optimization on the group of C candidate solutions by an optimization engine that performs collaborative optimization across two or more non-homogeneous parameters comprises selecting an optimization candidate solution having a collaborative optimization combination of: corresponding optimized base position and orientation of the respective base of each robot, optimized task assignment, and optimized motion planning.

15. A processor-based system for configuring multiple robots in a multi-robot operating environment in which multiple robots will operate, the processor-based system comprising: At least one processor; as well as At least one non-transitory processor-readable medium, the at least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions, the processor-executable instructions causing the processor to perform any of the methods according to claims 1 to 14 when executed by the at least one processor.

16. A processor-based system for configuring multiple robots in a multi-robot operating environment in which multiple robots will operate, the processor-based system comprising: At least one processor; as well as At least one non-transitory processor-readable medium, said at least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions, said processor-executable instructions causing the processor, when executed by said at least one processor, to: A population of C candidate solutions is generated via a population generator. Each candidate solution in the population of C candidate solutions specifies for each robot: a corresponding base position and orientation, at least one defined pose of a corresponding group, and a corresponding target sequence. The corresponding base position and orientation specifies the corresponding position and orientation of the base of the corresponding robot in the multi-robot operating environment. The at least one defined pose of the corresponding group specifies at least the corresponding in-situ pose of the corresponding robot in the multi-robot operating environment. The corresponding target sequence includes a corresponding ordered list of targets through which the corresponding robot moves to complete the corresponding task sequence. The optimization engine performs group optimization on the C candidate solutions, co-optimizing two or more of the following across two or more non-homogeneous parameters: the robot's corresponding base position and orientation, task assignment to the corresponding robots within the robot, and the robot's corresponding target sequence; and The outputs provided are: the corresponding base position and orientation of each robot, the corresponding task assignment of each robot, and the corresponding motion plan of each robot. The processor-executable instructions, when executed by the at least one processor, cause the processor to provide an optimized task assignment as output, wherein the optimized task assignment specifies for each robot a corresponding sequence of tasks to be executed in the form of an optimized ordered list of targets in the C-space of the respective robot and one or more dwell times at one or more of the targets, and provides optimized motion planning that specifies a set of collision-free paths that specify the corresponding collision-free paths between each pair of consecutive targets in the ordered list of targets.

17. The processor-based system of claim 16, wherein the processor-executable instructions, when executed by the at least one processor, further cause the processor to: Modeling performed via a robot environment simulator determines, for each of the group of C candidate solutions, the corresponding time for completing the task sequence and the corresponding collision value representing the ratio or probability of collisions occurring in the task sequence.

18. The processor-based system of claim 17, wherein, in order to perform optimization on a group of C candidate solutions by an optimization engine that performs collaborative optimization across two or more non-homogeneous parameters, the processor-executable instructions, when executed by the at least one processor, cause the processor to: One of the candidate solutions is selected via the optimization engine based at least in part on the corresponding cost associated with the candidate solution, the corresponding cost being based at least in part on the time to complete the task sequence and the collision value determined for the corresponding candidate solution.

19. The processor-based system of claim 17, wherein, in order to determine the corresponding time for completing a task sequence and the corresponding collision value representing the ratio or probability of a collision occurring via modeling by the optimization engine, the processor executes instructions that, when executed by the at least one processor, cause the processor to: Each task in the task sequence is virtually executed via a multi-robot environment simulator; For each of the multiple periods, the C-space position of at least one part of the robot is sampled via the multi-robot environment simulator; as well as Use forward kinematics to check for collisions to identify possible collisions between one or more parts of a robot and another part of a robot, possible collisions between a robot and another robot in the environment, and possible collisions between a robot and another object in a multi-robot operating environment that is not another robot.

20. The processor-based system of claim 19, wherein the ordered list of tasks is equivalent to an ordered list of trajectories in the robot's C-space, and includes multiple trajectories, one or more dwell durations at one or more poses, in-situ poses, and one or more other defined functional poses in the robot's C-space, and for virtually executing each task, the processor-executable instructions, when executed by the at least one processor, cause the processor to virtually execute: multiple trajectories and one or more of the following: one or more dwell durations at the one or more poses, in-situ poses, or one or more other defined functional poses.

21. The processor-based system of claim 16, wherein the processor-executable instructions, when executed by the at least one processor, further cause the processor to: For each of the multiple candidate solutions in the group of C candidate solutions, for at least one iteration: Perturb the corresponding candidate solutions to generate perturbated candidate solutions; Model the perturbated candidate solutions; Determine whether the perturbed candidate solution has a lower associated cost than the corresponding candidate solution; as well as In response to determining that a perturbed candidate solution has a lower associated cost than the corresponding candidate solution, the corresponding candidate solution in the group of C candidate solutions is replaced with the perturbed candidate solution.

22. The processor-based system of claim 21, wherein the processor-executable instructions, when executed by the at least one processor, further cause the processor to repeatedly perform the perturbation, modeling, determination, and replacement in multiple iterations until convergence, an iteration limit is reached, or an iteration time limit is reached.

23. The processor-based system of claim 21, wherein, in order to perturb a corresponding candidate solution to generate a perturbed candidate solution, the processor-executable instructions, when executed by the at least one processor, cause the processor to perturb a candidate solution vector, the candidate solution vector comprising a plurality of real vector elements, including one real vector element for each task, the real vector element representing a corresponding one in the task, the priority of the corresponding one in the task, and a corresponding combination of one of the robots identified as performing the corresponding one in the task.

24. The processor-based system of claim 16, wherein the processor-executable instructions, when executed by the at least one processor, further cause the processor to: The system receives input, which includes at least one model of a multi-robot operating environment, a corresponding model of each of at least two robots that will operate in the multi-robot operating environment, and a set of tasks.

25. The processor-based system of claim 16, wherein the processor-executable instructions, when executed by the at least one processor, further cause the processor to: Receive input, which includes at least one model of a multi-robot operating environment, a corresponding model of each of at least two robots that will operate in the multi-robot operating environment, a set of tasks, and at least one of the following: one or more dwell times at one or more targets while at least one of the robots is performing at least one task, a set of boundary or constraint variables, or a set of time intervals specifying simulation collision time limits.

26. The processor-based system of claim 25, wherein the processor-executable instructions, when executed by the at least one processor, further cause the processor to receive as input at least one of the following: a maximum number of tasks allowed for each robot and a limit on the total number of robots allowed to operate in a multi-robot operating environment.

27. The processor-based system of claim 16, wherein the population generator is a pseudo-random population generator, and wherein generating a population of C candidate solutions via a population seed generator includes generating a population of C candidate solutions pseudo-randomly via the pseudo-random population generator.

28. The processor-based system of claim 16, wherein generating a population of C candidate solutions via a population generator includes generating a population of C candidate solutions that has a lower probability of being invalid candidate solutions compared to a population of C candidate solutions generated purely by pseudo-randomness.

29. The processor-based system of claim 16, wherein, in order to perform optimization on a group of C candidate solutions by an optimization engine that performs collaborative optimization across two or more non-homogeneous parameters, the processor-executable instructions, when executed by the at least one processor, cause the processor to select an optimization candidate solution having a collaborative optimization combination of: corresponding optimized base position and orientation of the respective base of each robot, optimized task assignment, and optimized motion planning.

Images