Robust motion planning and / or control for multi-robot environments

Abstract

The motion planner generates a motion plan for a robot operating in a shared workspace, the motion plan including a nominal trajectory (e.g., a specified trajectory). An acceptable lag time is determined for the nominal trajectory, and adherence to the acceptable lag time during actual operation ensures self-collision-free operation. The actual motion of the robot (e.g., actual trajectory) is monitored for adherence to the respective acceptable lag time. Optionally, intervention is selected and / or taken as necessary (e.g., if the actual lag time approaches or exceeds, for example, a margin or threshold for the respective acceptable lag time). The motion planner can use the determined acceptable lag time to select a trajectory that provides more robust operation of the robot.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This patent application claims priority to U.S. Patent Application No. 63 / 358,422, filed July 5, 2022, the entire disclosure of which is incorporated herein by reference for all purposes.

[0002] <Technical field> The present disclosure generally relates to motion planning and operation of robots operating within a shared workspace, and relates to the computationally efficient generation of robust motion plans that can be executed by the robots in an efficient manner (e.g., reducing or even eliminating collisions and unplanned stoppages), as well as monitoring the actual execution of the motion plans and optionally taking remedial action when warranted. [Background technology]

[0003] <Description of Related Art> Various applications use two or more robots operating in a shared workspace, for example, where two or more robots perform tasks on or with one or more objects or workpieces in the shared workspace, such as threading bolts into chassis where portions of the robots overlap in their range of motion.

[0004] Planning typically includes task planning and motion planning. Task planning, for example, may determine that a robot's execution of a given task is to move between two poses (e.g., from a start pose to an end pose). In contrast, motion planning focuses on how to move the robot between the two poses.

[0005] Motion planning is a fundamental problem in robot control and robotics. Motion planning specifies a trajectory that a robot can follow from a starting pose, configuration, or state to a target (or goal) pose, configuration, or state, generally to complete a task without colliding with obstacles in a shared workspace or with a reduced likelihood of colliding with obstacles in the shared workspace. Motion planning challenges include the ability to execute the motion plan quickly (i.e., in real time) while taking into account environmental changes (e.g., changes in the position or orientation of obstacles in the shared workspace) as much as possible. Challenges further include executing the motion plan using relatively low-cost equipment, with relatively low energy consumption, and with a limited amount of computing power and / or storage (e.g., memory circuitry, e.g., "on-processor" circuitry).

[0006] The operation of two or more robots in a shared workspace (also called a workcell or multi-robot operating environment) presents a particular class of problems: For example, motion planning should consider and avoid situations where the robots or their robotic appendages may interfere with each other while performing a task.

[0007] One approach to operating multiple robots within a shared workspace can be called the task-level approach. Engineers can define portions of the shared workspace where robots may collide with each other (i.e., interference regions) and manually ensure (or guarantee) that the robots will not collide by programming each individual robot so that only one robot is in the interference region of the shared workspace at any given time. For example, when a first robot begins to move into the interference region of the shared workspace, the first robot sets a flag. A controller (e.g., a programmable logic controller (PLC)) reads the flag and prevents other robots from moving into the interference region of the shared workspace until the first robot de-asserts the flag upon leaving the interference region. While this approach is intuitive and easy to understand, it is typically difficult and time-consuming to implement and may not produce optimal results. This approach necessarily has low work throughput, since the use of collision avoidance (or de-confliction) typically results in at least one robot being idle for a significant amount of time (even if the idle robot is technically capable of performing useful work in the shared workspace). Summary of the Invention [Problem to be solved by the invention]

[0008] In traditional approaches, a team of engineers typically divides the problem and optimizes the resulting smaller subproblems independently of each other (e.g., assigning tasks to robots, sequencing the assigned tasks for each robot, and planning motion for each robot). This can involve iteratively simulating the motion to ensure that the robots / robot appendages do not collide with each other, which can require hours of computational time and may not result in an optimized solution. In addition, if modifications to the shared workspace result in a change in the actual trajectory of one of the robots / robot appendages, the entire workflow must be revalidated. Such an approach is, of course, suboptimal and typically requires experts to go through a slow process of iteratively attempting to find a combination of solutions that, when taken together, produce a good result. In any case, such a process often fails to guarantee collision-free operation when motion planning is performed by robots operating under real-world conditions. [Means for solving the problem]

[0009] Described herein are various methods and apparatus for generating motion plans for robots operating in a shared workspace, the motion plans including nominal trajectories (e.g., specified trajectories) with associated allowable lag times, where adherence to the allowable lag times during actual operation ensures self-collision-free operation (i.e., collisions of one robot with itself and with other robots in the robotic system). The nominal trajectories may, for example, represent respective collision-free paths. Described herein are various methods and apparatus for monitoring the actual motion (e.g., actual trajectories) of the robots with respect to the allowable lag times, and optionally taking one or more corrective actions as necessary (or when warranted) (e.g., when the actual lag times approach or exceed a threshold value, e.g., the respective allowable lag times).

[0010] A particularly advantageous approach is described in International Patent Application PCT / US2021 / 013610, published as WO 2021150439A1, which describes a system for generating optimized multi-robot motion plans. More specifically, a motion plan is optimized when it minimizes or attempts to minimize some cost function related to system performance (e.g., the probability or likelihood of collision). Each motion plan is also time-based as it specifies a nominal trajectory for the robotic system. A nominal trajectory is a specified, ordered sequence of poses, configurations, or states of each moving part of one or more robots in the robotic system, parameterized by time (e.g., for each time unit throughout the trajectory). The nominal trajectory can, for example, represent each collision-free path. A motion plan can be "multi-robot" because a robotic system can include multiple individual robots.

[0011] A motion plan is represented by a set of discrete robot trajectories: Trj(t) = {trj_r(t), r∈{1...N}, t∈{0, ΔT,2ΔTt,...,T}, where N is the total number of robots in the robot system, ΔT is the sample time step, and T is the total duration of the motion plan. If the value of ΔT is small enough, T rj (t) is guaranteed to be collision-free for all robots in the robot system. In other words, the trajectory T of a robot system with two or more robots rj The motion specified by the set of trajectories Trj(t) does not result in self-collisions (i.e., collisions of one robot with itself and collisions of one robot with other robots in the robot system). The set of trajectories Trj(t) of the robot system is not necessarily collision-free with other objects or obstacles in the environment, but collision evaluation with other objects or obstacles in the shared workspace can be performed. This self-collision-free condition can be easily checked or verified in a simulation, and once checked or verified, it can be considered safe to execute the motion prescribed by the motion plan.

[0012] However, in practice, while a motion plan is actually executed by a physical robotic system comprising two or more robots, many factors can adversely affect synchronization between the robots. These factors can include, for example, signal communication, control delays, sensor noise, numerical approximation errors, etc. This can cause the actual physical robotic system to deviate (or diverge) from the trajectory Trj(t) specified by the motion plan. In addition to this “reality gap” between the simulated environment used to generate and validate the motion plan(s) and the actual behavior of the actual physical robotic system, many other factors can cause the robotic system to deviate from a motion plan that provides a set of nominal trajectories. For example, in some cases, task execution may require more time than expected to complete, and therefore the robot may need to stay or dwell longer at a particular position (e.g., a target position, and therefore, a particular pose, configuration, or state) than specified by the trajectory in the motion plan. Deviation from a specific motion plan that has been validated and deemed safe can be dangerous and compromise the safety of the robotic system.

[0013] A set of trajectories Trj(t) specified by a motion plan is referred to herein as a set of nominal trajectories, indicating that they are specified trajectories. A single trajectory specified by a motion plan is referred to herein as a nominal trajectory. A set of trajectories executed by the actual physical movement of the robot is referred to herein as a set of actual trajectories to distinguish such trajectories from specified or nominal trajectories. A single trajectory executed by the actual physical movement of the robot is referred to herein as an actual trajectory.

[0014] The approach described herein advantageously extends safe operating regimes beyond those associated with a set of nominal trajectories for the motion plan by, for example, calculating neighborhoods of nominal trajectories for the motion plan along which the robotic system can operate without resulting in the aforementioned self-collisions. For each robot operating in the shared workspace, a maximum acceptable lag time (e.g., time delay or "lag") is calculated for the nominal trajectories the robot potentially executes. During online execution (i.e., runtime), the actual lag time of each robot is monitored. If the actual lag time is within a defined margin or threshold, continued execution of the motion plan is deemed safe because self-collision-free movement between the robots is guaranteed. If the actual lag time of any robot exceeds the corresponding margin or threshold, safety is deemed compromised because self-collision-free movement is no longer guaranteed. Optionally, the processor-based system can select and / or take one or more corrective actions in such an event. [Brief explanation of the drawings]

[0015] In the drawings, identical reference numbers indicate similar elements or acts. The sizes and relative positions of elements in the drawings are not necessarily drawn to scale. For example, the shapes and angles of various elements are not drawn to scale, and some of these elements have been appropriately enlarged and positioned from time to time to improve drawing readability. Furthermore, the particular shapes of the elements as drawn are not intended to convey any information regarding the actual shape of the particular elements, but have been selected solely for ease of recognition in the drawings.

[0016] FIG. 1 is a schematic diagram of a shared workspace in which multiple robots operate to perform tasks, and a configuration optimization system that performs optimization for configuring the robots, according to one illustrated embodiment.

[0017] FIG. 2 is a functional block diagram of at least a first robot and a robot control system communicatively coupled to control the movement of the first robot, according to at least another illustrated implementation, the robot control system including a motion planner that advantageously determines and uses an acceptable lag time for each of the robots' nominal trajectories in motion planning, optionally monitors the actual movement of the at least first robot, compares the actual lag time with a margin or threshold (e.g., the acceptable lag time), and optionally takes corrective action as necessary.

[0018] FIG. 3 illustrates a method of operation of a processor-based system for executing motion plans to control robots operating within a shared workspace, according to at least one illustrated embodiment, in which an allowable lag time is determined for each robot based on the nominal trajectories of the other robots, without necessarily considering the effect of lag time on the nominal trajectories of the other robots, and a motion plan is generated or selected that specifies the nominal trajectories, which are associated with respective allowable lag times.

[0019] FIG. 4 illustrates a method of operation of a processor-based system for executing motion plans to control robots operating within a shared workspace, according to at least one illustrated implementation, in which an allowable lag time is determined for each robot based on the nominal trajectories of the other robots, and a motion plan is generated or selected that specifies the nominal trajectories, which are associated with their respective allowable lag times, while necessarily taking into account the effects of lag time on the nominal trajectories of the other robots.

[0020] FIG. 5 illustrates a method of operation of a processor-based system for generating a swept volume for one or more trajectories of each of multiple robots operating within a shared workspace according to at least one illustrated implementation, which may optionally be employed in performing the methods of FIGS. 3, 4, and / or 6.

[0021] FIG. 6 illustrates, according to at least one illustrated implementation, a method of operation of a processor-based system for performing collision assessment for multiple robots and for multiple trajectories for each of the robots, where the movement of the robot or a portion thereof along the trajectory is represented as a swept volume, and the collision assessment can be used to determine an acceptable lag time for one or more robots operating within a shared workspace.

[0022] FIG. 7 illustrates a method of operation of a processor-based system for generating or selecting motion plans for one or more robots operating within a shared workspace based at least in part on an acceptable lag time, and optionally based at least in part on other criteria, for example, expressed as a cost or cost function, according to at least one illustrated implementation.

[0023] FIG. 8 illustrates a method of operation of a processor-based system for controlling the movement of a robot operating within a shared workspace based at least in part on an acceptable lag time, according to at least one illustrated implementation. DETAILED DESCRIPTION OF THE INVENTION

[0024] <Detailed explanation> Details of the disclosure are described below to provide an appreciation of various embodiments. However, those skilled in the art will readily understand that the present invention may be practiced without one or more of these specific details, or with other methods, components, or materials. In other instances, well-known structures related to computer systems, actuator systems, robots, and / or communication networks have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments. In other instances, well-known computer vision methods and techniques for generating sensory data and volumetric representations of one or more objects, etc., have not been described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0025] Unless the context otherwise requires, throughout the following specification and claims, the word "comprise" and variations thereof, such as "comprises" and "comprising," are to be interpreted in their open and inclusive sense, meaning "including, but not limited to."

[0026] Throughout this specification, a reference to an "implementation" or "one implementation" or "embodiment" or "one embodiment" means that a particular feature, structure, or characteristic described in connection with that implementation or embodiment is included in at least one implementation or at least one embodiment. Thus, the appearances of the phrases "implementation" or "one implementation" or "in an embodiment" or "in one embodiment" in various places throughout this specification do not necessarily all refer to the same implementation or embodiment. Furthermore, particular features, structures, or characteristics may be combined in any suitable manner in one or more implementations or embodiments.

[0027] As used in this specification and the appended claims, the singular includes the plural unless the content clearly dictates otherwise, and the term "or" generally includes "and / or" unless the content clearly dictates otherwise.

[0028] As used in this specification and the appended claims, the terms "optimizing," "optimize," and "optimized" mean providing, producing, or creating an improved result. Such terms are used in their relative sense and do not necessarily imply that an absolute optimum value has been provided, produced, or created.

[0029] As used herein and in the appended claims, the term "workspace" or "shared workspace" is used to refer to an operating environment in which two or more robots operate, where one or more portions of the shared workspace are volumes within which the robots may collide with one another and therefore may be referred to as interference regions. The operating environment may include obstacles (i.e., items with which the robots must avoid collisions) and / or workpieces (i.e., items with which the robots interact, act, or work together).

[0030] As used herein and in the appended claims, the term "task" is used to refer to a robot task in which the robot transitions from pose A to pose B, preferably without colliding with obstacles in its environment. A task may possibly include grasping or ungrasping an item, moving or dropping an item, rotating an item, picking up or placing an item. The transition from pose A to pose B may optionally include transitions between one or more intermediate poses.

[0031] As used herein and in the appended claims, the term "trajectory" or "trajectories" is used to refer to an ordered sequence, parameterized by time, of one or more robot poses or configurations or states through which the robot(s) or at least a portion thereof may move, e.g., to perform a task. Trajectories are preferably represented in the configuration space (also known as C-space) of each robot, but may alternatively be represented in the real space or real-world space of a shared workspace. Trajectories may include pauses, changes in direction, or even reversals in one or more poses, and are not necessarily smooth with respect to direction, time, velocity, or acceleration.

[0032] As used herein and in the appended claims, the term "nominal trajectory" or "nominal trajectories" is used to refer to a "specified" trajectory or trajectories. As used herein and in the appended claims, the term "actual trajectory" or "actual trajectories" is used to refer to a trajectory that is actually performed by the physical movement of a physical robot, as distinguished from a "nominal trajectory." In some cases, an "actual trajectory" may match a "nominal trajectory," but in many cases, there will be discrepancies between the two, for example, due to the "actual trajectory" lagging (in time) behind the "nominal trajectory."

[0033] As used herein and in the appended claims, the terms "self-collision" and "self-collisions," when used in the context of or with reference to a robotic system including two or more robots, encompass both i) a collision of one robot with itself, and ii) a collision of one robot with other robots in the robotic system. As used herein and in the appended claims, the term "self-collision," when used in the context of or with reference to a single robot, encompasses a collision of one robot with itself.

[0034] The foregoing description and summary are for convenience only and do not interpret the scope or meaning of the invention.

[0035] The approach described herein advantageously extends the safety of robotic operation beyond that guaranteed by a motion plan's nominal trajectory by determining an acceptable lag time within which a physical robot of a robotic system can operate while still ensuring that such operation of the robotic system is self-collision-free. In at least some implementations, for each robot of a robotic system operating within a shared workspace, a maximum acceptable lag time (e.g., time delay or "lag") is calculated for each of one or more nominal trajectories. This can advantageously be calculated during configuration time before the robot executes the motion plan. During online execution (i.e., runtime, following configuration time), each robot's actual motion is monitored, including each robot's actual lag time along its actual trajectory. If the actual lag time value is within a defined margin or threshold (e.g., an acceptable lag time), continued execution of the motion plan is deemed safe, since self-collision-free movement is guaranteed. If a robot's actual lag time exceeds the corresponding margin or threshold, safety is compromised, since self-collision-free movement is no longer guaranteed. For example, the safety of all robots may be deemed to be adversely affected even if only one robot experiences an actual lag time that exceeds its respective margin or threshold. Optionally, the processor-based system may select and / or take one or more corrective actions in such an event.

[0036] 1 shows a robotic system 100 including multiple robots 102a, 102b, 102c (collectively 102) operating in a shared workspace 104 (also referred to as a multi-robot environment) to perform tasks, according to one illustrated embodiment. In the robotic system 100 of FIG. 1, an acceptable lag time is determined as part of optimization, and the acceptable lag time is provided to one or more robot control systems along with an optimized motion plan including nominal trajectories for the robots.

[0037] Multiple robots can be configured to perform a set of tasks. The tasks can be specified as a task plan. A task plan can specify T tasks that need to be performed by N robots. The task plan can be modeled as a vector for each robot, where the vector is an ordered list of tasks to be performed by each robot (e.g., {task 7, task 2, task 9}). The task vector can also optionally include a dwell time duration, which specifies how long the robot or a portion thereof should dwell in a given configuration or target. The task vector can also specify a home pose and / or other "functional poses" not directly related to solving the task (e.g., a "move" or storage pose). Poses can be specified in the robot's C-space.

[0038] The robot 102 can take any of a wide variety of forms. Typically, the robot 102 takes the form of or has one or more robotic appendages 103 (only one shown) and a base 105 (only one shown) from which the robotic appendages 103 extend. The robot 102 may include one or more linkages having one or more joints and actuators (e.g., electric motors, stepper motors, solenoids, pneumatic actuators, or hydraulic actuators) coupled and operable to move the linkages in response to control or drive signals. A pneumatic actuator may include, for example, one or more pistons, cylinders, valves, a gas reservoir, and / or a pressure source (e.g., a compressor, a blower). A hydraulic actuator may include, for example, one or more pistons, cylinders, valves, a fluid reservoir (e.g., a low-compressibility hydraulic fluid), and / or a pressure source (e.g., a compressor, a pump, a blower). The robotic system 100 can use other forms of robots 102, for example, autonomous vehicles.

[0039] The shared workspace 104 typically represents a three-dimensional space in which the robots 102 can operate and move, although in certain limited implementations the shared workspace 104 can represent a two-dimensional space. The shared workspace 104 is a volume or region in which at least portions of the robots 102 may overlap or otherwise collide in space and time if movement is not controlled to avoid collisions. Note that the shared workspace 104 is a physical space or volume, position and orientation, which may be conveniently represented, for example, via Cartesian coordinates relative to some frame of reference, for example, the frame of reference represented by orthogonal axes X, Y, and Z shown in FIG. 1 . Note also that the frame of reference of the shared workspace 104 is distinct from the respective “configuration space” or “C-space” of any of the robots 102, which is typically represented by a set of joint positions, orientations, or configurations in the respective frame of reference of any of the robots 102.

[0040] As described herein, the robot 102a, or portions thereof, may constitute obstacles when viewed from the perspective of other robots 102b (i.e., when planning the motions of the other robots 102b). The shared workspace 104 may further include other obstacles, such as machinery (e.g., conveyors 106), supports, pillars, walls, ceilings, floors, tables, humans, and / or animals. The shared workspace 104 may further include one or more work items or workpieces that the robots 102 manipulate as part of performing a task, such as, for example, one or more parcels, packaging, fasteners, tools, items, or other objects.

[0041] The robotic system 100 optionally includes one or more processor-based multi-robot configuration optimization systems 108 (one is shown in FIG. 1 ). The optional multi-robot configuration optimization system(s) 108 receive a set of inputs 109 and generate as output 111 one or more solutions specifying the configuration of the robots 102, including one or more motion plans that specify or include a workcell layout (e.g., a respective base position and orientation for each robot 102), task plan(s) (e.g., a respective task plan for each robot 102), and optionally one or more nominal trajectories for the robots 102 a- c (e.g., a respective nominal trajectory for each robot 102), along with acceptable lag times for each of the nominal trajectories. One or more of the components of the output 111 may be optimized, at least to some extent.

[0042] The optional multi-robot configuration optimization system 108 can include a population generator 110, a multi-robot environment simulator 112, a multi-robot optimization engine 114, and an acceptable lag time evaluator 115.

[0043] The population generator 110 generates a set of candidate solutions 116 based on the provided input 109. The candidate solutions 116 represent possible solutions to the configuration problem (i.e., how to configure the robots 102 in the shared workspace 104 to accomplish a set of tasks). Any given candidate solution 116 may or may not actually be feasible; that is, an initial candidate may be invalid (e.g., having a robot in an impossible location, an unreachable target, or an infeasible task plan that would result in a collision). In some implementations, the population generator can attempt to find better candidate solutions.

[0044] The multi-robot environment simulator 112 models the workspace or multi-robot environment based on each candidate solution to determine certain attributes, such as the amount of time required to complete the task, the probability or rate (or speed / rate) of collisions in completing the task, and the feasibility or infeasibility of the particular configuration specified by the candidate solution. The multi-robot environment simulator 112 may reflect such in terms of cost, e.g., cost generated via one or more cost functions.

[0045] The cost or cost function may represent, for example, the probability or likelihood of a collision. The cost or cost function may optionally represent one or more of an acceptable lag time or “robustness,” a severity of a collision, an energy expenditure or wastage and / or a time or wait time to execute or complete an operation corresponding to a nominal trajectory. In some implementations, the cost or cost function represents a determined acceptable lag time for a given nominal trajectory for a given robot. The determined acceptable lag time represents a maximum, near-maximum, or optimized lag time that can be incurred when executing the respective nominal trajectory while maintaining, guaranteeing, or even committing to at least self-collision-free movement for itself and other robots operating within a shared workspace or workcell. Thus, the lag time can represent the amount of delay that can be introduced into or that can occur when actually executing the nominal trajectory without giving up a safety factor (e.g., self-collision-free operation), thus increasing the robustness of the corresponding motion plan. The safety factor can be checked or verified, for example, via a simulation of the robot's motion using a nominal trajectory, so that if the robot's actual trajectory lags behind the nominal trajectory by more than a specified margin or threshold (e.g., an acceptable lag time), collision-free motion can no longer be guaranteed.

[0046] The multi-robot optimization engine 114 evaluates candidate solutions based at least in part on associated costs 119 and advantageously co-optimizes over two or more non-homogenous parameter sets, for example, over two or more of the robots' respective base positions and orientations, the assignment of tasks to each one of the robots, the robots' respective target sequences, and / or respective trajectories or paths (e.g., collision-free trajectories or paths) between successive targets. For ease of illustration, straight-line trajectories between successive targets may be used, although the trajectories are not necessarily straight-line trajectories.

[0047] The input 109 may include one or more static environment models representing or characterizing the operating environment or shared workspace 104, for example, representing floors, walls, ceilings, pillars, other obstacles, etc. The operating environment or shared workspace 104 may be represented by one or more models, for example, geometric models (e.g., point clouds) representing the floors, walls, ceilings, obstacles, and other objects in the operating environment, which may be represented, for example, in Cartesian coordinates.

[0048] The input 109 may include one or more robot models representing or characterizing each of the robots 102, e.g., specifying the geometry and kinematics, such as dimensions or lengths, number of links, number of joints, types of joints, range of motion, velocity limits, acceleration or jerk limits, etc. The robots 102 may be represented by one or more robot geometric models that define the geometry of a given robot 102a-102c, e.g., in terms of joints, degrees of freedom, dimensions (e.g., linkage lengths), and / or in terms of the C-space of each of the robots 102a-102c.

[0049] The input 109 may include, for example, one or more sets of tasks to be performed, expressed as target objectives (e.g., poses, configurations, states, or positions or placements). The tasks may be expressed, for example, in terms of end poses, configurations, or states, and / or intermediate poses, configurations, or states, of each robot 102a-102c. The poses, configurations, or states may be defined, for example, in terms of joint positions and joint angles / rotations (e.g., joint poses, joint coordinates) of each robot 102a-102c. The input 109 may optionally include one or more dwell durations that specify a nominal amount of time a robot, or a portion thereof, should dwell on a given target to complete a task (e.g., picking and placing an object with the goal of tightening a screw or nut, sorting a stack of objects into two or more separate stacks of each type of object by two or more robots operating in a common workspace).

[0050] In some implementations, the multi-robot configuration optimization system(s) 108 can generate nominal trajectories for each robot to perform one or more tasks. A nominal trajectory is a “specified” trajectory, and each nominal trajectory includes a time-parameterized, ordered set or sequence of poses, configurations, or states for the robot between an initial or starting pose, configuration, or state and a final or ending pose, configuration, or state of the nominal trajectory, with respective timings for each pose, configuration, or state. The poses or configurations are preferably represented in the configuration space (also known as C-space) of each robot or in the real or real-world space of the workspace. Timing can be specified in relative terms (e.g., timing defined by a relative offset (or deviation) from the timing of the immediately preceding pose) or in absolute terms (e.g., timing defined by, for example, a duration from the start of the trajectory execution and, for example, with respect to a common clock). A nominal trajectory in at least some instances may specify or include one or more pauses in the motion or path of the robot or a portion thereof, and / or may specify a reversal in direction of the motion or path of the robot or a portion thereof, or may not otherwise be smooth in direction or time. Applicant notes that while any given trajectory may correspond to smooth motion of the robot, the term trajectory as used herein is not so limited and often specifies motion that is not smooth, nor does it define a straight path for the robot or a portion thereof. A nominal trajectory may, for example, specify a time-parameterized, ordered set or sequence of poses through which the robot or a portion thereof is moved to accomplish a task or portion of a task. Performing any given task may use one nominal trajectory or two or more nominal trajectories. As described herein, the actual motion or actual trajectory of the robot or a portion thereof may deviate from the respective nominal trajectory due, for example, to unexpected delays in transitioning between poses (e.g., due to a need to linger or dwell longer at (e.g., on) the target object than would otherwise be expected).

[0051] The allowable lag time evaluator 115 evaluates various candidate lag times for a given nominal trajectory to determine an allowable lag time that ensures self-collision-free operation even if the corresponding robot's actual trajectory lags the nominal trajectory by less than the allowable lag time. In a preferred approach, the allowable lag time evaluator 115 not only considers the effect(s) of lag(s) in the given robot's nominal trajectory, but also considers the effect or lag in the nominal trajectory of each of the other robots operating within the shared workspace. Thus, the allowable lag time evaluator 115 can identify an allowable lag time, or in other words, a time lag, for each nominal trajectory that assumes a worst-case scenario in which all actual trajectories of the robots operating within the shared workspace experience their respective allowable lag times. Thus, the allowable lag time evaluator 115 can, for example, determine a maximum allowable lag time for each robot that still ensures self-collision-free operation even if all robots experience their respective maximum allowable lag times. Several approaches to determining allowable lag times are described herein.

[0052] The input 109 may optionally include a limit on the number of robots that can be configured within the shared workspace 104. The input 109 may optionally include a limit on the number of tasks or targets that can be assigned to a given robot 102 a-102 c, referred to herein as task capacity, that can be configured within the shared workspace 104; for example, to limit the complexity of the configuration problem to ensure that the configuration problem is solvable or solvable within some acceptable time period using available computational resources, or to preempt certain solutions that are estimated to be too slow given an apparent over-allocation of tasks or targets to a given robot 102 a-102 c. The input 109 may optionally include one or more bounds or constraints on variables or other parameters. The input 109 may optionally include a limit on the total number of iteration cycles or time for iterations that can be used in refining candidate solutions, for example, to ensure that the configuration problem is solvable or solvable within some acceptable time period using available computational resources.

[0053] The robotic system 100 may optionally include one or more robot control systems 118 (only one shown in FIG. 1 ) communicatively coupled to control the robot 102. The robot control system(s) 118 may, for example, provide control signals (e.g., drive signals) to various actuators to move the robot 102 between various configurations to various designated targets to perform designated tasks.

[0054] The robotic system 100 may optionally include one or more motion planners 120 (only one shown in FIG. 1 ) communicatively coupled to control the robot 102. The motion planner(s) 120 create, generate, select, or refine motion plans for the robot 102, as described elsewhere herein, for example, to account for slight time deviations with respect to the motion plans provided by the multi-robot optimization engine 114 or to account for the unexpected appearance of obstacles (e.g., a human entering the operating environment or the shared workspace 104). The optional motion planner 120 may operate to dynamically create motion plans that cause the robot 102 to perform tasks in the operating environment. The motion planner 120, as well as other structure and / or operation, may be those described in U.S. Patent Application No. 62 / 865,431, filed June 24, 2019.

[0055] If included, the motion planner 120 is optionally communicatively coupled to receive as input sensory data, for example, provided by a perception subsystem (not shown). The sensory data describes static and / or dynamic objects within the shared workspace 104 that are not known a priori. The sensory data may be raw data, such as sensed via one or more sensors (e.g., camera, stereo camera, time-of-flight camera, LIDAR), and / or converted into digital representations of obstacles by the perception subsystem, which may generate respective discretizations of representations of the environment in which the robot 102 operates to perform various different scenario tasks.

[0056] Various communication paths are illustrated in FIG. 1 as lines between various structures, and in some cases, arrows indicating the direction of inputs 109 and outputs 111. The communication paths may take the form of, for example, one or more wired communication paths (e.g., electrical conductors, signal buses, or optical fibers) and / or one or more wireless communication paths (e.g., via RF or microwave radios and antennas, infrared transceivers). Communication channels may include, for example, one or more transmitters, receivers, transceivers, radios, routers, wired ports, such as Ethernet ports, etc. The general operation of the robotic system 100, and in particular the general operation of the multi-robot configuration optimization system 108, is shown and described in International Patent Application PCT / US2021 / 013610, published as WO 2021 / 150439, and will not be repeated here for the sake of brevity. Only some of the more significant differences in operation are described herein, such as the multi-robot configuration optimization system 108 performing various methods described herein to determine an acceptable lag time and determining the generation or selection of a motion plan based, at least in part, on the determined acceptable lag time, and / or the robot control system(s) 118 performing various methods described herein to monitor the actual lag time of the robot executing the motion plan, compare the actual lag time to a margin or threshold (e.g., an acceptable lag time), and / or control the robot accordingly, for example, by triggering one or more corrective actions if one or more actual lag times exceed one or more acceptable lag times or associated thresholds.

[0057] FIG. 2 illustrates a robotic system in which a first robotic control system 200a includes a first motion planner 204a that generates a first motion plan 206a for controlling the motion of a first robot 202, and optionally provides the first motion plan 206a and / or a representation of the motion as an obstacle to another motion planner 204b in another robotic control system 200b via at least one communication channel (e.g., a transmitter, receiver, transceiver, radio, router, Ethernet, indicated by proximity arrows) to control the other robot (not shown in FIG. 2), according to one example implementation. In the robotic control systems 200a, 200b of FIG. 2, an acceptable lag time is determined by the motion planner along with the motion plans 206a, 206b that include one or more nominal trajectories for execution by the robot(s) 202. In contrast to the robotic control system 100 (FIG. 1), the robotic control systems 200a, 200b of FIG. 2 do not necessarily perform optimization of the workcell layout or task planning. Additionally, the robotic control systems 200a, 200b of FIG. 2 monitor the actual lag time as described herein and optionally select and / or take corrective action, if necessary.

[0058] Similarly, the other motion planner 204b of the other robot control system(s) 200b generates and optionally provides the other motion plan 206b to the first motion planner 204a and other of the other motion planners 204b of the other robot control system 200b. The motion planners 204a, 204b can also optionally receive motion completion messages 209 indicating when the motions of the various robots 202 are completed. This may allow the motion planners 204a, 204b to generate new or updated motion plans based on the current or updated status of the shared workspace. For example, after the first robot 202 completes an action that is part or all of a set of actions to be formed as part of the first robot's 202 completing a task, a portion of the shared workspace may be blocked, unblocked, or otherwise available for a second robot to perform a task. Additionally or alternatively, the motion planners 204a, 204b can receive information (e.g., images, occupancy grids, joint positions, and joint angles / rotations) collected by various sensors or generated by other motion planners 204b that indicates when a portion of the shared workspace may be blocked, unblocked, or otherwise available for a second robot to perform a task after the first robot 202 completes an action that is part or all of a set of actions to be formed as part of the completion of a task by the first robot 202.

[0059] As described herein, the motion plans 206a, 206b specify a nominal trajectory for each robot, for example, to perform one or more tasks. As previously explained, a nominal trajectory is a "specified" trajectory, where each nominal trajectory includes a set or sequence of time-parameterized, ordered poses, configurations, or states for the robot, with respective timing for each pose, configuration, or state. The poses, configurations, or states are preferably represented in the configuration space (also known as C-space) of each robot.

[0060] The robot control systems 200a, 200b are optionally communicatively coupled, for example, via at least one communication channel (e.g., transmitter, receiver, transceiver, radio, router, Ethernet, indicated by proximity arrows) and can optionally receive the motion planning graph 208 and / or swept volume representation 211 from one or more sources 212 of the motion planning graph 208 and / or swept volume representation 211. The source(s) 212 of the motion planning graph 208 and / or swept volume representation 211 may be separate and distinct from the motion planners 204a, 204b, according to one illustrated implementation. The source(s) 212 of the motion planning graph 208 and / or swept volume representation 211 may be, for example, one or more processor-based computer systems (e.g., server computers), which may be operated or controlled by the respective manufacturers of the robots 202 or by some other entity. Each motion planning graph 208 may include a set of nodes 214 (only two are shown in FIG. 2 ) representing a pose, configuration, or state of the respective robot, and a set of edges 216 (only two are shown in FIG. 2 ) connecting each pair of nodes 214 and representing legal or valid transitions between the poses, configurations, or states. The poses, configurations, or states may be defined in the robot's respective configuration space (C-space), for example, representing a set of joint positions, orientations, poses, or coordinates for each of the joints of the respective robot 202. Thus, each node 214 may represent a pose, configuration, or state of the robot 202, or a portion thereof, as completely defined by the pose configurations or states of the joints that make up the robot 202. The motion planning graph 208 may be determined, set up, or defined before runtime (i.e., defined before the execution of a task), for example, during pre-runtime or configuration time. The optional swept volume representation 211 represents the respective volumes that the robot 202 or a portion thereof will occupy when performing movements or transitions between poses corresponding to the respective edges 216 of the motion plan graph 208 .The optional swept volume representation 211 may be represented in any of a variety of forms, for example, as voxels, a Euclidean distance field, or a hierarchy of geometric objects. This advantageously allows some of the most computationally intensive work to be performed before runtime, when responsiveness is not a particular concern. While swept volumes are used herein, such is exemplary, and any of a wide variety of other approaches to collision assessment may be used.

[0061] Each robot 202 may optionally include a base (not shown in FIG. 2 ). The base may be fixed within the environment or may be mobile therein (e.g., an autonomous or semi-autonomous vehicle). Each robot 202 may optionally include links, joints, end-of-arm tools, end effectors, and / or actuators 218 a, 218 b, 218 c (three of which are collectively designated 218) operable to move the links about the joints. A set of links, joints, end-of-arm tools, or end effectors typically comprises one or more appendages of the robot, which may be movably coupled to the robot's base. Each robot 202 may optionally include one or more motion controllers (e.g., motor controllers) 220 (only one shown) that receive control signals, for example, in the form of a motion plan 206 a, and provide drive signals for driving the actuators 218. Alternatively, the motion controller 220 may be separate from and communicatively coupled to the robots 202. Each robot 202 may be positioned and oriented within a shared workspace based on an optimized workcell layout, for example, as described in International Patent Application PCT / US2021 / 013610, published as WO 2021 / 150439.

[0062] There may be a respective robot control system 200a, 200b for each robot 202 (only one robot is shown in FIG. 2), or one robot control system 200a may execute the motion plans for more than one robot 202. A first robot control system 200a is described in detail for purposes of explanation. Those skilled in the art will recognize that this description may be applied to similar or even identical additional instances of other robot control systems 200b.

[0063] The first robotic control system 200a may include one or more processor(s) 222 and one or more associated non-transitory computer-readable or processor-readable storage media, such as a system memory 224a, a disk drive(s) 224b, and / or memory or registers (not shown) of the processor(s) 222. The non-transitory computer-readable or processor-readable storage media (e.g., the system memory 224a, the disk drive(s) 224b) are communicatively coupled to the processor(s) 222a via one or more communication channels, such as a system bus 234. The system bus 234 may employ any known bus structure or architecture, including a memory bus with a memory controller, a peripheral bus, and / or a local bus. One or more of such components may also, or instead, communicate with each other via one or more other communication channels, such as one or more parallel cables, serial cables, or high-speed wireless network channels, such as Universal Serial Bus (“USB”) 3.0, Peripheral Component Interconnect Express (PCIe), or Thunderbolt®.

[0064] The first robotic control system 200a may also be communicatively coupled to one or more remote computer systems, such as a server computer (e.g., source 212), desktop computer, laptop computer, ultraportable computer, tablet computer, smartphone, wearable computer, and / or sensor (not shown in FIG. 2 ), which are directly or indirectly communicatively coupled to various components of the first robotic control system 200a, for example, via interface 227. The remote computing system, such as a server computer (e.g., source 212), may be used to program, configure, control, or otherwise interface with or input data (e.g., motion plan graph 208, swept volume representation 211, task specification 215, candidate paths or nominal trajectories) to the first robotic control system 200a and various components within the first robotic control system 200a. Such connections may be over one or more communication channels 210, such as one or more wide area networks (WANs), e.g., Ethernet, or the Internet, using Internet Protocol. As mentioned above, the pre-runtime calculations may be performed by a system separate from the first robot control system 200a or the first robot 202, while the runtime calculations may be performed by the processor 222 of the first robot control system 200a while one or more robots are performing a task. In some implementations, one or more of the robot control systems 200a, 200b may be onboard a respective robot (e.g., the first robot 202).

[0065] As previously mentioned, the first robotic control system 200a may include one or more processors 222 (i.e., circuitry), non-transitory storage media (e.g., system memory 224a, disk drive 224b), and a system bus 234 coupling various system components. The processor 222 may be any logic processing unit, such as one or more central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), programmable logic controllers (PLCs), etc. The system memory 224a may include read-only memory ("ROM") 226, random access memory ("RAM") 228, flash memory 230, EEPROM (not shown), or any combination thereof. A basic input / output system ("BIOS") 232 may form part of the ROM 226 and contains the basic routines that help transfer information between elements within the first robotic control system 200a, such as during start-up.

[0066] Disk drive 224b may be, for example, a hard disk drive for reading from and writing to a magnetic disk, a solid-state (e.g., flash memory) drive for reading from and writing to a solid-state memory, and / or an optical disk drive for reading from and writing to a removable optical disk. First robotic control system 200a may also include any combination of such disk drives in various different embodiments. Disk drive 224b may communicate with processor 222 via system bus 234. Disk drive 224b may include an interface or controller (not shown) coupled between such a drive and system bus 234, as known to those skilled in the art. Disk drive 224b and its associated computer-readable media provide non-volatile storage of computer-readable and / or processor-readable and / or executable instructions, data structures, program modules, and other data for first robotic control system 200a. Those skilled in the art will appreciate that other types of computer-readable media capable of storing data accessible by a computer may be used, such as a WORM drive, a RAID drive, a magnetic cassette, a digital video disk ("DVD"), a Bernoulli cartridge, RAM, ROM, smart cards, etc.

[0067] Executable instructions and data may be stored in system memory 224a, such as an operating system 236, one or more application programs 238, other programs or modules 240, and program data 242. The application programs 238 may include processor-executable instructions that cause the processor 222 to do one or more of the following:

[0068] The application program 238 may include processor-executable instructions that cause the processor 222 to receive or generate a discretized representation of the shared workspace in which the robot 202 operates, including obstacles and / or target objects or workpieces in the shared workspace in which the planned movements of other robots may be represented as obstacles.

[0069] The application program 238 may include processor-executable instructions that cause the processor 222 to generate motion plans 206a, 206b that specify nominal trajectories, each of which generally defines a respective ordered sequence of poses, configurations, or states of the robot or portions thereof, parameterized by time.

[0070] The application program 238 may include processor-executable instructions that cause the processor 222 to determine respective allowable lag times relative to the nominal trajectories. The allowable lag time is generally the amount of time by which the actual motion (e.g., actual trajectory) of the robot 202 may lag behind or differ from the nominal time specified by the corresponding nominal trajectory, while still ensuring at least self-collision-free motion relative to one or more other robots operating within the shared workspace. While in some implementations self-collision-free motion assumes that the other robots do not experience any lag time in execution, preferably self-collision-free motion assumes all robots operating within the respective allowable lag times of their respective nominal trajectories, thus taking into account that each robot experiences lag time in executing its respective nominal trajectory.

[0071] To generate a motion plan, to generate a nominal trajectory, and / or to determine an acceptable lag time, the application program 238 may include processor-executable instructions that cause the processor(s) 222 to invoke or otherwise perform collision assessment. Although collision assessment is commonly referred to as "collision detection" or "collision check," such typically determines the probability or likelihood of a collision and is usually performed prior to the actual movement of the robot, rather than referring to the detection of an actual physical collision of the robot during its physical movement. Collision assessment is referred to interchangeably herein as "collision detection" or "collision check" or "collision analysis."

[0072] The application program 238 may include processor-executable instructions that cause the processor(s) 222 to set cost values ​​or cost functions for edges in a motion plan graph, for example, that reflect a determined probability or likelihood of experiencing a collision, and optionally other parameters. The application program 238 may include processor-executable instructions that cause the processor(s) 222 to set cost values ​​or cost functions for nominal trajectories, for example, that reflect a respective acceptable lag time, and alternatively, that further reflect a determined probability or likelihood of experiencing a collision, and optionally other parameters.

[0073] The application program 238 may include processor-executable instructions that cause the processor(s) 222 to evaluate available nominal trajectories generated from the motion plan graph, identify (e.g., select, determine, generate) a nominal trajectory, e.g., based on a cost or cost function, and / or identify or generate a motion plan executable by the robot(s) to cause the robot(s) to perform an action, e.g., further performance of one or more tasks by the robot(s). The application program 238 may include processor-executable instructions that cause the processor(s) 222, optionally, to store the determined motion plan and / or provide instructions to one or more robots to execute or otherwise move according to the motion plan. Motion planning and motion plan construction (e.g., collision assessment or detection, setting, e.g., updating or adjusting costs or cost functions based at least in part on collision assessment or detection, and optionally based in part on the determined acceptable lag time), and nominal trajectory generation, analysis, or evaluation of candidate nominal trajectories (e.g., selecting between two nominal trajectories based at least in part on their respective acceptable lag times) may be performed as described herein (e.g., with reference to the methods of Figures 3, 4, 5, 6, 7, and 8) and as described in the references incorporated herein by reference. Collision assessment or detection may use any of the various structures, techniques, and algorithms described herein, as well as suitable structures, techniques, and / or algorithms described elsewhere.

[0074] The application program 238 may also include one or more machine-readable and machine-executable instructions that cause the processor(s) 222 to monitor robot motion (e.g., actual trajectories) and evaluate the actual delay or lag time of these actual trajectories compared to a margin or threshold (e.g., based on an acceptable lag time or tolerable lag time) of the corresponding nominal trajectories. The application program 238 may also optionally include one or more machine-readable and machine-executable instructions that cause the processor(s) 222 to select and / or take corrective action(s) (e.g., slow down one or more robots, stop one or more robots, and / or accelerate one or more robots) as necessary (e.g., if the actual lag time approaches or exceeds an acceptable lag time).

[0075] The application program 238 may also optionally include one or more machine-readable and machine-executable instructions that cause the processor 222 to monitor the robot in an environment to determine when a path along the actual trajectory is unblocked or cleared, and, in response to the path along the actual trajectory being unblocked or cleared, to move the robot toward a goal.

[0076] The application program 238 may further include one or more machine-readable and machine-executable instructions that cause the processor 222 to perform other operations, for example, optionally process sensory data (captured via the sensors). The application program 238 may further include one or more machine-executable instructions that cause the processor 222 to perform various other methods described herein and in the references incorporated herein by reference.

[0077] In various embodiments, one or more of the above-described operations may be performed by one or more remote processing devices or computers linked via a communication channel 210 (e.g., a network) via interface 227.

[0078] While shown in FIG. 2 as being stored in system memory 224a, operating system 236, application programs 238, other programs / modules 240, and program data 242 may be stored on other non-transitory computer-readable or processor-readable media, such as disk drive 224b.

[0079] The motion planner 204a of the first robot control system 200a may include dedicated motion planner hardware, or may be implemented in whole or in part via the processor 222 and processor-executable instructions stored in system memory 224a and / or disk drive 224b.

[0080] Motion planner 204a may include or implement a motion converter 250, a path generator 252, a collision evaluator 253, a cost setter 254, optionally a path analyzer 255, a trajectory generator 256, an acceptable lag time evaluator 257, and optionally a nominal trajectory analyzer 258. Each of these may be implemented via one or more processors (e.g., circuitry) executing logic, such as executable software instructions, firmware instructions, hardwired logic, or any combination of the like.

[0081] The motion converter 250 converts the motion of an object (e.g., other robot, human) into a representation of an obstacle. The motion converter 250 receives a motion plan 206b or other representation of motion from another motion planner 204b.

[0082] The motion converter 250 may include a trajectory predictor 251 for predicting the trajectory of a transient object (e.g., another robot, e.g., another object, including a human) when the trajectory of the transient object is unknown (e.g., when the object is another robot but a motion plan for the other robot has not been received, or when the object is not another robot but is, e.g., a human). The trajectory predictor 251 may, for example, assume that the object will continue its existing motion in all directions, velocities, and accelerations without change. The trajectory predictor 251 may, for example, consider expected changes in the object's motion or path, e.g., when the object's path will result in a collision and therefore the object may be expected to stop or change direction, or when the object's goal is known, the object may be expected to stop upon reaching the goal. The trajectory predictor 251 may, at least in some examples, use a learned behavior model of the object to predict the object's trajectory. The trajectory predictor 251 may, for example, extrapolate known motions and expected changes to generate a predicted trajectory of the transient object.

[0083] The motion converter 250 then optionally determines an area or volume corresponding to the object's known motion and / or extrapolated motion(s). For example, the motion converter can convert the motion to a corresponding swept volume, i.e., a volume swept by a corresponding robot or portion thereof as it moves or transitions between poses as represented by the motion plan, by, for example, generating a volumetric representation of the robot or portion thereof and projecting the volumetric representation along a path defined by the trajectory of the robot or portion thereof. Also, for example, the motion converter can convert the motion to a corresponding swept volume, by, for example, generating a volumetric representation of an object (e.g., a non-robotic object, e.g., a human) and projecting the volumetric representation of the object along a path defined by the object's known and / or extrapolated trajectory. Advantageously, the motion planner 204a need only queue obstacles (e.g., swept volumes) and need not determine, track, or indicate the time of the corresponding motion or swept volume. Although generally described as a motion converter 250 for the first robot 202 that can convert the motion of other robots (not shown in FIG. 2 ) into obstacles, in some implementations other robot control systems 200 b of other robots operating within a shared workspace can provide obstacle representations (e.g., swept volumes) of particular motions to the motion planner 204 a for the first robot 202.

[0084] The path generator 252 generates a path from one pose, configuration, or state (e.g., a start pose, configuration, or state) to another pose, configuration, or state (e.g., an end pose, configuration, or state; or a goal pose, configuration, or state). The path generator 252 can determine or identify one or more feasible paths from a start (e.g., a start node or start pose or start state) to a goal (e.g., a goal node or goal pose or goal state; an end node or end pose or end state). For example, the path generator 252 can determine or identify an ordered sequence of nodes in a motion plan graph, where the ordered sequence of nodes provides a complete path from the start or current node to the goal or end node (i.e., an ordered set of nodes, where each consecutive pair of nodes in the complete path has a respective valid transition between them, represented by the existence of an edge connecting the nodes of the node pair). As described above, each node can correspond to a respective pose, configuration, or state of a respective robot. The path generator 252 may use or implement any of a variety of path-finding approaches, techniques, and / or algorithms. For example, the path generator 252 may use a variety of approaches, techniques, and / or algorithms to randomly or pseudo-randomly generate paths, selecting a sequence of nodes in a motion plan graph in which a node is connected to a next node in the sequence via an edge (i.e., an edge representing a valid transition between the poses represented by the connected nodes). The path generator 252 may generate a relatively large number of candidate paths between a start node and an end node, and thus between a start pose, configuration, or state and an end pose, configuration, or state. In some implementations, the path generator 252 may determine or identify feasible paths or trajectories regardless of cost (e.g., a cost representing the probability or likelihood of experiencing a collision along the path), creating a set of feasible paths or candidate paths that may later be evaluated based at least in part on the probability or likelihood of experiencing a collision.In other cases, cost (e.g., a cost representing the probability or likelihood of experiencing a collision along a path) may be taken into consideration when the path generator 252 determines or identifies a feasible path or trajectory.

[0085] Collision evaluator 253 performs collision evaluation, also referred to as collision detection or collision analysis. In particular, collision evaluator 253 optionally performs collision evaluation as part of determining whether a candidate path, representing a transition or movement of a given robot 202 or portion thereof, as specified by a nominal trajectory, results in and is likely to result in a collision with an obstacle. As noted above, the movements of other robots may advantageously be represented as obstacles. Thus, collision evaluator 253 can determine whether the movement of one robot results in and is likely to result in a collision with another robot moving through the shared workspace.

[0086] As described herein, collision assessment, detection, or analysis may be performed not only on candidate paths but also, or alternatively, on nominal trajectories (e.g., specified trajectories), particularly on nominal trajectories with various lag times introduced (e.g., simulating actual trajectories that may lag behind the nominal trajectory). In at least some implementations, collision assessment, detection, or analysis of nominal trajectories with various lag times introduced may be performed for each of two or more robots, for example, in the case of a pair of robots. Collision assessment, detection, or analysis may be performed for each set of one or more nominal trajectories for each robot in the pair of robots. Collision assessment, detection, or analysis may be performed for each nominal trajectory with a lag time equal to zero introduced, and for each trajectory with various (e.g., one, two, or more) non-zero lag times introduced, for example, by evaluating the pair of trajectories for each permutation of the lag times of the pair of trajectories. As described herein, collision assessment, detection, or analysis need not result in a binary outcome, but rather can result in a non-binary value representing, for example, the probability or likelihood of a collision between a pair of robots resulting from a path or trajectory.

[0087] In some implementations, the collision evaluator 253 implements software-based collision evaluation, detection, or analysis, e.g., performing bounding box-bounding box collision evaluation, detection, or analysis based on a hierarchy of geometric (e.g., spherical) representations of volumes swept by a robot (e.g., the first robot 202) or portions thereof during movement. In some implementations, the collision evaluator 253 implements hardware-based collision evaluation, detection, or analysis, e.g., employing a set of dedicated hardware logic circuits to represent obstacles and streaming representations of movements through the dedicated hardware logic circuits. In hardware-based collision evaluation, detection, or analysis, the collision detector may use one or more configurable arrays of circuits, e.g., one or more FPGAs 259, and may optionally generate Boolean collision assessments.

[0088] The cost setter 254 may set, update, and / or adjust costs or cost functions associated with transitions (e.g., edges in a motion plan graph) or movements (e.g., trajectories in a motion plan). The cost setter 254 may set, update, and / or adjust costs or cost functions based, for example, at least in part, on collision assessment, detection, or analysis. For example, the cost setter 254 may set relatively high cost values ​​for edges or trajectories representing transitions or movements between nodes that result in collisions or are likely to result in collisions. Also, for example, the cost setter 254 may set relatively low cost values ​​for edges or trajectories representing transitions or movements between nodes that do not result in collisions or are unlikely to result in collisions. Setting, updating, and / or adjusting costs or cost functions may include setting, updating, or adjusting costs or cost functions logically associated with corresponding edges or trajectories via some data structure (e.g., a field in a record, a pointer in a list, a table).

[0089] In some implementations, the cost setter 254 can optionally set, update, or adjust, for example, a cost or cost function to represent, at least in part, the determined tolerable lag time for each nominal trajectory. Such advantageously enables the first robot control system 200a to select a nominal trajectory from a set of available or candidate nominal trajectories for the robot to execute to perform a given task, e.g., selecting the nominal trajectory with the longest or largest determined tolerable lag time for use in generating a motion plan that achieves more robust operation as the robot experiences real-world conditions while performing the task.

[0090] In some implementations, the cost setter 254 may additionally or alternatively set, update, or adjust a cost or cost function to represent, at least in part, one or more other parameters, such as one or more of the severity of the collision, energy consumption or depletion, and / or time or latency to execute or complete.

[0091] The optional path analyzer 255 can use the motion planning graph 208 together with the costs or cost functions to determine or identify one or more suitable paths and / or select a single path (i.e., a selected path, e.g., an optimal or optimized path). The optional path analyzer 255 can, for example, identify one or more paths that meet some specified criteria (e.g., cost within a threshold limit), or even select a single path (e.g., select a single lowest-cost path) from a set of candidate viable paths determined or identified by the path generator 252. The identified paths can be referred to as suitable paths because they represent options with an acceptably low probability or likelihood of collision. The optional path analyzer 255 can, for example, configure a least-cost path optimizer that determines a lowest-cost or relatively low-cost path between two nodes (thus between two poses, configurations, or states represented by respective nodes in the motion planning graph). The optional path analyzer 255 may use or execute any of a variety of path-finding algorithms, such as a least-cost path-finding algorithm, taking into account a cost value associated with each edge, a probability or likelihood of collision, and optionally a cost value representing one or more other parameters (e.g., severity of collision, energy consumption or depletion, and / or time or wait to execute or complete). In some implementations, cost-based optimization may alternatively or additionally be applied to the nominal trajectory, advantageously allowing, for example, an acceptable lag time to be represented in a cost value or cost function, as described herein, in addition to the probability or likelihood of collision, and optionally in addition to or instead of one or more of severity of collision, energy consumption or depletion, and / or time or wait to execute or complete.

[0092] The nominal trajectory generator 256 can generate a nominal trajectory that the robot can follow, for example, to complete a task. As described herein, a trajectory includes an ordered sequence of poses or configurations or states, parameterized by time, that the robot, or at least a portion thereof, can move through, for example, to complete a task. The nominal trajectory generator 256 can generate trajectories for all of the generated paths, or for only selected paths, or for only one selected path if the optional path analyzer 255 is used.

[0093] The allowable lag time evaluator 257 evaluates various candidate lag times for a given nominal trajectory to determine an allowable lag time that ensures self-collision-free operation even if the actual trajectory of the corresponding robot lags the nominal trajectory by no more than the allowable lag time. In a preferred approach, the allowable lag time evaluator 257 not only considers the effect of lag in the nominal trajectory of a given robot, but also considers the effects of lags (or effects or lags) in the nominal trajectories of each of the other robots operating within the shared workspace. Thus, the allowable lag time evaluator 257 identifies an allowable lag time for each nominal trajectory that assumes a worst-case scenario in which all actual trajectories of the robots operating within the shared workspace experience their respective allowable lag times. Thus, the allowable lag time evaluator 257 can determine, for example, an optimized (e.g., maximum) lag time for each robot that still ensures self-collision-free operation. Several approaches for determining an acceptable lag time are described herein, for example, with respect to method 300 (FIG. 3), method 400 (FIG. 4), method 500 (FIG. 5), method 600 (FIG. 6), and / or method 700 (FIG. 7).

[0094] The optional nominal trajectory analyzer 258 can use cost values ​​or cost value functions or some other objective function to determine, identify, or select one or more suitable trajectories and / or determine, identify, or select a single trajectory (i.e., a selected trajectory, e.g., an optimal or optimized trajectory). The nominal trajectory analyzer 258 can, for example, determine, identify, or select one or more trajectories that meet some specified criteria (e.g., cost within a threshold upper limit), or can even determine, identify, or select a single trajectory (e.g., the lowest cost trajectory) from a set of feasible or candidate trajectories generated by the trajectory generator 256 (e.g., determine, identify, or select the lowest cost trajectory). The nominal trajectory analyzer 258 can, for example, comprise a minimum-cost trajectory optimizer that determines the lowest-cost or relatively low-cost trajectory between two poses, configurations, or states represented by respective nodes in the motion planning graph. Nominal trajectory analyzer 258 may, for example, select a set of trajectories for all robots that optimizes robust operation, such as by maximizing the sum of all lag times across all robots for a given time period, a given task, or a given set of tasks to be performed by the robots. Nominal trajectory analyzer 258 may use or implement any of a variety of algorithms, such as a lowest-cost trajectory finding algorithm, taking into account a cost associated with each trajectory, where the cost or cost function represents the probability or likelihood of a collision and the acceptable lag time, and optionally represents one or more of the severity of the collision, the energy consumption or waste, and / or the time or wait time to execute or complete.

[0095] Various algorithms and structures can be used to determine the minimum-cost path and / or minimum-cost trajectory, including those implementing the Bellman-Ford algorithm, although others may also be used, including, but not limited to, any such process in which a minimum-cost path or minimum-cost trajectory is determined as a path between two nodes in the motion planning graph 208, or as a trajectory specified by a time-parameterized ordered sequence of poses such that the sum of the costs or weights of its component edges or movements is minimized. This process improves the motion planning technique for the robot 102 (FIG. 1), 202 (FIG. 2) by determining an acceptable lag time, optionally selecting a trajectory based on the determined acceptable lag time, monitoring the actual trajectory to evaluate whether the actual lag time approaches or exceeds a margin or threshold (e.g., the acceptable lag time), and optionally selecting and / or taking corrective action as needed.

[0096] Although not shown, motion planner 204a can optionally include a look-ahead evaluator that can cause remedial action to be taken, for example, in response to a determination of the existence or current existence of a blocking or near-blocking condition, or a determination that a blocking or near-blocking position will occur if the given robot and / or other robots move along their respective trajectories. In at least some implementations, the look-ahead evaluator can determine or select the type of remedial action to be taken, for example, selecting from a set of different types of remedial actions based on one or more criteria. As described in U.S. Patent Application No. 63 / 327,917, filed April 6, 2022, one or more various types of remedial actions (referred to herein as corrective actions) can be implemented. For example, a new, modified, or replacement first motion plan can be generated to move the given robot to the first target based on an analysis of a second motion plan to move the given robot from the first target. Also, for example, a new, modified, or replacement motion plan may be generated for another robot that is or is likely to be blocking the given robot. Also, for example, a new order of the set of goals may be determined or generated, the set of goals including a first goal and at least a second goal. Such may be determined or generated pseudo-randomly or based on one or more heuristics (e.g., always attempting to move one position of the first goal downstream with respect to the order of the set of goals being modified).

[0097] Although not shown, motion planner 204a may optionally include an optional multi-path analyzer that can analyze the total or aggregate costs associated with the trajectories of two or more motion plans (e.g., the total cost of a first motion plan and a second motion plan), as described, for example, in U.S. patent application Ser. No. 63 / 327,917, filed April 6, 2022. Such may be used to identify a combination of motion plans having the lowest overall cost. The multi-path analyzer may, for example, consider the total or aggregate costs of two or more options for a first motion plan in conjunction with a second motion plan, using or implementing any of a variety of lowest-cost-finding algorithms, taking into account cost values ​​that represent the likelihood of an associated collision, and optionally one or more of the acceptable lag time, the severity of the collision, the energy consumption or drain, and / or the time or wait time to execute or complete the transition represented by each edge.

[0098] Motion planner 204a may optionally include a pruner 260. Pruner 260 may receive information representing the completion of a motion by another robot, referred to herein as a motion completion message 209. Alternatively, a flag may be set to indicate completion. In response, pruner 260 may remove the obstacle or portion of an obstacle representing the now-completed motion. This may enable the generation of a new motion plan for the given robot, which may be more efficient, or may allow the given robot to participate in the performance of a task previously prevented by the motion of another robot. This approach advantageously allows motion converter 250 to ignore the timing of the motion when generating the obstacle representation of the motion, while still achieving better throughput than using other techniques. The motion planner 204a may further send signals, prompts, or triggers to cause the collision evaluator 253 to perform new collision detection or evaluation given the obstacle modifications, generate an updated motion plan graph with modified edge weights or costs associated with the edges, and cause the cost setter 254, optional path analyzer 255, and nominal trajectory analyzer 258 to update cost values ​​and determine new or modified paths, trajectories, and / or motion plans accordingly.

[0099] The motion planner 204a may optionally include an environment converter 263 that converts output (e.g., a digitized representation of the environment) from optional sensors 262 (e.g., a digital camera) into representations of obstacles. Thus, the motion planner 204a can execute motion plans that take into account transient objects in the environment, such as people, animals, etc.

[0100] Processor 222 and / or motion planner 204a may be or include any logic processing unit, such as one or more central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic controllers (PLCs), etc. Non-limiting examples of commercially available computer systems include, but are not limited to, the Celeron, Core, Core2, Itanium, and Xeon families of microprocessors offered by Intel® Corporation, USA; the K8, K10, Bulldozer, and Bobcat series microprocessors offered by Advanced Micro Devices, Inc., USA; the A5, A6, and A7 series microprocessors offered by Apple Computer, Inc., USA; the Snapdragon series microprocessors offered by Qualcomm Incorporated, USA; and the SPARC series microprocessors offered by Oracle Corporation, USA. The construction and operation of the various structures shown in FIG. 2 may implement or employ structures, techniques, and algorithms described in, or similar to, International Patent Application No. PCT / US2017 / 036880, filed June 9, 2017, International Patent Application No. WO2016 / 122840, filed January 5, 2016, U.S. Patent Application No. 62 / 616,783, filed January 12, 2018, International Patent Application No. PCT / US2021 / 013610, published as WO2021 / 150439, and / or U.S. Patent Application No. 63 / 327,917, filed April 6, 2022.

[0101] Although not required, many of the implementations are described in the general context of computer-executable instructions, such as program application modules, objects, or macros, stored on a computer- or processor-readable medium and executed by one or more computers or processors capable of performing obstacle representation, collision assessment, and other motion planning operations.

[0102] Motion planning operations may include, but are not limited to, generating or converting one or more or all of the following: a representation of the robot geometry based on a geometric model, the task specification 215, and, optionally, a representation of a volume (e.g., a swept volume) occupied by the robot in various states or poses and / or while moving between states or poses; a representation in a digital format, such as, for example, a point cloud, a Euclidean distance field, a data structure format (e.g., hierarchical format, non-hierarchical format), and / or a curve (e.g., a polynomial or spline representation). Motion planning operations may optionally include, but are not limited to, generating or converting one or more or all of the sensory data representing static or permanent obstacles and / or static or transient obstacles into a digital format, such as, for example, a point cloud, a Euclidean distance field, a data structure format (e.g., hierarchical format, non-hierarchical format), and / or a curve (e.g., a polynomial or spline representation).

[0103] Motion planning operations may include, but are not limited to, using various collision assessment techniques or algorithms (e.g., software-based, hardware-based) to assess, detect, determine, or predict collisions for transitions between various poses, configurations, or states of the robot or movements of the robot between states or poses along each trajectory.

[0104] In some implementations, motion planning operations may include, but are not limited to, determining one or more motion planning graphs, motion plans, or road maps with nominal trajectories, an acceptable lag time for the nominal trajectories, storing the determined planning graph(s), motion plan(s), or road map(s), or acceptable lag times, and / or providing the planning graph(s), motion plans, or road maps, or acceptable lag times to control the motion of one or more robots, and optionally monitoring the motion of the robots and selecting and / or taking corrective action if warranted.

[0105] In one implementation, collision detection or evaluation is performed in response to a function call or similar process, which returns a Boolean value. Collision evaluator 253 can be implemented via one or more field programmable gate arrays (FPGAs) 259 and / or one or more application specific integrated circuits (ASICs) to perform collision detection with low latency, relatively low power consumption, and an increased amount of information that can be processed.

[0106] In various implementations, such operations may be performed entirely in hardware circuitry or as software stored in memory storage, such as system memory 224a, and may be performed by one or more hardware processors 222a, such as one or more microprocessors, digital signal processors (DSPs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), graphics processing unit (GPU) processors, programmed logic controllers (PLCs), electrically programmable read-only memories (EEPROMs), or as a combination of hardware circuitry and software stored in memory storage.

[0107] Various aspects of perception, planning graph construction, collision detection, and pathfinding that may be used in whole or in part are also described in International Patent Application No. PCT / US2017 / 036880, filed June 9, 2017; International Patent Application Publication No. WO2016 / 122840, filed January 5, 2016; U.S. Patent Application No. 62 / 616,783, filed January 12, 2018; U.S. Patent Application No. 62 / 616,783, filed June 3, 2019; No. 62 / 856,548, filed June 23, 2020, published as WO2020 / 263861, International Patent Application No. PCT / US2020 / 039193, filed June 23, 2020, published as WO2020 / 263861, International Patent Application No. PCT / US2021 / 013610, published as WO2021 / 150439, and / or U.S. Patent Application No. 63 / 327,917, filed April 6, 2022. Those skilled in the art will appreciate that the illustrated implementation, as well as other implementations, may be practiced with other system architectures and arrangements, including those of robots, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, personal computers (“PCs”), networked PCs, minicomputers, mainframe computers, and the like, and / or other computing system architectures and arrangements. Implementations or embodiments, or portions thereof (e.g., configuration time and runtime), may be practiced in a distributed computing environment where tasks or modules are executed by remote processing devices linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices or media. However, where and how certain types of information are stored is important to help improve operational planning.

[0108] For example, there are various motion planning solutions that "burn" a roadmap (i.e., motion planning graph) into a processor (e.g., FPGA 259), where each edge in the roadmap corresponds to non-reconfigurable Boolean circuitry in the processor. Designs in which the planning graph is "burned" into the processor pose the problem of having limited processor circuitry to store multiple or large planning graphs and are generally not reconfigurable for use with different robots.

[0109] One solution provides a reconfigurable design that puts the planning graph information into memory storage. This approach stores the information in memory instead of being burned into the circuit. Another approach uses templated reconfigurable circuits instead of memory.

[0110] As mentioned above, some of the information (e.g., the robot geometric model) may be captured, received, input, or provided during configuration time, i.e., before runtime. The received information may be processed during configuration time to create processed information (e.g., the motion plan graph 208) to accelerate operation or reduce computational complexity during runtime.

[0111] During runtime, collision detection can be performed on the entire environment (i.e., the shared workspace), including determining, for any pose or movement between poses, whether any part of the robot will collide or is predicted to collide with another part of the robot itself, other robots or parts thereof, permanent or static obstacles in the environment, or transient obstacles in the environment with unknown trajectories (e.g., people).

[0112] The first robotic control system 200a, or any other processor-based system, may include a lag monitor 264 (interchangeably referred to as a lag time monitor 264) and / or a corrective action selector 266. The lag time monitor 264 and / or corrective action selector 266 operate during runtime (e.g., during operation of one or more robots to complete one or more tasks) to monitor the operation of the robots and, optionally, select and take corrective action to slow or stop one or more robots as needed (e.g., if the actual lag time approaches or exceeds a margin or threshold, such as a corresponding acceptable lag time).

[0113] The lag time monitor 264 monitors the actual lag time of the actual trajectories executed by the robots and compares the actual lag time with a margin or threshold for the corresponding nominal trajectory (e.g., a margin or threshold representing or equal to a corresponding acceptable lag time). As previously explained, in some cases the actual trajectory may match the nominal trajectory, but in many cases the actual trajectory does not match the nominal trajectory, and typically the timing of the execution of at least some of the poses, configurations, states, or movements of the actual trajectory lags behind the timing of those poses, configurations, states, or movements specified by the nominal trajectory. The lag time monitor 264 determines the actual lag time and determines whether the monitored amount of actual lag time for the robots' actual trajectories approaches or exceeds a margin or threshold for each robot (e.g., each determined acceptable lag time for each nominal trajectory). The lag time monitor 264 can, for example, provide an indication (e.g., set a flag, send a message, and / or invoke the corrective action selector 266) if the actual lag time of any of the robots operating within the shared workspace exceeds a respective threshold (e.g., approaches or exceeds a corresponding acceptable lag time). In some implementations, the margin or threshold can be set to include a desired safety factor, such as a set amount or a defined percentage below the determined acceptable lag time for each nominal trajectory. Thus, in some implementations, the margin or threshold can be set equal to the corresponding acceptable lag time, while in other implementations the margin or threshold can be less (e.g., 10% less) than the corresponding acceptable lag time, thereby allowing detection and corrective action to be taken before a self-collision could possibly occur.

[0114] An optional corrective action selector 266 selects and / or causes the selected corrective action to be taken in response to a determination that the monitored amount of lag time exceeds a respective threshold (e.g., a determined acceptable lag time) for a respective trajectory of any robot. A determination that the actual lag time exceeds a respective threshold (e.g., a determined acceptable lag time) means that self-collision-free operation is no longer guaranteed, and therefore, implementation of one or more corrective actions may be committed.

[0115] The corrective action may include one or more of stopping the movement of one or more of the robots, slowing down the movement of one or more of the robots, and / or accelerating the movement of one or more of the robots. For example, the movement of one, two, more, or all of the robots may be stopped. For example, the movement of one, two, more, or all of the robots may be slowed down, e.g., by different amounts. For example, the movement of one, two, more, or all of the robots may be accelerated, e.g., by different amounts. For example, the movement of one or more robots may be stopped while the movement of one or more robots is being slowed down. For example, the movement of one or more robots may be stopped while the movement of one or more robots is accelerated or held constant. For example, the movement of one or more robots may be slowed down while the movement of one or more robots is accelerated or held constant. Also, for example, the movement of one or more robots may be stopped while the movement of one or more robots is decelerated and the movement of one or more other robots is accelerated or held constant. To take at least one corrective action, the processor-based system may send commands to one or more actuators (e.g., actuators 218a-218c via motion controller 220) to stop, decelerate, accelerate, or even advance the movement of the robots along their respective trajectories specified by their respective nominal trajectories. Such may, for example, be followed by a restart of the robot's movement, e.g., from the point where the movement was stopped or decelerated, but in some examples may include returning to and restarting from the start of the respective nominal trajectory.

[0116] FIG. 3 illustrates a method 300 of operation of a processor-based system for executing motion plans to control robots operating in a shared workspace, according to at least one illustrated implementation. According to at least one illustrated implementation, the processor-based system includes one or more processors that execute processor-executable instructions to determine an acceptable lag time for each robot based on the nominal trajectories of the other robots, without necessarily considering the effect of lag time on the nominal trajectories of the other robots, and generate or select a motion plan that specifies the nominal trajectories, the nominal trajectories being associated with the respective acceptable lag times. Method 300 can be performed, for example, during a configuration or “pre-run” time, i.e., the time during which some or all of the motion planning is performed to generate the nominal trajectories and / or motion plans for each of the robots. The configuration or “pre-run” time can occur before the robot movements specified by each robot’s nominal trajectory, e.g., before runtime (runtime is the time during which one or more robots are executing their respective motion plans). This advantageously allows some of the most computationally intensive work to be performed before runtime, when responsiveness is not a particular concern. In at least some implementations, the method 300, or portions thereof, may proceed to a runtime (i.e., when one or more robots perform a task (e.g., a pre-runtime) following a configuration time or pre-runtime). *訳注* Although "preform" is used, it is believed to be a misspelling of "perform" (hereinafter referred to as "perform" or "execution"). In yet another implementation, method 300 is performed during runtime (i.e., the time when one or more robots are performing (preforming) a task) following configuration time or pre-runtime.

[0117] The method 300 begins at 302, for example, in response to booting or powering up the system or a component thereof, receiving information or data, or being called or invoked by a calling routine or program.

[0118] Although not shown in Figure 3, method 300 may generate paths, perform collision assessment or detection or analysis on those paths, set, update, or adjust costs for the edges of the paths, and optionally perform analysis to identify or select paths (e.g., least-cost paths) based on cost, as described with respect to Figure 2 above, and at least some aspects of such are described in the references referenced herein. Such is omitted from method 300 for the sake of brevity.

[0119] Optionally, at 304, at least one processor of the processor-based system generates or accesses (e.g., receives, retrieves) a set of nominal trajectories for each robot I. As previously mentioned, nominal trajectories are “specified” trajectories, each of which includes an ordered set or sequence of poses, configurations, or states of the robot, the ordered set or sequence extending between two poses, configurations, or states (e.g., a starting pose, configuration, or state; an ending pose, configuration, or state), along with respective timing for each pose or configuration. The timing may be specified in relative terms (e.g., timing defined by a relative offset from the previous pose) or in absolute terms (e.g., timing defined by a relative offset from the start of the trajectory's execution relative to a common clock). Applicant notes that while any given trajectory can correspond to smooth motion of the robot, the term trajectory as used herein is not so limited and often designates non-smooth motion and does not define a straight-line path for the robot or portion thereof. A nominal trajectory in at least some cases may specify or include one or more pauses in the motion of the robot or a portion thereof, and / or may specify a change in direction or reversal of direction, a change in velocity, a change in speed, a change in acceleration of the motion or path of the robot or a portion thereof, which may not otherwise be smooth with respect to direction, speed, velocity, or time. A nominal trajectory may, for example, specify a time-parameterized, ordered set or sequence of poses through which the robot or a portion thereof is moved to perform a task or portion of a task. Performing any given task may use one nominal trajectory or two or more nominal trajectories. As described herein, the actual motion or actual trajectory of the robot or a portion thereof may deviate from the respective nominal trajectory due, for example, to unexpected delays in transitioning between poses (e.g., due to a need to linger or dwell longer than expected at (e.g., on) a target object).

[0120] Any of a wide variety of techniques and / or algorithms can be used to generate the nominal trajectory, such as, for example, a probabilistic road map (PRM) or a sampling-based motion planner (SBMP) such as a rapidly-exploring random tree (RRT, RTT*), stable sparse RRT* (SST*) and / or a Fast Matching Tree (FMT). In at least one implementation, the nominal trajectory can be generated via the systems, methods, and techniques described in International Patent Application PCT / US2021 / 013610, published as WO2021150439A1. The teachings of this patent application are not limited to any particular form of nominal trajectory generation. Advantageously, the teachings herein include computationally efficient use of nominal trajectories to generate motion plans with associated acceptable lag times, monitoring divergence of actual motion from the nominal trajectories with respect to lag times, optionally taking corrective action for deviations that exceed the respective acceptable lag times, and / or optionally using the determined lag times to select a set of nominal trajectories for the motion plan to improve robustness of operation, in any combination or permutation of these aspects.

[0121] In 306, at least one processor of the processor-based system initializes a robot counter I, e.g., sets the robot counter equal to the integer value 1. In 308, at least one processor of the processor-based system executes an outer iterative loop, performing iterations for each of two or more robots operating within the shared workspace or until a stopping condition is reached (e.g., determining a maximum lag time that ensures self-collision-free operation).

[0122] At 310, at least one processor of the processor-based system initializes a nominal trajectory counter J, for example, setting the nominal trajectory counter equal to the integer value one.

[0123] At 312, at least one processor of the processor-based system executes an inner iterative loop, performing iterations for each of the nominal trajectories for a given robot at least until a stopping condition is reached.

[0124] In 314, at least one processor of the processor-based system determines a respective allowable lag time for the current nominal trajectory (i.e., currently in the inner iterative loop) of the current robot (i.e., currently in the outer iterative loop). Each allowable lag time reflects a maximum allowable delay or lag in the current nominal trajectory J that still guarantees self-collision-free movement of the current robot relative to other robots, each moving according to their respective nominal trajectories, when the current robot executes the nominal trajectory with the corresponding allowable lag time introduced into the current robot's nominal trajectory. The allowable lag time for a given robot I guarantees self-collision freedom as long as all other robots are operating according to their respective nominal trajectories (i.e., lag time = 0). In other words, as long as each current actual lag time for a given robot is less than or equal to (i.e., less than or equal to) the robot's respective allowable lag time and the other robots are operating according to their nominal trajectories, the robots will not collide with each other (i.e., freedom of self-collision between robots is guaranteed).

[0125] In 316, at least one processor of the processor-based system determines whether each of the nominal trajectories for the given robot has been considered, such as by determining whether a nominal trajectory counter J is equal to the total number of nominal trajectories for the given robot (e.g., J=M?). If each of the nominal trajectories for the given robot has not been considered, control proceeds to 318, where the nominal trajectory counter is incremented (e.g., J=J+1), and control returns to 312 to consider the next nominal trajectory for the given robot. If each of the nominal trajectories for the given robot has been considered (e.g., J=M), control proceeds directly to 320.

[0126] Optionally, at 320, at least one processor of the processor-based system selects a maximum value of the set of determined acceptable lag times determined for the given robot.

[0127] At 322, at least one processor of the processor-based system provides one or more determined acceptable lag times (e.g., provides a selected maximum value of the determined acceptable lag times) for the given robot for use in determining a motion plan for at least the given robot and / or for controlling the motion of at least the given robot, which may include, for example, providing the determined acceptable lag time(s) to a different processor or transferring the determined acceptable lag time to a different register of the processor.

[0128] At 324, at least one processor of the processor-based system determines whether each robot has been considered, such as determining whether the robot counter I is equal to the total number of robots (e.g., I = N?). If each robot has not been considered (e.g., I < N), the control proceeds to 326, where at least one processor of the processor-based system iterates the robot counter (e.g., I = I + 1), and then the control returns to 308 to consider the next robot. If each robot has been processed (e.g., I = N), the control proceeds directly to 328.

[0129] At 328, at least one processor of the processor-based system generates or selects an operation plan for each respective robot. The operation plan may be, or represent, the nominal trajectory of the robot associated with the maximum value of the acceptable lag time. The operation plan can be, or represent, one or more nominal trajectories of the robots. This approach can advantageously enhance the robustness of the operation plan generated by motion planning, and thus improve the operation of the robot executing the resulting operation plan.

[0130] At 330, at least one processor of the processor-based system provides each respective operation plan to each robot or to the motion controller to move the robots according to their respective operation plans.

[0131] At 332, method 300 can, for example, end until called again. Although method 300 is described with respect to an ordered flow, various acts or operations are executed simultaneously or in parallel in many implementations, and / or include additional acts, and / or some acts can be omitted.

[0132] FIG. 4 illustrates a method 400 of operation of a processor-based system for executing motion plans to control robots operating within a shared workspace, according to at least one illustrated implementation. According to at least one illustrated implementation, the processor-based system includes one or more processors that execute processor-executable instructions to determine an acceptable lag time for each robot based on the nominal trajectories of the other robots, necessarily considering the effect of lag time on the nominal trajectories of the other robots, and generate or select a motion plan that specifies the nominal trajectories, the nominal trajectories being related to their respective acceptable lag times. Method 400 can be performed, for example, during a configuration or “pre-run” time, i.e., the time during which some or all of the motion planning is performed, to generate the nominal trajectories and / or motion plans for each of the robots. The configuration or “pre-run” time can occur before the robot movements specified by each robot’s nominal trajectory, e.g., before runtime (runtime is the time during which one or more robots are executing their respective motion plans). This advantageously allows some of the most computationally intensive work to be performed before runtime, when responsiveness is not a particular concern. In at least some implementations, method 400, or portions thereof, is performed during runtime (i.e., the time when one or more robots perform (preform) a task) following configuration time or pre-runtime. In still other implementations, method 400 is performed during runtime (i.e., the time when one or more robots perform (preform) a task) following configuration time or pre-runtime.

[0133] The method 400 begins at 402, for example, in response to booting or powering up the system or a component thereof, receiving information or data, or being called or invoked by a calling routine or program.

[0134] Although not shown in Figure 4, method 400 may generate paths, perform collision assessment or detection or analysis on those paths, set, update, or adjust costs for the edges of the paths, and optionally perform analysis to identify or select paths (e.g., least-cost paths) based on cost, as described with respect to Figure 2 above, and at least some aspects of such are described in the references referenced herein. Such is omitted from method 400 for the sake of brevity.

[0135] Optionally, at 404, at least one processor of the processor-based system generates or accesses (e.g., receives, retrieves) a set of nominal trajectories for each robot (i). As previously mentioned, nominal trajectories are “specified” trajectories, where each trajectory includes an ordered set or sequence of poses, configurations, or states of the robot between two poses, configurations, or states, along with the respective timing of each pose or configuration. The timing may be specified in relative terms (e.g., timing defined by a relative offset from the previous pose) or in absolute terms (e.g., timing defined by a relative offset from the start of the trajectory's execution relative to a common clock). Applicant notes that while any given trajectory may correspond to smooth motion of the robot, the term trajectory as used herein is not so limited and often designates motion that is not smooth, nor does it define a straight-line path for a robot or portion thereof. A nominal trajectory in at least some cases may specify or include one or more pauses in the motion of the robot or a portion thereof, and / or may specify a change in direction or reversal, a change in velocity, a change in speed, a change in acceleration of the motion or path of the robot or a portion thereof, which may not otherwise be smooth with respect to direction, speed, velocity, or time. A nominal trajectory may, for example, specify a time-parameterized, ordered set or sequence of poses through which the robot or a portion thereof is moved to perform a task or portion of a task. Performing any given task may use one nominal trajectory or two or more nominal trajectories. As described herein, the actual motion or actual trajectory of the robot or a portion thereof may deviate from the respective nominal trajectory due, for example, to unexpected delays in transitioning between poses (e.g., due to a need to linger or dwell longer than expected at (e.g., on) a target object).

[0136] To generate the nominal trajectory, for example, a probabilistic roadmap (PRM) or a rapid search random tree (RRT, RTT) can be used. * ), stable sparse RRT * (SST* Any of a wide variety of techniques and / or algorithms can be used, such as sampling-based motion planners (SBMPs) such as RTK (RTK-based motion planning) and / or fast matching trees (FMTs). In at least one embodiment, the nominal trajectories can be generated via the systems, methods, and techniques described in International Patent Application PCT / US2021 / 013610, published as WO2021150439A1. The teachings of this patent application are not limited to a particular form of nominal trajectory generation. Advantageously, the teachings herein include computationally efficient use of nominal trajectories to generate motion plans with associated acceptable lag times, monitoring deviations of actual motion from the nominal trajectories with respect to lag times, optionally taking corrective action for deviations exceeding each acceptable lag time, and / or optionally using the determined lag times to select a set of nominal trajectories for motion plans to improve robustness of operation, in any combination or permutation of these aspects.

[0137] At 406, at least one processor of the processor-based system initializes a robot counter I, e.g., sets the robot counter equal to the integer value 1. At 408, at least one processor of the processor-based system executes an outer iterative loop, performing iterations for each of two or more robots operating within the shared workspace or until a stopping condition is reached (e.g., determining a maximum lag time that ensures self-collision-free operation between the robots).

[0138] At 410, at least one processor of the processor-based system initializes a nominal trajectory counter J, e.g., sets the nominal trajectory counter equal to the integer value 1. At 412, the at least one processor of the processor-based system executes an inner iterative loop, performing iterations for each of the nominal trajectories of the given robot at least until a stopping condition is reached.

[0139] In 414, at least one processor of the processor-based system determines a respective allowable lag time for the current nominal trajectory (i.e., currently in the inner iterative loop) of the current robot (i.e., currently in the outer iterative loop). Each allowable lag time reflects the maximum allowable delay or lag in the current nominal trajectory J that still guarantees self-collision-free movement of the current robot relative to other robots moving according to their respective nominal trajectories, each with their respective allowable lag times introduced, when the current robot executes the nominal trajectory with the corresponding allowable lag time introduced into the nominal trajectory of the current robot. The allowable lag time for a given robot I guarantees self-collision-free movement as long as all other robots have current actual lag times that are less than their own allowable lag times. That is, the other robots do not need to be operating on their respective nominal trajectories (i.e., lag time = 0). In other words, as long as the current actual lag time of each of all robots operating within the shared workspace is less than or equal to (i.e., less than or equal to) the robot's respective allowable lag time, the robots will not collide with each other or with themselves (i.e., no self-collisions between robots are guaranteed).

[0140] In 416, at least one processor of the processor-based system determines whether each of the nominal trajectories for the given robot has been considered, such as by determining whether a nominal trajectory counter J is equal to the total number of nominal trajectories for the given robot (e.g., J=M?). If each of the nominal trajectories for the given robot has not been considered, control proceeds to 418, where the nominal trajectory counter is incremented (e.g., J=J+1), and control returns to 412 to consider the next nominal trajectory for the given robot. If each of the nominal trajectories for the given robot has been considered (e.g., J=M), control proceeds directly to 420.

[0141] Optionally, at 420, at least one processor of a processor-based system selects the maximum value of a set of determined acceptable lag times determined for a given robot. At 422, at least one processor of a processor-based system provides one or more determined acceptable lag times (e.g., the selected maximum value of the determined acceptable lag times) for a given robot for use in determining, and / or for controlling, at least the operation plan of the given robot. Such can include, for example, providing the determined acceptable lag times to different processors, or transferring the determined acceptable lag times to different registers of a processor.

[0142] At 424, at least one processor of a processor-based system determines, for example, whether each robot has been considered, such as by determining whether robot counter I is equal to the total number of robots (e.g., I = N?). If each robot has not been considered (e.g., I < N), control proceeds to 426, where at least one processor of the processor-based system iterates the robot counter (e.g., I = I + 1), and then control returns to 408 to consider the next robot. If each robot has been processed (e.g., I = N), control proceeds directly to 428.

[0143] At 428, at least one processor of a processor-based system generates or selects an operation plan for each respective robot. The operation plan may be, or represent, the nominal trajectory of the robot associated with the maximum value of the acceptable lag time. The operation plan can be, or represent, one or more of the nominal trajectories of the robots. This approach can advantageously enhance the robustness of the operation plans generated by planning the operation, and thus improve the operation of the robots executing the resulting operation plans.

[0144] At 430, at least one processor of the processor-based system provides a respective motion plan to each robot or to a motion controller to cause the robot to move according to the respective motion plan.

[0145] At 432, method 400 may end, for example, until called again. Although method 400 is described with respect to an ordered flow, various acts or operations may be performed simultaneously or in parallel in many implementations and / or may include additional acts and / or omit some acts.

[0146] As described herein, to determine an acceptable lag time, a processor-based system can perform a collision assessment to determine which of a set of candidate lag times will result in a collision or is likely to result in a collision. One approach to collision assessment is to perform the collision assessment using a swept volume for the robot. The swept volume represents the volume swept by the robot, or a portion thereof, when moving along a given trajectory. The swept volume is specified by the geometry and kinematics of the given robot and by the start and end positions specified by the given trajectory. Notably, the swept volume itself is not affected by the specific time at which the operation specified by the trajectory is performed. Therefore, the swept volume can be advantageously determined during configuration or pre-runtime, prior to run-time when the robot executes or performs a task. Thus, for each robot operating within a shared workspace, the processor-based system can generate a swept volume for the robot for each of several (e.g., multiple) paths from which trajectories are generated. The processor-based system can use the swept volume to perform the collision assessment. For example, for each robot of the two or more robots, the processor-based system may perform a collision assessment between i) at least a portion of a respective sample trajectory representing a respective nominal trajectory of the robot with at least one respective lag time introduced, and ii) at least a portion of each respective sample of a respective trajectory of each other robot of the two or more robots with at least one respective lag time introduced in the respective trajectory of each other robot of the two or more robots. In response to determining that a collision between the robot and at least one other robot of the two or more robots will occur or is likely to occur (e.g., expressed as a probability or likelihood), the processor-based system may identify as respective allowable lag times, for one or more nominal trajectories of the robots, that are shorter than the respective lag times that resulted in the determination that a collision will occur.The allowable lag time can represent the maximum allowable delay from timing relative to the poses of the nominal trajectory that still ensures that the movement of the robot through the sequence of poses specified by the nominal trajectory remains collision-free with respect to the movements of other robots of two or more robots (self-collision-free for the robot system). A non-limiting approach for determining the allowable lag time using a swept volume is described in Figures 5 and 6 below.

[0147] FIG. 5 illustrates a method 500 of operation of a processor-based system for generating swept volumes for one or more trajectories for each of multiple robots operating in a shared workspace, according to at least one illustrated implementation. The processor-based system includes one or more processors that execute processor-executable instructions to generate swept volumes, according to at least one illustrated implementation. Method 500 may be performed, for example, during configuration or “pre-run” time, which is a time during which some or all of the motion plans may be performed, to generate nominal trajectories and / or motion plans for each of the robots. Configuration or “pre-run” time, for example, may precede runtime, which is the time during which one or more robots are executing their respective motion plans. This advantageously allows some of the most computationally intensive work to be performed before runtime, when responsiveness is not a particular concern. In at least some implementations, robot configuration method 500, or portions thereof, is performed during runtime (i.e., the time during which one or more robots are performing (preforming) a task) following configuration or pre-run time. The configuration-time operations may optionally be performed via a processor-based system different from the processor-based system that performs the run-time operations. Method 500 may optionally be used, for example, when performing the methods of Figures 3, 4, and / or 6, such as as part of performing collision detection (method 600, see Figure 6) when determining the respective lag-tolerable lag times 314 (Figure 3) and 414 (Figure 4).

[0148] The method 500 for generating a swept volume begins at 502, for example, in response to booting or powering up the system or a component thereof, receiving information or data, or being called or invoked by a calling routine or program.

[0149] At 504, at least one processor of the processor-based system initializes a robot counter I (e.g., I=1). At 506, the at least one processor of the processor-based system executes an outer robot processing loop. The outer robot processing loop enables the at least one processor of the processor-based system to perform swept volume generation for each of the robots operating within the shared workspace.

[0150] At 508, at least one processor of the processor-based system initializes a trajectory counter J (e.g., J=1). At 510, at least one processor of the processor-based system executes an inner nominal trajectory processing loop. The inner nominal trajectory processing loop is nested within the outer robot processing loop. The inner nominal trajectory processing loop enables the at least one processor of the processor-based system to generate a respective swept volume for each of one or more nominal trajectories for a given robot.

[0151] In 512, at least one processor of the processor-based system generates a swept volume representation that represents a volume swept by at least a portion of a given robot I when moving through a set or sequence of poses specified by a given nominal trajectory J. Thus, for each of two or more robots, for each of one or more nominal trajectories, the processor-based system can generate a swept volume representation that represents a volume swept by at least a portion of the robot when moving through a set of sequences of poses specified by the nominal trajectory from at least one time in the nominal trajectory to another time in the nominal trajectory. Any of a wide variety of techniques can be used to generate the swept volume, including digitally representing the robot or portion thereof in one or more (e.g., hierarchical) data structures or point clouds, and projecting the digital representation along the path or trajectory to generate a digital representation of the swept volume (e.g., a set of voxels).

[0152] In 514, at least one processor in the processor-based system determines whether all trajectories for a given robot have been processed (e.g., J=M?). If all trajectories for a given robot have not been processed, control proceeds to 516, where a trajectory counter is incremented (e.g., J=J+1), after which control returns to the top of the inner nominal trajectory processing loop 510. If all nominal trajectories have been processed (e.g., J=M), control proceeds directly to 518.

[0153] In 518, at least one processor in the processor-based system determines whether all robots have been processed (e.g., I=N?). If not, control proceeds to 520, where a robot counter is incremented (e.g., I=I+1), and then control returns to the top of the outer robot processing loop 506. If all robots have been processed (e.g., I=N), control proceeds directly to 522.

[0154] At 522, method 500 may end, for example, until called again. Although method 500 is described with respect to an ordered flow, various acts or operations may be performed simultaneously or in parallel in many implementations and / or may include additional acts and / or omit some acts.

[0155] FIG. 6 illustrates a method 600 of operation of a processor-based system for performing collision assessment for multiple robots and for multiple nominal trajectories for each of the robots, according to at least one illustrated implementation. Method 600 can be used to perform collision assessment to identify feasible paths between a start node and an end node in a motion planning graph. Additionally or alternatively, method 600 can be used to perform collision assessment to determine an acceptable lag time for each of one or more nominal trajectories for robots operating in a shared workspace. A processor-based system includes, for example, one or more processors that execute processor-executable instructions for performing collision assessment using a swept volume according to at least one illustrated implementation. A swept volume represents or models, for example, a volume swept by a robot or portion thereof as it transitions between various poses, configurations, or states when transitioning between a start node and an end node along a path represented in a motion planning graph, or as it moves along a nominal trajectory, for example, from a start pose, configuration, or state to an end pose, configuration, or state. Method 600 can advantageously use, for example, the swept volume generated via method 500. Method 600 can be performed, for example, during a configuration or “pre-run” time, which is a time during which some or all of the motion planning can be performed, to generate a nominal trajectory and / or motion plan for each of the robots. The configuration or “pre-run” time can, for example, precede runtime, which is a time during which one or more robots are executing their respective motion plans. This advantageously allows some of the most computationally intensive work to be performed before runtime, when responsiveness is not a particular concern. In at least some implementations, robot configuration method 600, or portions thereof, is performed during runtime (i.e., the time during which one or more robots are performing (or preforming) a task), which follows configuration time or pre-runtime.

[0156] The method 600 for determining an acceptable lag time begins at 602, for example, in response to booting or powering up the system or a component thereof, receiving information or data, or being called or invoked by a calling routine or program.

[0157] At 604, at least one processor of the processor-based system initiates an outer robot pair collision evaluation loop. The outer robot pair collision evaluation loop enables the at least one processor of the processor-based system to evaluate or assess the probability or likelihood of a collision between robots for each pair of robots operating within the shared workspace. The outer robot pair collision evaluation loop can address each combination or permutation of robot pairs for potential collisions.

[0158] At 606, at least one processor of the processor-based system initializes a trajectory counter J (e.g., J=1). At 608, at least one processor of the processor-based system executes an inner nominal trajectory collision evaluation loop. The inner nominal trajectory collision evaluation loop is nested within the outer robot collision evaluation loop. The inner nominal trajectory collision evaluation loop enables the at least one processor of the processor-based system to evaluate or assess the probability or likelihood of a collision for each of one or more nominal trajectories of a given pair of robots.

[0159] At 610, at least one processor of the processor-based system initializes a lag time counter K (e.g., K=1). At 612, at least one processor of the processor-based system executes a nested inner lag time collision evaluation loop. The nested inner lag time collision evaluation loop is nested within the inner nominal trajectory collision evaluation loop. The nested inner lag time collision evaluation loop enables the at least one processor of the processor-based system to evaluate or assess the effect that each of a plurality of possible lag times (i.e., a candidate lag time from a set of candidate lag times) has when introduced into a given nominal trajectory. Thus, for example, nominal trajectories with a lag time equal to zero and with a non-zero number of lag times (one, two, or more) introduced can be evaluated to determine the probability or likelihood of a collision when a trajectory corresponding to the nominal trajectory with each lag time introduced is executed. The evaluation can be performed for successively longer lag times.

[0160] At 614, at least one processor of the processor-based system performs collision assessment using the generated swept volumes for corresponding trajectories with lag times (e.g., zero and non-zero lag times) introduced. The at least one processor of the processor-based system can, for example, determine whether the respective swept volumes intersect, where the respective swept volumes correspond to respective volumes swept by the robots of the pair of robots when executing a nominal trajectory with a lag time equal to zero and / or when executing nominal trajectories with each of non-zero numbers of lag times. For example, for each robot of the two or more robots, the processor-based system can perform collision assessment between i) at least a portion of a respective sample trajectory representing the robot's respective nominal trajectory with at least one respective lag time introduced (e.g., zero lag time and at least one non-zero lag time) and ii) at least a portion of each respective sample of each of the other robots of the two or more robots with at least one respective lag time introduced in the respective trajectory of each of the other robots of the two or more robots (e.g., zero lag time and at least one non-zero lag time). Thus, for example, all permutations of "candidate" lag times for each of the nominal trajectories and each of the robots can be evaluated to identify when the probability or likelihood of a collision equals or exceeds some collision detection threshold.

[0161] Although the collision assessment is described with respect to a swept volume, the teachings herein are not necessarily limited to such, and the approach and system may employ a variety of other collision assessment or detection techniques or algorithms.

[0162] In 616, at least one processor of the processor-based system determines whether a collision will occur or is likely to occur for the given robot pair based on the collision assessment. In at least some implementations, the collision assessment can generate a non-binary value representing the likelihood or probability of a collision. The determination of whether a collision will occur or is likely to occur can be based on the non-binary value exceeding a defined collision detection threshold (e.g., collision probability > 50%; > 10% > 5%). In response to determining that a collision has been detected or is likely to be detected (e.g., probability of collision equal to or greater than the collision detection threshold), the processor-based system can, in 617, identify and / or store a previous lag time value that did not result in a potential collision and, optionally, exit the nested inner lag time collision assessment loop 612, returning control to the top of the inner nominal trajectory collision assessment loop 608. Thus, for example, in response to a determination that a collision between the robot and at least one of the other robots of the two or more robots has occurred or is likely to occur, for one or more nominal trajectories of at least one of the other robots of the two or more robots for which a collision has been determined to occur, the processor-based system identifies as respective allowable lag times that are less than the respective lag times that resulted in the determination that a collision will occur, the allowable lag times reflecting a maximum tolerable delay from timing with respect to the poses of the nominal trajectory that still ensures that movement of the robot through the sequence of poses specified by the respective nominal trajectory will remain collision-free, relative to the movement of the other robot of the two or more robots. In response to a determination that a collision has been detected or is likely (e.g., the probability of collision is greater than or equal to the collision detection threshold), control proceeds directly to 618.

[0163] In 618, at least one processor in the processor-based system determines whether all lag times for the current trajectory of the current robot pair have been considered (e.g., K=P?). If not, then in 620, a lag time counter K is incremented (e.g., K=K+1) to select the next candidate lag time for evaluation from the set of candidate lag times, and control returns to the top of the nested inner lag time collision evaluation loop 612. If all lag times have been considered, control proceeds directly to 622.

[0164] At least one processor in the processor-based system determines whether all trajectories for a given robot pair have been processed (e.g., J=M?) in 622. If not, the trajectory counter is incremented (e.g., J=J+1) in 624 and control returns to the top of the inner nominal trajectory collision assessment loop 608. If all nominal trajectories have been processed (e.g., J=M), control passes to 626.

[0165] In 626, at least one processor of the processor-based system determines whether additional robot pairs remain to be processed. Method 600 may process each permutation of robot pairs, each permutation of robot pair trajectories, and each permutation of candidate lag times at least until a stopping condition is reached (e.g., an unacceptably high collision probability is found). Thus, the determined acceptable lag time (e.g., maximum lag time) for any given robot may be a function of the respective selected trajectories of each of the other robots operating within the shared workspace and the determined acceptable lag times associated with those selected trajectories. In this sense, the term maximum lag time does not necessarily refer to the absolute maximum lag time for a given robot alone, but may refer to the maximum lag time for a given robot taking into account the trajectories and associated lag times of the other robots operating within the shared workspace. If additional robot pairs remain to be processed, control proceeds to 628, where a new robot pair is selected, and then control proceeds to the top of the outer robot pair collision assessment loop. If no additional robot pairs remain to be processed, control proceeds directly to 630.

[0166] At 630, at least one processor of the processor-based system returns the acceptable lag time for each of the one or more nominal trajectories. The acceptable lag times may be stored, for example, in a non-transitory processor-readable medium, such as in one or more data structures associated with the nominal trajectories. Control then passes to 632.

[0167] At 632, method 600 may end, for example, until called again. Although method 600 is described with respect to an ordered flow, various acts or operations may be performed simultaneously or in parallel in many implementations and / or may include additional acts and / or omit some acts.

[0168] To determine each allowable lag time for the nominal trajectory, the processor-based system may, for example, iterate through each of a plurality of candidate lag times, from relatively small candidate lag times to relatively large candidate lag times, in a sequence, at least until a stopping condition is achieved, iterate through each of a plurality of times covered by the nominal trajectory, and check for a collision between one robot of the two or more robots and at least one other robot of the two or more robots based on a current one of the candidate lag times. In response to a determination that a collision between one robot of the two or more robots and at least one other robot of the two or more robots has occurred or is likely to occur, the processor-based system may set each allowable lag time for the nominal trajectory of the robot and at least one other robot of the two or more robots for which a collision has been determined to occur to the most immediate previous candidate lag time (e.g., a lag time at which the probability of collision is below a collision detection threshold indicating an acceptable risk of collision). To check for a collision, the processor-based system may, for example, determine whether a swept volume of one robot of the pair of robots intersects with a swept volume of another robot of the pair of robots. For example, the processor-based system may, in response to determining that the swept volume of one robot of a pair of robots intersects with the swept volume of another robot of the pair, generate an indication of a collision and an identification of the robots determined to be colliding.

[0169] An exemplary approach for determining and applying an acceptable lag time to provide a safety margin is provided below.

[0170] First, a motion plan including a set of nominal trajectories (Trj(t)) is generated using, for example, the optimization described in International Patent Application PCT / US2021 / 013610, published as WO2021150439A1. Thus, for each active robot r in the motion plan, an allowable lag time (e.g., a maximum safe lag value (max_lag_r)) for the motion plan is calculated. It is important to note that: (i) the allowable lag time value (max_lag) may be different for each robot r (e.g., max_lag_r≠max_lag_r); (ii) the allowable lag time value (max_lag) strictly depends on the set of nominal trajectories Trj(t) and the relative position of the robot r; and (iii) static obstacles and static robots (if any) do not affect the allowable lag time value (max_lag). In this example, a set of nominal trajectories Trj(t) and an acceptable lag time value (max_lag_r) are calculated sequentially during configuration time, preferably offline, prior to runtime, although this is not intended to limit the approach described herein. During execution (e.g., online; during runtime), a central clock (t) runs, which is common to all of the robots operating in the shared workspace. For each robot r, the lag (lag_r) relative to the nominal motion plan is calculated frequently, i.e., monitoring of the actual lag time is performed frequently enough relative to the robot's movement speed that unwanted actual collisions are avoided.

[0171] In the ideal case, each robot r is perfectly synchronized with its motion plan, and the robot's actual lag is zero (lag_r = 0). However, in the general case, one or more robots may fall behind schedule, and the current robot state is given by the trajectory trj_r(t-nΔt) and actual lag lag_r = nΔt.

[0172] The processor-based system monitors the actual lag values ​​to ensure the safety of the robotic system. If all actual lag values ​​are within a safety margin specified by the allowable lag time (e.g., lag_r≦max_lag_r), the execution is considered safe since the self-collision-free condition is guaranteed. Otherwise, the self-collision-free condition is not guaranteed and, therefore, the safety of the system is considered to be compromised.

[0173] To keep the robot in consistent, safe operation, the system can monitor the actual lag value and simply stop the movement of the entire robot system when a safety margin is exceeded (e.g., a margin or threshold is exceeded). To resume safe operation, the robot follows a set of nominal trajectories Trj(t * ), and the central clock is moved to any (safe) state of t * will be reset to.

[0174] A less disruptive solution could involve applying repair or corrective actions to robots that are within the safe boundary but relatively close to the limit. For example, a robot approaching the limit could be accelerated (increased in speed) to reduce its effective lag time. This action directly increases the distance from the limit, increasing the safety level. Alternatively, other robots could be slowed down. This action could use time dilation of the central clock to maintain consistency. Hybrid actions can also be very effective.

[0175] An exemplary processor-executable algorithm for determining the safety margin is set forth in the following pseudocode:

[0176] For better understanding, the algorithm can be divided into four components: · main_loop: Iterate the nominal trajectory Trj(t) over candidate lag values ​​from 0 to L. CheckLagCollision(): Evaluates the impact of candidate lag values ​​applied at a particular time t. It determines whether any robot is in a collision at a given time t, given a set of candidate lag values ​​for each of the robots. In a lag-free world, a robot occupies a volume determined by its current pose. Now, with lag, the robot sweeps the volume based on the trajectory it takes from its pose at time t - lag to time t. ComputeSweptVolume_r(t1,t2): Computes the volume swept by robot r while moving along the segment of the trajectory (trj_r(t1...t2)) from time t1 to time t2. VolumesIntersect(): Geometrically finds the intersection between two volumes.

[0177] Below, main_loop() and CheckLagCollision() are described in detail, while ComputeSweptVolume_r() and VolumesIntersect() are not, since many algorithms for such implementations are available.

[0178] Inputs include: trj[1...N]: a set of collision-free trajectories for N robots; ΔT: sample time; T: maximum duration of robot activity; ΔL: sample lag time value for exploration; and L: maximum lag value for exploration (L<=T).

[0179] Outputs include: max_lag[1...N] The maximum lag value per robot that ensures self-collision-free movement between robots.

[0180] The internal variables include: lag_candidate: the lag value evaluated in the current iteration; lag[1...N]: the lag value evaluated in the current iteration per robot; lag_prev[1...N]: the lag value evaluated in the previous iteration per robot; and found_max_lag[1...N]: a boolean value per robot to stop searching. TIFF0007814081000001.tif196169

[0181] FIG. 7 illustrates a method 700 of operation of a processor-based system for generating or selecting motion plans for one or more robots operating within a shared workspace based at least in part on an acceptable lag time and, optionally, at least in part on other criteria, according to at least one illustrated implementation. The lag time and, optionally, other criteria may be expressed as a cost or cost function. Method 700 may be performed, for example, during a configuration or “pre-run” time, which is a time during which some or all of the motion planning may be performed, e.g., to generate nominal trajectories and / or motion plans for each of the robots. The configuration or “pre-run” time may, for example, precede runtime, which is the time during which one or more robots are executing their respective motion plans. This advantageously allows some of the most computationally intensive work to be performed before runtime, when responsiveness is not a particular concern. In at least some implementations, method 700, or portions thereof, is performed during runtime (i.e., the time during which one or more robots are performing (preforming) a task) following the configuration or pre-run time.

[0182] The method 700 for determining an acceptable lag time begins at 702, for example, in response to booting or powering up the system or a component thereof, receiving information or data, or being called or invoked by a calling routine or program.

[0183] Optionally, at 704, at least one processor of the processor-based system generates or accesses (e.g., receives, retrieves) a nominal trajectory for each robot. As discussed above, the nominal trajectories are "specified" trajectories, where each trajectory includes an ordered sequence of poses or configurations for the robot, along with the respective timing of each pose or configuration. Also, as described herein, the actual movement or actual trajectory of the robot, or portions thereof, may deviate from the respective nominal trajectory due to, for example, unexpected delays in transitioning between poses (e.g., due to a need to linger or dwell longer at (e.g., on) the target object than would otherwise be expected).

[0184] As previously explained, to generate the nominal trajectory, for example, a probabilistic roadmap (PRM) or a rapid search random tree (RRT, RTT) can be used. * ), stable sparse RRT * (SST * Any of a wide variety of techniques and / or algorithms may be used, such as sampling-based motion planners (SBMPs) such as LAGS (Laser Based Motion Planners) and / or Fast Matching Trees (FMTs). Advantageously, the teachings herein include computationally efficient use of the determined lag time to select a set of nominal trajectories for motion planning that improves the robustness of the operation.

[0185] At 706, at least one processor of the processor-based system initializes a robot counter / , e.g., sets the robot counter equal to an integer value of 1. At 708, at least one processor of the processor-based system executes an outer robot processing iterative loop, performing iterations for each of two or more robots operating within the shared workspace, or until a stopping condition is reached (e.g., determining a maximum lag time that ensures collision-free movement).

[0186] At 710, at least one processor of the processor-based system initializes a nominal trajectory counter J, e.g., sets the nominal trajectory counter equal to the integer value 1. At 712, at least one processor of the processor-based system executes an inner nominal trajectory processing iterative loop, performing an iteration for each of the nominal trajectories for a given robot. The inner nominal trajectory processing iterative loop 712 is nested within the outer robot processing iterative loop 708.

[0187] In 714, at least one processor of the processor-based system determines a respective allowable lag time for the current nominal trajectory J for the current robot I. Each allowable lag time reflects a maximum allowable delay or lag in the current nominal trajectory J that still guarantees self-collision-free movement of the current robot I relative to the other robots that each move per their respective nominal trajectories, at least when the current robot I (the current robot of the current outer robot processing iteration loop) executes the current nominal trajectory J (the current nominal trajectory of the current inner nominal trajectory iteration loop) with a corresponding allowable lag time introduced in the current nominal trajectory J for the current robot I. Such can be evaluated against the nominal trajectories of the other robots, preferably with various candidate lag times (e.g., zero and non-zero lag times) introduced therein, or with the respective allowable lag times introduced therein, if already known. The allowable lag time for a given robot I guarantees no self-collisions as long as all other robots are operating at or within their respective nominal trajectories (i.e., lag time = 0), or at or along their respective nominal trajectories with their respective lag times introduced into the nominal trajectories of the other robots. Thus, in at least one implementation, as long as the respective current actual lag time of a given robot is less than the robot's respective allowable lag time and the other robots are operating according to their nominal trajectories, the robots will not collide with each other (no self-collisions between robots is guaranteed). Thus, in at least one preferred implementation, as long as the respective current actual lag times of all of the robots are less than the respective allowable lag times for each of the robots, the robots will not collide with each other (no self-collisions between robots is guaranteed).

[0188] In 716, at least one processor in the processor-based system determines whether each of the nominal trajectories for the given robot has been considered, e.g., determines whether a nominal trajectory counter J is equal to the total number of nominal trajectories for the given robot (e.g., J=M?). If each of the nominal trajectories for the given robot has not been considered, control proceeds to 718, where the nominal trajectory counter is incremented (e.g., J=J+1), and control returns to the top of the inner nominal trajectory processing iterative loop 712 to consider the next nominal trajectory for the given robot I. If each of the nominal trajectories for the given robot I has been considered (e.g., J=M), control proceeds directly to 720.

[0189] At 720, at least one processor of the processor-based system selects a maximum value among the set of determined acceptable lag times determined for the given robot I. At 722, at least one processor of the processor-based system provides one or more determined acceptable lag times for the given robot I (e.g., provides a selected maximum value among the determined acceptable lag times) for use in determining a motion plan for at least the given robot I. Such may include, for example, providing the determined acceptable lag time(s) to a different processor, or transferring the determined acceptable lag time(s) to a different register of a processor, or otherwise storing such in a non-transitory processor-readable medium.

[0190] In 724, at least one processor of the processor-based system determines whether each robot has been considered, for example, determines whether the robot counter I is equal to the total number of robots (e.g., I = N?). If each robot has not been considered (e.g., I < N), the control proceeds to 726, where at least one processor of the processor-based system iterates the robot counter (e.g., I = I + 1), and then the control returns to the top of the outer iterative robot processing loop 708 to consider the next robot. If each robot has been processed (e.g., I = N), the control proceeds directly to 728.

[0191] In 728, at least one processor of the processor-based system generates or selects an operation plan for each respective robot. The operation plan can be or represent the respective nominal trajectory of the robot associated with the maximum value of the acceptable lag time. Thereby, the robustness of the operation plan generated by the operation plan can be advantageously improved, and thus the operation of the robot executing the resulting operation plan can be improved.

[0192] When considering multiple trajectories per robot, the determined acceptable lag time (e.g., maximum lag time) for any given robot can be a function of the respective selected trajectories of each of the other robots operating within the shared workspace and the determined acceptable lag times associated with those selected trajectories. Thus, as described herein, in at least some implementations, the system determines an acceptable lag time for each robot and each candidate trajectory for that robot, given every other possible combination of candidate trajectories and candidate lag times for all other robots. Once completed, the system (e.g., nominal trajectory analyzer 258 of FIG. 2) can select a set of all trajectories for the robots operating within the shared workspace. The system (e.g., nominal trajectory analyzer 258 of FIG. 2) can, for example, partition (or sect) the set of trajectories by optimizing an objective function (e.g., the maximum sum of all lag times across all robots).

[0193] At 730, at least one processor of the processor-based system provides a respective motion plan to each robot or to a respective motion controller to cause the robot to move according to the respective motion plan, which may include, for example, providing a motion plan to a respective motion controller of each of the robots.

[0194] At 732, method 700 may end, for example, until called again. Although method 700 is described with respect to an ordered flow, various acts or operations may be performed simultaneously or in parallel in many implementations and / or may include additional acts and / or omit some acts.

[0195] Thus, for example, the processor-based system can determine, for each of two or more robots, and for each of two or more nominal trajectories for each robot of the two or more robots, a respective lag time for the nominal trajectory that reflects a maximum allowable delay from timing with respect to the poses of the nominal trajectory that still ensures that movement of the robot through a sequence of poses specified by the nominal trajectory is collision-free with movement through a respective sequence of poses specified by each nominal trajectory of each other robot of the two or more robots, the lag time being delayed by the respective allowable lag times for each nominal trajectory of each other robot of the two or more robots. The processor-based system can, for example, select between two or more nominal trajectories based at least in part on the respective allowable lag times of the two or more nominal trajectories, and can provide a motion plan for each robot based at least in part on the selected one of the nominal trajectories to control operation of each robot of the two or more robots. The processor-based system can, for example, select a nominal trajectory having a maximum one of the allowable lag times of the two or more nominal trajectories for each robot, resulting in a more robust motion plan than could otherwise be generated. The processor-based system can, for example, select a nominal trajectory based on a respective cost function representing an acceptable lag time and optionally a risk or probability of collision, respectively associated with each of two or more nominal trajectories for each robot.The processor-based system can, for example, select a nominal trajectory based on a respective cost function representing an acceptable lag time, a risk or probability of collision, and optionally a severity of collision, respectively associated with each of two or more nominal trajectories for each robot.The processor-based system can, for example, select a nominal trajectory based on a respective cost function representing at least one of an acceptable lag time, a risk or probability of collision, a severity of collision, and optionally a duration to completion or an expenditure of energy, each associated with each of two or more nominal trajectories for each robot, wherein each variable in the cost function representing at least one of an acceptable lag time, a risk or probability of collision, a severity of collision, and a duration to completion or an expenditure of energy is weighted in the respective cost function. The processor-based system can, for example, select a respective nominal trajectory for each of two or more robots that maximizes the acceptable lag time in a collective assembly of all of the two or more robots. The processor-based system can, for example, select a respective nominal trajectory for each of two or more robots that optimizes the respective cost function for each of the robots in a collective assembly of all of the two or more robots, each cost function representing an acceptable lag time and at least one risk or probability of collision, each associated with a respective one of two or more nominal trajectories for each robot. The processor-based system may, for example, select a respective nominal trajectory for each of two or more robots that optimizes a respective cost function for each of the robots in a collection across all robots of the two or more robots, each cost function representing an acceptable lag time, at least a risk or probability of collision, and a severity of collision, respectively associated with a respective one of the two or more nominal trajectories for the respective robot.The processor-based system may, for example, select a respective nominal trajectory for each of two or more robots that optimizes a respective cost function for each of the robots in a collection across all robots of the two or more robots, each cost function representing at least one of an acceptable lag time, at least a risk or probability of collision, a severity of the collision, and a duration to completion or energy expenditure, each associated with a respective one of the two or more nominal trajectories for a respective robot, and each variable in the cost function representing at least one of an acceptable lag time, at least a risk or probability of collision, a severity of the collision, and a duration to completion or energy expenditure is weighted in the respective cost function.

[0196] The processor-based system may, for example, determine an allowable lag time for each of the nominal trajectories that reflects a maximum allowable delay from timing relative to the poses of the nominal trajectory that still ensures that movement of the robot through the sequence of poses specified by the nominal trajectory remains self-collision-free between the two or more robots in the shared workspace, which may include performing a collision assessment for each robot of the two or more robots.

[0197] The processor-based system may, for example, perform collision assessment between i) at least a portion of each sample trajectory representing a respective nominal trajectory of a robot with at least one respective lag time introduced, and ii) at least a portion of each respective sample of a respective trajectory of each other robot of the two or more robots with at least one respective lag time introduced in the respective trajectory of each other robot of the two or more robots.

[0198] FIG. 8 illustrates a method 800 of operation of a processor-based system for controlling operation of robots operating within a shared workspace based at least in part on an acceptable lag time, according to at least one illustrated implementation. According to at least one illustrated implementation, the processor-based system includes one or more processors that monitor actual lag time compared to an acceptable lag time and, optionally, take corrective action if the actual lag time is outside a threshold. Method 800 can, for example, follow execution of methods 300 ( FIG. 3 ), 400 ( FIG. 4 ), 500 ( FIG. 5 ), 600 ( FIG. 6 ), and / or 700 ( FIG. 7 ). Method 800 can, for example, be performed during runtime, i.e., during the time one or more robots are executing their respective motion plans and / or performing tasks. Runtime can, for example, follow a configuration or “pre-run” period during which some or all of the motion planning can occur, e.g., to generate nominal trajectories and / or motion plans for each of the robots.

[0199] The method 800 begins at 802, for example, in response to booting or powering up the system or a component thereof, receiving information or data, or being called or invoked by a calling routine or program.

[0200] Optionally, at least one processor of the processor-based system accesses (e.g., receives, retrieves) a respective motion plan for each robot at 804. The respective motion plans may be stored and accessed from one or more non-transitory processor-readable media.

[0201] At 806, at least one processor of the processor-based system causes one or more robots to execute their respective motion plans. For example, the at least one processor may provide instructions to one or more motion controllers of the one or more robots. The motion controller(s) provide control signals to one or more actuators (e.g., electric motors, solenoids, valves, pumps) coupled to drive various linkages of the robots to move the robots or portions thereof.

[0202] At 808, at least one processor of the processor-based system monitors the amount of lag time (actual lag time) of each actual trajectory as it is executed by each one of the robots compared to the corresponding nominal trajectory. As previously described, an actual trajectory is the actual sequence of poses and timings for those poses executed by each robot. While in some instances the actual trajectories may match the nominal trajectories, in many instances the actual trajectories do not match the nominal trajectories, and typically the timing of the execution of at least some of the poses of one or more robots' actual trajectories lags the timing of those poses specified by the respective nominal trajectories for each robot. The monitoring can be performed in any of a variety of ways. For example, the processor-based system can use input sensed via one or more sensors positioned to sense the position and / or movement and / or pose or configuration or state of the robots within the shared workspace. The sensors can monitor the entire workspace, taking the form of, for example, a camera, video camera, stereo camera, motion detector, etc., positioned to encompass all or at least a portion of the shared workspace and having a field of view. The sensors may additionally or alternatively monitor the position and / or movement and / or pose or configuration or state of a particular robot and may include, for example, any one or more of cameras, video cameras, motion detectors, position or rotation encoders, Hall effect sensors, and / or reed switches associated with one or more joints, linkages, and / or actuators of the robot. One or more processors may perform processing (e.g., machine-vision processing) to monitor the actual trajectory.Additionally or alternatively, the processor-based system may use information from the robot control system and / or drive system (e.g., motor controllers, pneumatic controllers, hydraulic controllers, etc.) that represent the control signals (e.g., PWM motor control signals) used to drive the robot or portions thereof, and / or may use feedback signals (e.g., back EMF) received from the robot or the robot's drive system. The one or more processors may perform processing (e.g., machine vision processing) to determine the respective actual lag times relative to the actual trajectories.

[0203] At 810, at least one processor of the processor-based system determines whether the monitored amount of lag time (actual lag time) exceeds a margin or threshold (e.g., each determined acceptable lag time; a percentage of each determined acceptable lag time) for each trajectory of each of the robots operating within the shared workspace.

[0204] In response to a determination that the monitored amount of lag time (i.e., actual lag time) exceeds a respective margin or threshold (e.g., a determined acceptable lag time; a percentage of a respective determined acceptable lag time) for each trajectory of each robot for any robot, optionally, at 812, at least one processor of the processor-based system selects and / or takes one or more corrective actions.

[0205] For example, the processor-executable instructions, when executed by at least one processor of a processor-based system, to take at least one corrective action can cause the processor to one or more of: stop the movement of one or more of the robots; slow down the movement of one or more of the robots; and / or accelerate the movement of one or more of the robots. For example, the at least one processor can stop the movement of one, two, or even all of the robots. For example, the at least one processor can slow down the movement of one, two, more, or all of the robots. For example, the at least one processor can accelerate the movement of one, two, more, or all of the robots. For example, the at least one processor can stop the movement of one or more robots while slowing down the movement of one or more robots. For example, the at least one processor can stop the movement of one or more robots while accelerating or holding constant the movement of one or more robots. Also, for example, at least one processor may decelerate the movement of one or more robots while accelerating or holding constant the movement of one or more robots. Also, for example, at least one processor may stop the movement of one or more robots while decelerating the movement of one or more robots and while accelerating or holding constant the movement of one or more other robots. The term "holding movement constant" means that there is no change in the nominal trajectory of the robots, but the velocity (i.e., speed and direction) of the robot's movement may change as specified by the nominal trajectory. Also, for example, to take at least one corrective action, the processor-executable instructions, when executed by at least one processor of a processor-based system, may cause the processor to one or more of: stop the movement of one or more robots; advance one or more robots along their respective trajectories as specified by their respective nominal trajectories; and then restart the movement of the robots.Restarting a movement typically involves restarting the movement from the point where the movement was stopped, but in some instances may involve returning and restarting from the start of the respective trajectory.

[0206] Optionally, at 814, at least one processor of the processor-based system monitors the amount of lag time (actual lag time) of each actual trajectory compared to the corresponding nominal trajectory as the actual trajectory is executed by each one of the robots.

[0207] Optionally, at 816, for each of the robots operating within the shared workspace, at least one processor of the processor-based system determines whether the monitored amount of lag time (i.e., actual lag time) no longer exceeds a respective margin or threshold (e.g., determined acceptable lag time; percentage of each determined acceptable lag time) for each trajectory of each robot. The system may enter a waiting loop and continue to monitor the amount of lag time (i.e., actual lag time) until it no longer exceeds a respective margin or threshold (e.g., determined acceptable lag time; percentage of each determined acceptable lag time) for all robots. Once such condition is achieved, control proceeds to 818.

[0208] Optionally, at 818, at least one processor of the processor-based system causes one or more of the robots to continue executing their respective motion plans (eg, resume movement).

[0209] Method 800 may, for example, terminate at 820 until called again. Although method 800 is described with respect to an ordered flow, various acts or operations may be performed simultaneously or in parallel in many implementations and / or may include additional acts and / or omit some acts.

[0210] The foregoing detailed description sets forth various apparatus and / or method embodiments using block diagrams, schematic diagrams, and examples. While these block diagrams, schematic diagrams, and examples include one or more functions and / or operations, those skilled in the art will appreciate that each function and / or operation in these block diagrams, flow diagrams, and examples, individually and / or collectively, can be implemented by various hardware, software, firmware, or virtually any combination thereof. In one embodiment, the subject matter may be implemented via Boolean circuits, application-specific integrated circuits (ASICs), and / or FPGAs. However, it should be recognized that the embodiments disclosed herein may be implemented, in whole or in part, in a variety of different implementations in standard integrated circuits, as one or more computer programs executing on one or more computers (e.g., as one or more programs executing on one or more computer systems), as one or more programs executing on one or more processors (e.g., microprocessors), as firmware, or virtually any combination thereof, and that designing circuitry and / or code for software and / or firmware is well within the skill of one of ordinary skill in the art in light of the present disclosure.

[0211] Those skilled in the art will recognize that many of the methods or algorithms described herein may employ additional actions, omit some actions, and / or perform actions in a different order than specified.

[0212] Furthermore, those skilled in the art will appreciate that the mechanisms taught herein can be implemented in hardware, for example, in one or more FPGAs or ASICs.

[0213] The various embodiments described above can be combined to provide further embodiments. International Patent Application No. PCT / US2017 / 036880, filed June 9, 2017, International Patent Application No. WO2016 / 122840, filed January 5, 2016, U.S. Patent Application No. 62 / 616,783, filed January 12, 2018, U.S. Patent Application No. 62 / 626,939, filed February 6, 2018, U.S. Patent Application No. 62 / 856,548, filed June 3, 2019, U.S. Patent Application No. 62 / 865,431, filed June 24, 2019, U.S. Patent Application No. 62 / 865,431, filed January 22, 20 ... All assigned U.S. patent application publications, U.S. patent applications, and foreign patent applications described herein and / or listed in the Application Data Sheet, including, but not limited to, U.S. Patent Application No. 62 / 964,405 filed April 6, 2022, U.S. Patent Application No. 63 / 327,917 filed April 6, 2022, and International Patent Application PCT / US2021 / 013610, published as WO2021150439A1, are incorporated herein by reference in their entirety. These and other variations can be made to the embodiments in light of the above detailed description. Generally, in the following claims, the terms used should not be construed to limit the claims to the specific embodiments disclosed in the specification and the claims, but should be construed to include all possible embodiments, along with the full scope of equivalents to which such claims are entitled. Accordingly, the scope of the claims is not limited by the present disclosure.

[0214] The following is the invention as originally described in the present application. <Claim 1> 1. A method of operation in a processor-based system for facilitating the operation of multiple robots for a multi-robot operating environment in which multiple robots operate, the method comprising: For each of the two or more robots, each of one or more nominal trajectories for a respective robot of two or more robots, the nominal trajectory specifying timing for poses and a respective sequence of the poses for the respective robot; determining a respective allowable lag time for the nominal trajectory that reflects a maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of a robot through a sequence of poses specified by the nominal trajectory remains collision-free with respect to movement through a respective sequence of poses specified by a respective nominal trajectory of each of the other robots of the two or more robots; providing the acceptable lag time to at least one processor to control operation of each of the two or more robots. <Claim 2> determining each allowable lag time for the nominal trajectory, For each of the two or more robots, For each of the one or more nominal orbits: 10. The method of claim 1, comprising generating a swept volume representation that represents a volume swept by at least a portion of the robot as it moves through the set of sequences of poses specified by the nominal trajectory from at least one time within the nominal trajectory to another time within the nominal trajectory. <Claim 3> determining each allowable lag time for the nominal trajectory, The method of claim 2 , further comprising performing a collision assessment using the generated swept volume. <Claim 4> The step of performing collision assessment using the generated swept volumes includes, for each robot of the two or more robots: performing a collision assessment between i) at least a portion of a respective sample trajectory representing the respective nominal trajectory of the robot with at least one respective lag time introduced, and ii) at least a portion of each respective sample of the respective nominal trajectory of each other robot of the two or more robots with at least one respective lag time introduced into the respective trajectory of each other robot of the two or more robots; and in response to determining that a collision will occur between the robot and at least one other robot of the two or more robots, identifying as the respective allowable lag times for the one or more nominal trajectories of the robot that are shorter than the respective lag times that resulted in the determination that a collision will occur, the allowable lag times reflecting the maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of the robot through a sequence of poses specified by the nominal trajectory remains collision-free with respect to movement of the other robots of the two or more robots. <Claim 5> determining each allowable lag time for the nominal trajectory, 5. The method of claim 4, further comprising: in response to determining that a collision will occur between the robot and at least one other robot of the two or more robots, identifying as the respective allowable lag times, for the one or more nominal trajectories of at least one other robot of the two or more robots for which a collision has been determined to occur, that are shorter than the respective lag times that resulted in the determination that a collision will occur, wherein the allowable lag times reflect the maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of the robot through a sequence of poses specified by the respective nominal trajectory remains collision-free with respect to movement of the other robot of the two or more robots. <Claim 6> determining each allowable lag time for the nominal trajectory, further comprising a first iteration step and a second iteration step; the first iteration step repeats the second iteration step through each of a plurality of candidate lag times in sequence from a relatively small candidate lag time to a relatively large candidate lag time at least until a stopping condition is achieved; The second iteration step includes, through each of a plurality of times covered by the nominal trajectory: checking for a collision between one robot of the two or more robots and at least one other robot of the two or more robots based on a current one of the candidate lag times; and 2. The method of claim 1, further comprising: repeating, in response to determining that a collision will occur between one robot of the two or more robots and at least one other robot of the two or more robots, the step of setting the respective allowable lag times for the nominal trajectories of the robot and the at least one other robot of the two or more robots, where a collision has been determined to occur, to a most recent candidate lag time. Or, determining each allowable lag time for the nominal trajectory, iterating through each of a plurality of candidate lag times in sequence from relatively small candidate lag times to relatively large candidate lag times at least until a stopping condition is achieved; iterating through each of a plurality of times covered by said nominal trajectory; checking for a collision between one robot of the two or more robots and at least one other robot of the two or more robots based on a current one of the candidate lag times; and 2. The method of claim 1, further comprising: in response to determining that a collision will occur between one robot of the two or more robots and at least one other robot of the two or more robots, setting the respective allowable lag times for the nominal trajectories of the robot and the at least one other robot of the two or more robots, where a collision has been determined to occur, to a most recent candidate lag time. <Claim 7> The step of checking for a collision between the robot and the other robot based on a current one of the candidate lag times includes: For each of the two or more robots, 7. The method of claim 6, comprising performing a collision assessment for at least a portion of the respective nominal trajectories of the robots having the current one of the candidate lag times or another previously determined lag time against at least a portion of each of the respective trajectories of each of the other robots of the two or more robots having the current one of the candidate lag times. <Claim 8> 8. The method of claim 7, wherein performing a collision assessment for at least a portion of the respective nominal trajectory of the robot having the current one of the candidate lag times or another previously determined lag time, and for at least a portion of each of the respective trajectories of each of the other robots of the two or more robots having the current one of the candidate lag times, comprises performing a collision assessment to determine whether at least a portion of the robot will collide with at least a portion of any of the other robots of the two or more robots while the robot and the other robot are delayed along at least the portion of the respective nominal trajectory such that the robot and the other robot are delayed by the candidate lag time. <Claim 9> The step of checking for a collision between the robot and the other robot based on a current one of the candidate lag times includes: For each of the two or more robots, generating a swept volume representation representing a volume swept by at least a portion of the robot when moving through the set of sequences of poses specified by the respective nominal trajectories from at least one time within the respective nominal trajectories to another time within the respective nominal trajectories; For each pair of the two or more robots determining whether a swept volume of one robot of the pair of robots intersects with a swept volume of the other robot of the pair of robots for the timing relative to the pose specified by the nominal trajectory with no lag time introduced and for the timing relative to the pose specified by the nominal trajectory with each of a plurality of candidate lag times introduced into the respective nominal trajectory, at least until an intersection is detected; and 7. The method of claim 6, comprising generating an indication of a collision in response to determining that the swept volume of one robot of the pair of robots intersects the swept volume of the other robot of the pair of robots, and generating an identification of the robots determined to be in collision. <Claim 10> For each of the two or more robots, For each of one or more actual trajectories executed by the respective robot, monitoring the amount of delay between the nominal trajectory and the respective actual trajectory performed by the respective robot; determining whether the monitored amount of lag exceeds the respective determined allowable lag time for the respective actual trajectory of the respective robot; and taking at least one corrective action in response to the monitored amount of lag exceeding the respective determined acceptable lag time for the respective trajectory of the respective robot. <Claim 11> 11. The method of claim 10, wherein taking at least one corrective action comprises one or more of: stopping one or more movements of the robot; slowing down one or more movements of the robot; and accelerating one or more movements of the robot. <Claim 12> 11. The method of claim 10, wherein taking at least one corrective action comprises stopping the movement of the robots, advancing one or more of the robots along respective trajectories specified by the respective nominal trajectories, and thereafter resuming the movement of the robots. <Claim 13> 11. The method of claim 10, wherein the step of determining respective allowable lag times for the nominal trajectories occurs during a configuration time prior to movements of the at least two robots specified by the one or more nominal trajectories of the respective robots. <Claim 14> 14. The method of claim 13, wherein monitoring the amount of lag between the nominal trajectory and the respective actual trajectories performed by the respective robots occurs during runtime while the at least two robots are performing moves specified by the one or more nominal trajectories of the respective robots, the runtime following the configuration time. <Claim 15> 11. The method of claim 10, wherein monitoring the amount of lag between the nominal trajectory and the respective actual trajectory performed by the respective robot comprises using a common central clock for the monitoring of each of the robots of the one or more robots. <Claim 16> 16. The method of any combination of claims 1 to 15, further comprising receiving a respective motion plan for each of the robots, each of the motion plans specifying a respective one of the one or more nominal trajectories for the respective robot, the nominal trajectories representing a respective collision-free path. <Claim 17> 1. A processor-based system for facilitating operation of multiple robots for a multi-robot operating environment in which multiple robots operate, the processor-based system comprising: at least one processor; and at least one non-transitory processor-readable medium storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 16. <Claim 18> 1. A processor-based system for configuring a plurality of robots for a multi-robot operating environment in which the plurality of robots operate, the processor-based system comprising: at least one processor; At least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions, the processor-executable instructions, when executed by the at least one processor, causing the processor to: For each of the two or more robots, each of one or more nominal trajectories for a respective robot of the two or more robots, the nominal trajectories specifying a respective sequence of poses and timings for the poses for the respective robot; determining a respective allowable lag time for the nominal trajectory that reflects a maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of the robot through the sequence of poses specified by the nominal trajectory remains collision-free with respect to movement through the respective sequences of poses specified by the respective nominal trajectories of each of the other robots of the two or more robots; and a non-transitory processor-readable medium that causes at least one processor to perform the steps of providing the acceptable lag time to control operation of each of the two or more robots. <Claim 19> determining each allowable lag time for the nominal trajectory, For each of the two or more robots, For each of the one or more nominal orbits: 20. The processor-based system of claim 18, further comprising generating a swept volume representation that represents a volume swept by at least a portion of the robot as it moves through the set of sequences of poses specified by the nominal trajectory from at least one time within the nominal trajectory to another time within the nominal trajectory. <Claim 20> determining each allowable lag time for the nominal trajectory, The processor-based system of claim 19 , further comprising performing a collision assessment using the generated swept volume. <Claim 21> To perform collision assessment using the generated swept volumes, the at least one processor-executable instructions, when executed by the at least one processor, cause the at least one processor to, for each robot of the two or more robots: performing a collision evaluation between i) at least a portion of a respective sample trajectory representing the respective nominal trajectory of the robot with at least one respective lag time introduced, and ii) at least a portion of each of the respective samples of the respective nominal trajectory of each of the other robots of the two or more robots with at least one respective lag time introduced into the respective trajectory of each of the other robots of the two or more robots; 21. The processor-based system of claim 20, wherein in response to determining that a collision will occur between the robot and at least one other robot of the two or more robots, the processor-based system causes, for the one or more nominal trajectories of the robot, respective lag times that are shorter than the respective lag times that resulted in the determination that a collision will occur to be identified as the respective allowable lag times, the allowable lag times reflecting the maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that the movement of the robot through the sequence of poses specified by the nominal trajectory remains collision-free with respect to the movement of the other robots of the two or more robots. <Claim 22> To determine each allowable lag time for the nominal trajectory, the processor-executable instructions, when executed by the at least one processor, further cause the at least one processor to: 22. The processor-based system of claim 21, wherein in response to determining that a collision will occur between the robot and at least one other robot of the two or more robots, identifying as the respective allowable lag times, for the one or more nominal trajectories of at least one other robot of the two or more robots for which a collision will occur, that are shorter than the respective lag times that resulted in the determination that a collision will occur, the allowable lag times reflecting the maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of the robot through the sequence of poses specified by the respective nominal trajectories remains collision-free with respect to movement of the other robot of the two or more robots. <Claim 23> The processor-executable instructions, when executed by the at least one processor, further cause the processor to perform a first iteration step and a second iteration step to determine each allowable lag time for the nominal trajectory; the first iteration step repeats the second iteration step through each of a plurality of candidate lag times in sequence from a relatively small candidate lag time to a relatively large candidate lag time at least until a stopping condition is achieved; The second iteration step includes, through each of a plurality of times covered by the nominal trajectory: checking for a collision between one robot of the two or more robots and at least one other robot of the two or more robots based on a current one of the candidate lag times; and 20. The processor-based system of claim 18, wherein in response to determining that a collision will occur between one robot of the two or more robots and at least one other robot of the two or more robots, iteratively sets the respective allowable lag times for the nominal trajectories of the robot and the at least one other robot of the two or more robots where a collision has been determined to occur to a most recent candidate lag time. Or, To determine each allowable lag time for the nominal trajectory, the processor-executable instructions, when executed by the at least one processor, further cause the processor to: iterating through each of a plurality of candidate lag times in sequence from relatively small candidate lag times to relatively large candidate lag times at least until a stopping condition is achieved; iterating through each of a plurality of times covered by said nominal trajectory; checking for a collision between one robot of the two or more robots and at least one other robot of the two or more robots based on a current one of the candidate lag times; and 20. The processor-based system of claim 18, wherein in response to determining that a collision will occur between one robot of the two or more robots and at least one other robot of the two or more robots, setting the respective allowable lag times for the nominal trajectories of the robot and the at least one other robot of the two or more robots where a collision has been determined to occur to a most recent candidate lag time. <Claim 24> To check for a collision between the robot and the other robot based on a current one of the candidate lag times, the processor-executable instructions, when executed by the at least one processor, cause the processor to: For each of the two or more robots, 24. The processor-based system of claim 23, further comprising: causing collision assessment to be performed for at least a portion of the respective nominal trajectories of the robots having the current one of the candidate lag times or another previously determined lag time against at least a portion of each of the respective trajectories of each of the other robots of the two or more robots having the current one of the candidate lag times. <Claim 25> 25. The processor-based system of claim 24, wherein the processor-executable instructions, when executed by the at least one processor to perform a collision assessment for at least a portion of the respective nominal trajectory of the robot having the current one of the candidate lag times or another previously determined lag time, against at least a portion of each of the respective trajectories of each of the other robots of the two or more robots having the current one of the candidate lag times, cause the processor to perform a collision assessment to determine whether at least a portion of the robot will collide with at least a portion of any of the other robots of the two or more robots while the robot and the other robot are delayed by the candidate lag times along at least the portion of the respective nominal trajectory. <Claim 26> To check for a collision between the robot and the other robot based on a current one of the candidate lag times, the processor-executable instructions, when executed by the at least one processor, cause the processor to: For each of the two or more robots, generating a swept volume representation representing a volume swept by at least a portion of the robot when moving through the set of sequences of poses specified by the respective nominal trajectories from at least one time within the respective nominal trajectories to another time when a current one of the candidate lag times is introduced into the respective nominal trajectories; For each pair of the two or more robots, determining whether a swept volume of one robot of the pair of robots intersects with a swept volume of the other robot of the pair of robots for the timing with respect to the pose specified by the nominal trajectory without a lag time introduced and for the timing with respect to the pose specified by the nominal trajectory with each of a plurality of candidate lag times introduced into the respective nominal trajectory, at least until an intersection is detected; 24. The processor-based system of claim 23, wherein in response to determining that the swept volume of one robot of the pair of robots intersects with the swept volume of the other robot of the pair of robots, an indication of a collision is generated and an identification of the robots determined to be in collision is generated. <Claim 27> The processor-executable instructions, when executed by the at least one processor, further cause the processor to: For each of the two or more robots, For each of one or more actual trajectories executed by the respective robot, monitoring an amount of delay between the nominal trajectory and the respective actual trajectory executed by the respective robot; determining whether the monitored amount of delay exceeds the respective determined allowable lag time for the respective actual trajectory of the respective robot; 20. The processor-based system of claim 18, wherein the processor-based system causes at least one corrective action to be taken in response to the monitored amount of lag exceeding the respective determined allowable lag time for the respective trajectory of the respective robot. <Claim 28> 28. The processor-based system of claim 27, wherein the processor-executable instructions, when executed by the at least one processor, cause the processor to perform one or more of the following to take at least one corrective action: stop one or more movements of the robot, slow down one or more movements of the robot, or accelerate one or more movements of the robot. <Claim 29> 28. The processor-based system of claim 27, wherein the processor-executable instructions, when executed by the at least one processor, cause the processor to stop movement of the robots, advance one or more of the robots along respective trajectories specified by the respective nominal trajectories, and then resume the movement of the robots, to take at least one corrective action. <Claim 30> 28. The processor-based system of claim 27, wherein the determination of each allowable lag time for the nominal trajectory occurs during a configuration time prior to a movement of the at least two robots specified by the one or more nominal trajectories of the respective robots. <Claim 31> 31. The processor-based system of claim 30, wherein the monitoring of the amount of delay between the nominal trajectory and the respective actual trajectories performed by the respective robots occurs during runtime while the at least two robots are performing moves specified by the one or more nominal trajectories of the respective robots, the runtime following the configuration time. <Claim 32> 28. The processor-based system of claim 27, wherein the monitoring of the amount of delay between the nominal trajectory and the respective actual trajectory performed by the respective robot includes using a common central clock for the monitoring of each of the robots of the one or more robots. <Claim 33> The processor-executable instructions, when executed by the at least one processor, further cause the processor to: 33. The processor-based system of any combination of claims 18 to 32, further comprising: receiving a respective motion plan for each of the robots, each of the motion plans specifying a respective one of the one or more nominal trajectories for the respective robot, the nominal trajectories representing a respective collision-free path. <Claim 34> 1. A method of operation in a processor-based system for facilitating the operation of multiple robots for a multi-robot operating environment in which multiple robots operate, the method comprising: For each of the two or more robots, each of two or more nominal trajectories for a respective robot of two or more robots, the nominal trajectories specifying a respective sequence of poses and timings for the poses for the respective robot; determining a respective allowable lag time for the nominal trajectory that reflects a maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of the robot through the sequence of poses specified by the nominal trajectory remains collision-free with respect to movement through the respective sequences of poses specified by the respective nominal trajectory of each of the other robots of the two or more robots delayed by a respective allowable lag time for the nominal trajectory of at least one other robot of the two or more robots; selecting between the two or more nominal trajectories based at least in part on the respective allowable lag times for the two or more nominal trajectories; and providing a motion plan for the respective robot based at least in part on the selected one of the nominal trajectories to control motion of the respective robot of the two or more robots. <Claim 35> 35. The method of claim 34, wherein selecting between the two or more nominal trajectories based at least in part on the respective allowable lag times for the two or more nominal trajectories for each robot comprises selecting a nominal trajectory having a largest one of the allowable lag times of the two or more nominal trajectories for the respective robot. <Claim 36> 35. The method of claim 34, wherein selecting between the two or more nominal trajectories based at least in part on the respective acceptable lag times for the two or more nominal trajectories for each robot comprises selecting the nominal trajectories based on an acceptable lag time and a respective cost function representing at least a risk or probability of collision, respectively associated with each of the two or more nominal trajectories for the respective robot. <Claim 37> 35. The method of claim 34, wherein selecting between the two or more nominal trajectories based at least in part on the respective acceptable lag times for the two or more nominal trajectories for each robot comprises selecting the nominal trajectories based on respective cost functions representing an acceptable lag time, a risk or probability of collision, and a severity of collision, respectively associated with each of the two or more nominal trajectories for the respective robot. <Claim 38> 35. The method of claim 34, wherein selecting between the two or more nominal trajectories based at least in part on the respective acceptable lag times for the two or more nominal trajectories for each robot comprises selecting the nominal trajectories based on respective cost functions associated with each of the two or more nominal trajectories for the respective robot, the cost functions representing at least one of the acceptable lag time, the risk or probability of collision, the severity of a collision, and the duration to completion or the expenditure of energy, wherein each variable in the cost functions representing at least one of the acceptable lag time, the risk or probability of collision, the severity of a collision, and the duration to completion or the expenditure of energy is weighted in the respective cost function. <Claim 39> 35. The method of claim 34, wherein selecting between the two or more nominal trajectories based at least in part on the respective allowable lag times for the two or more nominal trajectories comprises selecting a respective nominal trajectory for each of the two or more robots that maximizes the allowable lag time in a collective across all robots of the two or more robots. <Claim 40> 35. The method of claim 34, wherein selecting between the two or more nominal trajectories based at least in part on the respective acceptable lag times for the two or more nominal trajectories for each robot comprises selecting a respective nominal trajectory for each of the two or more robots that optimizes a respective cost function for each of the robot in an aggregate across all robots of the two or more robots, each cost function representing the acceptable lag time and at least a risk or probability of collision, respectively, associated with the respective nominal trajectory of a respective one of the two or more nominal trajectories for the respective robot. <Claim 41> 35. The method of claim 34, wherein selecting between the two or more nominal trajectories based at least in part on the respective acceptable lag times for the two or more nominal trajectories for each robot comprises selecting a respective nominal trajectory for each of the two or more robots that optimizes a respective cost function for each of the robot in an aggregate across all robots of the two or more robots, each cost function representing the acceptable lag time, at least a risk or probability of collision, and a severity of collision, respectively, associated with the respective nominal trajectory of a respective one of the two or more nominal trajectories for the respective robot. <Claim 42> 35. The method of claim 34, wherein selecting between the two or more nominal trajectories based at least in part on the respective acceptable lag times for the two or more nominal trajectories for each robot comprises selecting a respective nominal trajectory for each of the two or more robots that optimizes a respective cost function for each of the robot in a collection across all robots of the two or more robots, wherein each cost function represents at least one of an acceptable lag time, at least a risk or probability of collision, a severity of collision, and a duration to completion or an energy expenditure, respectively associated with the respective nominal trajectory of a respective one of the two or more nominal trajectories for the respective robot, and wherein each variable in the cost function representing at least one of the acceptable lag time, the risk or probability of collision, the severity of collision, and the duration to completion or an energy expenditure, respectively, is weighted in the respective cost function. <Claim 43> 35. The method of claim 34, wherein determining a respective allowable lag time for the nominal trajectory reflecting a maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of the robot through the sequence of poses specified by the nominal trajectory remains collision-free comprises performing a collision assessment for each robot of the two or more robots. <Claim 44> The step of performing a collision assessment for each of the two or more robots includes: 44. The method of claim 43, comprising performing a collision evaluation between i) at least a portion of a respective sample trajectory representing the respective nominal trajectory of the robot with at least one respective lag time introduced, and ii) at least a portion of each of the respective samples of the respective nominal trajectory of each of the other robots of the two or more robots with at least one respective lag time introduced into the respective trajectory of each of the other robots of the two or more robots. <Claim 45> The step of determining each allowable lag time for the nominal trajectory further comprises: 45. The method of claim 44, further comprising: in response to determining that a collision will occur between the robot and at least one other robot of the two or more robots, identifying as the respective allowable lag times for the one or more nominal trajectories of the robot that are shorter than the respective lag times that resulted in the determination that a collision will occur, the allowable lag times reflecting the maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that the movement of the robot through the sequence of poses specified by the nominal trajectory remains collision-free with respect to the movement of the other robots of the two or more robots. <Claim 46> The step of determining each allowable lag time for the nominal trajectory further comprises: 46. ​​The method of claim 45, comprising, in response to determining that a collision between the robot and at least one other robot of the two or more robots will occur, identifying as the respective allowable lag times, for one or more nominal trajectories of the at least one other robot of the two or more robots for which a collision has been determined to occur, respective lag times that are shorter than the respective lag times that resulted in the determination that a collision will occur, the allowable lag times reflecting a maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that the movement of the robot through the sequence of poses specified by the respective nominal trajectories remains collision-free with respect to the movement of the other robot of the two or more robots. <Claim 47> 1. A processor-based system for facilitating operation of multiple robots for a multi-robot operating environment in which multiple robots operate, comprising: at least one processor; and at least one non-transitory processor-readable medium storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to perform the method of any one of claims 34 to 46. <Claim 48> 1. A processor-based system for configuring a plurality of robots for a multi-robot operating environment in which the plurality of robots operate, the processor-based system comprising: at least one processor; At least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions, the processor-executable instructions, when executed by the at least one processor, causing the processor to: For each of the two or more robots, each of two or more nominal trajectories for a respective robot of two or more robots, the nominal trajectories specifying a respective sequence of poses and timings for the poses for the respective robot; determining a respective allowable lag time for the nominal trajectory that reflects a maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of a robot through a sequence of poses specified by the nominal trajectory remains collision-free with respect to movement through the respective sequences of poses specified by the respective nominal trajectory of each of the other robots of the two or more robots delayed by a respective allowable lag time for the nominal trajectory of the at least one other robot of the two or more robots; selecting between the two or more nominal trajectories based at least in part on the respective allowable lag times for the two or more nominal trajectories; providing a motion plan for each robot based at least in part on a selected one of the nominal trajectories to control motion of the each robot of the two or more robots. <Claim 49> 49. The processor-based system of claim 48, wherein the processor-executable instructions, when executed, for selecting between the two or more nominal trajectories for each robot based at least in part on the respective allowable lag times for the two or more nominal trajectories cause the at least one processor to select a nominal trajectory having a largest one of the allowable lag times of the two or more nominal trajectories for the respective robot. <Claim 50> 49. The processor-based system of claim 48, wherein the processor-executable instructions, when executed to select between the two or more nominal trajectories based at least in part on the respective allowable lag times for the two or more nominal trajectories for each robot, cause the at least one processor to select the nominal trajectory based on an allowable lag time and a respective cost function representing at least a risk or probability of collision, respectively associated with the respective nominal trajectory of the two or more nominal trajectories for the respective robot. <Claim 51> 49. The processor-based system of claim 48, wherein the processor-executable instructions, when executed to select between the two or more nominal trajectories based at least in part on the respective acceptable lag times for the two or more nominal trajectories for each robot, cause the at least one processor to select the nominal trajectory based on respective cost functions representing an acceptable lag time, a risk or probability of collision, and a severity of collision, respectively associated with the respective nominal trajectory of the two or more nominal trajectories for the respective robot. <Claim 52> 49. The processor-based system of claim 48, wherein the processor-executable instructions, when executed, to select between the two or more nominal trajectories for each robot based at least in part on the respective acceptable lag times for the two or more nominal trajectories cause the at least one processor to select the nominal trajectory based on a respective cost function representing at least one of an acceptable lag time, a risk or probability of collision, a severity of collision, and a duration to completion or an expenditure of energy, respectively associated with each of the two or more nominal trajectories for the respective robot, wherein each variable in the cost function representing at least one of the acceptable lag time, the risk or probability of collision, the severity of collision, and the duration to completion or the expenditure of energy, respectively, is weighted in the respective cost function. <Claim 53> 49. The processor-based system of claim 48, wherein the processor-executable instructions for selecting between the two or more nominal trajectories based at least in part on the respective allowable lag times for the two or more nominal trajectories, when executed, cause the at least one processor to select a respective nominal trajectory for each of the two or more robots that maximizes the allowable lag time in aggregate across all robots of the two or more robots. <Claim 54> 49. The processor-based system of claim 48, wherein the processor-executable instructions, when executed, to select between the two or more nominal trajectories based at least in part on the respective allowable lag times for the two or more nominal trajectories for each robot, cause the at least one processor to select a respective nominal trajectory for each of the two or more robots that optimizes a respective cost function for each of the robot in an aggregate across all robots of the two or more robots, each cost function representing the allowable lag time and at least a risk or probability of collision, respectively, associated with the respective one of the two or more nominal trajectories for the respective robot. <Claim 55> 49. The processor-based system of claim 48, wherein the processor-executable instructions, when executed, to select between the two or more nominal trajectories based at least in part on the respective allowable lag times for the two or more nominal trajectories for each robot cause the at least one processor to select a respective nominal trajectory for each of the two or more robots that optimizes a respective cost function for each of the robot in an aggregate across all robots of the two or more robots, each cost function representing the allowable lag time, at least a risk or probability of collision, and a severity of collision, respectively, associated with the respective nominal trajectory of a respective one of the two or more nominal trajectories for the respective robot. <Claim 56> 49. The processor-based system of claim 48, wherein the processor-executable instructions, when executed, to select between the two or more nominal trajectories based at least in part on the respective allowable lag times for the two or more nominal trajectories for each robot cause the at least one processor to select a respective nominal trajectory for each of the two or more robots that optimizes a respective cost function for each of the robot in an aggregate across all robots of the two or more robots, wherein each cost function represents at least one of the allowable lag time, at least a risk or probability of collision, a severity of collision, and a duration to completion or an expenditure of energy, respectively associated with the respective one of the two or more nominal trajectories for the respective robot, and wherein each variable in the cost function representing at least one of the allowable lag time, the risk or probability of collision, the severity of collision, and the duration to completion or the expenditure of energy, respectively, is weighted in the respective cost function. <Claim 57> 49. The processor-based system of claim 48, wherein the processor-executable instructions, when executed, cause the at least one processor to perform collision assessment for each robot of the two or more robots to determine a respective allowable lag time for the nominal trajectory that reflects a maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of the robot through the sequence of poses specified by the nominal trajectory remains collision-free. <Claim 58> To perform collision assessment for each robot of the two or more robots, the processor-executable instructions, when executed, cause the at least one processor to: 58. The processor-based system of claim 57, further comprising: i) at least a portion of each sample trajectory representing the respective nominal trajectory of the robot with at least one respective lag time introduced; and ii) at least a portion of each respective sample of each respective trajectory of each other robot of the two or more robots with at least one respective lag time introduced in the respective trajectory of each other robot of the two or more robots. <Claim 59> To determine each allowable lag time for the nominal trajectory, the processor-executable instructions, when executed, further cause the at least one processor to: 59. The processor-based system of claim 58, wherein in response to a determination that a collision between the robot and at least one other robot of the two or more robots will occur, the processor causes the one or more nominal trajectories of the robot to identify respective lag times as the respective allowable lag times that are shorter than the respective lag times that resulted in the determination that a collision will occur, the allowable lag times reflecting the maximum allowable delay from timing for the poses of the nominal trajectory that still ensures that movement of the robot through the sequence of poses specified by the nominal trajectory remains collision-free with respect to movement of the other robots of the two or more robots. <Claim 60> To determine each allowable lag time for the nominal trajectory, the processor-executable instructions, when executed, further cause the at least one processor to: 60. The processor-based system of claim 59, wherein, in response to a determination that a collision between the robot and at least one other robot of the two or more robots will occur, the processor causes, for the one or more nominal trajectories of the at least one other robot of the two or more robots for which a collision has been determined to occur, respective lag times that are shorter than the respective lag times that resulted in the determination that a collision will occur to be identified as the respective allowable lag times, the allowable lag times reflecting the maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of the robot through the sequence of poses specified by the respective nominal trajectories remains collision-free with respect to movement of the other robot of the two or more robots.

Claims

1. 1. A method of operation in a processor-based system for facilitating operation of multiple robots for a multi-robot operating environment in which multiple robots operate, the method of operation comprising: For each of the two or more robots, each of one or more nominal trajectories for a respective robot of the two or more robots, the nominal trajectory specifying timing for poses and a respective sequence of the poses for the respective robot; determining a respective allowable time lag for the nominal trajectory that reflects a maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of a robot through a sequence of poses specified by the nominal trajectory remains collision-free with respect to movement through a respective sequence of poses specified by a respective nominal trajectory of each of the other robots of the two or more robots; selecting between two or more nominal trajectories based at least in part on the respective allowable time lags for the two or more nominal trajectories, wherein selecting between the two or more nominal trajectories for each robot based at least in part on the respective allowable time lags for the two or more nominal trajectories includes selecting a nominal trajectory having a largest one of the allowable time lags of the two or more nominal trajectories for the respective robot; providing the allowable time lag to at least one processor to control operation of each of the two or more robots.

2. determining each allowable time lag for the nominal trajectory, For each of the two or more robots, For each of the one or more nominal trajectories:

2. The method of claim 1, comprising generating a swept volume representation that represents a volume swept by at least a portion of the robot as it moves through the set of sequences of poses specified by the nominal trajectory from at least one time within the nominal trajectory to another time within the nominal trajectory.

3. determining each allowable time lag for the nominal trajectory, The method of claim 2 further comprising performing a collision assessment using the generated swept volume.

4. The step of performing collision assessment using the generated swept volumes includes, for each robot of the two or more robots: performing a collision assessment between i) at least a portion of a respective sample trajectory representing the respective nominal trajectory of the robot with at least one respective time lag introduced, and ii) at least a portion of each of the respective samples of the respective nominal trajectory of each of the other robots of the two or more robots with at least one respective time lag introduced into the respective nominal trajectory of each of the other robots of the two or more robots; 4. The method of claim 3, comprising: in response to determining that a collision will occur between the robot and at least one other robot of the two or more robots, identifying as the respective allowable time lags for the one or more nominal trajectories of the robot that are shorter than the respective time lags that resulted in the determination that a collision will occur, the allowable time lags reflecting the maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of the robot through a sequence of poses specified by the nominal trajectory remains collision-free with respect to movement of the other robots of the two or more robots.

5. determining each allowable time lag for the nominal trajectory, 5. The method of claim 4, further comprising: in response to determining that a collision will occur between the robot and at least one other robot of the two or more robots, identifying as the respective allowable time lags, for the one or more nominal trajectories of at least one other robot of the two or more robots for which a collision has been determined to occur, the respective time lags being shorter than the respective time lags that resulted in the determination that a collision will occur, the allowable time lags reflecting the maximum allowable delay from the timing for the poses of the nominal trajectories that still ensures that movement of the robot through a sequence of poses specified by the respective nominal trajectories remains collision-free with respect to movement of the other robot of the two or more robots.

6. determining each allowable time lag for the nominal trajectory, further comprising a first iteration step and a second iteration step; the first iteration step repeats the second iteration step through each of a plurality of candidate time lags in sequence from a relatively small candidate time lag to a relatively large candidate time lag at least until a stopping condition is met; The second iteration step includes, through each of a plurality of times covered by the nominal trajectory: checking for a collision between one robot of the two or more robots and at least one other robot of the two or more robots based on a current one of the candidate time lags; and 2. The method of claim 1, further comprising: repeating the step of: in response to determining that a collision will occur between one robot of the two or more robots and at least one other robot of the two or more robots, setting the respective allowable time lags for the nominal trajectories of the robot and the at least one other robot of the two or more robots, where a collision has been determined to occur, to a most recent candidate time lag.

7. The step of checking for a collision between the robot and the other robot based on a current one of the candidate time lags comprises: For each of the two or more robots, 7. The method of claim 6, comprising performing a collision assessment for at least a portion of the respective nominal trajectories of the robots having the current one of the candidate time lags or another previously determined time lag against at least a portion of each of the respective nominal trajectories of each of the other robots of the two or more robots having the current one of the candidate time lags.

8. 8. The method of claim 7, wherein performing a collision assessment for at least a portion of the respective nominal trajectories of the robots having the current one of the candidate time lags or another previously determined time lag, and for at least a portion of each of the respective nominal trajectories of each of the other robots of the two or more robots having the current one of the candidate time lags, comprises performing a collision assessment to determine whether at least a portion of the robot will collide with at least a portion of any of the other robots of the two or more robots while the robot and the other robot are delayed along at least the portion of the respective nominal trajectory such that the robot and the other robot are delayed by the candidate time lag.

9. The step of checking for a collision between the robot and the other robot based on a current one of the candidate time lags comprises: For each of the two or more robots, generating a swept volume representation representing a volume swept by at least a portion of the robot when moving through the set of sequences of poses specified by the respective nominal trajectories from at least one time within the respective nominal trajectories to another time within the respective nominal trajectories; For each of the two or more pairs of robots determining whether a swept volume of one robot of the pair of robots intersects a swept volume of the other robot of the pair of robots for the timing relative to the pose specified by the nominal trajectory with no time lag introduced and for the timing relative to the pose specified by the nominal trajectory with each of a plurality of candidate time lags introduced into the respective nominal trajectory, at least until an intersection is detected; and 7. The method of claim 6, comprising generating an indication of a collision and generating an identification of the robots determined to be in collision in response to determining that the swept volume of one robot of the pair of robots intersects the swept volume of the other robot of the pair of robots.

10. For each of the two or more robots, For each of one or more actual trajectories executed by the respective robot, monitoring the amount of delay between the nominal trajectory and each of the actual trajectories performed by each of the robots; determining whether the monitored amount of delay exceeds the respective determined allowable time lag for the respective actual trajectory of the respective robot; 2. The method of claim 1, further comprising: taking at least one corrective action in response to the monitored amount of delay exceeding the respective determined allowable time lag for the respective nominal trajectory of the respective robot.

11. 11. The method of claim 10, wherein taking at least one corrective action comprises one or more of: stopping one or more movements of the robot; slowing down one or more movements of the robot; and accelerating one or more movements of the robot.

12. 11. The method of claim 10, wherein taking at least one corrective action comprises stopping the movement of the robots, advancing one or more of the robots along respective trajectories specified by the respective nominal trajectories, and thereafter resuming the movement of the robots.

13. 11. The method of claim 10, wherein the step of determining respective allowable time lags for the nominal trajectories occurs during a configuration time prior to movement of the at least two robots specified by the one or more nominal trajectories of the respective robots.

14. 14. The method of claim 13, wherein monitoring the amount of delay between the nominal trajectory and the respective actual trajectories performed by the respective robots occurs during runtime while the at least two robots are performing moves specified by the one or more nominal trajectories of the respective robots, the runtime following the configuration time.

15. 11. The method of claim 10, wherein monitoring the amount of delay between the nominal trajectory and the respective actual trajectory performed by the respective robot comprises using a common central clock for the monitoring of each of the robots of the one or more robots.

16. 16. The method of any one of claims 1 to 15, further comprising receiving a respective motion plan for each of the robots, each of the motion plans specifying a respective one of the one or more nominal trajectories for the respective robot, the nominal trajectories representing a respective collision-free path.

17. 1. A processor-based system for facilitating operation of multiple robots for a multi-robot operating environment in which multiple robots operate, the processor-based system comprising: at least one processor; and at least one non-transitory processor-readable medium storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 15.

18. 1. A method of operation in a processor-based system for facilitating operation of multiple robots for a multi-robot operating environment in which multiple robots operate, the method of operation comprising: For each of the two or more robots, each of two or more nominal trajectories for a respective robot of the two or more robots, the nominal trajectories specifying a respective sequence of poses and timings for the poses for the respective robot; determining a respective allowable time lag for the nominal trajectory that reflects a maximum tolerable delay from the timing for the poses of the nominal trajectory that still ensures that movement of the robot through the sequence of poses specified by the nominal trajectory remains collision-free with respect to movement through the respective sequences of poses specified by the respective nominal trajectories of each of the other robots of the two or more robots delayed by the respective allowable time lag for the nominal trajectory of at least one other robot of the two or more robots; selecting between the two or more nominal trajectories based at least in part on the respective allowable time lags for the two or more nominal trajectories, wherein selecting between the two or more nominal trajectories for each robot based at least in part on the respective allowable time lags for the two or more nominal trajectories includes selecting the nominal trajectories based on an allowable time lag and a respective cost function representing at least a risk or probability of collision, respectively associated with each of the two or more nominal trajectories for the respective robot; and providing a motion plan for the respective robot based at least in part on the selected one of the nominal trajectories to control movement of the respective robot of the two or more robots.

19. 20. The method of claim 18, wherein selecting between the two or more nominal trajectories based at least in part on the respective allowable time lags for the two or more nominal trajectories for each robot comprises selecting the nominal trajectories based on the respective cost functions representing the allowable time lag, the risk or probability of collision, and a severity of collision, respectively associated with each of the two or more nominal trajectories for the respective robot.

20. 20. The method of claim 18, wherein selecting between the two or more nominal trajectories based at least in part on the respective allowable time lag for the two or more nominal trajectories for each robot comprises selecting the nominal trajectories based on respective cost functions representing at least one of the allowable time lag, the risk or probability of collision, the severity of a collision, and the duration to completion or the expenditure of energy, respectively associated with each of the two or more nominal trajectories for the respective robot, wherein each variable in the cost function representing at least one of the allowable time lag, the risk or probability of collision, the severity of a collision, and the duration to completion or the expenditure of energy, respectively, is weighted in the respective cost function.

21. 20. The method of claim 18, wherein selecting between the two or more nominal trajectories based at least in part on the respective allowable time lags for the two or more nominal trajectories comprises selecting a respective nominal trajectory for each of the two or more robots that maximizes the allowable time lag in a collective across all robots of the two or more robots.

22. 20. The method of claim 18, wherein selecting between the two or more nominal trajectories based at least in part on the respective allowable time lags for the two or more nominal trajectories for each robot comprises selecting a respective nominal trajectory for each of the two or more robots that optimizes a respective cost function for each of the robot in an aggregate across all robots of the two or more robots, each cost function representing the allowable time lag and at least a risk or probability of collision, respectively, associated with the respective nominal trajectory of a respective one of the two or more nominal trajectories for the respective robot.

23. 20. The method of claim 18, wherein selecting between the two or more nominal trajectories based at least in part on the respective allowable time lags for the two or more nominal trajectories for each robot comprises selecting a respective nominal trajectory for each of the two or more robots that optimizes a respective cost function for each of the robot in an aggregate across all robots of the two or more robots, each cost function representing the allowable time lag, at least a risk or probability of collision, and a severity of collision, respectively associated with the respective nominal trajectory of a respective one of the two or more nominal trajectories for the respective robot.

24. 20. The method of claim 18, wherein selecting between the two or more nominal trajectories based at least in part on the respective allowable time lags for the two or more nominal trajectories for each robot comprises selecting a respective nominal trajectory for each of the two or more robots that optimizes a respective cost function for each of the robot in a population across all robots of the two or more robots, wherein each cost function represents at least one of the allowable time lag, at least a risk or probability of collision, a severity of collision, and a duration to completion or an expenditure of energy, respectively associated with the respective nominal trajectory of a respective one of the two or more nominal trajectories for the respective robot, and wherein each variable in the cost function representing at least one of the allowable time lag, at least a risk or probability of collision, a severity of collision, and a duration to completion or an expenditure of energy, respectively, is weighted in the respective cost function.

25. 20. The method of claim 18, wherein determining a respective allowable time lag for the nominal trajectory reflecting a maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of the robot through the sequence of poses specified by the nominal trajectory remains collision-free comprises performing a collision assessment for each robot of the two or more robots.

26. The step of performing a collision assessment for each of the two or more robots includes:

26. The method of claim 25, comprising performing a collision evaluation between i) at least a portion of a respective sample trajectory representing the respective nominal trajectory of the robot with at least one respective time lag introduced, and ii) at least a portion of each of the respective samples of the respective nominal trajectories of each of the other robots of the two or more robots with at least one respective time lag introduced into the respective nominal trajectory of each of the other robots of the two or more robots.

27. The step of determining each allowable time lag for the nominal trajectory further comprises:

27. The method of claim 26, comprising, in response to determining that a collision will occur between the robot and at least one other robot of the two or more robots, identifying as the respective allowable time lags for the one or more nominal trajectories of the robot that are shorter than the respective time lags that resulted in the determination that a collision will occur, the allowable time lags reflecting the maximum allowable delay from the timing for the poses of the nominal trajectory that still ensures that movement of the robot through the sequence of poses specified by the nominal trajectory remains collision-free with respect to movement of the other robots of the two or more robots.

28. The step of determining each allowable time lag for the nominal trajectory further comprises:

28. The method of claim 27, comprising, in response to determining that a collision between the robot and at least one other robot of the two or more robots will occur, identifying as the respective allowable time lags, for the one or more nominal trajectories of at least one other robot of the two or more robots for which a collision has been determined to occur, respective time lags that are shorter than the respective time lags that resulted in the determination that a collision will occur, wherein the allowable time lags reflect the maximum allowable delay from the timing for the poses of the nominal trajectories that still ensures that movement of the robot through the sequence of poses specified by the respective nominal trajectories remains collision-free with respect to movement of the other robot of the two or more robots.

29. 1. A processor-based system for facilitating operation of multiple robots for a multi-robot operating environment in which multiple robots operate, comprising: at least one processor; and at least one non-transitory processor-readable medium storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to perform the method of any one of claims 18 to 28.

Images