Robust motion planning and / or control for multi-robotic environments

Abstract

The motion planning generates a motion plan for a robot operating in a shared workspace, the motion plan including a nominal trajectory (e.g., a specified trajectory). The acceptable lag time is determined for the nominal trajectory, where compliance with the acceptable lag time during the actual operation can ensure no self-collision operation. It is monitored whether an actual operation (e.g., an actual trajectory) of the robot conforms to a corresponding acceptable lag time. Optionally, a remedial action is selected and / or taken if necessary (e.g., the actual lag time approaches or exceeds a margin or threshold, such as a corresponding acceptable lag time). Motion planning can employ a determined acceptable lag time to select a trajectory, thereby providing operation with better robustness for the robot.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This patent application claims priority to U.S. Patent Application No. 63 / 358,422, filed on July 5, 2022, the disclosure of which is incorporated herein by reference in its entirety for all purposes. Technical Field

[0003] The present disclosure generally relates to motion planning and operation of robots operating in a shared workspace, and to the computationally efficient generation of robust motion plans that can be executed by the robots in an efficient manner (e.g., reducing or even eliminating collisions and unplanned downtime), as well as the monitoring of the actual execution of the motion plans, and optionally taking remedial action when necessary. Background Art

[0004] Description of Related Technology

[0005] There are a variety of applications for using two or more robots to operate in a shared workspace. For example, two or more robots may be used to perform tasks on or with one or more objects or workpieces in a shared workspace, such as tightening bolts onto a chassis, where parts of the robots may overlap within their range of motion.

[0006] Planning usually involves task planning and motion planning. For example, task planning determines that in order to perform a given task, the robot needs to move between two poses (e.g., from a starting pose to an ending pose). In contrast, motion planning focuses on how to move the robot between two poses.

[0007] Motion planning is a fundamental problem in robot control and robotics. A motion plan specifies a trajectory that a robot can follow to transition from a starting pose, configuration, or state to a destination pose, configuration, or state, typically in order to complete a task without colliding with, or with a reduced probability of colliding with, any obstacles in a shared space. Challenges in motion planning involve the ability to perform motion planning quickly (i.e., in real time) while potentially taking into account environmental changes (e.g., changes in the position or orientation of obstacles in the shared workspace). Challenges also include using relatively low-cost devices to perform motion planning with relatively low energy consumption and limited computational power and / or storage space (e.g., memory circuits, such as on a "processor chip" circuit).

[0008] Operating two or more robots in a shared workspace (also called a workcell or multi-robot operating environment) poses a specific class of problems. For example, motion planning should consider and avoid situations where the robots or their robotic appendages may interfere with each other during the execution of a task.

[0009] One approach to operating multiple robots in a shared workspace can be called a task-level approach. Engineers can manually ensure that robots do not collide by defining portions of the shared workspace where robots may collide with each other (i.e., interference zones) and programming individual robots so that only one robot is in the interference zone of the shared workspace at any given point in time. For example, when the first robot begins to move into the interference zone of the shared workspace, the first robot sets a flag. A controller (e.g., a programmable logic controller (PLC)) reads the flag and prevents other robots from moving into the interference zone of the shared workspace until the first robot removes the flag when it exits the interference zone. This approach is intuitive and easy to understand, but is often difficult and time-consuming to implement and may not produce optimized results. Because this approach leaves at least one of the robots idle for a considerable period of time, it inevitably results in lower work throughput, even though the idle robot technically has the potential to perform useful work in the shared workspace.

[0010] In a traditional approach, a team of engineers typically breaks down a problem and optimizes the resulting smaller subproblems independently of each other (e.g., assigning tasks to robots, sequencing the tasks assigned to each robot, performing motion planning for each robot). This can take the form of iterative simulations of motion to ensure that the robots / robot attachments do not collide with each other, a simulation that can take hours of computational time and may not result in an optimized solution. Furthermore, if a modification to the shared workspace causes the actual trajectory of one of the robots / robot attachments to change, the entire workflow must be revalidated. Clearly this approach is suboptimal and typically requires experts to go through a slow iterative process in an attempt to find a combination of solutions that yields good results. In any case, this approach typically fails to ensure collision-free operation when motion planning is performed by robots operating under real-world conditions. Summary of the invention

[0011] Various methods and apparatus are described herein to generate a motion plan for a robot operating in a shared workspace, the motion plan including a nominal trajectory (e.g., a specified trajectory) having an associated acceptable lag time, wherein compliance with the acceptable lag time during actual operation can ensure self-collision-free operation (i.e., collisions of a robot with itself and collisions of a robot with other robots of a robot system). For example, the nominal trajectory can represent a corresponding collision-free path. Various methods and apparatus are described herein that monitor the actual operation of the robot (e.g., the actual trajectory) relative to the acceptable lag time and optionally take one or more remedial actions when necessary (e.g., the actual lag time approaches or exceeds a threshold, such as the corresponding acceptable lag time).

[0012] A particularly advantageous approach is described in international patent application PCT / US2021 / 013610 (published as WO2021150439A1), which describes a system for generating optimized multi-robot motion plans. More specifically, the motion plan is optimized by minimizing or attempting to minimize some cost function related to the performance of the system (e.g., the probability or likelihood of a collision). Since the motion plan specifies a nominal trajectory for the robotic system, each motion plan is also time-based. The nominal trajectory is a specified ordered sequence of poses, configurations, or states of each movable part of one or more robots of the robotic system, and is parameterized in time (e.g., at each time unit throughout the trajectory). For example, the nominal trajectory can represent a corresponding collision-free path. Since the robotic system can include multiple individual robots, the motion plan can be "multi-robot".

[0013] The motion plan can be represented as a set of discrete robot trajectories: ,in , where N represents the total number of robots of the robotic system, ΔT represents the sample time step, and T represents the total duration of the motion plan. For sufficiently small values ​​of ΔT, Trj(t) ensures collision-free behavior with respect to all robots of the robotic system. In other words, the motion specified by a set of trajectories Trj(t) of a robotic system having two or more robots does not result in self-collisions (i.e., collisions of one robot with itself and collisions of one robot with other robots of the robotic system). Although collision assessment can be performed for other objects or obstacles in the shared workspace, the set of trajectories Trj(t) of the robotic system does not necessarily mean that there will be no collisions with respect to other objects or obstacles in the environment. This no-self-collision condition can be easily checked or reviewed in simulation; and once checked or reviewed, it can be considered safe to execute the motion specified by the motion plan.

[0014] However, in practice, during the actual execution of a motion plan by a physical robotic system comprising two or more robots, many factors can adversely affect the synchronization between the robots. For example, these factors may include: signal communication, control delays, sensor noise, numerical approximation errors, etc. This can cause the actual physical robotic system to deviate from the trajectory Trj(t) specified by the motion plan. In addition to this "reality gap" between the simulation environment used to generate and review one or more motion plans and the actual operation of the real physical robotic system, many other factors may also cause the robotic system to deviate from the motion plan that provides a set of nominal trajectories. For example, in some cases, the execution of a task may take more time to complete than expected, and therefore the robot may need to continue or stay in a specific position (e.g., the destination position, and therefore in a specific posture, configuration or state) for a longer time than the time specified by the trajectory in the motion plan. Deviating from a specific motion plan that has been reviewed and considered safe can be dangerous and may endanger the safety of the robotic system.

[0015] A set of trajectories Trj(t) specified by the motion plan is named as a set of nominal trajectories in this article to indicate that these trajectories are specified trajectories. A single trajectory specified by the motion plan is named as a nominal trajectory in this article. A set of trajectories executed by the actual physical motion of the robot is named as a set of actual trajectories in this article to distinguish it from the specified trajectory or nominal trajectory. A single trajectory executed by the actual physical motion of the robot is referred to as an actual trajectory in this article.

[0016] The methods described herein advantageously extend safe operating regimes beyond those associated with a set of nominal trajectories or nominal trajectories of one or more motion plans, for example, by calculating a neighborhood of one or more nominal trajectories of a motion plan in which the robotic system can operate without causing the self-collision described above. For each robot that will operate in a shared workspace, a maximum acceptable lag time (e.g., time delay or "hysteresis") for one or more nominal trajectories that the robot may execute is calculated. During online execution (i.e., run time), the actual lag time of each robot is monitored. If the actual lag time is within a defined margin or threshold, it is considered safe to continue executing the motion plan because self-collision-free movement between robots is ensured. If the actual lag time of any of the robots exceeds the corresponding margin or threshold, safety is compromised because self-collision-free operation can no longer be ensured. Optionally, a processor-based system is capable of selecting and / or taking one or more remedial actions in this case. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In the drawings, the same reference numerals identify similar elements or actions. The sizes and relative positions of elements in the drawings are not necessarily drawn to scale. For example, the angles and shapes of various elements are not drawn to scale, and portions of such elements are arbitrarily enlarged and positioned to improve the legibility of the drawings. In addition, the particular shapes of the drawn elements are not intended to convey any information about the actual shape of the particular element, but are selected only for ease of identification in the drawings.

[0018] Figure 1 is a schematic diagram of a shared workspace and a configuration optimization system that performs optimization to configure robots according to one illustrated embodiment, wherein multiple robots operate in the shared workspace to perform a task.

[0019] Figure 2 is a functional block diagram of at least a first robot and a robot control system according to at least another illustrated embodiment, wherein the robot control system is communicatively coupled to control operation of the first robot, wherein the robot control system includes a motion planner that advantageously determines and employs acceptable lag times for corresponding nominal trajectories of the robots in motion plans, and optionally monitors actual operation of at least the first robot, compares the actual lag time to a margin or threshold (e.g., the acceptable lag time), and optionally, takes remedial action if necessary.

[0020] Figure 3 A method of operating a processor-based system performing motion planning to control robots to be operated in a shared workspace according to at least one illustrated embodiment is shown, wherein an acceptable lag time for each robot is determined based on nominal trajectories of other robots without considering the effects of lag times in the nominal trajectories of other robots, and a motion plan specifying a nominal trajectory is generated or selected, the nominal trajectory being associated with a corresponding acceptable lag time.

[0021] Figure 4 A method of operating a processor-based system performing motion planning to control robots to be operated in a shared workspace according to at least one illustrated embodiment is shown, wherein an acceptable lag time for each robot is determined based on nominal trajectories of other robots, while the effects of lag times in the nominal trajectories of other robots must be considered, and a motion plan specifying a nominal trajectory is generated or selected, which nominal trajectory is associated with a corresponding acceptable lag time.

[0022] Figure 5 A method of operating a processor-based system according to at least one illustrated embodiment is shown for generating a swept volume for one or more trajectories for each of a plurality of robots to be operated in a shared workspace, optionally capable of being used to perform Figure 3 ,4 and / or 6 methods.

[0023] Figure 6 A method of operating a processor-based system according to at least one illustrated embodiment is shown for performing collision assessment for a plurality of robots and a plurality of trajectories for each of the robots, wherein movement of the robots or portions thereof along the trajectories is represented as a swept volume, and wherein the collision assessment can be used to determine an acceptable lag time for one or more robots operating in a shared workspace.

[0024] Figure 7 A method of operation of a processor-based system according to at least one illustrated embodiment is shown for generating or selecting motion plans for one or more robots to operate in a shared workspace based at least in part on acceptable lag time, and optionally at least in part on other criteria such as expressed as a cost or cost function.

[0025] Figure 8 A method of operating a processor-based system for controlling operation of a robot operating in a shared workspace based at least in part on acceptable lag time is shown in accordance with at least one illustrated embodiment. DETAILED DESCRIPTION

[0026] In the description that follows, certain specific details are set forth in order to provide a thorough understanding of the various disclosed embodiments. However, one skilled in the relevant art will recognize that the embodiments may be implemented without one or more of these specific details, or using other methods, components, materials, etc. In other cases, well-known structures associated with computer systems, robots, actuator systems, and / or communication networks are not shown or described in detail to avoid unnecessarily obscuring the description of the embodiments. In other cases, well-known computer vision methods and techniques for generating perceptual data and volumetric representations of one or more objects and the like are not described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0027] Unless the context requires otherwise, throughout the specification and the appended claims, the word "comprise" and variations such as "include" and "comprising" should be construed in the open inclusive sense, ie, "including, but not limited to."

[0028] References throughout this specification to "one embodiment" or "an embodiment", or "an example" or "an example" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment or at least one implementation example. Therefore, the phrases "one embodiment" or "an embodiment" or "in one example" or "in an example" appearing in various places in this specification do not necessarily refer to the same embodiment or example. Furthermore, particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0029] Unless the content clearly indicates otherwise, an undefined number of elements in this specification and the appended claims means that there is at least one of the elements. It should also be noted that the term "or" is generally used to include the meaning of "and / or" unless the content clearly indicates otherwise.

[0030] As used in this specification and the appended claims, the term "optimize" and its variations mean preparing, generating, or producing an improved result, or having prepared, generated, or produced an improved result. These terms are used in a relative sense and do not necessarily mean that the absolute optimal value has been prepared, generated, or produced.

[0031] As used in this specification and the appended claims, the term "workspace" or "shared workspace" is used to refer to an operating environment in which two or more robots operate, one or more portions of the shared workspace being a volume in which the robots may potentially collide with each other, and thus these portions may be named interference zones. The operating environment may include obstacles (i.e., items that the robots are to avoid colliding with) and / or artifacts (i.e., items that the robots are to interact with or act on or act on).

[0032] As used in this specification and the appended claims, the term "task" is used to refer to a robot task for which the robot transitions from posture A to posture B, preferably without colliding with obstacles in its environment. The task may involve grasping or releasing an object, moving or setting down an object, rotating an object, or retrieving or placing an object. The transition from posture A to posture B may optionally include transitions between one or more intermediate postures.

[0033] As used in this specification and the appended claims, the term "a trajectory" or "trajectories" is used to refer to an ordered sequence of poses or configurations or states of one or more robots, parameterized in time, through which one or more robots or at least a portion thereof can move, e.g., perform a task. The trajectory is preferably represented in the configuration space (also called C-space) of the respective robot, but can alternatively also be represented in real space or real world space of a shared workspace. The trajectory can include pauses, changes in direction or even reversals of one or more poses, and is not necessarily smooth in terms of direction, time, velocity or acceleration.

[0034] As used in this specification and the appended claims, the term "a nominal trajectory" or "a plurality of nominal trajectories" is used to refer to a "specified" one or more trajectories. As used in this specification and the appended claims, the term "an actual trajectory" or "a plurality of actual trajectories" is used to refer to one or more trajectories actually performed by the physical movement of a physical robot, as distinguished from the "a nominal trajectory" or "a plurality of nominal trajectories". In some cases, the "a actual trajectory" or "a plurality of actual trajectories" may match the "a nominal trajectory" or "a plurality of nominal trajectories", although in many cases there will be differences between the two, for example, because the "a actual trajectory" or "a plurality of actual trajectories" lags behind the "a nominal trajectory" or "a plurality of nominal trajectories" in time.

[0035] As used in this specification and the appended claims, when used or referring to a robot system including two or more robots, the term "self-collision" includes both: i) a collision of a robot with itself, and ii) a collision of a robot with other robots of the robot system. As used in this specification and the appended claims, when used or referring to a single robot, the term "self-collision" includes a collision of a robot with itself.

[0036] The titles and abstracts of the disclosure provided herein are for convenience only and do not interpret the scope or meaning of the embodiments.

[0037] The methods described herein advantageously extend the safety of robot operation beyond the safety ensured by the nominal trajectory of the motion plan by determining the acceptable lag time that the physical robots of the robot system can operate while still ensuring that such operation of the robot system will not self-collision. In at least some embodiments, for each robot of the robot system that will operate in a shared workspace, a maximum acceptable lag time (e.g., time delay or "lag") is calculated for each nominal trajectory in one or more nominal trajectories. Advantageously, the maximum acceptable lag time can be calculated during the configuration time before the robot executes the motion plan. During online execution (i.e., at runtime, after the configuration time), the actual movement of each robot is monitored, including monitoring the actual lag time of each robot along its actual trajectory. If the actual lag time value is within a defined margin or threshold (e.g., an acceptable lag time), it can be considered safe to continue to execute the motion plan because it can ensure that there is no self-collision movement. If the actual lag time of the robot exceeds the corresponding margin or threshold, then safety will be compromised because it can no longer ensure self-collision operation. For example, the safety of all robots may be deemed to be adversely affected even if only one robot is experiencing an actual lag time that exceeds a corresponding margin or threshold. Optionally, the processor-based system is capable of selecting and / or taking one or more remedial actions in such a situation.

[0038] Figure 1 A robotic system 100 is shown, according to one illustrated embodiment, comprising a plurality of robots 102a, 102b, 102c (collectively 102) operating in a shared workspace 104 (also referred to as a multi-robot environment) to perform tasks. Figure 1 In the robotic system 100, as part of the optimization, an acceptable lag time is determined and provided to one or more robotic control systems along with an optimized motion plan including a nominal trajectory of the robot.

[0039] Multiple robots can be configured to perform a set of tasks. Tasks can be specified as task plans. A task plan can specify T tasks that need to be performed by N robots. The task plan can be modeled as a vector for each robot, where the vector is an ordered list of tasks to be performed by the corresponding robot (e.g., {task 7, task 2, task 9}). The task vector can also optionally include a dwell duration that specifies the duration that the robot or part thereof should stay at a given configuration or destination. The task vector can also specify a home pose and / or other "functional poses" that are not directly related to solving the task (e.g., "get out of the way" or storage pose). The pose can be specified in the robot's C-space.

[0040] The robot 102 can take any of a variety of forms. Typically, the robot 102 will take the form of or have one or more robot attachments 103 (only one is labeled) and a base 105 (only one is labeled) from which the robot attachment 103 extends. The robot 102 may include one or more links having one or more joints, and an actuator (e.g., an electric motor, a stepper motor, a solenoid, a pneumatic actuator, or a hydraulic actuator) coupled and operable to move the links in response to a control or drive signal. For example, a pneumatic actuator may include one or more pistons, cylinders, valves, gas reservoirs, and / or pressure sources (e.g., compressors, blowers). For example, a hydraulic actuator may include one or more pistons, cylinders, valves, liquid reservoirs (e.g., low-compressibility hydraulic fluids), and / or pressure sources (e.g., compressors, blowers). The robot system 100 may take other forms of the robot 102, such as an autonomous vehicle.

[0041] The shared workspace 104 generally represents a three-dimensional space in which the robots 102 can operate and move, although in certain limited embodiments, the shared workspace 104 can represent a two-dimensional space. The shared workspace 104 is a volume or area in which at least a portion of the robots 102 can overlap in space and time, or collide if movement is not controlled to avoid collision. Notably, the shared workspace 104 is a physical space or volume, where the position and orientation of the physical space or volume can be determined, for example, relative to some reference frame (e.g., Figure 1 104) is conveniently represented by a Cartesian coordinate system (a reference system represented by orthogonal axes X, Y, and Z as shown in FIG. 104). It should also be noted that the reference system of the shared workspace 104 is different from the corresponding "configuration space" or "C-space" of any of the robots 102, which is generally represented by a set of joint positions, orientations, or configurations in the corresponding reference system of any of the robots 102.

[0042] As described herein, a robot 102a or a portion thereof may constitute an obstacle when considered from the perspective of another robot 102b (i.e., when performing motion planning for another machine 102b). The shared workspace 104 may also include other obstacles, such as mechanical parts (e.g., conveyor belt 106), columns, pillars, walls, ceilings, floors, tables, people and / or animals. The shared workspace 104 may also include one or more work items or workpieces that the robot 102 manipulates (e.g., one or more packages, packages, fasteners, tools, projects, or other objects) as part of performing a task.

[0043] Optionally, the robot system 100 includes one or more processor-based multi-robot configuration optimization systems 108 ( Figure 1 One is shown). The optional one or more multi-robot configuration optimization systems 108 receive a set of inputs 109 and generate one or more solutions specifying the configuration of the robots 102 as outputs 111, which include a work cell layout (e.g., a respective base position and orientation of each robot 102), one or more mission plans (e.g., a respective mission plan for each of the robots 102), and optionally one or more motion plans that specify or include one or more nominal trajectories of the robots 102a-102c (e.g., a respective nominal trajectory for each of the robots 102) and an acceptable lag time for each of the nominal trajectories. One or more components of the outputs 111 may be optimized at least to some extent.

[0044] The optional one or more multi-robot configuration optimization systems 108 may include a population generator 110 , a multi-robot environment simulator 112 , a multi-robot optimization engine 114 , and an acceptable lag time evaluator 115 .

[0045] The population generator 110 generates a set of candidate solutions 116 based on the input 109 provided. The candidate solutions 116 represent possible solutions to the configuration problem (i.e., how to configure the robots 102 in the shared workspace 104 to complete a set of tasks). Any given candidate solution 116 may or may not actually be feasible. That is, the initial candidate may be invalid (e.g., the robot is in an impossible place, has an unreachable goal, or an infeasible task plan that will result in a collision). In some embodiments, the population generator may attempt to find better candidate solutions.

[0046] The multi-robot environment simulator 112 models the workspace or multi-robot environment based on each candidate solution to determine certain properties, such as the amount of time required to complete the task, the probability or rate of collision when completing the task, and the feasibility or infeasibility of a particular configuration specified by the candidate solution. For example, the multi-robot environment simulator 112 can reflect these properties in terms of cost via costs generated by one or more cost functions.

[0047] For example, the cost or cost function can represent the probability or possibility of a collision. Optionally, the cost or cost function can represent one or more of the following: acceptable lag time or "robustness", the severity of the collision, the energy consumption or loss and / or the time or lag to execute or complete the motion corresponding to the nominal trajectory. In some embodiments, the cost or cost function represents the determined acceptable lag time of a given nominal trajectory of a given robot. The determined acceptable lag time represents the maximum or approximately maximum or optimized lag time that may be caused when executing the corresponding nominal trajectory while still maintaining, ensuring or even guaranteeing at least no self-collision movement relative to itself and relative to other robots operating in a shared workspace or work cell. Therefore, the lag time can represent the lag amount caused by the actual execution of the nominal trajectory or the actual execution of the nominal trajectory without giving up the safety factor (e.g., no self-collision operation), thereby enhancing the robustness of the corresponding motion plan. For example, the safety factor can be checked or reviewed via the use of the nominal trajectory simulation robot operation. Therefore, if the actual trajectory of the robot lags behind the nominal trajectory by more than a specified margin or threshold (e.g., acceptable lag time), collision-free operation can no longer be ensured.

[0048] The multi-robot optimization engine 114 evaluates candidate solutions based at least in part on the associated costs 119 and advantageously performs collaborative optimization across a set of two or more non-homogeneous parameters, for example, across two or more of the following parameters: respective basic positions and orientations of the robots, assignments of tasks to respective robots, respective goal sequences of the robots, and / or respective trajectories or paths between consecutive goals (e.g., collision-free trajectories or paths). Straight-line trajectories between consecutive goals may be used to simplify the description, but the trajectories need not necessarily be straight-line trajectories.

[0049] The input 109 may include one or more static environment models representing or characterizing the operating environment or shared workspace 104, for example, representing floors, walls, ceilings, columns, other obstacles, etc. The operating environment or shared workspace 104 may be represented by one or more models, for example, geometric models (e.g., point clouds) representing floors, walls, ceilings, obstacles, and other objects in the operating environment. For example, the model may be represented using Cartesian coordinates.

[0050] The input 109 may include one or more robot models representing or characterizing each of the robots 102, e.g., specifying geometry and kinematics, e.g., size or length, number of links, number of joints, range of motion of joint types, velocity limits, acceleration limits, or jerk limits. The robots 102 may be represented by one or more robot geometry models that define the geometry of a given robot 102a-102c, e.g., in terms of joints, degrees of freedom, size (e.g., link lengths), and / or respective C-spaces of the robots 102a-102c.

[0051] The input 109 may include one or more sets of tasks to be performed, for example, represented as target objectives (e.g., poses, configurations, states, or positions or locations). For example, the tasks may be represented by an end pose, end configuration, or end state and / or an intermediate pose, intermediate configuration, or intermediate state of the respective robots 102a to 102c. For example, the poses, configurations, or states may be defined based on joint positions and joint angles / rotations (e.g., joint poses, joint coordinates) of the respective robots 102a to 102c. Optionally, the input 109 may include one or more dwell times that specify a nominal amount of time that a robot or part thereof should stay at a given objective in order to complete a task (e.g., tightening a screw or nut, picking and placing an object, with the goal of sorting a pile of objects into two or more piles of objects of different types by two or more robots operating in a common workspace).

[0052] In some embodiments, one or more multi-robot configuration optimization systems 108 can generate a nominal trajectory for each robot to perform one or more tasks. A nominal trajectory is a "specified" trajectory, wherein each nominal trajectory includes a time-parameterized ordered set or sequence of postures, configurations, or states of the robot between an initial or starting posture, configuration, or state of the nominal trajectory and a final or ending posture, configuration, and state of the nominal trajectory, each posture, configuration, state having a corresponding timing. The postures or configurations are preferably represented in the configuration space (also referred to as C-space) of the corresponding robot, or alternatively in the real space or real world space of the workspace. The timing can be specified in relative terms (e.g., timing defined by a relative offset from the timing of the immediately preceding posture) or in absolute terms (e.g., timing defined by a duration of execution of the trajectory from the beginning, and, for example, relative to a common clock). In at least some cases, the nominal trajectory can specify or include one or more pauses in the motion or path of the robot or part thereof, and / or can specify a reversal of the direction or path of motion of the robot or part thereof, which otherwise may not be a smooth motion in terms of direction or timing. Applicants note that while any given trajectory can correspond to smooth motion of a robot, the term "trajectory" as used herein is not limited thereto and will generally specify motion that is not smooth, nor will it define a straight path for a robot or portion thereof. For example, a nominal trajectory can specify a time-parameterized ordered set or sequence of poses through which a robot or portion thereof moves to complete a task or portion thereof. One or more nominal trajectories can be employed to perform any given task. As described herein, the actual motion or actual trajectory of a robot or portion thereof may deviate from a corresponding nominal trajectory, for example, due to unexpected delays in transitioning between poses (e.g., due to the need to linger or stay at a target object for a longer time than expected).

[0053] The acceptable lag time evaluator 115 evaluates each candidate lag time for a given nominal trajectory to determine an acceptable lag time that ensures self-collision-free operation even if the actual trajectory of the corresponding robot lags behind the nominal trajectory by no more than the acceptable lag time. In a preferred method, the acceptable lag time evaluator 115 not only considers the effect of the lag in the nominal trajectory of the given robot, but also considers one or more lags or effects of the corresponding nominal trajectories of other robots operating in the shared workspace. Therefore, the acceptable lag time evaluator 115 is able to identify the acceptable lag time, or in other words, the time lag, for each nominal trajectory, assuming a worst-case scenario in which the actual trajectories of all robots operating in the shared workspace experience their corresponding acceptable lag times. Therefore, for example, the acceptable lag time evaluator 115 is able to determine the maximum acceptable lag time for each robot that can still ensure self-collision-free operation, even if it is assumed that all robots experience their corresponding maximum acceptable lag times. Several methods for determining acceptable lag times are described herein.

[0054] The input 109 may optionally include a limit on the number of robots that can be configured in the shared workspace 104. The input 109 may optionally include a limit on the number of tasks or goals that can be assigned to a given robot 102a to 102c, referred to herein as a task capacity, that can be configured in the shared workspace 104, for example, to limit the complexity of the configuration problem to ensure that the configuration problem is solvable, or solvable within an acceptable time period using available computing resources, or to pre-eliminate certain solutions that are considered too slow because the tasks or goals are significantly over-allocated to a given robot 102a to 102c. The input 109 may optionally include one or more bounds or constraints on variables or other parameters. The input 109 may optionally include a total number of iteration cycles or a time limit for iterations, which can be used to refine candidate solutions, for example, to ensure that the configuration problem is solvable or solvable within an acceptable time period using available computing resources.

[0055] The robotic system 100 may optionally include one or more robotic control systems 118 ( Figure 1 The robot 102 may include a plurality of control systems 118 (only one of which is shown) that are communicatively coupled to control the robot 102. For example, one or more robot control systems 118 may provide control signals (e.g., drive signals) to various actuators to move the robot 102 between various configurations to various designated targets to perform designated tasks.

[0056] The robotic system 100 may optionally include one or more motion planners 120 ( Figure 1102 ). As described elsewhere herein, one or more motion planners 120 generate, generate, select, or improve motion plans for the robot 102, for example, to account for small deviations in time from the motion plan provided by the multi-robot optimization engine 114, or to account for the unexpected appearance of obstacles (e.g., humans entering the operating environment or shared workspace 104). The optional motion planner 120 is operable to dynamically generate motion plans to enable the robot 102 to perform tasks in the operating environment. The motion planner 120 and other structure and / or operation may be as described in U.S. patent application serial number 62 / 865,431 filed on June 24, 2019.

[0057] Where a motion planner 120 is included, the motion planner 120 is optionally communicatively coupled to receive as input perception data, for example, provided by a perception subsystem (not shown). The perception data represents previously unknown static and / or dynamic objects in the shared workspace 104. The perception data may be raw data sensed via one or more sensors (e.g., a camera, a stereo camera, a time-of-flight camera, a lidar) and / or raw data converted by a perception subsystem into digital representations of obstacles, which may generate corresponding discretizations of representations of the environment in which the robot 102 will operate to perform tasks for a variety of different scenarios.

[0058] Figure 1Various communication paths are shown as lines between various structures, and in some cases, arrows indicate the direction of inputs 109 and outputs 111. For example, the communication paths can take the form of one or more wired communication paths (e.g., electrical conductors, signal buses, or optical fibers) and / or one or more wireless communication paths (e.g., via RF or microwave radios and antennas, infrared transceivers). For example, a communication channel can include one or more transmitters, receivers, transceivers, radios, routers, wired ports (e.g., Ethernet ports), etc. The general operation of the robot system 100, and in particular the general operation of one or more multi-robot configuration optimization systems 108, is shown and described in International Patent Application PCT / US2021 / 013610 (published as WO2021 / 150439), and for the sake of brevity, it will not be repeated herein. Only some of the more significant differences in operation are described herein, e.g., in embodiments, one or more multi-robot configuration optimization systems 108 perform the various methods described herein to determine acceptable lag times, generate or select motion plans based at least in part on the determined acceptable lag times, and / or one or more robot control systems 118 perform the various methods described herein to monitor actual lag times of robots executing motion plans, compare actual lag times to margins or thresholds (e.g., acceptable lag times), and / or control the robots accordingly, e.g., taking one or more remedial actions if one or more actual lag times exceed one or more acceptable lag times or related thresholds.

[0059] Figure 2 A robotic system according to at least one illustrated embodiment is shown, wherein a first robotic control system 200a includes a first motion planner 204a that generates a first motion plan 206a to control the operation of a first robot 202, and optionally provides the first motion plan 206a and / or the motion representation as an obstacle to other motion planners 204b of other robotic control systems 200b via at least one communication channel (represented by approximate arrows, e.g., transmitter, receiver, transceiver, radio, router, Ethernet) to control other robots ( Figure 2 ). Figure 2 In the robot control system 200a, 200b of FIG. 1 , the acceptable lag time is determined by a motion planner together with a motion plan 206a, 206b, which includes one or more nominal trajectories to be executed by one or more robots 202. Figure 1 )compared to, Figure 2 The robot control system 200a, 200b does not necessarily perform optimization of work cell layout or task planning. In addition, as described herein, Figure 2The robot control system 200a, 200b monitors the actual lag time and, optionally, selects and / or takes remedial action when necessary.

[0060] Likewise, one or more other motion planners 204b of one or more other robot control systems 200b generate one or more other motion plans 206b to control one or more other robots ( Figure 2 200b), and optionally provide one or more other motion plans 206b to the first motion planner 204a and one or more other motion planners 204b of the one or more other robot control systems 200b. The motion planners 204a, 204b may also optionally receive motion completion information 209 indicating when the motion of each robot 202 has been completed. This may allow the motion planners 204a, 204b to generate new or updated motion plans based on the current or updated state of the shared workspace. For example, after the first robot 202 completes part or all of a set of motions (the set of motions forming part of the first robot 102 completing a task), a portion of the shared workspace may become blocked, unblocked, or otherwise available for the second robot to perform a task. Additionally or alternatively, the motion planners 204a, 204b can receive information collected by various sensors or generated by other motion planners 204b (e.g., images, occupancy grids, joint positions, and joint angles / rotations) that indicate when a portion of the shared workspace may become blocked, unblocked, or otherwise available for a second robot to perform a task after the first robot 202 has completed part or all of a set of motions that form part of a task completed by the first robot 102.

[0061] As described herein, the motion plans 206a, 206b specify a nominal trajectory for each robot, e.g., for performing one or more tasks. As described above, a nominal trajectory is a "specified" trajectory, wherein each nominal trajectory comprises a time-parameterized ordered set or sequence of postures, configurations, or states of the robot, each posture, configuration, and state having a corresponding timing. The postures, configurations, or states are preferably represented in a configuration space (also referred to as C-space) of the corresponding robot.

[0062] The robotic control systems 200a, 200b may be optionally communicatively coupled, e.g., via at least one communication channel (represented by approximate arrows, e.g., a transmitter, receiver, transceiver, radio, router, Ethernet), to optionally receive the motion planning graph 208 and / or swept volume representation 211 from one or more sources 212 of the motion planning graph 208 and / or swept volume representation 211. According to one illustrated embodiment, the one or more sources 212 of the motion planning graph 208 and / or swept volume representation 211 may be separate and distinct from the motion planners 204a, 204b. For example, the one or more sources 212 of the motion planning graph 208 and / or swept volume representation 211 may be one or more processor-based computer systems (e.g., server computers) that may be operated or controlled by the respective manufacturer of the robot 202 or some other entity. The motion planning graphs 208 may each include a set of nodes 214 ( Figure 2 Only two are marked) and a set of edges 216 ( Figure 2 208 ), nodes 214 represent poses, configurations, or states of the corresponding robots, and edges 216 couple the nodes 214 of the corresponding node pairs 214 and represent legal or valid transitions between poses, configurations, and states. For example, the poses, configurations, or states may be defined in the corresponding configuration space (C-space) of the robot, representing a set of joint positions, orientations, poses, or coordinates of each of the joints of the corresponding robot 202. Thus, each node 214 may represent a pose, configuration, or state of the robot 202 or a portion thereof, which is completely defined by the pose configuration or state of the joints that make up the robot 202. The motion planning graph 208 may be determined, set, or defined prior to runtime (i.e., defined prior to performing a task), such as during pre-run time or configuration time. The optional swept volume representation 211 represents the corresponding volume that the robot 202 or a portion thereof will occupy when performing a motion or transition between poses corresponding to the corresponding edge 216 of the motion planning graph 208. The optional swept volume representation 211 can be represented in any of a variety of forms, such as, for example, as voxels, Euclidean distance fields, hierarchies of geometric objects. This advantageously allows some of the most computationally intensive work to be performed before run time, when responsiveness is not a particular concern. Although a swept volume is used herein, this is exemplary, and any of a variety of other collision assessment methods can be employed.

[0063] Each robot 202 may optionally include a base ( Figure 2206a). The base may be fixed in the environment or movable in the environment (e.g., an autonomous or semi-autonomous vehicle). Each robot 202 may optionally include a set of links, joints, end-of-arm tools or end effectors and / or actuators 218a, 218b, 218c (three are shown, collectively referred to as 218) that are operable to move the links about the joints. The set of links, joints, end-of-arm tools or end effectors typically include one or more accessories of the robot that can be movably coupled to the base of the robot. Each robot 202 may optionally include one or more motion controllers (e.g., motor controllers) 220 (only one is shown) that receive control signals (e.g., in the form of motion plans 206a) and provide drive signals to drive the actuators 218. Alternatively, the motion controllers 220 can be separate from the robot 202 and communicatively coupled to the robot 202. For example, as described in international patent application PCT / US2021 / 013610 (published as WO 2021 / 150439), each robot 202 can be positioned and oriented in a shared workspace based on an optimized work cell layout.

[0064] Each robot 202 ( Figure 2 Only one robot is shown in the figure) can have corresponding robot control systems 200a, 200b, or alternatively, one robot control system 200a can perform motion planning for two or more robots 202. For the purpose of illustration, one robot control system 200 will be described in detail. Those skilled in the art will recognize that the description can be applied to similar or even identical additional examples of other robot control systems 200.

[0065] The first robot control system 200a may include one or more processors 222, and one or more associated non-transitory computer or processor readable storage media, such as a system memory 224a, one or more disk drives 224b, and / or a memory or register (not shown) of the processor 222. The non-transitory computer or processor readable storage media (e.g., system memory 224a, one or more disk drives 224b) are communicatively coupled to the one or more processors 222a via one or more communication channels (e.g., a system bus 234). The system bus 234 can employ any known bus structure or architecture, including a memory bus with a memory controller, a peripheral bus, and / or a local bus. One or more such components may also, or alternatively, communicate with each other via one or more other communication channels, such as one or more parallel cables, serial cables, or wireless network channels capable of high-speed communication, such as Universal Serial Bus ("USB") 3.0, Peripheral Component Interconnect Express (PCIe), or via Thunderbolt.® .

[0066] The first robotic control system 200a may also be communicatively coupled to one or more remote computer systems, such as a server computer (e.g., source 212), a desktop computer, a laptop computer, an ultraportable computer, a tablet computer, a smartphone, a wearable computer, and / or a sensor ( Figure 2 227) are directly or indirectly communicatively coupled to the various components of the first robotic control system 200a, for example, via an interface 227. A remote computing system, for example, a server computer (e.g., source 212), can be used to program, configure, control, or otherwise interact with the first robotic control system 200a and various components in the first robotic control system 200a, or input data (e.g., motion planning graph 208, swept volume representation 211, task specification 215, or even candidate paths or nominal trajectories). Such connection can be through one or more communication channels 210, for example, one or more wide area networks (WANs), such as Ethernet, or the Internet using Internet Protocol. As described above, pre-runtime calculations can be performed by a system separate from the first robotic control system 200a or the first robot 202, while runtime calculations can be performed by one or more processors 222 of the first robotic control system 200a when one or more robots perform tasks. In some implementations, one or more of the robot control systems 200a, 200b may be installed on a corresponding robot (eg, the first robot 202).

[0067] As described above, the first robot control system 200a may include one or more processors 222 (i.e., circuits), non-transitory storage media (e.g., system memory 224a, one or more disk drives 224b), and a system bus 234 that couples various system components. The processor 222 may be any logical processing unit, such as one or more central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), programmable logic controllers (PLCs), etc. The system memory 224a may include a read-only memory (“ROM”) 226, a random access memory (“RAM”) 228, a flash memory 230, an EEPROM (not shown), or any combination thereof. A basic input / output system (“BIOS”) 232, which can form part of the ROM 226, contains basic routines that facilitate the transfer of information between elements within the first robot control system 200a, for example, during startup.

[0068] For example, the one or more disk drives 224b may be a hard disk drive for reading from and writing to a disk, a solid-state (e.g., flash) drive for reading from and writing to a solid-state memory, and / or an optical drive for reading from and writing to a removable optical disk. In various embodiments, the first robot control system 200a may also include any combination of such disk drives. The one or more disk drives 224b may communicate with the one or more processors 222 via a system bus 234. As known to those skilled in the relevant art, the one or more disk drives 224b may include an interface or controller (not shown) coupled between such drives and the system bus 234. The one or more disk drives 224b and their associated computer-readable media provide non-volatile storage of computer or processor-readable and / or executable instructions, data structures, program modules, and other data for the first robot control system 200a. Those skilled in the relevant art will appreciate that other types of computer-readable media capable of storing computer-accessible data may be employed, such as WORM drives, RAID drives, cassette tapes, digital video disks (“DVDs”), Bernoulli boxes, RAM, ROM, smart cards, etc.

[0069] Executable instructions and data can be stored in system memory 224a, such as operating system 236, one or more applications 238, other applications or modules 240, and program data 242. Applications 238 may include processor-executable instructions that cause one or more processors 222 to perform one or more of the following.

[0070] The application 238 may include processor-executable instructions that cause one or more processors 222 to receive or generate a discretized representation of a shared workspace in which the robot 202 will operate, including obstacles and / or target objects or workpieces in the shared workspace, wherein planned motions of other robots may be represented as obstacles.

[0071] The application 238 may include processor executable instructions that cause the one or more processors 222 to generate motion plans 206a, 206b that specify nominal trajectories. Each of the nominal trajectories typically defines a respective ordered sequence of poses, configurations, or states of the robot or part thereof, and these sequences are parameterized in time.

[0072] The application 238 may include processor executable instructions that cause the one or more processors 222 to determine a corresponding acceptable lag time for a nominal trajectory. The acceptable lag time is generally the amount of time that the actual motion (e.g., actual trajectory) of the robot 202 can lag behind or differ from the nominal time specified for the corresponding nominal trajectory while ensuring at least self-collision-free operation of the robot 202 relative to one or more other robots operating in the shared workspace. In some embodiments, the self-collision-free operation is premised on the fact that the other robots do not experience any lag time during execution, although preferably, the self-collision-free operation is premised on the fact that all robots operate within the corresponding acceptable lag time of their corresponding nominal trajectory, thereby taking into account the lag time experienced by each robot when executing the corresponding nominal trajectory.

[0073] To generate a motion plan, generate a nominal trajectory, and / or determine an acceptable lag time, the application 238 may include processor executable instructions that cause one or more processors 222 to call or otherwise perform a collision assessment. Collision assessment is often referred to as "collision detection" or "collision checking," even though the assessment often determines the probability or likelihood of a collision and often occurs before the actual movement of the robot, rather than referring to the detection of an actual physical collision of the robot during the physical movement of the robot. Collision assessment may alternatively be referred to herein as "collision detection," "collision checking," or "collision analysis."

[0074] The application 238 may include processor-executable instructions that cause the one or more processors 222 to set a cost value or cost function for an edge in the motion planning graph, for example, reflecting a determined probability or likelihood of a collision, and optionally other parameters. The application 238 may include processor-executable instructions that cause the one or more processors 222 to set a cost value or cost function for a nominal trajectory, for example, reflecting a corresponding acceptable lag time, and alternatively additionally reflecting a determined probability or likelihood of a collision, and optionally other parameters.

[0075] The application 238 may include processor-executable instructions that cause the one or more processors 222 to evaluate available nominal trajectories generated from the motion planning graph; identify (e.g., select, determine, generate) a nominal trajectory based on, for example, a cost or cost function, and / or identify or generate a motion plan executable by one or more robots to cause the robot to perform a motion, for example, to further perform one or more tasks by the one or more robots. The application 238 may include processor-executable instructions that cause the one or more processors 222 to optionally store the determined motion plan and / or provide instructions to cause the one or more robots to execute or otherwise move according to the motion plan. Motion planning and motion plan construction (e.g., collision evaluation or detection, setting, updating, or adjusting a cost or cost function, for example, based at least in part on collision evaluation or detection, and optionally based in part on a determined acceptable lag time), as well as nominal trajectory generation, analysis or evaluation of candidate nominal trajectories (e.g., selecting between two nominal trajectories based at least in part on corresponding acceptable lag times), can be described herein (e.g., with reference to Figure 3 , 4 , 5, 6, 7 and 8) and the methods described in the references incorporated herein by reference. Collision detection or evaluation can adopt various structures, techniques and algorithms described in this article and elsewhere.

[0076] The application 238 may also include one or more machine-readable and machine-executable instructions that cause the one or more processors 222 to monitor robot motions (e.g., actual trajectories) and evaluate actual delays or actual lag times of these actual trajectories compared to margins or thresholds (e.g., acceptable lag times or based on acceptable lag times) of corresponding nominal trajectories. Optionally, the application 238 may also include one or more machine-readable and machine-executable instructions that cause the one or more processors 222 to select and / or take one or more remedial actions (e.g., slowing down one or more robots, stopping one or more robots, and / or accelerating one or more robots) when necessary (e.g., if the actual lag time approaches or exceeds the acceptable lag time).

[0077] The application 238 may also include one or more machine-readable and machine-executable instructions that optionally cause the one or more processors 222 to monitor the robot in the environment to determine when a path along the actual trajectory becomes unblocked or unobstructed, and in response to the path along the actual trajectory becoming unblocked or unobstructed, cause the robot to move toward a target.

[0078] The application 238 may also additionally include one or more machine-readable and machine-executable instructions that cause the one or more processors 222 to perform other operations, such as, optionally, processing sensory data (captured via sensors). The application 238 may also additionally include one or more machine-executable instructions that cause the one or more processors 222 to perform various other methods described herein and in the references incorporated herein by reference.

[0079] In various embodiments, one or more of the operations described above may be performed by one or more remote processing devices or computers coupled via interface 227 through communication channel 210 (eg, a network).

[0080] Although in Figure 2 224a, but operating system 236, application programs 238, other programs / modules 240, and program data 242 can be stored on other non-transitory computer or processor-readable media (eg, one or more disk drives 224b).

[0081] The motion planner 204a of the first robotic control system 200a may include dedicated motion planner hardware, or may be implemented in whole or in part via one or more processors 222 and processor-executable instructions stored in a system memory 224a and / or one or more disk drives 224b.

[0082] The motion planner 204a may include or implement a motion converter 250, a path generator 252, a collision evaluator 253, a cost setter 254, an optional path analyzer 255, a trajectory generator 256, an acceptable lag time evaluator 257, and an optional nominal trajectory analyzer 258. Each of these can be implemented via one or more processors (e.g., circuits) executing logic, such as executable software instructions, firmware instructions, hard-wired logic, or any combination thereof.

[0083] The motion converter 250 converts the motion of an object (eg, other robot, person) into a representation of an obstacle. The motion converter 250 receives a motion plan 206b or other motion representation from the other motion planner 204b.

[0084] The motion converter 250 can include a trajectory predictor 251 for predicting the trajectory of a transient object (e.g., other robots, other objects, including, for example, a person), for example, when the trajectory of the transient object is unknown (e.g., when the object is another robot but the motion plan of the other robot has not been received, the object is not another robot, such as a person). For example, the trajectory predictor 251 can assume that the object will continue the existing movement in two directions, speeds, and accelerations without changes. For example, the trajectory predictor 251 can take into account expected changes in the motion or path of the object, such as when the path of the object will result in a collision, so that the object can be expected to stop or change direction, or when the object's destination is known, so that the object can be predicted to stop when the destination is reached. In at least some cases, the trajectory predictor 251 can use the learned behavior model of the object to predict the trajectory of the object. For example, the trajectory predictor 251 can extrapolate known motions and expected changes to generate a predicted trajectory for the transient object.

[0085] The motion converter 250 then optionally determines an area or volume corresponding to a known and / or one or more inferred motions of the object. For example, the motion converter can convert the motion to a corresponding swept volume, i.e., a volume swept by a corresponding robot or part thereof when moving or transitioning between poses represented by the motion plan, for example, by generating a volume representation of the robot or part thereof and projecting the volume representation along a path defined by the trajectory of the robot or part thereof. Also for example, the motion converter can convert the motion to a corresponding swept volume, for example, by generating a volume representation of an object (e.g., a non-robotic object such as a human) and projecting the volume representation of the object along a path defined by the known and / or inferred trajectory of the object. Advantageously, the motion planner 204a can simply queue obstacles (e.g., swept volumes) and may not need to determine, track, or indicate the timing of corresponding motions or swept volumes. Although generally described as being for the first robot 202 to move other robots ( Figure 2 204a of the first robot 202, but in some embodiments, other robot control systems 200b of other robots operating in the shared workspace can provide obstacle representations (e.g., swept volumes) for specific motions to the motion planner 204a of the first robot 202.

[0086] The path generator 252 generates a path (e.g., an ending posture, configuration, state; or a destination posture, configuration, and state) from one posture, configuration, or state (e.g., a starting posture, configuration, or state) to another posture, configuration, or state. The path generator 252 can determine or identify one or more feasible paths from the start (e.g., a starting node or a starting posture or a starting state) to the destination (e.g., a destination node or a destination posture or a destination state; an ending node or an ending posture or an ending state). For example, the path generator 252 can determine or identify an ordered sequence of nodes in a motion planning graph that provides a complete path from a starting or current node to a destination or an ending node (i.e., a set of ordered nodes, for which each pair of consecutive nodes in the complete path has a corresponding valid transition, for example, represented by the presence of an edge of a node coupling the pair of nodes). As described above, each node can correspond to a corresponding posture, configuration, or state of a corresponding robot. The path generator 252 can use or execute any type of path finding method, technique, and / or algorithm. For example, the path generator 252 can employ various methods, techniques and / or algorithms to randomly or pseudo-randomly generate a path, select a sequence of nodes in a motion planning graph, wherein the node is connected to the next node in the sequence via an edge (i.e., an edge representing a valid transition between the postures represented by the connecting nodes). The path generator 252 can generate a relatively large number of candidate paths between the starting node and the ending node, thereby generating a large number of candidate paths between the starting posture, configuration or state and the ending posture, configuration or state. In some embodiments, the path generator 252 can determine or identify a feasible path or trajectory independently of a cost (e.g., a cost representing the probability or possibility of experiencing a collision along the path), thereby creating a set of feasible paths or candidate paths, which can be evaluated later based at least in part on the probability or possibility of experiencing a collision. In other cases, the path generator 252 can consider the cost (e.g., a cost representing the probability or possibility of experiencing a collision along the path) when determining or identifying a feasible path or trajectory.

[0087] The collision evaluator 253 performs collision assessment, also referred to as collision detection or collision analysis. Specifically, the collision evaluator 253 optionally performs collision assessment as part of determining whether a candidate path representing a transition or motion specified by a given robot 202 or a portion thereof according to a nominal trajectory will result or may result in a collision with an obstacle. As described above, the motion of other robots can advantageously be represented as obstacles. Thus, the collision evaluator 253 is able to determine whether the motion of one robot will result or may result in a collision with another robot moving in a shared workspace.

[0088] As described herein, collision assessment, detection or analysis can be performed not only on candidate paths, but also, additionally or alternatively, on nominal trajectories (e.g., specified trajectories), and in particular, on nominal trajectories that introduce various lag times (e.g., simulate actual trajectories that may lag behind the nominal trajectories). In at least some embodiments, collision assessment, detection or analysis of nominal trajectories that introduce various lag times can be performed on each of two or more robots (e.g., a pair of robots). Collision assessment, detection or analysis can be performed on one or more nominal trajectories of each robot in the robot pair. Collision assessment, detection or analysis can be performed on each of those nominal trajectories (e.g., one, two or more) that introduce zero lag time and non-zero lag time, for example, evaluating the trajectory pair for each permutation of the lag time of the trajectory pair. As described herein, collision assessment, detection or analysis does not necessarily produce a binary result, but can produce a non-binary value, for example, representing the probability or likelihood of a collision between the robot pair caused by the path or trajectory.

[0089] In some embodiments, the collision evaluator 253 implements software-based collision evaluation, detection, or analysis, for example, performing bounding box-bounding box collision evaluation, detection, or analysis based on a hierarchy of geometric (e.g., sphere) representations of the volume swept by the robot (e.g., the first robot 202) or its parts during movement. In some embodiments, the collision evaluator 253 implements hardware-based collision evaluation, detection, or analysis, for example, using a set of dedicated hardware logic circuits to represent obstacles and streaming the representation of the movement through the dedicated hardware logic circuits. In hardware-based collision evaluation, detection, or analysis, the collision detector can use one or more configurable circuit arrays, for example, one or more FPGA259, and can optionally generate a Boolean collision evaluation.

[0090] Cost setter 254 can set, update and / or adjust the cost or cost function associated with conversion (e.g., edge in motion planning graph) or motion (e.g., trajectory of motion plan). For example, cost setter 254 can set, update and / or adjust the cost or cost function based at least in part on collision assessment, detection or analysis. For example, cost setter 254 can set a relatively high cost value for the edge or trajectory representing the conversion between nodes or motions that cause or may cause a collision. For example, cost setter 254 can set a relatively low cost value for the edge or trajectory representing the conversion between nodes or motions that will not cause or are unlikely to cause a collision. Setting, updating and / or adjusting cost or cost function can include setting, updating or adjusting the cost or cost function logically associated with corresponding edge or trajectory via some data structures (e.g., fields in records, pointers in lists, tables).

[0091] In some embodiments, for example, the cost setter 254 can optionally set, update, or adjust a cost or cost function to at least partially represent a determined acceptable lag time for a corresponding nominal trajectory. This can advantageously enable the first robot control system 200a to select a nominal trajectory from a set of available or candidate nominal trajectories for the robot to perform a given task, for example, selecting a nominal trajectory with the longest or maximum determined acceptable lag time for generating a motion plan that achieves more robust operation when the robot experiences real-world conditions while performing the task.

[0092] In some embodiments, additionally or alternatively, the cost setter 254 can set, update, or adjust a cost or cost function to at least partially represent one or more other parameters, such as one or more of: severity of the collision, consumption or loss of energy, and / or time or delay in execution or completion.

[0093] The optional path analyzer 255 can use the motion planning graph 208 and the cost or cost function to determine or identify one or more suitable paths and / or select a single path (i.e., a selected path, e.g., an optimal or optimized path). For example, the optional path analyzer 255 can identify one or more paths that meet some specified criteria (e.g., the cost is within an upper threshold limit), or even select a single path from a set of candidate feasible paths determined or identified by the path generator 252 (e.g., select a single path with the lowest cost). The identified paths can be named suitable paths because these paths represent options with an acceptably low probability or likelihood of collision. For example, the optional path analyzer 255 can constitute a minimum cost path optimizer that determines the lowest or relatively low cost path between two nodes (thus, between two postures, configurations, or states represented by corresponding nodes in the motion planning graph). The optional path analyzer 255 can use or execute any type of path finding algorithm (e.g., a minimum cost path finding algorithm) taking into account cost values ​​associated with each edge, which cost values ​​represent the probability or likelihood of a collision, and optionally represent one or more other parameters (e.g., severity of the collision, consumption or loss of energy, and / or time or delay to execute or complete). In some embodiments, cost-based optimization can alternatively or additionally be applied to the nominal trajectory to advantageously cause acceptable lag times to be represented in the cost value or cost function, as described herein, for example, in addition to the probability or likelihood of a collision, and optionally in addition to or in place of one or more of: severity of the collision, consumption or loss of energy, and / or time or delay to execute or complete.

[0094] The nominal trajectory generator 256 can generate a nominal trajectory that the robot can follow, e.g., to complete a task. As described herein, the trajectory includes an ordered sequence of poses or configurations or states parameterized by time through which the robot or at least a portion thereof can move, e.g., to complete a task. The nominal trajectory generator 256 can generate trajectories for all generated paths, or if the optional path analyzer 255 is employed, only for selected paths or only for one selected path.

[0095] The acceptable lag time evaluator 257 evaluates various candidate lag times for a given nominal trajectory to determine an acceptable lag time that ensures self-collision-free operation even if the actual trajectory of the corresponding robot does not lag the nominal trajectory by more than the acceptable lag time. In a preferred method, the acceptable lag time evaluator 257 not only considers the effects of lag in the nominal trajectory of the given robot, but also considers the effects of lag in the corresponding nominal trajectories of other robots operating in the shared workspace. Therefore, the acceptable lag time evaluator 257 identifies an acceptable lag time for each nominal trajectory that assumes a worst-case scenario in which the actual trajectories of all robots operating in the shared workspace experience their corresponding acceptable lag times. Therefore, for example, the acceptable lag time evaluator 257 is able to determine an optimized (e.g., maximum) lag time for each robot that ensures self-collision-free operation. Several methods for determining acceptable lag times are described herein, such as with respect to method 300 ( Figure 3 )、Method 400( Figure 4 )、Method 500( Figure 5 )、Method 600( Figure 6 ) and / or method 700 ( Figure 7 ).

[0096] The optional nominal trajectory analyzer 258 can use a cost value or cost value function or some other objective function to determine, identify or select one or more suitable trajectories and / or determine, identify or select a single trajectory (i.e., select a trajectory, e.g., an optimal or optimized trajectory). For example, the nominal trajectory analyzer 258 can determine, identify or select one or more trajectories that meet some specified criteria (e.g., the cost is within an upper threshold limit), or even determine, identify or select a single trajectory (e.g., a trajectory with the lowest cost) from a set of feasible or candidate trajectories generated by the trajectory generator 256 (e.g., determine, identify or select the lowest cost trajectory). For example, the nominal trajectory analyzer 258 can constitute a minimum cost trajectory optimizer that determines the lowest or relatively low cost trajectory between two poses, configurations, or states represented by corresponding nodes in the motion planning graph. For example, the nominal trajectory analyzer 258 can select a set of trajectories that optimize robust operation for all robots, for example, by maximizing the sum of all lag times for all robots, a given task, or a given set of tasks to be performed by the robots within a given time period. The nominal trajectory analyzer 258 is capable of using or executing any type of algorithm, such as a minimum cost trajectory finding algorithm, while taking into account a cost associated with each trajectory, wherein the cost or cost function represents a probability or likelihood of a collision and an acceptable lag time, and optionally, represents one or more of a severity of a collision, a consumption or loss of energy, and / or a time or delay in execution or completion.

[0097] A variety of algorithms and structures may be used to determine the minimum cost path and / or minimum cost trajectory, including algorithms and structures implementing the Bellman-Ford algorithm, but other algorithms and structures may also be used, including but not limited to any such process that determines the minimum cost path or minimum cost trajectory as a path between two nodes in the motion planning graph 208, or a trajectory specified by a time-parameterized ordered sequence of poses, such that the sum of the costs or weights of its constituent edges or motions is minimized. The process improves the robot 102 (by determining an acceptable lag time, optionally selecting a trajectory based on the determined acceptable lag time, monitoring the actual trajectory to assess whether the actual lag time approaches or exceeds a margin or threshold (e.g., the acceptable lag time), and optionally selecting and / or taking remedial action (if necessary). Figure 1 )、202( Figure 2 ) motion planning technology.

[0098] Although not shown, the motion planner 204a may optionally include a forward-looking evaluator that can cause corrective measures to be taken, such as in response to a determination of the presence or occurrence of a blocking or possible blocking condition, or a determination of a location where a blocking or possible blocking will occur when a given robot and / or other robots move along a corresponding trajectory. In at least some embodiments, the forward-looking evaluator is capable of determining or selecting the type of corrective measures to be taken, for example, selecting from a set of different types of corrective measures based on one or more criteria. As described in U.S. patent application 63 / 327,917 filed on April 6, 2022, one or more of various types of corrective measures (referred to as remedial measures) can be implemented. For example, a new, revised, or replaced first motion plan can be generated based on an analysis of a second motion plan that moves a given robot from a first destination to move the given robot to a first destination. In addition, for example, a new, revised, or replaced motion plan can be generated for another robot that is or may be blocking the given robot. Also, for example, a new order of a set of goals can be determined or generated, the set of goals including a first goal and at least a second goal. This can be determined or generated pseudo-randomly or based on one or more heuristics (eg, always trying to move the first target one position downstream relative to the order of the set of targets being modified).

[0099] Although not shown, the motion planner 204a can optionally include an optional multi-path analyzer that can analyze the total cost or aggregate cost associated with the trajectories of two or more motion plans (e.g., the total cost of a first motion plan and a second motion plan), such as described in U.S. patent application 63 / 327,917 filed on April 6, 2022. This can be used to identify the combination of motion plans with the lowest total cost. For example, the multi-path analyzer can consider the total cost or aggregate cost of two or more options for the first motion plan in combination with the second motion plan, for example, using or executing any type of minimum cost search algorithm while considering a cost value that represents the likelihood of a related collision and optionally represents one or more of: an acceptable lag time, a severity of a collision, a consumption or loss of energy, and / or a time or delay associated with executing or completing a transformation represented by a corresponding edge.

[0100] Optionally, the motion planner 204a may include a trimmer 260. The trimmer 260 may receive information indicating that other robots have completed motion, which is referred to herein as a motion completion message 209. Alternatively, a flag may be set to indicate completion. In response, the trimmer 260 may remove an obstacle or a portion of an obstacle representing a now completed motion. This may allow a new motion plan to be generated for a given robot, which may be more efficient, or allow a given robot to perform a task previously blocked by the motion of another robot. The method advantageously allows the motion converter 250 to ignore the timing of motion when generating an obstacle representation of motion, while still achieving better throughput than using other techniques. The motion planner 204a may also send a signal, prompt, or trigger to cause the collision evaluator 253 to perform a new collision detection or evaluation in the case where an obstacle is modified to generate an updated motion planning graph, wherein the edge weight or cost associated with the edge has been modified, and causes the cost setter 254, the optional path analyzer 255, and the nominal trajectory analyzer 258 to update the cost value, and accordingly determine a new or revised path, trajectory, and / or motion plan.

[0101] The motion planner 204a may optionally include an environment converter 263 that converts output from an optional sensor 262 (e.g., a digital camera) (e.g., a digitized representation of the environment) into a representation of obstacles. Thus, the motion planner 204a is able to perform motion planning that takes into account transient objects (e.g., people, animals, etc.) in the environment.

[0102] One or more processors The processor 222 and / or the motion planner 204a may be, or may include, any logical processing unit, such as, for example, one or more central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic controllers (PLCs), etc. Non-limiting examples of commercially available computer systems include, but are not limited to: Intel ® The company provides Celeron, Core, Core2, Itanium and Xeon series microprocessors; the Advanced Micro Devices Inc. provides K8, K10, Bulldozer and Bobcat series microprocessors; the Apple Computer Inc. provides A5, A6 and A7 series microprocessors; the Qualcomm Incorporated provides Snapdragon series microprocessors; and the Oracle Corporation provides SPARC series microprocessors. Figure 2The construction and operation of the various structures shown in the invention may implement or adopt structures, techniques and algorithms described in the following or similar thereto: International Patent Application No. PCT / US2017 / 036880 filed on June 9, 2017; International Patent Application Publication No. WO 2016 / 122840 filed on January 5, 2016; U.S. Patent Application No. 62 / 616783 filed on January 12, 2018; International Patent Application PCT / US2021 / 013610, published as WO 2021 / 150439; and / or U.S. Patent Application 63 / 327917 filed on April 6, 2022.

[0103] Although not required, many embodiments will be described in the general context of computer-executable instructions, such as program application modules, objects, or macros stored on a computer or processor readable medium and executed by one or more computers or processors, that are capable of performing obstacle representation, collision assessment, and other motion planning operations.

[0104] The motion planning operations may include, but are not limited to, generating or converting one, more, or all of the following: a geometric representation of the robot based on a geometric model, a task specification 215, and an optional representation of the volume (e.g., a swept volume) occupied by the robot in various states or poses and / or during movement between states or poses, in digital form, such as a point cloud, a Euclidean distance field, a data structure format (e.g., a hierarchical format, a non-hierarchical format), and / or a curve (e.g., a polynomial or spline representation). The motion planning operations may optionally include, but are not limited to, generating or converting one, more, or all of the following: a representation of static or persistent obstacles and / or perception data of static or transient obstacles into digital form, such as a point cloud, a Euclidean distance field, a data structure format (e.g., a hierarchical format, a non-hierarchical format), and / or a curve (e.g., a polynomial or spline representation).

[0105] Motion planning operations may include, but are not limited to, using various collision assessment techniques or algorithms (e.g., software-based, hardware-based) to assess, detect, determine, or predict collisions for transitions between various poses, configurations, or states of the robot, or for movement of the robot between states or poses along corresponding trajectories.

[0106] In some embodiments, motion planning operations may include, but are not limited to: determining one or more motion planning maps, motion plans, or roadmaps having a nominal trajectory; an acceptable lag time for the nominal trajectory, storing the determined planning maps, motion plans, or roadmaps or the acceptable lag time, and / or providing planning maps, motion plans, or roadmaps or the acceptable lag time to control the operation of one or more robots, and optionally monitoring the operation of the robots, and selecting and / or taking remedial measures when necessary.

[0107] In one embodiment, collision detection or evaluation is performed in response to a function call or similar process and a Boolean value is returned thereto. The collision evaluator 253 may be implemented via one or more field programmable gate arrays (FPGAs) 259 and / or one or more application specific integrated circuits (ASICs) to achieve low latency, relatively low power consumption, and increase the amount of information that can be processed while performing collision detection.

[0108] In various embodiments, these operations may be performed entirely in hardware circuitry or as software stored in a memory such as system memory 224 a and performed by one or more hardware processors 222 a, such as one or more microprocessors, digital signal processors (DSPs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), graphics processing unit (GPU) processors, programmable logic controllers (PLCs), electrically programmable read-only memories (EEPROMs), or as a combination of hardware circuitry and software stored in memory.

[0109] Various aspects of sensing, map construction, collision detection, and path searching that may be employed in whole or in part are also described in International Patent Application No. PCT / US2017 / 036880, filed on June 9, 2017; International Patent Application No. WO 2016 / 122840, filed on January 5, 2016; U.S. Patent Application No. 62 / 616783, filed on January 12, 2018; U.S. Patent Application No. 62 / 856548, filed on June 3, 2019; International Patent Application No. PCT / US2020 / 039193, filed on June 23, 2020 (published as WO 2020 / 263861); International Patent Application No. PCT / US2021 / 013610, published as WO 2021 / 150439; and / or U.S. Patent Application No. 63 / 327917, filed on April 6, 2022. Those skilled in the relevant art will appreciate that the illustrated embodiments and other embodiments can be practiced with other system structures and arrangements and / or other computing system structures and arrangements, including robots, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, personal computers ("PCs"), networked PCs, microcomputers, mainframe computers, etc. These embodiments or examples or portions thereof (e.g., at configuration time and runtime) can be practiced in a distributed computing environment when tasks or modules are performed by a remote processing device connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote storage devices or media. However, where and how certain types of information are stored is important for helping to improve motion planning.

[0110] For example, various motion planning solutions "bake in" a roadmap (i.e., a motion planning graph) into a processor (e.g., FPGA 259), and each edge in the roadmap corresponds to a non-reconfigurable Boolean circuit of the processor. A problem with designs that "bake" the planning graph into the processor is that the processor circuits are limited and cannot store multiple or large planning graphs, and are generally not reconfigurable for use with different robots.

[0111] One solution is to provide a reconfigurable design that puts the map information into memory. This approach stores the information in memory instead of embedding it in the circuit. Another approach is to use templated reconfigurable circuits instead of memory.

[0112] As described above, some information (e.g., robot geometry) may be captured, received, input, or provided during configuration time (i.e., before runtime). The received information may be processed during configuration time to generate processed information (e.g., motion planning graph 208) to speed up operations or reduce computational complexity during runtime.

[0113] During runtime, collision detection can be performed on the entire environment (i.e., the shared workspace), including determining, for any pose or movement between poses, whether any part of the robot will or is expected to collide with another part of the robot itself, with other robots or parts thereof, with persistent or static obstacles in the environment, or with transient obstacles with unknown trajectories in the environment (e.g., people).

[0114] The first robot control system 200a or some other processor-based system may include a lag monitor 264 (interchangeably referred to as a lag time monitor 264) and / or a remedial action selector 266. The lag time monitor 264 and / or the remedial action selector 266 operate during runtime (e.g., during operation of one or more robots to complete one or more tasks), monitor the operation of the robots, and select and take one or more remedial actions when necessary (e.g., slowing or stopping one or more robots if the actual lag time approaches or exceeds a margin or threshold (e.g., a corresponding acceptable lag time)).

[0115] The lag time monitor 264 monitors the actual lag time of the actual trajectory executed by the robot and compares the actual lag time with a margin or threshold of the corresponding nominal trajectory (e.g., a margin or threshold representing or even equal to a corresponding acceptable lag time). As previously described, although the actual trajectory may match the nominal trajectory in some cases, in many cases, the actual trajectory will not match the nominal trajectory, typically, the execution timing of at least some postures, configurations, states, or motions of the actual trajectory lags behind the timing of those postures, configuration states, or motions specified by the nominal trajectory. The lag time monitor 264 determines the actual lag time, and whether the monitored amount of the actual lag time of the actual trajectory of the robot approaches or exceeds the margin or threshold of the corresponding robot (e.g., the corresponding determined acceptable lag time of the corresponding nominal trajectory). For example, if the actual lag time of any robot operating in the shared workspace exceeds the corresponding threshold (e.g., approaches or exceeds the corresponding acceptable lag time), the lag time monitor 264 can provide an indication (e.g., set a flag, send a message, and / or call a remedial action selector 266). In some embodiments, the margin or threshold can be set to include a desired safety factor, such as a set amount or defined percentage less than a determined acceptable lag time of a corresponding nominal trajectory. Thus, in some embodiments, the margin or threshold can be set equal to the corresponding acceptable lag time, while in other embodiments, the margin or threshold can be less than (e.g., less than 10%) the corresponding acceptable lag time, thereby allowing detection and remedial measures to be taken before a self-collision may occur.

[0116] The selectable remedial action selector 266 selects one or more appropriate remedial actions and / or causes the selected remedial action to be taken in response to determining that the monitored amount of lag time exceeds a corresponding threshold value (e.g., a determined acceptable lag time) for a corresponding trajectory of any robot. Determining that the actual lag time exceeds the corresponding threshold value (e.g., a determined acceptable lag time) means that self-collision-free movement is no longer ensured, and therefore one or more remedial actions may need to be taken.

[0117] The remedial measures can include one or more of the following: stopping the movement of one or more robots, slowing down the movement of one or more robots and / or speeding up the movement of one or more robots. For example, the movement of one, two, more or even all robots can be stopped. In addition, for example, the movement of one, two, more or even all robots can be slowed down, for example, by different amounts relative to each other. In addition, for example, the movement of one, two, more or even all robots can be speeded up, for example, by different amounts relative to each other. In addition, for example, the movement of one or more robots can be stopped while slowing down the movement of one or more robots. In addition, for example, the movement of one or more robots can be stopped while speeding up the movement of one or more robots or keeping the movement of one or more robots constant. In addition, for example, the movement of one or more robots can be slowed down while speeding up the movement of one or more robots or keeping the movement of one or more robots constant. In addition, for example, the movement of one or more robots can be stopped while slowing down the movement of one or more robots and speeding up the movement of one or more robots or keeping the movement of one or more robots constant. To take at least one remedial action, the processor-based system can send instructions to one or more actuators (e.g., via the actuators 218a to 218c of the motion controller 220) to stop, slow down, speed up, or even advance the movement of the robot along the corresponding trajectory specified by the corresponding nominal trajectory. This can be followed by restarting the movement of the robot, for example, restarting the movement from where the movement stopped or slowed down, although in some cases it may include returning to the beginning of the corresponding nominal trajectory and restarting from there.

[0118] Figure 3A method 300 of operating a processor-based system according to at least one illustrated embodiment is shown that performs motion planning to control robots that will operate in a shared workspace. According to at least one illustrated embodiment, the processor-based system includes one or more processors that execute processor-executable instructions to determine an acceptable lag time for each robot based on the nominal trajectory of other robots without considering the effect of lag time in the nominal trajectory of other robots, and generate or select a motion plan that specifies the nominal trajectory, which is associated with the corresponding acceptable lag time. For example, the method 300 can be performed during configuration or "pre-run" time, that is, the time when some or all of motion planning occurs, for example, generating a nominal trajectory and / or motion plan for each robot. The configuration or "pre-run" time can occur before the robot moves according to the nominal trajectory of the corresponding robot, for example, before the run time, where the run time is the time when one or more robots execute the corresponding motion plan. This advantageously allows some of the most computationally intensive work to be performed before the run time, because responsiveness is not particularly concerned at this time. In at least some embodiments, the method 300 or portions thereof is performed during run time (i.e., the time when one or more robots are performing a task) after a configuration time or a pre-run time. In other embodiments, the method 300 is performed during run time (i.e., the time when one or more robots are performing a task) after a configuration time or a pre-run time.

[0119] The method 300 begins at 302, for example, in response to startup or power-up of a system or component thereof, receipt of information or data, or invocation or invocation of a calling routine or program.

[0120] Although Figure 3 Although not shown, method 300 can generate paths; perform collision evaluation or detection or analysis on these paths; set, update or adjust the cost of the edges of the paths; and optionally perform analysis to identify or select a path based on the cost (e.g., the lowest cost path). Figure 2 This is described, and at least some aspects of the references cited herein are described. For the sake of brevity, these are omitted in method 300.

[0121] Optionally, at 304, at least one processor of the processor-based system generates or accesses (e.g., receives, retrieves) a set of nominal trajectories for each robot 1. As previously described, a nominal trajectory is a "specified" trajectory, wherein each trajectory includes an ordered set or sequence of postures, configurations, or states of the robot, the ordered set or sequence extending between two postures, configurations, and states (e.g., a starting posture, configuration, or state; an ending posture, configuration, state), and includes a corresponding timing for each posture or configuration. The timing can be specified in relative terms (timing is defined by a relative offset from a previous posture) or absolute terms (timing is defined by a relative offset from the start of trajectory execution, for example, relative to a common clock). Applicants note that while any given trajectory can correspond to a smooth motion of a robot, the term "trajectory" as used herein is not limited thereto, and will generally specify motion that is not smooth, and will not define a straight path for a robot or part thereof. In at least some cases, the nominal trajectory can specify or include one or more pauses in the motion of the robot or its part, and / or can specify a change in the direction of motion or path direction of the robot or its part, even a reversal, a change in rate, a change in speed, and a change in acceleration, and may not be a smooth motion in terms of direction, rate, speed, or time. For example, the nominal trajectory can specify a time-parameterized ordered set of postures or a sequence of postures through which the robot or its part moves to perform a task or a portion of a task. One or more nominal trajectories can be used to perform any given task. As described herein, the actual motion or actual trajectory of the robot or its part may deviate from the corresponding nominal trajectory, for example, due to unexpected delays in transitioning between postures (e.g., due to the need to stay or dwell on a target object for a longer time than expected).

[0122] The nominal trajectory can be generated using any of a variety of techniques and / or algorithms, such as a sample-based motion planner (SBMP), such as a probabilistic roadmap (PRM) or a rapidly exploring random tree (RRT, RTT*s), a stable sparse RRT* (SST*) and / or a fast matching tree (FMT). In at least one embodiment, the nominal trajectory can be generated via the systems, methods and techniques described in international patent application PCT / US2021 / 013610 (published as WO 2021150439A1). The teachings of this patent application are not limited to the generation of a specific form of nominal trajectory. Advantageously, the teachings herein include any combination or arrangement of the following aspects: computationally efficiently using nominal trajectories to generate motion plans with associated acceptable lag times, monitoring deviations of actual motion from the nominal trajectory in terms of lag times, optionally taking remedial measures for deviations exceeding corresponding acceptable lag times, and / or optionally using a determined lag time to select a set of nominal trajectories for the motion plan to enhance the robustness of the operation.

[0123] At 306, at least one processor of the processor-based system initializes a robot counter 1, e.g., sets the robot counter equal to an integer value of 1. At 308, at least one processor of the processor-based system executes an outer iteration loop, performing iterations for each of two or more robots to be operated in the shared workspace, or until a stop condition is reached (e.g., determining a maximum lag time that ensures self-collision-free movement).

[0124] At 310 , at least one processor of the processor-based system initializes a nominal trace counter J, for example, setting the nominal trace counter equal to an integer value of one.

[0125] At 312, at least one processor of the processor-based system executes an inner iteration loop, performing iterations for each nominal trajectory of a given robot, at least until a stop condition is reached.

[0126] In 314, at least one processor of the processor-based system determines a corresponding acceptable lag time for a current nominal trajectory (i.e., current in an inner iteration loop) of a current robot (i.e., current in an outer iteration loop). The corresponding acceptable lag time reflects a maximum acceptable delay or lag in the current nominal trajectory J that still ensures self-collision-free movement of the current robot relative to other robots that each move according to a corresponding nominal trajectory when the current robot executes the nominal trajectory (the corresponding acceptable lag time is introduced into the nominal trajectory of the current robot). As long as all other robots are running according to their corresponding nominal trajectories (i.e., lag time=0), the acceptable lag time of a given robot I can ensure that no self-collision will occur. In other words, as long as the corresponding current actual lag time of a given robot is less than or equal to (i.e., not greater than) the corresponding acceptable lag time of the robot, and the other robots are running according to their nominal trajectories, the robots will not collide with each other (i.e., it is guaranteed that no self-collision will occur between the robots).

[0127] At 316, at least one processor of the processor-based system determines whether each of the nominal trajectories of the given robot has been considered, e.g., whether the nominal trajectory counter J is equal to the total number of nominal trajectories of the given robot (e.g., J=M?). If each of the nominal trajectories of the given robot has not been considered, control passes to 318, where the nominal trajectory counter is incremented (e.g., J=J+1), and then control returns to 312 to consider the next nominal trajectory of the given robot. If each of the nominal trajectories of the given robot has been considered (e.g., J=M), control passes directly to 320.

[0128] Optionally, in 320, at least one processor of the processor-based system selects the maximum value from a set of determined acceptable latency times determined for a given robot.

[0129] In 322, at least one processor of the processor-based system provides one or more determined acceptable latency times (e.g., provides the maximum value of the selected determined acceptable latency times) for the given robot to be used in determining at least the motion plan of the given robot and / or controlling the operation of the given robot. For example, this can include providing one or more determined acceptable latency times to different processors, or transmitting one or more determined acceptable latency times to different registers of the processor.

[0130] In 324, at least one processor of the processor-based system determines whether each robot among the robots has been considered, e.g., determines whether the robot counter I is equal to the total number of robots (e.g., I = N?). If not every robot has been considered (e.g., I < N), the control goes to 326, where at least one processor of the processor-based system iterates the robot counter (e.g., I = I + 1), and then the control returns to 308 to consider the next robot. If every robot has been processed (e.g., I = N), the control goes directly to 328.

[0131] In 328, at least one processor of the processor-based system generates or selects a corresponding motion plan for each robot. The motion plan can be or can represent the nominal trajectory of the robot associated with the maximum acceptable latency time. The motion plan can be or can represent the nominal trajectory of one or more robots. This approach can advantageously enhance the robustness of the motion plan generated by motion planning and thus improve the operation of the robots executing the resulting motion plan.

[0132] In 330, at least one processor of the processor-based system provides the corresponding motion plan to each robot or motion controller to cause the robot to move according to the corresponding motion plan.

[0133] In 332, method 300 can terminate, e.g., until called again. Although method 300 is described as an ordered process, in many embodiments, various actions or operations will be performed simultaneously or in parallel, and / or can include additional actions and / or omit some actions.

[0134] Figure 4A method 400 of operating a processor-based system according to at least one illustrated embodiment is shown, the system performing motion planning to control robots that will operate in a shared workspace. The processor-based system includes one or more processors that execute processor-executable instructions to determine an acceptable lag time for each robot based on the nominal trajectory of other robots, and at the same time, the effect of lag time in the nominal trajectory of other robots must be considered, and a motion plan for a specified nominal trajectory is generated or selected, and the nominal trajectory is associated with the corresponding acceptable lag time. For example, the method 400 can be performed during configuration or "pre-run" time, that is, the time when some or all of the motion planning occurs, for example, generating a nominal trajectory and / or motion plan for each robot. The configuration or "pre-run" time can occur before the robot moves according to the nominal trajectory of the corresponding robot, for example, before the run time, wherein the run time is the time when one or more robots execute the corresponding motion plan. This advantageously allows some of the most computationally intensive work to be performed before the run time, because responsiveness is not particularly concerned at this time. In at least some embodiments, the method 400 or a portion thereof is performed during run time (i.e., the time when one or more robots are performing a task) after the configuration time or pre-run time. In still other embodiments, the method 400 is performed during run time (i.e., the time when one or more robots are performing a task) after the configuration time or pre-run time.

[0135] The method 400 begins at 402, for example, in response to startup or power-up of a system or component thereof, receipt of information or data, or invocation or invocation of a calling routine or program.

[0136] Although Figure 4 , but method 400 can generate paths; perform collision evaluation or detection or analysis on these paths; set, update or adjust the cost of the edges of the paths; and optionally perform analysis to identify or select a path based on the cost (e.g., the lowest cost path). Figure 2 This is described, and at least some aspects of the references cited herein are described. For the sake of brevity, these are omitted in method 400.

[0137] Optionally, at 404, at least one processor of the processor-based system generates or accesses (e.g., receives, retrieves) a set of nominal trajectories for each robot (i). As previously described, a nominal trajectory is a "specified" trajectory, wherein each trajectory includes an ordered set or sequence of postures, configurations, and states of the robot between two postures, configurations, or states, and includes corresponding timing for each posture or configuration. Timing can be specified in relative terms (timing is defined by a relative offset from a previous posture) or absolute terms (timing is defined by a relative offset from the start of trajectory execution, for example, relative to a common clock). Applicants note that while any given trajectory can correspond to smooth motion of the robot, the term "trajectory" as used herein is not limited to this, and will generally specify motion that is not smooth, and will not define a straight path for a robot or part thereof. In at least some cases, a nominal trajectory can specify or include one or more pauses in the motion of a robot or part thereof, and / or can specify a change in direction or path direction of motion of a robot or part thereof, even a reversal, a change in rate, a change in velocity, and a change in acceleration, and may not be a smooth motion in terms of direction, rate, speed, or time. For example, a nominal trajectory can specify a time-parameterized ordered set of poses or a sequence of poses through which a robot or a portion thereof moves to perform a task or a portion thereof. One or more nominal trajectories can be employed to perform any given task. As described herein, the actual motion or actual trajectory of a robot or a portion thereof may deviate from a corresponding nominal trajectory, for example, due to unexpected delays in transitioning between poses (e.g., due to a need to linger or stay on a target object for a longer time than expected).

[0138] The nominal trajectory can be generated using any of a variety of techniques and / or algorithms, such as a sample-based motion planner (SBMP), such as a probabilistic roadmap (PRM) or a rapidly exploring random tree (RRT, RTT*s), a stable sparse RRT* (SST*) and / or a fast matching tree (FMT). In at least one embodiment, the nominal trajectory can be generated via the systems, methods and techniques described in international patent application PCT / US2021 / 013610 (published as WO 2021150439A1). The teachings of this patent application are not limited to the generation of a specific form of nominal trajectory. Advantageously, the teachings herein include any combination or arrangement of the following aspects: computationally efficiently using nominal trajectories to generate motion plans with associated acceptable lag times, monitoring deviations of actual motion from the nominal trajectory in terms of lag times, optionally taking remedial measures for deviations exceeding corresponding acceptable lag times, and / or optionally using a determined lag time to select a set of nominal trajectories for the motion plan to enhance the robustness of the operation.

[0139] At 406, at least one processor of the processor-based system initializes a robot counter 1, e.g., sets the robot counter equal to an integer value of 1. At 408, at least one processor of the processor-based system executes an outer iteration loop, performing iterations for each of two or more robots to be operated in the shared workspace, or until a stop condition is reached (e.g., determining a maximum lag time that ensures self-collision-free motion between the robots).

[0140] At 410, at least one processor of the processor-based system initializes a nominal trajectory counter J, e.g., sets the nominal trajectory counter equal to an integer value of 1. At 412, at least one processor of the processor-based system executes an inner iteration loop, performing iterations for each nominal trajectory of a given robot, at least until a stop condition is reached.

[0141] At 414, at least one processor of the processor-based system determines a corresponding acceptable lag time for a current nominal trajectory (i.e., current in an inner iteration loop) of a current robot (i.e., current in an outer iteration loop). The corresponding acceptable lag time reflects a maximum acceptable delay or lag in the current nominal trajectory J that still ensures that the current robot moves relative to the respective robot according to the corresponding nominal trajectory (the corresponding acceptable lag time is introduced into the nominal trajectory of the current robot) when the current robot executes the nominal trajectory (the corresponding acceptable lag time is introduced into the nominal trajectory of the current robot).

[0142] The acceptable lag time of a given robot 1 guarantees that self-collision will not occur as long as the current actual lag time of all other robots is less than their corresponding acceptable lag time. That is, the other robots do not have to operate on their corresponding nominal trajectories (i.e., lag time=0). In other words, as long as the corresponding current actual lag time of each of all robots operating in the shared workspace is less than or equal to (i.e., not greater than) the corresponding acceptable lag time of the robot, the robots will not collide with each other or with themselves (i.e., self-collision between robots is guaranteed to be non-existent).

[0143] In 416, at least one processor of a processor-based system determines whether each nominal trajectory in a given robot's nominal trajectories has been considered, e.g., determines whether a nominal trajectory counter J is equal to the total number of nominal trajectories of the given robot (e.g., J = M?). If not every nominal trajectory in the given robot's nominal trajectories has been considered, control passes to 418, where the nominal trajectory counter is incremented (e.g., J = J + 1), and then control returns to 412 to consider the next nominal trajectory of the given robot. If every nominal trajectory of the given robot has been considered (e.g., J = M), control passes directly to 420.

[0144] Optionally, in 420, at least one processor of a processor-based system selects the maximum value among a set of determined acceptable lag times determined for a given robot. In 422, at least one processor of the processor-based system provides one or more determined acceptable lag times (e.g., the maximum value of the selected determined acceptable receive lag times) for the given robot for determining at least the motion plan of the given robot and / or controlling the operation of the given robot. For example, this can include providing one or more determined acceptable lag times to different processors or transmitting one or more determined acceptable lag times to different registers of a processor.

[0145] In 424, at least one processor of a processor-based system determines whether each robot among the robots has been considered, e.g., determines whether a robot counter I is equal to the total number of robots (e.g., I = N?). If not every robot among the robots has been considered (e.g., I < N), control passes to 426, where at least one processor of the processor-based system iterates the robot counter (e.g., I = I + 1), and then control returns to 408 to consider the next robot. If every robot among the robots has been processed (e.g., I = N), control passes directly to 428.

[0146] In 428, at least one processor of a processor-based system generates or selects a corresponding motion plan for each robot. The motion plan can be or can represent the nominal trajectory of the robot associated with the maximum acceptable lag time. The motion plan can be or can represent the nominal trajectories of one or more robots. This approach can advantageously enhance the robustness of the motion plans generated by motion planning and thus improve the operation of the robots executing the resulting motion plans.

[0147] In 430, at least one processor of a processor-based system provides the corresponding motion plan to

[0148] each robot or each motion controller to cause the robots to move according to the corresponding motion plans.

[0149] In 432, method 400 may terminate, for example, until called again. Although method 400 is described as an ordered flow, in many implementations, various actions or operations will be performed simultaneously or in parallel, and / or may include additional actions and / or omit some actions.

[0150] As described herein, in order to determine an acceptable lag time, a processor-based system can perform a collision assessment to determine which of a set of candidate lag times will or may cause a collision. One method of collision assessment is to use a swept volume of a robot to perform the collision assessment. A swept volume represents a volume swept by a robot or a portion thereof as it moves along a given trajectory. The swept volume is specified by the geometry and kinematics of a given robot and the start and end positions specified by a given trajectory. It is noteworthy that the swept volume itself is not affected by the specific time to perform the motion specified by the trajectory. Therefore, the swept volume can be advantageously determined during configuration or pre-run time before the runtime of the robot performing or executing a task. Therefore, the processor-based system can generate a swept volume for each of a plurality of (e.g., many) paths for each robot that will operate in a shared workspace, from which a trajectory is generated for the robot. The processor-based system can use the swept volume to perform the collision assessment. For example, for each of the two or more robots, the processor-based system can perform a collision assessment between i) and ii) below, wherein i) represents at least a portion of a corresponding sample trajectory of a corresponding nominal trajectory of the robot with at least one corresponding lag time introduced, and ii) at least a portion of a corresponding sample of each of the corresponding trajectories of each of the other robots of the two or more robots, wherein at least one corresponding lag time is introduced into the corresponding trajectory of each of the other robots of the two or more robots. In response to determining that a collision will or may occur between the robot and at least one of the other robots of the two or more robots (e.g., represented as a probability or likelihood), the processor-based system can identify, for one or more nominal trajectories of the robot, a corresponding lag time that is less than the corresponding lag time that resulted in the determination that a collision will occur as the corresponding acceptable lag time. The acceptable lag time can represent a maximum acceptable delay in the timing of a pose relative to the nominal trajectory that still ensures that the movement of the robot through the sequence of poses specified by the nominal trajectory remains collision-free (self-collision-free for the robot system) relative to the movement of the other robots of the two or more robots. The following Figure 5 and Figure 6 A non-limiting method for determining acceptable lag times using swept volumes is described.

[0151] Figure 5A method 500 of operating a processor-based system according to at least one illustrated embodiment is shown for generating a swept volume for one or more trajectories for each of a plurality of robots to be operated in a shared workspace. According to at least one illustrated embodiment, the processor-based system includes one or more processors that execute processor-executable instructions to generate a swept volume. For example, the method 500 can be performed during configuration or "pre-run" time, i.e., the time when some or all of motion planning occurs, e.g., generating a nominal trajectory and / or motion plan for each robot. For example, the configuration or "pre-run" time can be before run time, where run time is the time when one or more robots execute the corresponding motion plan. This advantageously allows some of the most computationally intensive work to be performed before run time, when responsiveness is not particularly concerned. In at least some embodiments, the robot configuration method 500 or portions thereof is performed during run time (i.e., the time when one or more robots are performing a task) after the configuration time or pre-run time. Optionally, the configuration-time operations can be performed via a processor-based system that is different from the processor-based system that performs the run-time operations. For example, the method 500 can optionally be used to perform Figure 3 , 4 and / or the method of 6, for example, as in determining the corresponding acceptable lag time 314 ( Figure 3 ) and 414 ( Figure 4 ) when performing collision detection (see method 600, Figure 6 ) part.

[0152] The method 500 of generating a swept volume begins at 502 , for example, in response to startup or power-up of a system or component thereof, receipt of information or data, or invocation or invocation of a calling routine or program.

[0153] At least one processor of the processor-based system initializes a robot counter I (e.g., I=1) in 504. At least one processor of the processor-based system executes an external robot processing loop in 506. The external robot processing loop allows at least one processor of the processor-based system to perform generation of a swept volume for each of the robots to be operated in the shared workspace.

[0154] At 508, at least one processor of the processor-based system initializes a trajectory counter J (e.g., J=1). At 510, at least one processor of the processor-based system executes an inner nominal trajectory processing loop. The inner nominal trajectory processing loop is nested within the outer robot processing loop. The inner nominal trajectory processing loop allows the at least one processor of the processor-based system to generate a corresponding swept volume for each of one or more nominal trajectories of a given robot.

[0155] At 512, at least one processor of the processor-based system generates a swept volume representation representing a volume swept by at least a portion of a given robot I as it moves between poses of a set or sequence of poses specified by a given nominal trajectory J. Thus, for each of the two or more robots, and for each of the one or more nominal trajectories, the processor-based system can generate a swept volume representation representing a volume swept by at least a portion of the robot as it moves between a set of poses specified by the nominal trajectory, from at least one time of the nominal trajectory to another time of the nominal trajectory. The swept volume can be generated using any of a variety of techniques, including digitally representing the robot or portion thereof with one or more (e.g., hierarchical) data structures or point clouds, and projecting the digital representation along a path or trajectory to generate a digital representation (e.g., a set of voxels) of the swept volume.

[0156] At least one processor of the processor-based system determines whether all trajectories for a given robot have been processed (e.g., J=M?) at 514. If all trajectories for a given robot have not been processed, control passes to 516 where a trajectory counter is incremented (e.g., J=J+1) and control then returns to the top of the inner nominal trajectory processing loop 510. If all nominal trajectories have been processed (e.g., J=M), control passes directly to 518.

[0157] At least one processor of the processor-based system determines whether all robots have been processed (e.g., I=N?) at 518. If all robots have not been processed, control passes to 520 where the robot counter is incremented (e.g., I=I+1) and control then returns to the top of the outer robot processing loop 506. If all robots have been processed (e.g., I=N), control passes directly to 522.

[0158] In 522, method 500 may terminate, for example, until called again. Although method 500 is described as an ordered flow, in many implementations, various actions or operations will be performed simultaneously or in parallel, and / or may include additional actions and / or omit some actions.

[0159] Figure 6A method 600 of operating a processor-based system according to at least one illustrated embodiment is shown for performing collision assessment for a plurality of robots and a plurality of trajectories of each of the robots. The method 600 can be used to perform collision assessment to identify feasible paths between a start node and an end node in a motion planning graph. Additionally or alternatively, the method 600 can be used to perform collision assessment to determine corresponding acceptable lag times for one or more nominal trajectories of robots operating in a shared workspace. According to at least one illustrated embodiment, the processor-based system includes one or more processors that execute processor-executable instructions to perform collision assessment, for example, using a swept volume. The swept volume represents or simulates a volume swept by a robot or a portion thereof in the following process: for example, in transitions between various postures, configurations, or states of transitions between a start node and an end node of a path represented in a motion planning graph, or, for example, in nominal trajectory movement between a start posture, configuration, or state to an end posture, configuration, or state. For example, the method 600 can advantageously use a swept volume generated via the method 500. For example, the method 600 can be executed during configuration or "pre-run" time, i.e., the time when some or all of motion planning occurs, e.g., generating a nominal trajectory and / or motion plan for each robot. For example, the configuration or "pre-run" time can be before run time, where run time is the time when one or more robots execute a corresponding motion plan. This advantageously allows some of the most computationally intensive work to be performed before run time, when responsiveness is not a particular concern. In at least some embodiments, the robot configuration method 600, or portions thereof, is executed during run time (i.e., the time when one or more robots are performing a task) after the configuration time or pre-run time.

[0160] The method 600 of determining acceptable latency begins at 602, for example, in response to startup or power-up of a system or component thereof, receipt of information or data, or invocation or invocation of a calling routine or program.

[0161] At least one processor of the processor-based system initiates an external robot pair collision assessment loop at 604. The external robot pair collision assessment loop allows the at least one processor of the processor-based system to assess or estimate the probability or likelihood of a collision between robots in each pair of robots that will operate in the shared workspace. The external robot pair collision assessment loop is capable of handling collisions that may occur in each combination or arrangement of robot pairs.

[0162] At 606, at least one processor of the processor-based system initializes a trajectory counter J (e.g., J=1). At 608, at least one processor of the processor-based system executes an inner nominal trajectory collision evaluation loop. The inner nominal trajectory collision evaluation loop is nested within the outer robot collision evaluation loop. The inner nominal trajectory collision evaluation loop allows the at least one processor of the processor-based system to evaluate or estimate a collision probability or likelihood for each of one or more nominal trajectories of a given robot pair.

[0163] At 610, at least one processor of the processor-based system initializes a lag time counter K (e.g., K=1). At 612, at least one processor of the processor-based system executes a nested inner lag time collision evaluation loop. The nested inner lag time collision evaluation loop is nested within the inner nominal trajectory collision evaluation loop. The nested inner lag time collision evaluation loop allows the at least one processor of the processor-based system to evaluate or estimate the impact of each of a plurality of possible lag times (i.e., candidate lag times from a set of candidate lag times) when introduced into a given nominal trajectory. Thus, for example, nominal trajectories introducing a lag time equal to zero and introducing multiple (one, two, or more) non-zero lag times may be evaluated to determine the probability or likelihood of a collision if a trajectory corresponding to the nominal trajectory introducing the corresponding lag times is executed. Successively longer lag times may be evaluated.

[0164] At least one processor of the processor-based system uses the generated swept volumes to perform collision assessments on corresponding trajectories that introduce lag times (e.g., for zero and non-zero lag times) at 614. For example, at least one processor of the processor-based system can determine whether corresponding swept volumes intersect, wherein the corresponding swept volumes correspond to corresponding volumes swept by a robot in a pair of robots when executing a nominal trajectory that introduces a lag time equal to zero and introduces each of a plurality of non-zero lag times. For example, for each of the two or more robots, the processor-based system can perform a collision assessment between i) and ii) representing at least a portion of a corresponding sample trajectory of a corresponding nominal trajectory of the robot that introduces at least one corresponding lag time (e.g., zero lag time and at least one non-zero lag time), and ii) at least a portion of each corresponding sample of the corresponding trajectory of each of the other of the two or more robots, wherein at least a portion of the corresponding sample of each of the corresponding trajectories of each of the other of the two or more robots wherein at least one corresponding lag time (e.g., zero lag time and at least one non-zero lag time) is introduced into the corresponding trajectory of each of the other of the two or more robots. Thus, for each of the robots and each of the nominal trajectories, all permutations of "candidate" lag times can be evaluated, e.g., to identify when the probability or likelihood of a collision equals or exceeds a certain collision detection threshold.

[0165] Although collision assessment is described in terms of swept volumes, the teachings herein are not necessarily limited in this regard, and the methods and systems can employ various other collision assessment or detection techniques or algorithms.

[0166] At 616, at least one processor of the processor-based system determines whether a given pair of robots will or may cause a collision based on the collision assessment. In at least some embodiments, the collision assessment can produce a non-binary value representing a chance or probability of a collision. Whether a collision will or may cause a collision can be determined based on the non-binary value being above a defined collision detection threshold (e.g., collision probability >50%; collision probability >10%; collision probability >5%). At 617, in response to determining that a collision has been detected or may occur (e.g., collision probability is equal to or greater than the collision detection threshold), the processor-based system can identify and / or store previous lag time values ​​that did not result in a potential collision and optionally exit the nested inner lag time collision assessment loop 612, returning control to the top of the inner nominal trajectory collision assessment loop 608. Thus, for example, in response to determining that a collision will or may occur between the robot and at least one other robot of the two or more robots, for one or more nominal trajectories of at least one of the two or more other robots that have been determined to be colliding, the processor-based system identifies a corresponding lag time that is less than the determined corresponding lag time that led to the determination that a collision will occur as the corresponding acceptable lag time, wherein the acceptable lag reflects a maximum acceptable delay in gesture timing relative to the nominal trajectory that still ensures that the movement of the robot through the gesture sequence specified by the corresponding nominal trajectory remains collision-free relative to the movement of the other robots of the two or more robots. In response to determining that a collision has been detected or may be detected (e.g., the collision probability is equal to or greater than the collision detection threshold), control directly passes to 618.

[0167] At least one processor of the processor-based system determines whether all lag times for the current trajectory of the current robot pair have been considered (e.g., K=P?) at 618. If all lag times have not been considered, then at 620, a lag time counter K is incremented (e.g., K=K+1) to select the next candidate lag time from the set of candidate lag times for evaluation, and control returns to the top of the nested inner lag time collision evaluation loop 612. If all lag times have been considered, then control passes directly to 622.

[0168] At least one processor of the processor-based system determines whether all trajectories for the given robot pair have been processed (e.g., J=M?) at 622. If all trajectories for the given robot pair have not been processed, the trajectory counter is incremented at 624 (e.g., J=J+1) and control returns to the top of the inner nominal trajectory collision evaluation loop 608. If all nominal trajectories have been processed (e.g., J=M), control passes to 626.

[0169] At 626, at least one processor of the processor-based system determines whether there are additional robot pairs that need to be processed. The method 600 can process each arrangement of robot pairs, each arrangement of the trajectories of these robot pairs, and each arrangement of candidate lag times, at least until a stop condition is reached (e.g., an unacceptably high collision probability is found). Therefore, for any given robot, the determined acceptable lag time (e.g., maximum lag time) can be a function of the corresponding selected trajectories of each other robot operating in the shared workspace and the determined acceptable lag times associated with these selected trajectories. In this sense, the term "maximum lag time" does not necessarily mean the absolute maximum lag time of a given robot in isolation, but can mean the maximum lag time of a given robot taking into account the trajectories and associated lag times of other robots operating in the shared workspace. If there are additional robot pairs that need to be processed, control passes to 628, in which a new robot pair is selected, and then control passes to the top of the external robot pair collision evaluation loop. If there are no remaining robot pairs that need to be processed, control passes directly to 630.

[0170] At 630, at least one processor of the processor-based system returns corresponding acceptable lag times for one or more nominal trajectories. For example, the acceptable lag times can be stored in a non-transitory processor-readable medium, such as in one or more data structures associated with the nominal trajectories. Control then passes to 632.

[0171] In 632, method 600 may terminate, for example, until called again. Although method 600 is described as a sequential flow, in many implementations, various actions or operations will be performed simultaneously or in parallel, and / or may include additional actions and / or omit some actions.

[0172] To determine the corresponding acceptable lag time for the nominal trajectory, for example, the processor-based system can iterate each of the plurality of candidate lag times in order from a relatively smaller candidate lag time to a relatively larger candidate lag time, at least until a stop condition is reached, iterating the nominal trajectory over each of the plurality of times, and checking for a collision between one of the two or more robots and at least one other of the two or more robots based on a current one of the candidate lag times. In response to determining that a collision will or may occur between one of the two or more robots and at least one other of the two or more robots, the processor-based system can set the corresponding acceptable lag time for the nominal trajectory of the robot determined to be colliding and at least one of the other of the two or more robots to the closest previous candidate lag time (e.g., a lag time at which a probability of collision is below a collision detection threshold indicating an acceptable risk of collision). To check for collision, for example, the processor-based system can determine whether a swept volume of one of the robots in the pair of robots intersects a swept volume of another robot in the pair of robots. For example, the processor-based system can generate a collision indication in response to determining that a swept volume of one robot in the pair of robots intersects a swept volume of another robot in the pair of robots and identifying the robots that were determined to have collided.

[0173] Listed below are exemplary methods for determining and applying acceptable lag times to provide a safety margin.

[0174] First, a motion plan including a set of nominal trajectories (Trj(t)) is generated, for example, using the optimization described in International Patent Application PCT / US2021 / 013610 (published as WO2021150439A1). Thus, for each mobile robot r in the motion plan, an acceptable lag time relative to the motion plan (e.g., a maximum safe lag value ( )). It is important to note the following: (i) The acceptable lag time value for each robot r ( ) may be different (for example, ); (ii) acceptable lag time value ( ) strictly depends on a set of nominal trajectories Trj(t) and the relative position of the robot r; and (iii) static obstacles and static robots (if any) do not affect the acceptable lag time value ( In this example, a set of nominal trajectories Trj(t) and acceptable lag time values ​​are calculated sequentially ( ), and is preferably calculated offline during configuration time prior to runtime; however, this is not intended to limit the methods described herein. A central clock (t) is implemented during execution (e.g., online; during runtime) that is common to all robots operating in a shared workspace. For each robot r, the lag relative to the nominal motion plan ( ), that is, the monitoring of the actual lag time is performed with a sufficiently high frequency relative to the movement speed of the robot to avoid undesired actual collisions.

[0175] In an ideal situation, each robot r is perfectly synchronized with the motion plan and the actual lag of the robot is zero ( ). However, in the general case, one or more robots may fall behind schedule, where the current robot state is determined by the trajectory and the actual lag Given.

[0176] The processor-based system monitors the actual hysteresis values ​​to ensure the safety of the robot system. If all actual hysteresis values ​​are within the safety margin specified by the acceptable hysteresis time (e.g. ), then it is considered safe to continue execution because the no-self-collision condition is ensured. Otherwise, the safety of the system is considered to be compromised because the no-self-collision condition cannot be ensured.

[0177] To keep the robot in a safe state at all times, the system can monitor the actual hysteresis value and simply stop the movement of the entire robot system if a safety margin is exceeded (e.g., a margin or threshold is exceeded). To safely restart the operation, the robot is able to move to a set of nominal trajectories. any (safe) state of the , and the central clock is reset to .

[0178] Less destructive solutions can include taking remedial or corrective action on robots that are still within the safety margin but relatively close to the limit. For example, the movement of the robot that is about to reach the limit can be accelerated (increased in speed) to reduce the value of its actual lag time. This action will directly increase the distance to the limit and improve the safety level. Alternatively, other robots can be slowed down. This action may use time dilation of a central clock to maintain consistency. Hybrid actions may also be very effective.

[0179] An exemplary processor-executable algorithm for determining the safety margin is set forth in the pseudo-code below.

[0180] For better understanding, the algorithm can be divided into 4 parts:

[0181] • main_loop: It spans candidate lag values ​​from 0 to L, and iterates over the nominal trajectory Trj(t).

[0182] ● CheckLagCollision(): Evaluates the impact of a candidate lag value applied at a specific time t. Given a set of candidate lag values ​​for each robot in the robot group, it determines if any robot is in collision at a given time t. In the region without lag, the robot occupies a volume determined by its current pose. Currently, in the case of lag, the robot can sweep the volume based on the trajectory it takes from time t-lag to its pose at time t.

[0183] ● ComputeSweptVolume_r(t1, t2): Computes the volume swept by the robot r as it moves along a segment of the trajectory from time t1 to time t2 ( ).

[0184] ● VolumesExcept(): Geometrically searches for the intersection between two volumes.

[0185] main_loop() and CheckLagCollision() are described in detail below. ComputeSweptVolume_r() and VolumesIntersect() are not described in detail because they can be implemented using many algorithms.

[0186] The input includes: : a set of collision-free trajectories for N robots; ΔT: sampling time; T: maximum duration of robot activity; ΔL: sample lag time value searched; and L: maximum lag value searched (L<=T).

[0187] Outputs include: Maximum hysteresis value for each robot to ensure self-collision-free motion between robots .

[0188] Internal variables include: lag_candidate: the lag value evaluated in the current iteration; : The lag value evaluated by each robot in the current iteration; : The lag value evaluated by each robot in the previous iteration; : Boolean value for each robot to stop searching.

[0189]

[0190] Figure 7A method 700 of operating a processor-based system according to at least one illustrated embodiment is shown for generating or selecting motion plans for one or more robots to be operated in a shared workspace based at least in part on acceptable lag time, and optionally at least in part on other criteria. The lag time and optionally other criteria can be expressed as a cost or cost function. For example, the method 700 can be executed during configuration or "pre-run" time, i.e., the time when part or all of motion planning occurs, for example, generating a nominal trajectory and / or motion plan for each robot. The configuration or "pre-run" time can precede the run time, which is the time when one or more robots execute the corresponding motion plan. This advantageously allows some of the most computationally intensive work to be performed before the run time, because responsiveness is not particularly concerned at this time. In at least some embodiments, the method 700 or a portion thereof is executed during the run time (i.e., the time when one or more robots are performing tasks) after the configuration time or pre-run time.

[0191] The method 700 of determining acceptable latency begins at 702, for example, in response to startup or power-up of a system or component thereof, receipt of information or data, or invocation or invocation of a calling routine or program.

[0192] Optionally, at 704, at least one processor of the processor-based system generates or accesses (e.g., receives, retrieves) a nominal trajectory for each robot. As previously described, a nominal trajectory is a "specified" trajectory, wherein each trajectory includes an ordered sequence of poses or configurations of the robot, and corresponding timings for each pose or configuration. As described herein, actual motion or actual trajectories of a robot or part thereof may deviate from corresponding nominal trajectories, for example, due to unexpected delays in transitioning between poses (e.g., due to a need to linger or stay on a target object for a longer time than expected).

[0193] As previously described, any of a variety of techniques and / or algorithms can be employed to generate nominal trajectories, such as a sample-based motion planner (SBMP), such as a probabilistic roadmap (PRM) or rapidly exploring random trees (RRT, RTT*s), stable sparse RRT* (SST*), and / or fast matching trees (FMT). Advantageously, the teachings herein include computationally efficiently using a determined lag time to select a set of nominal trajectories for motion planning to enhance robustness of operation.

[0194] At 706, at least one processor of the processor-based system initializes a robot counter 1, e.g., sets the robot counter equal to an integer value of 1. At 708, at least one processor of the processor-based system executes an external robot processing iteration loop, executing iterations for each of two or more robots to be operated in the shared workspace, or until a stop condition is reached (e.g., determining a maximum lag time to ensure collision-free motion).

[0195] At 710, at least one processor of the processor-based system initializes a nominal trajectory counter J, e.g., sets the nominal trajectory counter equal to an integer value of 1. At 712, at least one processor of the processor-based system executes an inner nominal trajectory processing iteration loop, performing iterations for each nominal trajectory of a given robot. The inner nominal trajectory processing iteration loop 712 is nested within the outer robot processing iteration loop 708.

[0196] At 714, at least one processor of the processor-based system determines a corresponding acceptable lag time for the current nominal trajectory J of the current robot I. The corresponding acceptable lag time reflects the maximum acceptable delay or hysteresis in the current nominal trajectory J that still ensures self-collision-free movement of the current robot I relative to the respective movements according to the corresponding nominal trajectory, at least when the current robot I (the current robot of the outer robot processing iteration loop) executes the current nominal trajectory J (the current nominal trajectory of the inner current nominal trajectory iteration loop), wherein the corresponding acceptable lag time is introduced into the current nominal trajectory J of the current robot I. Preferably, this can be evaluated relative to the nominal trajectories of other robots that introduce various candidate lag times (e.g., zero and non-zero lag times) or even introduce corresponding acceptable lag times where known. As long as all other robots are running on or within their corresponding nominal trajectories (i.e., lag time=0), or all other robots are running on or within their corresponding nominal trajectories to which they introduce corresponding lag times, the acceptable lag time of the given robot I can ensure that self-collision will not occur. Therefore, in at least one embodiment, as long as the corresponding current actual lag time of a given robot is less than the corresponding acceptable lag time of the robot, and the other robots are running according to their nominal trajectories, the robots will not collide with each other (ensuring that no self-collision occurs between the robots). Therefore, in at least one preferred embodiment, as long as the corresponding current actual lag time of all robots is less than the corresponding acceptable lag time of each robot, the robots will not collide with each other (ensuring that no self-collision occurs between the robots).

[0197] In 716, at least one processor of the processor-based system determines whether each of the nominal trajectories of a given robot has been considered. For example, it determines whether the nominal trajectory counter J is equal to the total number of nominal trajectories of the given robot (e.g., J = M?). If not all of the nominal trajectories of the given robot have been considered, control passes to 718, where the nominal trajectory counter is incremented (e.g., J = J + 1), and then control returns to the top of the internal nominal trajectory processing iteration loop 712 to consider the next nominal trajectory of the given robot I. If all of the nominal trajectories of the given robot I have been considered (e.g., J = M), control passes directly to 720.

[0198] In 720, at least one processor of the processor-based system selects the maximum value from a set of determined acceptable lag times determined for the given robot I. In 722, at least one processor of the processor-based system provides one or more of the determined acceptable lag times (e.g., the selected maximum value of the determined acceptable lag times) for the given robot for use in determining at least the motion plan of the given robot. For example, this can include providing one or more of the determined acceptable lag times to different processors, or transferring one or more of the determined acceptable lag times to different registers of the processor, or otherwise storing it in a non-transitory processor-readable medium.

[0199] In 724, at least one processor of the processor-based system determines whether each of the robots has been considered. For example, it determines whether the robot counter I is equal to the total number of robots (e.g., I = N?). If not all of the robots have been considered (e.g., I < N), control passes to 726, where at least one processor of the processor-based system iterates the robot counter (e.g., I = I + 1), and then control returns to the top of the external iterative robot processing loop 708 to consider the next robot. If all of the robots have been processed (e.g., I = N), control passes directly to 728.

[0200] In 728, at least one processor of the processor-based system generates or selects a corresponding motion plan for each robot. The motion plan can be or can represent the corresponding nominal trajectory of the robot associated with the maximum acceptable lag time. This can advantageously enhance the robustness of the motion plan generated by the motion planning and thus improve the operation of the robot executing the resulting motion plan.

[0201] When considering multiple trajectories for each robot, the acceptable lag time (e.g., maximum lag time) determined for any given robot can be a function of the corresponding selected trajectories of each of the other robots operating in the shared workspace and the determined acceptable lag times associated with those selected trajectories. Thus, as described herein, in at least some embodiments, the system determines, for each robot and each candidate trajectory for that robot, an acceptable lag time given all other possible combinations of candidate trajectories and candidate lag times for all other robots. Once completed, the system (e.g., Figure 2 The nominal trajectory analyzer 258 of the system can select a set of trajectories for all robots operating in the shared workspace. Figure 2 The nominal trajectory analyzer 258 can partition the set of trajectories, for example, by optimizing an objective function (eg, the maximum sum of all lag times of all robots).

[0202] At least one processor of the processor-based system provides a corresponding motion plan to each robot or each motion controller so that the robot moves according to the corresponding motion plan in 730. For example, this can include providing the motion plan to a corresponding motion controller of each robot in the one or more robots.

[0203] At 732, method 700 may terminate, for example until called again. Although method 700 is described as an ordered flow, in many implementations, various actions or operations will be performed simultaneously or in parallel, and / or may include additional actions and / or omit some actions.

[0204] Thus, for example, the processor-based system can determine, for each of the two or more robots and each of the two or more nominal trajectories of the corresponding robot of the two or more robots, a corresponding acceptable lag time of the nominal trajectory, the lag time reflecting the maximum acceptable delay in the pose timing relative to the nominal trajectory, which delay still ensures that the movement of the robot through the pose sequence specified by the nominal trajectory remains collision-free relative to the movement of the corresponding pose sequence specified by the corresponding nominal trajectory of each of the other robots of the two or more robots, because the nominal trajectory of each of the other robots of the two or more robots is itself delayed by the corresponding acceptable lag time. For example, the processor-based system can select between the two or more nominal trajectories based at least in part on the corresponding acceptable lag times of the two or more nominal trajectories; and provide a motion plan for the corresponding robot based at least in part on the selected one of the nominal trajectories to control the operation of the corresponding robot of the two or more robots. For example, the processor-based system can select the nominal trajectory with the maximum acceptable lag time among the two or more nominal trajectories for the corresponding robot, thereby generating a more robust motion plan than otherwise possible. For example, a processor-based system can select a nominal trajectory based on a corresponding cost function, the corresponding cost function representing an acceptable lag time and an optional risk or probability of collision associated with a corresponding nominal trajectory of two or more nominal trajectories of the corresponding robot. For example, a processor-based system can select a nominal trajectory based on a corresponding cost function, the cost function representing an acceptable lag time, a risk or probability of collision, and an optional severity of collision associated with a corresponding nominal trajectory of two or more nominal trajectories of the corresponding robot. For example, a processor-based system can select a nominal trajectory based on a corresponding cost function, the cost function representing an acceptable lag time, a risk or probability of collision, a severity of collision, and optionally at least one of the following: duration of completion or energy consumption, wherein each variable in the cost function represents an acceptable lag time, a risk or probability of collision, a severity of collision, and at least one of the following: duration of completion or energy consumption, respectively, and these variables are weighted in the corresponding cost function. For example, the processor-based system can select a respective nominal trajectory for each of the two or more robots to maximize an overall acceptable lag time for all of the two or more robots.For example, the processor-based system can select a respective nominal trajectory for each of the two or more robots such that a respective cost function for each robot is optimized across all of the two or more robots, wherein each cost function represents an acceptable lag time and at least a risk or probability of collision, respectively, associated with a respective one of the two or more nominal trajectories of the respective robot. For example, the processor-based system can select a respective nominal trajectory for each of the two or more robots such that a respective cost function for each robot is optimized across all of the two or more robots, wherein each cost function represents an acceptable lag time, at least a risk or probability of collision, and a severity of collision, respectively, associated with a respective one of the two or more nominal trajectories of the respective robot. For example, a processor-based system is capable of selecting a respective nominal trajectory for each of two or more robots such that a respective cost function for each robot is optimized overall for all of the two or more robots, wherein each cost function represents an acceptable lag time, at least a risk or probability of collision, a severity of collision, and at least one of: duration of completion or energy consumption, respectively, associated with a respective nominal trajectory of the two or more nominal trajectories of the respective robot, wherein each variable in the cost function represents an acceptable lag time, a risk or probability of collision, a severity of collision, and at least one of: duration of completion or energy consumption, respectively, and these variables are weighted in the respective cost function.

[0205] For example, the processor-based system can determine a corresponding acceptable lag time for the nominal trajectory, the lag time reflecting a maximum acceptable delay in pose timing relative to the nominal trajectory that still ensures that movement of the robot through the pose sequence specified by the nominal trajectory remains self-collision-free between two or more robots in a shared workspace, which can include performing a collision assessment for each of the two or more robots.

[0206] For example, the processor-based system is capable of performing a collision assessment between i) and ii), wherein i) at least a portion of a corresponding sample trajectory representing a corresponding nominal trajectory of the robot into which at least one corresponding lag time is introduced, and ii) at least a portion of a corresponding sample of each of the corresponding trajectories of each of the other robots of the two or more robots, wherein at least one corresponding lag time is introduced into the corresponding trajectory of each of the other robots of the two or more robots.

[0207] Figure 8A method 800 of operating a processor-based system for controlling operation of a robot operating in a shared workspace based at least in part on an acceptable lag time is shown in accordance with at least one illustrated embodiment. According to at least one illustrated embodiment, the processor-based system includes one or more processors that monitor an actual lag time compared to an acceptable lag time and, if

[0208] If the actual lag time exceeds the threshold, one or more remedial actions may be optionally taken. Method 800 may be performed, for example, in method 300 ( Figure 3 )、Method 400( Figure 4 )、Method 500( Figure 5 )、Method 600( Figure 6 ) and / or method 700 ( Figure 7 ). For example, the method 800 can be executed during runtime, i.e., during the time when one or more robots are executing corresponding motion plans and / or performing tasks. For example, runtime can be after configuration or "pre-run" time, during which some or all motion planning can be performed, e.g., generating nominal trajectories and / or motion plans for each robot.

[0209] The method 800 begins at 802, for example, in response to startup or power-up of a system or component thereof, receipt of information or data, or invocation or initiation of a calling routine or program.

[0210] Optionally, at least one processor of the processor-based system accesses (eg, receives, retrieves) a respective motion plan for each robot at 804. The respective motion plans can be stored and accessed from one or more non-transitory processor-readable media.

[0211] At least one processor of the processor-based system causes one or more robots to execute the corresponding motion plan at 806. For example, the at least one processor can provide instructions to one or more motion controllers of the one or more robots. The one or more motion controllers provide control signals to one or more actuators (e.g., motors, solenoids, valves, pumps) that are coupled to drive various links of the robot, thereby moving the robot or a portion thereof.

[0212] In 808, at least one processor of the processor-based system monitors the amount of lag time (actual lag time) of the corresponding actual trajectory performed by the corresponding one of the robots compared to the corresponding nominal trajectory. As previously described, the actual trajectory is an actual sequence of postures and the timing of these postures performed by the corresponding robot. Although in some cases, the actual trajectory may match the nominal trajectory, in many cases, the actual trajectory will not match the nominal trajectory, and typically the execution timing of at least some of the postures of the actual trajectory of one or more robots lags behind the timing of the postures specified by the corresponding nominal trajectory of the corresponding robot. Monitoring can be performed in any of a variety of ways. For example, the processor-based system can use inputs sensed via one or more sensors, which are positioned to sense the position and / or movement and / or posture or configuration or state of the robot in the shared workspace. The sensor can monitor the entire workspace, for example, the sensor takes the form of a camera, a video camera, a stereo camera, a motion detector, etc., which is positioned and has a field of view covering all or at least part of the shared workspace. Sensors can additionally or alternatively monitor the position and / or movement and / or posture or configuration or state of a particular robot, for example, including any one or more of the following: cameras, video cameras, motion detectors, position encoders or rotary encoders, Hall effect sensors and / or reed switches associated with one or more joints, links and / or actuators of the robot. One or more processors can perform processing (e.g., machine vision processing) to monitor the actual trajectory. Additionally or alternatively, the processor-based system can use information from the robot control system and / or drive system (e.g., motor controller, pneumatic controller, hydraulic controller, etc.) representing control signals (e.g., PWM motor control signals) used to drive the robot or its parts and / or use feedback signals (e.g., back EMF) received from the robot or the robot's drive system. One or more processors can perform processing (e.g., machine vision processing) to determine the corresponding actual lag time of the actual trajectory.

[0213] In 810, at least one processor of the processor-based system determines whether the monitored amount of lag time (actual lag time) exceeds a margin or threshold (e.g., a corresponding determined acceptable lag time; a percentage of a corresponding determined acceptable lag time) for a corresponding trajectory of any of the robots operating in the shared workspace.

[0214] In response to determining that the monitored amount of lag time (i.e., actual lag time) exceeds a corresponding margin or threshold (e.g., a determined acceptable lag time; a corresponding percentage of determined acceptable lag time) for a corresponding trajectory of any robot, optionally in 812, at least one processor of the processor-based system selects and / or takes one or more remedial actions.

[0215] For example, to take at least one remedial action, when at least one processor of the processor-based system executes the processor-executable instructions, the processor can be caused to perform one or more of the following operations: stop the movement of one or more robots, slow down the movement of one or more robots, and / or speed up the movement of one or more robots. For example, at least one processor can stop the movement of one, two, more, or even all robots. Also, for example, at least one processor can slow down the movement of one, two, more, or even all robots. Also, for example, at least one processor can speed up the movement of one, two, more, or even all robots. Also, for example, at least one processor can stop the movement of one or more robots while slowing down the movement of one or more robots. Also, for example, at least one processor can stop the movement of one or more robots while speeding up the movement of one or more robots or keeping the movement of one or more robots constant. Also, for example, at least one processor can slow down the movement of one or more robots while speeding up the movement of one or more robots or keeping the movement of one or more robots constant. Likewise, for example, at least one processor is capable of stopping the movement of one or more robots, slowing down the movement of one or more robots, speeding up the movement of one or more robots, or causing the movement of one or more robots to remain constant. The term "keeping the movement constant" means that there is no change in the nominal trajectory of the robot, although the speed (i.e., velocity and direction) of the movement of the robot may change as specified by the nominal trajectory. Also, for example, in order to take at least one remedial action, when at least one processor of the processor-based system executes the processor-executable instructions, the processor is capable of performing one or more of the following operations: stopping the movement of one or more robots, causing one or more robots to advance along a corresponding trajectory specified by the corresponding nominal trajectory, and then being capable of restarting the movement of the robot. Restarting the movement typically involves restarting the movement from where the movement was stopped, although in some cases it may include returning to the starting point of the corresponding trajectory and restarting from the starting point.

[0216] Optionally, at 814, at least one processor of the processor-based system monitors an amount of lag time (actual lag time) of a respective actual trajectory executed by a respective robot compared to a respective nominal trajectory.

[0217] Optionally, at 816, for each robot operating in the shared workspace, at least one processor of the processor-based system determines whether the monitored amount of lag time (i.e., actual lag time) no longer exceeds a corresponding margin or threshold (e.g., determined acceptable lag time; corresponding percentage of determined acceptable lag time) for the corresponding trajectory of the corresponding robot. The system can enter a waiting loop, continuing to monitor the amount of lag time (i.e., actual lag time) until it no longer exceeds the corresponding margin or threshold (e.g., determined acceptable lag time; corresponding percentage of determined acceptable lag time) for all robots. Once this condition is met, control passes to 818.

[0218] Optionally, at 818, at least one processor of the processor-based system causes one or more robots to continue executing (eg, resume moving) the corresponding motion plan.

[0219] Method 800 terminates at 820, for example, until called again. Although method 800 is described as a sequential flow, in many implementations, various actions or operations will be performed simultaneously or in parallel, and / or can include additional actions and / or omit some actions.

[0220] The above detailed description sets forth various embodiments of the device and / or process through the use of block diagrams, schematics, and examples. To the extent that these block diagrams, schematics, and examples include one or more functions and / or operations, it will be understood by those skilled in the art that each function and / or operation in these block diagrams, flowcharts, or examples can be implemented individually and / or collectively by a variety of hardware, software, firmware, or any combination thereof. In one embodiment, the subject matter can be implemented via Boolean circuits, application specific integrated circuits (ASICs), and / or FPGAs. However, it will be appreciated by those skilled in the art that the embodiments disclosed herein can be implemented in whole or in part in a variety of different embodiments in standard integrated circuits, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more controllers (e.g., microcontrollers), as one or more programs running on one or more processors (e.g., microprocessors), as firmware, or as almost any combination thereof, and that designing circuits and / or writing code for software and / or firmware will be completely within the technical scope of those skilled in the art according to the present disclosure.

[0221] Those skilled in the art will recognize that many of the methods or algorithms described herein may employ additional acts, may omit some acts, and / or may perform acts in an order different from that specified.

[0222] Furthermore, those skilled in the art will appreciate that the mechanisms taught herein can be implemented in hardware, such as in one or more FPGAs or ASICs.

[0223] The various embodiments described above can be combined to provide further embodiments. All commonly assigned U.S. patent application publications, U.S. patent applications, foreign patents, and foreign patent applications referred to in this specification and / or listed in the Application Data Sheet include, but are not limited to, International Patent Application No. PCT / US2017 / 036880 filed on June 9, 2017; International Patent Application No. WO2016 / 122840 filed on January 5, 2016; U.S. Patent Application No. 62 / 616783 filed on January 12, 2018; U.S. Patent Application No. 62 / 616783 filed on February 6, 2018; 2 / 626939; U.S. Patent Application No. 62 / 856548 filed on June 3, 2019; U.S. Patent Application No. 62 / 865,431 filed on June 24, 2019; U.S. Patent Application No. 62 / 964405 filed on January 22, 2020; U.S. Patent Application No. 63 / 327917 filed on April 6, 2022; International Patent Application PCT / US2021 / 013610 (published as WO2021150439A1), all of which are incorporated herein by reference. These and other changes can be made to the embodiments in light of the above detailed description. Generally speaking, in the appended claims, the terms used should not be interpreted as limiting the claims to the specific embodiments disclosed in the specification and claims, but should be understood to include all possible embodiments and the full range of equivalents to which such claims are entitled. Therefore, the claims are not limited by the present disclosure.

Claims

1. A method of operating in a processor-based system to facilitate operation of a plurality of robots, the method being for use in a multi-robot operating environment in which the plurality of robots are to operate, the method comprising: For each of the two or more robots, for each of the one or more nominal trajectories for a respective robot of the two or more robots, the nominal trajectory specifies a respective sequence of poses and pose timings for the respective robot, determining a corresponding acceptable lag time for the nominal trajectory, the corresponding acceptable lag time reflecting a maximum acceptable delay in gesture timing relative to the nominal trajectory that still ensures that movement of the robot through a sequence of gestures specified by the nominal trajectory remains collision-free relative to movement of a corresponding sequence of gestures specified by a corresponding nominal trajectory of each of the other robots of the two or more robots; as well as The acceptable lag time is provided to at least one processor to control operation of each of the two or more robots.

2. The method according to claim 1, wherein: Determining the corresponding acceptable lag time for the nominal trajectory includes: For each of the two or more robots, For each of the one or more nominal trajectories, A swept volume representation is generated that represents a volume swept by at least a portion of the robot moving between a set of pose sequences specified by the nominal trajectory, from at least one time in the nominal trajectory to another time in the nominal trajectory.

3. The method according to claim 2, wherein: Determining a corresponding acceptable lag time for the nominal trajectory further includes: Use the resulting swept volume to perform collision evaluation.

4. The method according to claim 3, wherein: Performing collision assessment using the generated swept volume comprises: for each of the two or more robots, performing collision assessment between i) and ii), wherein i) at least a portion of a respective sample trajectory representing a respective nominal trajectory of the robot into which at least one respective lag time is introduced, ii) at least a portion of a respective sample of each of the respective trajectories of each of the other robots of the two or more robots, wherein the at least one respective lag time is introduced into the respective trajectory of each of the other robots of the two or more robots, and In response to determining that a collision will occur between the robot and at least one of the other robots of the two or more robots, for one or more nominal trajectories of the robot, a corresponding lag time that is less than the corresponding lag time that caused the determination that a collision will occur is identified as the corresponding acceptable lag time, wherein the acceptable lag time reflects a maximum acceptable delay in gesture timing relative to the nominal trajectory, and the maximum acceptable delay still ensures that the movement of the robot through the gesture sequence specified by the nominal trajectory remains collision-free relative to the movement of the other robots of the two or more robots.

5. The method according to claim 4, wherein: Determining a corresponding acceptable lag time for the nominal trajectory further includes: In response to determining that a collision will occur between the robot and at least one of the other robots of the two or more robots, for one or more nominal trajectories of at least one of the other robots of the two or more robots that have been determined to collide, identifying a corresponding lag time that is less than the corresponding lag time that caused the determination that a collision will occur as the corresponding acceptable lag time, wherein the acceptable lag time reflects a maximum acceptable delay in gesture timing relative to the nominal trajectory, and the maximum acceptable delay still ensures that the movement of the robot through the gesture sequence specified by the corresponding nominal trajectory remains collision-free relative to the movement of the other robots of the two or more robots.

6. The method according to claim 1, wherein: Determining a corresponding acceptable lag time for the nominal trajectory further includes: iterating each of the plurality of candidate lags in order from relatively smaller candidate lags to relatively larger candidate lags, at least until a stopping condition is reached, iterating each of a plurality of times covered by the nominal trajectory, checking for collision between one of the two or more robots and at least one other of the two or more robots based on a current one of the candidate lag times; and In response to determining that a collision will occur between one of the two or more robots and at least one other of the two or more robots, setting corresponding acceptable lag times of nominal trajectories of the robot determined to be colliding and at least one other of the two or more robots to the closest previous candidate lag times.

7. The method according to claim 6, wherein: Checking the collision between the robot and other robots based on a current one of the candidate lag times comprises: For each of the two or more robots, A collision assessment is performed on at least a portion of a corresponding nominal trajectory of the robot having a current one of the candidate lag times or another previously determined lag time relative to at least a portion of each trajectory of a corresponding trajectory of each of the other robots of the two or more robots having a current one of the candidate lag times.

8. The method according to claim 7, wherein: Performing a collision assessment on at least a portion of a corresponding nominal trajectory of the robot having a current one of the candidate lag times or another previously determined lag time relative to at least a portion of each trajectory of a corresponding trajectory of each of the other robots of the two or more robots having a current one of the candidate lag times includes: performing a collision assessment to determine whether at least a portion of the robot collides with at least a portion of any of the other robots during movement of the robot and the other robots of the two or more robots along at least a portion of the corresponding nominal trajectory delayed by the candidate lag time.

9. The method according to claim 6, wherein: Checking the collision between the robot and the other robots based on a current one of the candidate lag times comprises: For each of the two or more robots, generating a swept volume representation representing a volume swept by at least a portion of the robot moving from at least one time in the respective nominal trajectories to another time in the respective nominal trajectories between a set of pose sequences specified by the respective nominal trajectories; and For each pair of robots among the two or more robots, determining, for a pose timing specified for a nominal trajectory without introducing any lag time and for a pose timing specified for a nominal trajectory with each of a plurality of candidate lag times introduced into the corresponding nominal trajectory, whether a swept volume of one of the pair of robots intersects a swept volume of the other of the pair of robots, at least until an intersection is detected, and In response to determining that a swept volume of one of the pair of robots intersects a swept volume of the other of the pair of robots and identifying the robots determined to have collided, a collision indication is generated.

10. The method according to claim 1, further comprising: For each of the two or more robots, For each actual trajectory of the one or more actual trajectories executed by the corresponding robot, monitoring an amount of lag between a nominal trajectory and a corresponding actual trajectory executed by the corresponding robot; determining whether the monitored amount of lag exceeds a corresponding determined acceptable lag time of a corresponding actual trajectory of the corresponding robot; as well as In response to the monitored amount of lag exceeding a corresponding determined acceptable lag time for a corresponding trajectory of the corresponding robot, at least one remedial action is taken.

11. The method according to claim 10, wherein: Taking at least one remedial action includes at least one or more of the following: stopping the movement of one or more of the robots, slowing down the movement of one or more of the robots, and speeding up the movement of one or more of the robots.

12. The method according to claim 10, wherein: Taking at least one remedial action includes stopping the movement of the robots, advancing one or more of the robots along respective trajectories specified by respective nominal trajectories, and then restarting the movement of the robots.

13. The method according to claim 10, wherein: The determining of the respective acceptable lag times of the nominal trajectories occurs during a configuration time prior to movement of at least two robots as specified by one or more nominal trajectories of the respective robots.

14. The method according to claim 13, wherein: The amount of lag between the monitored nominal trajectories and the respective actual trajectories executed by the respective robots occurs during a runtime during which the at least two robots are executing movements specified by one or more nominal trajectories of the respective robots, the runtime being after the configuration time.

15. The method according to claim 10, wherein: The monitoring of a lag between a nominal trajectory and a corresponding actual trajectory executed by the corresponding robot includes using a central clock common to a monitor of each of the one or more robots.

16. The method according to any one of claims 1 to 15 and any combination thereof, further comprising: A respective motion plan is received for each of the robots, each motion plan specifying a respective one of one or more nominal trajectories for the respective robot, the nominal trajectory representing a respective collision-free path.

17. A processor-based system for facilitating operation of a plurality of robots, the system being for use in a multi-robot operating environment in which the plurality of robots are to operate, the processor-based system comprising: at least one processor; as well as At least one non-transitory processor-readable medium storing processor-executable instructions which, when executed by the at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 16.

18. A processor-based system for configuring a plurality of robots for use in a multi-robot operating environment in which the plurality of robots are to operate, the processor-based system comprising: at least one processor; as well as at least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: For each of the two or more robots, for each of the one or more nominal trajectories for a respective robot of the two or more robots, the nominal trajectory specifies a respective sequence of poses and pose timings for the respective robot, determining a corresponding acceptable lag time for the nominal trajectory, the corresponding acceptable lag time reflecting a maximum acceptable delay in gesture timing relative to the nominal trajectory that still ensures that movement of the robot through the sequence of gestures specified by the nominal trajectory remains collision-free relative to movement of the corresponding sequence of gestures specified by the corresponding nominal trajectory of each of the other robots of the two or more robots; and The acceptable lag time is provided to at least one processor to control operation of each of the two or more robots.

19. The processor-based system of claim 18, wherein: Determining the corresponding acceptable lag time for the nominal trajectory includes: For each of the two or more robots, For each of the one or more nominal trajectories, A swept volume representation is generated that represents a volume swept by at least a portion of the robot moving between a set of pose sequences specified by the nominal trajectory, from at least one time in the nominal trajectory to another time in the nominal trajectory.

20. The processor-based system of claim 19, wherein: Determining a corresponding acceptable lag time for the nominal trajectory further includes: Use the resulting swept volume to perform collision evaluation.

21. The processor-based system of claim 20, wherein: To perform collision assessment using the generated swept volume, the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to perform, for each of the two or more robots, collision assessment between i) and ii), wherein i) at least a portion of a respective sample trajectory representing a respective nominal trajectory of the robot into which at least one respective lag time is introduced, ii) at least a portion of a respective sample of each of the respective trajectories of each of the other robots of the two or more robots, wherein the at least one respective lag time is introduced into the respective trajectory of each of the other robots of the two or more robots, and In response to determining that a collision will occur between the robot and at least one of the other robots of the two or more robots, for one or more nominal trajectories of the robot, a corresponding lag time that is less than the corresponding lag time that caused the determination that a collision will occur is identified as the corresponding acceptable lag time, wherein the acceptable lag time reflects a maximum acceptable delay in gesture timing relative to the nominal trajectory, and the maximum acceptable delay still ensures that the movement of the robot through the gesture sequence specified by the nominal trajectory remains collision-free relative to the movement of the other robots of the two or more robots.

22. The processor-based system of claim 21, wherein: To determine a corresponding acceptable lag time for the nominal trajectory, the processor-executable instructions, when executed by the at least one processor, further cause the at least one processor to: In response to determining that a collision will occur between the robot and at least one of the other robots of the two or more robots, for one or more nominal trajectories of at least one of the other robots of the two or more robots that have been determined to collide, identifying a corresponding lag time that is less than the corresponding lag time that caused the determination that a collision will occur as the corresponding acceptable lag time, wherein the acceptable lag time reflects a maximum acceptable delay in gesture timing relative to the nominal trajectory, and the maximum acceptable delay still ensures that the movement of the robot through the gesture sequence specified by the corresponding nominal trajectory remains collision-free relative to the movement of the other robots of the two or more robots.

23. The processor-based system of claim 18, wherein: To determine a corresponding acceptable lag time for the nominal trajectory, the processor-executable instructions, when executed by the at least one processor, further cause the at least one processor to: iterating each of the plurality of candidate lags in order from relatively smaller candidate lags to relatively larger candidate lags, at least until a stopping condition is reached, iterating each of a plurality of times covered by the nominal trajectory, checking for collision between one of the two or more robots and at least one other of the two or more robots based on a current one of the candidate lag times; as well as In response to determining that a collision will occur between one of the two or more robots and at least one other of the two or more robots, setting corresponding acceptable lag times of nominal trajectories of the robot determined to be colliding and at least one other of the two or more robots to the closest previous candidate lag times.

24. The processor-based system of claim 23, wherein: To check for collision between the robot and the other robots based on a current one of the candidate lag times, the processor executable instructions, when executed by the at least one processor, cause the processor to: For each of the two or more robots, A collision assessment is performed on at least a portion of a corresponding nominal trajectory of the robot having a current one of the candidate lag times or another previously determined lag time relative to at least a portion of each trajectory of a corresponding trajectory of each of the other robots of the two or more robots having a current one of the candidate lag times.

25. The processor-based system of claim 24, wherein: In order to perform a collision assessment of at least a portion of a corresponding nominal trajectory of the robot having a current one of the candidate lag times or another previously determined lag time relative to at least a portion of each trajectory of a corresponding trajectory of each of the other robots of the two or more robots having a current one of the candidate lag times, the processor-executable instructions, when executed by the at least one processor, cause the processor to perform a collision assessment to determine whether at least a portion of the robot collides with at least a portion of any of the other robots during movement of the robot and the other robots of the two or more robots along at least a portion of the corresponding nominal trajectory delayed by the candidate lag time.

26. The processor-based system of claim 23, wherein: To check for collision between the robot and the other robots based on a current one of the candidate lag times, the processor executable instructions, when executed by the at least one processor, cause the processor to: For each of the two or more robots, generating a swept volume representation representing a volume swept by at least a portion of the robot moving from at least one time in the respective nominal trajectories to another time in the respective nominal trajectories between a set of pose sequences specified by the respective nominal trajectories, wherein a current one of the candidate lag times is introduced into the respective nominal trajectories; as well as For each pair of robots among the two or more robots, determining, for a pose timing specified for a nominal trajectory without introducing any lag time and for a pose timing specified for a nominal trajectory with each of a plurality of candidate lag times introduced into the corresponding nominal trajectory, whether a swept volume of one of the pair of robots intersects a swept volume of the other of the pair of robots, at least until an intersection is detected, and In response to determining that a swept volume of one of the pair of robots intersects a swept volume of the other of the pair of robots and identifying the robots determined to have collided, a collision indication is generated.

27. The processor-based system of claim 18, wherein: The processor-executable instructions, when executed by the at least one processor, further cause the processor to: For each of the two or more robots, For each actual trajectory of the one or more actual trajectories executed by the corresponding robot, monitoring an amount of lag between a nominal trajectory and a corresponding actual trajectory executed by the corresponding robot; determining whether the monitored amount of lag exceeds a corresponding determined acceptable lag time of a corresponding actual trajectory of the corresponding robot; as well as In response to the monitored amount of lag exceeding a corresponding determined acceptable lag time for a corresponding trajectory of the corresponding robot, at least one remedial action is caused to occur.

28. The processor-based system of claim 27, wherein: In order to take at least one remedial action, the processor executable instructions, when executed by the at least one processor, cause the processor to perform one or more of the following: stop the movement of one or more of the robots, slow down the movement of one or more of the robots, and speed up the movement of one or more of the robots.

29. The processor-based system of claim 27, wherein: In order to take at least one remedial action, the processor executable instructions, when executed by the at least one processor, cause the processor to: stop the movement of the robot, move one or more of the robots along a corresponding trajectory specified by a corresponding nominal trajectory, and then restart the movement of the robot.

30. The processor-based system of claim 27, wherein: The determining of the respective acceptable lag times of the nominal trajectories occurs during a configuration time prior to movement of at least two robots as specified by one or more nominal trajectories of the respective robots.

31. The processor-based system of claim 30, wherein: The amount of lag between the monitored nominal trajectories and the respective actual trajectories executed by the respective robots occurs during a runtime during which the at least two robots are executing movements specified by one or more nominal trajectories of the respective robots, the runtime being after the configuration time.

32. The processor-based system of claim 27, wherein: The monitoring of a lag between a nominal trajectory and a corresponding actual trajectory executed by the corresponding robot includes using a central clock common to a monitor of each of the one or more robots.

33. The processor-based system of any one of claims 18 to 32 and any combination thereof, wherein the processor-executable instructions, when executed by the at least one processor, further cause the processor to: A respective motion plan is received for each of the robots, each motion plan specifying a respective one of one or more nominal trajectories for the respective robot, the nominal trajectory representing a respective collision-free path.

34. A method of operating in a processor-based system to facilitate operation of a plurality of robots, the method for use in a multi-robot operating environment in which the plurality of robots are to operate, the method comprising: For each of the two or more robots, for each of two or more nominal trajectories for a respective robot of the two or more robots, the nominal trajectory specifies a respective sequence of poses and pose timings for the respective robot, determining a respective acceptable lag time for the nominal trajectory, the respective acceptable lag time reflecting a maximum acceptable delay in gesture timing relative to the nominal trajectory that still ensures that movement of the robot through the sequence of gestures specified by the nominal trajectory remains collision-free relative to movement of a respective sequence of gestures specified by a respective nominal trajectory of each of the other robots of the two or more robots that is delayed by the respective acceptable lag time of the nominal trajectory of at least one of the other robots of the two or more robots; selecting between the two or more nominal trajectories based at least in part on respective acceptable lag times of the two or more nominal trajectories; as well as A motion plan is provided for the respective robot based at least in part on the selected one of the nominal trajectories to control operation of the respective robot of the two or more robots.

35. The method of claim 34, wherein: Based at least in part on respective acceptable lag times for the two or more nominal trajectories for each robot, selecting between the two or more nominal trajectories includes selecting a nominal trajectory having a maximum acceptable lag time among the two or more nominal trajectories for the respective robot.

36. The method of claim 34, wherein: Based at least in part on the respective acceptable lag times for the two or more nominal trajectories for each robot, selecting between the two or more nominal trajectories includes selecting a nominal trajectory based on a respective cost function representing an acceptable lag time and at least a risk or probability of collision each associated with a respective one of the two or more nominal trajectories for the respective robot.

37. The method of claim 34, wherein: Based at least in part on the respective acceptable lag times for the two or more nominal trajectories for each robot, selecting between the two or more nominal trajectories includes selecting a nominal trajectory based on a respective cost function representing an acceptable lag time, a risk or probability of collision, and a severity of collision, each associated with a respective one of the two or more nominal trajectories for the respective robot.

38. The method of claim 34, wherein: Based at least in part on respective acceptable lag times for two or more nominal trajectories of each robot, selecting between the two or more nominal trajectories includes selecting a nominal trajectory based on a respective cost function representing an acceptable lag time, a risk or probability of collision, a severity of collision, and at least one of duration of completion or energy consumption, respectively, associated with a respective one of the two or more nominal trajectories of the respective robot, wherein each variable in the cost function represents an acceptable lag time, a risk or probability of collision, a severity of collision, and at least one of duration of completion or energy consumption, respectively, and each variable is weighted in the respective cost function.

39. The method of claim 34, wherein: Based at least in part on respective acceptable lag times of the two or more nominal trajectories, selecting between the two or more nominal trajectories includes selecting a respective nominal trajectory for each of the two or more robots such that an overall acceptable lag time for all of the two or more robots is maximized.

40. The method of claim 34, wherein: Based at least in part on the respective acceptable lag times for the two or more nominal trajectories for each robot, selecting between the two or more nominal trajectories includes selecting the respective nominal trajectory for each of the two or more robots such that a respective cost function for each robot is optimized overall across all of the two or more robots, wherein each cost function represents an acceptable lag time and at least a risk or probability of collision respectively associated with a respective one of the two or more nominal trajectories for the respective robot.

41. The method of claim 34, wherein: Based at least in part on the respective acceptable lag times for the two or more nominal trajectories for each robot, selecting between the two or more nominal trajectories includes selecting the respective nominal trajectory for each of the two or more robots such that a respective cost function for each robot is optimized overall across all of the two or more robots, wherein each cost function represents an acceptable lag time, at least a risk or probability of a collision, and a severity of a collision, respectively, associated with a respective one of the two or more nominal trajectories for the respective robot.

42. The method of claim 34, wherein: Based at least in part on the respective acceptable lag times of the two or more nominal trajectories for each robot, selecting between the two or more nominal trajectories includes selecting a respective nominal trajectory for each of the two or more robots such that a respective cost function for each robot is optimized overall for all of the two or more robots, wherein each cost function represents an acceptable lag time, at least a risk or probability of collision, a severity of collision, and at least one of the duration of completion or energy consumption, respectively, associated with a respective one of the two or more nominal trajectories for the respective robot, wherein each variable in the cost function represents an acceptable lag time, a risk or probability of collision, a severity of collision, and at least one of the duration of completion or energy consumption, respectively, and each variable is weighted in the respective cost function.

43. The method of claim 34, wherein: Determining a corresponding acceptable lag time for a nominal trajectory that reflects a maximum acceptable delay in gesture timing relative to the nominal trajectory, the maximum acceptable delay still ensuring that movement of the robot through the gesture sequence specified by the nominal trajectory will remain collision-free, including: performing a collision assessment for each of the two or more robots.

44. The method of claim 43, wherein: Performing a collision assessment on each of the two or more robots includes: A collision assessment is performed between i) and ii), wherein i) at least a portion of a respective sample trajectory representing a respective nominal trajectory of the robot into which at least one respective lag time is introduced, and ii) at least a portion of a respective sample of each of the respective trajectories of each of the other robots of the two or more robots, wherein at least one respective lag time is introduced into the respective trajectory of each of the other robots of the two or more robots.

45. The method of claim 44, wherein: Determining a corresponding acceptable lag time for the nominal trajectory further includes: In response to determining that a collision will occur between the robot and at least one of the other robots of the two or more robots, for one or more nominal trajectories of the robot, a corresponding lag time that is less than the corresponding lag time that caused the determination that a collision will occur is identified as the corresponding acceptable lag time, wherein the acceptable lag time reflects a maximum acceptable delay in gesture timing relative to the nominal trajectory, and the maximum acceptable delay still ensures that the movement of the robot through the gesture sequence specified by the nominal trajectory remains collision-free relative to the movement of the other robots of the two or more robots.

46. ​​The method of claim 45, wherein: Determining a corresponding acceptable lag time for the nominal trajectory further includes: In response to determining that a collision will occur between the robot and at least one of the other robots of the two or more robots, for one or more nominal trajectories of at least one of the other robots of the two or more robots that have been determined to collide, identifying a corresponding lag time that is less than the corresponding lag time that caused the determination that a collision will occur as the corresponding acceptable lag time, wherein the acceptable lag time reflects a maximum acceptable delay in gesture timing relative to the nominal trajectory, and the maximum acceptable delay still ensures that the movement of the robot through the gesture sequence specified by the corresponding nominal trajectory remains collision-free relative to the movement of the other robots of the two or more robots.

47. A processor-based system for facilitating operation of a plurality of robots, the system for use in a multi-robot operating environment in which the plurality of robots are to operate, the processor-based system comprising: at least one processor; as well as At least one non-transitory processor-readable medium storing processor-executable instructions which, when executed by the at least one processor, cause the at least one processor to perform the method of any one of claims 34 to 46.

48. A processor-based system for configuring a plurality of robots for use in a multi-robot operating environment in which the plurality of robots are to operate, the processor-based system comprising: at least one processor; as well as at least one non-transitory processor-readable medium storing at least one of data and processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: For each of the two or more robots, for each of two or more nominal trajectories for a respective robot of the two or more robots, the nominal trajectory specifies a respective sequence of poses and pose timings for the respective robot, determining a respective acceptable lag time for the nominal trajectory, the respective acceptable lag time reflecting a maximum acceptable delay in gesture timing relative to the nominal trajectory that still ensures that movement of the robot through the sequence of gestures specified by the nominal trajectory remains collision-free relative to movement of a respective sequence of gestures specified by a respective nominal trajectory of each of the other robots of the two or more robots that is delayed by the respective acceptable lag time of the nominal trajectory of at least one of the other robots of the two or more robots; selecting between the two or more nominal trajectories based at least in part on respective acceptable lag times of the two or more nominal trajectories; as well as A motion plan is provided for the respective robot based at least in part on the selected one of the nominal trajectories to control operation of the respective robot of the two or more robots.

49. The processor-based system of claim 48, wherein: To select between the two or more nominal trajectories for each robot based at least in part on respective acceptable lag times for the two or more nominal trajectories, the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to select the nominal trajectory having the greatest acceptable lag time among the two or more nominal trajectories for the respective robot.

50. The processor-based system of claim 48, wherein: To select between two or more nominal trajectories for each robot based at least in part on respective acceptable lag times for the two or more nominal trajectories, the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to select a nominal trajectory based on respective cost functions representing the acceptable lag times and at least the risk or probability of collision respectively associated with respective ones of the two or more nominal trajectories of the respective robot.

51. The processor-based system of claim 48, wherein, to select between two or more nominal trajectories for each robot based at least in part on respective acceptable lag times for the two or more nominal trajectories, the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to select a nominal trajectory based on respective cost functions representing respective acceptable lag times, risks or probabilities of collisions, and severity of collisions associated with respective ones of the two or more nominal trajectories of the respective robot.

52. The processor-based system of claim 48, wherein: In order to select between two or more nominal trajectories for each robot based at least in part on respective acceptable lag times for the two or more nominal trajectories, the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to select a nominal trajectory based on a respective cost function, the cost function representing an acceptable lag time, a risk or probability of collision, a severity of collision, and at least one of the duration of completion or energy consumption, respectively, associated with a respective one of the two or more nominal trajectories of the respective robot, wherein each variable in the cost function represents an acceptable lag time, a risk or probability of collision, a severity of collision, and at least one of the duration of completion or energy consumption, respectively, and each variable is weighted in the respective cost function.

53. The processor-based system of claim 48, wherein: To select between two or more nominal trajectories based at least in part on respective acceptable lag times for the two or more nominal trajectories, the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to select a respective nominal trajectory for each of the two or more robots such that an overall acceptable lag time for all of the two or more robots is maximized.

54. The processor-based system of claim 48, wherein: To select between two or more nominal trajectories for each robot based at least in part on respective acceptable lag times for the two or more nominal trajectories, the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to select a respective nominal trajectory for each of the two or more robots such that a respective cost function for each robot is optimized overall across all of the two or more robots, wherein each cost function represents an acceptable lag time and at least a risk or probability of collision respectively associated with a respective one of the two or more nominal trajectories for the respective robot.

55. The processor-based system of claim 48, wherein: To select between two or more nominal trajectories for each robot based at least in part on respective acceptable lag times for the two or more nominal trajectories, the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to select a respective nominal trajectory for each of the two or more robots such that a respective cost function for each robot is optimized overall across all of the two or more robots, wherein each cost function represents an acceptable lag time, at least a risk or probability of a collision, and a severity of a collision, respectively, associated with a respective one of the two or more nominal trajectories for the respective robot.

56. The processor-based system of claim 48, wherein: In order to select between two or more nominal trajectories for each robot based at least in part on respective acceptable lag times for the two or more nominal trajectories, the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to select a respective nominal trajectory for each of the two or more robots such that a respective cost function for each robot is optimized overall for all of the two or more robots, wherein each cost function represents an acceptable lag time, at least a risk or probability of collision and a severity of collision, and at least one of: duration of completion or energy consumption, respectively, associated with a respective one of the two or more nominal trajectories of the respective robot, wherein each variable in the cost function represents an acceptable lag time, a risk or probability of collision, a severity of collision, and at least one of: duration of completion or energy consumption, each variable being weighted in the respective cost function.

57. The processor-based system of claim 48, wherein: To determine a corresponding acceptable lag time for a nominal trajectory that reflects a maximum acceptable delay in pose timing relative to the nominal trajectory that still ensures that movement of the robot through the sequence of poses specified by the nominal trajectory will remain collision-free, the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to perform a collision assessment for each of the two or more robots.

58. The processor-based system of claim 57, wherein: To perform a collision assessment for each of the two or more robots, the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to: A collision assessment is performed between i) and ii), wherein i) at least a portion of a respective sample trajectory representing a respective nominal trajectory of the robot into which at least one respective lag time is introduced, and ii) at least a portion of a respective sample of each of the respective trajectories of each of the other robots of the two or more robots, wherein at least one respective lag time is introduced into the respective trajectory of each of the other robots of the two or more robots.

59. The processor-based system of claim 58, wherein: To determine a corresponding acceptable lag time for the nominal trajectory, the processor-executable instructions, when executed by the at least one processor, further cause the at least one processor to: In response to determining that a collision will occur between the robot and at least one of the other robots of the two or more robots, for one or more nominal trajectories of the robot, a corresponding lag time that is less than the corresponding lag time that caused the determination that a collision will occur is identified as the corresponding acceptable lag time, wherein the acceptable lag time reflects a maximum acceptable delay in gesture timing relative to the nominal trajectory, and the maximum acceptable delay still ensures that the movement of the robot through the gesture sequence specified by the nominal trajectory remains collision-free relative to the movement of the other robots of the two or more robots.

60. The processor-based system of claim 59, wherein: To determine a corresponding acceptable lag time for the nominal trajectory, the processor-executable instructions, when executed by the at least one processor, further cause the at least one processor to: In response to determining that a collision will occur between the robot and at least one of the other robots of the two or more robots, for one or more nominal trajectories of at least one of the other robots of the two or more robots that have been determined to collide, identifying a corresponding lag time that is less than the corresponding lag time that caused the determination that a collision will occur as the corresponding acceptable lag time, wherein the acceptable lag time reflects a maximum acceptable delay in gesture timing relative to the nominal trajectory, and the maximum acceptable delay still ensures that the movement of the robot through the gesture sequence specified by the corresponding nominal trajectory remains collision-free relative to the movement of the other robots of the two or more robots.

Images