Motion planning and control of robots in shared workspace employing look-ahead planning

Abstract

The structures and algorithms described herein employ look-ahead motion planning in which at least two targets are motion-planned and the ability to transition from a previous motion plan to a next (e.g., subsequent) motion plan is evaluated prior to the robot executing the motion plan thus generated. Thus, the system can determine whether the robot will be trapped (e.g., blocked by another robot) at the end of the first motion plan, thereby preventing, limiting, or delaying execution of the second motion plan. Detecting this situation may result in taking one or more remedial actions, such as generating a new, modified, or alternative first exercise plan. Other remedial actions may move another robot, perform a motion plan for another robot to mitigate the obstructed condition and / or determine a new order of targets.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This patent application claims priority to U.S. Patent Application No. 63 / 327,917, filed on April 6, 2022, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0002] The present disclosure relates generally to robotic motion planning and robotic control, and more particularly to systems and methods for performing collision detection by a processor circuit to generate motion plans to efficiently drive a robot, etc., in a shared workspace. Background Art

[0003] Description of Related Technology Motion planning is a fundamental problem in robot control and robotics. A motion plan specifies a path that a robot can follow to transition from a starting or current state to a goal state, typically to complete a task without colliding with any obstacles in the operating environment, or with a reduced likelihood of colliding with any obstacles in the operating environment. Challenges in motion planning include the ability to perform motion planning at very high speeds, even when environmental characteristics change. For example, one or more characteristics, such as the location and / or orientation of one or more obstacles in the environment, may change over time during robot operation. Challenges also include generating motion plans that allow the robot to complete a task efficiently, such as with respect to the time required to complete the task and / or the energy required to complete the task. Challenges also include performing motion planning using relatively low-cost equipment, with relatively low energy consumption, and with limited amounts of storage (e.g., memory circuits, such as memory circuits on processor chip circuits).

[0004] A problem in robotics is operating two or more robots in a shared workspace (a workspace is usually referred to as a cell or environment), e.g., during the execution of a task in the shared workspace, the robots or their accessories may interfere with each other.

[0005] One method of operating multiple robots in a common workspace can be referred to as a task-level method. The task-level method can employ teach-and-repeat training. Engineers can ensure that robots do not collide by defining shared portions of the workspace and programming the individual robots so that only one robot is in the shared workspace at any given point in time. For example, when the first robot begins to move into the workspace, the first robot sets a flag. A controller (e.g., a microprocessor, a microcontroller, a programmable logic controller (PLC)) reads the flag and prevents other robots from moving into the shared workspace until the first robot cancels the flag when exiting the workspace. This method is intuitive, easy to understand, easy to implement, and easy to troubleshoot. However, this method necessarily has a lower work throughput because using task-level conflict elimination typically results in at least one robot being idle for a considerable period of time, even if it is technically possible for the idle robot to perform useful work in the shared workspace.

[0006] Another approach to operating multiple robots in a common workspace employs offline planning (e.g., during pre-operational configuration) to achieve higher work throughput than the task-level conflict-resolving approaches described above. To do this, the system can attempt to solve a planning problem in the combined joint space of all robots or robot attachments. For example, if there are two 6-degree-of-freedom (DOF) attachments in the workspace, a 12 DOF planning problem must be solved. While this approach allows for higher performance, planning can be extremely time consuming. The 12-DOF problem is too large for traditional motion planning algorithms to solve using existing architectures.

[0007] One strategy to address these issues is to optimize the motion of the first robot / robotic accessory and then manually optimize the motion of the second robot / robotic accessory. This can take the form of iterative simulations to ensure that the robots / robotic accessories do not collide with each other, which can take hours of computational time. Additionally, if modifications to the workspace result in a change in the trajectory of one of the robots / robotic accessories, the entire workflow must be revalidated. Summary of the invention

[0008] Motion planning is performed to determine or generate a motion plan that can be executed by a given robot to perform one or more tasks. The motion plan can specify a set of poses that a given robot needs to go through in order to reach a goal (e.g., to perform a task or as part of performing a task). For example, each pose The motion plan may be specified by joint angles that define the physical configuration of one or more parts of the robot (e.g., an attachment, an end effector, or a link of an end-of-arm tool), and these joint angles may be mapped into real world space. The motion plan may, for example, specify a pose that positions at least a portion of a given robot (e.g., an end effector or an end-of-arm tool) at a target to perform or run an assigned task. When planning motion for a given robot, the position, trajectory, and / or expected trajectory of one or more other robots operating in a shared workspace may be taken into account.

[0009] The structure and algorithm described herein employs look-ahead motion planning, wherein motion plans are made for at least two consecutive goals and the ability to transition from a previous motion plan to a subsequent (e.g., subsequent or next) motion plan is evaluated before the robot executes the resulting motion plan. Thus, the system is able to determine whether the robot will become stuck (e.g., blocked by another robot) or even deadlocked at the end of running the first motion plan, thereby preventing, limiting, or delaying the execution of the second motion plan. In response to detecting In this case, the structures and algorithms described herein can take one or more remedial actions, such as generating a new, revised, or replacement first motion plan. Other remedial actions can include moving another robot and / or performing motion planning on another robot to alleviate the blocking or potential blocking condition and / or determine a new order of a set of goals, the set of goals including the first goal and at least the second goal.

[0010] The structures and algorithms described herein can advantageously perform motion planning to: i) determine a first motion plan to move a given robot from a pose to a first end pose, the first end pose positioning at least a portion of the first robot at a first target; and ii) determine at least a second motion plan to move a given robot from the first end pose to a second end pose, the second motion plan being determined based on, for example, the first motion plan or a modified first motion plan before moving the first robot. This can, for example, allow motion planning to be performed for two or more targets before a given robot runs a motion plan. This can, for example, allow the system to consider how one or more subsequent motion plans for the corresponding targets affect one or more previous motion plans (e.g., the subsequent motion plan is the next motion plan, i.e., the motion plan executed immediately after the execution of the previous motion plan). In response to determining that another robot (e.g., the second robot) will block or may block the movement of a given robot (e.g., the first robot), various remedial actions can be taken. For example, when running a first motion plan will cause the given robot to be in a pose that makes it impossible or difficult to run or even delays running a subsequent motion plan (e.g., trapped, deadlocked, blocked, or may be blocked by another robot), one or more remedial actions can be taken.

[0011] The remedial action may, for example, include generating a new or revised or replacement motion plan to transform the first robot to the first destination at an end pose that is different from the end pose of the previously generated first motion plan. The new or revised or replacement motion plan may, for example, result in a pose at the first destination that better configures or orients the given robot to achieve a next or subsequent goal (e.g., a second goal). The new or revised or replacement motion plan may, for example, result in the pose at the first destination improving the ability or likelihood of the given robot to run the next motion plan to achieve the next or subsequent goal. The new or revised or replacement motion plan may, for example, result in the pose at the first destination being transformed from the movement specified by the new or revised or replacement first motion plan to the movement specified by the next motion plan.

[0012] Remedial action may include, for example, moving another robot that is blocking or potentially blocking the given robot so as to not block or no longer block the trajectory of the given robot running the next or subsequent motion plan.

[0013] Remedial action may include, for example, generating a motion plan or a modified motion plan for the other robot, which motion plan or modified motion plan specifies a trajectory for the other robot so that when the given robot moves along the trajectory specified by its motion plan, the other robot will not block or is unlikely to block the given robot, or otherwise reduce the probability of a collision between the given robot and the other robot.

[0014] The remedial action may include, for example, determining or generating a new order for a set of objectives, the set of objectives including the first objective and at least the second objective.

[0015] The various techniques described herein can advantageously facilitate the operation of two or more robots operating in a shared workspace or workroom, preventing or at least reducing the risk of collision between the robots or their robotic accessories, while operating to efficiently move one or more robots to one or more targets, respectively, to perform respective tasks in the shared workspace.

[0016] The structures and algorithms described herein enable high degree of freedom robots to avoid collisions and continue to work efficiently in a constantly changing shared environment. Efficient planning methods can be accelerated with or without hardware acceleration to produce collision-free motion plans in milliseconds. Ultrafast "real-time" motion planning allows robot paths to be determined at run time during task execution without the need for training or time-consuming path optimization. This can advantageously allow multiple robots in a shared workspace to be coordinated in an efficient manner.

[0017] In at least some embodiments, the structures and algorithms described herein can enable efficient, collision-free robotic motion in a workspace shared by multiple robots. All parts of the robot (e.g., base, robot appendages, end-of-arm tooling, end effectors) can perform collision-free motion, even at high speeds.

[0018] The structures and algorithms described herein can advantageously reduce the programming workload of a multi-robot workspace by performing autonomous planning (e.g., during the runtime of one or more robots). In at least some embodiments, an operator is not required to program any safety zones, time synchronization, or joint space trajectories. The input can be limited to a description of the task to be performed or a target pose or a kinematic model of the target and the robot. The input can also include representations of fixed objects, or selectable objects (e.g., people) with unpredictable trajectories.

[0019] The structures and algorithms described herein can advantageously perform motion planning dynamically, including considering the impact of the motion plan for the next goal on the motion plan for the previous goal, and taking remedial actions when necessary, to efficiently move the robot through various postures to achieve continuous goals to perform one or more tasks while keeping the probability of collision zero or very low.

[0020] In at least some embodiments, the structures and algorithms described herein allow robots to share information, for example, via non-proprietary communication channels (e.g., Ethernet connections), which can advantageously facilitate the integration of robots in a shared workspace, even the integration of robots from different manufacturers.

[0021] In at least some embodiments, the structures and algorithms described herein can operate without cameras or other perception sensors. In at least some embodiments, coordination between robots relies on kinematic models of the robots, the ability of the robots to communicate their respective motion plans, and a geometric model of the shared workspace. In other embodiments, vision or other perception can optionally be employed, for example to avoid humans or other dynamic obstacles (e.g., other robots) that may enter or occupy parts of the shared workspace.

[0022] A variety of algorithms are used to solve the motion planning problem. Each of these algorithms generally requires the ability to determine whether a given pose of the robot or the movement from one pose to another pose will result in a collision with the robot itself or an obstacle in the environment. The collision assessment or check can be performed "in software" using a processor that executes processor executable instructions from a stored set of processor executable instructions to execute the algorithm. The collision assessment or check can be performed "in hardware" using a set of dedicated hardware circuits (e.g., collision checking circuits implemented in a field programmable gate array (FPGA), an application specific integrated circuit (ASIC)). Such a circuit can, for example, represent a volume swept by the robot / robot attachment or part thereof during a corresponding movement or transition between two states (i.e., a swept volume). The circuit can, for example, generate a Boolean assessment indicating whether the movement will collide with any obstacles, wherein at least some of the obstacles represent volumes swept when other robots operating in a shared workspace perform movements or transitions. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In the drawings, the same reference numerals indicate similar elements or actions. The sizes and relative positions of the elements in the drawings are not necessarily drawn to scale. For example, the shapes and angles of the various elements are not drawn to scale, and some of the elements may be arbitrarily enlarged and positioned to improve the readability of the drawings. In addition, the particular shape of the drawn element is not intended to convey any information about the actual shape of the particular element, and the particular shape may be selected simply for ease of identification in the drawings.

[0024] Figure 1 is a schematic diagram of a robotic system according to at least one illustrated embodiment, the robotic system including multiple robots operating in a shared workspace to perform a task, and including a motion planner that dynamically generates a motion plan for the robot that takes into account the planned motions of the other robots, and the robotic system optionally includes a perception subsystem.

[0025] Figure 2 is a functional block diagram of controlling an environment of a first robot via a robotic control system, the system including a motion planner, optionally providing motion plans to other motion planners of other robots, and the robotic system also including a planning map source separate and distinct from the motion planner, according to at least one illustrated embodiment.

[0026] Figure 3A An example motion planning graph for a robot operating in a shared workspace according to at least one illustrated embodiment has a current node representing a current pose and a path between a first target and a corresponding first pose that positions the robot or a portion of the robot at the first target.

[0027] Figure 3Bis an example motion planning graph for a robot operating in a shared workspace having a path between a first target node and a second target and a corresponding second pose positioning the robot or a portion of the robot at the second target according to at least one illustrated embodiment.

[0028] Figure 3C An example motion planning graph for a robot operating in a shared workspace according to at least one illustrated embodiment having a new or revised or alternative path between a current node representing a current pose and a first target and a corresponding alternative first pose that positions the robot or a part thereof at the first target.

[0029] Figure 3D is an example motion planning graph for a robot operating in a shared workspace according to at least one illustrated embodiment, having another new or revised or alternative path between a current node representing a current pose and a first target and a corresponding alternative first pose that positions the robot or a part thereof at the first target.

[0030] Figure 4 is a flow chart illustrating a method of operation for generating a motion planning graph and a swept volume in a processor-based system, according to at least one illustrative embodiment.

[0031] Figure 5 is a flow chart illustrating a method of controlling the operation of one or more robots in a processor-based system, such as during robot operation, according to at least one illustrative embodiment.

[0032] Fig. 6A is a high-level flow chart illustrating a method for a processor-based system to perform motion planning operations for one or more robots, such as during operation of the robots, which may be performed as Figure 5 is run as part of a method.

[0033] Figure 6B is a low-level flow chart illustrating a method for a processor-based system to perform motion planning operations for one or more robots, such as during operation of the robots, which may be performed as Figure 5 and Fig. 6A A portion of a method is run.

[0034] Figure 7 is a flow chart illustrating a method of operating a processor-based system for controlling the operation of one or more robots in a multi-robot environment, which may be performed as an example, according to at least one illustrative embodiment. Figure 5 , 6A or as part of a 6B approach. DETAILED DESCRIPTION

[0035] In the following description, certain specific details are set forth to provide a full understanding of the various disclosed embodiments. However, those skilled in the relevant art will recognize that the embodiments may be practiced without one or more of these specific details, or with other methods, components, materials, etc. In other cases, well-known structures related to computer systems, robots, actuator systems, and / or communication networks are not shown or described in detail to avoid unnecessarily obscuring the description of the embodiments. In other cases, well-known computer vision methods and techniques for generating perceptual data and volume representations of one or more objects, etc. are not described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0036] Unless the context requires otherwise, in the following description and claims, the word "comprise" and variations such as "include" and "comprising" should be construed in the open inclusive sense, ie, "including, but not limited to."

[0037] Throughout this specification, reference to "one embodiment" or "an embodiment" or "an example" or "an example" means that a particular feature, structure, or characteristic described in conjunction with the example is included in at least one embodiment or at least one example. Therefore, the phrases "in one embodiment" or "in an embodiment" or "in an example" or "in an example" that appear in multiple places throughout this specification do not necessarily all refer to the same embodiment or the same example. Furthermore, particular features, structures, or characteristics may be combined in one or more embodiments or examples in any suitable manner.

[0038] Unless the content clearly indicates otherwise, an element with an undefined number in this specification and the appended claims means that there is at least one of the element. It should also be noted that the term "or" is generally used to include the meaning of "and / or" unless the content clearly indicates otherwise.

[0039] As used in this specification and the appended claims, the various variations of the term "determine" when used in the context of whether a collision will occur or result means to evaluate or predict whether a given posture or movement between two postures through multiple intermediate postures will or may result in a collision between a part of the robot and an object (e.g., another part of the robot, part of another robot, a persistent obstacle, a transient obstacle (e.g., a person)).

[0040] As used in this specification and the appended claims, the term "robot" refers to a machine that can autonomously or semi-autonomously perform a series of complex actions. A robot may take the form of a base, a robot attachment movably coupled to the base, an end effector or end-of-arm tool carried by the robot attachment, and one or more actuators that drive its movement. Such a robot may or may not be similar to a human. A robot may alternatively or additionally take the form of a vehicle (e.g., an autonomous or semi-autonomous vehicle).

[0041] As used in this specification and the appended claims, the terms "first", "second", "third", etc. are used in a relative sense, not in an absolute sense. Thus, the term "first motion plan" refers to a motion plan to be executed by a given robot before the "second motion plan", even though the "first motion plan" may not be the absolute first motion plan executed by the given robot. Similarly, the terms "first target" and "first final pose" refer to a target or final pose that occurs before the next or subsequent target or the next or subsequent final pose, respectively, such as before the "second target" or "second final pose". In addition, the term "subsequent motion plan" refers to a motion plan to be executed by a given robot after the "previous motion plan". When used with robots, the terms "first robot", "second robot" and "third robot" are used only to distinguish between two or more different robots operating in a shared workspace, and the "first robot" in various examples is generally the robot that is the object of a specific instance of the motion plan. Similarly, the term "given robot" is used to refer to the robot for which the motion plan is being executed (the object of a specific instance of the motion plan), and the term "other robots" is used to refer to robots other than the "given robot" in the shared workspace.

[0042] As used in this specification and the appended claims, the term "feasible path" refers to a complete path from a starting or current node to a target node, wherein the nodes of each pair of consecutive nodes in the path are coupled by respective edges, which represent valid conversions between two configurations or postures represented by respective nodes. As used in this specification and the appended claims, the term "suitable path" refers to one or more feasible paths that meet one or more conditions or constraints, for example, a feasible path with an associated cost or cost function that is lower than a threshold cost or an acceptable cost. As used in this specification and the appended claims, the term "selected path" refers to a feasible path that has been selected from a set of feasible paths or a set of suitable paths based on one or more criteria (e.g., selected based on a cost or cost function, such as selected based on a low cost or a low cost function). As used in this specification and the appended claims, the term "lowest cost path" refers to a feasible path with the lowest cost of a feasible path in a set of feasible paths or a set of suitable paths. It should be noted that the cost or cost function can represent an assessment of the risk of collision or probability of collision, the severity of the collision, the energy expenditure or consumption associated with the feasible path, and / or the time or delay.

[0043] As used in this specification and the appended claims, the term "blocked" or "likely to be blocked" means that a given robot is blocked or at least delayed or will likely be blocked or will likely be at least delayed by an obstacle (e.g., another robot) when executing a motion plan (e.g., a subsequent motion plan or a second motion plan). In this case, the given robot may be trapped, i.e., blocked or at least delayed when executing the subsequent motion plan, until a new, revised or replaced motion plan is generated for the given robot and / or until the obstacle (e.g., another robot) is removed so as to no longer block or no longer have the potential to block the given robot from moving along the trajectory defined by the motion plan (e.g., a subsequent motion plan or a second motion plan). In some cases, the given robot may even become deadlocked and unable to move along any path or trajectory or to transition from a particular posture, at least until the obstacle (e.g., another robot) is removed so as to no longer block or no longer have the potential to block the given robot. The delay can be any delay (e.g., any amount of delay other than the ability to execute the subsequent or second motion plan without delay once commanded to execute the subsequent or second motion plan), or the delay can be a delay longer than a specified delay duration or a threshold delay duration (e.g., a specified non-zero amount of delay in the ability to execute the subsequent or second motion plan once commanded to run the subsequent or second motion plan).

[0044] The titles and abstracts of the disclosure provided herein are for convenience only and are not used to interpret the scope or meaning of the embodiments.

[0045] According to at least one illustrated embodiment, Figure 1 A robotic system 100 is shown, which includes a plurality of robots 102a, 102b, 102c (collectively referred to as robots 102) operating in a shared workspace 104 to perform tasks that employ look-ahead motion planning to better position the robot when a subsequent trajectory or path of the robot is blocked or has the potential to be blocked (e.g., by another robot) and / or when a suitable and unblocked subsequent trajectory or path cannot be found. This can improve the operation of the robot in various ways, such as reducing the overall time to complete a task, allowing tasks that cannot be completed in other ways to be completed, reducing overall energy usage, reducing the occurrence of collisions, and / or reducing the consumption of computing resources.

[0046] The robot 102 may take any of a variety of forms. Typically, the robot will take the form of or have one or more robot attachments. The robot 102 may include one or more links having one or more joints, and an actuator (e.g., an electric motor, a stepper motor, a solenoid, a pneumatic actuator, or a hydraulic actuator) coupled and operable to move the links in response to a control or drive signal. For example, a pneumatic actuator may include one or more pistons, cylinders, valves, air reservoirs, and / or pressure sources (e.g., compressors, blowers). For example, a hydraulic actuator may include one or more pistons, cylinders, valves, fluid reservoirs (e.g., low-compressibility hydraulic fluids), and / or pressure sources (e.g., compressors, blowers). The robot system 100 may take other forms of the robot 102 (e.g., an autonomous or semi-autonomous vehicle).

[0047] The shared workspace 104 generally represents a three-dimensional space in which the robots 102a-102c can operate and move, although in certain limited embodiments, the shared workspace 104 can represent a two-dimensional space. The shared workspace 104 is a volume or area in which at least portions of the robots 102 can overlap in space and time or collide if movement is not controlled to avoid collision. It is noted that the workspace 104 is different from the "configuration space" or "C-space" of each of the robots 102a-102c, which will be described below, for example, with reference to the accompanying drawings. Figures 3A-3D Give a description.

[0048] As described herein, a robot 102a or a portion thereof may constitute an obstacle when considered from the perspective of another robot 102b (i.e., when performing motion planning for another robot 102b). The shared workspace 104 may also include other obstacles, such as machinery (e.g., conveyors 106), piles, columns, walls, ceilings, floors, tables, people, and / or animals. The shared workspace 104 may also include one or more work items or workpieces 108 (e.g., one or more packages, packages, fasteners, tools, items, or other objects) that are manipulated as part of a task performed by the robot 102.

[0049] The robotic system 100 can include one or more robotic control systems 109a, 109b, 109c, (three are shown, collectively referred to as the robotic control system 109), which include one or more motion planners, such as respective motion planners 110a, 110b, 110c (three are shown, collectively referred to as the motion planner 110) for each robot 102a, 102b, 102c, respectively. In at least some embodiments, a single motion planner 110 may be employed to generate motion plans for two, more, or all of the robots 102. The motion planners 110 are communicatively coupled to control each of the robots 102, respectively. The motion planners 110 are also communicatively coupled to receive various types of inputs, including, for example, robot kinematic models 112a, 112b, 112c (collectively referred to as the kinematic models 112) and / or task messages 114a, 114b, 114c. The motion planners 110 are optionally communicatively coupled (not shown) to the robot kinematic model 112. Figure 1 ) to receive motion plans or other representations of motions 116a, 116b, 116c (collectively 116) of other robots 102 operating in the shared workspace 104. The robot kinematic model 112 defines the geometry of a given robot 102, such as joints, degrees of freedom, dimensions (e.g., link lengths), and / or C-space aspects of each robot 102. (e.g., Figure 2As shown, the conversion of the robot kinematic model 112 to the motion planning graph can occur at configuration time before the robot is operational or before the robot performs a task, for example, by a processor-based system that is distinct and separate from the robot system 100 using any of a variety of techniques.) Task messages 114a-114c specify the task to be performed, for example in terms of a target (e.g., a target position, a target pose or final pose, a target configuration or a target state and / or a final configuration or final state). The execution of the task may also involve transitions or movements through a plurality of intermediate poses, intermediate configurations, or intermediate states of the corresponding robot 102. The pose, configuration, or state may be defined, for example, in the configuration space (C-space) of the robot, for example, in terms of joint positions and joint angles / rotations (e.g., joint poses, joint coordinates) of the corresponding robot 102. The target position may be specified in real-world coordinates of a workspace (e.g., 2-space or 2D space or 3-space or 3D space), or alternatively in C-space. The term "(one or more) targets" refers to a position or area or range at which a robot or part thereof is positioned to perform or implement a task, and within which the robot can perform or implement a task. In this regard, it should be noted that the position at which the robot can perform a task is not necessarily limited to a single point in a space or area, but may encompass a relatively large space or area, and the size along any given axis in a given coordinate system may be non-uniform. Therefore, the term "(one or more) targets", "(one or more) target positions" are not necessarily limited to a single point in a space or area, but may encompass a space, area or range, such as a space, area or range in which the robot can perform a defined task.

[0050] The motion planner 110 is optionally communicatively coupled to receive static object data 118a, 118b, 118c as input. The static object data 118a, 118b, 118c represents static objects 118 in the workspace 104 (e.g., size, shape, location, space occupied), which may be known a priori, for example. The static objects 118 may, for example, include one or more fixed structures in the workspace, such as piles, columns, walls, ceilings, floors, conveyors 106. Since the robots 102 operate in a shared workspace, the static objects 118 are generally the same for each robot. Therefore, in at least some embodiments, the static object data 118a, 118b, 118c provided to the motion planner 110 will be the same. In other embodiments, the static object data 118a, 118b, 118c provided to the motion planner 110 may be different for each robot, for example based on the position or orientation of the robot 102 in the environment, or based on the robot 102's view of the environment. Additionally, as noted above, in some implementations, a single motion planner 110 may generate motion plans for two or more robots 102 .

[0051] The motion planner 110 is optionally communicatively coupled to receive as input perception data 120, for example provided by a perception subsystem 124. The perception data 120 represents a priori unknown static and / or dynamic objects in the workspace 104. The perception data 120 may be raw data perceived by one or more sensors (e.g., cameras 122a, 122b) and / or converted by the perception subsystem 124 into digital representations of obstacles.

[0052] The optional perception subsystem 124 may include one or more processors that may execute one or more machine-readable instructions that cause the perception subsystem 124 to generate corresponding discretizations of representations of the environment in which the robot 102 will operate to perform tasks for various different scenarios.

[0053] Optional sensors (e.g., cameras 122a, 122b) provide raw sensory information (e.g., point clouds) to the perception subsystem 124. The optional perception subsystem 124 can process the raw sensory information and can provide the resulting sensory data as a point cloud, an occupancy grid, a box (e.g., a bounding box) or other geometric object, or a voxel stream (i.e., a "voxel" is equivalent to a 3D or volume pixel) that represents obstacles present in the environment. The representation of the obstacles can optionally be stored in an on-chip memory. The sensory data 120 can represent which voxels or sub-volumes (e.g., boxes) are occupied in the environment at the current time (e.g., during runtime). In some embodiments, when representing a robot or another obstacle in the environment, the surface of each robot or obstacle (e.g., including other robots) can be represented as voxels or polygonal meshes (usually triangles). In some cases, it is advantageous to represent these objects as boxes (rectangular prisms, bounding boxes, hierarchical data structures, such as octrees or ball trees) or other geometric objects. Since the shape of an object is not random, there may be a lot of structure in the way voxels are organized; many voxels inside an object are directly adjacent to each other in 3D space. Therefore, representing an object as a box may require fewer bits (i.e., only Cartesian coordinates x, y, z may be needed to represent the two opposite corners of the box). Also, the complexity of performing intersection checks on boxes is comparable to the complexity of performing intersection checks on voxels.

[0054] At least some embodiments may combine the output of multiple sensors, and these sensors may provide very fine-grained voxelization. However, in order for the motion planner to perform motion planning efficiently, coarse-grained voxels (i.e., "processor voxels") may be used to represent the environment and the swept volume in 3D space of the robot 102 or its parts as it transitions between various states, configurations, or poses. Accordingly, the optional perception subsystem 124 may transform the output of the sensors (e.g., cameras 122a, 122b) accordingly. At runtime, if the robot control system 200 determines that any sensor voxel within a processor voxel is occupied, the robot control system 200 considers the processor voxel to be occupied and generates an occupancy grid accordingly.

[0055] Figure 1Arrows in the figure show various communication paths. The communication paths can, for example, take the form of one or more wired communication paths (e.g., electrical conductors, signal buses, or optical fibers) and / or one or more wireless communication paths (e.g., via RF or microwave radios and antennas, infrared transceivers). In particular, each motion planner 110a-110c can be communicatively coupled to each other directly or indirectly to provide the motion plan of the corresponding robot in the robots 102a-102c to other motion planners in the motion planners 110a-110c. For example, the motion planners 110a-110c can be communicatively coupled to each other via a network infrastructure, such as a non-proprietary network infrastructure (e.g., an Ethernet network infrastructure) 126. This can advantageously allow robots from different manufacturers to operate in a shared workspace.

[0056] The term "environment" is used to refer to the robot's current workspace, which is a shared workspace where two or more robots operate in the same workspace. The environment may include obstacles and / or artifacts (i.e., items that the robot will interact with or act on or with). The term "task" is used to refer to the robot's task of transitioning from pose A to a target pose B, preferably without colliding with obstacles in its environment. The target pose is the pose of the robot performing a specified task at a location called the target position or goal, usually on an object. Tasks may involve grasping or releasing an item, moving or dropping an item, rotating an item, or retrieving or placing an item. The transition from pose A to target pose B may optionally include transitions between one or more intermediate poses. The term "scene" is used to refer to a class of environment / task pairs. For example, a scene may be "Perform a pick and place task between x and y obstacles of a given range of sizes and shapes in an environment with a 3-legged table or conveyor". There may be many different task / environment pairs that meet this criterion, depending on the location of the goals and the size and shape of the obstacles.

[0057] The motion planner 110 is operable to dynamically generate motion plans 116 for the robots 102 to perform tasks in the environment while taking into account the planned motions of other robots in the robots 102 (e.g., motions represented by respective motion plans 116 or generated swept volumes). The motion planner 110 may optionally take into account representations of a priori static objects 118 represented by static object data 118a, 118b, 118c and / or perception data 120 when generating the motion plans 116. The motion planner 110 may advantageously take into account the known or predicted positions, poses, and / or motion states of other robots 102 at a given time (e.g., whether another robot 102 has completed a given motion or task), and allow for recalculation of motion plans based on the motion or task being completed by one of the other robots, for example, to provide previously excluded paths or trajectories for selection.

[0058] As described herein, the motion planner 110 uses forward motion planning to facilitate efficient operation of two or more robots operating in a shared workspace or workroom, prevent or at least reduce the risk of collision between robots or robot accessories of robots, and efficiently move one or more robots to one or more targets to perform their respective tasks in the shared workspace. The motion planner 110 can, for example, identify (e.g., select, generate) a first motion plan for a first robot, the first motion plan specifies a plurality of poses and a corresponding first end pose for converting the first robot from a pose to a first target, the first end pose positions at least a portion of the first robot at the first target, and the motion planner 110 also identifies (e.g., selects, generates) a second motion plan for the first robot, the second motion plan specifies a plurality of poses and a corresponding second end pose for converting the first robot from the first end pose to a second target, the second end pose positions at least a portion of the first robot at the second target. The second motion plan can be determined before the first robot runs the first motion plan. The motion planner 110 can, for example, consider how the first motion plan will affect the ability of the first robot to execute a subsequent second motion plan (i.e., the second motion plan). For example, the motion planner 110 can, for example, determine that the first motion plan will place the first robot in a first final posture, in which the subsequent motion plan cannot or is difficult to execute. For example, another robot (e.g., a second robot) can block or may block the first robot from moving according to the subsequent motion plan from the first final posture, such as causing the first robot to be trapped, deadlocked, or at least delayed when running the second motion plan. As described elsewhere herein, one or more remedial actions can be taken, such as generating a new or revised or replaced first motion plan to place the first robot or part thereof at the first target with a better final posture than the first final posture in the first motion plan. Such remedial actions can be performed, for example, during the robot's runtime. Various heuristic methods can be used to identify new or revised or replaced first motion plans to effectively position or configure the robot to complete its task. The system can monitor other robots and move the robot to the target in response to a suitable path becoming unobstructed or unblocked or otherwise finding a suitable path.

[0059] Optionally, the motion planner 110 may take into account operating conditions of the robot 102, such as the occurrence or detection of a blocked trajectory or path, or the occurrence or detection of a predicted blocked trajectory or path and / or the inability to find a suitable and unblocked path.

[0060] According to one illustrative embodiment, Figure 2An environment is shown in which a first robot control system 200 includes a first motion planner 204a that generates a first motion plan 206a to control the operation of a first robot 202, and optionally provides the first motion plan 206a and / or the motion representation as an obstacle to other motion planners 204b of other robot control systems 200b through at least one communication channel (represented by approximate arrows, such as a transmitter, receiver, transceiver, radio, router, Ethernet) to control other robots ( Figure 2 not shown).

[0061] Similarly, other motion planners 204b of other robot control systems 200b generate other motion plans 206b to control other robots ( Figure 2 206b) and optionally provide other motion plans 206b to the first motion planner 204a and other motion planners 204b of other robot control systems 200b. The motion planners 204a, 204b may also optionally receive motion completion messages 209 indicating when the motion of each robot 202 is completed. This may allow the motion planners 204a, 204b to generate new or updated motion plans based on the current or updated state of the environment. For example, after the first robot 202 completes some or all of a set of motions formed as part of the first robot completing a task, a portion of the shared workspace may become unblocked or otherwise enable the second robot to perform a task therein. Additionally or alternatively, the motion planner 204a can receive information collected by various sensors or generated by other motion planners 204b (e.g., images, occupancy grids, joint positions, and joint angles / rotations) that indicate when a portion of the shared workspace may become unblocked or otherwise enable a second robot to perform a task therein after the first robot 202 completes some or all of a set of motions formed as part of the first robot completing a task.

[0062] The robotic control system 200 can be communicatively coupled, such as by at least one communication channel (represented by approximate arrows, such as a transmitter, receiver, transceiver, radio, router, Ethernet), to receive the motion planning graph 208 and / or swept volume representation 211 from one or more sources 212 of the motion planning graph 208 and / or swept volume representation 211. According to one illustrated embodiment, the source 212 of the motion planning graph 208 and / or swept volume representation 211 can be separate and distinct from the motion planners 204a, 204b. The source 212 of the motion planning graph 208 and / or swept volume representation 211 can be, for example, one or more processor-based computing systems (e.g., server computers) that can be operated or controlled by the respective manufacturer of the robots 202 or by other entities. The motion planning graphs 208 can each include a set of nodes 214 (in Figure 2 Only two are marked in FIG. 2 ) and a set of edges 216 (in FIG. 2 ) coupling nodes 214 of corresponding node pairs 214. Figure 2 Only two are marked in the figure), where the node 214 represents the state, configuration or posture of the corresponding robot, and the edge 216 represents the legal or valid conversion between the state, configuration or posture. The state, configuration or posture can, for example, represent the set of joint positions, orientations, postures or coordinates of each joint of the corresponding robot 202. Therefore, each node 214 can represent the posture of the robot 202 or its part, which is completely defined by the posture of the joints that constitute the robot 202. The motion planning graph 208 can be determined, set or defined before runtime, such as in pre-run time or configuration time (that is, defined before performing the task). The swept volume representation 211 represents the corresponding volume that the robot 202 or its part will occupy when performing the motion or conversion corresponding to the corresponding edge 216 of the motion planning graph 208. The swept volume representation 211 can be represented in various forms, such as represented as voxels, Euclidean distance fields, and hierarchical structures of geometric bodies. When responsiveness is not particularly concerned, this advantageously allows some of the most computationally intensive work to be performed before runtime.

[0063] Each robot 202 may optionally include a base ( Figure 2The base 202 may be fixed to the environment or movable in the environment (e.g., an autonomous or semi-autonomous vehicle). Each robot 202 may optionally include a set of links, joints, end-of-arm tools or end effectors, and / or actuators 218a, 218b, 218c (three are shown, collectively referred to as 218) operable to move the links about the joints. The set of links, joints, end-of-arm tools or end effectors typically includes one or more robot attachments that are movably coupled to the base of the robot. Each robot 202 may optionally include one or more motion controllers (e.g., motor controllers) 220 (only one is shown) that receive control signals (e.g., in the form of a motion plan 206a) and provide drive signals to drive the actuators 218. Alternatively, the motion controller 220 may be separate from the robot 202 but communicatively coupled to the robot 202.

[0064] Each robot 202 can have its own robot control system 200, or alternatively one robot control system 200 can perform motion planning for two or more robots 202. For illustrative purposes, one robot control system 200 will be described in detail. Those skilled in the art will recognize that the description can be applied to similar or even identical additional examples of other robot control systems 200.

[0065] The robot control system 200 may include one or more processors 222 and one or more associated non-transitory computer or processor readable storage media, such as system memory 224a, disk drive 224b and / or memory or registers (not shown) of the processor 222. The non-transitory computer or processor readable storage media (e.g., system memory 224a, drive 224b) is communicatively coupled to the processor 222 via one or more communication channels (e.g., system bus 234). The system bus 234 may employ any known bus structure or architecture, including a memory bus with a memory controller, a peripheral bus, and / or a local bus. One or more such components may also or alternatively communicate with each other via one or more other communication channels (e.g., a wireless network channel capable of high-speed communication, one or more parallel cables or serial cables, such as Universal Serial Bus ("USB") 3.0, Peripheral Component Interconnect Express (PCIe) or via Thunderbolt®).

[0066] The robotic control system 200 may also be communicatively coupled to one or more remote computer systems, such as a server computer (e.g., source 212 of motion planning graphs), a desktop computer, a laptop computer, an ultraportable computer, a tablet computer, a smartphone, a wearable computer, and / or a sensor ( Figure 2200 ), the remote computer system is directly or indirectly communicatively coupled to the various components of the robot control system 200, for example, via a network interface 227. The remote computing system (e.g., a server computer (e.g., a source of motion planning graphs 212)) can be used to program, configure, control, or otherwise interact with or input data (e.g., motion control graphs 208, swept volume representations 211, task specifications 215) the robot control system 200 and the various components of the robot control system 200. Such connection can be through one or more communication channels using the Internet protocol, such as one or more wide area networks (WANs), such as Ethernet or the Internet. As described above, pre-runtime calculations can be performed by a system separate from the robot control system 200 or the robot 202, while runtime calculations can be performed by a processor 222 of the robot control system 200, which in some embodiments can be carried by the robot 202.

[0067] As described above, the robot control system 200 may include one or more processors 222 (i.e., circuits), non-transitory storage media (e.g., system memory 224a, drive 224b), and a system bus 234 that couples various system components. The processor 222 may be any logical processing unit, such as one or more central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), programmable logic controllers (PLCs), etc. The system memory 224a may include a read-only memory ("ROM") 226, a random access memory ("RAM") 228, a flash memory 230, and an EEPROM (not shown). The basic input / output system ("BIOS") 232, which can constitute part of the ROM 226, contains basic routines that help transfer information between elements within the robot control system 200 (e.g., during startup).

[0068] The drive 224b may be, for example, a hard disk drive for reading from and writing to a disk, a solid-state (e.g., flash) drive for reading from and writing to a solid-state memory, and / or an optical drive for reading from and writing to a removable optical disk. The robot control system 200 may also include any combination of such drives in various different embodiments. The drive 224b may communicate with the processor 222 via the system bus 234. As will be appreciated by those skilled in the relevant art, the drive 224b may include an interface or controller (not shown) coupled between these drives and the system bus 234. The drive 224b and its associated computer-readable medium provide the robot control system 200 with non-volatile storage of computer or processor-readable and / or executable instructions, data structures, program modules, and other data. Those skilled in the relevant art will appreciate that other types of computer-readable media (e.g., WORM drives, RAID drives, tape cassettes, digital video disks ("DVDs"), Bernoulli cassettes, RAM, ROM, smart cards, etc.) capable of storing computer-accessible data may be used.

[0069] Executable instructions and data can be stored in the system memory 224a, such as in an operating system 236, one or more application programs 238, other programs or modules 240, and program data 242. The application programs 238 may include processor-executable instructions that cause the processor 222 to perform one or more of the following: generate a discretized representation of the environment in which the robot 202 will operate, including obstacles and / or target objects or artifacts in the environment, in which the planned motions of other robots can be represented as obstacles; generate a motion plan, which includes calling or otherwise obtaining the results of collision assessment, setting cost values ​​for edges in the motion planning graph, and evaluating available paths in the motion planning graph; identify (e.g., select, generate) feasible paths and / or motion plans, optionally store the determined motion plans, and / or provide instructions to cause one or more robots to move according to the motion plans. Motion planning and motion plan construction (e.g., collision detection or evaluation, updating the cost of edges in the motion planning graph based on collision detection or evaluation, and path searching or estimation including identification of feasible paths, evaluation of feasible paths to find a suitable path, and optionally selecting a selected path from the feasible paths or suitable paths) can be performed as described herein (e.g., with reference to the attached Figure 4 , 5, 6A, 6B and 7) and the execution described in the references incorporated herein by reference. Collision detection or evaluation can be performed using various structures and techniques described elsewhere in this article. The application 238 can also include one or more machine-readable and machine-executable instructions that cause the processor 222 to monitor the robot in the environment to determine when a trajectory or path (e.g., a feasible path) becomes unobstructed or unblocked, and cause the robot to move toward the target in response to the trajectory or path becoming unobstructed or unobstructed. The application 238 can also include one or more machine-readable and machine-executable instructions that cause the processor 222 to perform other operations, such as optionally processing sensory data (captured via sensors). The application 238 can also include one or more machine-executable instructions that cause the processor 222 to perform various other methods described in this article and the references incorporated herein by reference.

[0070] In various embodiments, one or more of the operations described above may be performed by one or more remote processing devices or computers connected through a communications network (eg, network 210 ) via network interface 227 .

[0071] Although in Figure 2 224a, but operating system 236, application programs 238, other programs / modules 240, and program data 242 can be stored on other non-transitory computer or processor-readable media, such as drive 224b.

[0072] The motion planner 204 of the robotic control system 200 may include dedicated motion planner hardware, or may be implemented in whole or in part via the processor 222 and processor-executable instructions stored in the system memory 224a and / or the driver 224b.

[0073] The motion planner 204 may include or implement a motion converter 250, a collision detector 252, a cost setter 253, a path finder 254, a path analyzer 255; a look-ahead evaluator 256, and optionally a multi-path analyzer 257. Each of these may be implemented by one or more processors (e.g., circuits) executing logic (e.g., executable software or firmware instructions, or hard-wired logic). The motion converter 250 converts the motion of the other robot into an obstacle representation. The motion converter 250 receives a motion plan 206b or other representation of motion from the other motion planner 204b.

[0074] The motion converter 250 may include a trajectory predictor 251 to predict the trajectory of a transient object (e.g., other robots, other objects including, for example, humans), such as when the trajectory of the transient object is unknown (e.g., where the object is another robot but the motion plan of the other robot is not received, where the object is not another robot but is, for example, a human). The trajectory predictor 251 may, for example, assume that the object will continue the existing motion without changes in direction, speed, and acceleration. The trajectory predictor 251 may, for example, interpret expected changes in the motion or path of the object, such as when the path of the object will result in a collision, the object may be expected to stop or change direction, or when the target of the object is known, the object may be expected to stop when reaching the target. In at least some instances, the trajectory predictor 251 may employ a learned behavior model of the object to predict the trajectory of the object.

[0075] The motion converter 250 then determines an area or volume corresponding to the motion. For example, the motion converter can convert the motion into a corresponding swept volume, i.e., a volume swept by the corresponding robot or portion thereof when moving or transitioning between poses represented by the motion plan. Advantageously, the motion planner 250 can simply queue obstacles (e.g., swept volumes) and may not need to determine, track, or indicate the timing of corresponding motions or swept volumes. Although described as the motion converter 250 of a given robot 202 converting the motion of other robots 202b into obstacles, in some embodiments, the other robots 202b can provide obstacle representations (e.g., swept volumes) for a particular motion to the given robot 202.

[0076] The collision detector 252 performs collision detection or analysis to determine whether a given robot 202 or part thereof's transition or motion will result in a collision with an obstacle. As described above, the motion of other robots can advantageously be represented as obstacles. Thus, the collision detector 252 can determine whether a robot's motion will result in a collision with another robot moving in a shared workspace.

[0077] In some embodiments, the collision detector 252 implements software-based collision detection or evaluation, such as performing bounding box-bounding box collision evaluation or evaluation based on a hierarchy of geometric (e.g., sphere) representations of the volume swept by the robot 202, 202b or a portion thereof during movement. In some embodiments, the collision detector 252 implements hardware-based collision detection or evaluation, such as using a set of dedicated hardware logic circuits to represent obstacles and streaming the representation of the movement through the dedicated hardware logic circuits. In hardware-based collision detection or evaluation, the collision detector can use one or more configurable circuit arrays, such as one or more FPGAs 258, and can optionally generate a Boolean collision evaluation.

[0078] The cost setter 253 can set or adjust the cost of the edges in the motion planning graph based at least in part on collision detection or evaluation. For example, the cost setter 253 can set a relatively high cost value for an edge representing a transition between states or a motion between postures that causes or may cause a collision. For another example, the cost setter 253 can set a relatively low cost value for an edge representing a transition between states or a motion between postures that will not cause or is unlikely to cause a collision. Setting the cost can include setting a cost value logically associated with the corresponding edge through some data structure (e.g., field, pointer, table).

[0079] The path identifier 254 can determine or identify one or more feasible paths from the starting node to the target node. For example, the path identifier 254 can determine or identify a set of nodes in a motion planning graph that provides a complete path from the starting or current node to the target node, i.e., a set of nodes, for which each pair of consecutive nodes in the complete path has respective valid transitions as represented by the presence of edges coupling the nodes of the node pair. The path identifier 254 can use or execute any type of path finding algorithm. In some embodiments, the path identifier 254 can determine or identify feasible paths independently of cost, creating a set of feasible paths. In other cases, the path identifier 254 can consider the cost when determining or identifying a feasible path.

[0080] The optional path analyzer 255 can determine or identify one or more suitable paths and / or select a single path (i.e., a selected path, such as an optimal or optimized path) using a motion planning graph with cost values. The path analyzer 255 can, for example, identify one or more paths that meet certain specific criteria (e.g., the cost is within an upper threshold limit), or even select a single path (e.g., select the lowest cost path) from a set of feasible paths determined or identified by the path identifier 254. For example, the path analyzer 255 can construct a lowest cost path optimizer that determines the lowest or relatively low cost path between two states, configurations, or postures, which are represented by respective nodes in the motion planning graph. The path analyzer 255 can use or execute any type of path finding algorithm (e.g., lowest cost path finding algorithm), taking into account the cost value associated with each edge that represents the possibility of a collision, which cost value also optionally represents one or more of the following: the severity of the collision, the expenditure or consumption of energy, and / or the time or delay of execution or completion.

[0081] Various algorithms and structures may be used to determine the least-cost path, including algorithms and structures that implement the Bellman-Ford algorithm, but other algorithms and structures may also be used, including but not limited to any method in which the least-cost path is determined to be the path between two nodes in the motion planning graph 208 such that the sum of the costs or weights of the edges that make up the motion planning graph 208 is minimized. This process improves the motion planning techniques of the robots 102, 202 by using a motion planning graph that represents the motion of other robots as obstacles and collision detection to improve the efficiency and response time of finding the "best" path to perform a task without collisions.

[0082] The look-ahead evaluator 256 may evaluate whether the final pose (e.g., the first final pose) resulting from the execution of a previous motion plan (e.g., the first motion plan) is unfavorable for the execution of a subsequent motion plan (e.g., the second motion plan), and take remedial action if the final pose (e.g., the first final pose) is unfavorable. The look-ahead evaluator 256 may, for example, determine whether the execution of a previous motion plan (e.g., the first motion plan) by a given robot will result in the given robot being placed in a pose (e.g., a pose at a first target) that will result in the given robot being stuck or even deadlocked (e.g., blocked or likely to be blocked by (e.g., another robot)) when subsequently executing or attempting to execute a subsequent motion plan (e.g., the second motion plan). The look-ahead evaluator 256 may, for example, determine whether the execution of a previous motion plan (e.g., the first motion plan) by a given robot will result in the given robot being blocked or likely to be blocked (e.g., blocked by another robot) when subsequently executing or attempting to execute a subsequent motion plan (e.g., the second motion plan), and the blocking probability is above a certain defined threshold. The look-ahead evaluator 256 may, for example, determine whether the execution of a previous motion plan (e.g., a first motion plan) by a given robot will result in difficulty in generating a subsequent motion plan or difficulty in generating a subsequent motion plan with an acceptable collision-free probability (e.g., a probability equal to or higher than a defined collision-free probability, or a probability lower than a defined collision probability). The look-ahead evaluator 256 may, for example, determine whether the execution of a previous motion plan (e.g., a first motion plan) by a given robot will result in a delay or an unacceptably long delay in executing the subsequent motion plan because the given robot is blocked or is likely to be blocked during the transition from the first motion plan to the subsequent motion plan. The look-ahead evaluator 256 may, for example, compare the respective delays associated with two or more different motion plans and determine whether any of the motion plans is acceptable (e.g., has an associated delay below a defined delay threshold), and / or decide whether additional motion plans should be generated in an attempt to find a motion plan with an acceptably small delay.

[0083] The forward evaluator 256 can take remedial action, such as in response to determining that there is or may be a blocking situation, or determining that if a given robot and / or other robots move along their respective trajectories, a blocking or possible blocking position will occur. In at least some embodiments, the forward evaluator 256 can determine or select the type of remedial action taken, such as selecting from a group of different types of remedial actions based on one or more criteria. As described herein, one or more various types of remedial actions can be implemented. For example, based on an analysis of a second motion plan that moves a given robot from a first target, a new, revised or replaced first motion plan can be generated to move the given robot to the first target. For example, a new, revised or replaced motion plan can be generated for another robot that is blocking or may block the given robot. For example, a new order of a set of targets can be determined or generated, the set of targets including a first target and at least a second target. This can be determined or generated pseudo-randomly or based on one or more heuristic methods (e.g., always trying to move the first target downstream by one position relative to the order of the set of targets being modified).

[0084] The multipath analyzer 257 can analyze the total cost or the total cost associated with two or more motion plans (e.g., the total cost of the first motion plan and the second motion plan). This can be used to identify the combination of motion plans with the lowest total cost. The multipath analyzer 257 can, for example, consider the total cost or the total cost of two or more options of the first motion plan in conjunction with the second motion plan. This is not limited to the two-layer motion planning depth, but can also take into account motion plans at other levels (e.g., the third motion plan for switching from the second target to the third target). The multipath analyzer 257 can construct a minimum cost path optimizer that determines the minimum or relatively low cost combination of two or more paths between two states, configurations, postures, or targets, which are represented by respective nodes in the motion planning graph (e.g., the starting posture from the first target to the second target). The multipath analyzer 257 may use or execute any type of path finding algorithm (e.g., a minimum cost path finding algorithm) taking into account the cost values ​​associated with each edge, which cost values ​​represent the associated collision likelihood, and which cost values ​​may also optionally represent one or more of the severity of the collision, the energy expenditure or consumption, and / or the time or delay to execute or complete associated with the transition represented by the corresponding edge. In at least some embodiments, the multipath analyzer 257 may form part of the look-ahead evaluator 256.

[0085] The motion planner 204a may optionally include a trimmer 260. The trimmer 260 may receive information indicating that the motion of the other robot is complete, which is referred to herein as a motion complete message 209. Alternatively, a flag may be set to indicate completion. In response, the trimmer 260 may remove the obstacle or portion of the obstacle indicating that the motion is now complete. This may allow a new motion plan to be generated for a given robot that may be more efficient, or allow the given robot to participate in performing a task that was previously blocked by the motion of another robot. This approach advantageously allows the motion converter 250 to ignore the timing of the motion when generating obstacle representations of the motion, while still achieving better throughput than using other techniques. The motion planner 204a can also send a signal, prompt, or trigger to cause the collision detector 252 to perform a new collision detection or evaluation if the obstacle has been modified to generate an updated motion planning graph in which the edge weights or costs associated with the edges have been modified, and cause the cost setter 253, path identifier 254, path analyzer 255, look-ahead evaluator 256, and optional multi-path analyzer 257 to update the cost values ​​and determine a new or revised motion plan accordingly.

[0086] The motion planner 204a may optionally include an environment converter 263 that converts output from an optional sensor 262 (e.g., a digital camera) (e.g., a digitized representation of the environment) into a representation of obstacles. Thus, the motion planner 204a may perform motion planning taking into account transient objects (e.g., people, animals, etc.) in the environment.

[0087] The processor 222 and / or the motion planner 204a may be, or may include, any logic processing unit, such as one or more central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic controllers (PLCs), etc. Non-limiting examples of commercially available computer systems include, but are not limited to, Celeron, Core, Core 2, Itanium, and Xeon series microprocessors provided by Intel Corporation; K8, K10, Bulldozer, and Bobcat series microprocessors provided by Advanced Micro Devices; A5, A6, and A7 series microprocessors provided by Apple Computer; Snapdragon series microprocessors provided by Qualcomm; and SPARC series microprocessors provided by Oracle. Figure 2The construction and operation of the various structures shown may be implemented or adopted in accordance with International Patent Application No. PCT / US2017 / 036880, filed on June 9, 2017, entitled “MOTIONPLANNING FOR AUTONOMOUS VEHICLES AND RECONFIGURABLE MOTION PLANNING PROCESSORS”; International Patent Application Publication No. WO 2016 / 122840, filed on January 5, 2016, entitled “SPECIALIZED ROBOT MOTION PLANNING HARDWARE AND METHODS OF MAKING AND USINGSAME”; and / or U.S. Patent Application No. WO2016 / 122840, filed on January 12, 2018, entitled “APPARATUS, METHOD AND ARTICLE TO FACILITATE MOTION PLANNING OF AN AUTONOMOUS VEHICLE IN AN ENVIRONMENT HAVING DYNAMIC OBJECTS”. 62 / 616,783 or similar structures, techniques and algorithms.

[0088] Although not required, many embodiments will be described in the general context of computer-executable instructions, such as program application modules, objects, or macros stored on a computer or processor readable medium and executed by one or more computers or processors that can perform obstacle representation, collision assessment, and other motion planning operations.

[0089] The motion planning operation may include, but is not limited to, generating or converting one, more, or all of the following: Figure 1 )、Task 114( Figure 1 ), and a representation of the volume (e.g., a swept volume) occupied by the robot in various states or poses and / or during movement between states or poses, into a digital form, such as a point cloud, a Euclidean distance field, a data structure format (e.g., a hierarchical format, a non-hierarchical format), and / or a curve (e.g., a polynomial or spline representation). The motion planning operation may optionally include, but is not limited to, generating or converting one, more, or all of the following: a representation of a static or persistent obstacle or static object 118 ( Figure 1 ) and / or represent static or transient obstacles 118 ( Figure 1 ) is converted into a digital form, such as a point cloud, a Euclidean distance field, a data structure format (e.g., a hierarchical format, a non-hierarchical format), and / or a curve (e.g., a polynomial or spline curve representation).

[0090] Motion planning operations may include, but are not limited to, using various collision assessment techniques or algorithms (e.g., software-based, hardware-based) to determine or detect or predict collisions for various states or poses of the robot, or the movement of the robot between states or poses.

[0091] In some embodiments, motion planning operations may include, but are not limited to: determining one or more motion planning graphs, motion plans, or roadmaps; storing the determined motion planning graphs, motion plans, or roadmaps; and / or providing motion planning graphs, motion plans, or roadmaps to control the operation of the robot.

[0092] In one embodiment, collision detection or evaluation is performed in response to a function call or similar process and a Boolean value is returned thereto. The collision detector 252 may be implemented via one or more field programmable gate arrays (FPGAs) and / or one or more application specific integrated circuits (ASICs) to perform collision detection while achieving low latency, relatively low power consumption, and increasing the amount of information that can be processed.

[0093] In various embodiments, such operations may be performed entirely in hardware circuitry or as software stored in a storage memory (e.g., system memory 224a) and performed by one or more hardware processors 222), such as one or more microprocessors, digital signal processors (DSPs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), graphics processing unit (GPU) processors, programmable logic controllers (PLCs), electrically programmable read-only memories (EEPROMs), or as a combination of hardware circuitry and software stored in a storage memory.

[0094] Various aspects of perception, planning graph construction, collision detection, and path searching may also be described in whole or in part in the following: International Patent Application No. PCT / US2017 / 036880, filed on June 9, 2017, entitled “MOTION PLANNING FOR AUTONOMOUS VEHICLES AND RECONFIGURABLE MOTION PLANNING PROCESSORS,” International Patent Application Publication No. WO 2016 / 122840, filed on January 5, 2016, entitled “SPECIALIZED ROBOT MOTION PLANNING HARDWARE AND METHODS OF MAKING AND USING SAME,” and U.S. Patent Application No. WO 2016 / 122840, filed on January 12, 2018, entitled “APPARATUS, METHOD AND ARTICLE TO FACILITATE MOTION PLANNING OF AN AUTONOMOUS VEHICLE IN AN ENVIRONMENT HAVING DYNAMIC OBJECTS.” 62 / 616,783; U.S. Patent Application No. 62 / 856,548, filed on June 3, 2019, entitled “APPARATUS, METHODS AND ARTICLES TO FACILITATE MOTION PLANNING IN ENVIRONMENTS HAVING DYNAMIC OBSTACLES” and International Patent Application No. PCT / US2020 / 039193, filed on June 23, 2020, entitled “MOTION PLANNING FOR MULTIPLE ROBOTS IN SHARED WORKSPACE” and published as WO 2020 / 263861. Those skilled in the relevant art will understand that the illustrated embodiments, as well as other embodiments, can be practiced with other system structures and arrangements and / or other computing system structures and arrangements, including robots, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, personal computers (“PCs”), networked PCs, minicomputers, mainframe computers, or the like. Implementations or embodiments or parts thereof (e.g., at configuration time and runtime) can be practiced in a distributed computing environment, where tasks or modules are performed by remote processing devices that are connected via a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices or media. However, the storage location and manner of certain types of information are very important to help improve motion planning.

[0095] For example, various motion planning solutions bake a roadmap (i.e., motion planning graph) into a processor (e.g., FPGA), and each edge in the roadmap corresponds to a non-reconfigurable Boolean circuit of the processor. The design in which the planning graph is "baked" into the processor brings a problem that the processor circuits for storing multiple or large planning graphs are limited and usually cannot be reconfigured for different robots.

[0096] One solution provides a reconfigurable design that places the map information into storage memory. This approach stores the information in memory rather than embedding it in the circuit. Another approach uses templated reconfigurable circuits instead of memory.

[0097] As described above, some information (e.g., robot geometry model) may be captured, received, input, or provided during configuration time (i.e., before runtime). The received information may be processed during configuration time to produce processed information (e.g., motion planning graph) to speed up operations during runtime or reduce computational complexity during runtime.

[0098] At runtime, collision detection can be performed on the entire environment, including determining, for any pose or movement between poses, whether any part of the robot will collide or be predicted to collide with another part of the robot itself, with other robots or their parts, with persistent or static obstacles in the environment, or with transient obstacles with unknown trajectories in the environment (such as people).

[0099] Figure 3A , 3B , 3C and 3D show the robot 102 ( Figure 1 )、202( Figure 2 ) in which the goal of the robot 102, 202 is to avoid collisions with static and dynamic obstacles while performing a task, which obstacles may include other robots operating in a shared workspace.

[0100] In particular, the motion planning graph 300a ( Figure 3A ) shows a first feasible path 312a between a current node 308a representing a current posture and a first target. In this example, the first target may correspond to, for example, node 308i, which represents a corresponding target posture for positioning the robot or a part thereof at the first target. In this example, the first feasible path 312a is composed of edges between nodes 308a, 308b, 308c, 308d, 308e, 308f, 308g, 308h, and 308i in sequence, and the first feasible path 312a is in Figure 3A300a. For ease of reference, the first feasible path 312a may be named the previous or first motion plan.

[0101] Motion planning diagram 300b ( Figure 3B ) shows a feasible path 314 between a first target and a second target. In this example, the first target may, for example, correspond to node 308j, which represents a corresponding target posture for positioning the robot or a part thereof at the first target. It is worth noting that in at least some cases, there may be one, two, or even more postures for positioning the robot or a part thereof at a given target. Therefore, there may be one, two, or even more nodes for positioning the robot or a part thereof at a given target. Certain postures may be more suitable for avoiding the robot from being trapped or even deadlocked at a given target. In this example, the second target may correspond to node 308m, which represents a corresponding state for positioning the robot or a part thereof at the second target. In this example, the feasible path 314 is composed of edges between nodes 308j, 308k, 308l, and 308m in sequence, and the first feasible path 314 is in Figure 3B 300b with a thicker line width than other edges in the motion planning graph 300b. For ease of reference, the feasible path 314 may be named a subsequent or second motion plan because it is intended to be executed after the previous or first motion plan.

[0102] It is noteworthy that the edge of the node 308i directly coupling the node 308j of the second feasible path 314 at the end of the first feasible path 312a is associated with a relatively high risk of collision. For example, if two feasible paths are selected for the motion plan, there may be a risk that the robot is trapped or even deadlocked. The forward motion planning described herein can take this into account and remedial action can be taken. As mentioned above, in the case of at least some motion plans, there may be more than one posture (for example, two or more target postures) that positions the robot or its part at a given target, thereby allowing the robot to perform a task. These postures (for example, target postures) are represented by each node in the motion planning graph. In this example, the posture associated with node 308i and the posture associated with node 308j both position the robot or its part at the first target. In this case, one of the feasible remedial actions is to perform motion planning again to determine the first motion plan that is new, revised or replaced to adapt to or consider this second motion plan. Figure 3C and 3D Two different new, modified or replaced first motion plans are respectively shown.

[0103] Movement planning diagram 300c ( Figure 3C) shows an alternative of a new, revised or replaced first feasible path 312b between a current node 308a representing a current posture and a node 308j positioning the robot or a part thereof at a first target. In this example, the new, revised or replaced first feasible path 312b is composed of edges between nodes 308a, 308b, 308c, 308d, 308e, 308f, 308g, 308h, 308n and 308j in sequence, and the first feasible path 312b is Figure 3C 300c. For ease of reference, the new, revised or replaced first feasible path 312b may be named a new, revised or replaced previous motion plan because it is intended to be executed before a subsequent or second motion plan. It is noteworthy that the motion plan may have a relatively low associated collision risk (e.g., indicated by the total value associated with its various edges, which in this example totals 1), despite the fact that it does include a relatively large number of gestures (e.g., nine different gestures in this example, although in many practical applications, the number of different gestures may be much larger).

[0104] Movement planning diagram 300d ( Figure 3D ) shows another alternative of a new, revised or replaced first feasible path 312c between the current node 308a representing the current posture and the node 308j positioning the robot or its part at the first target. In this example, the new, revised or replaced first feasible path 312c is composed of the edges between nodes 308a, 308o, 308p, 308n and 308j in sequence, and the first feasible path 312c is Figure 3D 300d. For ease of reference, the new, modified or replaced first feasible path 312d may be named a new, modified or replaced previous motion plan because it is intended to be executed before the subsequent or second motion plan. Figure 3C In comparison, this motion plan may have a relatively high associated collision risk (e.g., as indicated by the aggregate values ​​associated with its various edges, which in this example totals 7), despite being relatively Figure 3D A relatively small number of gestures are indeed included (e.g., five different gestures in this example, although in many practical applications, the number of different gestures may be much larger). In at least some embodiments, fewer gestures can be associated with less delay and / or less energy consumption. Therefore, in some embodiments, a higher risk of collision can be considered a reasonable trade-off for saving delay and / or energy consumption.

[0105] As used herein, the term "feasible path" refers to a complete path from a starting or current node to a target node. Feasible paths in a set of feasible paths may be evaluated based at least in part on an associated cost or cost function for each feasible path, and optionally based on a threshold cost or acceptable cost, to identify one or more suitable or even optimal feasible paths. As described herein, a cost or cost function may represent an assessment of the risk of collision, severity of collision, expenditure or consumption of energy, and / or time or delay associated with a feasible path.

[0106] The motion planning graphs 300a, 300b, 300c, 300d respectively include a plurality of nodes (represented as hollow circles in the figure) connected by edges (represented as straight lines between pairs of nodes in the figure). A subset of nodes is referred to as nodes 308a-308p, and a subset of edges is referred to as edges 310a-310h. Each node implicitly or explicitly represents the time and variables that characterize the state of the robot 102, 202 in the configuration space of the robot 102, 202. The configuration space is generally referred to as C-space, and is the space of the state, configuration, or posture of the robot 102, 202 represented in the motion planning graphs 300a, 300b, 300c, 300d. For example, each node may represent the state, configuration, or posture of the robot 102, 202, which may include, but is not limited to, position, orientation, or posture (i.e., position and orientation). The state, configuration, or posture may be represented, for example, by a set of joint positions and joint angles / rotations (e.g., joint postures, joint coordinates) of the joints of the robot 102, 202.

[0107] The edges in the motion planning graphs 300a, 300b, 300c, 300d represent valid or allowed transitions between these states, configurations, or postures of the robot 102, 202. The edges in the motion planning graphs 300a, 300b, 300c, 300d do not represent actual movements in Cartesian coordinates (2-space or 3-space), but rather represent transitions between states, configurations, or postures in the robot's C-space. Each edge in the motion planning graphs 300a, 300b, 300c, 300d represents a transition of the robot 102, 202 between corresponding pairs of nodes. For example, edge 310a represents a transition of the robot 102, 202 between two nodes. Specifically, edge 310a represents a transition between the state of the robot 102, 202 in a specific configuration associated with node 308b and the state of the robot 102, 202 in a special configuration associated with node 308c. For example, the robot 102, 202 may currently be in a particular configuration associated with the node 308a. Although the nodes are shown at different distances from each other, this is for illustrative purposes only and is not related to any physical distance. There is no limit to the number of nodes or edges in the motion planning graphs 300a, 300b, 300c, 300d, however, the more nodes and edges used in the motion planning graphs 300a, 300b, 300c, 300d, the more accurately and precisely the motion planner can determine a suitable or even optimal path to perform a task based on one or more states, configurations, or postures of the robot 102, 202, because there are more feasible paths to choose from with the lowest cost path.

[0108] Each edge is assigned or associated with a cost value. For example, the cost value may represent a collision assessment for the motion represented by the corresponding edge. The cost value may also represent other information, such as representing the energy consumption associated with the conversion or the duration of completing the relevant movement or conversion or task.

[0109] Typically, it is desirable for the robot 102, 202 to avoid certain obstacles, such as other robots in a shared workspace. In some cases, it may be desirable for the robot 102, 202 to contact or approach certain objects in a shared workspace, such as to grasp or move an object or workpiece. Figure 3A , 3C 3D and 3D show a "first" motion planning graph 300a and a pair of "new" or "revised" or "alternative" "first" motion planning graphs 300c, 300d, respectively, which the motion planner uses to identify feasible paths 312a, 312b, 312c (indicated by bold lines) for the robot 102, 202 to move to the first target. The feasible paths 312a, 312b, 312c are generated or selected to move the robot 102, 202 to the first target through several "intermediate poses" while avoiding collisions with one or more obstacles. Figure 3BA "second" motion planning graph 300b is shown, which the motion planner uses to identify feasible paths 314 (indicated by bold lines) for the robot 102, 202 to move from a first target to a second target. The target is typically a location where the robot 102, 202 performs one or more specific tasks (e.g., picking up and placing an object), and the target pose is the pose of the robot for performing the specific task. As previously described, two or more nodes can be associated with corresponding poses, each of which places the robot or a part thereof at a given target (e.g., two or more poses, each of which places the robot or a part thereof at a given physical location or within a given physical volume or area or space from which a task can be performed, such as a physical location, volume or area or space in real world space). Therefore, a given target can be reached via two or more paths, which have different target nodes and therefore have different end poses or target poses from each other.

[0110] Obstacles may be represented digitally, for example, as bounding boxes, oriented bounding boxes, curves (e.g., splines), Euclidean distance fields, or a hierarchy of geometric entities, whichever digital representation is most appropriate for the obstacle type and the type of collision detection to be performed, which itself may depend on the specific hardware circuitry employed. In some embodiments, the swept volume in the roadmap of the master agent (e.g., robot 102) is pre-computed. Examples of collision assessment are described in International Patent Application No. PCT / US2017 / 036880, filed on June 9, 2017, and entitled “MOTIONPLANNING FOR AUTONOMOUS VEHICLES AND RECONFIGURABLE MOTION PLANNING PROCESSORS”; U.S. Patent Application No. 62 / 722,067, filed on August 23, 2018, and entitled “COLLISION DETECTION USEFUL IN MOTION PLANNING FOR ROBOTICS”; and International Patent Application Publication No. WO 2016 / 122840, filed on January 5, 2016, and entitled “SPECIALIZED ROBOT MOTION PLANNING HARDWARE AND METHODS OF MAKING AND USING SAME”.

[0111] Motion planners 110a, 110b, 110c ( Figure 1 )、204( Figure 2 ) or parts thereof (e.g. collision detectors, Figure 2) determines or evaluates the likelihood or probability that a pose (represented by a node) and / or a motion or transition (represented by an edge) will result in a collision with an obstacle. In some cases, the determination is a Boolean value, while in other cases the determination may be expressed as a probability.

[0112] For nodes in the motion planning graphs 300a, 300b, 300c, and 300d, if direct transitions between nodes may result in collisions with obstacles, the motion planner (e.g., the cost setter 253, Figure 2 ) Edges in the motion planning graph 300a, 300b, 300c, 300d that transition between these nodes (e.g., edges 310a, 310b, 310c, 310d, 310e, 310f, 310g, 310h) are assigned cost values ​​or weights indicating the probability of a collision with an obstacle.

[0113] For example, the motion planner can assign a cost value or weight equal to zero or close to zero for each of the multiple edges in the motion planning graph 300a, 300b, 300c, 300d where the probability of colliding with an obstacle is lower than a defined threshold collision probability. In this example, the motion planner has assigned a zero cost value or zero weight to those edges in the motion planning graph 300a, 300b, 300c, 300d where the probability of a transformation or movement of the robot 102, 202 colliding with an obstacle is zero or very small. For each of the multiple edges in the motion planning graph 300a, 300b, 300c, 300d where the probability of colliding with an obstacle in the environment is higher than a defined threshold collision probability, the motion planner assigns a cost value or weight far greater than zero. In this example, the motion planner has assigned a cost value or weight greater than zero to those edges in the motion planning graph 300a, 300b, 300c, 300d where the probability of colliding with an obstacle is relatively high. The specific threshold value for collision probability may be different. For example, the threshold value can be 40%, 50%, 60% or lower or higher collision probability. In addition, the cost value or weight greater than zero can be included in the weight greater than zero corresponding to the corresponding collision probability. For example, as shown in motion planning diagram 300a, 300b, 300c, 300d, the motion planner has assigned a relatively high cost value or weight 10 to some edges with a higher collision probability, but the motion planner determines that other edges with much lower collision probability have been assigned a relatively low value cost value or weight 0. In other embodiments, the cost value or weight can present a binary choice between collision and no collision, and only two cost values ​​or weights are available when the cost value or weight is assigned to the edge.

[0114] Motion planner (e.g., cost designer 253, Figure 2) In addition to being based on the associated collision probability, the cost value or weight of each edge can also be assigned, set or adjusted based on factors or parameters such as energy consumption, total duration to complete the associated move or transition or task.

[0115] After the motion planner sets a cost value or weight representing the probability of the robot 102, 202 colliding with an obstacle based at least in part on the collision assessment, the motion planner (e.g., path identifier 254, path analyzer 255, Figure 2 ) identifies or generates feasible paths (e.g., feasible paths with valid transitions between a current or start node and a target node), and optionally performs optimization to identify (e.g., determine, select) a suitable path (a feasible path that satisfies one or more conditions or thresholds), or even selects a selected feasible path 312a, 312b, 312c, 312d (e.g., an optimal feasible path or an optimal feasible path or an optimal suitable feasible path) in the resulting motion planning graph 300a, 300b, 300c, 300d, which provides the robot 102, 202 with a motion plan specified by the identified suitable or even optimal feasible path 312a, 312b, 312c, 312d without colliding with obstacles (including other robots operating in a shared workspace) or with a relatively low probability of colliding with obstacles.

[0116] In one embodiment, once all edge costs in the motion planning graphs 300a, 300b, 300c, 300d are assigned or set, the motion planner (e.g., path analyzer 255, Figure 2 ) can perform calculations to determine the lowest cost path to or toward the destination represented by the destination node 308i. For example, the path analyzer 255 ( Figure 2 ) can execute a minimum cost path algorithm from the current state of the robot 102, 202 represented by the current node 308a in the motion planning graph 300a, 300b, 300c, 300d to a possible state, configuration or posture. The motion planner then selects the lowest cost (close to zero) path in the motion planning graph 300a, 300b, 300c, 300d. As described above, the cost can reflect not only the probability of collision, but also other factors or parameters (such as the energy consumption and total duration to complete the relevant movement or transformation or task). Figure 3A and 3B As shown in the example, the current state, configuration or posture of the robot 102, 202 in the motion planning graph 300a, 300b is located at node 308a, and the first target state, configuration or posture of the robot 102, 202 that places at least a portion of a given robot (e.g., an end effector, an end-of-arm tool) at a target to perform a given task is located at node 308i. Figure 3AIn FIG. 3 , a feasible path between the current node 308 a and the target node 308 i is depicted as a feasible path 312 a in the motion planning graph 300 a (a thick line path consisting of line segments extending successively from the node 308 a to the node 308 i ). Figure 3B , a feasible path (eg, revised) between the first target node 308j and the second target node 308m is depicted as feasible path 314 in the motion planning graph 300b (a thick line path consisting of a line segment extending from node 308j to node 308m with intermediate nodes 308k and 308l).

[0117] Although feasible paths 312a, 314, 312b, 312c are shown as having many sharp turns in the motion planning graphs 300a, 300b, 300c, 300d, respectively, such turns do not represent corresponding physical turns in the routes, but rather logical transitions between states, configurations, or postures of the robots 102, 202. For example, each edge in the identified feasible paths 312a, 314, 312b, 312c may represent a state change relative to the physical configuration of the robots 102, 202 in the environment, but does not necessarily represent a state change with respect to the physical configuration of the robots 102, 202 in the environment. Figure 3A , 3B , 3C and 3D respectively illustrate the direction changes of the robots 102 and 202 corresponding to the angles of the feasible paths 312a, 314, 312b, and 312c shown.

[0118] Figure 4 A method 400 of operation for generating a motion planning graph and a swept volume in a processor-based system according to at least one illustrative embodiment is shown. The method 400 may be performed prior to runtime, such as during configuration time. The method 400 may be performed by a processor-based system (e.g., a server computer) that is separate and distinct from, and possibly remote from, one or more robots and / or one or more robot control systems.

[0119] Constructing a motion planning graph takes a lot of time and resources, but as described herein, it only needs to be constructed once, for example, at configuration time before run time. Once generated, the motion planning graph can be stored, for example, in a planning graph side information memory or other non-transitory computer or processor readable medium, and the processor can relatively quickly and efficiently swap motion planning graphs back and forth, or select which motion planning graph to use, for example based on the current characteristics of the robot (e.g., when the robot is grasping an object of a certain size).

[0120] As described above, some preprocessing activities can be performed before runtime, so in some embodiments, these operations can be performed by a remote processing device connected to the robot control system via a network interface through a communication network. For example, a pre-run programming or configuration phase allows the robot to be prepared for the problem of interest. In such embodiments, a large amount of preprocessing is used to avoid runtime calculations. For example, preprocessing can include generating pre-calculated data (i.e., data calculated before the robot runs) about the volume in 3D space swept by the robot when it transitions from one state to another in the motion planning graph, and this transition is represented by the edge in the motion planning graph. The pre-calculated data can be stored in a memory (e.g., a planning graph edge information memory) and accessed by an appropriate processor during runtime. The system can also build a series of motion planning graphs before runtime, which correspond to different dimensional feature changes that may occur during the robot's runtime. The system then stores these planning graphs in memory.

[0121] At 402, at least one component of the processor-based system receives a kinematic model representing a robot for which motion planning will be performed. The kinematic model, for example, represents a robot attachment with a plurality of links and a plurality of joints, which are located between respective pairs of links. The kinematic model can be in any known or to-be-developed format. Optionally, at least one component of the processor-based system generates a data structure representation of the robot based at least in part on the kinematic model of the robot. The data structure representation includes representations of a plurality of links and joints. Various suitable data structures and techniques can be used, such as various hierarchical data structures (e.g., tree data structures) or non-hierarchical data structures (e.g., EDF data structures). This can, for example, use any structure, method, or technique described in PCT / US2019 / 045270 (published as WO 2020 / 040979).

[0122] At 404, the processor-based system generates a motion planning graph for the robot based on the corresponding robot kinematic model. The motion planning graph represents each state, configuration, or posture of the robot as a corresponding node, and represents valid transitions between pairs of states, configurations, or postures as edges connecting corresponding pairs of nodes. Although described as a graph, the motion planning graph does not necessarily need to be represented or stored as a conventional graph, but can be represented, for example, logically or in a storage circuit or computer processor using any type of data structure (e.g., records and fields, tables, linked lists, pointers, trees).

[0123] At 406, the processor-based system optionally generates a swept volume for each edge of the motion planning graph that will be used to perform motion planning for the robot. The swept volume represents the volume swept by the robot or a portion thereof when performing a motion or transformation equivalent to the corresponding edge. The swept volume can be represented as Various forms, such as representation as voxels, Euclidean distance fields, spheres, or other hierarchies of geometric objects.

[0124] At 408, the processor-based system provides the motion planning map and / or swept volume to the robot control system and / or motion planner. The processor-based system can provide the motion planning map and / or swept volume via a non-dedicated communication channel (e.g., Ethernet). In some embodiments, various robots from different manufacturers can operate in a shared workspace. In some embodiments, various robot manufacturers can operate proprietary processor-based systems (e.g., server computers) that generate motion planning maps and / or swept volumes for various robots produced by the robot manufacturers. Each of the robot manufacturers can provide a corresponding motion planning map and / or swept volume for use by the robot controller or motion planner.

[0125] The processor-based system may use, for example, any one or more of the structures and methods described in PCT / US2019 / 045270 (published as WO 2020 / 040979), PCT / US2020 / 034551 (published as WO 2020 / 247207), and / or PCT / US2019 / 016700 (published as WO 2019 / 156984) to generate a motion planning graph for a robot to perform motion planning, and / or generate a swept volume representing a robot to perform motion planning.

[0126] Figure 5 A method 500 of operating a processor-based system for controlling one or more robots according to at least one illustrated embodiment is shown. The method 500 may be performed during runtime of the robot. Alternatively, a portion of the method 500 may be performed during configuration time, before runtime of the robot, and a portion of the method 500 may be performed during runtime of the robot.

[0127] The method 500 may be performed for one, two or more robots operating in a multi-robot operating environment or workspace. For example, the method 500 may be performed sequentially for each of two or more robots (e.g., a first robot R1, a second robot R2, and a third robot R3). For ease of discussion, the method 500 is described relative to the control of the first robot R1, wherein the second robot R2 in the operating environment represents a potential obstacle to the movement of the first robot R1. Those skilled in the art will recognize from the discussion that the method can be extrapolated to control one robot in the case of two or more other robots in the operating environment, or to control two or more robots in the case of two or more robots in the operating environment. This can be performed, for example, by running the method 500 sequentially for each robot, and / or, for example, iteratively executing the method 500 sequentially for a given robot relative to each other robot that poses a potential obstacle to the movement of the given robot. Those skilled in the art will also recognize that the method 500 may omit some of the illustrated actions, include additional actions, and may be run or executed in a different order than the illustrated order and / or may be run or executed simultaneously with each other.

[0128] The method 500 may be performed at run time, during which at least one robot is operating (e.g., moving, performing a task), and typically two or more robots are operating, such as after configuration time. The method 500 may be performed by one or more processor-based systems employing one or more robot control systems, although the method 500 will be described with respect to one processor-based system. For example, the robot control system may be co-located or "piggybacked" with each robot.

[0129] The method 500 begins at 502, for example, in response to powering on a robot and / or a robot control system, in response to a call or invocation from a calling routine, or in response to receiving a task to be performed by the robot.

[0130] At 504, the processor-based system optionally receives a motion planning graph for one or more robots. For example, the processor-based system may receive the motion planning graph from another processor-based system that generates the motion planning graph at configuration time, such as with reference to Figure 4 As stated and Figure 4As shown. The motion planning graph represents each state, configuration, or posture of the robot as a corresponding node, and represents the valid transitions between pairs of states, configurations, or postures as edges connecting the corresponding nodes. Although described as a graph, each motion planning graph does not necessarily need to be represented or stored as a conventional graph, but can be represented using any type of data structure (e.g., records and fields, tables, linked lists, pointers, trees), for example, logically or in a storage circuit or computer processor.

[0131] At 506, the processor-based system optionally receives a set of swept volumes for one or more robots, such as the robot R1 for which motion is being planned and optionally other robots R2, R3 operating in the environment. For example, the processor-based system may receive the set of swept volumes from another processor-based system that generates a motion planning graph at configuration time, such as with reference to Figure 4 As stated and Figure 4 As shown. Alternatively, the processor-based system can generate the set of swept volumes itself, for example based on the motion planning graph. The swept volume represents the corresponding volume swept by the robot or its part when performing the motion or transformation corresponding to the corresponding edge. The swept volume can be represented in a variety of forms, such as a hierarchy of voxels, Euclidean distance fields, spheres, or other geometric objects.

[0132] At 508, the processor-based system receives a plurality of tasks, such as in the form of a task specification. The task specification specifies a robot task to be performed or carried out by the robot. The task specification may, for example, specify a first task and a second task, in which the robot moves from a first position and a first posture to a second position (e.g., a first target) and a second posture (e.g., a first target posture), and grasps an object at the second position; in the second task, the robot moves the object to a third position (e.g., a second target) and a third posture (e.g., a second target posture) and releases the object at the third position. Task specifications may take a variety of forms, such as high-level specifications (e.g., prose and syntax) that need to be parsed into low-level specifications. Task specifications may take the form of low-level specifications, such as specifying a set of joint positions and joint angles / rotations (e.g., joint postures, joint coordinates).

[0133] At 510, the processor-based system optionally receives or generates an environmental representation, such as a representation of an environment sensed by one or more sensors (e.g., a camera, LIDAR). Such a representation may represent static or persistent objects in the environment and / or represent dynamic or transient objects in the environment. In some embodiments, one or more sensors capture sensory data and send it to one or more processors. The sensory data may be, for example, occupancy information representing obstacles in the environment at their respective locations. Occupancy information is typically generated by one or more sensors that are positioned and oriented to sense the environment in which the robot will operate. The occupancy information may be in the form of raw data or pre-processed data, and may be formatted into any existing or later created format or pattern. The sensory data may, for example, be a stream of voxels or boxes indicating that the current environment has been occupied by objects. Receiving or generating a representation of the environment may occur over a period of time, such as during the entire runtime, and may occur periodically, non-periodically, constantly, and / or continuously.

[0134] For each of the plurality of iterations, the system receives or generates a respective discretization of a representation of the environment in which the robot 102 is operating. Each discretization may, for example, include respective sets of voxels. The voxels of each discretization may, for example, be non-uniform in at least one of size and shape within the respective discretization. The respective distributions of the non-uniformity of the voxels of the respective discretizations may, for example, be different from one another. The obstacles may be digitally represented, for example, as bounding boxes, oriented bounding boxes, or curves (e.g., splines), whichever digital representation is most appropriate for the obstacle type and the type of collision detection to be performed, which itself may depend on the specific hardware circuitry or software algorithm employed.

[0135] The processor-based system may, for example, represent one or more static or persistent obstacles in the environment and / or one or more dynamic or transient obstacles in the environment, including one or more other robots.

[0136] For example, at least one component of the processor-based system may receive occupancy information indicating the number of persistent obstacles in the environment and / or the number of transient obstacles in the environment. The persistent obstacles are obstacles that are generally static or remain fixed and do not move during at least the period of operation by the robot and during the robot's performance of one or more tasks. Transient obstacles are obstacles that appear / enter, disappear / leave, or are dynamic or moving, at least during the operation of the robot. Thus, transient obstacles (if any) are obstacles that have a known position and orientation in the environment at least for a portion of the operating time, but do not have a known fixed position or orientation in the environment at the configuration time. The occupancy information may, for example, indicate the corresponding volume occupied by each obstacle in the environment. The occupancy information may be in any known or yet to be developed format. The occupancy information may be generated by one or more sensors, or may be defined or specified by a computer or even by a person. The occupancy information of transient obstacles is generally received during the operation of the robot. Persistent or static objects are sometimes identified and planned during the configuration time, while transient or dynamic objects are generally identified and planned during the operating time.

[0137] At least one component of the processor-based system may generate one or more data structure representations of one or more objects or obstacles in the environment in which the robot will operate. The data structure representation may include representations of multiple obstacles that have known fixed positions and orientations in the environment at configuration time, and are therefore referred to as persistent obstacles because these objects are assumed to persist in the environment at known positions and orientations from configuration time to runtime. The data structure representation may include representations of multiple obstacles that do not have known fixed positions and orientations in the environment at configuration time and / or are expected to move during runtime, and are therefore referred to as transient obstacles because these objects are assumed to appear, disappear, or move in the environment at unknown positions and orientations during runtime. Various suitable data structures may be employed, such as various hierarchical data structures (e.g., tree data structures) or non-hierarchical data structures (e.g., EDF data structures) and techniques, such as described in PCT / US2019 / 045270 (published as WO 2020 / 040979). Thus, for example, at least one component of the processor-based system receives or generates a representation of obstacles in the environment as any one or more of: a Euclidean distance field, a hierarchy of bounding volumes; an axis-aligned bounding box (AABB) tree, an oriented (non-axis-aligned) bounding box tree, a sphere tree; a bounding box hierarchy with triangle meshes as leaf nodes; a sphere hierarchy; a k-ary sphere tree; a hierarchy of axis-aligned bounding boxes (AABBs); a hierarchy of oriented bounding boxes, and / or an octree storing voxel occupancy information. It is worth noting that the leaves of any tree data structure can be a different shape than other nodes of the data structure, for example, all nodes are AABBs except the root node or leaves which can be in the form of triangle meshes. Using spheres as bounding volumes facilitates fast comparison (i.e., it is easy to determine by calculation whether spheres overlap each other).

[0138] The processor-based system can, for example, use any one or more of the structures and methods described in PCT / US2019 / 045270 (published as WO 2020 / 040979) to represent obstacles in the environment.

[0139] Optionally at 512, the processor-based system receives motion plans for one or more other robots, i.e., motion plans for other robots in the environment other than the given robot that is executing the particular instance of motion planning. The processor-based system can advantageously use the motion plans of the one or more other robots to determine whether a trajectory of a given robot (e.g., the first robot R1) will result in a collision or a non-zero probability of a collision with another robot (e.g., the second robot R2, the third robot R3), as described herein.

[0140] Optionally at 514, the processor-based system predicts the movement of obstacles in the environment, such as the movement or trajectory of other robots (e.g., the second robot R2, the third robot R3) in the environment. The processor-based system can infer future trajectories, for example, based on a sample of the current trajectory and / or based on knowledge or prediction of the goals of other robots or predictions or expected collisions with other robots. This may be particularly useful when the processor-based system does not know the motion plans of one or more other robots.

[0141] At 516, the processor-based system represents other robots and / or other robot motions as obstacles to the robot that is executing the particular instance of motion planning. For example, the processor-based system may queue the swept volume corresponding to each motion as an obstacle in an obstacle queue. The swept volume for each edge may have been predetermined or calculated and logically associated with each edge in memory via a data structure (e.g., a pointer). As previously described, the swept volume may be represented in various forms, such as as a hierarchy of voxels, Euclidean distance fields, spheres, or other geometric objects.

[0142] At 518, the processor-based system optionally initializes or sets a target counter I to 1, which allows the processor-based system to loop through each of the total target number N.

[0143] At 520, the processor-based system performs motion planning for a given robot (e.g., a first robot R1) to perform several tasks (e.g., one, two, or more tasks), the motion planning being based on a number N of at least two or more consecutive goals. The motion planning is performed for at least two or more consecutive goals before the motion specified by the resulting two or more motion plans occurs. Thus, the processor-based system is able to consider motion plans for subsequent goals (e.g., second goals) when evaluating the compatibility or adequacy of motion plans (e.g., first motion plans) with previous goals (e.g., first goals), and optionally implement remedial actions if the remedial actions are deemed useful or necessary (as described below with respect to Fig. 6A This can advantageously prevent the robot from becoming stuck or even deadlocked, for example, if the transition from a pose associated with a first target specified by a previous or first motion plan to a next or subsequent pose specified by a subsequent or second motion plan is blocked or has the potential to be blocked (e.g., by another robot in the shared workspace). Fig. 6A and 6B , at least one embodiment of motion planning is shown and described, including collision checking and assigning costs to edges representing transitions in a motion planning graph. Fig. 6A and Figure 6BThe examples are not intended to be limiting, and other implementations of motion planning may be employed.

[0144] At 522, the processor-based system optionally controls the operation of a given robot (e.g., a first robot R1) for which a motion plan is generated, so that the given robot moves according to the motion plan. For example, the processor-based system can send a control signal or a drive signal to one or more motion controllers (e.g., a motor controller) to cause one or more actuators to move one or more links according to the motion plan. It should be noted that although an edge generally defines a valid transition between pairs of nodes representing corresponding postures, any edge can correspond to one, two, or even more movements of a robot or part thereof. Therefore, an edge can correspond to a single movement of the robot from one posture to another posture in a pair of postures, or alternatively an edge can correspond to multiple movements of the robot from one posture to another posture in a pair of postures.

[0145] At 524, the processor-based system monitors the planned trajectory of a given robot (e.g., the first robot R1) for which a motion plan is generated. Monitoring can include monitoring to determine whether the robot or the trajectory of the robot is blocked, such as by another robot. Monitoring can include monitoring to determine whether the given robot or the trajectory of the given robot may (e.g., potentially) be blocked by a known or predicted movement of another robot, optionally including an assessment of the probability of such blocking. Monitoring can include monitoring to determine whether the given robot or the trajectory of the given robot has become unblocked or may be unblocked, such as due to the movement of another robot.

[0146] Monitoring can include monitoring to determine whether a given robot has reached a goal (e.g., a final goal) or achieved a target pose (e.g., a final goal pose) that will complete a task. Monitoring can also include monitoring the robot or other robots for motion completion, which can advantageously allow a processor-based system to no longer consider obstacles corresponding to the motion during subsequent collision detection or assessment. In some embodiments, this can result in the generation of a motion completion message, which can be used, for example, to trim obstacles, as described below with reference to Figure 7 Discussion of . A processor-based system can rely on the coordinates of a corresponding robot. The coordinates can be based on information from a motion controller, actuators, and / or sensors (e.g., cameras including DOF cameras and LIDAR with or without structured lighting, rotary encoders, reed switches). In at least some embodiments, monitoring for obstruction or unobstructed traffic can include performing collision detection using any of a variety of collision detection techniques, such as collision detection using a swept volume representing the volume swept by a given robot and / or the volume swept by other robots or other obstacles in the environment; or collision detection using an evaluation of whether two spheres intersect.

[0147] At 526, the processor-based system determines or evaluates whether a given robot (e.g., a first robot R1) or a trajectory of a given robot has been blocked or may be blocked, or alternatively determines or evaluates whether another trigger condition has occurred. For example, the processor-based system can perform a collision check on the path or trajectory of a given robot relative to one or more objects or obstacles in the operating environment. A variety of forms of collision checking can be employed, although this process should be relatively fast because collision checking is typically performed during robot runtime. A swept volume can be used, for example, to represent an area or volume swept by a given robot when it transitions from one posture to another. A swept volume can be used, for example, to represent an area or volume swept by other robots when they transition from one posture to another. The swept volume can be advantageously calculated during configuration time prior to runtime. The collision check can employ a determination of the intersection of a sphere representing a given robot portion and an obstacle.

[0148] If the given robot (e.g., the first robot R1) or the trajectory of the given robot is blocked or is likely to be blocked (e.g., the probability of being blocked or the probability of collision is equal to or exceeds a defined threshold) or alternatively some other triggering condition occurs, control returns to 520. If the given robot or the trajectory of the given robot is not blocked or is not likely to be blocked (e.g., the probability of being blocked or the probability of collision is below a defined threshold), control is passed to 528.

[0149] At 528, the processor-based system determines whether a given robot (e.g., the first robot R1) has reached the current target I or has achieved a target pose that positions the robot or a portion thereof at the target I. The processor-based system may rely on the coordinates of the given robot. The coordinates may be based on information from a motion controller, actuators, and / or sensors (including DOF cameras and LIDAR with or without structured lighting, rotary encoders, cameras with reed switches).

[0150] If the given robot has not reached the current target I, nor achieved the target pose that positions the robot or part thereof at the target I, control returns to 522 or optionally to 524, allowing the given robot to continue to move toward the current target I according to the motion plan. If the given robot has reached the current target I or achieved the target pose, control is passed to 530.

[0151] At 530 , the processor-based system increments the target counter (I=I+1). Control then passes to 532 .

[0152] At 532, the processor-based system determines whether the total target number N has been reached (I>N) based on the target counter. If the total target number N has not been fully reached (I>N), control returns to 522. If the total target number N has been fully reached (I>N), control passes to 534.

[0153] At 534, method 500 terminates at 534 until, for example, called again. Alternatively, method 500 may be repeated until a stop is determined, such as a power off state or condition. In some embodiments, method 500 may be executed as a multithreaded process on one or more cores of one or more processors.

[0154] It is noteworthy that in many embodiments, method 500 will be repeated for additional tasks and / or additional robots. For example, a processor-based system can determine whether the end of a task queue and / or a motion planning request queue has been reached. For example, a processor-based system can determine whether there are any remaining motion planning requests in the motion planning request queue, and / or whether all tasks in the task queue have been completed. In at least some embodiments, a group of tasks (e.g., a task queue) may be temporarily exhausted, and new or additional tasks may arrive later. In such embodiments, a processor-based system can execute a waiting loop, checking from time to time whether there are new or additional tasks, or waiting for a signal indicating that there are new or additional tasks available for processing and execution.

[0155] In some embodiments, multiple instances of method 500 may be executed as multithreaded processes on one or more cores of one or more processors, for example, in parallel with each other. Thus, method 500 may optionally repeat until a stop is determined, such as a power off state or condition.

[0156] Although the method of operation 500 is described in terms of an ordered flow, in many embodiments various actions or operations will be performed simultaneously or in parallel. Typically, motion planning for one robot to perform a task can be performed while one or more robots are performing tasks. Thus, the performance of a robot's task can overlap, be performed simultaneously, or be parallel to the performance of a motion plan by one or more motion planners. The performance of a robot's task can overlap, be performed simultaneously, or be parallel to the performance of other robots' tasks. In some embodiments, at least some portions of a motion plan for one robot can overlap, be performed simultaneously, or be parallel to at least some portions of a motion plan for one or more robots.

[0157] Fig. 6AA high-level method 600 of operations for a processor-based system to perform motion planning for one or more robots according to at least one illustrated embodiment is shown. The method 600 may be performed during the robot's runtime. Alternatively, a portion of the method 600 may be performed during configuration time, before the robot's runtime, and a portion of the method 600 may be performed during the robot's runtime. The method 600 may be performed, for example, during execution of the method 500 ( Figure 5 ) when executing method 600, although motion planning 520 is not limited to the motion planning described and shown in method 600.

[0158] The method 600 can be performed for one, two or more robots operating in a multi-robot operating environment. For example, the method 600 can be performed for each robot (e.g., the first robot R1, the second robot R2, the third robot R3 or even more robots) in sequence. For ease of discussion, the method 600 is described with respect to the control of a given robot (e.g., the first robot R1), wherein other robots (e.g., the second robot R2, the third robot R3) in the operating environment may constitute obstacles to the movement of the given robot (e.g., the first robot R1). Those skilled in the art will recognize from the discussion herein that the method 600 can be extrapolated to motion planning for one robot in the case of two or more other robots in the operating environment, or motion planning for two or more robots in the case of two or more other robots in the operating environment. This can be performed, for example, by running the method 600 for each robot in sequence, and / or, for example, iteratively executing the method 600 for the given robot in sequence relative to each other robot that poses a potential obstacle to the movement of the given robot. Those skilled in the art will also recognize that method 600 may omit some of the illustrated actions, include additional actions, and that many of the actions of method 600 may be run or performed in a different order than illustrated and / or concurrently with each other.

[0159] The method 600 may be performed at run time, during which at least one robot is operating (e.g., moving, performing a task), such as after a configuration time. The method 600 may be performed by one or more processor-based systems employing one or more robot control systems. For example, the robot control systems may be co-located or "piggybacked" with the respective robots.

[0160] Method 600 begins at 602, for example, in response to powering on a robot and / or a robot control system, in response to a call or invocation from a calling routine (e.g., a call from a calling routine that executes method 500), in response to receiving a motion planning request or a motion planning request in a queue of motion planning requests to be executed by the robot, or in response to receiving a task to be executed by the robot.

[0161] At 604, the processor-based system performs collision detection or assessment for a given robot (e.g., the first robot R1). The processor-based system can perform collision detection or assessment using any of the various structures and algorithms described herein or described in the materials incorporated herein by reference or described elsewhere. The collision detection or assessment can include performing collision detection or assessment for each motion of each obstacle in the obstacle queue. In International Patent Application No. PCT / US2017 / 036880, filed on June 9, 2017, entitled “MOTION PLANNING FOR AUTONOMOUS VEHICLES AND RECONFIGURABLE MOTION PLANNING PROCESSORS”; U.S. Patent Application No. 62 / 722,067, filed on August 23, 2018, entitled “COLLISION DETECTION USEFUL IN MOTION PLANNING FOR ROBOTICS”; in International Patent Application Publication No. WO 2016 / 122840, filed on January 5, 2016, entitled “SPECIALIZED ROBOT MOTION PLANNING HARDWARE AND METHODS OF MAKING AND USING SAME”; and PCT / US2019 / 045270 (published as WO 2020 / 040979) describes an example of collision evaluation to generate a motion planning graph, a swept volume, and / or perform collision checks (e.g., sphere intersection evaluation). As described herein, a processor-based system can use collision evaluation to set cost values ​​or cost functions for each edge in a motion planning graph, or use the collision evaluation as part of setting cost values ​​or cost functions for each edge in a motion planning graph, which can be used to generate a motion plan that specifies a suitable path (e.g., a set of ordered poses) for a robot to complete a task.

[0162] In 606, the processor-based system sets the cost value or cost function of the edge in the motion planning graph based at least in part on the collision detection or evaluation of a given robot (e.g., the first robot R1). The cost value or cost function can represent a collision evaluation or collision risk (e.g., probability or occurrence). The cost value can also represent the severity of the collision in the event of a collision (e.g., the degree of damage that may be caused). For example, a collision with a person or other living object may be considered more serious than a collision with a wall, a table or other inanimate object, and therefore has a higher cost. The cost value or cost function can also represent the use or consumption of resources associated with the conversion corresponding to the corresponding edge, such as the amount of energy consumed or the amount of time that will be spent to perform the relevant conversion. The processor-based system can use any of the various structures and algorithms described herein or incorporated by reference into the material description of this article to perform the cost value or cost function setting, usually setting or adjusting the cost of the edge with no collision risk or low collision risk to a relatively low value (e.g., zero), and setting or adjusting the cost of the edge that causes a collision or has a high collision risk to a relatively high value (e.g., one hundred thousand). For example, the processor-based system may set the cost value or cost function of an edge by logically associating each edge in the motion planning graph with an associated cost value or cost function via one or more data structures.

[0163] At 608, the processor-based system generates a motion plan (e.g., a first motion plan) that specifies a set of valid transformations to transform or move a given robot from a starting or current configuration to a first target I. The first motion plan represents feasible paths between the starting point and the first target I, and in at least some instances represents a suitable path (e.g., a complete path that satisfies a specified criterion (e.g., a cost value below a threshold cost)) or even represents a selected path (e.g., an optimal path, such as a complete path with the lowest cost among all paths). Thus, the motion plan specifies a trajectory for the given robot to transform between the starting point and the target through various poses corresponding to corresponding nodes. Figure 6B Exemplary embodiments of generating a motion plan are described and illustrated.

[0164] At 610, the processor-based system generates a motion plan (e.g., a second motion plan) that specifies a set of valid transitions to transform or move a given robot from a first destination I to a second destination I+1. The second motion plan represents a feasible path between the first destination I and the second destination I+1, and in at least some instances represents a suitable path (e.g., a complete path that satisfies a specified criterion (e.g., a cost value below a threshold cost)) or even represents a selected path (e.g., an optimal path, such as a complete path with the lowest cost among all paths). Therefore, the motion plan specifies a trajectory for the given robot to transform between the first destination and the second destination through various postures corresponding to corresponding nodes. Refer to the following Figure 6B An exemplary embodiment of the generation of a motion plan is shown and described.

[0165] It should be noted that in some embodiments, the processor-based system can repeatedly generate subsequent motion plans for any number of subsequent goals (e.g., I+2, I+3, I+4), for example to allow for deeper consideration or inference of the impact that a given robot's early motion plans will have on the given robot's ability to achieve or reach one or even more subsequent goals.

[0166] At 612, the processor-based system determines or evaluates whether other robots will or are likely (i.e., with a relatively high probability) block the given robot or block the given robot from moving from a previous target (e.g., first target I) to a subsequent or next target (e.g., second target I+1) when the given robot moves along the trajectory specified by the corresponding motion plan (e.g., second motion plan). For example, the processor-based system can determine whether the given robot will be trapped, delayed, or even deadlocked when switching from a previous motion plan (e.g., first motion plan) to a subsequent (or referred to as "next" or "subsequent") motion plan (e.g., second motion plan). For example, the processor-based system can determine whether the given robot will be blocked or likely to be blocked from switching from the posture of the given robot at the first target according to the first motion plan to the posture of the given robot during at least the initial portion of the execution of the second motion plan. For example, the processor-based system can determine whether the given robot will or is likely to be unable to execute a subsequent motion plan (e.g., next or subsequent motion plan) due to being blocked by another robot. For another example, the processor-based system can determine whether the given robot will or is likely to delay the execution of a subsequent motion plan (e.g., next or subsequent motion plan) due to being blocked by another robot. In at least some instances, the delay can be any delay, such as any amount of delay other than the ability to execute the subsequent or second motion plan without delay once commanded to execute the subsequent or second motion plan. In other cases, the delay can be a delay longer than a specified or threshold delay duration, such as a specified non-zero amount of delay in the ability to execute the subsequent or second motion plan once commanded to execute the subsequent or second motion plan.

[0167] The processor-based system may, for example, employ collision detection to determine or assess whether other robots will collide with each other when a given robot moves along a trajectory specified by a corresponding motion plan (e.g., a second motion plan). or is likely (i.e., with a relatively high probability) to block the given robot or to block the given robot from moving toward a subsequent target (e.g., a second target I+1). For example, a collision probability exceeding a threshold collision probability or a cost value or cost function exceeding a threshold cost or threshold cost value may indicate a blocking condition. Various collision detection methods may be employed, such as described elsewhere herein. If the other or more robots will block or are likely to block the given robot or to block the given robot from moving toward the target, control is transferred to 614. If the other or more robots will not block or are unlikely (i.e., with a relatively low probability) to block the given robot or to block the given robot from moving toward the target, control is transferred to 618.

[0168] Optionally at 614, the processor-based system selects one or more remedial actions in response to determining that the given robot is blocked or potentially blocked when attempting to transition from a previous goal (e.g., first goal I) to a subsequent or subsequent or next goal (e.g., second goal I+1) when the given robot moves along the trajectory specified by the corresponding motion plan (e.g., the second motion plan).

[0169] Examples of remedial actions include generating a new or revised or replaced motion plan to convert a given robot to a previous target (e.g., generating a new or revised or replaced first motion plan). The new or revised or replaced motion plan can, for example, position a given robot or part thereof at a first target position, volume, region, or space, but its target pose is different from the target pose of the previously generated first motion plan to place the given robot. Different target poses may make it easier for the given robot to convert to a subsequent target (e.g., a second target I+1), thereby avoiding being trapped or even falling into a deadlock. In at least some embodiments, when generating a new or revised or replaced motion plan for a given robot (e.g., generating a new or revised or replaced first motion plan), a processor-based system may produce a deviation or otherwise cause the generation of a new or revised or replaced motion plan, the target pose of which is different from the target pose of the previously generated motion plan (e.g., the first motion plan) of the given robot. This can be achieved, for example, by at least temporarily increasing the cost associated with the edge of the node corresponding to the end pose of the previously generated motion plan connected to the given robot.

[0170] The processor-based system may optionally select between the first motion plan and at least one or more revised motion plans, for example, based at least in part on a comparison of the corresponding delay amounts associated with each of the first motion plan and at least one or more revised motion plans. For example, the first motion plan may be associated with a first delay amount (e.g., 90 seconds), during which the given robot will be or may be blocked by another robot, and thus be stuck at the first target with a first final posture before being able to execute the second motion plan. For another example, the first revised motion plan may be associated with a second delay amount (e.g., 60 seconds), during which the given robot will be or may be blocked by another robot, and thus be stuck at the first target with a second final posture before being able to execute the second motion plan. In this example, the processor-based system may select the first revised motion plan to be executed by the first robot because this will produce the minimum delay among the feasible options. As a further example, the second revised motion plan may be associated with a third delay amount (e.g., 15 seconds), during which the given robot will be or may be blocked by another robot, and thus be stuck at the first target with a third final posture before being able to execute the second motion plan. In this example, the processor-based system may select the second modified motion plan to be executed by the first robot because this will produce the least delay among the feasible options. Although the selection is illustrated based solely on delay, duration, or waiting time, the processor-based system may consider other parameters, such as collision risk or collision probability, collision severity, and / or energy consumption.

[0171] Another example of a remedial action includes moving the other robot if the other robot is blocking or is likely to block the given robot. This may include, for example, causing the other robot to move out of the way or otherwise move so as not to block the given robot from moving along the trajectory specified by the motion plan (e.g., first motion plan, second motion plan) generated for the given robot.

[0172] Additional examples of remedial actions include generating a new, revised, or replacement motion plan for another robot (whose trajectory or planned trajectory will block or is likely to block the given robot), such as causing the other robot to move in a manner such that the other robot does not block or is less likely to block the given robot from moving along the motion trajectory specified by the motion plan (e.g., first motion plan, second motion plan) generated for the given robot.

[0173] Another example of a remedial action includes determining or generating a new order of a set of goals, the set of goals including a first goal and at least a second goal. Thus, the original first goal can become the second goal, and the original second goal can also become the first goal.

[0174] Optionally at 616, the processor-based system causes one or more remedial actions to be taken. This may, for example, include the processor-based system invoking another iteration of motion planning for the given robot. This may, for example, alternatively or additionally include the processor-based system invoking another iteration of motion planning for one or more other robots that are blocking or are likely to block the movement of the given robot. This may, for example, alternatively or additionally include the processor-based system sending or transmitting control signals or drive signals to one or more motion controllers (e.g., motor controllers) to cause one or more actuators to move one or more links of one or more other robots to move the other robots so that they do not block or are less likely to block the given robot. This may, for example, alternatively or additionally include determining or generating a new order of a set of goals, the set of goals including a first goal and at least a second goal.

[0175] At 618, the processor-based system provides a motion plan that implements the selected path (e.g., selected path, selected feasible path, selected suitable path, selected suitable feasible path) to control the operation of a given robot (e.g., the first robot R1) for which the motion plan has been generated. For example, the processor-based system can send control signals or drive signals to one or more motion controllers (e.g., motor controllers) to cause one or more actuators to move one or more links to move the given robot or a part thereof (e.g., an attachment, an end effector, an end-of-arm tool) along a trajectory specified by the motion plan.

[0176] Method 600 may terminate at 620 until, for example, called again. Alternatively, method 600 may repeat until a stop is determined, such as a power off state or condition. In some embodiments, method 600 may be executed as a multithreaded process on one or more cores of one or more processors.

[0177] Although the method of operation 600 is described in terms of an ordered flow, in many embodiments various actions or operations will be performed simultaneously or in parallel. Typically, the motion planning of one robot to perform a task can be performed at the same time as one or more robots are performing the task. Thus, the execution of a robot's task can overlap, be simultaneous, or be parallel to the execution of a motion plan of one or more motion planners. The execution of a robot's task can overlap, be simultaneous, or be parallel to the execution of other robots' tasks. In some embodiments, at least some portions of a robot's motion plan can overlap, be simultaneous, or be parallel to at least some portions of a motion plan of one or more robots.

[0178] Figure 6BA low-level method 630 of operations for a processor-based system to perform motion planning for one or more robots according to at least one illustrated embodiment is shown. The method 630 may be performed during the robot's runtime. Alternatively, a portion of the method 630 may be performed during configuration time, before the robot's runtime, and a portion of the method 630 may be performed during the robot's runtime. The method 630 may be performed, for example, during execution of the method 500 ( Figure 5 ) when performing motion planning 520, although motion planning 520 is not limited to that described and shown in method 630.

[0179] Method 630 can be performed for one, two or more robots operating in a multi-robot operating environment. For example, method 630 can be performed for each robot (e.g., a first robot R1, a second robot R2, a third robot R3, or even more robots) in sequence. For ease of discussion, method 630 is described with respect to the control of a given robot (e.g., a first robot R1), wherein other robots (e.g., a second robot R2, a third robot R3) in the operating environment may represent obstacles to the movement of the first robot. Those skilled in the art will recognize from the discussion herein that method 630 can be extrapolated to motion planning for one robot in the case of two or more other robots in the operating environment, or motion planning for each of two or more robots in the case of one, two or more other robots in the operating environment. This can be performed, for example, by running method 630 for each robot in sequence, and / or, for example, iteratively executing method 630 for a given robot in sequence relative to each other robot that poses a potential obstacle to the movement of the given robot. Those skilled in the art will also recognize that method 630 may omit some of the illustrated actions, include additional actions, and that many of the actions of method 630 may be run or performed in a different order than illustrated and / or concurrently with each other.

[0180] The method 630 may be performed at runtime, during which at least one robot is operating (e.g., moving, performing a task), such as after configuration time. The method 630 may be performed by one or more processor-based systems employing one or more robot control systems. For example, the robot control systems may be co-located or "piggybacked" with the respective robots.

[0181] Method 630 begins at 632, for example, in response to powering on the robot and / or robot control system, in response to a call or invocation from a calling routine (e.g., a call from a routine executing method 500 or a call from a routine executing method 600), in response to receiving a motion planning request or receiving a motion planning request in a queue of motion planning requests to be executed by the robot, or in response to receiving a task to be executed by the robot.

[0182] At 634, the processor-based system generates one or more feasible paths using the motion planning graph. A feasible path is a complete path extending from a start node or current node through a target node via valid transitions. A feasible path specifies possible trajectories for a given robot through various poses corresponding to corresponding nodes.

[0183] It should be noted that one or more of the generated feasible paths may not be suitable feasible paths, for example because the cost or cost function of the transformation or edge along the path is too high, or even the cumulative cost or cumulative cost function along the path is too high (for example, higher than a threshold cost or value or size). The processor-based system may ultimately reject those feasible paths because they are not suitable for a given robot to perform a specific task, although it is possible and therefore feasible to perform the task via the transformation defined by the path. Therefore, as discussed below with reference to 636, the processor-based system can optionally identify one or more suitable feasible paths from a set of one or more feasible paths, for example based on a cost threshold. In addition, as discussed below with reference to 638, the processor-based system can select one of the feasible paths, or even select one of the suitable feasible paths when a suitable feasible path is identified, for example, by minimum cost analysis to identify the selected path (or referred to as a selected feasible path).

[0184] Optionally at 636, the processor-based system identifies one or more suitable paths (also interchangeably referred to as suitable feasible paths or identified suitable feasible paths) from one or more feasible paths. The processor-based system can, for example, identify a set of feasible paths, each path having a relevant cumulative cost on the corresponding feasible path that is equal to or lower than a threshold cost. Therefore, the set of suitable feasible paths can include those feasible paths that meet certain defined cost conditions or constraints. As noted elsewhere, the cost can reflect various parameters, including, for example, collision risk or collision probability, severity of collision, energy consumption, duration or delay in executing or completing a path or assigned task. The processor-based system can enforce other conditions or constraints in addition to or in place of the cost condition to identify the suitable feasible path.

[0185] Optionally at 638, the path selected by the processor-based system is referred to as a selected path. A processor-based system can, for example, select a path from one or more feasible paths (therefore interchangeably referred to as a selected feasible path). A processor-based system can, for example, optionally select a path from one or more suitable paths identified (therefore interchangeably referred to as a selected suitable path or a selected suitable feasible path). For example, a processor-based system can perform a minimum cost analysis on a set of feasible paths or a set of suitable paths to find the path with the lowest cost or relatively low cost in all feasible paths. Since the cost or cost function at least partially represents a collision check, a path is selected at least partially based on collision detection or evaluation (such as collision risk or collision probability). The cost value or cost function can also represent other criteria and / or parameters, such as the severity of the collision, energy consumption and / or time consumption when a collision occurs when performing the corresponding motion. The processor-based system can select a path (e.g., select a path, select a feasible path, select a suitable path, select a suitable feasible path) using any of the various structures and algorithms described herein or in materials incorporated herein by reference, such as by performing a minimum cost analysis on feasible paths or suitable paths generated from a motion planning graph having associated cost values.

[0186] Optionally at 640, the processor-based system selects a multi-path combination or a multi-motion plan combination, for example, based on one or more parameters (e.g., a total cost or total cost of the combination), selects a combination of a previous path or a previous motion plan (e.g., a first path or a first motion plan) and a subsequent path or a subsequent motion plan (e.g., a second path or a second motion).

[0187] The processor-based system may, for example, generate a first set of candidate paths or candidate motion plans, each of which specifies a complete path between a current or starting node and a first target. Additionally or alternatively, the processor-based system may, for example, generate a second set of candidate paths or candidate motion plans, each of which specifies a complete path between a first target and a subsequent, subsequent, or second target. The processor-based system may, for example, select a path or motion plan from the first set and / or a path or motion plan from the second set based on one or more criteria.

[0188] The processor-based system can, for example, perform a minimum cost analysis on various combinations (each combination includes a first path or a first motion from a first group of candidates and a second path or a second motion plan from a second group of candidates) to find a combination with the lowest total aggregate cost or a relatively low total aggregate cost from the current or starting posture to the Nth target, where N is an integer equal to or greater than 2. Since the cost or cost function at least partially represents a collision check, the combination is selected at least partially based on collision detection or evaluation (e.g., collision risk or collision probability). The cost value or cost function can also represent other criteria and / or parameters, such as the severity of the collision when a collision occurs when performing the corresponding motion, energy consumption and / or time consumption, etc. The processor-based system can use any of the various structures and algorithms described in this article or the materials described by reference to this article to select a path (e.g., a selected path, a selected feasible path, a selected suitable path, a selected suitable feasible path), such as by performing a minimum cost analysis on a feasible path or a suitable path generated from a motion planning graph with an associated cost value.

[0189] Although not explicitly stated, combinations can be identified and selected to minimize the total count of gestures or conversions experienced when reaching two or more targets. In some cases, this can advantageously minimize delays and / or energy consumption. It should be noted that the duration and / or energy consumption of the conversion between any given gesture pair may not necessarily be equal to the duration and / or energy consumption of the conversion between some other gesture pairs, so minimizing the total count of gestures or conversions may not necessarily limit or reduce delays and / or energy consumption.

[0190] Method 630 may terminate at 642 until, for example, called again. Alternatively, method 630 may repeat until a stop is determined, such as a power off state or condition. In some embodiments, method 630 may be executed as a multithreaded process on one or more cores of one or more processors.

[0191] Although the method 630 of operation is described according to an ordered flow, in many embodiments various actions or operations will be performed simultaneously or in parallel. Typically, motion planning for one robot to perform a task can be performed at the same time as one or more robots perform the task. Thus, the performance of a robot's task can overlap, be simultaneous, or be parallel to the performance of a motion plan by one or more motion planners. The performance of a robot's task can overlap, be simultaneous, or be parallel to the performance of other robots' tasks. In some embodiments, at least some portions of a motion plan for one robot can overlap, be simultaneous, or be parallel to at least some portions of a motion plan for one or more robots.

[0192] Figure 7A method 700 of operating a processor-based system for controlling one or more robots in a multi-robot environment according to at least one illustrated embodiment is shown. The method 700 may be performed, for example, during the runtime of one or more robots as a method of executing the method 500 ( Figure 5 ) and / or perform method 600 ( Figure 6A-6B ) is executed as part of a system. The method 700 may be performed by one or more processor-based systems in the form of one or more robotic control systems. For example, the robotic control systems may be co-located or "piggybacked" with the respective robots.

[0193] Method 700 begins at 702, for example in response to powering on a robot and / or robot control system, in response to a call or invocation from a calling routine (eg, method 500), or in response to receiving a set or list or fleet of tasks.

[0194] Optionally at 704, the processor-based system optionally determines whether the corresponding motion has been completed. Monitoring the completion of a motion can advantageously allow the processor-based system to no longer consider obstacles corresponding to the motion during subsequent collision detection or evaluation. The processor-based system can rely on the coordinates of the corresponding robot. The coordinates can determine whether a given motion has been completed based on information from a motion controller, an actuator, and / or a sensor (e.g., a camera, a rotary encoder, a reed switch).

[0195] Optionally at 706, the processor-based system generates or transmits a motion completion message in response to determining that the given motion has been completed. Alternatively, the processor-based system may set and reset a flag.

[0196] At 708, a processor-based system (e.g., obstacle trimming 260, Figure 2 ) In response to determining that a motion completion message has been generated or received or a flag has been set, pruning one or more obstacles corresponding to a given motion. For example, the processor-based system can remove the corresponding obstacle from the obstacle queue. This is conducive to collision detection or evaluation in an environment where the swept volume or other representations (such as spheres, bounding boxes) that no longer exist as obstacles have been cleared. For another example, the processor-based system can remove the swept volume corresponding to the edge representing the completed motion of the given robot. This also advantageously allows tracking of the motion and the corresponding swept volume without tracking the time of the motion or the corresponding swept volume.

[0197] Method 700 may terminate at 710 until, for example, called again. Alternatively, method 700 may repeat until a stop is determined, such as a power off state or condition. In some embodiments, method 700 may be executed as a multithreaded process on one or more cores of one or more processors.

[0198] Example Example 1. A method for controlling the operation of one or more robots in a multi-robot environment by a processor-based system, the method comprising: For a first robot of the one or more robots, performing motion planning for the first robot by at least one processor to determine a first motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the first motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose that positions at least a portion of the first robot at the first target; for a second destination, performing motion planning for the first robot by at least one processor to attempt to determine a second motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the second motion plan specifying a plurality of poses that transform the first robot from a first end pose to a second end pose that positions at least a portion of the first robot at the second destination; and After performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, the first robot is moved.

[0199] Example 2. The method of Example 1, wherein performing motion planning for the first robot includes determining whether movement of the first robot along a trajectory will result in a collision with the second robot, or whether a probability of causing a collision with the second robot exceeds a threshold probability.

[0200] Example 3. The method as described in Example 2 further includes: In response to determining that the trajectory of the first robot will result in a collision with the second robot or that the probability of resulting in a collision with the second robot exceeds a threshold probability, the second robot is moved out of the path of the first robot.

[0201] Example 4. The method as described in Example 2 further includes: In response to determining that the trajectory of the first robot will result in a collision with the second robot or that the probability of resulting in a collision with the second robot exceeds a threshold probability, the second robot is moved out of a path of the first robot specified by at least one of a first motion plan of the first robot or a second motion plan of the first robot.

[0202] Example 5. The method as described in Example 2 further includes: In response to determining that the trajectory of the first robot will result in a collision with the second robot or that the probability of a collision with the second robot exceeds a threshold probability, motion planning is performed by at least one processor for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose while avoiding a collision with the second robot or at least reducing the probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target.

[0203] Example 6. The method as described in Example 5 further includes: The first robot is caused to move according to the revised motion plan of the first robot.

[0204] Example 7. A method as described in Example 1, wherein performing motion planning for the first robot includes determining whether the first robot moving along a first trajectory will cause a collision with the second robot moving along a second trajectory, or whether the probability of causing a collision with the second robot moving along the second trajectory exceeds a threshold probability.

[0205] Example 8. The method as described in Example 7 further includes: in response to determining that the first robot moving along a first trajectory will result in a collision with a second robot moving along a second trajectory or that a probability of causing a collision with the second robot moving along a second trajectory exceeds a threshold probability, performing motion planning by at least one processor for the second robot to determine a motion plan for the second robot, the motion plan for the second robot specifying at least one pose that transitions the second robot out of a path of the first robot; and The second robot is moved according to a second trajectory of the second robot.

[0206] Example 9. The method as described in Example 7 further includes: In response to determining that the first robot moving along a first trajectory will cause a collision with the second robot moving along a second trajectory or that a probability of a collision with the second robot moving along the second trajectory exceeds a threshold probability, motion planning is performed by at least one processor for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least a second trajectory of the second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transition the first robot from a pose to a first end pose while avoiding or at least reducing a probability of a collision with the second robot as the second robot moves along the second trajectory of the second robot, the first end pose positioning at least a portion of the first robot at the first target.

[0207] Example 10. The method as shown in Example 9, further comprising: The first robot is caused to move according to the revised motion plan of the first robot.

[0208] Example 11. The method of any one of Examples 1 to 10, wherein causing the first robot to move comprises causing the first robot to move to the first target after performing motion planning for the first robot to attempt to determine a second motion plan for the first robot.

[0209] Example 12. The method of any of Examples 1 to 10, wherein moving the first robot comprises moving the first robot to the second target after moving the first robot to the first target.

[0210] Example 13. The method of any one of Examples 1 to 10, further comprising: for a third destination, performing motion planning for the first robot by at least one processor to attempt to determine a third motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the third motion plan specifying a plurality of poses that transform the first robot from the second end pose to a third end pose that positions at least a portion of the first robot at the third destination, Wherein causing the first robot to move occurs after performing motion planning for the first robot to attempt to determine a third motion plan for the first robot.

[0211] Example 14. The method of any one of Examples 1 to 10, wherein performing motion planning for the first robot comprises: representing at least a second robot as at least one obstacle; and Collision detection is performed, by at least one processor, on at least one movement of at least a portion of the first robot relative to a representation of at least one obstacle.

[0212] Example 15. The method of any one of Examples 1 to 10, wherein performing motion planning for the first robot comprises: representing a plurality of motions of at least a second robot as at least one obstacle; and Collision detection is performed, by at least one processor, on at least one movement of at least a portion of the first robot relative to a representation of at least one obstacle.

[0213] Example 16. A method as described in Example 15, wherein representing multiple motions of at least the second robot as obstacles includes: using a set of swept volumes, each of which represents a corresponding volume swept by at least the portion of the second robot as the at least portion of the second robot moves along a trajectory represented by a corresponding motion.

[0214] Example 17. The method of Example 16 further includes: A set of swept volumes previously calculated at pre-runtime is received by at least one processor, each of the swept volumes representing a respective volume swept by the at least portion of the second robot as the at least portion of the second robot moves along a trajectory represented by a respective motion.

[0215] Example 18. The method of Example 15, wherein representing the plurality of movements of the second robot as obstacles comprises: representing the movements of the robot as at least one of: an occupancy grid, a hierarchical tree, or a Euclidean distance field.

[0216] Example 19. The method of any one of Examples 1 to 10, further comprising: For each of at least the first robot and the second robot, a corresponding motion planning graph is generated by at least one processor, each motion planning graph includes a plurality of nodes and edges, the nodes representing the corresponding states of the corresponding first robot and the second robot, and the edges representing the valid transitions between the corresponding states represented by the corresponding nodes of the corresponding node pairs connected by the corresponding edges.

[0217] Example 20. A method as described in any of Examples 1 to 10, wherein motion planning is performed for the first robot to determine a first motion plan for the first robot, and motion planning is performed for the first robot to attempt to determine a second motion plan for the first robot, both of which occur during runtime of at least one of the one or more robots.

[0218] Example 21. The method of any one of Examples 1 to 10, further comprising: Responsive to determining that the trajectory of the first robot will result in a collision with the second robot or that a probability of a collision with the second robot exceeds a threshold probability, taking remedial action by the at least one processor.

[0219] Example 22. The method of Example 21 further includes: A remedial action to be taken is autonomously selected by the at least one processor from a set of two or more defined remedial actions.

[0220] Example 23. A method as described in any of Examples 21 or 22, wherein taking remedial action by the at least one processor includes taking at least one of the following remedial actions: i) performing motion planning by the at least one processor for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least a second robot among the one or more robots, the revised motion plan specifying multiple poses that transform the first robot from one pose to a first end pose while avoiding a collision with the second robot or at least reducing the probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target; ii) moving the second robot out of a path of the first robot specified by at least one of the first motion plan of the first robot or the second motion plan of the first robot; iii) performing motion planning by the at least one processor for the second robot to determine a revised motion plan for the second robot, the revised motion plan specifying multiple poses that transform the second robot out of the path of the first robot; and iv) determining a new order of a set of targets, the set of targets including the first target and at least the second target.

[0221] Example 24. The method of any one of Examples 1 to 10, further comprising: A motion plan is autonomously selected, by the at least one processor, from a set of two or more candidate motion plans based at least in part on total costs of the two or more motion plans between corresponding ones of the two or more objectives.

[0222] Example 25. A processor-based system for controlling one or more robots, the system comprising: at least one processor; at least one non-transitory processor-readable medium communicatively coupled to the at least one processor and storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: Perform the method of any one of Examples 1 to 24.

[0223] Example 26. A processor-based system for controlling one or more robots, the system comprising: The first robot in a robotic environment; at least a second robot in the robotic environment; at least one processor; and at least one non-transitory processor-readable medium communicatively coupled to the at least one processor and storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: For a first robot of the one or more robots, performing motion planning for the first robot to determine a first motion plan for the first robot for a first destination, wherein the motion planning takes into account at least a second robot of the one or more robots, the first motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose that positions at least a portion of the first robot at the first destination; for a second destination, performing motion planning for a first robot to attempt to determine a second motion plan for the first robot, wherein the motion planning takes into account at least a second robot of the one or more robots, the second motion plan specifying a plurality of poses that transform the first robot from the first end pose to a second end pose that positions at least a portion of the first robot at the second destination; and After performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, the first robot is moved.

[0224] Example 27. The processor-based system of Example 26, wherein to perform motion planning for the first robot, the at least one processor determines whether movement of the first robot along a trajectory will result in a collision with the second robot, or whether a probability of causing a collision with the second robot exceeds a threshold probability.

[0225] Example 28. The processor-based system of Example 27, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: In response to determining that the trajectory of the first robot will result in a collision with the second robot or that the probability of resulting in a collision with the second robot exceeds a threshold probability, the second robot is moved out of the path of the first robot.

[0226] Example 29. The processor-based system of Example 27, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: In response to determining that the trajectory of the first robot will result in a collision with the second robot or that the probability of resulting in a collision with the second robot exceeds a threshold probability, the second robot is moved out of a path of the first robot specified by at least one of a first motion plan of the first robot or a second motion plan of the first robot.

[0227] Example 30. The processor-based system of Example 27, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: In response to determining that the trajectory of the first robot will result in a collision with the second robot or that the probability of a collision with the second robot exceeds a threshold probability, motion planning is performed for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose while avoiding a collision with the second robot or at least reducing the probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target.

[0228] Example 31. The processor-based system of Example 30, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: The first robot is caused to move according to the revised motion plan of the first robot.

[0229] Example 32. A processor-based system as described in Example 26, wherein to perform motion planning for the first robot, the at least one processor determines whether the first robot moving along a first trajectory will cause a collision with the second robot moving along a second trajectory, or whether the probability of causing a collision with the second robot moving along the second trajectory exceeds a threshold probability.

[0230] Example 33. The processor-based system of Example 32, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: in response to determining that the first robot moving along a first trajectory will result in a collision with a second robot moving along a second trajectory or will result in a collision with the second robot moving along a second trajectory with a probability exceeding a threshold probability, performing motion planning for the second robot to determine a motion plan for the second robot, the motion plan for the second robot specifying at least one pose that transitions the second robot out of a path of the first robot; and The second robot is moved according to a second trajectory of the second robot.

[0231] Example 34. The processor-based system of Example 32, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: In response to determining that the first robot moving along a first trajectory will result in a collision with a second robot moving along a second trajectory or that a probability of a collision with the second robot moving along the second trajectory exceeds a threshold probability, motion planning is performed for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least a second trajectory of a second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transition the first robot from a pose to a first end pose while avoiding or at least reducing a probability of a collision with the second robot as the second robot moves along the second trajectory of the second robot, the first end pose positioning at least a portion of the first robot at the first target.

[0232] Example 35. The processor-based system of Example 34, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: The first robot is caused to move according to the revised motion plan of the first robot.

[0233] Example 36. A method as described in any of Examples 26 to 35, wherein to cause the first robot to move, after performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, the at least one processor causes the first robot to move to the first target.

[0234] Example 37. The processor-based system of any of Examples 26 to 35, wherein to cause the first robot to move, the at least one processor causes the first robot to move to the second target after causing the first robot to move to the first target.

[0235] Example 38. The processor-based system of any of Examples 26 to 35, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: for a third destination, performing motion planning for the first robot to attempt to determine a third motion plan for the first robot, wherein the motion planning takes into account at least a second robot of the one or more robots, the third motion plan specifying a plurality of poses that transform the first robot from the second end pose to a third end pose that positions at least a portion of the first robot at the third destination, Wherein causing the first robot to move occurs after performing motion planning for the first robot to attempt to determine a third motion plan for the first robot.

[0236] Example 39. The processor-based system of any of Examples 26 to 35, wherein to perform motion planning for the first robot, the at least one processor: representing at least a second robot as at least one obstacle; and Collision detection is performed on at least one movement of at least a portion of the first robot relative to a representation of at least one obstacle.

[0237] Example 40. The processor-based system of any of Examples 26 to 35, wherein to perform motion planning for the first robot, the at least one processor: representing a plurality of motions of at least a second robot as at least one obstacle; and Collision detection is performed on at least one movement of at least a portion of the first robot relative to a representation of at least one obstacle.

[0238] Example 41. A processor-based system as described in Example 40, wherein to represent multiple motions of at least a second robot as obstacles, the at least one processor uses a set of swept volumes, each of the swept volumes representing a corresponding volume swept by the at least portion of the second robot as the at least portion of the second robot moves along a trajectory represented by a corresponding motion.

[0239] Example 42. The processor-based system of Example 41, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: A set of swept volumes previously calculated at pre-run time is received, each of the swept volumes representing a respective volume swept by the at least portion of the second robot as the at least portion of the second robot moves along a trajectory represented by a respective motion.

[0240] Example 43. A processor-based system as described in Example 40, wherein to represent multiple movements of the second robot as obstacles, the at least one processor: represents the movements of the robot as at least one of the following: an occupancy grid, a hierarchical tree, or a Euclidean distance field.

[0241] Example 44. The processor-based system of any of Examples 26 to 35, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: For each of at least the first robot and the second robot, a corresponding motion planning graph is generated, each motion planning graph includes a plurality of nodes and edges, the nodes representing the corresponding states of the corresponding first robot and the second robot, and the edges representing the valid transitions between the corresponding states represented by the corresponding nodes of the corresponding node pairs connected by the edges.

[0242] Example 45. A processor-based system as described in any of Examples 26 to 35, wherein performing motion planning for the first robot to determine a first motion plan for the first robot, and performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, both occur during runtime of at least one of the one or more robots.

[0243] Example 46. The processor-based system of any of Examples 26 to 36, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: Responsive to determining that the trajectory of the first robot will result in a collision with the second robot or that a probability of a collision with the second robot exceeds a threshold probability, taking remedial action.

[0244] Example 47. The processor-based system of Example 46, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: Autonomously select a remedial action to take from a set of two or more defined remedial actions.

[0245] Example 48. A processor-based system as described in any of Examples 46 or 47, wherein in order to take remedial action, the at least one processor causes at least one of the following remedial actions to occur: i) performing motion planning for the first robot by the at least one processor to determine a revised motion plan for the first robot, the motion planning taking into account at least a second robot among the one or more robots, the revised motion plan specifying multiple poses that transform the first robot from one pose to a first end pose while avoiding a collision with the second robot or at least reducing the probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target; ii) moving the second robot out of a path of the first robot specified by at least one of the first motion plan of the first robot or the second motion plan of the first robot; iii) performing motion planning for the second robot to determine a revised motion plan for the second robot, the revised motion plan specifying multiple poses that transform the second robot out of the path of the first robot; and iv) determining a new order for a set of targets, the set of targets including the first target and at least the second target.

[0246] Example 49. The processor-based system of any of Examples 26 to 36, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: A motion plan is autonomously selected from a set of two or more candidate motion plans based at least in part on a total cost of the two or more motion plans between corresponding ones of the two or more objectives.

[0247] Example 50. A method for controlling the operation of one or more robots in a multi-robot environment by a processor-based system, the method comprising: performing motion planning, by at least one processor, for a first robot of the one or more robots to determine a first motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the first motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose that positions at least a portion of the first robot at a first destination; performing motion planning for a first robot by at least one processor in an attempt to determine a second motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the second motion plan specifying a plurality of poses that transform the first robot from the first end pose to a second end pose that positions at least a portion of the first robot at a second destination; determining whether the first robot will at least one of become stuck, delayed, or deadlocked when transitioning from the first motion plan to the second motion plan; and In response to determining that the first robot will at least one of be stuck, delayed, or deadlocked when transitioning from the first motion plan to the second motion plan, at least one remedial action is performed by the at least one processor.

[0248] Example 51. The method of Example 50 further includes: A remedial action to be taken is autonomously selected by the at least one processor from a set of remedial actions.

[0249] Example 52. A method as described in any of Examples 50 or 51, wherein performing remedial action by the at least one processor includes performing at least one of the following remedial actions: i) performing motion planning by the at least one processor for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least a second robot among the one or more robots, the revised motion plan specifying multiple poses that transform the first robot from one pose to a first end pose while avoiding a collision with the second robot or at least reducing the probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target; ii) moving the second robot out of a path of the first robot specified by at least one of the first motion plan of the first robot or the second motion plan of the first robot; iii) performing motion planning by the at least one processor for the second robot to determine a revised motion plan for the second robot, the revised motion plan specifying multiple poses that transform the second robot out of the path of the first robot; and iv) determining a new order of a set of targets, the set of targets including the first target and at least the second target.

[0250] Example 53. A method as described in any of Examples 50 or 51, wherein the remedial action performed by the at least one processor includes: performing additional motion planning by the at least one processor for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least the second robot among the one or more robots, the revised motion plan specifying multiple poses that transform the first robot from one pose to a first final pose while avoiding a collision with the second robot or at least reducing the probability of a collision with the second robot, the first final pose positioning at least a portion of the first robot at the first target.

[0251] Example 54. The method of Example 53 further includes: A selection is made between the first motion plan and the at least revised motion plan based at least in part on a comparison of respective delay amounts associated with each of the first motion plan and the at least revised motion plan.

[0252] Example 55. The method of any one of Examples 50 to 54, further comprising: After performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, the first robot is moved.

[0253] Example 56. A processor-based system for controlling one or more robots, the processor-based system comprising: at least one processor; at least one non-transitory processor-readable medium communicatively coupled to the at least one processor and storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: Execute the method described in any one of Examples 50 to 55.

[0254] Example 57. A processor-based system for controlling one or more robots, the processor-based system comprising: at least one processor; at least one non-transitory processor-readable medium communicatively coupled to the at least one processor and storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: performing motion planning for a first robot of the one or more robots to determine a first motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the first motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose that positions at least a portion of the first robot at a first destination; performing motion planning for a first robot in an attempt to determine a second motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the second motion plan specifying a plurality of poses that transform the first robot from the first end pose to a second end pose that positions at least a portion of the first robot at a second destination; and determining whether the first robot will at least one of become stuck, delayed, or deadlocked when transitioning from the first motion plan to the second motion plan; and In response to determining that the first robot would become at least one of stuck, delayed, or deadlocked when transitioning from the first motion plan to the second motion plan, at least one remedial action is performed.

[0255] Example 58. The processor-based system of Example 57, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: Autonomously select the remedial action to take from a set of remedial actions.

[0256] Example 59. A processor-based system as described in any of Examples 57 or 58, wherein in order to perform at least one remedial action, the at least one processor performs at least one of the following remedial actions: i) performing additional motion planning to determine a revised motion plan for the first robot, the motion planning taking into account at least a second robot among the one or more robots, the revised motion plan specifying multiple poses that transform the first robot from one pose to a first end pose while avoiding a collision with the second robot or at least reducing the probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target; ii) moving the second robot out of a path of the first robot specified by at least one of the first motion plan of the first robot or the second motion plan of the first robot; iii) performing motion planning for the second robot to determine a revised motion plan for the second robot, the revised motion plan specifying multiple poses that transform the second robot out of the path of the first robot; and iv) determining a new order for a set of targets, the set of targets including the first target and at least the second target.

[0257] Example 60. A processor-based system as described in any of Examples 57 or 58, wherein to perform remedial action, the at least one processor performs additional motion planning for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least the second robot of the one or more robots, the revised motion plan specifying multiple poses that transform the first robot from one pose to a first end pose while avoiding a collision with the second robot or at least reducing the probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target.

[0258] Example 61. The processor-based system of Example 60, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: A selection is made between the first motion plan and the at least revised motion plan based at least in part on a comparison of respective delay amounts associated with each of the first motion plan and the at least revised motion plan.

[0259] Example 62. The processor-based system of any one of Examples 57 to 61, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: After performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, the first robot is moved.

[0260] The foregoing detailed description has described various embodiments of the device and / or process by using block diagrams, schematic diagrams, and examples. As long as these block diagrams, schematic diagrams, and examples contain one or more functions and / or operations, those skilled in the art will understand that each function and / or operation in these block diagrams, flow charts, or examples can be implemented individually and / or collectively by a wide range of hardware, software, firmware, or any combination thereof. In one embodiment, the subject matter can be implemented via Boolean circuits, application specific integrated circuits (ASICs), and / or FPGAs. However, those skilled in the art will recognize that all or part of the embodiments disclosed herein can be implemented in various different embodiments in standard integrated circuits, such as one or more computer programs running on one or more computers (e.g., one or more programs running on one or more computer systems), as one or more programs running on one or more controllers (e.g., microcontrollers), as one or more programs running on one or more processors (e.g., microprocessors), as firmware, or almost any combination thereof, and according to the present disclosure, designing circuits and / or writing codes for software and / or firmware will be completely within the skill of those skilled in the art.

[0261] Those skilled in the art will recognize that many of the methods or algorithms set forth herein may employ additional acts, may omit certain acts, and / or may perform acts in an order different from that specified.

[0262] Furthermore, those skilled in the art will appreciate that the mechanisms taught herein can be implemented in hardware, such as in one or more FPGAs or ASICs.

[0263] The various embodiments described above can be combined to provide further embodiments. All commonly assigned U.S. patent application publications, U.S. patent applications, foreign patents, and foreign patent applications referred to in this specification and / or listed in the Application Data Sheet, including, but not limited to, International Patent Application No. PCT / US2017 / 036880, filed on June 9, 2017, entitled “MOTION PLANNING FOR AUTONOMOUS VEHICLES AND RECONFIGURABLE MOTION PLANNING PROCESSORS”; International Patent Application Publication No. WO 2016 / 122840, filed on January 5, 2016, entitled “SPECIALIZED ROBOT MOTION PLANNING HARDWARE AND METHODS OF MAKING AND USING SAME”; U.S. Patent Application No. 666, filed on January 12, 2018, entitled “APPARATUS, METHOD AND ARTICLE TO FACILITATE MOTION PLANNING OF AN AUTONOMOUS VEHICLE IN AN ENVIRONMENT HAVING DYNAMIC OBJECTS”. 62 / 616,783, filed on February 6, 2018; U.S. Patent Application No. 62 / 626,939, filed on February 6, 2018, entitled “MOTION PLANNING FROM A ROBOT STORING A DISCRETIZED ENVIRONMENT ON ONE OR MORE PROCESSORS AND IMPROVED OPERATION OF SAME”; U.S. Patent Application No. 62 / 856,548, filed on June 3, 2019, entitled “APPARATUS, METHODS AND ARTICLES TO FACILITATE MOTION PLANNING INENVIRONMENTS HAVING DYNAMIC OBSTACLES”; U.S. Patent Application No. 62 / 856,548, filed on June 24, 2019, entitled “MOTION PLANNING FOR MULTIPLE ROBOTS IN SHARED WORKSPACE”62 / 865,431; International Patent Application No. PCT / US2019 / 016700, filed on February 5, 2019, entitled “MOTION PLANNING OF A ROBOTSTORING A DISCRETIZED ENVIRONMENT ON ONE OR MORE PROCESSORS AND IMPROVEDOPERATION OF SAME,” and published as WO 2019 / 156984; International Patent Application No. PCT / US2020 / 034551, filed on May 26, 2020, entitled “APPARATUS, METHODS AND ARTICLES TO FACILITATE MOTIONPLANNING ENVIRONMENTS HAVING DYNAMIC OBSTACLES,” and published as WO 2020 / 247207; International Patent Application No. PCT / US2020 / 034551, filed on June 23, 2020, entitled “MOTION PLANNING FORMULTIPLE ROBOTS IN SHARED SYSTEMS” and published as WO 2020 / 247207 WORKSPACE” and published as WO 2020 / 263861; International Patent Application No. PCT / US2020 / 047429, filed on August 21, 2020, entitled “MOTION PLANNING FOR ROBOTS TOOPTIMIZE VELOCITY WHILE MAINTAINING LIMITS ON ACCELERATION AND JERK” and published as WO2021041223; U.S. Patent Application No. 17 / 153,662, filed on January 20, 2021, entitled “CONFIGURATION OF ROBOTS IN MULTI-ROBOT OPERATIONAL ENVIRONMENT” and published as US 2021 / 0220994; and International Patent Application No. PCT / US2020 / 039193, filed on August 21, 2020, entitled “MOTION PLANNING FOR ROBOTS TOOPTIMIZE VELOCITY WHILE MAINTAINING LIMITS ON ACCELERATION AND JERK” and published as WO2021041223; U.S. Patent Application No. 17 / 153,662, filed on January 20, 2021, entitled “CONFIGURATION OF ROBOTS IN MULTI-ROBOT OPERATIONAL ENVIRONMENT” and published as US 2021 / 0220994; and WORKSPACE EMPLOYING STAGING POSES” U.S. Application No.63 / 318,933; all of which are incorporated herein by reference in their entirety. These and other changes can be made to the embodiments in light of the above detailed description. In general, in the appended claims, the terms used should not be construed to limit the claims to the specific embodiments disclosed in the specification and claims, but should be construed to include all possible embodiments and the full scope of equivalents to the claims. Therefore, the claims are not limited by the present disclosure. .

Claims

1. A method for controlling the operation of one or more robots in a multi-robot environment by a processor-based system, the method comprising: For a first robot of the one or more robots, performing motion planning for the first robot by at least one processor to determine a first motion plan for the first robot for a first destination, the motion planning taking into account at least a second robot of the one or more robots, the first motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose that positions at least a portion of the first robot at the first destination; for a second destination, performing motion planning by at least one processor for the first robot to attempt to determine a second motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the second motion plan specifying a plurality of poses that transform the first robot from the first end pose to a second end pose that positions at least a portion of the first robot at the second destination; and After performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, the first robot is moved.

2. The method of claim 1 , wherein performing motion planning for the first robot comprises determining whether movement of the first robot along a trajectory will result in a collision with the second robot, or whether a probability of causing a collision with the second robot exceeds a threshold probability.

3. The method according to claim 2, further comprising: In response to determining that the trajectory of the first robot will result in a collision with the second robot or that the probability of resulting in a collision with the second robot exceeds a threshold probability, the second robot is moved out of the path of the first robot.

4. The method according to claim 2, further comprising: In response to determining that the trajectory of the first robot will result in a collision with the second robot or that the probability of resulting in a collision with the second robot exceeds a threshold probability, the second robot is moved out of a path of the first robot specified by at least one of a first motion plan of the first robot or a second motion plan of the first robot.

5. The method according to claim 2, further comprising: In response to determining that the trajectory of the first robot will result in a collision with the second robot or that the probability of a collision with the second robot exceeds a threshold probability, motion planning is performed by at least one processor for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose while avoiding a collision with the second robot or at least reducing the probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target.

6. The method according to claim 5, further comprising: The first robot is caused to move according to the revised motion plan of the first robot.

7. The method of claim 1 , wherein performing motion planning for the first robot comprises determining whether the first robot moving along a first trajectory will result in a collision with the second robot moving along a second trajectory, or whether a probability of causing a collision with the second robot moving along the second trajectory exceeds a threshold probability.

8. The method according to claim 7, further comprising: in response to determining that the first robot moving along a first trajectory will result in a collision with a second robot moving along a second trajectory or will result in a probability of a collision with the second robot moving along the second trajectory exceeding a threshold probability, performing motion planning by at least one processor for the second robot to determine a motion plan for the second robot, the motion plan for the second robot specifying at least one pose that transitions the second robot out of a path of the first robot; and The second robot is moved according to a second trajectory of the second robot.

9. The method according to claim 7, further comprising: In response to determining that the first robot moving along a first trajectory will cause a collision with the second robot moving along a second trajectory or that a probability of a collision with the second robot moving along the second trajectory exceeds a threshold probability, motion planning is performed by at least one processor for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least a second trajectory of the second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transition the first robot from a pose to a first end pose while avoiding or at least reducing a probability of a collision with the second robot as the second robot moves along the second trajectory of the second robot, the first end pose positioning at least a portion of the first robot at the first target.

10. The method according to claim 9, further comprising: The first robot is caused to move according to the revised motion plan of the first robot.

11. The method of any one of claims 1 to 10, wherein moving the first robot comprises: After performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, the first robot is moved to the first target.

12. The method of any one of claims 1 to 10, wherein moving the first robot comprises: After moving the first robot to the first destination, the first robot is moved to the second destination.

13. The method according to any one of claims 1 to 10, further comprising: for a third destination, performing motion planning for the first robot by at least one processor to attempt to determine a third motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the third motion plan specifying a plurality of poses that transform the first robot from the second end pose to a third end pose that positions at least a portion of the first robot at the third destination, Wherein causing the first robot to move occurs after performing motion planning for the first robot to attempt to determine a third motion plan for the first robot.

14. The method of any one of claims 1 to 10, wherein performing motion planning for the first robot comprises: representing at least a second robot as at least one obstacle; and Collision detection is performed, by at least one processor, on at least one movement of at least a portion of the first robot relative to a representation of at least one obstacle.

15. The method of any one of claims 1 to 10, wherein performing motion planning for the first robot comprises: representing a plurality of motions of at least a second robot as at least one obstacle; and Collision detection is performed, by at least one processor, on at least one movement of at least a portion of the first robot relative to a representation of at least one obstacle.

16. The method of claim 15, wherein representing a plurality of motions of at least the second robot as obstacles comprises: A set of swept volumes is used, each of the swept volumes representing a respective volume swept by the at least portion of the second robot as the at least portion of the second robot moves along a trajectory represented by a respective motion.

17. The method according to claim 16, further comprising: A set of swept volumes previously calculated at pre-runtime is received by at least one processor, each of the swept volumes representing a respective volume swept by the at least portion of the second robot as the at least portion of the second robot moves along a trajectory represented by a respective motion.

18. The method of claim 15, wherein representing the plurality of motions of the second robot as obstacles comprises: The robot's motion is represented as at least one of: an occupancy grid, a hierarchical tree, or a Euclidean distance field.

19. The method according to any one of claims 1 to 10, further comprising: For each of at least the first robot and the second robot, a corresponding motion planning graph is generated by at least one processor, each motion planning graph includes a plurality of nodes and edges, the nodes representing the corresponding states of the corresponding first robot and the second robot, and the edges representing the valid transitions between the corresponding states represented by the corresponding nodes of the corresponding node pairs connected by the corresponding edges.

20. The method of any one of claims 1 to 10, wherein performing motion planning for the first robot to determine a first motion plan for the first robot, and performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, both occur during runtime of at least one of the one or more robots.

21. The method according to any one of claims 1 to 10, further comprising: Responsive to determining that the trajectory of the first robot will result in a collision with the second robot or that the probability of a collision with the second robot exceeds a threshold probability, taking remedial action by the at least one processor.

22. The method according to claim 21, further comprising: A remedial action to be taken is autonomously selected by the at least one processor from a set of two or more defined remedial actions.

23. The method of any one of claims 21 or 22, wherein taking remedial action by the at least one processor comprises taking at least one of the following remedial actions: i) performing motion planning by the at least one processor for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose while avoiding a collision with the second robot or at least reducing a probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target; ii) moving the second robot out of a path of the first robot specified by at least one of the first motion plan of the first robot or the second motion plan of the first robot; iii) performing motion planning for the second robot by the at least one processor to determine a revised motion plan for the second robot, the revised motion plan specifying a plurality of poses that transition the second robot out of a path of the first robot; and iv) determining a new order of a set of targets, said set of targets comprising said first target and at least said second target.

24. The method according to any one of claims 1 to 10, further comprising: A motion plan is autonomously selected, by the at least one processor, from a set of two or more candidate motion plans based at least in part on total costs of the two or more motion plans between corresponding ones of the two or more objectives.

25. A processor-based system for controlling one or more robots, the system comprising: at least one processor; at least one non-transitory processor-readable medium communicatively coupled to the at least one processor and storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: A method according to any one of claims 1 to 24 is performed.

26. A processor-based system for controlling one or more robots, the system comprising: The first robot in a robotic environment; at least a second robot in the robotic environment; at least one processor; as well as at least one non-transitory processor-readable medium communicatively coupled to the at least one processor and storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: For a first robot of the one or more robots, performing motion planning for the first robot to determine a first motion plan for the first robot for a first destination, wherein the motion planning takes into account at least a second robot of the one or more robots, the first motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose that positions at least a portion of the first robot at the first destination; for a second destination, performing motion planning for a first robot to attempt to determine a second motion plan for the first robot, wherein the motion planning takes into account at least a second robot of the one or more robots, the second motion plan specifying a plurality of poses that transform the first robot from the first end pose to a second end pose that positions at least a portion of the first robot at the second destination; and After performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, the first robot is moved.

27. The processor-based system of claim 26, wherein to perform motion planning for the first robot, the at least one processor determines whether movement of the first robot along a trajectory will result in a collision with the second robot, or whether a probability of resulting in a collision with the second robot exceeds a threshold probability.

28. The processor-based system of claim 27, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: In response to determining that the trajectory of the first robot will result in a collision with the second robot or that the probability of a collision with the second robot exceeds a threshold probability, the second robot is moved out of the path of the first robot.

29. The processor-based system of claim 27, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: In response to determining that the trajectory of the first robot will result in a collision with the second robot or that the probability of resulting in a collision with the second robot exceeds a threshold probability, the second robot is moved out of a path of the first robot specified by at least one of a first motion plan of the first robot or a second motion plan of the first robot.

30. The processor-based system of claim 27, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: In response to determining that the trajectory of the first robot will result in a collision with the second robot or that the probability of a collision with the second robot exceeds a threshold probability, motion planning is performed for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose while avoiding a collision with the second robot or at least reducing the probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target.

31. The processor-based system of claim 30, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: The first robot is caused to move according to the revised motion plan of the first robot.

32. The processor-based system of claim 26, wherein to perform motion planning for the first robot, the at least one processor determines whether the first robot moving along a first trajectory will result in a collision with the second robot moving along a second trajectory, or whether a probability of causing a collision with the second robot moving along the second trajectory exceeds a threshold probability.

33. The processor-based system of claim 32, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: in response to determining that the first robot moving along a first trajectory will result in a collision with a second robot moving along a second trajectory or will result in a collision with the second robot moving along a second trajectory with a probability exceeding a threshold probability, performing motion planning for the second robot to determine a motion plan for the second robot, the motion plan for the second robot specifying at least one pose that transitions the second robot out of a path of the first robot; and The second robot is moved according to a second trajectory of the second robot.

34. The processor-based system of claim 32, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: In response to determining that the first robot moving along a first trajectory will result in a collision with a second robot moving along a second trajectory or that a probability of a collision with the second robot moving along the second trajectory exceeds a threshold probability, motion planning is performed for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least a second trajectory of a second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transition the first robot from a pose to a first end pose while avoiding or at least reducing a probability of a collision with the second robot as the second robot moves along the second trajectory of the second robot, the first end pose positioning at least a portion of the first robot at the first target.

35. The processor-based system of claim 34, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: The first robot is caused to move according to the revised motion plan of the first robot.

36. The processor-based system of any one of claims 26 to 35, wherein to cause the first robot to move, after performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, the at least one processor causes the first robot to move to the first target.

37. The processor-based system of any one of claims 26 to 34, wherein to move the first robot, the at least one processor causes the first robot to move to the second target after causing the first robot to move to the first target.

38. The processor-based system of any one of claims 26 to 35, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: for a third destination, performing motion planning for the first robot to attempt to determine a third motion plan for the first robot, wherein the motion planning takes into account at least a second robot of the one or more robots, the third motion plan specifying a plurality of poses that transform the first robot from the second end pose to a third end pose that positions at least a portion of the first robot at the third destination, Wherein causing the first robot to move occurs after performing motion planning for the first robot to attempt to determine a third motion plan for the first robot.

39. The processor-based system of any one of claims 26 to 35, wherein to perform motion planning for the first robot, the at least one processor: representing at least a second robot as at least one obstacle; and Collision detection is performed on at least one movement of at least a portion of the first robot relative to a representation of at least one obstacle.

40. The processor-based system of any one of claims 26 to 35, wherein to perform motion planning for the first robot, the at least one processor: representing a plurality of motions of at least a second robot as at least one obstacle; and Collision detection is performed on at least one movement of at least a portion of the first robot relative to a representation of at least one obstacle.

41. The processor-based system of claim 40, wherein to represent multiple motions of at least a second robot as obstacles, the at least one processor uses a set of swept volumes, each of the swept volumes representing a respective volume swept by the at least portion of the second robot as the at least portion of the second robot moves along a trajectory represented by a respective motion.

42. The processor-based system of claim 41, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: A set of swept volumes previously calculated at pre-run time is received, each of the swept volumes representing a respective volume swept by the at least portion of the second robot as the at least portion of the second robot moves along a trajectory represented by a respective motion.

43. The processor-based system of claim 40, wherein to represent the plurality of motions of the second robot as obstacles, the at least one processor: represents the motions of the robot as at least one of: an occupancy grid, a hierarchical tree, or a Euclidean distance field.

44. The processor-based system of any one of claims 26 to 35, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: For each of at least the first robot and the second robot, a corresponding motion planning graph is generated, each motion planning graph includes a plurality of nodes and edges, the nodes representing the corresponding states of the corresponding first robot and the second robot, and the edges representing the valid transitions between the corresponding states represented by the corresponding nodes of the corresponding node pairs connected by the edges.

45. The processor-based system of any one of claims 26 to 35, wherein performing motion planning for the first robot to determine a first motion plan for the first robot, and performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, both occur during runtime of at least one of the one or more robots.

46. ​​The processor-based system of any one of claims 26 to 36, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: Responsive to determining that the trajectory of the first robot will result in a collision with the second robot or that a probability of a collision with the second robot exceeds a threshold probability, taking remedial action.

47. The processor-based system of claim 46, wherein the processor-executable instructions, when executed by at least one processor, cause the at least one processor to further: Autonomously select a remedial action to take from a set of two or more defined remedial actions.

48. The processor-based system of any one of claims 46 or 47, wherein in order to take remedial action, the at least one processor causes at least one of the following remedial actions to occur: i) performing motion planning by the at least one processor for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose while avoiding or at least reducing the probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target; ii) causing the second robot to move out of a path of the first robot specified by at least one of the first motion plan of the first robot or the second motion plan of the first robot; iii) performing motion planning for the second robot to determine a revised motion plan for the second robot, the revised motion plan specifying a plurality of poses that transition the second robot out of a path of the first robot; and iv) determining a new order of a set of targets, said set of targets comprising said first target and at least said second target.

49. The processor-based system of any one of claims 26 to 36, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: A motion plan is autonomously selected from a set of two or more candidate motion plans based at least in part on a total cost of the two or more motion plans between corresponding ones of the two or more objectives.

50. A method for controlling the operation of one or more robots in a multi-robot environment by a processor-based system, the method comprising: performing motion planning, by at least one processor, for a first robot of the one or more robots to determine a first motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the first motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose that positions at least a portion of the first robot at a first destination; performing motion planning for a first robot by at least one processor in an attempt to determine a second motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the second motion plan specifying a plurality of poses that transform the first robot from the first end pose to a second end pose that positions at least a portion of the first robot at a second destination; determining whether the first robot will become at least one of stuck, delayed, or deadlocked when transitioning from the first motion plan to the second motion plan; and In response to determining that the first robot will become at least one of stuck, delayed, or deadlocked when transitioning from the first motion plan to the second motion plan, at least one remedial action is performed by the at least one processor.

51. The method of claim 50, further comprising: A remedial action to be taken is autonomously selected by the at least one processor from a set of remedial actions.

52. The method of any one of claims 50 or 51, wherein performing remedial action by the at least one processor comprises performing at least one of the following remedial actions: i) performing motion planning by the at least one processor for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose while avoiding a collision with the second robot or at least reducing a probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target; ii) moving the second robot out of a path of the first robot specified by at least one of the first motion plan of the first robot or the second motion plan of the first robot; iii) performing motion planning for the second robot by the at least one processor to determine a revised motion plan for the second robot, the revised motion plan specifying a plurality of poses that transition the second robot out of a path of the first robot; and iv) determining a new order of a set of targets, said set of targets comprising said first target and at least said second target.

53. The method of any one of claims 50 or 51, wherein performing remedial action by the at least one processor comprises: Additional motion planning is performed by the at least one processor for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least the second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose while avoiding a collision with the second robot or at least reducing a probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target.

54. The method of claim 53, further comprising: A selection is made between the first motion plan and the at least revised motion plan based at least in part on a comparison of respective delay amounts associated with each of the first motion plan and the at least revised motion plan.

55. The method according to any one of claims 50 to 54, further comprising: After performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, the first robot is moved.

56. A processor-based system for controlling one or more robots, the processor-based system comprising: at least one processor; at least one non-transitory processor-readable medium communicatively coupled to the at least one processor and storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: Perform a method according to any one of claims 50 to 55.

57. A processor-based system for controlling one or more robots, the processor-based system comprising: at least one processor; at least one non-transitory processor-readable medium communicatively coupled to the at least one processor and storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: performing motion planning for a first robot of the one or more robots to determine a first motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the first motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose that positions at least a portion of the first robot at a first destination; performing motion planning for a first robot in an attempt to determine a second motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the second motion plan specifying a plurality of poses that transform the first robot from the first end pose to a second end pose that positions at least a portion of the first robot at a second destination; and determining whether the first robot will become at least one of stuck, delayed, or deadlocked when transitioning from the first motion plan to the second motion plan; and In response to determining that the first robot would become at least one of stuck, delayed, or deadlocked when transitioning from the first motion plan to the second motion plan, at least one remedial action is performed.

58. The processor-based system of claim 57, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: Autonomously select the remedial action to take from a set of remedial actions.

59. The processor-based system of any one of claims 57 or 58, wherein to perform at least one remedial action, the at least one processor performs at least one of the following remedial actions: i) performing additional motion planning to determine a revised motion plan for the first robot, the motion planning taking into account at least a second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose while avoiding or at least reducing the probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target; ii) moving the second robot out of a path of the first robot specified by at least one of the first motion plan of the first robot or the second motion plan of the first robot; iii) performing motion planning for the second robot to determine a revised motion plan for the second robot, the revised motion plan specifying a plurality of poses that transition the second robot out of a path of the first robot; and iv) determining a new order of a set of targets, said set of targets comprising said first target and at least said second target.

60. The processor-based system of any one of claims 57 or 58, wherein to perform remedial action, the at least one processor performs additional motion planning for the first robot to determine a revised motion plan for the first robot, the motion planning taking into account at least the second robot of the one or more robots, the revised motion plan specifying a plurality of poses that transform the first robot from a pose to a first end pose while avoiding a collision with the second robot or at least reducing a probability of a collision with the second robot, the first end pose positioning at least a portion of the first robot at the first target.

61. The processor-based system of claim 60, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: A selection is made between the first motion plan and the at least revised motion plan based at least in part on a comparison of respective delay amounts associated with each of the first motion plan and the at least revised motion plan.

62. The processor-based system of any one of claims 57 to 61, wherein the processor-executable instructions, when executed by the at least one processor, cause the at least one processor to further: After performing motion planning for the first robot to attempt to determine a second motion plan for the first robot, the first robot is moved.

Images