Robot System Simulation Engine

Abstract

To provide techniques for using robotic system simulation to control a robotic system.SOLUTION: In various embodiments, a communication indicating an action to be performed by a robotic element is received from a robotic control system. The performance of the action by the robotic element is simulated. State tracking data is updated to reflect a virtual change in one or more state variables as a result of the simulated performance of the action. Successful completion of the action by the robotic element is reported to the robotic control system.SELECTED DRAWING: Figure 4B

Description

[Technical Field]

[0001] CROSS-REFERENCE TO OTHER APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 62 / 926,172, "ROBOTIC SYSTEM SIMULATION ENGINE," filed October 25, 2019, which is incorporated herein by reference for all purposes. [Background technology]

[0002] Robotic kitting systems have been provided for assembling kits of similar and / or dissimilar items. Items to be included are picked from a source location (such as a bin, shelf, or other container) and placed in a destination container (such as a box, bin, tray, or tote).

[0003] To achieve a required throughput, in some contexts it may be necessary or desirable to use multiple robots working cooperatively to perform a set of tasks, such as assembling many and / or various kits. When multiple robots are utilized, planning and control must achieve goals such as avoiding collisions or interference, and optimizing utilization of the robots to achieve a desired throughput or other performance measure.

[0004] Algorithms for planning and / or scheduling robots to perform actions and tasks toward achieving an overall goal may rely on simplifying assumptions to enable plans to be generated in a time that is computationally and / or operationally practical. [Brief explanation of the drawings]

[0005] Various embodiments of the present invention are disclosed in the following detailed description and the accompanying drawings.

[0006] [Figure 1]FIG. 1 is a block diagram illustrating an embodiment of a robotic system for picking and placing items from a source container to a destination container.

[0007] [Figure 2] FIG. 1 illustrates an embodiment of a robot system.

[0008] [Figure 3] FIG. 1 is a block diagram illustrating one embodiment of a robotic system and workspace.

[0009] [Figure 4A] FIG. 1 is a block diagram illustrating an embodiment of a robot control system.

[0010] [Figure 4B] FIG. 1 is a block diagram illustrating one embodiment of a robotic system simulation engine.

[0011] [Figure 5A] FIG. 1 is a block diagram illustrating one embodiment of a robotic system simulation engine.

[0012] [Figure 5B] FIG. 1 is a functional flow diagram illustrating one embodiment of a process for simulating a robotic system.

[0013] [Figure 6] 1 is a flow chart illustrating one embodiment of a process for controlling a robotic system.

[0014] [Figure 7] 1 is a flow chart illustrating one embodiment of a process for performing a simulation to verify and / or refine a plan.

[0015] [Figure 8] 1 is a flow chart illustrating one embodiment of a process for controlling a robotic system using a plan.

[0016] [Figure 9A] FIG. 1 illustrates an example of optimizing across multiple physical locations in one embodiment of a robotic system simulation engine.

[0017] [Figure 9B] 1 is a flow chart illustrating one embodiment of a process for optimizing across multiple physical locations. DETAILED DESCRIPTION OF THE INVENTION

[0018] The present invention may be embodied in various forms, including as a process, an apparatus, a system, a composition of matter, a computer program product embodied on a computer-readable storage medium, and / or a processor configured to execute instructions stored in and / or provided by a memory coupled to the processor. These embodiments, or any other form the present invention may take, may be referred to herein as technology. In general, the order of steps in a disclosed process may be varied within the scope of the present invention. Unless otherwise noted, components, such as a processor or memory, described as configured to perform a task may be implemented as general components temporarily configured to perform the task at a given time, or as specific components manufactured to perform the task. As used herein, the term “processor” refers to one or more devices, circuits, and / or processing cores configured to process data, such as computer program instructions.

[0019] The following is a detailed description of one or more embodiments of the present invention with reference to figures that illustrate the principles of the invention. While the present invention has been described in connection with such embodiments, it is not limited to any particular embodiment. The scope of the present invention is limited only by the claims, and the present invention includes many alternatives, modifications, and equivalents. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. These details are for the purpose of example, and the present invention may be practiced according to the claims without some or all of these specific details. For simplicity, technical matters that are well known in the art related to the present invention have not been described in detail so as not to unnecessarily obscure the present invention.

[0020] A robotic system simulation engine is disclosed. The simulation engine models a system based on the system's attributes, physical dimensions, constraints, capabilities, performance characteristics, etc. In various embodiments, the robotic system simulation engine disclosed herein is used to determine and adjust utilization of a robotic system to perform kitting operations. In some embodiments, multiple plans may be simulated to determine an optimal plan to be implemented. In some embodiments, a plan programmatically generated, for example, by a scheduler or other system, module, or component using a scheduling algorithm that relies at least in part on one or more simplifying assumptions, is verified and / or refined by simulating the operation of the robotic system according to the plan.

[0021] In some embodiments, the results of the simulation may be used to provide displays such as video sequences, to visually demonstrate how operations are (or can / could be) performed, to demonstrate that desired results, throughput, or other performance measures can / are / will be achieved, etc.

[0022] In various embodiments, the robotic system simulation engine disclosed herein models a multi-robot system. The simulation engine generates simulation results that are used in various embodiments to plan and execute robotic operations in which multiple robots are used to perform a series of kitting operations, such as at a desired throughput.

[0023] In various embodiments, process simulations are used to ensure that a robotic facility / system meets throughput requirements and / or to help optimize robotic picking and movement patterns to maximize throughput. In some embodiments, process simulations are generated and / or regenerated / updated, for example, as the environment changes, unexpected events occur, etc., to generate simulation results that enable the robotic system to verify that the expected and / or desired results achieved by the system are achieved, and / or to test alternative scenarios or plans for performing a series of tasks and select an optimal or relatively optimal solution.

[0024] In some embodiments, the kitting process simulation engine disclosed herein may be used to provide visual displays (e.g., video sequences) for the purpose of demonstrating to a human user that desired throughput, goals, etc. can / will be achieved.

[0025] 1-3 illustrate examples of robotic systems and associated workspaces in which process simulation is used as disclosed herein to ensure that the robotic equipment / system meets throughput and / or other requirements and / or to help optimize robotic picking and movement patterns for maximizing throughput, collision avoidance, etc. While specific environments are shown in FIGS. 1-3, the techniques disclosed herein may be used in other robotic workspaces, environments, and contexts as well, in various embodiments.

[0026] 1 is a block diagram illustrating one embodiment of a robotic system for picking and placing items from a source container to a destination container. In the illustrated example, system 100 includes a robotic arm 102 rotatably mounted on a carriage 104 configured for translational movement along rails 106. For example, carriage 104 may include a computer-controlled mechanical and / or electromechanical drive mechanism configured for use under robot / computer control to move carriage 104 along rails 106, e.g., to reposition robotic arm 102 to a desired location. In this example, robotic arm 102 is movably mounted on carriage 104 and rails 106; however, in various other embodiments, robotic arm 102 may be fixed or may be fully or partially movable other than translational movement along a rail, e.g., mounted on a carousel, fully movable on a motor-driven chassis, etc.

[0027] In the illustrated example, the robotic arm 102 has an end effector 108 at its working distal end (the end farthest from the carriage 104). The end effector 108 includes a compliant vacuum (or "suction") cup 110. In various embodiments, the suction cup 110 comprises silicone or another natural or synthetic material that is durable yet compliant enough to at least slightly "give in" when the robotic system 100 makes contact (initially and / or gradually) with an item that it is attempting to grasp using the robotic arm 102 and end effector 108, e.g., by suction (such as a loaf of bread or other soft and / or fragile item in a non-rigid package like a plastic bag or wrapping).

[0028] In this example, the end effector 108 has a camera 112 mounted to the side of the end effector 108. In other embodiments, the camera 112 may be more centrally located, such as on a downward-facing surface of the body of the end effector 108 (in the position and orientation shown in FIG. 1 ). Additional cameras may be mounted elsewhere on the robotic arm 102 and / or end effector 108 (e.g., on the arm segments that comprise the robotic arm 102). Additionally, cameras 114 and 116, which are wall-mounted in this example, provide additional image data that can be used to construct a 3D view of the scene in which the system 100 is positioned and configured to operate.

[0029] In various embodiments, the robotic arm 102 is used to position the suction cup 110 of the end effector 108 over the item to be picked up, as shown, and a vacuum source provides a suction force to grasp the item, lift the item from its source location, and place the item in its destination location.

[0030] In the example shown in FIG. 1 , the robotic arm 102 is configured to be used to pick any items, which in this example are heterogeneous, such as bread items picked from a tray of items received from a bakery, from a source tray 120 and place those items in a destination tray. In the illustrated example, the destination tray 118 and the source tray 120 comprise stackable trays configured to be stacked on a base with wheels on each corner. In some embodiments, the trays 118 and / or 120 may be pushed into position by a human worker. In some embodiments, the trays 118 and / or 120 may be stacked on a motorized and / or robotically controlled base configured to be used, for example, to move a bin stack into a predetermined position to be picked and / or placed, and / or to move a completed stack of trays to a staging area and / or transport area for loading into a delivery vehicle for delivery to a retail store. In some embodiments, other robots not shown in FIG. 1 may be used to push trays 118 and / or 120 into position for loading / unloading and / or onto a truck or other destination for transport, etc.

[0031] In various embodiments, 3D or other image data generated by one or more of cameras 112, 114, and 116 may be used to generate a 3D view of the work area of ​​system 100 and items within the work area. The 3D image data may be used to identify items to be picked / placed, such as by color, shape, or other attributes. In various embodiments, one or more of cameras 112, 114, and 116 may be used to read text, logos, photographs, drawings, images, marks, barcodes, QR codes, or other encoded and / or graphical information or content visible on and / or comprising items within the work area of ​​system 100.

[0032] 1 , in the depicted example, system 100 includes a control computer 122 configured to communicate with elements such as robotic arm 102, carriage 104, effector 108, and sensors (such as cameras 112, 114, and 116, and / or weight, force, and / or other sensors not shown in FIG. 1), in this example via wireless communication (although in various embodiments, via one or both of wired and wireless communication). In various embodiments, control computer 122 is configured to use input from the sensors (such as cameras 112, 114, and 116, and / or weight, force, and / or other sensors not shown in FIG. 1) to observe, identify, and determine one or more attributes of items being loaded and / or unloaded from tray 120 to tray 118. In various embodiments, control computer 122 identifies the item and / or its attributes using item model data in a library stored in control computer 122 and / or accessible to control computer 122, for example, based on image and / or other sensor data. Control computer 122 uses the model corresponding to the item to determine and implement a plan for stacking the item, along with other items, in / on a destination (e.g., tray 118). In various embodiments, the item attributes and / or model are utilized to determine a strategy for grasping, moving, and placing the item at a destination location (e.g., a location where the item has been determined to be placed as part of the planning / re-planning process for stacking the item in / on tray 118).

[0033] In the illustrated example, control computer 122 is connected to an “on-demand” teleoperator 124. In some embodiments, if control computer 122 is unable to continue in fully automated mode, e.g., if a strategy for grasping, moving, and placing an item becomes indeterminable and / or fails such that control computer 122 does not have a strategy for completing the pick and place of an item in fully automated mode, control computer 122 instructs human user 126 to intervene, e.g., by manipulating one or more of robotic arm 102, carriage 104, and / or end effector 108 using teleoperator 124 to grasp, move, and place an item.

[0034] 2 illustrates one embodiment of a robotic system. In various embodiments, the robotic system 200 uses the robotic system simulation engine disclosed herein to generate, evaluate, and refine plans to fulfill operational requests, such as picking and placing various items from source containers to respective destination containers according to one or more sets of manifests, orders, invoices, etc.

[0035] In the depicted example, system 200 includes robotic arms 208 and 210 mounted for computer-controlled movement along rails 204 and 206, respectively. Robotic arms 208 and 210 terminate in suction-type end effectors 212 and 214, respectively. In various embodiments, robotic arms 208 and 210 and end effectors 212 and 214 are controlled by a robotic control system including a control computer (such as control computer 122 in FIG. 1 ).

[0036] 2, robotic arms 208 and 210 and end effectors 212 and 214 are used to move items, such as bread, from a source tray on wheeled base 202 to a destination tray on wheeled bases 216 and 218. In various embodiments, a human and / or robot may position the source tray and wheeled base 202 at a starting location between rails 204 and 206, as shown. Wheeled base 202 may be advanced through a path formed by rails 204 and 206, for example, in the direction of the arrow beginning at the far end of base 202, as shown. In various embodiments, base 202 may be advanced using one or both of robotic arms 208 and 210, by being pushed manually by one or more human operators, one or more fixed or unfixed robots, etc., by conveyor belt or chain type mechanisms operating along and / or between rails 204 and 206, by robotically controlled propulsion and / or transport mechanisms (such as computer-controlled motorized wheels and brakes) incorporated into base 202, etc.

[0037] As wheeled base 202 advances along / between rails 204 and 206 and / or while base 202 is temporarily in a stationary position between rails 204 and 206, in various embodiments, the robotic system uses robotic arms 208 and 210 and end effectors 212 and 214 to move items (e.g., bread) from a source tray on wheeled base 202 to a destination tray on wheeled bases 216 and 218 according to a plan. For example, the plan may be calculated based on inventory and / or sensor data indicating which one or more types of bread are available in source trays on base 202 and / or other bases in the work area, and further based on a manifest or other data indicating which combinations of bread or other items are to be placed in destination trays for delivery to respective final delivery locations (e.g., retail stores).

[0038] In the illustrated example, once the destination trays on bases 216 and 218 are filled, bases 216 and 218 are moved outwardly away from rails 204 and 206, respectively, e.g., to a staging area and a loading area for loading onto a delivery vehicle.

[0039] 2 shows a single set of source trays on a single base 202, in various embodiments, one or more additional bases (each having zero or more trays filled with stacked items) may be positioned between rails 204 and 206 and / or in nearby staging areas depending on the high-level operations being performed. Similarly, in some embodiments, multiple sets of destination trays, each on a corresponding wheeled base, may be staged adjacent to rail 204 or rail 206, with the top tray of each being in the process of being filled simultaneously and / or sequentially, for example, according to a manifest and associated execution plan.

[0040] In various embodiments, once a source tray is empty, the system may use the robotic arms 208 and / or 210 to move the tray to a staging area and / or to the top of a stack of destination trays, thereby creating a supply of trays to be filled and sent to a final destination and / or exposing the next source tray from which items can be picked to fulfill an order.

[0041] While FIG. 2 shows a single robotic arm (208, 210) positioned on each rail (204, 206), in some embodiments, two or more robotic arms may be provided on each rail. In some embodiments, two robotic arms on a single rail may be used to pick up and move empty trays. For example, each arm may be used to engage opposite sides of the tray, either mechanically by engaging structures on the tray or, in various embodiments, by suction or gripping, and the two arms may be coordinated to maintain control of the empty tray while moving it to a new location (e.g., a buffer or staging area, or the top of a stack of destination trays). In some embodiments, one or more robotic arms may be used to move a full or partially full tray from the top of the stack (e.g., to be moved to a staging area) to expose trays below the stack for purposes of adding additional items, rearranging items according to a new or revised plan, etc.

[0042] Cameras 220 and 222 and / or cameras 224 and 226 mounted on end effectors 212 and 214, respectively, are used in various embodiments to generate image data for planning and executing the pick and place operations disclosed herein.

[0043] 3 is a block diagram illustrating one embodiment of a robotic system and workspace. In various embodiments, the robotic system 300 uses the robotic system simulation engine disclosed herein to generate, evaluate, and refine plans to fulfill operational requests, such as picking and placing various items from source containers to individual destination containers according to one or more sets of manifests, orders, invoices, etc.

[0044] In the illustrated example, system 300 comprises robotic arms 302 and 304 mounted for computer-controlled movement along rail 306, and robotic arms 308 and 310 mounted for computer-controlled movement along rail 312. In the illustrated example, the two rail-mounted robots arm are attached to each of rails 306 and 312, although in various embodiments, more or fewer robot arms may be attached to each rail 306, 312. Similarly, a different number of robot arms may be attached to each rail. In various embodiments, simulations may be performed as disclosed herein using different numbers, positions, and configurations of robot arms, and the placement and configuration may be selected and implemented based at least in part on the simulation results. For example, the number, placement, and configuration of robot arms may be selected to achieve the operational requirements while minimizing a cost function applicable to other simulated combinations.

[0045] In various embodiments, robotic arms 302, 304, 308, and 310 may have the same or different end effectors. For example, in some embodiments, robotic arms 302, 304, 308, and 310 are shown being picked and placed by system 300. 3The items shown in may include relatively light, fragile products (e.g., bread or other bakery products) wrapped in soft, easily deformable packaging (e.g., plastic bags). In such embodiments, each of the robotic arms 302, 304, 308, and 310 may be equipped with a suction-type end effector, allowing the robotic arms to lift and move the items by applying a vacuum to suction cups on the working surfaces or ends of the end effectors so that the plastic wrapping around the items to be picked and placed is sucked into the suction cups. In some embodiments, some of the robotic arms 302, 304, 308, and 310 may have suction-type end effectors, while others may have other types of end effectors, such as gripper-type end effectors. In various embodiments, the type and combination of end effectors may be selected based on the type of items to be picked and placed. In some embodiments, the simulations disclosed herein may be performed using different types, combinations, and locations of end effectors, and the types, combinations, and locations of end effectors may be selected based at least in part on the results of the simulation. For example, one or more types of end effectors and their respective locations (e.g., which end effectors are located on which of the robot arms 302, 304, 308, and 310) may be selected as a result of the simulation that shows the configuration that achieves the best results.

[0046] In various embodiments, system 300 of FIG. 3 operates similarly to robotic system 200 of FIG. 2 , except that multiple robotic arms 302, 304, 308, 310 are present on rails 306 and 312, respectively. In various embodiments, stacks, each including one or more pallets, trays, bins, and / or other stackable containers, travel along a central path defined by guides or tracks 314 and 316. The stacks are fed to the central path from staging locations 318, to which the stacks may be transported by one or more humans, robots, and other devices. Once in the central path defined by guides or tracks 314 and 316, the stacks / containers are advanced by operation under robotic system control of propulsion mechanisms that, in various embodiments, comprise or are otherwise associated with guides or tracks 314 and 316. For example, guides or tracks 314 and 316 may comprise conveyors, pushers, belts, gears, chains, or other electromechanical structures for advancing a stack of one or more trays or other containers along a central path defined by guides or tracks 314 and 316.

[0047] In the illustrated example, robotic arms 302, 304, 308, and 310 are used to pick items from the top trays of stacks in a central path area defined by guides or tracks 314 and 316 and place the items into the top trays of assembled stacks in output stack work areas 320 and 322. Completed stacks in areas 320 and 322 (such as stacks 326 and 328 in the illustrated example) are moved from areas 320 and 322 to other work spaces not shown in FIG. 3 (such as staging areas for transport to downstream destinations in other geographic locations).

[0048] When a tray in the picking work area between rails 306 and 312 is emptied (e.g., tray 330), the empty tray is moved to the top of the stack in output work areas 320, 322 (e.g., tray 332 in the illustrated example) using robotic arms 302, 304, 308, and 310. For example, the empty tray may be placed on top of a stack of output trays that has not reached the maximum stack height and currently has a full tray above the stack, leaving no more room to place an item.

[0049] 3 is used to assemble stacks of trays for distribution to respective downstream destinations, each tray containing a combination of one or more different types of items, and the set of trays associated with each destination collectively containing a mix of items, and for each item, containing a quantity of the item corresponding to the destination. For example, in some embodiments, system 300 may be used at a bakery or associated central distribution center to fill and send to each of multiple retail stores, each corresponding set of baked goods, in the quantity ordered by or for that store.

[0050] In various embodiments, one or more computers and / or other processing elements (not shown in FIG. 3 ) are used to process orders, invoices, manifests, etc., and to programmatically generate and implement plans, as disclosed herein, for fulfilling the requirements defined by the orders, invoices, etc., without human intervention. In various embodiments, the computer applies one or more scheduling algorithms to generate an initial plan, which is then validated and / or refined using a simulation engine, as disclosed herein. In various embodiments, the simulation engine models the behavior and operation of the robotic elements (e.g., robotic arms 302, 304, 308, and 310), along with the shape, function, and performance characteristics of the robotic elements, in light of constraints defined by the workspace, other robotic elements, and environmental factors, to simulate how the system 300 would perform under the plan initially generated by the computer.

[0051] In various embodiments, the simulation engine and / or model take into account the need for the robotic arms 302, 304, 308, 310 to avoid collisions with each other, with objects in the environment, or with other actors (e.g., human workers) in the same workspace. For example, to avoid collisions and / or to assess whether operation under the generated plan will result in (or poses too high a risk of) collision, the simulation engine may calculate, for each robotic arm 302, 304, 308, 310, corresponding turning circles (such as turning circles 334, 336, 338, and 340 in the example shown in FIG. 3 ) at successive simultaneous positions and poses of the respective robotic arms. If turning circles overlap at a given point in the simulation, in some embodiments, the system 300 flags that portion of the plan for review (e.g., review by a human user) and / or refinement, e.g., to generate a revised plan that does not result in such overlaps. In some embodiments, the turning circle may be used to generate a corresponding visual representation of such circle, for example, in an animation or other visual representation of the simulation.

[0052] 4A is a block diagram illustrating one embodiment of a robotic control system. In various embodiments, system 400 of FIG. 4 is used to control robotic elements within a physical workspace to fulfill a set of requirements, such as invoices, orders, manifests, etc., associated with items to be delivered to their respective downstream destinations. In some embodiments, system 400 may be used to control robotic elements (such as robotic arms 302, 304, 308, and 310 of FIG. 3).

[0053] In the depicted example, system 400 includes a scheduler 402 configured to receive a set of orders from an order management system 404. Scheduler 402 may be implemented as a set of software components and / or processes configured to use order information from order management system 404 to generate, validate, and refine a plan for utilizing a set of robotic elements to fulfill the set of requirements defined by the order information. Order management system 404 tracks order information (e.g., which customers ordered which products in what quantities, what each customer received, etc.).

[0054] In various embodiments, scheduler 402 obtains system state from an internal state machine 416 that tracks system state (e.g., where all products / items are, how high the input and output stacks of trays or other containers are, the position and pose of the robot, etc.), obtains order information from order management system 404, and calculates and provides to dispatcher 406 a schedule for agents 408, 410, 412 to execute. For example, scheduler 402 may generate a schedule for each of multiple destinations (e.g., a set of retail stores) that indicates a set of one or more items to be delivered to that destination, along with the quantity of each item. 1 set of Orders (manifests, invoices, etc.) may be received from the order management system 404. In various embodiments, the scheduler 402 may use one or more well-known scheduling algorithms to generate an initial plan. The initial plan may be generated by making one or more simplifying assumptions.

[0055] The schedule generated by scheduler 402 is provided to and configured for use by dispatcher 406 to execute the plan / schedule by operating robotic elements via a set of agents (represented in FIG. 4A as agents 408, 410, and 412). Agents 408, 410, and 412 may include control software, drivers, and / or communication interfaces to one or more robotic control elements (e.g., robotic arms, conveyors, etc.) and / or sensors (e.g., 3D or other cameras, optical code scanners, scales, pressure sensors, contact sensors, etc.).

[0056] In various embodiments, the dispatcher 406 receives schedules (e.g., lists of actions for the robots to perform) from the scheduler 402 and communicates them (via agents 408, 410, 412) to the respective robots to perform them. The dispatcher 406 handles error handling, unexpected or erratic behavior by the robots, etc. After the scheduled actions are performed, the dispatcher 406 updates the internal state machine 416 and the order management system 404 and requests a new schedule.

[0057] The dispatcher 406 communicates with the agents 408, 410, 412 via an interface 414. The dispatcher 406 sends specific control commands to the agents 408, 410, 412 via the interface 414, and the agents 408, 410, 412 provide feedback via the same interface 414, reporting, for example, task success / failure, sensor readings (e.g., force sensors on a robotic arm), etc. The dispatcher 406 uses information from the scheduler 402 and feedback from the agents 408, 410, 412 to update and maintain an internal state machine 416.

[0058] In various embodiments, an internal state machine 416 is used to maintain a current, consistent view of the state of the elements comprising system 400 and the robotic elements controlled by system 400 (e.g., via agents 408, 410, 412), including the location and state of the items the robotic system manipulates (e.g., trays and items being picked from / placed onto trays in the example shown in FIG. 3 ). The state machine may be used to track the state and location of each tray and each order / manifest, along with, for example, the location and state (e.g., pose) of each robotic arm or other robotic element comprising the system. The internal state machine 416 may be used to track the state (e.g., open, in progress, completed) of each order and the location or other current disposition of the tray or other element relative to each other.

[0059] 4B is a block diagram illustrating one embodiment of a robotic system simulation engine. In the illustrated example, system 420 includes the same scheduler 402, order management system 404, dispatcher 406, and internal state machine 416. However, in system 420, dispatcher 406 communicates with simulator (also referred to as a "simulator engine") 422 via interface 414. Simulator 422 includes simulated agents 428, 430, and 432 corresponding to each of the real agents 408, 410, and 412 of system 400 of FIG. 4A, each of which is configured to communicate with dispatcher 406 via the same interface 414 as its "real" (non-simulated) counterpart.

[0060] In various embodiments, simulated agents 428, 430, 432 are configured to respond to commands from dispatcher 406 in the same manner as their real-world counterparts. Each simulated agent 428, 430, 432 uses a corresponding model and / or configuration file to simulate the behavior that the robotic element it represents will exhibit in response to commands from dispatcher 406, and each generates and provides a response communication to dispatcher 406 that includes information that the corresponding real-world robotic element is expected to return based on the simulated execution of actions commanded by dispatcher 406.

[0061] In the illustrated example, simulator 422 includes a state tracker 434 that is used to track the simulated state of the system. In various embodiments, simulator 422 uses state tracker 434 to track the system state independently of internal state machine 416. This state is used to estimate behavior completion times and is updated by simulated agents 428, 430, 432.

[0062] The simulated agents 428, 430, 432 are communicatively coupled to a state tracker 434 and may read state information from the state tracker 434, e.g., to estimate how long it will take the simulated agents to perform their assigned tasks / operations, and are configured to update the state tracker 434 with their simulated states (e.g., positions, trajectories, task success / failure, etc.). In various embodiments, consistency logic configuring the state tracker 434 ensures that a consistent state is maintained, e.g., by detecting and preventing inconsistent updates from two or more of the simulated agents 428, 430, 432.

[0063] FIG. 5A is a block diagram illustrating one embodiment of a robotic system simulation engine. In various embodiments, the system of FIG. 5A is used to implement the simulator 422 of FIG. 4B. In the illustrated example, the state tracker 434 of FIG. 4B comprises a set of system state data structures and associated logic 502. The data structures may include, for example, one or more tables or other data structures that are read and / or updated by the simulated agents 428, 430, 432 as actions are simulated as being performed in response to a schedule received by the dispatcher 406 and communicated to the simulated agents 428, 430, 432 for simulated execution. Individual actions, action results, and other events may be logged in a log file 504, for example, by the simulated agents 428, 430, 432. In some embodiments, each simulated agent 428, 430, 432 and / or each resource (e.g., tray or other container) that a robotic element may manipulate may be represented in the log file 504 by a separate log file or document.

[0064] In various embodiments, the simulated agents 428, 430, 432 use the model 506 to determine how long and / or in what specific and / or fine-grained manner each action will be performed by the robotic element (e.g., a robotic arm) whose behavior the simulated agents 428, 430, 432 are configured to simulate.

[0065] In various embodiments, the simulation / state logic 502 includes logic for detecting actual or structural collisions or other inconsistent and / or incompatible behavior. For example, in some embodiments, the logic 502 detects whether two or more robot arms are updated simultaneously as being in the same three-dimensional space. In some embodiments, a turning circle (or other 2D or 3D shape or volume) is calculated for each robot arm, e.g., based on each robot's current position and pose (e.g., arm segment orientation, etc.). If the turning circles (or other shapes or volumes) of two robots intersect, an exception event is triggered. In some embodiments, each simulated agent determines whether the planned action is consistent with the current system state before posting a simulated action update. If it is inconsistent, e.g., if taking the action would cause the robot to collide with another robot or object in the workspace, the simulated agent does not post the action. In some embodiments, the simulated agent may retry the action, e.g., after a predetermined delay period. If the retry fails, the simulated agent may notify the dispatcher of the failure.

[0066] 5B is a functional flow diagram illustrating one embodiment of a process for simulating a robotic system. In the illustrated example, dispatcher 406 assigns actions to simulated agents 522 to be executed. Simulated agents 522 simulate the execution of actions assigned by dispatcher 406, such as by retrieving state variables 524 from a state tracker data structure 526 and using the state information (and robot / other resource-specific models) to determine updated values ​​for each of one or more state variables to reflect the final state resulting from the simulated execution of the actions. The simulated agents provide updates 528 to consistency / state transition logic 530.

[0067] In various embodiments, consistency / state transition logic 530 validates each update 528 to ensure, for example, that there are not too many robots or other elements in an inconsistent or otherwise impermissible state (e.g., a collision between two robot arms). If the update is permissible, consistency / state transition logic 530 writes update 532 to the appropriate location in state tracker data structure 526, including any updates to global or other state variables determined by consistency / state transition logic 530 based on the update 528 received from simulated agent 522. The above process repeats as successive behavior sets are assigned to simulated agent 522 by dispatcher 406.

[0068] In various embodiments, to simulate the execution of a behavior assigned by dispatcher 406, simulated agent 522 may provide a series of updates 528, for example, at predetermined intervals and / or upon completion of the simulation of each of multiple sub-behaviors simulated as being performed by the robot or other resource that the simulated agent is configured to simulate. In this manner, intermediate states between the state of the robot or other resource at the start of the behavior and the end state at completion may be simulated, and collisions or other problems that may occur while the robot or other resource is in such intermediate states may be detected. For example, by simulating intermediate positions / states, a collision between two robot arms on opposing rails (e.g., in the example shown in FIG. 3) may be detected.

[0069] FIG. 6 is a flow chart illustrating one embodiment of a process for controlling a robotic system. In various embodiments, process 600 of FIG. 6 may be performed by one or more components, modules, or processes operating on one or more control computers (such as control computer 122 of FIG. 1 ). In the illustrated example, at step 602, a request and initial state information are received. For example, in the example shown in FIGS. 4A and 4B , at step 602, scheduler 402 may receive order information from order management system 404 and state information from internal state machine 416. At step 604, a scheduling algorithm is used to determine a plan for fulfilling the request received at step 602 in light of the state information received at step 602. At step 606, one or more simulations are performed, e.g., as disclosed herein, to verify and / or refine the plan determined at step 604. At step 608, the plan is used to control the robotic system to fulfill all or part of the request received at step 602. For example, the plan may be used to control one or more robotic arms or other robotic elements, such as by dispatcher 406 in the example shown in FIGS. 4A and 4B.

[0070] In various embodiments, continuous and / or ongoing or successive iterations of one or more of steps 602, 604, and 606 may be performed when / if necessary to generate, regenerate, verify, and / or refine the plan. For example, as the initial plan or portions thereof are executed, status and / or order information may be updated in a manner that leads to the generation, verification, and / or refinement of a new or updated plan. Continuous iterations may be repeated until all requirements are met.

[0071] Figure 7 is a flow chart illustrating one embodiment of a process for performing simulations to validate and / or refine a plan. In various embodiments, the process of Figure 7 is used to implement step 606 of Figure 6. In the illustrated example, at step 702, the process begins with an initially received plan that is expected to achieve system requirements (e.g., fulfilling a set of product orders) within predetermined constraints (e.g., within a given work day or other work period). For example, a plan generated without simulation (e.g., a plan generated by applying one or more scheduling algorithms) may initially be received.

[0072] At step 704, operation according to the initial (or current) plan is simulated, for example, as disclosed herein. At step 706, if / as long as the simulated operation is proceeding as expected (or within predetermined and / or set or configurable tolerances of what is expected according to the plan), the simulated processing continues according to the plan until it is determined to be complete at step 708, or unless / until it is determined at step 706 that the simulated operation is not proceeding as expected. For example, at step 706, an indication may be received or a determination may be made that a scheduled action cannot or has not been successfully completed as simulated. Alternatively, an unexpected event or behavior of the robot or other operating entity or resource may be detected.

[0073] If it is determined in step 706 that the simulated operation is not progressing as expected, then in step 710 the plan (schedule) is refined. For example, the scheduler may be updated (via the dispatcher) with information indicating problems that occurred in the simulated operation according to the original / previous plan. The scheduler may include logic and / or processes for generating an improved (or new) plan that avoids the problems encountered in the simulation. For example, a different robot may be assigned to perform actions, or actions may be reordered, or other actions may be performed to avoid the problems encountered and / or predicted by the simulation. 、1 The above actions Timing may be modified in other ways (e.g., delayed, sequential instead of simultaneous, etc.) Once the improved plan is generated in step 710, the simulation resumes in step 704 based on the improved plan.

[0074] Successive iterations of steps 704, 706, and 710 may be performed to refine or further refine the plan as needed, for example, when problems arise in operation as simulated.

[0075] FIG. 8 is a flowchart illustrating one embodiment of a process for controlling a robotic system using a plan. In various embodiments, the process of FIG. 8 is used to implement step 608 of FIG. 6. In the illustrated example, at step 802, robotic movements are executed according to the (current) plan. For example, in the example shown in FIG. 4A , the plan generated by scheduler 402 may be provided to dispatcher 406, which then implements the plan by assigning actions to agents 408, 410, 412 configured to control corresponding robotic elements (e.g., robotic arms, other robots) to execute the actions. At step 804, an ongoing simulation is performed to ensure that the remaining / subsequent portions of the plan result in fulfilling the requirements without collisions or other errors. For example, the actions assigned by dispatcher 406 to real agents 408, 410, 412 may also (or first) be provided to simulated agents 428, 430, 432 for simulated execution of the actions. If, at step 806, it is determined that the simulation indicates a problem with the plan (or possible and / or necessary improvement), then the plan is refined at step 808, and the refined plan begins to be used to control the operation of the robotic system at step 802 and to simulate the operation of the robotic system at step 804. Processing continues until completion (step 810) (e.g., all requirements are met).

[0076] In various embodiments, simulations may be performed for robotic and / or other operations at multiple physical locations to achieve a global goal (e.g., to minimize a global cost function). For example, FIGS. 1, 2, and 3 show examples of robotic systems used to fulfill orders, such as at a warehouse or other distribution center. In some embodiments, simulations may also be performed for one or more destinations to which items are shipped. For example, a plan may be made to assemble trays or other containers for shipment / delivery to each destination (such as a retail store or further (e.g., regional or local) distribution node), and in various embodiments, operations of the robotic system at the source location may be performed, and robotic and / or non-robotic operations (e.g., manual / human operations) at each of one or more destinations may be simulated to generate, verify, and / or refine a plan for assembling trays / kits at the distribution node (source) so as to avoid inefficiencies or conflicts at the destination node.

[0077] FIG. 9A illustrates an example of optimizing across multiple physical locations in one embodiment of a robotics system simulation engine. In the illustrated example, a distribution network 900 includes a distribution center 902 and multiple destinations (represented in FIG. 9A by destinations 904, 906, and 908) to which items are shipped from distribution center 902. In various embodiments, a first robotic system (e.g., FIGS. 1-4) is deployed at distribution center 902 to fulfill orders, e.g., for transporting respective quantities of respective items to the destinations (e.g., 904, 906, and 908). Additionally, at least some of the destinations (e.g., 904, 906, and 908) are deployed with destination robotic (or other) systems, e.g., to receive and open trays or other containers, pick and place items onto shelves, etc. In various embodiments, simulations are performed of operations at distribution center 902 and one or more of destinations 904, 906, and 908 to determine and implement a plan to fulfill demand globally (end-to-end) across multiple locations.

[0078] FIG. 9B is a flowchart illustrating one embodiment of a process for optimizing across multiple physical locations. In various embodiments, process 920 is performed using the simulations disclosed herein to meet requirements in a manner that considers goals and requirements across multiple locations. In the illustrated example, at step 922, candidate plans are generated for meeting a set of requirements through robotic system operations at a first location. For example, a scheduling algorithm may be applied to generate a set of candidate plans. Each candidate plan may be scored, ranked, or otherwise weighted relative to one another with respect to a cost function applicable to the first location (e.g., considering time, energy usage, robot utilization, other efficiency measures, etc.). One or more of the higher (highest) ranked / scored plans may be simulated to validate and / or refine the plan for the first location.

[0079] In step 924, the top n plans are simulated at and / or for each of one or more destination locations to determine a cost associated with the plan for each destination. The destination location costs associated with each plan simulated at each location are aggregated and added to the cost at the first location to generate a total cost. The plan with the lowest total cost is selected and / or further refined to determine and implement a final plan.

[0080] In various embodiments, the techniques disclosed herein may be used to generate, verify, and refine plans for controlling a robotic system to meet a set of requirements. Scheduling algorithms may be applied in a computationally feasible manner to determine plans whose execution can be verified and / or refined in real time through simulation, as disclosed herein, to anticipate and avoid real-world problems that may arise.

[0081] Although the above-described embodiments have been described in some detail for ease of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and are not intended to be limiting. The present disclosure may be realized in the following forms. [Form 1] 1. A system comprising: a communication interface; a processor connected to the communication interface; Equipped with The processor: receiving a communication from a robotic control system via the communication interface indicative of an action to be performed by the robotic element; simulating performance of the behavior by the robotic element; updating the state tracking data to reflect hypothetical changes in one or more state variables as a result of the simulated execution of said behavior; and a system configured to report successful completion of the action by the robotic element to the robotic control system via the communication interface. [Form 2] 10. The system according to claim 1, The system further comprises a memory coupled to the processor and configured to store the state tracking data. [Form 3] 10. The system according to claim 1, The behavior indicated in the communication is determined by the robot control system according to a plan generated by the robot control system for satisfying a set of requirements using a set of robotic elements including the robotic element. [Form 4] The system according to aspect 3, The robotic control system is configured to verify or refine the plan using simulation results generated based at least in part on the simulated execution of the actions by the robotic elements. [Form 5] The system according to aspect 3, wherein the communication is received from, and the report is sent to, the robotic control system via the same interface used by the robotic control system to communicate with a non-simulated agent configured to control the movement of the robotic element to perform the action in the real physical world. [Form 6] 10. The system according to claim 1, The system, wherein the processor is further configured to emulate a robotic control agent associated with the robotic element. [Form 7] 10. The system according to claim 1, The processor is configured to simulate performance of the behavior by the robotic elements, at least in part, by using a model including data representing physical configurations and dynamic behavior characteristics of the robotic elements. [Form 8] 8. The system according to claim 7, The system, wherein the model includes data indicative of a time associated with the performance of the behavior by the robotic element. [Form 9] 10. The system according to claim 1, The system wherein the robotic element comprises a robotic arm having an end effector. [Form 10] 10. The system according to claim 9, The system, wherein the actions include using the robotic arm and end effector to pick an item from a source location and place the item at a destination location. [Form 11] 11. The system of claim 10, The system wherein the source location includes a source tray or other source container and the destination location includes a destination tray or other container. [Form 12] 10. The system according to claim 1, The system, wherein the processor is configured to detect an error condition associated with the simulated performance of the behavior by the robotic element. [Form 13] 13. The system according to claim 12, The processor is configured to provide feedback data to the robotic control system indicative of the error condition. [Form 14] 14. The system according to claim 13, The robotic control system is configured to refine a plan based at least in part on the feedback data. [Form 15] 10. The system according to claim 1, The behavior is included in a first set of behaviors that the processor is configured to simulate at a first physical location, the processor is further configured to simulate a second set of behaviors at a second physical location, and the robotic control system is configured to use the simulation of the first set of behaviors and the simulation of the second set of behaviors to determine a plan for performing an operation at the first physical location. [Form 16] 1. A method comprising: receiving a communication from a robotic control system via a communication interface indicative of an action to be performed by the robotic element; simulating performance of the behavior by the robotic element; updating state tracking data to reflect hypothetical changes in one or more state variables as a result of the simulated execution of said behavior; reporting successful completion of the action by the robotic element to the robotic control system via the communication interface; A method comprising: [Form 17] 17. The method of claim 16, The method, wherein the behavior indicated in the communication is determined by the robotic control system according to a plan generated by the robotic control system for satisfying a set of requirements using a set of robotic elements including the robotic element. [Form 18] 18. The method of claim 17, the robotic control system is configured to verify or refine the plan using simulation results generated based at least in part on the simulated execution of the actions by the robotic elements. [Form 19] 18. The method of claim 17, wherein the communication is received from, and the report is sent to, the robotic control system via the same interface used by the robotic control system to communicate with a non-simulated agent configured to control movement of the robotic element to perform the action in the real physical world. [Form 20] A computer program product embodied in a non-transitory computer-readable medium, computer instructions for receiving, from a robotic control system, via a communication interface, a communication indicative of an action to be performed by the robotic element; computer instructions for simulating performance of the behavior by the robotic element; computer instructions for updating state tracking data to reflect hypothetical changes in one or more state variables as a result of the simulated execution of said behavior; computer instructions for reporting successful completion of the action by the robotic element to the robotic control system via the communication interface; A computer program product comprising:

Claims

1. 1. A system comprising: a communication interface; a processor connected to the communication interface; Equipped with The processor: receiving, via said communication interface, communications indicative of actions to be performed by the robotic elements from a robotic control system including an internal state machine, said internal state machine being used to maintain a view of the state of said robotic elements; simulating performance of the behavior by the robotic element using at least one simulated agent and a state tracker that monitors simulated states of the at least one simulated agent independently of the internal state machine, and in simulating performance of the behavior, simulating a state at the start of the behavior, a state at the completion of the behavior, and intermediate states between the start and the completion; updating one or more state variables identifying a state of the robotic element to reflect state tracking data as a result of the simulated execution of the behavior, the state tracking data being used to estimate a behavior completion time; and a system configured to report successful completion of the action by the robotic element to the robotic control system via the communication interface.

2. 10. The system of claim 1, The system further comprises a memory coupled to the processor and configured to store the state tracking data.

3. 10. The system of claim 1, The behavior indicated in the communication is determined by the robotic control system according to a plan generated by the robotic control system for satisfying a set of requirements using a set of robotic elements including the robotic element.

4. 4. The system of claim 3, The robotic control system is configured to verify or refine the plan using simulation results generated based at least in part on the simulated execution of the actions by the robotic elements.

5. 4. The system of claim 3, wherein the communication is received from, and the report is sent to, the robotic control system via the same interface used by the robotic control system to communicate with a non-simulated agent configured to control the movement of the robotic element to perform the action in the real physical world.

6. 10. The system of claim 1, The system, wherein the processor is further configured to emulate a robotic control agent associated with the robotic element.

7. 10. The system of claim 1, The processor is configured to simulate the performance of the behavior by the robotic element, at least in part, by using a model including data representing physical configuration and dynamic behavior characteristics of the robotic element, the model being used by the at least one simulated agent.

8. 8. The system of claim 7, The system, wherein the model includes data indicative of a time associated with the performance of the behavior by the robotic element.

9. 10. The system of claim 1, The system wherein the robotic element comprises a robotic arm having an end effector.

10. 10. The system of claim 9, The system, wherein the actions include using the robotic arm and end effector to pick an item from a source location and place the item at a destination location.

11. 11. The system of claim 10, The system wherein the source location includes a source tray or other source container and the destination location includes a destination tray or other container.

12. 10. The system of claim 1, The system, wherein the processor is configured to detect an error condition associated with the simulated performance of the behavior by the robotic element.

13. 13. The system of claim 12, The processor is configured to provide feedback data to the robotic control system indicative of the error condition.

14. 14. The system of claim 13, The robotic control system is configured to refine a plan based at least in part on the feedback data.

15. 10. The system of claim 1, The at least one simulated agent reads the state tracking data to estimate the time it will take for the at least one agent to perform an assigned task or operation.

16. 1. A method comprising: receiving, via a communication interface, communications indicative of actions to be performed by a robotic element from a robotic control system including an internal state machine used to maintain a view of the state of the robotic element; simulating performance of the behavior by the robotic element using at least one simulated agent and a state tracker that monitors simulated states of the at least one simulated agent independently of the internal state machine, wherein in simulating performance of the behavior, a state at the start of the behavior, a state at the completion of the behavior, and intermediate states between the start and the completion; updating one or more state variables for specifying a state of the robotic element to reflect state tracking data as a result of simulated execution of the behavior, the state tracking data being used to estimate a behavior completion time; reporting successful completion of the action by the robotic element to the robotic control system via the communication interface; A method comprising:

17. 17. The method of claim 16, The method, wherein the behavior indicated in the communication is determined by the robotic control system according to a plan generated by the robotic control system for satisfying a set of requirements using a set of robotic elements including the robotic element.

18. 18. The method of claim 17, the robotic control system is configured to verify or refine the plan using simulation results generated based at least in part on the simulated execution of the actions by the robotic elements.

19. 18. The method of claim 17, wherein the communication is received from, and the report is sent to, the robotic control system via the same interface used by the robotic control system to communicate with a non-simulated agent configured to control movement of the robotic element to perform the action in the real physical world.

20. A non-transitory computer-readable medium, comprising: computer instructions for receiving, via a communication interface, communications indicative of actions to be performed by a robotic element from a robotic control system including an internal state machine used to maintain a view of the state of the robotic element; computer instructions for simulating performance of the behavior by the robotic element using at least one simulated agent and a state tracker that monitors simulated states of the at least one simulated agent independently of the internal state machine, wherein, in simulating performance of the behavior, the computer instructions simulate a state at the start of the behavior, a state at the completion of the behavior, and intermediate states between the start and the completion; computer instructions for updating one or more state variables for identifying a state of the robotic element to reflect state tracking data as a result of simulated execution of the behavior, the state tracking data being used to estimate a behavior completion time; computer instructions for reporting successful completion of the action by the robotic element to the robotic control system via the communication interface; A non-transitory computer-readable medium having recorded thereon:

Images