ROBOT SYSTEM, METHOD FOR CONTROLLING A ROBOT, AND COMPUTER PROGRAM PRODUCT FOR CONTROLLING A ROBOT - Patent application

Abstract

A robotic system is disclosed. The system includes a communication interface that receives sensor data from one or more sensors deployed in the workspace indicative of a current state of the workspace. The system includes one or more processors that use the sensor data to estimate a state of one or both of a pallet or other container and a set of zero or more items stacked on or in the container, and that use the estimated states to generate or update a plan for controlling a robotic arm to place a next set of items on or in the pallet or other container, or to remove a next set of items from the pallet or other container, the plan including an ordered sequence of placing or removing the items. The plan is generated or updated, at least in part, by performing a bounded tree search in which a subset of possible ordered sequences is explored.

Description

CROSS-REFERENCE TO OTHER APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 211,366, entitled "WORKFLOW FOR USING TREE SEARCH BASED APPROACH FOR PLACING BOXES ON PALLET WITH LIMITED KNOWLEDGE OF FUTURE SEQUENCE," filed June 16, 2021, which is incorporated herein by reference for all purposes. [Background technology]

[0002] Shipping and distribution centers, warehouses, loading docks, air cargo terminals, large retail stores, and other activities that ship and receive heterogeneous sets of items utilize strategies such as packing and unpacking heterogeneous items into boxes, crates, containers, conveyor belts, pallets, etc. Packing heterogeneous items into boxes, crates, pallets, etc. allows the resulting set of items to be handled by lifting equipment (forklifts, cranes, etc.), allowing the items to be packed more efficiently for storage (e.g., in a warehouse) and / or shipment (e.g., in a truck, cargo hold, etc.).

[0003] In some contexts, items may vary so much in size, weight, density, bulk, rigidity, packaging strength, etc. that they may or may not have attributes that allow any given item or set of items to support the size, weight, weight distribution, etc. of given other items that they may need to pack (e.g., in a box, container, pallet, etc.). When assembling heterogeneous items onto a pallet or other set, the items must be carefully selected and stacked to ensure that the palletized stack does not collapse, tip, or otherwise become unstable (e.g., so that it cannot be handled by a machine such as a forklift), and to avoid damage to the items.

[0004] The use of robotics is made more difficult in many environments due to the variety of items, for example, the variability in the order, number, and combination of items packed on a given pallet, and the variety in the types and locations of containers and / or delivery mechanisms from which items must be picked up to place them on a pallet or other container. [Brief explanation of the drawings]

[0005] Various embodiments of the present invention are disclosed in the following detailed description and the accompanying drawings.

[0006] [Figure 1] FIG. 1 illustrates a robotic system for palletizing and / or depalletizing heterogeneous items in accordance with various embodiments.

[0007] [Figure 2] 1 is a flowchart illustrating a process for palletizing one or more items according to various embodiments.

[0008] [Figure 3] 1 is a flowchart illustrating a process for determining a plan for moving an item according to various embodiments.

[0009] [Figure 4A] 10A-10C illustrate trees corresponding to different states or arrangements of items according to various embodiments.

[0010] [Figure 4B] 10A-10C illustrate trees corresponding to different states or arrangements of items according to various embodiments.

[0011] [Figure 4C] 1 illustrates a stack of items with potential placements according to various embodiments.

[0012] [Figure 4D] 1 illustrates a stack of items with potential placements according to various embodiments.

[0013] [Figure 5] 1 is a flowchart illustrating a process for determining a tree corresponding to an arrangement of a set of items, according to various embodiments.

[0014] [Figure 6] 10 is a flowchart illustrating a process for pruning nodes of a tree for placement of a set of items according to various embodiments.

[0015] [Figure 7] 10 is a flowchart illustrating a process for pruning nodes of a tree for placement of a set of items according to various embodiments.

[0016] [Figure 8] 10 is a flow chart illustrating one embodiment of determining an estimate of the state of a pallet and / or stack of items. DETAILED DESCRIPTION OF THE INVENTION

[0017] The present invention may be embodied in various forms, including as a process, an apparatus, a system, a composition of matter, a computer program product embodied on a computer-readable storage medium, and / or a processor configured to execute instructions stored in and / or provided by a memory coupled to the processor. These embodiments, or any other form the present invention may take, may be referred to herein as technology. In general, the order of steps in a disclosed process may be varied within the scope of the present invention. Unless otherwise noted, components, such as a processor or memory, described as configured to perform a task may be implemented as general components temporarily configured to perform the task at a given time, or as specific components manufactured to perform the task. As used herein, the term “processor” refers to one or more devices, circuits, and / or processing cores configured to process data, such as computer program instructions.

[0018] The following is a detailed description of one or more embodiments of the present invention with reference to figures that illustrate the principles of the invention. While the present invention has been described in connection with such embodiments, it is not limited to any particular embodiment. The scope of the present invention is limited only by the claims, and the present invention includes many alternatives, modifications, and equivalents. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. These details are for the purpose of example, and the present invention may be practiced according to the claims without some or all of these specific details. For simplicity, technical matters that are well known in the art related to the present invention have not been described in detail so as not to unnecessarily obscure the present invention.

[0019] As used herein, a geometric model may refer to a model of the state of a workspace (e.g., a programmatically determined state). For example, the geometric model is generated using geometric data determined in connection with generating a plan for moving items in the workspace and an expected result if the items are moved according to the plan. For example, the geometric model corresponds to the state of the workspace that is changed by controlling a robotic arm to pick, move, and / or place items in the workspace, and the picking, moving, and placing of the items is considered to be performed according to the plan (e.g., without errors (e.g., errors or noise) that may be caused by (i) misconfiguration or misalignment of the robotic arm or another component in the workspace, (ii) deformation of the items due to interaction with the robotic arm, (iii) other items in the workspace, other objects in the workspace, or (iv) collisions between the robotic arm or items being moved by the robotic arm and other objects in the workspace).

[0020] As used herein, a "pallet" includes a platform, receptacle, or other container on or within which one or more items can be stacked or placed. Additionally, as used herein, a pallet may be used in connection with packaging and dispensing a set of one or more items. By way of example, the term pallet includes a typical flat transport structure that supports items and is movable by a forklift, pallet jack, crane, or the like. As used herein, a pallet may be made from a variety of materials, including wood, metal, alloys, polymers, and the like.

[0021] As used herein, palletizing an item or set of items includes picking the items from a source location (e.g., a transport structure) and placing the items onto a pallet (e.g., onto a stack of items on a pallet).

[0022] As used herein, depalletization includes picking an item from a pallet (e.g., from a stack of items on a pallet), moving the item, and placing the item at a destination location (e.g., a conveying structure). Examples of palletization / depalletization systems and / or processes for palletizing / depalletizing sets of items are further described in U.S. Patent Application No. 17 / 343,609, which is incorporated herein by reference in its entirety for all purposes.

[0023] As used herein, singulation of items includes picking items from a source pile / stream and placing the items on a conveying structure (e.g., a partitioned conveyor or similar conveying means). Optionally, singulation may include sorting various items on a conveying structure, such as placing items one by one from the source pile / stream into slots or trays on a conveyor. Examples of singulation systems and / or processes for singulating sets of items are further described in U.S. Patent Application No. 17 / 246,356, which is incorporated herein by reference in its entirety for all purposes.

[0024] Kitting, as used herein, includes picking one or more items / objects from corresponding locations and placing one or more items in predetermined locations such that a set of one or more items corresponds to a kit. Examples of kitting systems and / or processes for kitting sets of items are further described in U.S. Patent Application No. 17 / 219,503, which is incorporated herein by reference in its entirety for all purposes.

[0025] As used herein, a vision system comprises one or more sensors that acquire sensor data (e.g., sensor data related to a workspace). The sensors may include one or more of a camera, a high-resolution camera, a 2D camera, a 3D (e.g., RGBD) camera, an infrared (IR) sensor, other sensors for generating a three-dimensional view of the workspace (or a portion of the workspace, such as a pallet and a stack of items on the pallet), any combination of the foregoing, and / or a sensor array including multiple sensors described above. Examples of vision systems are further described in U.S. Patent Application No. 16 / 667,661, which is incorporated herein by reference in its entirety for all purposes.

[0026] Various embodiments include a system, method, and / or apparatus for picking and placing items. The system includes a communications interface and one or more processors coupled to the communications interface. The communications interface is configured to receive sensor data from one or more sensors disposed in a workspace indicating a current state of the workspace, the workspace including a pallet or other container and zero or more items stacked on or in the container. The one or more processors are coupled to the communications interface and configured to (i) use the sensor data to estimate a state of one or both of the pallet or other container and the zero or more items stacked on or in the container, and (ii) use the estimated state to generate or update a plan for controlling a robotic arm to place a next set of items on or in the pallet or other container or to retrieve a next set of items from the pallet or other container, the plan including an ordered sequence of item placement or retrieval. The plan is generated or updated, at least in part, by performing a bounded tree search in which a subset of possible ordered sequences is explored. The subset is determined by either or both selecting branches of the search tree that satisfy one or more selection criteria and / or eliminating from consideration one or more branches that satisfy pruning criteria.

[0027] In various embodiments, the robotic palletization systems disclosed herein determine where to place a current item by considering not only the current item but also the next or future item (or a predetermined number of next items). The system determines where to place the current item based on (i) the estimated state of the pallet or stack of items and (ii) a combination / permutation of the placement of the current item and the next item (or a predetermined number of next items). In some embodiments, the system uses a scoring function (e.g., a function that scores placements, which may be the inverse of, i.e., negatively correlated with, the corresponding cost function) in connection with determining the best placement of the current item (e.g., the first item).

[0028] Some of the challenges that arise from utilizing knowledge of the future sequence of items to be palletized include: Each item may be placed in one of several positions and / or several orientations on a pallet / stack of items. Determining a plan for placing items in a particular position and / or orientation based at least in part on some optimization (e.g., packing density, time to move items, etc.) may involve determining / simulating the outcomes of different possible scenarios. Determining / simulating different outcomes may be computationally expensive / inefficient. When determining among multiple locations where a current and / or future item should be placed, if one or more future items are considered, the computational burden of determining / simulating the outcomes of different possible scenarios for placing each item in each possible location becomes exponentially cumbersome. As an example, assume a system knows the following sequence of five boxes (e.g., items) and the system can place the first box in four configurations (e.g., four different combinations of location and orientation). The system may simulate all four configurations, and then the system extends the search for the next box with four additional configurations. After a two-deep search, the system reaches 16 possible scenarios. This number of scenarios grows exponentially and becomes unwieldy at higher search depths.

[0029] Various embodiments include a system that utilizes tree searching in connection with determining where to place one or more items (e.g., a current item and / or one or more future items in a sequence of items). The system searches a tree corresponding to different scenarios in connection with determining where to place one or more items (e.g., to find a best or optimal arrangement of one or more items, pallets, etc.). The different scenarios may each correspond to an arrangement of one or more items at corresponding positions on a pallet / stack of items. In some embodiments, the system searches the tree to a particular depth. The depth to which the system searches the tree may be predetermined or may be based at least in part on a threshold (e.g., a threshold related to the computational load of searching a particular branch / scenario).

[0030] According to various embodiments, the system determines a search space corresponding to states according to various arrangements of the set of items. Evaluating the entire search space can be computationally expensive and add undesirable latency to determining a plan for picking and placing the items. To reduce the computational complexity (and required resources / time) of evaluating various combinations / permutations of arrangements for the set of items, the system implements a pruning process / technique to reduce the number of combinations / permutations considered. The system can prune the search space to remove arrangements whose predicted stability does not satisfy the stability condition (e.g., predicted stability is less than a stability threshold, heuristics suggest that the estimated state is considered unstable, etc.). After pruning the search space, the system evaluates the remaining combinations / permutations of arrangements to determine the arrangement that produces the best result (e.g., the best arrangement determined based on one or more of the predicted stability and the cost function, etc.). The system uses the arrangement that produces the best result to determine a plan for picking and placing an item (e.g., the next item in the set of items).

[0031] According to various embodiments, the system performs placement simulations for at least a subset of placement combinations / permutations (e.g., at each node in a tree structure representing the search space), and the system determines a predicted stability corresponding to each simulation. In some embodiments, the system constrains the search space based on a combination of (i) simulating a subset of placements and evaluating the resulting predicted stability, and (ii) for the subset of placements, determining whether the placement is predicted to produce a stable stack based on one or more predetermined heuristics. The system constrains the search space based on the one or more heuristics or placement simulations. In various embodiments, the system uses one or more rules or heuristics in connection with analyzing the search space (e.g., constraining / pruning the search space). For example, a domain expert may define one or more rules or heuristics for pruning based on observation, judgment, etc., about which sorts of placements or placement sequences are likely to be more or less worth considering.

[0032] Various embodiments include a system for picking and placing a set of items to form a stable and space-efficient pallet. In some embodiments, the system determines where to place the set of items (e.g., how the set of items are arranged on the pallet) to maximize a predetermined scoring function (or minimize a predetermined cost function).

[0033] 1 illustrates a robotic system for palletizing and / or depalletizing heterogeneous items, according to various embodiments. In some embodiments, system 100 performs at least a portion of process 200 of FIG. 2, process 300 of FIG. 3, process 500 of FIG. 5, process 600 of FIG. 6, process 700 of FIG. 7, and / or process 800 of FIG. 8.

[0034] In the illustrated example, system 100 includes a robotic arm 102. In this example, robotic arm 102 is fixed; however, in various alternative embodiments, robotic arm 102 may be fully or partially mobile, e.g., mounted on rails, fully mobile on a motor-driven chassis, etc. As shown, robotic arm 102 is used to pick random and / or heterogeneous items (e.g., boxes, packages, etc.) from a conveyor (or other source) 104 and stack them on a pallet (e.g., a platform or other container) 106. Pallet (e.g., a platform or container) 106 comprises a pallet, container, or base with wheels on the four corners and at least partially closed on three of its four sides, and is also referred to as a three-sided "roll pallet," "roll cage," and / or "roll," "cage," or "trolley." In other embodiments, rolls with more, fewer, and / or no sides or pallets without wheels may be used. In some embodiments, other robots not shown in FIG. 1 may be used to push the container 106 into position for loading / unloading and / or onto a truck or other destination for transport, etc.

[0035] In some embodiments, multiple bins 106 may be positioned around the robotic arm 102 (e.g., within a range near a threshold of the robotic arm or within the range of another robotic arm). The robotic arm 102 may simultaneously (e.g., side-by-side and / or contemporaneously) stack one or more items onto multiple pallets. Each of the multiple pallets may be associated with a manifest and / or order. For example, each of the pallets may be associated with a pre-defined destination (e.g., a customer, an address, etc.). In some examples, some of the multiple pallets may be associated with the same manifest and / or order. However, each of the multiple pallets may be associated with a different manifest and / or order. The robotic arm 102 may place multiple items corresponding to the same order onto multiple pallets. The system 100 may determine the arrangement (e.g., stacking of items) on the multiple pallets (e.g., how items for an order are divided among the multiple pallets, how items on any one pallet are stacked, etc.). System 100 may store one or more items (e.g., items for an order) in a buffer area or staging area while one or more other items are stacked on a pallet. As an example, one or more items may be stored in a buffer area or staging area until system 100 determines that the placement of each of the one or more items on a pallet (e.g., into a stack) meets (e.g., exceeds) a threshold suitability or threshold stability. The threshold suitability or threshold stability may be a predetermined value or may be a value empirically determined based at least in part on historical information. Machine learning algorithms may be implemented in connection with determining whether the placement of an item on a stack is expected to meet (e.g., exceed) the threshold suitability or threshold stability (e.g., a threshold against which a simulation or model is compared to evaluate whether to place an item on a stack).

[0036] In the illustrated example, the robotic arm 102 is equipped with a suction-type end effector (e.g., end effector 108). The end effector 108 has a plurality of suction cups 110. The robotic arm 102 is used to position the suction cups 110 of the end effector 108 over the item to be picked up, as shown, and a vacuum source provides a suction force to grasp the item, lift the item from the conveyor 104, and place the item at a destination location on the bin 106. Various types of end effectors may be implemented.

[0037] In various embodiments, the system 100 includes a vision system that is used to generate a model of the workspace (e.g., a 3D model and / or a geometric model of the workspace). For example, one or more of a 3D camera or other camera 112 mounted on the end effector 108 and cameras 114, 116 mounted in the space in which the system 100 is deployed are used to identify items on the conveyor 104 and / or determine a plan for grasping, picking / placing, and stacking items onto the bins 106 (or placing the items in a buffer or staging area, if applicable). In various embodiments, additional sensors not shown may be used to identify (e.g., determine) attributes of the item, grasp the item, pick up the item, move the item through the determined trajectory, and / or place the item at a destination location on or in the item in the bin 106 on the conveyor 104 and / or other source and / or staging area where the item may be placed and / or relocated, for example, by the system 100. Examples of such additional sensors not shown may include weight or force sensors embedded in and / or adjacent to the conveyor 104 and / or the robotic arm 102, force sensors in the xy plane and / or z direction (vertical direction) of the suction cup 110.

[0038] In the illustrated example, camera 112 is mounted to the side of the body of end effector 108, although in some embodiments camera 112 and / or additional cameras may be mounted in other locations (e.g., facing downward from between suction cups 110, mounted on the underside of the body of end effector 108, mounted on a segment or other structure of robotic arm 102, or mounted in other locations, etc.). In various embodiments, cameras such as 112, 114, and 116 may be used to read text, logos, photographs, drawings, images, marks, barcodes, QR codes, or other encoded and / or graphic information or content visible on and / or comprising items on conveyor 104.

[0039] In some embodiments, the system 100 includes a dispenser device (not shown) configured to dispense a quantity of spacer material from a supply of spacer material in response to a control signal. The dispenser device may be located on the robotic arm 102 or may be located near the workspace (e.g., within a threshold distance of the workspace). For example, the dispenser device may be positioned within the workspace of the robotic arm 102 such that the dispenser device dispenses spacer material onto or around a container 106 (e.g., a pallet) or within a predetermined distance of an end effector 108 of the robotic arm 102. arrangementIn some embodiments, the dispenser device comprises a mounting fixture configured to mount the dispenser device on or adjacent to the end effector 108 of the robotic arm 102. The mounting fixture may be at least one of a bracket, a strap, one or more fasteners, etc. As an example, the dispenser device may comprise a biasing device / mechanism that biases the feed material within the dispenser device to be extruded / dispensed from the dispenser device. The dispenser device may comprise a gating structure used to control the dispensing of the spacer material (e.g., actuation of the gating structure prevents dispensing of the spacer material in the absence of actuation, and actuation of the gating structure enables dispensing of the spacer material).

[0040] The dispenser device may include a communications interface configured to receive control signals. For example, the dispenser device may communicate with one or more terminals (e.g., control computer 118). The dispenser device may communicate with one or more terminals via one or more wired connections and / or one or more wireless connections. In some embodiments, the dispenser device communicates information to one or more terminals. For example, the dispenser device may send an indication of the status of the dispenser device (e.g., an indication of whether the dispenser device is operating normally), an indication of the type of spacer material included within the dispenser device, an indication of the supply level of spacer material within the dispenser device (e.g., an indication of whether the dispenser device is full, empty, half full, etc.), etc. to the control computer 118. The control computer 118 may be utilized in connection with controlling the dispenser device to dispense a quantity of spacer material. For example, the control computer 118 may determine that a spacer is utilized in connection with palletizing one or more items, such as to improve the predictive stability of the stack of items on / in the container 106. Control computer 118 may determine an amount of spacer material to utilize in connection with palletizing one or more items (e.g., number of spacers, amount of spacer material, etc.). For example, the amount of spacer material to utilize in connection with palletizing one or more items may be determined based at least in part on determining a plan for palletizing the one or more items.

[0041] In some embodiments, the dispenser device includes an actuator configured to dispense a quantity of spacer material from a supply of spacer material in response to a control signal. In response to determining that a spacer / spacer material is to be utilized in connection with palletizing one or more items, the control computer 118 may generate a control signal to cause the actuator to dispense a quantity of spacer material. The control signal may include an indication of the amount of spacer material to be utilized as a spacer.

[0042] According to various embodiments, the spacer or spacer material is a rigid block. For example, the spacer or spacer material may be a rigid block of foam. In some embodiments, the spacer or spacer material comprises polyurethane.

[0043] In some embodiments, the supply of spacer material includes a plurality of cut blocks, which may be pre-loaded into a spring-loaded cartridge that biases the plurality of cut blocks toward the dispensing end, such that in response to dispensing a cut block from the cartridge, another block of the plurality of cut blocks is pushed out of the cartridge into a position to be next dispensed.

[0044] In some embodiments, the supply of spacer material includes one or more of a larger block of spacer material, a strip of spacer material, and a roll of spacer material. The dispensing device or system 100 may include a cutter configured to cut a quantity of spacer material from the supply of spacer material. In response to providing a control signal to the actuator, the actuator may cause the cutter to cut a quantity of spacer material from the supply of spacer material.

[0045] In some embodiments, the source of spacer material includes a liquid precursor, and in response to providing a control signal to the actuator, the actuator causes a quantity of spacer material to be dispensed onto a surface of the pallet or a stack of items on the pallet, the dispensed precursor being capable of curing after being dispensed onto the surface of the pallet or a stack of items on the pallet.

[0046] In some embodiments, the source of spacer material includes an extruded material. In response to providing a control signal to the actuator, the extruded material is filled to one or more of a desired size and a desired hardness. The extruded material may be sealed in response to determining that the extruded material has filled to one or more of a desired size and a desired hardness. In some embodiments, the extruded material is filled with a fluid. The fluid may be one or more of air, water, etc. In some embodiments, the extruded material is filled with a gel.

[0047] In various embodiments, robotically controlled dispenser tooling or machines fill gaps between and / or adjacent to boxes to prepare the surface area for the next box / layer to be placed. In some embodiments, system 100 may use robotic arm 102 to pick / place pre-cut material and / or dynamically trim spacer material to fit the surface area needs of the next item to be placed. In some embodiments, a robotically controlled dispenser device or a robotic palletization system including a robotically controlled dispenser device includes equipment for trimming long tubes and / or packages to form cuboids and placing the cuboids on an existing pallet in connection with preparing the surface area for the next box or item that the system determines may not normally fit on the pallet surface area (e.g., on top of the previous layer). Spacers may include, but are not limited to, foam, inflated plastic bags, wood, metal, plastic, etc. The dispenser device may place (e.g., push, feed, etc.) the cuboids (e.g., spacers) directly onto the pallet, and / or the device may feed the cuboids (e.g., spacers) near a robotic arm and an end effector may reposition / place the cuboids (e.g., spacers) on the pallet surface area. The dispenser device may feed a predetermined amount (e.g., the correct amount or expected amount) of spacer material to correct or improve surface area discrepancies between boxes or items on a layer (e.g., on the top surface of a layer) to prepare the surface area for the next box or item.

[0048] 1 , in the depicted example, robotic system 100 includes a control computer 118 configured to communicate with elements such as robotic arm 102, conveyor 104, end effector 108, and sensors (such as cameras 112, 114, and 116 and / or weight, force, and / or other sensors not shown in FIG. 1 ), in this example via wireless communication (although in various embodiments, via one or both of wired and wireless communication). In various embodiments, control computer 118 is configured to use input from the sensors (such as cameras 112, 114, and 116 and / or weight, force, and / or other sensors not shown in FIG. 1 ) to observe, identify, and determine one or more attributes of items being loaded into and / or unloaded from container 106. In various embodiments, the control computer 118 identifies the item and / or its attributes using item model data in a library stored in the control computer 118 and / or accessible to the control computer 118, for example, based on image and / or other sensor data. The control computer 118 uses the model corresponding to the item to determine and implement a plan for stacking the item, along with other items, in / on a destination (e.g., a container 106). In various embodiments, the item attributes and / or model are utilized to determine a strategy for grasping, moving, and placing the item at a destination location (e.g., a location where the item has been determined to be placed as part of the planning / re-planning process for stacking the item in / onto the container 106).

[0049] In the illustrated example, control computer 118 is connected to an "on-demand" telemanipulator 122. In some embodiments, if control computer 118 is unable to continue in fully automated mode, e.g., if a strategy for grasping, moving, and placing an item becomes indeterminable and / or fails such that control computer 118 has no strategy for completing the pick and place of the item in fully automated mode, control computer 118 instructs human user 124 to intervene, e.g., by manipulating robotic arm 102 and / or end effector 108 using telemanipulator 122 to grasp, move, and place the item.

[0050] A user interface for operation of system 100 may be provided by control computer 118 and / or remote operator 122. The user interface may provide the current status of system 100, including information associated with the current state of a pallet (or stack of items associated therewith), the current order or manifest being palletized or depalletized, the performance of system 100 (e.g., number of items palletized / depalletized by time), etc. A user may select one or more elements on the user interface or otherwise provide input to the user interface to activate or pause system 100 and / or particular robotic arms within system 100.

[0051] According to various embodiments, the system 100 performs a machine learning process to model the condition of the pallet, such as to generate a model of the stacks on the pallet. The machine learning process may include an adaptive and / or dynamic process for modeling the condition of the pallet. The machine learning process may define and / or update / refine the process by which the system 100 generates the model of the condition of the pallet. The model may be generated based at least in part on input (e.g., information obtained from the sensors) from one or more sensors in the system 100 (e.g., one or more sensors or sensor arrays in the workspace of the robotic arm 102). The model may be generated based at least in part on the shape of the stack, the vision response (e.g., information obtained by one or more sensors in the workspace), and the machine learning process, etc. The system 100 may utilize the model in connection with determining an efficient (e.g., maximize / optimize efficiency) method for palletizing / depalletizing one or more items, where the method for palletizing / depalletizing may be constrained by a minimum threshold stability value. The process for palletizing / depalletizing one or more items may be configurable by a user / administrator. For example, one or more metrics at which the process for palletizing / depalletizing is maximized may be configurable (e.g., set by a user / administrator).

[0052] In the context of palletizing one or more items, system 100 may generate a model of the pallet's condition in connection with determining whether to place an item on a pallet (e.g., on a stack), selecting a plan for placing the item on the pallet, such as a destination location where the item will be placed, and a trajectory along which the item will be moved from a source location (e.g., a current destination, such as a conveyor) to the destination location. System 100 may also use the model in connection with determining a strategy for releasing or otherwise placing the item on the pallet (e.g., applying force to the item to fit it tightly on the stack). Modeling the pallet's condition may include simulating the placement of the item at different destination locations on the pallet (e.g., on a stack) and determining corresponding different predicted fitnesses and / or predicted stabilities (e.g., stability indices) that are predicted to result from placing the item at the different locations. System 100 may select destination locations where the predicted fitnesses and / or predicted stabilities meet (e.g., exceed) corresponding thresholds. Additionally or alternatively, system 100 may select a destination location that optimizes predicted fitness (eg, of the items on the stack) and / or predicted stability (eg, of the stack).

[0053] Conversely, in the context of depalletizing one or more items from a pallet (e.g., a stack on a pallet), the system 100 (e.g., control computer 118) may generate a model of the condition of the pallet in connection with determining whether to remove an item from the pallet (e.g., a stack) and selecting a plan for removing the items from the pallet. The model of the condition of the pallet may be utilized in connection with determining the order in which items are removed from the pallet. For example, the control computer 118 may utilize the model to determine whether removing an item is predicted to reduce the stability of the condition of the pallet (e.g., a stack) below a threshold stability. The system 100 (e.g., control computer 118) may simulate the removal of one or more items from the pallet and select an order for removing items from the pallet that optimizes the stability of the condition of the pallet (e.g., a stack). The system 100 may use the model to determine the next item to remove from the pallet. For example, the control computer 118 may select an item as the next item to be removed from the pallet based at least in part on a determination that the predicted stability of the stack during and / or after removal of the item exceeds a threshold stability. Models and / or machine learning processes may be utilized in connection with determining a strategy for picking an item from a stack. For example, after an item is selected as the next item to be removed from a stack, the system 100 may determine a strategy for picking the item. The strategy for picking the item may be based at least in part on the condition of the pallet (e.g., the determined stability of the stack), attributes of the item (e.g., size, shape, weight or predicted weight, center of gravity, type of packaging, etc.), the location of the item (e.g., position relative to one or more other items in the stack), attributes of other items on the stack (e.g., attributes of adjacent items, etc.), etc.

[0054] According to various embodiments, machine learning processes are performed in connection with improving a gripping strategy (e.g., a strategy for gripping an item). The system 100 may obtain attribute information about one or more items to be palletized / depalletized. The attribute information may include one or more of the item's orientation, material (e.g., packaging type), size, weight (or predicted weight), or center of gravity. The system 100 may also obtain information about a source location (e.g., information about an input conveyor from which the item is picked) and a pallet on which the item is placed (or a set of pallets from which a destination pallet is determined, such as a set of pallets corresponding to an order in which the item is stacked). In connection with determining a plan for picking and placing an item, the system 100 may use information about the item (e.g., attribute information, destination location, etc.) to determine a strategy for picking the item. The pick strategy may include a suggestion of a pick location (e.g., a location on the item where the robotic arm 102 engages the item via an end effector, etc.). The pick strategy may include a force applied to pick the item and / or a holding force to grasp the item while the robotic arm 102 moves the item from a source location to a destination location. The system 100 may utilize machine learning processes to improve the pick strategy based at least in part on associations between information about the item (e.g., attribute information, destination location, etc.) and performance in picking the item (e.g., historical information associated with past iterations of picking and placing the item or similar items (e.g., items sharing one or more similar attributes)).

[0055] According to various embodiments, system 100 may determine to utilize a spacer or an amount of spacer material in connection with palletizing one or more items in response to determining that utilizing a spacer or an amount of spacer material results in improved stacking of the items on the pallet (e.g., improves the stability of the stack of items). In some embodiments, the determination that placing one or more spacers in connection with placing a set of N items on a pallet improves the stacking of the items on the pallet is based at least in part on one or more of packing density, flat top surface, and stability. In some embodiments, the determination that placing one or more spacers in connection with placing a set of N items on a pallet improves the stacking of the items on the pallet is based at least in part on determining that the packing density of a stack of items including the set of N items is higher than the packing density if the set of N items were placed on the pallet without the one or more spacers. In some embodiments, the determination that placing one or more spacers in association with placing a set of N items on a pallet improves stacking of items on the pallet is based at least in part on a determination that the top surface is flatter than the top surface of the set of N items placed on the pallet without the one or more spacers. In some embodiments, the determination that placing one or more spacers in association with placing a set of N items on a pallet improves stacking of items on the pallet is based at least in part on a determination that the stability of a stack of items including the set of N items is greater than the stability of the set of N items placed on the pallet without the one or more spacers. N may be a positive integer (e.g., a positive integer less than the total number of items to be palletized on the completed pallet).

[0056] As an example, N may be less than the total number of items to be palletized, and thus the system 100 may be limited in optimizing the stack of items (e.g., the robotic system 100 may only plan to place N items at a time). Therefore, utilizing one or more spacers increases the number of degrees of freedom associated with placing the N items. The system 100 may utilize one or more spacers to optimize the stack of N items (or to achieve a "good enough" stack of N items, such as a stack that meets a minimum stability threshold). The system 100 may utilize a cost function in connection with determining whether to utilize one or more spacers, the number of spacers to utilize, the placement of the spacers, etc. For example, the cost function may include one or more of a stability value, a time to place one or more items, a packing density of the stack of items, a top flatness or variability of the top surface of the stack of items, a cost of the feed material, etc.

[0057] According to various embodiments, the control computer 118 controls the system 100 to place a spacer on the container 106 (e.g., a pallet) or a stack of items in connection with improving the stability of the stack of items on the container 106. As one example, the spacer may be placed in response to a determination that it is estimated (e.g., it is likely, such as having a probability above a predetermined likelihood threshold) that the stability of the stack of items will be improved if the spacer is utilized. As another example, the control computer 118 may control the robotic system 100 to utilize a spacer in connection with placing a set of items (e.g., a set of N items, where N is an integer) in response to a determination that the stability of the stack of items is less than a threshold stability value and / or that the stability of the stack of items is estimated to be less than a threshold stability value.

[0058] According to various embodiments, the control computer 118 may determine the stability of the stack of items based at least in part on a model of the stack of items and / or a simulation of placing one or more sets of items. The computer system may obtain (e.g., determine) a current model of the stack of items and model (e.g., simulate) the placement of the set of items. In connection with modeling the stack of items, a predicted stability of the stack of items may be determined. Modeling the stack of items may include modeling the placement of spacers in connection with modeling the placement of the set of items.

[0059] In some embodiments, the control computer 118 may determine the stability of a stack of items (or a simulated stack of items) based at least in part on one or more attributes of the top surface of the stack of items (or the simulated stack of items) and / or the spacer. For example, a measure of the degree to which the top surface is flat may be utilized in connection with determining the stability of the stack of items. Placing a box on a flat surface may result in a stable arrangement and / or stack of items. As another example, the surface area of ​​the flat area above the top surface may be utilized in connection with determining the stability or predicted stability of the arrangement of items on the stack of items. The greater the flat area above the top surface of the stack of items relative to the bottom surfaces of items placed on the stack of items, the more likely the stability of the stack of items will meet (e.g., exceed) a threshold stability value.

[0060] According to various embodiments, system 100 may generate a model of a pallet or a stack of one or more items on the pallet, and spacers or spacer material may be determined to be placed in association with palletizing the one or more items based at least in part on the model of the pallet or the stack of one or more items on the pallet. System 100 may generate a model of at least a top surface of the pallet or the stack of one or more items on the pallet, determine a set of N items to be next placed on the pallet (e.g., N is a positive integer), determine that placing one or more spacers in association with placing the set of N items on the pallet improves the stack of items on the pallet compared to a stack resulting from placing the set of N items without the spacers, generate one or more control signals to cause an actuator to supply an amount of spacer material corresponding to the one or more spacers, and provide the one or more control signals to the actuator in association with placing the set of N items on the pallet.

[0061] According to various embodiments, item (e.g., item type) variability among palletized items can complicate palletizing the items in a stable manner (e.g., in a manner where the stability of the stack of items meets a threshold stability value). In some embodiments, the control computer 118 may only be able to predict a certain number of items to be palletized. For example, the system may have a queue / buffer of N items to be palletized, where N is a positive integer. N may be a fraction of the total number of items to be stacked on the pallet. For example, N may be relatively small compared to the total number of items to be stacked on the pallet. Thus, the system 100 may only be able to optimize the stacking of items utilizing the next N known items. For example, the system 100 may determine a plan for stacking one or more items according to the current state of the stack of items (e.g., current model) and one or more attributes associated with the next N items to be stacked. In some embodiments, the use of one or more spacers may provide flexibility in how the next N items are stacked and / or improve the stability of the stack of items.

[0062] Various embodiments include palletizing a relatively large number of mixed boxes or items onto a pallet. The various boxes and items being palletized may have different attributes (height, shape, size, stiffness, packaging type, etc.). Variations in one or more attributes of the various boxes or items may make it difficult to place the items onto the pallet in a consistent manner. In some embodiments, the system 100 (e.g., control computer 118) may determine a destination location (e.g., a location where the item is to be placed) for an item that has a larger surface area (e.g., a larger base) than the box or other item below the item being placed. In some embodiments, items having different heights (e.g., different bin heights) may be placed in relatively tall areas of the pallet (e.g., above a height threshold equal to the maximum pallet height multiplied by 0.5, above a height threshold equal to the maximum pallet height multiplied by 2 / 3, above a height threshold equal to the maximum pallet height multiplied by 0.75, or above a height threshold equal to the maximum pallet height multiplied by another predetermined value).

[0063] According to various embodiments, machine learning processes are performed in connection with improving a spacer material supply / utilization strategy (e.g., a strategy for utilizing spacer material in connection with palletizing one or more items). The system 100 may obtain attribute information about one or more items to be palletized / depalletized and attribute information about one or more spacers utilized in connection with palletizing / depalletizing the one or more items. The attribute information may include one or more of the item's orientation, material (e.g., type of spacer material), size, weight (or predicted weight), center of gravity, stiffness, dimensions, etc. The system 100 may also obtain information about a source location (e.g., information about an input conveyor from which an item is picked) and a pallet onto which an item is placed (or a set of pallets from which a destination pallet is determined, such as a set of pallets corresponding to an order in which items are stacked). In connection with determining a plan for picking and placing items, the system 100 may use information about the items (e.g., attribute information, destination locations, etc.) to determine a strategy for palletizing the items (e.g., picking and / or placing the items). The palletizing strategy may include suggestions for pick locations (e.g., locations on the items where the robotic arm 102 engages the items, such as via an end effector) and destination locations (e.g., locations on the pallet / bin 106 or stack of items). The palletizing strategy may include suggestions for the force to be applied to pick the items and / or the holding force with which the robotic arm 102 grips the items while moving them from the source location to the destination location, the trajectory by which the robotic arm moves the items to the destination location, the amount of spacer material (if any) to be utilized in connection with placing the items at the destination location, and a plan for disposing the spacer material.System 100 may utilize machine learning processes to improve palletizing strategies based at least in part on relationships between information about the item (e.g., attribute information, destination location, etc.) and one or more of: (i) the performance of picking and / or placing the item (e.g., historical information associated with past iterations of picking and placing the item or similar items (e.g., items sharing one or more similar attributes)); (ii) the performance of the stability of the item after it has been placed in its destination location, e.g., against predicted stability generated using a model of the stack of items (e.g., historical information associated with past iterations of palletizing the item or similar items (e.g., items sharing one or more similar attributes)); and (iii) the performance of the stability of the stack of items after it has been placed in its destination location, e.g., against predicted stability generated using a model of the stack of items (historical information associated with past iterations of palletizing the item or similar items and / or spacers (e.g., items / spacers sharing one or more similar attributes)). In some embodiments, system 100 may use machine learning processes to improve the utilization of one or more spacers in association with a palletizing strategy based at least in part on a relationship between information associated with the spacers and / or one or more items to be palletized (e.g., information about the spacers and / or one or more items to be palletized in association with the palletizing strategy (e.g., attribute information, destination location, etc.)) and the stability performance of palletizing the set of items using one or more spacers relative to the predicted stability of palletizing the set of items using one or more spacers (e.g., predicted stability based on a simulation of palletizing the items using a model of the stack of items).

[0064] The model generated by system 100 may correspond to or be based at least in part on a geometric model. In some embodiments, system 100 generates the geometric model based at least in part on one or more placed items (e.g., items placed by system 100 controlling robotic arm 102), one or more attributes associated with at least a portion of one or more items, one or more objects in the workspace (e.g., predetermined objects such as pallets, robotic arms, shelving systems, chutes, or other infrastructure included in the workspace), etc. The geometric model may be determined at least in part based on executing a physics engine on control computer 118 to model the stacking of items (e.g., modeling the state / stability of a stack of items, etc.). The geometric model may be determined based on predicted interactions of various components of the workspace (e.g., interactions of an item with another item, object, or simulated forces applied to the stack (e.g., to model the use of a forklift or other equipment to lift / move a pallet or other container on which the stack of items is located)).

[0065] According to various embodiments, the system determines an estimated state of the stack of items. For example, system 100 determines the estimated state in response to the next item being placed, or the Nth next item being placed, etc. The estimated state may be determined based at least in part on one or more of a geometric model of the stack of items (or of the workspace) and / or sensor data (e.g., data acquired by a vision system of system 100).

[0066] In some embodiments, system 100 utilizes sensor data and geometric data (e.g., a geometric model) in connection with determining the location to place one or more items on a pallet (or in connection with depalletizing one or more items from a pallet). System 100 may use different data sources to model the condition of the pallet (or stack of items on a pallet). For example, system 100 estimates the location of one or more items on the pallet and one or more characteristics (or attributes) associated with the one or more items (e.g., item size). The one or more characteristics associated with the one or more items may include item size (e.g., item dimensions), center of gravity, item stiffness, package type, deformability, shape, identifier location, etc.

[0067] According to various embodiments, system 100 estimates a state of the workspace (also referred to herein as an estimated state) based at least in part on geometric data (e.g., a geometric model of the workspace) and sensor data (e.g., data acquired by one or more sensors deployed in the workspace). In response to obtaining the estimated state of the workspace, system 100 utilizes the estimated state in connection with moving items within the workspace. For example, system 100 uses the estimated state to determine a plan and / or strategy for picking an item from a source location and placing it at a target location (also referred to herein as a destination location).

[0068] According to various embodiments, the geometric model is determined based at least in part on one or more attributes of one or more items in the workspace. For example, the geometric model reflects the respective attributes of a set of items (e.g., one or more of a first set that are palletized / stacked and a second set of items that are to be palletized / stacked, etc.). AttributeExamples include item size (e.g., item dimensions), center of gravity, item rigidity, packaging type, identifier location, item deformability, item shape, etc. Various other attributes of items or objects within the workspace may also be implemented. As another example, the geometric model includes the predicted stability of one or more items stacked on or in a container (e.g., a pallet). The geometric model may include the predicted stability of a set of items (e.g., a stack of items) and / or the predicted stability of individual items included in a stack of items. In some embodiments, the system 100 determines the predicted stability of an item based at least in part on (i) one or more attributes of the item and (ii) one or more predicted interactions with the item and another item or object (e.g., a pallet) within the workspace. For example, the system 100 may determine the predicted stability based on determining attributes of another item or object in contact with the item whose predicted stability is being calculated. Examples of other item attributes that may affect the predicted stability of a particular item include rigidity, deformability, size, etc. As one example, if a particular item rests on another rigid item, the particular item may have a higher predicted stability than if the particular item rests on another non-rigid or less rigid item. As another example, if a particular item rests on another deformable item (such as one made of soft packaging), the particular item may have a lower predicted stability than if the particular item rests on another non-deformable or less deformable item. As another example, if a particular item rests on another item that has a top surface area that is larger than its bottom surface area, or if a relatively high percentage of the particular item's bottom surface is supported by the top surface of the other item, the predicted stability of the item may be relatively higher, or at least higher, than if the particular item has a top surface area that is smaller than its bottom surface area, or if a relatively high percentage of the particular item's bottom surface is not supported by / interacting with the top surface of the other item.

[0069] In some embodiments, the system 100 may detect noise (e.g., sensor noise). Z ) to account for noise. System 100 can estimate noise in the sensor data based at least in part on an empirical analysis of the vision system. For example, an empirical analysis of the vision system's performance can be performed to determine noise in (e.g., inherent in) the sensor data. In some embodiments, system 100 stores a predetermined sensor noise profile associated with the vision system. System 100 can utilize the sensor noise profile in connection with adjusting the sensor data to account for noise. For example, system 100 can apply adjustments to counteract predicted noise based at least in part on the sensor profile. An empirical analysis of the vision system's performance can include (i) manually / physically measuring an item or workspace, (ii) capturing the same using a vision system, and (iii) determining (e.g., using digital processing, etc.) a difference between the manual / physical measurement of the item / workspace and a measurement of the same using sensor data. System 100 can consider the difference between the manual / physical measurement of the item / workspace and a measurement of the same using sensor data as a noise profile. As an example, the system 100 determines variability in the sensor data and determines a sensor noise profile based at least in part on the variability. The empirical analysis may be performed on a set of statistically significant experiments / measurements. Examples of noise (or sensor data inaccuracies) may include (i) image inaccuracies at the edge of the vision system's field of view, (ii) glare / reflections from items or other objects in the workspace, etc.

[0070] In some embodiments, system 100 adjusts the geometric model to account for noise (e.g., geometric noise or inaccuracies resulting from the transition of the geometric model to the physical world, such as by controlling a robotic arm). System 100 can estimate the noise contained in the geometric model based at least in part on an empirical analysis of the accuracy of the robotic control or other objects in the workspace (e.g., estimated deformation of a pallet, deviation of the pallet's placement relative to the location utilized in the geometric model, etc.). For example, an empirical analysis of the performance of the control of the robotic arm (e.g., in performing a task such as placing an item) can be performed to determine the noise incorporated (e.g., inherent) in the geometric model. As an example, system 100 determines the variance of the geometric model and determines a geometric noise profile based at least in part on the variance. In some embodiments, system 100 stores a predetermined geometric noise profile associated with the vision system. System 100 can utilize the geometric noise profile in connection with adjusting the geometric model to account for noise. For example, the system 100 applies adjustments to counteract predictive noise contained in the geometric model (e.g., noise that occurs based on controlling a robot (e.g., robot arm 102) according to a plan determined based on the geometric model).

[0071] In some embodiments, system 100 updates the geometric model after each item movement (e.g., placement). For example, system 100 maintains (e.g., stores) a geometric model corresponding to the state of the workspace (e.g., the state / stability of the stack of items and the position of one or more items within the stack of items). The geometric model utilizes the current geometric model in connection with determining a plan for moving the items and controlling the robotic arm to move the items. In response to item movement, system 100 updates the geometric model to reflect the item movement. For example, when depalletizing a stack of items, in response to a particular item being picked and removed from the stack of items, system 100 updates the geometric model so that the particular item is no longer represented as being on the stack but is included within the geometric model at the particular item's placed destination location, or if the destination location is outside the workspace, the geometric model is updated to remove the item. Additionally, the geometric model is updated to reflect the stability of the stack of items after a particular item is removed from the stack. As another example, when palletizing a set of items, system 100 updates the geometric model to reflect the placement of particular items on / in the stack of items. System 100 can update the geometric model to include an updated stability of the stack of items based at least in part on the placement of items on / in the stack of items (e.g., to reflect interactions that particular items have with other items, or interactions between other items based on the placement of particular items, etc.).

[0072] In some embodiments, system 100 updates the current state (e.g., based on updates to the geometric model) after (i) the movement (e.g., placement) of a predetermined number of items, or (ii) the earlier of the movement of a predetermined number of items or the detection of an anomaly (e.g., an anomaly that meets one or more anomaly criteria (e.g., the degree of anomaly exceeds an anomaly threshold, etc.)). The predetermined number of items (e.g., X items, where X is a positive integer) may be set based on user selection, robot control system policy, or otherwise determined based on empirical analysis of item placement. As one example, the predetermined number of items is set based on a determination that the number of items will yield optimal / best results with respect to a predetermined cost function (e.g., a cost function reflecting efficiency, stability, expected change in stability, etc.). As an example, system 100 determines a current estimated state, uses the current estimated state to determine a plan for moving the next X items, and after moving the X items (e.g., stacking or de-stacking the items), system 100 determines an updated estimated state (e.g., a geometric update / model to reflect the placement of the X items). System 100 determines the updated state based at least in part on a combination of a geometric model and sensor data (e.g., a current geometric model and current sensor data, etc.). System 100 then utilizes the updated state in connection with determining a plan and controlling the robot to place the next set of items according to the plan.

[0073] In some embodiments, the frequency with which system 100 updates the estimated state is dynamically determined. For example, system 100 dynamically determines a value X corresponding to the number of items for which the system updates the estimated state after being moved. In some embodiments, system 100 dynamically determines value X (e.g., corresponding to the estimated state update frequency) based at least in part on one or more attributes of the items (e.g., attributes of previously moved / placed items and / or attributes of the item being moved). As an example, system 100 dynamically determines value X based on a determination that an irregularly placed item or a deformable item was placed before (e.g., immediately before) placing the set of X items using the current estimated state, or that the set of X items includes an irregularly shaped item or a deformable item.

[0074] In connection with determining a configuration for the set of items (e.g., a plan corresponding to the configuration) and controlling the robotic arm 102 to pick and place the set of items according to the configuration, the system 100 determines a state space, an action space, and a search space. The system 100 determines the search space based at least in part on determining various place positions and / or orientations for the set of items. The system 100 can further determine the search space based on changes in the order of placement of items within the set of items if the system 100 is configured to enable buffering of one or more items.

[0075] Determining a plan for picking and placing a set of items (e.g., palletizing a set of items) includes determining place locations (e.g., destination locations where the items will be placed) and orientations where the items will be placed. In some embodiments, determining or updating a plan for picking and placing one or more sets of items includes evaluating various place locations and orientations for items in the set of items. The system 100 determines a search space based on (i) a state space for the state of the pallet or other location where the set of items will be placed and (ii) an action space corresponding to placing each item in the set of items into its corresponding destination location and orientation. For example, the system 100 determines a set of plans for palletizing a set of items by determining a configuration (e.g., a place location and orientation, etc.) corresponding to each item included in the set of items. As another example, the system 100 determines multiple combinations / permutations of configurations (e.g., place locations and orientations, etc.) for each item in the set of items (or N items in the set of items, where N is an integer). Determining the plan may further include determining a predicted stability of a stack of items including at least a portion of the set of placed items at the corresponding destination positions and orientations. In some embodiments, system 100 runs multiple simulations, each corresponding to a different combination / permutation for placing / orienting the set of items. The plan is determined at least in part based on the best (e.g., optimal) combination / permutation of destination positions and orientations. The best combination / permutation of destination positions and orientations may be selected based on a cost function such that the cost of the best combination / permutation is the lowest-cost combination / permutation or is less than a cost threshold (e.g., an absolute threshold, a percentile of the cost among various costs for different combinations / permutations, etc.).

[0076] In some embodiments, system 100 determines a search space for placement of a set of N next items, where N is a positive integer. As an example, when an item is delivered to the workspace for a robot to pick and place at a destination location, system 100 may be able to determine the next M items to be placed, where M is a positive integer. M is greater than or equal to N (e.g., the next N items may be a subset of the next M items). System 100 may determine the next M items based on sensor data acquired by one or more sensors (e.g., a vision system) in the workspace. In some embodiments, system 100 determines the next N items based on a manifest or other predetermined list of items to be picked and placed (e.g., palletized).

[0077] In some embodiments, system 100 represents the search space as a tree, with each node corresponding to a different combination of arrangements for the set of items. System 100 determines the search space based at least in part on the state space and the action space. The state space corresponds to the current state of the workspace (e.g., the current state of the pallet). The action space corresponds to the space defined by the arrangements (e.g., place positions and orientations) of the set of items. The root node is the current state of the workspace (e.g., the current state of the pallet). The first step after the root node corresponds to branches / nodes for various permutations of place positions and orientations for the placement of the first item. The second step after the root node corresponds to branches / nodes for various permutations of place positions and orientations for the placement of the second item.

[0078] In some embodiments, system 100 represents the search space as a Markov decision process, where each node corresponds to a different combination of placements for the set of items. For example, if system 100 does not have knowledge of the entire set of items to be picked and placed, system 100 implements a Markov decision process because there is uncertainty regarding future items to be picked and placed.

[0079] The placement (e.g., destination location and orientation) of the set of items is selected by performing a search within a search space. For example, system 100 performs a search within the search space to identify a best / lowest cost solution or a sufficiently good solution (e.g., a solution that meets a predetermined cost threshold). However, traversing the entire search space, including all possible combinations of place locations and orientations, may be extremely computationally expensive and may add significant latency to determining a plan for placing the items. For example, if the search space is represented as a tree, traversing the various branches of the tree may be inefficient. The search space may grow exponentially as the number of items increases, the palette size increases, etc. In some embodiments, system 100 constrains the search space from which system 100 selects placements (e.g., place locations, orientations, etc.). However, constraining the search space too much may result in a suboptimal number of placement combinations / permutations for the set of items (from which the placement of the next item is determined).

[0080] According to various embodiments, system 100 constrains the search space to obtain a more computationally reasonable search space (e.g., to find a more computationally reasonable method for determining optimal locations for placed items). If the search space is represented as a tree, system 100 determines a method for pruning the tree, and system 100 prune the tree. As an example, pruning the tree includes constraining the search space of the tree such that system 100 removes states corresponding to pruned branches / nodes from consideration as possible place locations / orientations. In some embodiments, system 100 limits the analysis of potential configurations (locations and orientations) or searches for the best configuration (e.g., the best combination / permutation of destination locations and orientations, the best destination location and orientation for the placement of the next item that leads to a stable placement of the next set of items, etc.) relative to those configurations in the unpruned search space.

[0081] In some embodiments, system 100 prunes the search space (e.g., a tree) based at least in part on running simulations for various portions of the search space. As an example, each node in the tree corresponds to placing an item at a particular place position and orientation. In some embodiments, system 100 simulates the placement of the items according to the place position and orientation corresponding to the particular node in the tree. As an example, running the simulation includes querying a simulation service. The simulation service may be a physics engine or model trained using a machine learning process based on training data (e.g., information obtained from repeated physical / real-world trials of stacking items). The simulation service may be an application or other process running on system 100 or on a remote system with which system 100 communicates. In response to running the simulation, system 100 determines whether to prune the tree at such a node. System 100 may decide to prune the tree at such a node based at least in part on the predicted stability of the stack of items in response to the placement of the items (e.g., the search space may be pruned to remove / filter out placements deemed physically unstable). In some embodiments, the system 100 obtains (eg, calculates) a cost function value for a placement corresponding to the node.By way of example, the cost function may be based at least in part on one or more of: (i) the predicted stability of the stack of items; (ii) the time for the stack of items to be completed; (iii) the degree to which the stack of items fulfills predetermined criteria or heuristics (e.g., deformable members placed towards the top of the stack, heavy items placed towards the bottom of the stack, irregularly shaped items placed towards the top of the stack, etc.); (iv) collision avoidance or collision prediction (e.g., determining whether the trajectory to the place location will result in a collision between the item or robot arm and another object); (v) efficiency of item movement; and (vi) an indication of whether the robot is predicted to be configured in an improper pose when picking, moving, or placing an item for placement.

[0082] In some embodiments, for each node in the tree (or Markov decision process), the system 100 simulates placing a corresponding item according to its placement at the node. In response to simulating the placement of the corresponding item, the system 100 models a stack of items. For example, the system 100 determines a predicted stability of the stack of items (e.g., the modeled stack of items). In response to determining the predicted stability, the system 100 decides whether to prune the node (and its child nodes / branches). For example, the system 100 decides whether to prune the node based at least in part on comparing the predicted stability to a predetermined stability threshold. If the predicted stability is less than the stability threshold, the system 100 decides to prune the node. As another example, the system 100 decides whether to prune the node based on a value of a cost function (e.g., a cost) corresponding to the placement associated with the node. The system 100 compares the cost to a predetermined cost threshold. If the cost is higher than the predetermined cost threshold, the system 100 decides to prune the node.

[0083] According to various embodiments, system 100 performs a placement simulation for the next item (e.g., the first item in the set of items to be placed). The placement simulation is used in conjunction with determining the first level of nodes branching from a root node (corresponding to the current state of the palette / stack of items). Performing a placement simulation of items is computationally expensive. For example, system 100 queries a physics engine to run the simulation and receive results (e.g., a model of the estimated state). While simulation fidelity is highly desirable, high-fidelity placement simulation (e.g., determining a model of a stack of items based on such placement) is costly (e.g., computationally expensive, time-consuming, etc.).

[0084] In some embodiments, for next-level nodes (e.g., placement of items after the first item in a set of items), system 100 uses one or more heuristics in connection with determining the predicted stability of the estimated state (e.g., stack of items). One or more heuristics may be predefined. For example, one or more heuristics may be predefined based on stacking policies or system preferences. The heuristics may be empirically determined by an administrator and predefined accordingly. In some embodiments, one or more heuristics are based on attributes of corresponding items logically placed (e.g., according to their placement relative to the node) or items in the stack of items. According to various embodiments, system 100 performs item placement simulations only for first-level nodes branching from the root node, and for the N-1 next items, system 100 uses one or more heuristics to determine the predicted stability, or the impact on stability, of the placement of corresponding items according to their placement position and orientation relative to the respective node.

[0085] Example heuristics may include (i) predicted stability based on placement of non-rigid or deformable items at or near the bottom of a stack of items, (ii) predicted stability based on placement of large items at or near the top of a stack of items, (iii) predicted stability based on placement of heavy items at or near the bottom of a stack of items, (iv) predicted stability based on placement of heavy items at or near the bottom of a stack of items, (v) predicted stability based on placement of irregularly shaped items at or near the bottom of a stack of items, and (vi) predicted stability based on placement of irregularly shaped items at or near the top of a stack of items. Various other heuristics may be implemented. As an example, a heuristic suggests that a stack of items will become unstable if a non-rigid or deformable item is placed at or near the bottom of a stack of items. As an example, a heuristic suggests that the stability of a stack of items will not be adversely affected (e.g., by at least the amount of a threshold stability) by placement of a non-rigid or deformable item placed at or near the top of a stack of items. As an example, the heuristic suggests that a stack of items becomes unstable if a heavy item is placed at or near the top of the stack of items. As an example, the heuristic suggests that the stability of a stack of items is not adversely affected (e.g., by at least the amount of threshold stability) by the placement of a heavy item placed at or near the bottom of the stack of items. As an example, the heuristic suggests that a stack of items becomes unstable if an irregularly shaped item (e.g., a non-rectangular item, a round item, etc.) is placed at or near the bottom of the stack of items. As an example, the heuristic suggests that the stability of a stack of items is not adversely affected (e.g., by at least the amount of threshold stability) by the placement of an irregularly shaped item (e.g., a non-rectangular item) placed at or near the top of the stack of items.

[0086] According to various embodiments, the heuristics are computationally efficient variations of physical simulations, e.g., heuristics are defined to be similar to performing a simulation of the placement.

[0087] Using heuristics to determine the predicted stability of a stack of items or the impact on the stability of a stack of items based on a particular configuration is relatively efficient, especially compared to running a simulation of the configuration and determining a model of the stack of items based on the simulation.

[0088] Determining the stacking and predicted stability of items using simulated placements for first-order items (e.g., first-level nodes branching off from the root node) and using heuristics for placement of second and subsequent next items (e.g., second-level nodes and nodes branching off from the second-level nodes) provides an accurate estimate for the placement of the next items and a cost-effective method of filling the rest of the tree (e.g., determining predicted stability of its impact on predicted stability).

[0089] Various embodiments use information about the predicted stability of a stack of items at a node of the tree in connection with whether to prune such node from the tree (or Markov decision process). For example, system 100 traverses the tree starting from the root node, and system 100 determines for the next node whether the stack is predicted to be stable (e.g., stability greater than a stability threshold, heuristics suggesting the stack is predicted to be stable, no heuristics suggesting the stack is predicted to be unstable, etc.). In response to determining that the stack is predicted to be stable at a particular node, system 100 can proceed to a next-level node (e.g., a node branching from the particular node) and determine whether the stack is predicted to be stable at the next-level node. In response to determining that a particular node is unstable (predicted stability less than a stability threshold, heuristics suggesting the stack is predicted to be unstable), system 100 prune the node (and all next-level nodes branching from the particular node).

[0090] In some embodiments, system 100 is configured to allow / enable item buffering, and system 100 determines the search space based at least in part on combinations / permutations of item placement, including changing the order of item placement up to a threshold buffer amount. For example, if system 100 is configured to allow buffering of up to two items, system 100 may determine the search space based on selecting a first item to place from the next three items to be placed. System 100 may determine a node in the search space for each placement order and corresponding combination / permutation of place position and orientation.

[0091] In some embodiments, system 100 is configured to allow / not enable buffering of items, and system 100 determines the search space based at least in part on the placement of each next item in the order in which the set of items is provided to the workspace (e.g., to a robot, such as via a conveyor).

[0092] According to various embodiments, system 100 determines a plan (e.g., destination location and orientation) for placing a first item based at least in part on determining the best placement of the next N items (e.g., placement with the highest predicted stability and / or lowest cost, etc.). System 100 then controls a robot to pick and place the first item according to the plan. In response to placing the first item, system 100 determines a search space for placing the second and subsequent items. For example, the search space has a root node corresponding to a stack of items, where the stack of items includes the first item, and a first-level node in the search space corresponds to the placement of the second item. System 100 then determines the predicted stability (or effect on stability) of the stack of items for nodes in the search space, determines whether / how to prune the search space, and selects the best placement (e.g., highest predicted stability, lowest cost, etc.) for the second item. According to various embodiments, the system 100 can iteratively determine the search space, prune the search space, and determine an item placement for each item in the set of items to be placed.

[0093] In some embodiments, the system includes multiple zones, each with a pallet located in it, and can simultaneously determine the pallet / stack of items on which a particular item is to be placed and pick and place the item onto the selected pallet.

[0094] Although the above examples are discussed in the context of the system palletizing a set of items onto one or more pallets, the robotic system can be utilized in connection with depalletizing a set of items from one or more pallets.

[0095] 2 is a flowchart illustrating a process for palletizing one or more items according to various embodiments. In some embodiments, process 200 is performed, at least in part, by system 100 of FIG.

[0096] At step 210, a set of items is obtained. The set of items may correspond to a set of items to be palletized together on one or more pallets. According to various embodiments, the set of items to be palletized is determined at least in part based on a manifest or instructions for fulfilling an order. For example, in response to receiving an order, a list of items for the order may be generated. As another example, a list of items corresponding to multiple orders to be sent to the same recipient may be generated.

[0097] The items may be located on shelves or other locations within the warehouse. To palletize the items, the items are moved to a robotic system that palletizes the items. For example, the items may be placed on one or more conveyors that move the items into range of one or more robotic arms that palletize the items onto one or more pallets. In response to obtaining the list of items, at least some of the items are associated with a particular robotic arm, a predetermined zone corresponding to the particular robotic arm, and / or a particular pallet (e.g., a pallet identifier, a pallet located in a predetermined zone), etc.

[0098] At step 220, planning (or re-planning) is performed to generate a plan for picking and placing items based on the list of items and available sensor information. The plan may include one or more strategies for fetching one or more items on the list of items and placing such items on corresponding one or more conveyors for transporting the items to the robotic arms. According to various embodiments, the order in which items on the list of items are presented to appropriate robotic arms for palletizing is determined based at least in part on the list of items.

[0099] The order in which items are placed on the conveyor may be based at least loosely on the items and the expected stacks (e.g., modeled estimated states) of the items on one or more pallets. For example, a system that determines the order in which to place items may generate a model of the expected stacks of items and determine the order based on the model (e.g., to transport items that will form the base / bottom of the stack first and incrementally transport items toward the top of the stack). If the items on the list of items are palletized onto multiple pallets, items that are expected to form the base / bottom of each stack (or are otherwise relatively close to the bottom of the stack) may be placed on the conveyor before items that are expected to be substantially in the middle or at the top of the stack. Various items palletized onto multiple pallets may be interspersed with one another, and the robotic system may sort the items upon arrival at the robotic arm (e.g., the robotic arm may pick and place items on the appropriate pallet based at least on the items (e.g., item identifiers or item attributes). Thus, corresponding items at the base / bottom of corresponding stacks may be interspersed with one another, and the various items for each pallet / stack may be placed on a conveyor as the corresponding stack is built.

[0100] The computer system may generate one or more predictive stacking models for items in the list of items. The models may be generated based at least in part on one or more thresholds (e.g., a fitness threshold or a stability threshold, other packing metrics (e.g., density), etc.). For example, the computer system may generate models for stacks whose predicted stability values ​​meet (e.g., exceed) the stability threshold. The models may be generated using a machine learning process. The machine learning process may be iteratively updated based on historical information, such as previous stacks of items (e.g., attributes of items in previous stacks, performance metrics (stability, density, fitness, etc.) for previous stacks). In some embodiments, the stacking models for palletizing items on the list of items are generated based at least in part on one or more attributes of the items.

[0101] Various attributes of the items may be obtained before or during determining the plan. The attributes may include the size of the item, the shape of the item, the type of packaging for the item, an identifier for the item, the center of gravity of the item, an indication of whether the item is fragile, an indication of the top or bottom of the item, etc. As an example, one or more attributes associated with at least some of the items may be obtained based at least in part on a list of items. The one or more attributes may be obtained based at least in part on information obtained by one or more sensors and / or by performing a lookup in a mapping of attributes to items (e.g., item type, item identifier (serial number, model number, etc.)).

[0102] In some embodiments, generating a model of one or more predictive statistics for items belonging to the list of items includes generating (e.g., determining) an estimated state for a workspace (e.g., a workspace including one or more stacks of items). A computer system determines a plan for moving (e.g., palletizing or depalletizing, etc.) one or more sets of items, and the computer system controls a robot (e.g., a robotic arm) to move the one or more sets of items according to the plan. In response to moving the one or more sets of items according to the plan, the computer system determines an estimated state of the workspace. For example, the computer system updates the estimated state based at least in part on the movement of the sets of items. In some embodiments, the estimated state is determined based at least in part on the geometric model or the sensor data, or a combination of the geometric model and the sensor data, in response to determining that the geometric model and the sensor data are inconsistent (e.g., a difference between the geometric model and the sensor data is greater than a predetermined difference threshold, or includes an anomaly, etc.). The updated / current estimated state reflects the movement of the set of one or more items (e.g., in the case of palletizing, the updated estimated state includes information about the placement of the set of one or more items on a stack, etc.). In response to determining the updated / current estimated state, the computer system determines a plan for moving another set of one or more items, and the computer system controls the robot to move the another set of one or more items according to the plan.

[0103] In some embodiments, the computer system updates the current state (e.g., based on updates to the geometric model) after (i) the movement (e.g., placement) of a predetermined number of items, or (ii) the earlier of the movement (e.g., placement) of a predetermined number of items or the detection of an anomaly (e.g., an anomaly that meets one or more anomaly criteria (e.g., the degree of anomaly exceeds an anomaly threshold, etc.)). The predetermined number of items (e.g., X items, where X is a positive integer) may be set based on user selection, robot control system policy, or otherwise determined based on empirical analysis of item placement. As an example, the predetermined number of items is set based on a determination that the number of items will yield optimal / best results with respect to a predetermined cost function (e.g., a cost function reflecting efficiency, stability, predicted changes in stability, etc.). As an example, the computer system determines a current estimated state and uses the current estimated state to determine a plan for moving the next X items, and after the movement (e.g., stacking or de-stacking) of the X items, the computer system determines an updated estimated state (e.g., geometric updates / model to reflect the placement of the X items). The computer system determines an updated state based at least in part on a combination of the geometric model and the sensor data (e.g., the current geometric model and the current sensor data, etc.) The computer system then utilizes the updated state in connection with determining a plan and controlling the robot to place the next set of items according to the plan.

[0104] According to various embodiments, the computer system determines the estimated state based at least in part on performing an interpolation between the geometric model and the sensor data. For example, the system performs an interpolation between a particular portion of the geometric model and a corresponding portion of the sensor data (e.g., the particular portion may correspond to a difference between the geometric model and the sensor data that exceeds a difference threshold or includes an anomaly).

[0105] Various interpolation techniques may be implemented. A particular portion of the geometric model may correspond to a particular point (or set of points) in the point cloud for the geometric model, the corresponding portion of the sensor data may be the sensor data for that particular point in the point cloud for the sensor data, etc. In some embodiments, the system performs adaptive interpolation between the geometric model and the sensor data. In some embodiments, the system performs non-adaptive interpolation between the geometric model and the sensor data. Examples of adaptive interpolation processes include nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, spline interpolation, sinc interpolation, Lanczos interpolation, etc. Various other interpolation processes may be performed in connection with determining the estimated state.

[0106] In some embodiments, the system uses the current estimated state in connection with determining the next placement (e.g., placement of the next item in a set of items to be placed). For example, the system determines a search space of possible placements for the next item based at least in part on the estimated state. In some embodiments, the system uses the estimated state as the root node of a search space (e.g., a tree structure representing the search space, a Markov decision process, etc.), and the system determines various combinations / permutations of the next item or set of items (e.g., a set of N items to be placed next).

[0107] In response to determining the search space, the system determines whether to prune the search space based on, for example, quality of service or other criteria such as the amount of resources allocated to determining a plan for placing the next item or latency. The system prunes the search space to remove combinations / permutations of placements for the set of items that correspond to sub-optimal placements (e.g., placements that are predicted to result in unstable stacks of items, or where the cost of such placements exceeds a predetermined cost threshold). In response to pruning the search space, the system evaluates the remaining combinations / permutations of placements in the search space to identify the best placements (e.g., placements of the next items that are predicted to produce the best placement for the set of items (e.g., the set of N next items)). The best placement may be determined based at least in part on the predicted stability of the stack, the cost for placing the next item (or the cost for placing the set of items), etc.

[0108] In response to determining the placement for the next item, the system determines a plan for placing the item at the corresponding destination location and associated orientation.

[0109] In step 230, the item is picked according to the plan determined and / or updated in step 220, moved through a (predetermined / planned) trajectory to a location near where the item will be placed on the corresponding conveyor, and placed at the destination location.

[0110] In the illustrated example, (re)planning and plan execution (steps 220, 230) continues until the high-level goal of providing an item on the list of items is completed (step 240), at which point process 200 ends. In various embodiments, replanning (step 220) may be triggered by conditions such as the arrival of an unexpected and / or unidentifiable item, a sensor reading indicating an attribute having a value other than that predicted based on the item identification and / or associated item model information, etc. Other examples of unexpected conditions include, but are not limited to, determining that an expected item is missing, reevaluating an item identification to determine that the item is other than that originally identified, detecting an item weight or other attribute that does not match the identified item, dropping an item or needing to be re-grasped, determining that a later-arriving item is too heavy to be stacked on one or more other items as planned by the original plan and / or the current plan, and detecting instability in a set of items stacked in a container.

[0111] 3 is a flowchart illustrating a process for determining a plan for moving an item, according to various embodiments. In some embodiments, process 300 is performed, at least in part, by system 100 of FIG.

[0112] At step 305, a set of items is obtained. In some embodiments, the set of items is determined based at least in part on sensor data (such as information obtained by a vision system in the workspace). The system determines the next item to be placed based at least in part on the sensor data (e.g., the system determines the next N items to be transported to the workspace for palletization, etc.). In some embodiments, the set of items is determined at least in part on a predetermined manifest or list of items to be picked and placed.

[0113] At step 310, the current state of the pallet or stack of items is obtained. In some embodiments, the system determines an estimated current state of the pallet or stack of items. The system can determine the estimated state based on utilizing a geometric model of the stack of items, sensor data of the workspace, or a combination of the geometric model and sensor data. For example, the system performs interpolation on the geometric model and sensor data to determine the estimated state.

[0114] In step 315, the system determines a tree corresponding to scenarios for placement of at least a portion of the set of items. In some embodiments, the system determines a search space corresponding to various combinations / permutations (e.g., place positions and orientations) of placements of the next N items.

[0115] At step 320, the tree is pruned to remove branches and / or nodes corresponding to unfavorable scenarios. In some embodiments, the system determines branches / nodes corresponding to configurations that are predicted to produce unstable stacks of items (e.g., predicted stability less than a predetermined stability threshold, heuristics that suggest the stacks of items are predicted to be unstable, etc.) or where the configurations are predicted to have costs above a cost threshold (e.g., according to a predetermined cost function). In response to determining branches / nodes corresponding to configurations predicted to produce unstable stacks of items or costs above a threshold cost, the system determines to prune such configurations from the search space. For example, the system excludes such configurations from further analysis. In some embodiments, step 320 is performed after step 350.

[0116] In step 325, a scenario in the remaining tree is selected. In some embodiments, the system selects combinations / permutations of configurations in the pruned search space. The system may repeat steps 325-360 until all remaining scenarios in the pruned search space have been analyzed. In some embodiments, the scenarios correspond to nodes in a tree structure representing the search space. In some embodiments, the scenarios correspond to nodes in a Markov decision process representing the search space.

[0117] In step 330, an item to place is determined. In some embodiments, the system determines the next item to place. In some implementations, the system may allow for item buffering, in which case the system determines the next item from the set of next items that fall within the buffer criteria.

[0118] Configurations in which items may be placed are determined for the current scenario in step 335. The configurations may correspond to different place positions and orientations in which items may be placed.

[0119] A location and / or orientation for placing the item relative to the scenario is selected in step 340. As an example, the system selects a node in the search space that corresponds to the placement of the item and determines the location and / or orientation corresponding to the selected node.

[0120] At step 345, the pick-and-place of the item is simulated. In some embodiments, in response to selecting a placement for the item (e.g., determining a placement corresponding to a node in the search space), the system simulates the pick-and-place of the item. The system may utilize a geometric model in connection with simulating the pick-and-place of the item. If the item to be placed is a primary item, the system models placing the item into an existing stack of items using a geometric model corresponding to the current estimated state. If the item to be placed is a successor to the primary item, the system simulates placement of such item based on utilizing a geometric model corresponding to the current estimated state updated to reflect the logical (e.g., theoretical) placement of the primary item.

[0121] In step 350, one or more characteristics corresponding to the stack of items are determined. For example, the system determines characteristics of the stack of items corresponding to the scenario based at least in part on a model of the stack of items generated based on a simulation of the placement of the items. Examples of characteristics for the stack of items include (i) predicted stability, (ii) cost, (iii) time to perform the placement for the scenario, (iv) an indication of whether a collision is predicted to occur if the placement is performed, (v) an indication of whether the robot is predicted to be placed in an improper or inefficient pose during item placement, etc.

[0122] At step 355, the system determines whether further item placement simulations are to be performed. For example, the system determines whether any items to be placed according to the scenario remain in the set of items (or the set of the next N items). In response to determining that further item placement simulations are to be performed, process 300 returns to step 330, and process 300 repeats steps 330-355 until no further item placement simulations are to be performed for the selected scenario. In response to determining that no further items exist, process 300 proceeds to step 360.

[0123] At step 360, the system determines whether there are any additional scenarios for the arrangement of the set of items. For example, the system determines whether any other orders or combinations / permutations of stacking the items remain in the search space. In response to determining that there are additional scenarios, process 300 returns to step 325, and process 300 repeats steps 325-360 until there are no additional scenarios. In response to determining that there are no additional scenarios, process 300 proceeds to step 365.

[0124] In step 365, various scenarios within the search space are compared, and a best scenario is determined. The system can determine the best scenario (e.g., the set of items or the placement of the next item that is predicted to produce the best result) based at least in part on one or more features corresponding to the stack of items for the various scenarios. For example, the system determines the placement that produces the highest predicted stability. In another example, the system determines the placement that produces the lowest cost according to a predetermined cost function. As another example, the system determines the best scenario to be the first placement traversed in the search whose predicted stability meets a stability criterion (e.g., stability above a stability threshold, no heuristics that suggest the stack of items is unstable, etc.) and / or meets a cost criterion.

[0125] Pick and place of the item is performed according to the best case scenario in step 370. In some embodiments, the system determines a plan for placing the item based on the place location and orientation relative to the scenario.

[0126] 4A illustrates a tree corresponding to different states or placements of items, according to various embodiments. In some embodiments, the system 100 of FIG. 1 determines or utilizes a search space 400 in connection with determining the placement of a current item (e.g., the next item in a set of items to be placed).

[0127] 4A, the search space 400 is represented in a tree structure. The search space 400 includes multiple nodes, each of which may correspond to an item placement scenario, and each branch of the search space 400 represents a specific position and may correspond to the placement of the current item in a particular orientation.

[0128] The search space 400 includes a root node 402. The root node 402 corresponds to the current state of a palette / stack of items. For example, the system queries a state estimation service / module to obtain an estimated state and uses such estimated state as the state in the root node 402.

[0129] In response to determining the root node, the system iteratively determines placements for the next set of items. For example, the system progresses through various orders of the tree based on various next-item placements. Nodes 404, 406, 408, and 410 correspond to placements of the primary items (e.g., the items for which the system has determined the best placement). As an example, nodes 404, 406, 408, and 410 are primary nodes that correspond to different placements of the primary items (e.g., different positions and / or orientations, or different items if the system is allowed to buffer some items). As shown in FIGS. 4A and 4B, each of nodes 404, 406, 408, and 410 has secondary nodes branching off from it. For example, nodes 412, 414, 416, and 418 branch off from node 404. Each secondary node corresponds to a different arrangement of secondary items (e.g., different positions and / or orientations, or different items if the system is allowed to buffer some items). Once the system determines the arrangement corresponding to each node, it determines nodes branching from the node for the arrangement of the remaining items to be simulated. For example, in response to determining the arrangement for node 412, the system determines nodes 420, 422, 424, and 426 (e.g., the system determines estimated states and associated information for the nodes).

[0130] If the system analyzes stability at each node in the search space 400, such analysis can be computationally expensive. 。 Thus, as shown in FIG. 4B, various embodiments prune the search space to limit the number of nodes whose state / stability is determined in relation to determining the placement of an item (e.g., a primary item).

[0131] 4B illustrates a tree corresponding to different states or placements of items, according to various embodiments. In some embodiments, the system 100 of FIG. 1 determines or utilizes a search space 430 in connection with determining the placement of a current item (e.g., the next item in a set of items to be placed).

[0132] The system determines search space 430 based on determining whether and / or how to prune search space 400. Partitioned regions 442, 444, 446, and 448 correspond to regions of the tree that the system has determined to be unfavorable scenarios and therefore should be pruned from search space 400 (e.g., tree).

[0133] In some embodiments, the system determines whether a node corresponds to a favorable or unfavorable scenario based on one or more of: (i) simulating the state of the stack of items for a configuration associated with the node; and (ii) one or more heuristics. In some embodiments, the system simulates the state of the stack of items for a primary node (e.g., 404, 406, 408, and 410) in connection with determining the predicted stability of the stack of items. In some embodiments, for all nodes having a higher order than the primary node (e.g., 412, 414, 416, 418, 420, 422, 424, 426, etc.), the system uses one or more heuristics to determine whether the corresponding configuration is predicted to be an unfavorable scenario (e.g., whether it is predicted to lead to a stable or unstable stack, whether it is predicted to have a significant cost (e.g., a cost higher than a cost threshold), whether it is predicted to place the robot in an improper pose, etc.).

[0134] According to various embodiments, in response to determining the results of an analysis of the state, predicted stability, and / or favorability of the placement based on one or more heuristics, the system determines whether to prune the corresponding node. If the system determines to prune a node, it prunes the node and all potential nodes branching from the node. For example, in response to determining that node 426 is to be pruned (e.g., the node corresponds to an unfavorable scenario), the system correspondingly determines to prune nodes 432, 434, 436, etc. based on such nodes directly or indirectly branching from node 426. In some embodiments, the system determines to prune nodes 432, 434, 436, etc. without further analysis using one or more heuristics. For example, the system determines to prune nodes 432, 434, 436, etc. based on the determination to prune node 426.

[0135] In some embodiments, after determining the search space 430 (e.g., after pruning the search space 400), the system determines the best placement for the next item (e.g., the item relative to the primary node). The system can determine the best placement based on one or more of the predicted stability, the cost of the placement, the cost of placing the next item and M items after placing the next item, the predicted stability of the stack of items after placing the M items after placing the next item and the next item, and the desirability of the placement.

[0136] In some embodiments, the system determines that the best placement of the next item corresponds to a placement at node 408. For example, the system determines that the placement at node 408 is best based on determining that the state of the stack of items remains favorable (e.g., cost-effective, stable, etc.) through several placement orders (e.g., the state remains favorable for the placement of the most subsequent items). In some embodiments, the system determines that nodes 438 and 440 remain favorable through the placement of nine different items (e.g., the next item and M items after the next item, where M is 8 in this example). In some embodiments, the system performs a beam search with respect to search space 430 to determine the best placement.

[0137] In connection with determining whether to prune a node, the system can implement pruning criteria, which may include one or more of: (i) the state is predicted to have stability less than a state stability threshold, (ii) the state violates a heuristic (e.g., a deformable item is placed at or near the bottom of a stack of items, etc.), (iii) the cost of the corresponding placement is predicted to exceed a cost threshold, (iv) the robot is predicted to be placed in an improper pose, (v) the predicted density of the stack of items is less than a predetermined density threshold, etc.

[0138] 4C is a diagram illustrating a stack of items with potential placements, according to various embodiments. In some embodiments, the system 100 of FIG. 1 determines a model 450 of the stack of items.

[0139] In some embodiments, the system determines the model 450 based on a geometric model and / or sensor data. The system can obtain the model 450 using a machine learning model.

[0140] As shown in FIG. 4C , the system can use the current estimated state (e.g., based on a geometric model and / or sensor data) to determine edges and vertices that define various items and / or potential placement locations within the stack of items. For example, the system performs image processing or machine learning analysis to identify edges (such as edges 452, 454, 456, 458, 460, 462, 464, 466, and 468) and / or vertices where edges meet. In response to identifying one or more edges, the system determines a potential placement for the current item (e.g., the first item). The system may also determine potential placements for a future set of items (e.g., the set of N next items).

[0141] In some embodiments, the system may utilize one or more rules or heuristics in connection with determining the placement of items. Examples of rules or heuristics include (i) a bias / preference for placing items on edges, (ii) a bias / preference for placing items next to items, (iii) a top surface area of ​​the surface on which an item may be placed that is greater than the bottom surface area of ​​the item being placed, (iv) a top surface area of ​​the surface area on which an item may be placed that is greater than a threshold percent of the bottom surface area of ​​the item being placed, (v) a bias / preference for placing large or heavy items at or near the bottom of a stack of items, such as within a threshold distance of the pallet surface, (vi) a bias / preference for placing non-rigid / deformable items at or near the top of a stack of items, such as within a threshold distance of a maximum stack height, (vii) an indication that an item will fit within a potential position, and (viii) an indication of whether the top surface of an item, when placed in a position, will be substantially level with adjacent items (e.g., within a threshold height of adjacent items), etc. Various other rules / heuristics may be implemented.

[0142] 4C, the system determines that an item (e.g., the current item) can be placed in potential locations 470, 474, and 476. In the illustrated example, the identified potential locations 470, 474, and 476 are located along the edge defined by the top surface (e.g., exposed surface) of the stack of items.

[0143] In some embodiments, in response to determining the possible placements 470, 474, and 476, the system uses a machine learning model to evaluate the placements. For example, the system uses the machine learning model to determine a score for the placement based on a scoring function.

[0144] 4D is a diagram illustrating a stack of items with potential placements, according to various embodiments. In some embodiments, the system 100 of FIG. 1 determines a model 480 of the stack of items.

[0145] In some embodiments, the system determines model 480 based on a geometric model and / or sensor data. The system can obtain model 480 using a machine learning model. In some embodiments, the system evaluates locations 482 and 484 in relation to determining whether such locations are possible placements for item 486. As shown in FIG. 4D , item 486 is an irregularly shaped object. For example, item 486 may be a deformable member and / or a non-rigid type of packaging (e.g., a polybag). Stacking items on irregularly shaped item 486 may cause instability in the stack of items.

[0146] With respect to location 482, the system may determine that such a location is not feasible at least because the surface is substantially narrower than the width of item 486 (e.g., the top surface area corresponding to location 482 is less than a threshold percentage of the bottom surface area of ​​item 486). If the top surface of a location is less than a threshold percentage of the bottom surface area of ​​the item to be placed, placing the item in such a location may lead to instability.

[0147] 5 is a flowchart illustrating a process for determining a tree corresponding to the placement of a set of items, according to various embodiments. In some embodiments, process 500 is performed by system 100 of FIG. 1. In some embodiments, process 500 is invoked in connection with determining a search space for the placement of the set of items. For example, process 500 is invoked in response to determining that an item is to be placed on a palette or stack of items. As an example, the system may determine a search space for each item to be placed (e.g., process 500 is invoked for each item in the set of items to be placed).

[0148] In step 505, an estimated state is obtained. In some embodiments, the system obtains the current estimated state of the workspace (e.g., a pallet or stack of items). For example, the system can query a state estimation service / module for the estimated state. The estimated state may include a geometric model of the pallet / stack of items.

[0149] In step 510, an item is determined to be placed in a location corresponding to a node in the search space. For example, the system determines the next item to be placed. In some embodiments, the next item may correspond to the first item being transported to the workspace (e.g., by a conveyor). In some embodiments, the next item is selected from a set of items (e.g., a set of N items) based on conditions for item buffering. Conditions for item buffering may include (i) determining whether buffering is allowed in the system, (ii) the number of items that can be buffered, (iii) determining the items currently in the buffer, etc. In some embodiments, the item is determined based on a determination of a location corresponding to a node in the search space (e.g., in a tree structure representing the search space). For example, each node in the search space may correspond to a particular location (e.g., position and orientation) of a particular item.

[0150] In step 515, a destination location where the item will be placed is determined. In response to determining the item to be placed, the system selects a destination location from a set of possible destination locations where the item may be placed. As an example, the system determines a destination location that corresponds to placement for a particular node in the search space.

[0151] In step 520, an orientation in which the item will be placed at the destination location is determined. In some embodiments, the system selects an orientation from a set of possible orientations in which the item may be placed at the destination location. As an example, the system determines an orientation that corresponds to placement for a particular node in the search space.

[0152] At step 525, nodes are created in the search space based on the destination location and orientation for item placement. In some embodiments, the system represents the search space in a tree representation or as a Markov decision process, where the tree is configured to include nodes corresponding to destination locations and orientations for item placement.

[0153] At step 530, a determination is made as to whether the item can be placed in additional orientations. The system determines whether there are additional orientations for which placements are included in the search space (e.g., other additional orientations in which the item can be placed at the selected destination location). In response to determining at step 530 that there are additional orientations, process 500 returns to step 520 where another orientation is selected, and process 500 can repeat steps 520-530 until there are no additional orientations for placement of the item at the destination location.

[0154] At step 535, a determination is made as to whether the item can be placed at an additional destination location. The system determines whether additional destination locations exist for placement within the search space. In response to determining at step 535 that additional destination locations exist, process 500 returns to step 515 where another destination location is selected, and process 500 can repeat steps 515-535 until no additional destination locations exist for placement of the item.

[0155] At step 540, a determination is made as to whether placements for additional items are to be evaluated. In some embodiments, the system determines whether the set of items to be placed includes an item for which a set of placements has not yet been determined. In some embodiments, the system determines placements for a set of N next items, and the system determines whether the set of N next items includes an item for which a set of placements has not yet been determined. In response to determining at step 540 that placements for additional items are to be evaluated, process 500 continues to step 5 where another item is selected. 10 Return to step 5 until there are no more placements to evaluate for the item. 10 540 can be repeated. In response to determining at step 540 that there are no more items to be evaluated for placement, process 500 proceeds to step 545.

[0156] In step 545, a search space is obtained based at least in part on the nodes corresponding to the arrangements of the items. For example, the system determines a search space that includes nodes corresponding to various arrangements (e.g., destination positions and / or orientations) of the set of items. The system can utilize the search space in connection with determining the arrangement of the current items.

[0157] 6 is a flow chart illustrating a process for pruning nodes of a tree for placement of a set of items, according to various embodiments. In some embodiments, process 600 is performed by system 100 of FIG. 1. In some embodiments, process 600 is invoked in connection with determining a placement and / or a plan for placing items.

[0158] In step 605, a search space for various states for placing the set of items is obtained. In some embodiments, the system determines the search space for various combinations / permutations of placement of the set of items. As an example, the search space is obtained based on invoking process 500 of Figure 5. In some embodiments, the search space is represented as a tree structure or a Markov decision process.

[0159] At step 610, a search space node is selected for the current item placement. In some embodiments, the search space node corresponds to a particular item placement. For example, the search space node corresponds to a particular item placement in a particular location and orientation. As another example, the search space node corresponds to the state of a stack of items according to a particular item placement in a particular location and orientation.

[0160] In step 615, the model is used to simulate the placement of the current items according to the state of the node. In response to determining the place position and orientation associated with a particular node, the system simulates the placement of the items according to the position and orientation. The model can be a geometric model that is updated based on the placement of the current items. In some embodiments, the system queries a state estimation service / module for an updated state taking into account the placement.

[0161] At step 620, the stability of the stack of items is determined based on the current item placement. In some embodiments, the system determines the predicted stability of the stack of items based on simulating the placement (e.g., invoking a physics engine to simulate stability) and / or one or more heuristics.

[0162] At step 625, a determination is made as to whether the stability of the stack of items is greater than a stability threshold. In response to modeling the placement of the items relative to the nodes, the system determines a predicted stability of the stack of items and compares the predicted stability to a predetermined stability threshold. In response to determining at step 625 that the stability (e.g., predicted stability) of the stack of items is greater than the predetermined threshold, process 600 proceeds to step 630. Conversely, in response to determining at step 625 that the stability of the stack of items is not greater than the predetermined threshold, process 600 proceeds to step 650.

[0163] At step 630, a downstream node corresponding to the next item to be placed is selected. In some embodiments, the system selects a node branching from the node selected at step 610. If the node selected at step 610 is a primary node (e.g., a node branching directly from the root node of the tree / search space), the node selected at step 630 (at least at the first iteration of step 630) may correspond to a secondary node, and in later iterations of step 630, may include nodes branching from the secondary node (e.g., tertiary nodes, quaternary nodes, etc.). In some embodiments, the node selected at step 630 corresponds to the placement of an item to be placed after the current item (e.g., the first item in the set of items to be placed).

[0164] In step 635, heuristics are used to evaluate the placement of the next item according to the state of the node. In some embodiments, the system uses heuristics to generally determine whether the placement of the next item is predicted to adversely affect the stability of the stack of items. In some embodiments, the system uses one or more heuristics to determine whether the stack of items is predicted to be stable (e.g., remain stable) after the placement of the next item.

[0165] In some embodiments, for higher level nodes (e.g., placement of an item after a first item in a set of items), the system uses one or more heuristics in connection with determining the predicted stability of the estimated state (e.g., stacking of items). The one or more heuristics may be predefined. For example, the one or more heuristics may be defined based on stacking policies or system preferences. The heuristics may be empirically determined by an administrator and predefined accordingly.

[0166] Example heuristics may include (i) predicted stability based on placement of non-rigid or deformable items at or near the bottom of a stack of items, (ii) predicted stability based on placement of large items at or near the top of a stack of items, (iii) predicted stability based on placement of heavy items at or near the bottom of a stack of items, (iv) predicted stability based on placement of heavy items at or near the bottom of a stack of items, (v) predicted stability based on placement of irregularly shaped items at or near the bottom of a stack of items, and (vi) predicted stability based on placement of irregularly shaped items at or near the top of a stack of items. Various other heuristics may be implemented. As an example, a heuristic suggests that a stack of items will become unstable if a non-rigid or deformable item is placed at or near the bottom of a stack of items. As an example, a heuristic suggests that the stability of a stack of items will not be adversely affected (e.g., by at least the amount of a threshold stability) by placement of a non-rigid or deformable item placed at or near the top of a stack of items. As an example, the heuristic suggests that a stack of items becomes unstable if a heavy item is placed at or near the top of the stack of items. As an example, the heuristic suggests that the stability of a stack of items is not adversely affected (e.g., by at least the amount of threshold stability) by the placement of a heavy item placed at or near the bottom of the stack of items. As an example, the heuristic suggests that a stack of items becomes unstable if an irregularly shaped item (e.g., a non-rectangular item, a round item, etc.) is placed at or near the bottom of the stack of items. As an example, the heuristic suggests that the stability of a stack of items is not adversely affected (e.g., by at least the amount of threshold stability) by the placement of an irregularly shaped item (e.g., a non-rectangular item) placed at or near the top of the stack of items.

[0167] At step 640, the stability of the simulated placement of the next item is determined. In some embodiments, the system uses one or more heuristics to determine the predicted stability of the stack of items after placement of the next item. As an example, the system uses one or more heuristics to qualitatively determine the predicted stability of the stack of items. The system can use one or more heuristics to determine whether placing the item in the corresponding destination position and orientation will destabilize the stack of items, whether placing the item will improve stability, etc.

[0168] At step 645, a determination is made as to whether the stability of the stack of items is greater than a stability threshold. In response to modeling the placement of the items relative to the nodes, the system determines a predicted stability of the stack of items and compares the predicted stability to a predetermined stability threshold. In response to determining at step 645 that the stability (e.g., predicted stability) of the stack of items is greater than the predetermined threshold, process 600 proceeds to step 655. Conversely, in response to determining at step 645 that the stability of the stack of items is not greater than the predetermined threshold, process 600 proceeds to step 650.

[0169] At step 650, the corresponding node is pruned. In response to determining that the stack of items is not sufficiently stable (e.g., the predicted stability is less than a stability threshold), the system decides to prune a node from the search space. In some embodiments, pruning a node includes pruning a current node (e.g., for the placement of the next item selected at step 630 or for the placement of the current item selected at step 610). In some embodiments, pruning a node includes pruning the current node and nodes branching from the current node. The system prunes the corresponding node to remove such node as a potential placement for the set of items.

[0170] At step 655, a determination is made as to whether additional downstream nodes exist. For example, the system determines whether there are nodes that branch directly or indirectly from the node selected at step 610. In response to determining that additional downstream nodes exist, process 600 returns to step 630, where it repeats steps 630-655 until no additional downstream nodes exist. In response to determining that no additional downstream nodes exist, process 600 proceeds to step 660, where the system determines whether there are additional nodes that correspond to the current item's placement. For example, the system determines whether the search space (e.g., tree) has additional nodes that branch directly from the root node. The other nodes for the current item may correspond to different placements (e.g., different placement positions and / or different orientations).

[0171] In response to determining at step 660 that there are more nodes for placement of the current item, process 600 returns to step 610, and process 600 repeats steps 610-660 until there are no more nodes for placement of the current item.

[0172] 7 is a flowchart illustrating a process for pruning nodes of a tree for placement of a set of items, according to various embodiments. In some embodiments, process 700 is performed by system 100 of FIG. 1. In some embodiments, process 700 is invoked in connection with determining a placement and / or a plan for placing items.

[0173] In step 705, a search space for various states for placing the set of items is obtained. In some embodiments, the system determines the set of items and determines a search space for placement of the set of items. For example, the system determines various combinations / permutations for placement of the set of items (or a subset of the N next items in the set of items to be placed).

[0174] Nodes in the search space are selected at step 710. In some embodiments, the search space is represented as a tree, and the system sequentially selects nodes branching from a root node (e.g., a primary node) and each node branching directly or indirectly from the primary node.

[0175] In step 715, a placement of items corresponding to the node is simulated according to the state of the selected node. The selected node corresponds to a particular placement of items (e.g., a particular place position and / or orientation). The system simulates the placement of items according to the placement relative to the node to determine a probable state of the stack of items or one or more characteristics associated with the stack of items (e.g., predictive stability, results of heuristic analysis, etc.).

[0176] In some embodiments, simulating the placement of the items includes querying a state assessment service / module to provide a model of the state of the stack of items that reflects the placement of the items, or qualitatively determining the state of the stack of items based on one or more heuristics.

[0177] At step 720, the system determines the stability of the stack of items based on the placement of the items (e.g., based on the state of the stack of items corresponding to the selected node). According to various embodiments, the system determines the predicted stability of the stack of items. As another example, the system determines the impact on stability based at least in part on one or more heuristics.

[0178] At step 725, a determination is made as to whether the stability of the stack of items is greater than a stability threshold. In response to modeling the placement of the items relative to the nodes, the system determines a predicted stability of the stack of items and compares the predicted stability to a predetermined stability threshold. In response to determining at step 725 that the stability (e.g., predicted stability) of the stack of items is greater than the predetermined threshold, process 700 proceeds to step 735. Conversely, in response to determining at step 725 that the stability of the stack of items is not greater than the predetermined threshold, process 700 proceeds to step 730.

[0179] The corresponding node is pruned at step 730. In response to determining that the stack of items is not sufficiently stable (e.g., the predicted stability is less than a stability threshold), the system decides to prune a node from the search space.

[0180] At step 735, a determination is made as to whether there are additional nodes in the search space for which stability is to be determined / regenerated. In response to determining that there are additional nodes, process 700 returns to step 710, and process 700 may repeat steps 710-735 until the system has determined / analyzed the stability for all nodes (all remaining nodes, including unpruned nodes).

[0181] In some embodiments, in response to determining that there are no more nodes in step 735, the system can evaluate the search space to determine a configuration that produces the best or optimal result (e.g., best configuration). For example, the system analyzes the remaining nodes in the search space (e.g., unpruned nodes) to select the best configuration. In response to determining the best configuration, the system can determine a plan for placing the current item according to the best configuration, and the system can further control the robot to pick and place the item according to the plan.

[0182] 8 is a flow chart illustrating one embodiment of determining an estimate of the condition of a pallet and / or stack of items. In some embodiments, process 800 is performed, at least in part, by system 100 of FIG.

[0183] In some embodiments, process 800 is performed by one or more of an app 802, a server 804, a state estimator 806, a vision system 808, and a position determiner 810 running on a control system for the robotic arm.

[0184] At step 820, the app 802 sends a request to the server 804. The request may correspond to a placement request that calls for a plan and / or strategy for placing an item.

[0185] In response to receiving the placement request, the server 804 invokes state determination at step 822. For example, the server 804 sends a request or command to the state estimator 806 to determine (and provide) an estimated state. In some embodiments, the state estimator 806 is a module running on the server 804. In some embodiments, the state estimator 806 is a service that is queried by multiple different server / robot systems. For example, the state estimator 806 may be a cloud service.

[0186] In response to invoking the state determination, the state estimator 806 obtains the vision state. In some embodiments, the state estimator 806 sends a request for the vision state to the vision system 808.

[0187] In response to receiving the vision state request in step 824, the vision system 808 provides the vision state to the state estimator 806 in step 826. For example, in response to receiving the vision state request, the vision system captures a snapshot of the workspace using one or more sensors in the workspace.

[0188] In response to receiving the vision state, the state estimator 806 determines a pallet state (e.g., an estimated state of the pallet and / or stack of items). The state estimator 806 may determine the estimated state based on one or more of the geometric model and the vision state. In some embodiments, the state estimator 806 combines the geometric model and the vision state (for at least a portion of the stack).

[0189] In step 828 , the state estimator 806 provides the palette state to the server 804 .

[0190] In step 830, server 804 sends a placement request including the pallet status to placement determiner 810. In some embodiments, placement determiner 810 is a module running on server 804. In some embodiments, placement determiner 810 is a service that is queried by multiple different server / robot systems. For example, placement determiner 810 may be a cloud service.

[0191] In step 832, the placement determiner 810 provides the set of one or more potential placements to the server 804. The set of one or more potential placements may be determined based at least in part on the items to be placed (e.g., attributes associated with the items) and pallet conditions (e.g., available locations and attributes of items within the stack of items), etc.

[0192] In some embodiments, the set of one or more potential placements is a subset of all possible placements. For example, the placement determiner 810 may use a cost function to determine the set of one or more potential placements for provision to the server 804. The placement determiner 810 may determine potential placements that meet a cost criterion with respect to the cost function (e.g., have a cost less than a cost threshold).

[0193] In response to receiving the set of one or more potential placements, in step 834, the server 804 selects a placement and sends the selected placement to the app 802. For example, the selected placement may be provided as a response to the initial placement request in step 820.

[0194] The app 802 controls the robotic arm to place the item in step 836. In some embodiments, the app 802 determines a plan for moving the item to the selected location (e.g., based on attributes of the item and a location (e.g., coordinates in the workspace) corresponding to the selected location).

[0195] In step 838, the app 802 provides instructions to the server 804 to perform updates related to the geometric state. For example, the app 802 provides confirmation that the placing of the item was performed in step 836, and the server 804 considers such confirmation an indication that an update to the geometric state (e.g., the geometric model) is to be invoked.

[0196] In step 840, the server 804 sends a request to update the geometric state to the state estimator 806. For example, the server 804 requests that the state estimator 806 update the geometric model to reflect the placement of items according to the corresponding plan.

[0197] In response to receiving a request to update the geometric state, the state estimator 806 performs the corresponding update. At step 842, the state estimator 806 provides an indication to the server 804 that the geometric state was successfully updated.

[0198] In step 844, the server 804 provides an indication to the app 802 that the geometric state has been successfully updated to reflect the placement of the items.

[0199] The process 800 may be repeated for the set of items to be stacked.

[0200] While the above examples are described in the context of palletizing or depalletizing a set of items, various embodiments may be implemented in connection with singulating a set of items and / or kitting a set of items. For example, various embodiments are implemented to determine / estimate a state of a workspace (e.g., a chute, conveyor, bin, etc.) based at least in part on geometric data and sensor data (e.g., a combination of geometric data and sensor data, such as interpolation between the geometric data and the sensor data).

[0201] Various example embodiments described herein are described with reference to flowcharts. While the examples may include some steps performed in a particular order, according to various embodiments, various steps may be performed in different orders and / or various steps may be combined into a single step or performed in parallel.

[0202] Although the above-described embodiments have been described in some detail for ease of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and are not intended to be limiting.

Claims

1. 1. A robotic system comprising: a communications interface configured to receive sensor data from one or more sensors disposed in a workspace indicative of a current state of the workspace, the workspace including a pallet or other container and a set of zero or more items stacked on or in the container; one or more processors connected to the communication interface; Equipped with the one or more processors: using the sensor data to estimate a state of one or both of the pallet or other container and the set of zero or more items stacked on or in the container; using the estimated states to generate or update a plan for controlling a robotic arm to place or remove a next set of items onto or into the pallet or other container; the plan is configured to include an ordered sequence of item placement or removal; the one or more processors are configured to generate or update the plan, at least in part, by performing a bounded tree search in which a subset of possible ordered sequences is searched; the subset is determined by one or both of selecting branches of the search tree that satisfy one or more selection criteria and / or eliminating from consideration one or more branches that satisfy pruning criteria, the pruning criteria being related to determining that a branch does not contain a node corresponding to a preferred scenario; the pruning criteria being related to the instability of one or more items stacked on or in the container; The robotic system, wherein the instability is determined based at least in part on a simulation of stacking or removing at least one of the zero or more items stacked on or in the container.

2. The robotic system of claim 1 , wherein placing an item comprises placing the item at a specific location.

3. The robotic system of claim 1 , wherein placing an item includes placing the item in a specific orientation.

4. The robot system according to claim 3, the item is a box; The arrangement of the item is in one of six orientations.

5. The robotic system of claim 1 , wherein the instability is further determined based at least in part on one or more pieces of historical information.

6. The robotic system of claim 1 , wherein the instability is further determined based at least in part on one or more predetermined heuristics.

7. 7. The robotic system of claim 6, wherein the at least one heuristic includes a suggestion to stack items having weights above a predetermined weight threshold on top of items having deformable packaging.

8. The robotic system of claim 1 , wherein the pruning criterion is associated with a packing density that is less than a predetermined packing density threshold.

9. 2. The robotic system of claim 1, wherein the pruning criterion is associated with a determination that stacking items according to a particular sequence of the subset of possible ordered sequences is predicted to cause collisions.

10. 2. The robotic system of claim 1, wherein the pruning criterion is associated with a determination that stacking items according to a particular sequence of the subset of possible ordered sequences is predicted to cause collisions with other objects or movement of the items due to collisions with the other objects.

11. The robotic system of claim 1 , wherein the preferred scenario is determined to be a configuration for which a corresponding value of a value function exceeds a predetermined threshold.

12. 12. The robotic system of claim 11, wherein the value function is based at least in part on one or more of packing density, predicted stability, and predicted efficiency of item placement.

13. The robotic system of claim 1 , wherein the one or more selection criteria are associated with a predicted packing density exceeding a predetermined density threshold.

14. The robotic system of claim 1 , wherein the one or more selection criteria are associated with one or more predetermined heuristics.

15. 15. The robotic system of claim 14, wherein the one or more predetermined heuristics include a bias to place heavy or large items at the bottom of a stack.

16. The robotic system of claim 1 , wherein the one or more selection criteria are related to predicted stability.

17. 17. The robotic system of claim 16, wherein the predicted stability is associated with the topography of a layer or layers within a stack.

18. 2. The robotic system of claim 1, wherein the one or more selection criteria are associated with first exploring a branch that includes first locating a first subset of one or more items far away from the robot, followed by locating a second subset of one or more items closer to the robot.

19. 20. The robotic system of claim 18, wherein the first subset is determined based at least in part on performing a beam search of the tree branches, the beam search being limited to N possible configurations, where N is a positive integer.

20. The robot system according to claim 1, each node in the tree of possible ordered sequences corresponds to a scenario of the placement of items up to that point in the ordered sequence; Each branch of said tree of possible ordered sequences corresponds to a decision to place the next item in a particular position or in a particular orientation of the robotic system.

21. 1. A method for controlling a robot, comprising: receiving sensor data from one or more sensors disposed in a workspace indicative of a current state of the workspace, the workspace including a pallet or other container and a set of zero or more items stacked on or in the container; using the sensor data to estimate a state of one or both of the pallet or other container and the set of zero or more items stacked on or in the container; using the estimated states to generate or update a plan for controlling a robotic arm to place or remove a next set of items onto or into the pallet or other container, wherein the plan includes an ordered sequence of item placement or removal; generating or updating the plan includes performing a bounded tree search in which a subset of possible ordered sequences is searched; the subset is determined by one or both of selecting branches of the search tree that satisfy one or more selection criteria and / or eliminating from consideration one or more branches that satisfy pruning criteria, the pruning criteria being related to determining that a branch does not contain a node corresponding to a preferred scenario; the pruning criteria being related to the instability of one or more items stacked on or in the container; The method, wherein the instability is determined based at least in part on a simulation of stacking or removing at least one of the zero or more items stacked on or in the container.

22. 1. A computer program product for controlling a robot, embodied in a non-transitory computer-readable medium, comprising: computer instructions for receiving sensor data from one or more sensors disposed in a workspace indicative of a current state of the workspace, the workspace including a pallet or other container and a set of zero or more items stacked on or in the container; computer instructions for using the sensor data to estimate a state of one or both of the pallet or other container and the set of zero or more items stacked on or in the container; and computer instructions for using the estimated states to generate or update a plan for controlling a robotic arm to place or remove a next set of items onto or into the pallet or other container, the plan including an ordered sequence of item placement or removal; generating or updating the plan includes performing a bounded tree search in which a subset of possible ordered sequences is searched; the subset is determined by one or both of selecting branches of the search tree that satisfy one or more selection criteria and / or eliminating from consideration one or more branches that satisfy pruning criteria, the pruning criteria being related to determining that a branch does not contain a node corresponding to a preferred scenario; the pruning criteria being related to the instability of one or more items stacked on or in the container; The computer program product, wherein the instability is determined based at least in part on a simulation of stacking or removing at least one of the zero or more items stacked on or in the container.

Images