Automated product unloading, handling and distribution

Abstract

Disclosed is a robotic system including a telescoping transport conveyor with an automated unloader attached. The automated unloader includes a loading conveyor with at least two articulated robots attached to a first loading conveyor end. The automated unloader includes a control system having logic to control the loading and unloading of materials to be handled from a shipping container to the telescoping transport conveyor and thence to an automated palletizing system, where the materials are loaded onto pallets and supported by pallet sleeves during storage or during transport to a stabilizing system where the palletized load is stretch wrapped. Also disclosed are methods of controlling and operating the loading and unloading of products to fully automate their handling and distribution throughout a material handling facility.

Description

[Technical field]

[0001] (Reference to Related Application) This application claims the benefit of U.S. Provisional Patent Application No. 63 / 278,022, filed November 10, 2021, the contents of which are incorporated by reference herein in their entirety. [Background technology]

[0002] With the transportation and distribution of goods through logistical systems, manual loading and unloading of products / packages in trucks, trailers, and containers is a resource-intensive endeavor. Inbound receiving and outbound shipping docks found in logistical systems such as warehousing, distribution, cross-docking, etc. involve manual loading / unloading of freight from trailers, trucks, and shipping containers. In most instances, the loading and unloading process can take multiple hours to complete and is typically done manually or with the aid of a conveyor system to aid in material flow. The typical loading and unloading and palletizing rate per person in these operations is around 250-400 cases / hour, which typically translates to turning trucks per day with each associated role filled by multiple people throughout the day. To turn a loading dock faster, multiple related roles may work together. On average, in the United States, at a fully loaded labor rate, the cost of manual loading and unloading per day exceeds US$1,000 per dock when two to three related roles are assigned to the task.

[0003] Due to the distributed nature of global logistics, when products are shipped directly from manufacturers to the first node of distribution, cargo is typically configured and fluid loaded to optimize shipping density. Thus, products are grouped together when they have the same single stock-keeping unit (SKU). Typically, fluid loaded cargo arrives in one of three forms: single SKU, snake or wave SKU, or mixed-load containerization. In snake or wave SKU loads, there are multiple SKUs loaded into a fluid container, with each unique SKU loaded next to a similar SKU. In mixed-load containerization, multiple SKUs are loaded into a container without a specific order, with each SKU randomly placed.

[0004] Transportation from containers to warehouses is a difficult and repetitive task plagued by high turnover human labor due to the environment and the ergonomics of manually lifting some products that can weigh as much as 80 pounds. Moving materials in these environments takes constant labor turnover to minimize risk of damage and exposure. Manually loading cases (cargo) onto pallets in trailers or shipping containers is difficult when cases of different sizes and weights are presented in random order for palletization so that the stacked cases are stable. Thus, unloading typically requires pallet load stabilization, where pallets are usually stabilized by manually shrink-wrapping the cases they carry and the material around the pallet to prevent inadvertent displacement of the cases during downstream transportation. In addition to the manpower expended in loading and / or unloading trucks, trailers, or containers, downtime with trucks sitting idle at a loading dock can contribute to higher costs resulting from paying truck drivers who sit idle while trucks or trailers are loaded / unloaded or from fees associated with a limited supply of shipping containers.

[0005] Later in the logistics system value chain, pallets are broken down to support logistics needs. Inevitably, pallets require plastic material stretch wrapping or other means of load stabilization to enable defect-free transportation across logistics nodes and within facilities. It is difficult to store and retrieve pallets and automatically dismantle them. Typically, removal of the load stabilizing plastic stretch wrap is done manually, with commercially available systems costing hundreds of thousands of dollars per installation, which typically remove an entire layer of the pallet to feed an automated case storage and retrieval system.

[0006] The current state of logistics involving the transportation of palletized unit loads in and out of facilities is costly and difficult due to the many manual interventions that take place in the care of palletization, load stabilization, transportation, storage, and depalletization within the logistics system. To address this, several efforts have been made towards automation. However, these approaches lack high functional utility and attractive speed and return on investment due to low elegance in the design architecture, coupling of too many functions, and poor integration of technology. In most cases, there are two treatments of automation. One involves bulk flow of material from a trailer using parallel conveyors and end effectors, and the other involves a single robotic system with a low level of awareness of the package status. In the bulk flow architecture, packages become jammed and locked during the flow and are damaged. Similarly, expensive material handling is required to deal with the bulk unstructured flow of material. Single industrial robotic systems suffer from low throughput and utilization of the major robotic degrees of freedom can be less than 50%. Because the robot must pick up an item, move to drop it off, then go back and get another item, there are no designs that can automatically load and unload single and snake SKU loads, inspect the materials to be handled, and automatically palletize, transport, and store without stretch wrap load stabilization technology.

[0007] The present invention relates to the automated unloading / loading of fluid-loaded cases of various sizes, shapes and weights transported in trucks, trailers and shipping containers, as well as the automated palletization of cargo / cases into palletized stabilized unit loads. Additionally, the present invention relates to the transport and storage of unit load quantities of packages that enable a high level of automation in the distribution, cross-docking, storage and shipment of products across a logistics system. Summary of the Invention

[0008] In one aspect, a robotic system includes a telescopic transport conveyor. The robotic system also includes an automated unloader attached to the telescopic transport conveyor, the automated unloader including a loading conveyor having a first loading conveyor end and a second loading conveyor end. The automated unloader includes at least two articulated robots attached to the first loading conveyor end, each articulated robot including at least six degrees of freedom, at least one of a two-dimensional (2D) camera and a three-dimensional (3D) camera, and an end effector. The automated unloader includes a grounded base including wheels or tracks, a base support including a pivot, a support attached to the second loading conveyor end, and a translation actuator, the loading conveyor pivot configured to enable the lifting and lowering of the loading conveyor by extending or retracting the translation actuator. The robotic system also includes a control system having logic for coordinating the operation of the at least two articulated robots, the coordination including the position and movement of each robot and each robot's components with respect to time. The logic performs a perception scan of at least a portion of a visible wall of the material to be handled (MTBH), the perception scan captures 3D images and 2D images including 3D point cloud data of the MTBH. The logic constructs a 6 degree of freedom pose for each unit of the MTBH, including analyzing the 2D image to determine which segments of the 2D image belong to each MTBH, extrapolates at least one of a size, an orientation, a base of interest, and a center of mass from the 2D image for each unit of the MTBH, queries the 3D image to obtain depth information, an orientation of each MTBH in 3D, and a dimension of each side of each MTBH as expressed in the reference frames of the 2D and 3D cameras, and determines the reference frames of the 2D and 3D cameras as a function of the tilt angle of the unloading conveyor and the robot with the attached 2D and 3D cameras. The logic determines a primary reference frame of the automatic unloader using at least the reference frames of the 2D and 3D cameras.The logic constructs a virtual representation of each unit of the MTBH detected in the sensory scan, the virtual representation generates a six degree of freedom pose for each unit of the MTBH relative to a primary frame of reference of the automated unloader, and then calculates pick poses, each including a pick position and a pick orientation, the pick position being at the center of a face of each unit of the MTBH, and the pick orientation being normal to one face of each unit of the MTBH. The logic determines at least one motion plan using a search-based algorithm with inputs including a pick pose for each unit of the MTBH, at least one of minimizing torque for the robot motors, minimizing motion path length, and avoiding collisions with other dynamic or static elements of the environment. The logic selects a motion plan based on a scoring algorithm, the scoring algorithm based at least in part on the pick pose of each unit of the MTBH relative to the other units of the MTBH. The logic determines a pick sequence based at least in part on the selected motion plan for each MTBH, the pick pose of each unit of the MTBH relative to the other units of the MTBH, and the coordinated motion of the at least two jointed robots. The logic commands the automatic unloader to perform the pick-up sequence. [Brief description of the drawings]

[0009] To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced.

[0010] [Figure 1] 1 illustrates an automatic unloading and palletizing routine 100 according to one embodiment.

[0011] [Diagram 2] 2 illustrates a pallet load stabilization routine 200 according to one embodiment.

[0012] [Figure 3A]FIG. 3 shows a perspective view of a robotic loading / unloading system 300 that may be used to load and unload packages from a vehicle trailer to a conveyor system and an integrated robotic palletizer, according to various embodiments of the present disclosure.

[0013] [Figure 3B] FIG. 3 illustrates a side view of a robotic loading / unloading system 300 according to one embodiment.

[0014] [Figure 3C] FIG. 3 illustrates a top view of a robotic loading / unloading system 300 according to one embodiment.

[0015] [Figure 4] FIG. 4 illustrates a perspective view of an end-to-end system 400 showing robotic unloading from a trailer to a palletized unit load in accordance with various embodiments of the present disclosure.

[0016] [Figure 5A] A side view of an automated unloader 500 according to various embodiments of the present disclosure is shown.

[0017] [Figure 5B] A front view of an automated unloader 500 is shown according to one embodiment.

[0018] [Figure 6] 6 illustrates degrees of freedom 600 of a robotic loading / unloading system according to one embodiment.

[0019] [Figure 7] 7 illustrates a primary reference frame generation 700 according to one embodiment.

[0020] [Figure 8] 8 illustrates MTBH calculated picking data 800 according to one embodiment.

[0021] [Figure 9A]FIG. 9 illustrates a top view of an automated pallet load stabilization system 900 supported by a mobile robot, an interface between a robotic material handling system according to various embodiments of the present disclosure.

[0022] [Figure 9B] 9 illustrates a side view of an automated pallet load stabilization system 900 according to one embodiment.

[0023] [Figure 10] 10 illustrates a concurrent packaging and lifting operation 1000 according to one embodiment.

[0024] [Figure 11A] A side view of a pallet sleeve assembly 1100 is shown in accordance with various embodiments of the present disclosure.

[0025] [Figure 11B] A front view of a pallet sleeve assembly 1100 is shown according to one embodiment.

[0026] [Figure 11C] 11 illustrates a perspective exploded view of a pallet sleeve assembly 1100 according to one embodiment.

[0027] [Figure 12A] 12 illustrates a side view of an automated crane storage and retrieval system 1200 according to various embodiments of the present disclosure.

[0028] [Figure 12B] 12 illustrates a front view of an automated crane storage and retrieval system 1200 according to one embodiment.

[0029] [Figure 12C] 12 illustrates a top view of an automated crane storage and retrieval system 1200 according to one embodiment.

[0030] [Figure 13] 13 illustrates a loading / unloading control system 1300 according to one embodiment.

[0031] [Figure 14] 14 illustrates a palletization control system 1400 according to one embodiment.

[0032] [Figure 15] 15 illustrates a load stabilization control system 1500 according to one embodiment.

[0033] [Figure 16] 16 illustrates an end-to-end control system diagram 1600 according to one embodiment.

[0034] [Figure 17] 17 illustrates an automatic truck unloading routine 1700 according to one embodiment.

[0035] [Figure 18] 18 illustrates a pick routine for an individual articulated robot 1800 according to one embodiment.

[0036] [Figure 19] 19 illustrates an automated truck loading and unloading flow diagram 1900 according to one embodiment.

[0037] [Figure 20] 2 illustrates a detection cash flow diagram 2000 according to one embodiment.

[0038] [Figure 21] 21 illustrates a planner flow diagram 2100 according to one embodiment.

[0039] [Figure 22] 2 illustrates an execution flow diagram 2200 according to one embodiment.

[0040] [Diagram 23] 23 illustrates an embodiment of a computing device 2300 for implementing the components and process steps of the systems described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0041] The present disclosure relates to automating the entire process of loading and unloading stacked materials to be handled (MTBH) from trailers or containers of delivery vehicles at a material handling facility by automating the construction of pallets, storage, retrieval, transportation, and de-palletizing of the pallets into other pallets suitable for distribution without the need for human assistance and labor. The material handling facility may be a sortation center, shipping center, packaging center, distribution center, filling center, or other shipping, handling, or packaging facility, among other similar facilities. At the material handling facility, carrier vehicles may deliver packages of inventory, pre-packaged orders, and / or other types of packages via trucks, trailers, and other types of shipping containers.

[0042] Typically, when a vehicle carrying a container arrives at a material handling facility, workers manually remove packages from the container, manually construct pallets, and manually apply load stabilization in the form of stretch wrapping. The robotics and automation architecture disclosed herein enables end-to-end automation. By automating the associated activities of unloading, palletizing, load stabilizing, transporting, storing, retrieving, and depalletizing packages at a material handling facility, unloading time, costs, and potential worker injuries may be reduced. The disclosed end-to-end automated package unloading, palletizing, load stabilizing, transporting, storing, and retrieving system efficiently unloads stacked multi-layer packages into trailers or containers received at the material handling facility, converts them into pallets reinforced with pallet sleeves, transports them, places them in storage, retrieves them, and automatically applies load stabilization (e.g., stretch wrapping) through the use of mobile robot transport to enable all types of typical material handling activities and methods (e.g., sorting, storage, shipping, etc.). The disclosed system uses multiple robotic arms working together to load and unload containers. It uses multiple perception systems to understand where the MTBH is located in free space.

[0043] Although the making and use of various embodiments of the present disclosure are discussed in detail below, it should be understood that the concepts disclosed herein may be embodied in a wide variety of specific contexts. The specific embodiments discussed herein are merely illustrative of specific ways to make and use the disclosed systems and do not limit the scope of the disclosure.

[0044] Figure 1 illustrates an exemplary automated unloading and palletizing routine 100 for unloading MTBHs from a docked shipping container at a material handling facility through the action of a robotic unloading system 300 as shown in Figures 3A-3C. Some or all of the steps of the automated unloading and palletizing routine 100 may be performed by a control system such as the unloading control system 1300 shown in Figure 13 and the palletizing control system 1400 in Figure 14, which may interact with other control systems throughout the material handling facility according to an end-to-end control system diagram 1600 shown in Figure 16.

[0045] Although the exemplary automated unloading and palletizing routine 100 illustrates a particular sequence of operations, the sequence may be modified without departing from the scope of the present disclosure. For example, some of the operations illustrated may be performed in different sequences or in parallel without substantially affecting the functionality of the automated unloading and palletizing routine 100. In other examples, different components of an exemplary device or system implementing the automated unloading and palletizing routine 100 may perform functions substantially simultaneously or in a particular sequence. Some or all of the automated unloading and palletizing routine 100 may, in other embodiments, be augmented or replaced by routines such as the automated truck unloading routine 1700 shown in FIG. 17 and the pick routine for an individual articulated robot 1800 shown in FIG. 18, and may be supported by algorithms such as those described with respect to the automated truck unloading flow diagram 1900 in FIG. 19, the detection cache flow diagram 2000 in FIG. 20, the planner flow diagram 2100 in FIG. 21, and the execution flow diagram 2200 in FIG. 22.

[0046] According to some examples, the method includes positioning a robotic loading / unloading system at or within an entrance to a shipping container at block 102. For example, the operations team 412 shown in Figure 4 may position the robotic loading / unloading system at or within the entrance to the shipping container. The robotic loading / unloading system may be a system such as the robotic loading / unloading system 300 shown in Figures 3A-3C.

[0047] According to some examples, the method includes applying a downward force to a base support of the automated unloader at block 104. For example, the telescoping transport conveyor 308 shown in FIG. 3A may apply a downward force to the base support of the automated unloader. The application of this downward force stabilizes the automated unloader 500 against uneven loads as the MTBHs are picked up and dropped, and maintains firm ground contact with uneven or sloping container floors.

[0048] According to some examples, the method includes unloading the MTBH from the shipping container at block 106. For example, a robotic unloading system 300 shown in FIGURE 3A may unload the MTBH from the shipping container. The operations of block 104 may be supported by logic algorithms described through subroutine blocks 108-126.

[0049] According to some examples, the method includes coordinated operation of at least two articulated robots at subroutine block 108. For example, the loading / unloading control system 1300 shown in Figure 13 may coordinate the operation of at least two articulated robots. The coordination may include the position and movement of each robot and components of each robot with respect to time.

[0050] According to some examples, the method includes performing a perceptual scan of at least a portion of the visible walls of the material to be handled (MTBH) at subroutine block 110. For example, the robot-mounted camera 504 shown in FIG. 5A may perform a perceptual scan of at least a portion of the visible walls of the material to be handled (MTBH). The perceptual scan captures 3D and 2D images, including 3D point cloud data of the MTBH. The robot-mounted camera 504 may be a 2D camera and / or a 3D camera.

[0051] According to some examples, the method includes constructing a 6-DOF pose for each unit of the MTBH in subroutine block 112. For example, the loading / unloading control system 1300 shown in FIG. 13 may construct a 6-DOF pose for each unit of the MTBH. According to some examples, the method includes analyzing the 2D image to determine which segments of the 2D image belong to each MTBH. According to some examples, the method includes extrapolating at least one of a size, an orientation, an origin of interest, and a center of mass from the 2D image for each unit of the MTBH. According to some examples, the method includes querying the 3D image to obtain depth information, an orientation of each MTBH in 3D, and a dimension of each side of each MTBH as represented in a field of view of at least one of the 2D camera and the 3D camera of each articulated robot of the at least two articulated robots. According to some examples, the method includes, for each robot of at least two articulated robots, a 2D camera reference system (reference frame) and a 3D camera reference system (reference frame) as a function of the loading conveyor inclination angle and the position configuration of the robot joints.

[0052] According to some examples, the method includes determining a primary reference frame for the automated unloader at subroutine block 114. For example, the unloading control system 1300 shown in Figure 13 may determine the primary reference frame for the automated unloader. The primary reference frame may be determined using at least one of a 2D camera reference frame and a 3D camera reference frame.

[0053] According to some examples, the method includes constructing a virtual representation of each unit of the MTBH detected in the perception scan at subroutine block 116. For example, the offloading control system 1300 shown in Figure 13 may construct a virtual representation of each unit of the MTBH detected in the perception scan. The virtual representation generates a six degree of freedom pose for each unit of the MTBH relative to the primary frame of reference of the automated unloader.

[0054] According to some examples, the method includes calculating a picking pose including a picking position and a picking orientation in subroutine block 118. For example, the unloading control system 1300 shown in FIG. 13 may calculate a picking pose including a picking position and a picking orientation. The picking position may be at the center of a face of each unit of the MTBH, and the picking orientation is normal to one face of each unit of the MTBH.

[0055] According to some examples, the method includes determining at least one motion plan using a search-based algorithm at subroutine block 120. For example, the loading / unloading control system 1300 shown in FIG. 13 may determine at least one motion plan using a search-based algorithm. Inputs to the search-based algorithm may include picking poses for each unit of the MTBH, minimization of torque on the robot motors, minimization of motion path length, and collision avoidance with other dynamic or static elements of the surrounding environment.

[0056] According to some examples, the method includes, at subroutine block 122, selecting a motion plan based on a scoring algorithm. For example, the loading / unloading control system 1300 shown in FIG. 13 may select a motion plan based on a scoring algorithm. The scoring algorithm is based, at least in part, on a picking pose of each unit of the MTBH relative to other units of the MTBH. Attributes of the scoring algorithm may include at least one of minimizing torque on the robot motors, motion path length, position, velocity, acceleration, jerk, snap, minimizing trajectory normalized distance, and attributes related to configuration space, free space, target space, and obstacle space.

[0057] According to some examples, the method includes determining a picking sequence at subroutine block 124. For example, the unloading control system 1300 shown in FIG. 13 may determine the picking sequence. The picking sequence may be based, at least in part, on the motion plan selected for each MTBH, the picking pose of each unit of the MTBH relative to the other units of the MTBH, and the coordinated motion of the at least two articulated robots.

[0058] According to some examples, the method includes performing the picking sequence at subroutine block 126. For example, the articulated robot 302 shown in FIGURE 3A may perform the picking sequence.

[0059] According to some examples, the method includes removing at least one visible wall of the MTBH at block 128. For example, the robotic loading / unloading system 300 shown in FIG. 3A may remove at least one visible wall of the MTBH.

[0060] According to some examples, the method includes positioning the robotic offloading system further into the container at block 130. For example, the telescoping transport conveyor 308 shown in FIG. 3A may position the robotic offloading system further into the container, which allows at least two articulated robots to access the remaining MTBH.

[0061] According to some examples, the method includes unloading the remaining MTBH from the shipping container at block 132. For example, the robotic unloading system 300 shown in Figure 3A may unload the remaining MTBH from the shipping container.

[0062] According to some examples, the method includes receiving the unloaded MTBHs from the shipping container at block 134. For example, an automated palletizing system 318 shown in Figure 3A may receive the unloaded MTBHs from the shipping container. The automated palletizing system may be like the automated palletizing system 318 described in Figures 3A-3C.

[0063] According to some examples, the method includes forming an at least partially stacked pallet sleeve assembly with the received MTBH at block 136. For example, the palletizing robot 326 shown in FIG. 3A may form an at least partially stacked pallet sleeve assembly with the received MTBH. This action may be accomplished as described in subroutine blocks 138 and 140.

[0064] According to some examples, the method includes receiving the unit of MTBH with an end effector of the palletizing robot at subroutine block 138. For example, the palletizing robot 326 shown in FIG. 3A may receive the unit of MTBH with an end effector of the palletizing robot.

[0065] According to some examples, the method includes, at subroutine block 140, stacking each unit of the MTBH within at least one pallet sleeve assembly until an at least partially stacked pallet sleeve assembly is formed.

[0066] According to some examples, the method includes transporting the at least partially stacked pallet sleeve assembly to another location within the material handling facility at block 142. For example, the autonomous mobile robot or AMR 324 shown in FIG. 3A may transport the at least partially stacked pallet sleeve assembly to another location within the material handling facility. The at least partially stacked pallet sleeve assembly may be transported to the automated pallet load stabilization system 900 or the automated crane storage and retrieval system 1200 in some embodiments.

[0067] Figure 2 illustrates an exemplary pallet load stabilization routine 200 for an automated pallet load stabilization system 900 such as that shown in Figures 9A and 9B and 10. Some or all of the steps of the automated unloading and palletizing routine 100 may be performed by a control system such as the load stabilization control system 1500 shown in Figure 15, which may interact with other control systems across the material handling facility according to an end-to-end control system diagram 1600 shown in Figure 16.

[0068] Although the example pallet load stabilization routine 200 illustrates a particular sequence of operations, the sequence may be modified without departing from the scope of the present disclosure. For example, some of the illustrated operations may occur in parallel or in different sequences that do not substantially affect the functionality of the pallet load stabilization routine 200. In other examples, different components of an example device or system implementing the pallet load stabilization routine 200 may perform functions substantially simultaneously or in a particular sequence.

[0069] According to some examples, the method includes receiving an at least partially stacked pallet sleeve assembly at block 202. For example, the automated pallet load stabilization system 900 shown in FIG. 9A may receive the at least partially stacked pallet sleeve assembly. The automated pallet load stabilization system 900 is described in detail with respect to FIG. 9A and FIG. 9B. The at least partially stacked pallet sleeve assembly may arrive from the automated palletization system 318 via the AMR 324.

[0070] According to some examples, the method includes generating a wrapped palletized unit load at block 204. For example, a rotating pallet wrapping mechanism 916 shown in FIG. 9A may generate the wrapped palletized unit load. The steps included in this operation may be performed simultaneously with the steps included in the operation of block 206.

[0071] According to some examples, the method includes lifting the pallet sleeve completely from the wrapped palletized unit load at block 206. For example, a pallet sleeve lifter 910 shown in FIG. 9B may lift the pallet sleeve completely from the wrapped palletized unit load. This step and the steps of block 204 may be accomplished as described in subroutine block 208 through subroutine block 214.

[0072] According to some examples, the method includes placing the at least partially stacked pallet sleeve assemblies in a position suitable for interfacing with an actuator system and a pallet sleeve lifter at subroutine block 208. For example, the MHE 906 shown in FIG. 9A may place the at least partially stacked pallet sleeve assemblies in a position suitable for interfacing with an actuator system and a pallet sleeve lifter.

[0073] According to some examples, the method includes actuating a pallet sleeve lifter to interface with a pallet sleeve on the at least partially stacked pallet sleeve assemblies at subroutine block 210. For example, the load stabilization control system 1500 shown in FIG. 15 may actuate a pallet sleeve lifter to interface with a pallet sleeve on the at least partially stacked pallet sleeve assemblies.

[0074] According to some examples, the method includes incrementally lifting the pallet sleeve upwardly, thereby incrementally exposing the MTBH units for stretch wrapping by the rotating pallet wrap mechanism, at subroutine block 212. For example, a pallet sleeve lifter 910 shown in FIG 9B may incrementally lift the pallet sleeve upwardly, thereby incrementally exposing the MTBH units for stretch wrapping by the rotating pallet wrap mechanism.

[0075] According to some examples, the method includes, at subroutine block 214, actuating an actuator system with the packaging material spool pin to move in a spiral motion around the incrementally exposed MTBH units as the pallet sleeve lifter lifts the pallet sleeve. For example, a load stabilization control system 1500 shown in FIG. 15 may actuate an actuator system with the packaging material spool pin to move in a spiral motion around the incrementally exposed MTBH units as the pallet sleeve lifter lifts the pallet sleeve. Dispensing the packaging material around the at least partially stacked pallet sleeve assembly is performed according to a number of wraps around the exposed MTBH units, a percent elongation over a length of the packaging material, and / or an offset between layers of the packaging material.

[0076] According to some examples, the method includes transporting the packaged palletized unit load to another location within the material handling facility at block 216. For example, the AMR 324 shown in FIG. 3A may transport the packaged palletized unit load to another location within the material handling facility. The packaged palletized unit load, in some embodiments, is transferred to the automated crane storage and retrieval system 1200 or loaded onto a transport vehicle or container for delivery to another location.

[0077] According to some examples, the method includes placing the empty pallet under the fully raised pallet sleeve at block 218. For example, the automated forklift 920 shown in FIG 9A may place the empty pallet under the fully raised pallet sleeve.

[0078] According to some examples, the method includes lowering a pallet sleeve onto the empty pallet to form an empty pallet sleeve assembly at block 220.

[0079] According to some examples, the method includes transporting the empty pallet sleeve assembly to an automated palletization system where the MTBH is to be loaded, at block 222. For example, the AMR 324 shown in FIG. 3A may transport the empty pallet sleeve assembly to the automated palletization system where the MTBH is to be loaded.

[0080] Figure 3A shows a perspective view of a robotic loading / unloading system 300 that may be used to load and unload packages from a vehicle trailer to a conveyor system and an integrated robotic palletizer in accordance with various embodiments of the present disclosure. Figure 3B shows a side view of the robotic loading / unloading system 300, and Figure 3C shows a top view of the robotic loading / unloading system 300.

[0081] This robotic loading / unloading system 300 is utilized in systems and methods for automated truck loading / unloading, automated palletizing, and packaging of trailers, containers, and the like. Multiple robotic degrees of freedom (DOF) are used by an automated unloader 500 with an articulated robot 302 to grasp, manipulate, and transport packages from the loading / unloading area to an attached loading conveyor 304, which has the ability to pivot, translate up and down, or rotate. The DOFs present in one embodiment of the robotic loading / unloading system 300 are shown in FIG. 6. In this embodiment, two separate articulated robots 302 are presented in such a way that they may operate close to each other with overlapping kinematic reach volumes and coordinate with each other via an optimized 12 DOF kinematic solver. The ability to move up and down allows the robot platform to get closer to fluid loaded packages, shortening the distance the articulated robot 302 needs to travel. This may increase the throughput of the system.

[0082] The loading conveyor 304 transfers the material to be handled to the telescopic transport conveyor 308. In some cases, the transport conveyor may be flexible, but in most embodiments, the use of a fixed telescopic conveyor for the telescopic transport conveyor 308 may support specific material handling characteristics for the size of the MTBH that requires a continuous belt. The telescopic transport conveyor 308 may be a MaxReach telescopic conveyor or similar structure, and may be modified as necessary to accommodate the automatic unloader 500. A scanning tunnel 310 may be configured at a location along the telescopic transport conveyor 308. In the scanning tunnel 310, a visual sensor such as a camera, barcode scanner, etc. may provide a five- or six-sided scan of the MTBH moving along the telescopic transport conveyor 308. Information collected within the scanning tunnel 310 may be used for tracking and sorting of the MTBHs at later stages of the robotic loading / unloading system 300.

[0083] The automated unloader 500 may include multiple robot DOFs as well as various perception systems that enable the robot DOFs to identify the MTBH and the constraint space in which the automated unloader 500 may operate. The attached or fixed perception systems may be any combination of one-dimensional (1D), two-dimensional (2D) and three-dimensional (3D) perception systems. These perception systems allow the automated unloader 500 to determine where the package is and what orientation the package is in, allowing the robot end effector to optimize the contact normal at which the package may be grasped, the picking speed, and the probability of a successful pick. The automated unloader 500 may also provide six-sided perception of the package, including the package's weight and instantaneous center of gravity, during MTBH operations or after placement in the system. Because these sensory systems are typically very sensitive to noise and vibration, the automated unloader 500 may utilize some type of support base or grounded base 306, such as tracks or wheels, that provide a wide footprint and low contact pressure to distribute the weight of the system. The grounded base 306 includes a pivot point, allowing the automated unloader 500 to move in and out of containers while providing stability and support even when facing uneven or sloping terrain not possible with conventional flexible and stretchable transport conveyors.

[0084] The transport flexible or fixed telescopic transport conveyor 308 may have at least one connection point to a traditional material handling equipment (MHE) or may transition to a mobile material handling system. In this embodiment, the system transitions to a type of sorting MHE 312 that allows a single stream of goods to be split into n different lanes for sorting. In one embodiment, the sorting MHE 312 may be a 2-Way Sortation manufactured by Regal Rexnord, Cellmation, or Intralox. Such a sorting MHE 312 may very easily split from one feed lane into two or four sorting lanes in a continuous stream. This may allow downstream palletization to be optimized depending on the incoming material arriving at the material handling facility, or may allow a stream to enter directly into a set of cross-docked MHEs by bypassing palletization.

[0085] If loading / unloading to palletization is intended, a set of inclined MHEs 314 allows the product to be transferred from the lower height of the transport conveyor ("top of the rollers") to the automated palletization system 318. An Interoll Incline Conveyor may be used for this purpose. The inclined MHE 314 may decouple the two subsystems and bring the MTBH to the correct location. The automated palletization system 318 consistently brings the product to a known location in three-dimensional space so that the speed of the palletizer is optimized. In most systems on the market, the material flow is random and may require the use of additional calculations to place the packages. The automated palletization system 318 may use a palletization robot 326, such as the Robotic Palletizer System, to place the MTBH 402 once it arrives at the top of the inclined MHE 314. The palletization robot 326 may be supported by a palletization robot support gantry 328.

[0086] The MTBH 402 may be placed in a protective pallet sleeve assembly 1100, such as the empty pallet sleeve assembly 320, which allows for high speed palletization without worrying about packages tipping, slipping, or shifting. The pallet sleeve assembly 1100 allows for automated transport of empty, partially loaded, or full pallets throughout the system. The pallet sleeve assembly 1100 may be optimized for automated flow on the pallet MHE or with an independent agent-based automated mobile robot or AMR 324 material handling system. The AMR 324 may bring the pallet sleeve assembly 1100 directly into the automated palletization system 318 or may interface with the MHE. The 2DOF MHE 316 may allow the pallet sleeve assembly 1100 to flow in at a normal vector and transition to the loading area without rotation about the Z axis. An empty pallet sleeve assembly 320 and an at least partially stacked pallet sleeve assembly 322 are shown.

[0087] The AMR 324 may interface with a fixed MHE (such as the 2DOF MHE 316) and offload an empty pallet sleeve assembly 320. The empty pallet sleeve assembly 320 may be the entry point for the flow of pallets, and the at least partially stacked pallet sleeve assembly 322 may be the exit point. The AMR 324 may bring the empty pallet sleeve assembly 320 into the system and then pick up the at least partially stacked pallet sleeve assembly 322 at the same location or a different location. The MHE may be configured to support multiple different loading and unloading architectures and mappings depending on the material flow. The AMR 324 may navigate to any surrounding location to accept the transfer of the pallet sleeve assembly 1100 and transport the pallet sleeve assembly 1100. The AMR 324 may also transport pallets without sleeves as well as pallet sleeves that are not assembled around a pallet.

[0088] FIG. 4 is a perspective view of an end-to-end system 400 illustrating robotic offloading of MTBHs 402 from a shipping container 404 at a material handling facility 406 through to the production of a packaged palletized unit load 410 in accordance with various embodiments of the present disclosure, with little or no involvement by an operations team 412 at the material handling facility 406.

[0089] Here, three shipping containers 404, trailers or trucks are shown parked at a loading dock of a material handling facility 406. A dock 408 is typically found at material handling facilities 406 and is intended to provide a portal for shipping containers to interface with the interior of the facility while providing a weather seal.

[0090] An automated unloader 500 may be positioned by the operations team 412 inside each transport volume and may begin robotically and automatically unloading the MTBHs 402. The MTBHs 402 may be transported along a fixed or flexible sorting MHE 312 and then via the tilt MHE 314 to the automated palletizing system 318. A queue of empty pallet sleeve assemblies 320 as shown in Figure 3A may be maintained and available for the automated palletizing system 318 to bring the MTBHs 402 from the tilt MHE 314 into a pallet sleeve assembly 1100, which will be described in more detail with reference to Figure 11A.

[0091] The sorting MHE 312 logically sorts the MTBHs 402 into the correct lanes of the tilting MHE 314, which may include paths for separating palletization automated palletizing systems 318. Additionally, MTBHs may be diverted to exception paths for inspection, special handling, and other non-standard reasons during sorting on the sorting MHE 312. In this embodiment, the dual flows of the tilting MHE 314 go to the automated palletizing systems 318 to provide each robotic arm with its own source and supply of products.

[0092] Once full, the at least partially stacked pallet sleeve assemblies 322 may be moved either to a storage location or to an automated pallet load stabilization system 900 which may provide automatic stretch wrapping of the pallets.

[0093] In one embodiment, a shipping customer may provide data including a shipping manifest, a SKU master, and a desired pallet configuration. The shipping manifest may include the number of SKUs and total weight of the load expected in one shipping container 404. The SKU master may include the dimensions, weight, and type of MTBH 402 associated with a particular SKU. The pallet configuration may include the unique identifier, SKU, final destination, and weight associated with a particular pallet to be constructed from the MTBH 402 housed in the shipping container 404. The control system of the end-to-end system 400 may take this data as an input and use it to determine what actions should be taken at each stage of the process to palletize the MTBH 402 as desired by the customer, as shown in FIG. 16. The end-to-end system 400 computing system may return the configuration of each completed pallet for use by the customer, including the unique identifier, box build, quantity, inspection status, receiving indication, weight, timestamp of when the pallet was created and stabilized, and an image of the pallet. If the pallet is placed in storage, the pallet information may include a location indicator to facilitate retrieval when required.

[0094] 5A shows a side view of an automated unloader 500 according to various embodiments of the present disclosure. FIG. 5B shows a front view of the automated unloader 500.

[0095] The automated unloader 500 includes a high DOF robotic system including at least two articulated robots 302, each capable of at least 6 DOF of motion. The articulated robots 302 include robotic joints that support those DOF of motion to allow segments of the robot arm to bend, extend, and reorient in space relative to other parts of the robot and other objects in the environment. The articulated robots 302 may be positioned at a first loading conveyor end 502 of the loading conveyor 304. The articulated robots 302 may be mounted to a robot base 508 supported by the first loading conveyor end 502. The articulated robots 302 may incorporate an integrated 1D, 2D, and 3D perception system including a robot-mounted camera 504 including 2D and 3D cameras that can capture image data used to determine the quantity, size, location, and orientation of the MTBH items within the shipping container. The automated unloader 500 may use applied machine learning and artificial intelligence to detect boundaries of separate items of the MTBH, such as boxes, cartons, or other items, classify them, locate them in 3D space, and optimize pick points and approach patterns for the articulated robot 302. These computational tasks may be separated into a dedicated on-board computing cluster that allows the automated unloader 500 to achieve high levels of autonomy and high pick speeds. Such a computing cluster may utilize a loading / unloading control system 1300 as shown in FIG. 13.

[0096] The articulated robot 302 may use various vision sensors and sensor control feedback loops to grasp, manipulate, and transport packages to the loading conveyor 304. The articulated robot 302 may be configured with multiple types of end effectors 506 that may use vacuum generated suction or other techniques to grasp and manipulate packages. Typically, the number of robotic degrees of freedom in a system is greater than 12 and may be as high as 33. The DOFs for one embodiment are shown in FIG. 6.

[0097] The entire automated unloader 500 is supported by a grounded base 306 assembly, which distributes the weight of the system and provides stability during operation. The grounded base 306 attaches to the second loading conveyor 510 through a base support 512 connected to a grounded base pivot 514, which allows the grounded base 306 to orient itself as needed over uneven or inclined container surfaces while maintaining the loading conveyor 304 in its desired position. A feedback loop may be configured between the telescopic transport conveyor and strain gauges on the automated unloader 500 and the grounded base 306 that measures elastic deformations of the components of the automated unloader 500 corresponding to the forces on those components. A proportional-integral-derivative loop is established for force feedback to allow the upward and downward forces that may be applied by the telescopic transport conveyor to keep the grounded base 306 grounded to the bottom of the container. To accomplish this, a standard industrial controller, such as a Beckhoff controller, may be used. The grounded base 306 may include wheels 516 and / or tracks 518 or other passive moving components that allow the automated unloader 500 to be manipulated and repositioned through the operation of a telescoping transport conveyor and the efforts of an operating team.

[0098] The loading conveyor 304 provides an additional degree of freedom by rotation about the loading conveyor pivot 520, with tight loop control of position via a translation actuator 522. The rotational position is recorded with various 1D sensing techniques such as a rotary encoder, or a linear variable range transceiver. Encoder feedback can provide the location of the articulated robot 302 in free space at the end of the pivoting conveyor. Joint information can be provided based on the robot on-board camera 504. Each time this camera scans an area, it may highlight the field of view captured as a 2D image or 3D point cloud data. The 2D data can be used to classify boxes to identify where the boundaries and edges of each box are located. The 3D data can be used along with the lines defining the box outline to identify points on the surface to determine the pick location. In one embodiment, the pick location can be established as the coordinates of a point at the center of the picking surface of the MTBH.

[0099] The mounting gantry 524 provides attachment points for additional perception systems that record 1D, 2D, and 3D data to estimate the state and pose of the system. In one embodiment, these perception systems may include a forward-sensing multi-dimensional sensor system 526 and a multi-channel inertial measurement system 528. The robot base 508 may include a sensor cutout 530 that allows additional sensors, such as cameras, light detection and ranging (LIDAR), and other sensors to detect objects and obstacles ahead of the automated unloader 500. Vibrations, shocks, accelerations, and alignment to global magnetic field lines are recorded by such sensors during automated unloader 500 operation.

[0100] A local control system is presented that distributes power to various devices and components as well as provides edge distributed logic, and may include a motor 532 and an industrial control box 534 that supports a programmable logic controller.

[0101] 6 illustrates a robotic offloading system degree of freedom 600 according to one embodiment. The robotic offloading system 600 includes degrees of freedom (DOFs) achievable by the at least two articulated robot 302 along with the loading conveyor 304 and telescoping transport conveyor 308 of the robotic offloading system 300 that can be utilized in determining the handling of the MTBH 402 to be offloaded from the shipping container 404.

[0102] One DOF 602 may be provided by extension and contraction of the telescopic transport conveyor 308. The DOF 602 enables the telescopic transport conveyor 308 to propel the automated unloader 500 further along the motion vector into or out of the transport container 404. In this way, relocation of the automated unloader 500 to handle the subsequent wall of the MTBH 402 as the visible wall of the MTBH 402 is unloaded may be automated through control of the telescopic transport conveyor 308 without requiring intervention by an operations team.

[0103] In some embodiments, another DOF 604 may be provided, but for vertical movement of the telescoping transport conveyor 308. In another embodiment, the ability of the telescoping transport conveyor 308 to move vertically may instead be used to apply a downward preload force to the grounded base 306 such that the automated unloader 500 may traverse a sloped or uneven surface of the shipping container 404 without excessive vibration to the MTBH 402 or components of the automated unloader 500.

[0104] Another DOF 606 may be provided by upward and downward pivoting of the loading conveyor 304 about the loading conveyor pivot. Each of the two articulated robots 302 may be equipped to move with at least six DOF 608. In such an embodiment, at least fourteen robot loading / unloading system degrees of freedom 600 may be achieved.

[0105] In one embodiment, the loading conveyor 304 may be capable of pivoting horizontally and vertically to provide additional DOF. An articulated robot 302 capable of seven or more DOF may be utilized. Such a system may have 17 or 18 DOF. Those skilled in the art will readily understand how additional degrees of freedom may be achieved, as well as the technical complexities and advantages of achieving them in a system as disclosed herein.

[0106] 7 illustrates a primary frame of reference generation 700 according to one embodiment. 2D and 3D cameras 702 may take one or more visual perception scans 706 of at least a portion of the interior of a shipping container 404 carrying a MTBH 402. In an embodiment in which the 2D and 3D cameras 702 are mounted to a left hand joint robot 302 mounted to an automated unloader 500 as described above, the visual perception scan 706 performed by the 2D and 3D cameras 702 may be focused on a left hand portion of the shipping container 404 and the MTBH 402. Similarly, the 2D and 3D cameras 704 may perform one or more visual perception scans 708 as shown.

[0107] Each perception scan may cover an area of ​​overlap 710 that is scanned with more than one perception scan. Although an example is shown herein as overlap area 710, multiple bands are shown showing where visual field perception scans 706 may overlap one another, where visual field perception scans 708 may overlap one another, and where visual field perception scans 706 and 708 may overlap one another to provide complete scan coverage of the interior of shipping container 404 and MTBH 402 therein.

[0108] One or more visual field perception scans 706 may be used to develop a frame of reference 712 for the 2D and 3D cameras 702. Similarly, one or more visual field perception scans 708 may be used to develop a frame of reference 714 for the 2D and 3D cameras 704. Such a frame of reference may be determined as a function of the tilt angle of the loading conveyor 304 and the at least two articulated robot 302 with the attached 2D and 3D cameras 702 or 2D and 3D cameras 704. These two frames of reference, i.e., the frame of reference 712 and the frame of reference 714, may be used to generate a primary frame of reference 716 for the automated unloader 500.

[0109] 8 illustrates MTBH calculated picking data 800 according to one embodiment. A first wall of MTBH 802 and a second wall of MTBH 804 are shown, where the first wall of MTBH 802 includes the MTBH 402 determined from a perception scan that is closer to the nearest articulated robot 302, and the second wall of MTBH 804 is farther from the articulated robot 302.

[0110] The visible MTBH 402 may be detected through a sensory scan and may be analyzed to detect available picking faces, such as picking face 806, picking face 808, and picking face 810, as shown. Other picking faces may be inferred from the sensory scan data, but may or may not be included in the calculations to determine the picking data. In one embodiment, hidden faces may be analyzed, as a particular picking sequence may be expected to reveal picking faces that are not accessible during a sensory scan.

[0111] For the three picking faces visible in this figure for the MTBH 402 under inspection, a picking location may be determined for each face. The picking locations may lie within the calculated Cartesian center points on the picking faces. Such picking locations are shown as picking location 812 on picking face 806, picking location 814 on picking face 810, and picking location 816 on picking face 808.

[0112] For each picking surface, a picking orientation may be calculated. The picking orientation may be normal to the plane of the picking surface such that the end effector 506 of the articulated robot 302 may make firm contact with the picking surface and then begin to move normal to that surface to begin picking the MTBH 402. Examples of picking orientations are shown here as picking orientation 818 for picking surface 806, picking orientation 820 for picking surface 810, and picking orientation 822 for picking surface 808.

[0113] Figure 9A shows a top view of an automated pallet load stabilization system 900 supported by a mobile robot that is an interface between robotic material handling systems, according to various embodiments of the present disclosure. Figure 9B shows a side view of the automated pallet load stabilization system 900. The automated pallet load stabilization system 900 is shown during an automated pallet stretch wrap operation.

[0114] The AMR 324 may bring in the at least partially stacked pallet sleeve assemblies 322 loaded with MTBHs 402. The sequence and queue order may be controlled by a fleet management system that directs the pick-up of the pallet sleeve assemblies 1100 and pallets 904 and the drop-off of the pallet sleeve assemblies 1100 and pallets 904 to the system. The AMR 324 may interface with the at least partially stacked pallet sleeve assemblies 322 and transfer the at least partially stacked pallet sleeve assemblies 322 to the MHE 906. The at least partially stacked pallet sleeve assemblies 322 may be precisely moved to a suitable location for interfacing with the rotating ring 918 of the rotating pallet wrapping mechanism 916. The pallet sleeve lifter 910 may be actuated to interface with the pallet sleeve 908 and gradually pull the pallet sleeve 908 upward to expose the packages for stretch wrapping by the rotating pallet wrapping mechanism 916. The simultaneous wrap and lift operation 1000 is shown in more detail in FIG. 10 according to one embodiment.

[0115] The rotating pallet wrapping mechanism 916 travels in a spiral motion, carrying therewith packaging material spool pins 924 that hold spools of packaging material, wrapping the pallet with any combination of rotation per translation, at variable extension. The rotating pallet wrapping mechanism 916 automatically starts and stops the pallet wrapping. The rotating ring 918 provides clearance for the actuator system 914. The rotating ring 918, actuator system 914, packaging material spool pins 924, and other elements of the rotating pallet wrapping mechanism 916 may be provided through available industrial wrapping machines, such as a Smart Wasp X1300 Ring Type High Speed ​​Fully Automatic Pallet Wrapping Machine or similar machines.

[0116] When the actuator system 914 activates the process of wrapping the pallet and the MTBH 402 it carries, the at least partially stacked pallet sleeve assemblies 322 located within the rotating ring 918 may lift the pallet sleeve 908 by the pallet sleeve lifter 910. The wrapping may start at the bottom and the MTBH 402 is exposed through lifting of the pallet sleeve 908 by the pallet sleeve lifter 910. Once the pallet and MTBH 402 are wrapped, the pallet sleeve 908 may be lifted completely off the pallet and remain suspended. A pallet dispenser 902 may provide a repository for the pallets 904 near the rotating pallet wrapping mechanism 916. The automated forklift 920 may pick up the empty pallet 922 from the pallet dispenser 902 and then bring the empty pallet 922 to the MHE 906 as the MHE 906 moves the wrapped palletized unit 410 out of its position in the rotating ring 918. The AMR 324 may pick up the wrapped palletized unit load 410 and remove it from the automated pallet load stabilization system 900. The MHE 906 may receive the empty pallet 904 from the automated forklift 920 and move it to a position below the suspended pallet sleeve 908. The pallet sleeve lifter 910 may then reverse its action and deposit the pallet sleeve 908 onto the pallet 904 to form an empty pallet sleeve assembly 320, which may then be delivered by the MHE 906 to the AMR 324 and returned to the automated palletization system 318 for use as described with respect to FIG. 3A. Full automation of the rotating pallet wrapping mechanism 916 may be supported by a support structure 912 .

[0117] The packaging material may be made of Linear Low Density Polyethylene (LLDP) and may omit various attributes in terms of number of wraps, percent elongation over length, and offset between layers. Calculations for the stretch packaging film used in one embodiment are provided in the table below. [Table 1]

[0118] In one embodiment, the automated pallet load stabilization system 900 may process one pallet 904 in 60 seconds. In a typical configuration, a wrapped palletized unit load 410 may be removed from the automated pallet load stabilization system 900 and an empty pallet 904 may be provided within 10-12 seconds. In such a configuration, the automated pallet load stabilization system 900 may process a cycle time of 72 seconds, or 50 pallets 904 per hour. In one embodiment, three parallel configurations of robotic unloading systems 300 may be operated to simultaneously unload three trucks or shipping containers, palletize their contents, and send their contents to the automated pallet load stabilization system 900. This may provide an unloading, palletizing, and stabilization capacity of 3,000 cases per hour in some embodiments.

[0119] 10 illustrates a simultaneous wrapping and lifting operation 1000 according to one embodiment. The simultaneous wrapping and lifting operation 1000 may be performed by simultaneous operation of elements of a rotating pallet wrapping mechanism 916, such as an actuator system 914 and a pallet sleeve lifter 910 that run along a rotating ring 918 and carry a packaging material spool pin 924. These components may be supported by a support structure 912 as previously described, which is not shown here for ease of illustration. Those skilled in the art will readily appreciate the supports necessary to implement these configurations.

[0120] The at least partially stacked pallet sleeve assemblies 322 may be positioned within the rotating ring 918 of the rotating pallet wrapping mechanism 514. The pallet sleeves 908 of the at least partially stacked pallet sleeve assemblies 322 may be engaged by a pallet sleeve lifter 910. The pallet sleeve lifter 910 may begin to perform a pallet sleeve lift operation 1002 to begin raising the pallet sleeves 908 and exposing the MTBHs 402 to be loaded onto the pallets 904.

[0121] The actuator system 914 may initiate a rotational wrapping operation 1004 and a packaging material spool pin lifting operation 1006 to travel around the rotating ring 918 and thus move the packaging material spool pin 924 around the MTBH 402 to be wrapped. This results in a rotational wrapping 1008 of the MTBH 402 in the packaging material 1010 starting at the level of the pallet 904 and spiraling upward as the pallet sleeve 908 rises until all of the MTBH 402 are released from the pallet sleeve 908 and securely wrapped.

[0122] In one embodiment, the connection between the actuator system 914 and the packaging material spool pin 924 may be dynamically adjustable such that the actuator system 914 moves in a horizontal plane rather than vertically while lifting the packaging material spool pin 924. In another embodiment, the rotating ring 918 may be configured to simultaneously lift the pallet sleeve 908 and raise the actuator system 914, and the actuator system 914 may be statically connected to the packaging material spool pin 924, such that the packaging material spool pin may be lifted as well. In one embodiment, a stationary packaging material spool pin 924 may be used in conjunction with a device that rotates the pallet sleeve assembly, provided that the pallet sleeve lifter 910 accommodates the pallet sleeve rotation. One of ordinary skill in the art will readily appreciate other configurations for implementing this disclosure.

[0123] FIG. 11A shows a side view of a pallet sleeve assembly 1100 according to various embodiments of the present disclosure. FIG. 11B shows a front view of the pallet sleeve assembly 1100, and FIG. 11C shows a perspective exploded view of the pallet sleeve assembly 1100 showing the interface between a pallet 904 carrying a load requiring stabilization and a pallet sleeve 908. The pallet sleeve assembly 1100 is a multi-part assembly that supports palletizing, transporting, storing, retrieving, and de-palletizing of products. It is intended to be lightweight, low-cost, and mass producible, and therefore, it has few components.

[0124] The pallet sleeve 908 may include structural walls. These structural walls may be made of aluminum, steel, plastic, wood or other suitably lightweight, rigid, and sturdy materials so that they retain their shape during loading, transportation, storage, retrieval, and unloading. The primary purpose of the pallet sleeve assembly 1100 is to provide support during the palletization step, as well as during transportation and storage if the at least partially stacked pallet sleeve assembly 322 is not to be packaged for storage. A conventional commonly used pallet 904 may be 40 inches wide by 48 inches long by 5.6 inches high. Those skilled in the art will readily appreciate that the pallet sleeve 908 may be variously dimensioned in the horizontal plane based on the dimensions of the footprint of the different pallets 904 and rotating ring 918, and in the vertical plane to accommodate different load heights. The pallet sleeve 908 may be sized to easily slide over the pallet while fitting within the rotating ring 918 of the automated pallet load stabilization system 900 as shown in FIG. 9A, which may have an inside diameter of about 74 inches. For example, the pallet sleeve 908 may be 42 inches by 40 inches, providing 1 inch of clearance all around the circumference of a conventional pallet while still allowing it to fit within a conventional rotating ring as described above. The pallet sleeve 908 may be 66 inches tall to support loads up to 5 feet tall on a conventional 5.6 inch tall pallet, or may be made taller to support taller loads or shorter to support shorter loads while reducing material usage.

[0125] Lifting features 1102 may be provided to aid in the placement and lifting of the pallet sleeves 908 from the pallet 904. Such lifting features 1102 may be tooling holes that provide kinematic mounts for engagement during assembly or disassembly as part of the stretch wrap process, as described with respect to FIG. 9A. The lifting mechanism 1102 may be designed to allow an actuator, such as the pallet sleeve lifter 910, to engage the pallet sleeve, lift the pallet sleeve, and then lower the pallet sleeve during a pallet building operation.

[0126] The clearance holes 1106 may be configured to allow a forklift to enter the pallet sleeve 908 and engage the pallet 904 and pallet sleeve 908 together. The clearance holes 1106 may be provided on all sides of the pallet sleeve 908. A wear surface 1104 may be included to provide a non-metal to metal wear resistant interface between the sleeve and the MHE it travels through. The wear surface 1104 may be made from a material such as ultra-high molecular weight (UHMW) polyethylene, acetal, polyvinyl chloride (PVC), or acrylonitrile butadiene styrene (ABS). The four sides or sleeve walls 1108 of the pallet sleeve 908 may be unique pieces that are assembled in place. Sheet metal or other methods of fastening the pieces together may be utilized.

[0127] FIG 12A shows a side view of an automated crane storage and retrieval system 1200 according to various embodiments of the present disclosure. FIG 12B shows a front view of the automated crane storage and retrieval system 1200, and FIG 12C shows a top view of the automated crane storage and retrieval system 1200.

[0128] Manual systems, such as forklifts, very narrow aisle or reach trucks, may be used. For this embodiment, an automated crane storage and retrieval system 1200 is shown that includes a pallet crane 1202. This allows for full automation of the flow of material from the inbound dock to the palletizer, stretch wrapper, and storage, as shown. Once the product is placed in the queue 1212, the pallet crane 1202 may move to a pick-up location 1208 and the pallet end effector 1220 may grab the pallet 904. Similarly, when the pallet is being stored, the operation may be reversed. An additional drop-off location 1214 may allow a mobile robot or a manual forklift to drop off the product at the queue 1212. The storage racks 1204 may be configured such that each storage rack has sufficient clearance to place and retrieve pallet-sleeve pairs at the storage location. High bay pallet racking 1206 may be used, where the rack height may extend up to 130 feet high and provide hundreds of thousands of storage locations. The height of the pallet crane 1202 is determined by the available clearance height, the storage racks 1204 it needs to reach, and the stability of the crane configuration. The pallet sleeve 908 may be designed to accept the transfer from the pallet crane 1202. The pallet sleeve 908 may be designed to accept thousands of use cycles and may have an estimated return on investment of approximately 500 pallets, assuming each pallet costs $1.50-1.75 in stretch packaging materials. End stops 1210 on the pallet crane 1202 may allow side-to-side movement of the pallet crane 1202 between the storage racks 1204. During transportation of the pallet sleeve 908, the sleeve may remain on the outside of the pallet with its bottom surface riding with the pallet on the MHE. The pallet sleeve 908 may be designed to be a removable outer shell.Additionally, the pallet sleeve assembly 1100 may be designed so that a remote automated pallet crane 1202 that is not on rails 1216 can come and remove a case from storage as long as there is sufficient clearance between the pallet sleeve 908 and the storage rack 1204.

[0129] The pallet sleeve assemblies 1100 may enter from the MHE 1218 and the pallet crane 1202 may traverse its rails 1216 and pick them up. Once on the pallet crane 1202, the pallet sleeve assemblies 1100 may be transported to any storage location housed within a storage rack 1204. The end stops 1210 of the pallet crane 1202 are shown as being relatively close to each other. However, typical configurations of the automated crane storage and retrieval system 1200 can be very long. This illustrates one of the core capabilities of the system in that the pallet sleeve assemblies 1100 can be stored in very large storage systems without fear that the pallet stacks will become unstable, plus with a special type of robotic system, cases can be removed while the pallets are stored.

[0130] Figure 13 illustrates a loading / unloading control system 1300 according to one embodiment. The loading / unloading control system 1300 may include a computer system 1306 configured to perform the disclosed activities, industrial control systems 1304, and a perception system 1302, as described with respect to the automated truck loading / unloading routine 1700 of Figure 17 and the end-to-end control system diagram 1600 of Figure 16.

[0131] In one embodiment, the perception system 1302 may receive input from a camera system 1308, load cells 1310, pressure sensors 1312, and other sensors 1314 such as LIDAR, inertial measurement units, sonic locators, etc. The industrial control system 1304 may include controls for pneumatics 1316 and hydraulics 1318 for the industrial equipment of the robotic loading and unloading system 300, as well as encoders 1320 and motors 1322 for the mechanical and robotic components of the robotic loading and unloading system 300.

[0132] The computer system 1306 of the unloading control system 1300 may include a computing device 2300 as described with respect to Figure 23. The data and logic necessary to perform the disclosed activities may be stored in any number of computing devices 2300, including the computer system 1306, which may be connected via wired and / or wireless network technologies.

[0133] "Logic" refers to non-transitory machine-readable media, including machine memory circuits and machine-executable instructions (software and firmware), and / or circuitry (hardware) including control and / or procedural signals through its material and / or material-energy configurations, and / or settings and values ​​(such as resistance, impedance, capacitance, inductance, current / voltage ratings, etc.) that may be applied to affect the operation of a device. Magnetic media, electronic circuits, electrical and optical memory (both volatile and non-volatile), and firmware are examples of logic. Logic specifically excludes pure signals or software per se (but does not exclude machine memory, which includes software and thereby forms a configuration of matter).

[0134] Figure 14 illustrates an unloader palletization control system 1400 according to one embodiment. The palletization control system 1400 may include a computer system 1406, an industrial control system 1404, and a perception system 1402 configured to perform the disclosed activities as described with respect to the automated palletization system 318 of Figures 3A-3C and the end-to-end control system diagram 1600 of Figure 16.

[0135] In one embodiment, the perception system 1402 may receive input from a camera system 1408, load cells 1410, pressure sensors 1412, and other sensors 1414 such as LIDAR, inertial measurement units, sonic locators, etc. The industrial control system 1404 may include controllers for pneumatics 1416, hydraulics 1418, and motors 1420 for the industrial equipment of the automated palletizing system 318.

[0136] The computer system 1406 of the palletization control system 1400 may include a computing device 2300 as described with respect to Figure 23. Data and logic necessary to perform the disclosed activities may be stored in any number of computing devices 2300, including the computer system 1406, which may be connected via wired and / or wireless network technologies.

[0137] Figure 15 illustrates an unloader load stabilization control system 1500 according to one embodiment. The load stabilization control system 1500 may include a computer system 1506, an industrial control system 1504, and a perception system 1502 configured to perform the disclosed activities as described with respect to the automated pallet load stabilization system 900 of Figures 9A and 9B and the end-to-end control system diagram 1600 of Figure 16.

[0138] In one embodiment, the perception system 1502 may receive input from a camera system 1508, location sensors 1510, and other sensors 1512 such as LIDAR, inertial measurement units, sonic locators, etc. The industrial control system 1504 may include controls for pneumatics 1514, hydraulics 1516, and motors 1518 for the industrial equipment of the automated pallet load stabilization system 900.

[0139] The computer system 1506 of the load stabilization control system 1500 may include a computing device 2300 as described with respect to Figure 23. Data and logic necessary to perform the disclosed activities may be stored in any number of computing devices 2300, including the computer system 1506, which may be connected via wired and / or wireless network technology.

[0140] 16 illustrates an end-to-end control system diagram 1600 according to one embodiment. The end-to-end control system diagram 1600 illustrates key communication and MTBH flows between the robotic unloading system 300, the automated palletizing system 318, the mobile transport fleet manager 1604 and the AMRs 324 it manages, the automated palletized load stabilization system 900, and the automated crane storage and retrieval system 1200 to manage the unloading, handling, and storage of MTBHs 402 throughout a material handling facility.

[0141] The robotic unloading system 300 may be comprised of an unloading control system 1300 as shown in FIG. 13. The automated palletizing system 318 may be comprised of a robotic unloading system 300 as shown in FIG. 14. The automated palletizing system 900 may be comprised of a load stabilization control system 1500 as shown in FIG. 15. Each of these controllers may include a computer system that sends control communications 1602 to each other over a wired or wireless network. These controllers may be further networked to communicate with a mobile transport fleet manager 1604 that manages the AMRs 324 via wireless fleet control 1606. In one embodiment, all of the computer systems in the material handling facility may be wired into a facility-wide EtherCAT network.

[0142] The unloading control system 1300 may access and update stored unloading data 1608, such as container inventory 1610, picking sequence 1612, and picking rate 1614. The palletizing control system 1400 may access and update stored palletizing data 1616, such as incoming box sequence 1618, sorting logic 1620, pallet status 1622, and current MTBH amount per pallet 1624.

[0143] During a typical material unloading and handling job, the MTBHs 402 may be transferred from the robotic unloading system 300 to the automated palletization system 318 via conveyors and other MHEs. After palletization, the MTBHs 402 may be transported by the AMR 324 to the automated palletized load stabilization system 900 to be wrapped for storage without pallet sleeves or sent directly to the automated crane storage and retrieval system 1200 for storage in a pallet sleeve. The shrink-wrapped palletized load may be accepted by the AMR 324 and then transported to the automated crane storage and retrieval system 1200.

[0144] The speed at which the end-to-end system functions may be determined by the rate at which the MTBHs 402 are palletized by the automated palletization system 318. The palletization control system 1400 may receive manifest information including the quantity, size, and type of MTBHs 402 contained in the container to be unloaded. Using this information, the palletization control system 1400 may determine a set of pallet loads to be filled when an empty MTBH 402 arrives at the automated palletization system 318. This data may be used by sorting MHEs that feed the automated palletization system 318.

[0145] In one embodiment, the MHE may direct the MTBH to a small buffer lane. However, when the MTBH flows directly to the automated palletization system 318, the numbers in the shipping manifest may be accounted for as palletized and may be used to inform the automated unloader 500 regarding the picking sequence to optimally fill the pallet or pallets in progress. The automated unloader 500 may be programmed to pick-to-exhaustion to reduce the amount of MTBH that needs to be set aside or sorted into a buffer area.

[0146] In one embodiment, the manifest data, in combination with other stored unloading data 1608, as well as picking sequence 1612 data that may be provided by the unloading control system 1300, may be used as an input to a flow control feedback loop 1626 by the computer system 1406 of the palletizing control system 1400 to monitor or calculate the rate 1628 at which palletizing is performed by the automated palletizing system 318. The rate data 1630 may then be communicated as part of the control communications 1602 to other control components, such as the mobile transport fleet manager 1604, the unloading control system 1300 of the robotic unloading system 300, and the load stabilization control system 1500 of the automated palletized load stabilization system 900. These components may use the rate data 1630 to control how quickly they perform their local tasks to provide the smoothest flow of MTBH 402 throughout the end-to-end system.

[0147] 17 illustrates an example automated truck loading / unloading routine 1700 that may be performed by an automated unloader, such as the automated unloader 500 illustrated in FIGS. 5A and 5B. Although the example automated truck loading / unloading routine 1700 illustrates a particular sequence of operations, the sequence may be modified without departing from the scope of the present disclosure. For example, some of the illustrated operations may be performed in parallel or in different sequences that do not substantially affect the functionality of the automated truck loading / unloading routine 1700. In other examples, different components of an example device or system implementing the automated truck loading / unloading routine 1700 may perform functions simultaneously or in a particular sequence.

[0148] According to some examples, the automated truck loading and unloading routine 1700 includes inputting shipping manifest data provided by a customer into a computer control system at block 1702. The computer control system may include one or more computing devices 2300 as described with respect to FIGURE 23. The shipping manifest data may include an inventory of each item of MTBH in the container, including the SKU, weight, height, width, and length of each MTBH.

[0149] According to some examples, the automatic truck unloading routine 1700 includes positioning the automatic unloader 500 within a shipping container. An on-site operations team at a material handling facility may receive a container, trailer, or truck at a loading dock, as shown in FIG. 4. The container may be opened by the team, and the automatic unloader may be manually positioned within the container at a location suitable for initial scanning and movement of the automatic unloader after starting automatic operations. The team may then place the automatic unloader in its automatic mode, and the automatic unloader may begin automatic operations at block 1704.

[0150] According to some examples, the automated truck unloading routine 1700 includes scanning an area within the container to detect MTBHs at block 1706. The scan may be performed by a perception sensor, such as a camera, LIDAR, barcode scanner, etc. In one embodiment, the sensor may provide a detailed scan of a 3 foot by 3 foot area so that one scan does not cover the entire wall of a box or package containing MTBHs stacked within a trailer, truck, or other container. To provide complete coverage, each articulated robot may use its camera to create a map of its side of the container to ensure overlap between scans taken. In such a case, the automated unloader may begin scanning at a default top left position based on expected or entered dimensions of the shipping container or trailer. In another embodiment, a lower resolution overall scan may be acquired first to provide an initial dimension for the wall of MTBHs stacked within the container, and a more detailed scan may begin at the top left most MTBH detected. Scans are taken, in some embodiments, on the order of 0.5 seconds for acquisition and processing, allowing scans of multiple regions to be taken without detrimental loss of time.

[0151] The system may scan the faces of the boxes stacked in the container and may locate them in three-dimensional space in terms of three Cartesian locations and three rotational offsets from the principal axes. This is the amount of information required to define a three-dimensional object in three-dimensional space. The three translational and three rotational components may define the location and orientation of the box in space, and from there the contact normal point, represented as the centroid of each side face and the picking face.

[0152] In situations where the picking surface is not flat, an approximation of the center of gravity on the surface topology may be made. In one embodiment, the automated unloader 500 may include two separate industrial collaborative robots mounted on the base of the automated unloader, as shown in FIG. 5A. Each robot may be responsible for scanning its own portion of the shipping container (i.e., the right portion or the left portion, as sensed by the robot directly facing inside the container). The two portions may include some overlap to ensure that no gaps are left in the scanned MTBH data.

[0153] According to some examples, the automated truck loading / unloading routine 1700 includes building a virtual representation for each scanned container portion and unit of MTBH at block 1708. The virtual representation in one embodiment may be generated by a master robot controller or a robot operating system (ROS) computer. The program may have an underlying code that sets the expected environment and parameters, and an interface that provides TCP / IP control for the robot's motion. To complete the virtual representation from the robot scan, a behavior tree may be used. As will be well understood by those skilled in the art, a behavior tree provides a mathematical model that includes a root node, control flow nodes, and execution nodes, and is used to represent complex operations as a series of simple operations. The software stack for the system may run on a simulation framework such as Gazebo. RViz or a similar program may be used to represent the visualization of the perception sensors.

[0154] The virtual representation may use 6DOF estimates for the pose of each unit of MTBH detected in the container. In one embodiment, the automated unloader control system may access the shipping manifest entered in block 1702 via an application programming interface (API) connection to the customer's warehouse management system. The virtual representation may be verified for expected boxes, and data provided by the manifest may be used to verify that detected boxes are within the handling constraints of the unloading robot. For example, the unloading robot can handle MTBH with dimensions of 24 inches or less and weighing 30 pounds or less. If any detected and manifested items exceed those parameters, they may be ignored in the resulting unloading plan or may require handling, such as pushing to one side rather than picking up.

[0155] If the entire visible wall has not yet been scanned at decision block 1710, the automatic truck unloading routine 1700 may return to block 1706 to identify and scan the next area within the container. This next scan area may overlap the previous area scanned so that no gaps are left in the scanned MTBH data. If the entire visible wall has been scanned at decision block 1710, the automatic truck unloading routine 1700 may proceed to block 1712.

[0156] According to some examples, the automated truck loading and unloading routine 1700 includes using the virtual representation built iteratively through completion of block 1708 for the entire scanned visible wall to simulate a motion plan for the MTBH in block 1712. For each candidate item in the MTBH, a feasible, safe, and complete motion path from the source location to the destination location may be calculated. However, since each side of the MTBH is not necessarily available, picking sequences may also be considered. The motion plan for each MTBH may be a function of incremental positions assumed by a mobile base that attaches the industrial collaborative robot to the loading conveyor. The motion plans provided in the Open Motion Plan Library (OMPL) created by Kavraki Lab may be used.

[0157] Different motion plans may control the robot for different segments. For example, if it is important for the robot to follow a particular line of motion, Cartesian motion plans may be used. If the motion space is constrained, a probabilistic roadmap may be constructed and stored as a graph with nodes corresponding to collision-free configurations and edges corresponding to feasible motion paths between those configurations. The motion plan may take the state of the environment (such as the absolute and relative locations of the six sides of each box within the container's boundary), the state of the automatic unloader 500 (such as robot joint angles, encoder data, and information from other sensors), and a high-level plan from the behavior tree. The output of the motion plan may be a trajectory including a sequence of six floating-point variables representing joint angles as a function of time, which may be sent to the robot's motors for execution.

[0158] The scanned data may be added to a learning library in some embodiments and input to machine learning to train machine algorithms to more quickly detect and identify MTBH from classification and segmentation of configurations detected through the scan sensor. In one embodiment, a machine learning device such as Fizyr may be used. Machine learning may help understand the pose of the MTBH over time for a good selection given the detected candidates. Successful and unsuccessful picks may be used for reinforcement learning.

[0159] The virtual representation may be stored for use in internal testing and validation as well as the continuous integration of new data such that a "digital twin" of the shipping container's real-world packing configuration is maintained. The digital twin conceptualization, well known in the fields of automation and robotics, provides the ability to test operational code in a simulated environment prior to or in parallel with execution in the real environment.

[0160] According to some examples, block 1714 of the automated truck unloading routine 1700 includes scoring each motion plan generated in block 1712 based on the motion required by the robot base to accomplish the pick. As described with respect to FIGS. 5A and 5B, the industrial collaborative robot that accomplishes the picking may be mounted on a base located at the end of a loading conveyor that is capable of forward and backward lateral movement (i.e., extending further into the container or retracting into and out of the container opening) and left-to-right rotational movement (pivoting at the loading conveyor pivot as shown). In one embodiment, the load conveyor may be able to pivot upward and downward at the loading conveyor pivot through a lifting mechanism. These ranges of motion provided at the base are sometimes referred to as auxiliary degrees of freedom (as opposed to the six degrees of freedom that may be provided by each robot). The best score for each motion plan may be associated with the plan that minimizes this lateral and rotational movement of the base, and may be associated with vertical adjustments, if applicable. In one embodiment, factors such as position, velocity, acceleration, jerk, snap, trajectory normalized distance, and other attributes related to configuration space, free space, target space, and obstacle space may be included in the scoring. The scoring algorithm may apply different weightings to these variables based on different physical environment configurations and characteristics of the MTBH. For example, jerks and snaps may be weighted more if the MTBH is expected to contain fragile objects.

[0161] According to some examples, the automated truck unloading routine 1700 includes generating an optimal picking sequence from the candidate motion plans with the best scores at block 1716. The optimal picking sequence may include a dual-arm picking strategy for the difficult MTBH (as opposed to the typical case where one robot arm picks one item of the MTBH independent of the action of the other robot). The optimal picking sequence may also account for synchronization between the motion of the two arms to prevent the motion of one robot from interfering with the motion of the other robot or even damaging the other robot. The optimal picking sequence may also consider timing for the motion of the inclined conveyor and may maximize the maneuverability of the robot arms. To calculate these picking sequences, the motion plans on each feasible surface may be examined.

[0162] According to some examples, the automated truck unloading routine 1700 includes selecting a pick from the optimal picking sequence at block 1718. Each robot may independently perform a series of picks from the sequence following the steps of the pick routine for the individual articulated robot 1800 shown in FIG. 18. In one embodiment, the system may place the first item of the MTBH on the exception path conveyor instead of the loading conveyor so that the item may be easily inspected later by the operations team. If the container is loaded with snake SKUs or has mixed contents, the system may place the first of each unique SKU on the exception path conveyor.

[0163] The MTBH of different sizes can affect how fast an articulated robot may perform picks of a picking sequence. For very small boxes, the automated unloader 500 may load and unload 700-1400 cases per hour (CPH). Small and medium sized boxes, which an articulated robot's end effector may be primarily designed to handle, may be loaded and unloaded at speeds of 800-1,100 CPH and 800-1,200 CPH, respectively. Larger boxes, such as boxes over 20 inches in each dimension, may have typical loading and unloading speeds of 600-900 CPH.

[0164] At decision block 1720, the automated truck unloading routine 1700 may determine whether the pick executed via the pick routine for each articulated robot 1800 was successful. If it is determined that the pick was not successful, the reason for the failure may be evaluated at block 1722. The pick failure may be the result of an articulated robot fault, an end effector fault, a perception fault, a motion planning fault, or various other static or dynamic conditions in the unloading environment. If it is determined at decision block 1724 that the failure was a robot, end effector, or perception fault, the automated truck unloading routine 1700 may return to select a different pick from the optimal picking sequence at block 1718.

[0165] If at decision block 1724 it is determined that the failure is due to reasons other than a robot, end effector, or perception defect, the motion plan may be recalculated and the picking sequence updated at block 1726. The motion plan and picking sequence may be recalculated based on the aspect ratio of the picking face of the item being picked, the weight of the item as indicated on the shipping manifest, and / or the individual dimensions of the object, such as its height, its width, or its length. The motion plan and picking sequence may be updated based on the position of the item relative to other MTBHs, or other parameters of the environment, including the container and MTBH. The new pick may be executed at this time or may be included in the picking sequence for later execution. The automated truck unloading routine 1700 returns to block 1718 to continue picking the MTBH.

[0166] In one embodiment, if a defect, exception, or other failure occurs while operating in the automatic mode, the operations team may stop the automatic operation and enter a manual or maintenance mode. In one embodiment, the operations team may choose to manually remove the MTBH causing the exception and set it aside, and then allow the automatic unloader 500 to continue operating in the automatic mode.

[0167] If at decision block 1720 it is determined that the pick was successful, the automated truck unloading routine 1700 may proceed to block 1728. According to some examples, the automated truck unloading routine 1700 includes scanning the environment, updating the virtual representation, and revising the optimal picking sequence if necessary at block 1728. In this manner, the digital twin of the container environment may be updated to account for the removal of the picked MTBH and may include the newly revealed MTBH due to the gap created by picking the previous item. If a new item of MTBH is detected, its location may be resolved and it may be added to the virtual representation. If a box that is already part of the virtual representation is detected, its scanned location may be compared to its location in the virtual representation to ensure that it has not moved to an extent that would disrupt its previously identified pick strategy. For example, if the MTBH has moved more than a position change threshold, its virtual representation location and that portion of the motion plan may be updated. Such a threshold may be an absolute value, such as 3 inches, or may be a ratio, such as 40% of the item's dimension in the direction of motion (i.e., 4 inches for a 10 inch wide object shifted horizontally). In one embodiment, a robot-mounted camera or other sensor may be instructed to scan manually or automatically on-demand at any point during operation if a defect is detected in performance or visual representation.

[0168] From decision block 1720, for a successful pick, the automated truck unloading routine 1700 may branch to block 1732. At block 1732, the scan tunnel may be activated to inspect the MTBH placed on the conveyor by the articulated robot executing the pick routine for the individual articulated robot 1800. In this manner, the MTBH may be inspected as it moves along the telescopic transport conveyor toward the automated palletizing system. The scan tunnel may perform five or six side scans to identify the MTBH with respect to the expected item in the manifest using optical character recognition (OCR), machine learning, pattern matching, or other identification techniques that may be utilized based on 2D and 3D imaging performed in the scan tunnel. These techniques and operations may include turning on the light panel when the MTBH breaks the retro-reflective sensor beam, capturing an image from the scan tunnel camera, extracting logistics information such as SKU based on extraction and analysis of the MTBH label from the image data, and verifying the extracted label with the manifest for the container unloaded.

[0169] At decision block 1730, the auto truck unload routine 1700 may determine whether the visible walls scanned at blocks 1706-1710 have been completely cleared. If the walls have not yet been cleared, the auto truck unload routine 1700 may return to block 1718 to perform the next pick in the picking sequence. If the walls have been cleared at decision block 1730, the auto truck unload routine 1700 may return to block 1706 to check the next wall of MTBHs or determine that no additional MTBHs remain in the container. Once all the MTBHs have been removed from the container, the auto unloader 500 may be automatically pulled back to the dock and may wait to serve a new container.

[0170] 18 illustrates an example pick routine 1800 for an individual articulated robot. Although the example pick routine 1800 for an individual articulated robot illustrates a particular sequence of operations, the sequence may be modified without departing from the scope of the present disclosure. For example, some of the operations illustrated may be performed in parallel or in different sequences that do not substantially affect the functionality of the pick routine 1800 for an individual articulated robot. In other examples, different components of an example device or system implementing the pick routine 1800 for an individual articulated robot may perform functions simultaneously or in a particular sequence.

[0171] According to some examples, a pick routine 1800 for an individual articulated robot includes, at block 1802, transmitting 1D, 2D, and 3D sensor data indicative of the pose and position of the MTBH within the container to a control system. The 1D sensors may include pressure sensors, load cells, sonic locators, or similar sensors that capture information relative to one-dimensional points in space. The 2D sensors may include cameras capable of capturing two-dimensional images representative of a field of view within the container. The 3D sensors may include imaging technology capable of detecting or calculating depth within the field of view as well as planar data captured by the 2D sensors. The control system may include or be connected to the loading / unloading control system 1300 described with respect to FIG. 13 .

[0172] According to some examples, a pick routine 1800 for an individual articulated robot includes receiving a motion plan for an optimal pick sequence at block 1804. As described with respect to the automated truck loading / unloading routine 1700, the motion plan may act to minimize motion required across the non-robot or auxiliary DOFs. The pick sequence data may instruct the robot how to move across at least six of its DOFs in three-dimensional space to contact desired surfaces of items of the MTBH.

[0173] According to some examples, a pick routine 1800 for an individual articulated robot includes moving its end effector into position to grasp the MTBH and following the discrete steps provided by the motion plan, minimizing joint rotations and tool point arcs at block 1806.

[0174] According to some examples, a pick routine 1800 for an individual articulated robot includes positioning the end effector at a contact normal with the MTBH using a 1D sensor for positional contact control at block 1808. "Contact normal" refers to a position where the end effector may engage the MTBH such that motion normal or perpendicular to the contact surface allows the articulated robot to pick the MTBH from a wall or stack of other MTBHs.

[0175] According to some examples, the pick routine 1800 for an individual articulated robot includes initiating an end effector to engage the MTBH at block 1810. In one embodiment, the end effector may be a pneumatic vacuum end effector. Initiating such an end effector includes physically contacting a flat surface of the MTBH and engaging a vacuum suction to pull the MTBH securely against the end effector so that the MTBH is held by this vacuum force as the articulated robot pulls the MTBH against frictional forces exerted by adjacent MTBHs and remains securely held against gravity and inertial forces as the robot arm traverses the void space from the MTBH wall or stack to the loading conveyor. In another embodiment, the end effector may include fingers that engage a pinching motion or other fixed structure that may be engaged with the MTBH, as would be readily understood by one of ordinary skill in the art.

[0176] According to some examples, the pick routine for individual jointed robot 1800 includes verifying that the grip is complete using 1D and pressure sensors at block 1812. If the grip is not verified to be complete, the robot may return to the start of the pick routine for individual jointed robot 1800, or terminate the pick routine for individual jointed robot 1800 to allow other actions to be taken to achieve a successful grip.

[0177] According to some examples, the pick routine 1800 for an individual articulated robot includes lifting the MTBH via the contact normal vector once the grip is verified, at block 1814. In one embodiment, the offset of the MTBH from its original position may be verified, at block 1816. This may provide another layer of verification that the grip is complete and may allow for observation and calculation points to ensure that nothing is currently obstructing the intended motion path for the MTBH to the loading conveyor.

[0178] According to some examples, the pick routine 1800 for an individual articulated robot includes transferring the MTBH to a loading conveyor at block 1818. The automatic truck unloading routine 1700 may then weigh the MTBH using a 1D force sensor at block 1820. In some embodiments, the weighing may be completed by some other process, such as matching with a package manifest, using a sensor in the loading conveyor or along other waypoints away from the container. The pick routine 1800 for an individual articulated robot may end with depositing the MTBH on the loading conveyor at block 1822.

[0179] Figure 19 illustrates an automatic truck unload flow diagram 1900 according to one embodiment. The illustrated steps may be performed by the unload control system 1300 described in Figure 13. In general terms, the automatic truck unload flow diagram 1900 illustrates how the unload control system 1300 may perform the automatic truck unload routine 1700 in one embodiment.

[0180] The automated truck unloading flow diagram 1900 may begin with a full wall scanning step 1902. A move conveyor down step 1904a, a move conveyor middle step 1904b, and a move conveyor up step 1904c may be performed, each followed by process left and right images 1906. In this manner, or in more or fewer steps, all walls of the MTBH may be scanned within the container.

[0181] After the full wall scan 1902 is completed, the workflow may proceed to a detection cache flow diagram 2000, which is described in more detail with respect to Figure 20. In general terms, the detection cache flow diagram 2000 may include a generate detection cache 1908 step and an update detection cache 1910 step.

[0182] The data generated during the flows described in the detection cash flow diagram 2000 may be used by the planner operations as described in the planner flow diagram 2100 shown in Figure 21. Through operations according to the planner flow diagram 2100, a motion plan may be generated in a generate motion plan step 1912 and cached in a cache motion plan step 1914.

[0183] The sensed cache data from sensed cache flow diagram 2000 and the planned data from planner flow diagram 2100 may be provided for use during execution, as described with respect to execution flow diagram 2200 shown in FIG. 22. The execution flow diagram 2200 may proceed to a move conveyor 1916 step that positions the robot within a vicinity to perform a pick 1918 as indicated by a motion plan generated through operation according to the planner flow diagram 2100. Completing the perform a pick 1918 step may include operations to execute trajectories 1920 that contact the pick surface, and update world state 1922 based on changes that occurred through the execute trajectories 1920 step. A wall clear 1924 check may be performed, and if the wall is not cleared, the execution flow diagram 2200 may return to the move conveyor 1916 step. If the wall is cleared, the execution flow diagram 2200 may proceed and move to the next wall 1926. The execution flow diagram 2200 may continue as long as the MTBH is in the container. Execution may end when all walls are cleared, as determined by a container cleared check 1928.

[0184] 20 illustrates a detection cash flow diagram 2000 according to one embodiment. The detection cash flow diagram 2000 may begin by waiting to receive data for MTBH detection 2002. The detection cash flow diagram 2000 may receive MTBH detection 2004 data based on an analysis of image data and point cloud data captured during a sensory scan. If no data is available at the detection available 2006 check, the detection cash flow diagram 2000 continues to wait. If available, an order and filter detection 2008 step may be performed.

[0185] The MTBH detection data may include information about the material to be handled that corresponds to the picking poses for gripping the material and the segmentation of the path to be traversed to achieve the picking pose. Picking poses refer to the positions that an articulated robot needs to achieve to grip an item of the MTBH. These picking poses for the detected MTBH may or may not be related to an MTBH that is part of a scanned wall where loading / unloading is in progress. If a wall is determined as part of the wall 2010 check, it may be stored in the wall cache 2012. If it is not part of a wall, it may be stored in the container cache 2018.

[0186] The detection cache flow diagram 2000 may request a picking pose pair plan 2022 as an output from the planner flow diagram 2100. A picking pose pair refers to two picking poses shown for a coordinated motion step by two joint robots. One of the pair of picking poses may be for execution by the left hand joint robot, and the other of the pair may be for execution by the right hand joint robot. The picking pose pair plan may include a trajectory describing the movements that each of the two joint robots may need to perform to complete the next part of the motion plan. The planner flow diagram 2100 may include an operation of sending a request back to the detection cache for identifiers (IDs) (pick pose pair IDs 2024) for the pick pose pairs requested by the plan. These pick pose pair IDs 2026 may be provided by the detection cache to the planner or execution flow.

[0187] If the wall cache empty 2014 check indicates that the wall cache is empty, then a switch container cache to wall cache 2016 may be performed. At this point, the get new picking pose pair IDs 2020 task may be completed and a picking pose pair ID 2026 is output.

[0188] For the following parameters: ld: detection vector wallCache: a vector of all detections close to the system containerCache: a vector of all detections that do not belong in the wall cache Pseudocode corresponding to this detection cash flow diagram 2000 may be as follows:

number

[0189] The planner flow diagram 2100 may receive as input a requested pair of picking poses 2102 from the detection cache. These are picking poses for two different detections provided by the detection cache. The planner flow diagram 2100 may include waiting 2104 for the requested pose if a pose available 2106 check indicates that there is no available pose. If a picking pose is available, a dual arm trajectory planner 2108 may operate to determine a plan for the requested picking pose. If the plan available 2110 check indicates that there is no available plan, the planner may wait for the next request. In one embodiment, the planner may send a success or failure indication to the detection cache.

[0190] If a plan is available, as determined in the available plan 2110 check, a send trajectory to cache 2112 step may be executed. The planner may request picking of pose IDs from the detection cache 2114 and output planned trajectories for the requested pick pose pairs 2116. These are the generated trajectories. Each trajectory has three different sub-trajectories: 1) home, 2) pick, and 3) drop. Each sub-trajectory of each arm may be synchronized in orchestration with the entire system, as described with respect to the execution flow diagram 2200 shown in FIG. 22.

[0191] Regarding the following parameters: (outside 1) TIFF2024544928000005.tif638: Requested picking pose pair (outside 2) TIFF2024544928000006.tif836: Planned trajectory pair Pseudocode corresponding to this detection cash flow diagram 2000 may be as follows:

number

[0192] 22 shows an execution flow diagram 2200 according to one embodiment. The execution flow diagram 2200 may begin with the moving arm to start at position 2202 so that the articulated robot arm is in its intended initial position. This may be the same as the default home position for each articulated robot, or may be an optimized pose for the next expected movement based on the overall motion plan recognition and picking sequence. The execution flow diagram 2200 may obtain a picking pose 2204 by sending a signal requesting a picking pose from a detection cache 2206.

[0193] If the available picking poses 2208 indicates that picking poses are not available, the detection cash flow diagram 2000 may return to the get picking poses 2204 step. If picking poses are available, the execution flow diagram 2200 may execute a move conveyor 2210 step as necessary to move the loading conveyor between the up, center, and down positions to guide the articulated robot to a position suitable for the picking pose. The planner flow diagram 2100 may then request motion plan trajectories from the planner 2212. The available motion plans 2214 check may indicate that plans are not available, at which point the request motion plan trajectories from planner 2212 step may be repeated. If plans are available, these plans may be executed.

[0194] In one embodiment, executing the plan includes a first move left arm home and pick with right arm 2216 step followed by take picture on left side 2218 step. Next, a pick with left arm and drop with right arm 2220 step may be performed followed by a drop with left arm and move home with right arm 2222 step followed by a take picture on right side 2224 step. Based on the pictures taken on the left and right, a cleared wall 2226 check may be performed. If it is determined that the wall is not cleared, an additional picking pose may be obtained.

[0195] If it is determined that the wall is cleared, a move arms to home position step 2228 may be performed followed by a move conveyor to high position step 2230, followed by a move forward step 2232. A cleared container 2234 check may be performed. If the containers are not cleared, the execution flow diagram 2200 may return to and repeat the move first arm to start position step 2202. If the containers are cleared, the execution flow diagram 2200 may be terminated.

[0196] For the following parameters: startPose: This is the starting position of the arms (two arrays of 6 floats describing each robot's joint configuration). l graspPoses: This is a pair of IDs for grasp poses. l homePose: This is the home position for the arm. l dropPose: This is the 6D drop pose for the arm. l pickPose: This is the pick position for the arm depending on the grasp pose. l HighPose: This is the highest position for the conveyor. l wallCache: A vector of all detections close to the system. l containerCache: A vector of all detections that do not belong in the wall cache. Pseudocode corresponding to this detection cash flow diagram 2000 may be as follows:

number

[0197] 23 illustrates an embodiment of a computing device 2300 that implements the components and process steps of the systems described herein. The computing device 2300 and some or all of its operating logic may be contained within a physical component of a robot, within a traditional desktop computer, within a computer server, and / or within a cloud server that communicates with the robot and other computing devices. In one embodiment, aspects of the computing device 2300 on a networked or cloud-based computing platform may provide control for more than one robot at a time, allowing multiple robots to operate in a coordinated manner within a workspace such as a material handling facility.

[0198] Input devices 2304, such as robots, desktop computers, programmable logic controllers (PLCs), industrial personal computers (IPCs), etc., contain transducers that convert physical phenomena into machine internal signals, typically electrical, optical, or magnetic signals. The signals can be wireless in the form of electromagnetic radiation in the radio frequency (RF) range, but potentially also in the infrared or optical range. Examples of input devices 2304 are contact sensors that respond to touch or physical pressure from the proximity of an object against an object or surface, mice that respond to movement through space or across a plane, microphones that convert vibrations in a medium (typically air) into device signals, and scanners that convert optical patterns on a two-dimensional or three-dimensional object into device signals. Signals from the input devices 2304 are provided to memory 2306 via various machine signal conductors (e.g., buses or network interfaces) and circuits.

[0199] The memory 2306 is typically known as a first or second level memory device and provides storage of signals received from the input device 2304, instructions and information for controlling the operation of the central processing unit or CPU 2302, and signals (via configurations of matter or states of matter) from the storage device 2310. The memory 2306 and / or the storage device 2310 may store computer executable instructions and thus form logic 2314 that, when applied to and executed by the CPU 2302, implements embodiments of the processes disclosed herein. The logic 2314 may include portions of a computer program executed by the CPU 2302 or another processor, along with configuration data. The logic 2314 may include one or more machine learning models 2316 used to perform or supplement the performance of the disclosed operations.

[0200] Information stored in memory 2306 is typically directly accessible to the device's CPU 2302. Signals input to the device cause a reconfiguration of the internal material / energy state of memory 2306, essentially creating a new mechanical configuration and affecting the behavior of the computing device 2300 by configuring the CPU 2302 with control signals (instructions) and data provided in conjunction with the control signals.

[0201] A second or third level storage device 2310 may provide slower but larger mechanical memory capabilities. Examples of storage devices 2310 are hard disks, optical disks, large capacity flash memory or other non-volatile memory technologies, and magnetic memory.

[0202] In one embodiment, the memory 2306 may include a virtual storage device accessible through a connection to a cloud server using the network interface 2312, as described below. In such an embodiment, some or all of the logic 2314 may be stored and processed remotely.

[0203] The CPU 2302 may change the configuration of the memory 2306 by signals in the storage device 2310. In other words, the CPU 2302 may read data and instructions from the storage device 2310 in the memory 2306, which may affect the operation of the CPU 2302 as instructions and data signals, which may be provided to the output device 2308. The CPU 2302 may change the contents of the memory 2306 by signaling the machine interface of the memory 2306 to change the internal configuration, and then convert the signal to the storage device 2310 to change its material internal configuration. In other words, the data and instructions may be backed up from the memory 2306, which is often volatile, to the storage device 2310, which is often non-volatile.

[0204] The output device 2308 is a transducer that converts the signals received from the memory 2306 into the physical phenomenon of vibrations in the air, patterns of light on a mechanical display, or vibrations of ink or other material (i.e., tactile devices) or patterns (i.e., printers and 3D printers).

[0205] The network interface 2312 receives signals from the memory 2306 and converts them to electrical, optical, or wireless signals to other machines, typically over a machine network. The network interface 2312 also receives signals from the machine network and converts them to electrical, optical, or wireless signals to the memory 2306. The network interface 2312 may enable the robot to communicate with networked desktop or server computing systems, cloud servers, mobile devices, other robots, and other network-enabled devices.

[0206] While this disclosure has been described with reference to exemplary embodiments, this description is not intended to be construed in a limiting sense. Various modifications and combinations of the exemplary and other embodiments of the present disclosure will be apparent to those skilled in the art upon reference to this description. [Explanation of symbols]

[0207] 100 Automatic unloading and palletizing routines 102 Block 104 Block 106 Blocks 108 Subroutine Block 110 Subroutine Block 112 Subroutine Block 114 Subroutine Blocks 116 Subroutine Block 118 Subroutine Block 120 Subroutine Block 122 Subroutine Block 124 Subroutine Blocks 126 Subroutine Blocks 128 blocks 130 blocks 132 blocks 134 Blocks 136 blocks 138 Subroutine Block 140 Subroutine Block 142 blocks 200 Pallet Load Stabilization Routine Block 202 204 Block 206 Block 208 Subroutine Block 210 Subroutine Block 212 Subroutine Block 214 Subroutine Block 216 Blocks 218 Blocks 220 Block 222 Block 300 Robot loading and unloading system 302 Articulated Robot 304 Loading Conveyor 306 Ground Base 308 Telescopic Transport Conveyor 310 Scanning Tunnel 312 Sorting MHE 314 Inclined MHE 316 2DOF MHE 318 Automatic Palletizing System 320 Empty Pallet Sleeve Assembly 322 At least partially stacked pallet sleeve assembly 324 AMR 326 Palletizing Robot 328 Palletizing Robot Support Gantry 400 End-to-End System 402 MTBH 404 Shipping Container 406 Material Handling Facilities 408 Dock 410 Packaged palletized unit loads 412 Operations Team 500 Automatic Unloader 502 First loading conveyor end 504 Robot-mounted camera 506 End Effector 508 Robot Base 510 Second Loading Conveyor End 512 Bass Support 514 Ground Base Pivot 516 Wheels 518 Tracks 520 Loading Conveyor Pivot 522 Translation Actuator 524 Mounting Gantry 526 Forward Sensing Multi-Dimensional Sensor System 528 Multi-Channel Inertial Measurement System 530 Sensor notch 532 Motor 534 Industrial Control Box 600 degrees of freedom for robot loading / unloading system 602 DOF 604 DOF 606 DOF 608 6DOF 700 Primary reference system generation 702 2D and 3D Cameras 704 2D and 3D Cameras 706 Visual Perception Scan 708 Visual Perception Scan 710 Areas of overlap 712 Reference system 714 Reference system 716 Primary Reference System 800 MTBH Calculated Picking Data The first wall of 802 MTBH The second wall of 804 MTBH 806 Picking surface 808 Picking surface 810 Picking surface 812 Picking position 814 Picking position 816 Picking Position 818 Picking 820 Picking 822 Picking 900 Automatic Pallet Load Stabilization System 902 Pallet Dispenser 904 Palette 906 MHE 908 Pallet Sleeve 910 Pallet Sleeve Lifter 912 Support structure 914 Actuator System 916 Rotating pallet wrapping mechanism 918 Rotating Ring 920 Automatic Forklift 922 Empty Palette 924 Packaging material spool pin 1000 simultaneous wrapping and lifting operations 1002 Pallet sleeve lifting operation 1004 Rotational wrapping operation 1006 Packaging material spool pin lifting operation 1008 Rotary Packaging 1010 Packaging materials 1100 Pallet Sleeve Assembly 1102 Lifting configuration 1104 Friction surface 1106 Clearance Hole 1108 Sleeve Wall 1200 Automated Crane Storage and Retrieval System 1202 Pallet Crane 1204 Storage Rack 1206 High Bay Pallet Rack 1208 Pickup location 1210 Endstop 1212 Queue 1214 Additional Drop-off Locations 1216 Rail 1218 MHE 1220 Pallet End Effector 1300 Loading and unloading control system 1302 Perception Systems 1304 Industrial Control Systems 1306 Computer Systems 1308 Camera System 1310 Load Cell 1312 Pressure Sensor 1314 Other Sensors 1316 Pneumatic 1318 Hydraulic Pressure 1320 Encoder 1322 Motor 1400 Palletizing Control System 1402 Perception System 1404 Industrial Control Systems 1406 Computer Systems 1408 Camera System 1410 Load Cell 1412 Pressure Sensor 1414 Other Sensors 1416 Pneumatic 1418 Hydraulic Pressure 1420 Motor 1500 Load Stabilization Control System 1502 Perception System 1504 Industrial Control Systems 1506 Computer Systems 1508 Camera System 1510 Location Sensor 1512 Other Sensors 1514 Pneumatic 1516 Hydraulic Pressure 1518 Motor 1600 End-to-End Control System Diagram 1602 Control communication 1604 Mobile Transport Fleet Manager 1606 Wireless Fleet Control 1608 Stored loading and unloading data 1610 Container Inventory 1612 Picking Sequence 1614 Picking Rate 1616 Stored Paletted Data 1618 Input Box Sequence 1620 Sorting Logic 1622 Pallet Status 1624 Current MTBH per pallet 1626 Flow Control Feedback Loop 1628 Speed 1630 Speed ​​Data 1700 Automatic Truck Loading / Unloading Routine 1702 Block 1704 Block 1706 Block 1708 Block 1710 Decision Block 1712 Block 1714 Block 1716 Block 1718 Block 1720 Decision Block 1722 Block 1724 Decision Block 1726 Block 1728 Block 1730 Decision Block 1732 blocks 1800 Pick routine for individual joint robots 1802 Block 1804 Block 1806 Block 1808 Block 1810 Block 1812 Block 1814 Block 1816 Block 1818 Block 1820 Block 1822 Block 1900 Automatic truck loading and unloading flow chart 1902 Full wall scanning 1904a Conveyor downward movement 1904b Conveyor intermediate movement 1904c Conveyor upward movement 1906 Left and Right Image Processing 1908 Discovery Cache Generation 1910 Discovery Cache Update 1912 Motion Planning Generation 1914 Exercise Planning Cache 1916 Conveyor Movement 1918 Run the Pick 1920 Run the trajectory 1922 Update World State 1924 Cleared Wall 1926 Move to the next wall 1928 Cleared Containers 2000 Inspection Cash Flow Chart Waiting to receive data on 2002 MTBH detection 2004 MTBH detection 2006 Available Detections 2008 Order and Filter Discovery 2010 Wall section 2012 Wall Cache 2014 Wall Cache Empty 2016 Switching container cache to wall cache 2018 Container Cache 2020 Get new picking pose pair ID 2022 Picking Pose Pair Plan 2024 Picking Pose Pair ID 2026 Picking Pose Pair ID 2100 Planner Flow Diagram 2102 Picking pose requested pair from detection cache 2104 Wait for requested pause 2106 Available Poses 2108 Dual Arm Trajectory Planner 2110 Available Plans 2112 Sending trajectories to cache 2114 Request a picking pose ID from the detection cache 2116 Output the planned trajectory for the requested picking pose pair. 2200 Execution Flow Diagram 2202 Move the arm to the starting position 2204 Get a picking pose 2206 Request a picking pause from the discovery cache 2208 Available Picking Poses 2210 Move the conveyor 2212 Request a motion planning trajectory from the planner 2214 Available exercise plans 2216 Move the left arm to the home position and pick with the right arm 2218 Take a photo on the left 2220 Pick with left arm, drop with right arm 2222 Drop with left arm and move right arm to home 2224 Take a photo on the right 2226 Cleared Wall 2228 Move the arm to the home position 2230 Move the conveyor to a higher position 2232 Move forward 2234 Cleared Containers 2300 Computing Devices 2302 CPU 2304 Input Devices 2306 Memory 2308 Output Device 2310 Storage Device 2312 Network Interface 2314 Logic 2316 Machine Learning Models

Claims

1. positioning a robotic loading / unloading system within or at an entrance to a shipping container, said shipping container containing material to be handled (MTBH); The robot loading / unloading system includes: A telescopic transport conveyor; An automatic unloader attached to the telescopic transport conveyor, a loading conveyor having a first loading conveyor end and a second loading conveyor end; At least two articulated robots attached to an end of the first loading conveyor, each articulated robot comprising: At least six degrees of freedom; At least one of a 2D camera and a 3D camera; Robot joints; an end effector; at least two articulated robots; a ground-contact base including wheels or tracks; a base support including a load conveyor pivot and attached to the second load conveyor end; a translation actuator, the loading conveyor pivot configured to enable raising and lowering of the loading conveyor by extending or retracting the translation actuator; An automatic unloader; A control system; and logic, the logic comprising: coordinating the motion of the at least two articulated robots, the coordination including the position and movement of each robot and each robot component with respect to time; performing a perception scan of at least a portion of a visible wall of a material to be handled (MTBH), said perception scan capturing 3D and 2D images including 3D point cloud data of said MTBH; Constructing a 6-DOF pose for each unit of the MTBH analyzing the 2D image to determine which segments of the 2D image belong to each MTBH; extrapolating at least one of a size, an orientation, a fiducial of interest, and a center of mass from the 2D image for each unit of the MTBH; Querying the 3D image to obtain depth information, an orientation of each MTBH in 3D, and a dimension of each side of each MTBH as represented within a field of view of at least one of the 2D camera and the 3D camera of each articulated robot of the at least two articulated robots; determining, for each of the at least two articulated robots, a reference frame for the 2D camera and a reference frame for the 3D camera as a function of an inclination angle of the loading conveyor and a position configuration of the robot joints; determining a primary frame of reference for the automated unloader using at least one of the frame of reference for the 2D camera and the frame of reference for the 3D camera; constructing a virtual representation of each unit of the MTBH detected in the sensory scan, the virtual representation generating a six degree of freedom pose for each unit of the MTBH relative to the primary frame of reference of the automated unloader; Calculating a pick pose including a pick position and a pick orientation, the pick position being at the center of a face of each unit of the MTBH and the pick orientation being normal to one face of each unit of the MTBH; determining at least one motion plan using a search-based algorithm having an input, the input comprising: said picking pose for each unit of the MTBH; Minimizing torque on the robot motors; Minimizing the length of the movement path, and Collision avoidance with other dynamic and static elements of the surrounding environment and selecting a movement plan based on a scoring algorithm, the scoring algorithm being based, at least in part, on a picking pose of each unit of the MTBH relative to other units of the MTBH; At least in part, the selected exercise plan for each MTBH; the picking pose of each unit of MTBH relative to the other units of MTBH; and Coordinated movement of the at least two articulated robots determining a picking sequence based on the positioning a robotic loading / unloading system; and unloading the MTBH from the shipping container. method.

2. 10. The method of claim 1, further comprising, after removing at least one visible wall of a MTBH, positioning the robotic offloading system further within the shipping container to enable the at least two articulated robots to access a remaining MTBH.

3. 10. The method of claim 1, further comprising: operating the robotic offloading system to offload remaining MTBHs from the shipping container.

4. the robotic loading / unloading system further includes a ground contact base pivot rotationally coupling the ground contact base to the base support, the ground contact base pivot configured to allow the ground contact base wheels or tracks to remain in contact with a ground; The method further includes applying a downward force to the base support using the telescoping transport conveyor. The method of claim 1.

5. 2. The method of claim 1, wherein attributes of the scoring algorithm further include at least one of minimization of torque for the robot motors, minimization of motion path length, position, velocity, acceleration, jerk, snap, trajectory normalized distance, and attributes related to configuration space, free space, target space, and obstacle space.

6. receiving MTBHs unloaded from the shipping container into an automated palletization system; The automated palletization system comprises: at least one palletizing robot support gantry configured to receive at least one palletizing robot; said at least one palletizing robot mounted on said at least one palletizing robot support gantry; and at least one pallet sleeve assembly beneath the at least one palletizing robot, the at least one pallet sleeve assembly comprising: A palette and at least one pallet sleeve on the pallet, the at least one pallet sleeve including four structural walls; The at least one palletizing robot includes an articulated arm having a palletizing robot end effector, the at least one palletizing robot comprising: receiving, with said palletizing robot end effector, a unit of MTBH that has been picked up by performing said pick-up sequence; stacking each unit of the MTBH into at least one pallet sleeve assembly until an at least partially stacked pallet sleeve assembly is formed; It is configured as follows: receiving the unloaded MTBH; and operating the automated palletizing system to form an at least partially stacked pallet sleeve assembly. The method of claim 1.

7. receiving the at least partially stacked pallet sleeve assembly into an automated pallet load stabilization system; The automated pallet load stabilization system comprises:

1. A rotating pallet wrapping mechanism comprising: A support structure; a rotating ring having a diameter larger than the at least partially stacked pallet sleeve assemblies to provide clearance for an actuator system around the at least partially stacked pallet sleeve assemblies; an actuator system having at least two degrees of freedom configured to traverse vertically along the support structure; a packaging material spool pin attached to the actuator system and configured to hold a spool of packaging material and configured to dispense packaging material from the spool of packaging material onto MTBHs on the at least partially stacked pallet sleeve assemblies. A rotating pallet wrapping mechanism; a pallet sleeve lifter configured to lift the at least one pallet sleeve from the pallet, The logic is as follows: placing the at least partially stacked pallet sleeve assembly in a position suitable for interfacing with the actuator system and the pallet sleeve lifter; actuating the pallet sleeve lifter to interface with the pallet sleeves on the at least partially stacked pallet sleeve assemblies and incrementally lift the pallet sleeves upwardly, thereby incrementally exposing the MTBH for stretch wrapping by the rotating pallet wrapping mechanism; and actuating the actuator system with the packaging material spool pin to travel in a spiral motion around the incrementally exposed MTBH unit as the pallet sleeve lifter lifts the pallet sleeve, resulting in a wrapped palletized unit load and a fully lifted pallet sleeve. receiving the at least partially stacked pallet sleeve assemblies; and activating the automated pallet load stabilization system to produce the wrapped palletized unit load and the fully elevated pallet sleeve. The method according to claim 4.

8. receiving the at least partially stacked pallet sleeve assembly into an automated pallet load stabilization system; The automated pallet load stabilization system comprises:

1. A rotating pallet wrapping mechanism comprising: A support structure; a rotating base for receiving a pallet or said at least partially stacked pallet sleeve assemblies; an actuator system having at least two degrees of freedom configured to traverse vertically along the support structure; a packaging material spool pin attached to the actuator system and configured to hold a spool of packaging material and to dispense packaging material from the spool of packaging material. A rotating pallet wrapping mechanism; a pallet sleeve lifter configured to lift the at least one pallet sleeve from the pallet, The logic is as follows: positioning the at least partially stacked pallet sleeve assembly on the rotating base in a position suitable for interfacing with the actuator system and the pallet sleeve lifter; actuating the pallet sleeve lifter to interface with a pallet on the at least partially stacked pallet sleeve assembly and incrementally lift the pallet sleeve upwardly, thereby incrementally exposing the MTBH unit for stretch wrapping by the rotating pallet wrapping mechanism; and actuating the actuator system with the packaging material spool pin to dispense packaging material in a spiral motion around the incrementally exposed MTBH unit as the pallet sleeve lifter lifts the pallet sleeve, resulting in a wrapped palletized unit load and the fully lifted pallet sleeve. receiving the at least partially stacked pallet sleeve assembly into an automated pallet load stabilization system; and activating the automated pallet load stabilization system to produce the wrapped palletized unit load and the fully elevated pallet sleeve. The method according to claim 4.

9. Dispensing the packaging material around the at least partially stacked pallet sleeve assemblies comprises: the number of wraps around the exposed MTBH unit; the percent elongation relative to the length of the packaging material; and Offset between layers of packaging material 1. The method according to claim 1, The method according to claim 8.

10. 7. The method of claim 6, further comprising transporting the at least partially stacked pallet sleeve assemblies or the packaged palletized unit loads using an autonomous mobile robot (AMR).

11. operating an automated forklift configured to transport the empty pallet; placing the empty pallet under the fully elevated pallet sleeve using the automated forklift; and lowering the fully elevated pallet sleeve onto the empty pallet to form an empty pallet sleeve assembly. The method of claim 10.

12. A telescopic transport conveyor; An automatic unloader attached to the telescopic transport conveyor, a loading conveyor having a first loading conveyor end and a second loading conveyor end; At least two articulated robots attached to an end of the first loading conveyor, each articulated robot comprising: At least six degrees of freedom; At least one of a 2D camera or a 3D camera; Robot joints; an end effector; at least two articulated robots; A ground base, Wheels or tracks; a base support including a load conveyor pivot attached to the second load conveyor end; a translation actuator, the loading conveyor pivot configured to enable raising and lowering of the loading conveyor by extending or retracting the translation actuator; a ground base; An automatic unloader; A control system; and logic, the logic comprising: coordinating the motion of the at least two articulated robots, the coordination including the position and movement of each robot and each robot component with respect to time; performing a perception scan of at least a portion of a visible wall of a material to be handled (MTBH), said perception scan capturing 3D and 2D images including 3D point cloud data of said MTBH; Constructing a 6-DOF pose for each unit of the MTBH analyzing the 2D image to determine which segments of the 2D image belong to each MTBH; - extrapolating at least one of a size, an orientation, a base of interest, and a center of mass from said 2D image for each unit of the MTBH; Querying the 3D image to obtain depth information, an orientation of each MTBH in 3D, and a dimension of each side of each MTBH as represented within a field of view of at least one of the 2D camera and the 3D camera of each articulated robot of the at least two articulated robots; determining, for each of the at least two articulated robots, a reference frame for the 2D camera and a reference frame for the 3D camera as a function of an inclination angle of the loading conveyor and a position configuration of the robot joints; determining a primary frame of reference for the automated unloader using at least one of the frame of reference for the 2D camera and the frame of reference for the 3D camera; constructing a virtual representation of each unit of the MTBH detected in the sensory scan, the virtual representation generating a six degree of freedom pose for each unit of the MTBH relative to the primary frame of reference of the automated unloader; Calculating a pick pose including a pick position and a pick orientation, the pick position being at the center of a face of each unit of the MTBH and the pick orientation being normal to each unit of the MTBH; determining at least one motion plan using a search-based algorithm on an input, the input comprising: Picking poses for each unit of MTBH, Minimizing torque on the robot motors; Minimizing the length of the movement path, and Collision avoidance with other dynamic or static elements of the surrounding environment and selecting a movement plan based on a scoring algorithm, said scoring algorithm being based at least in part on a picking pose of each unit of the MTBH relative to other units of the MTBH; The following: the selected exercise plan for each MTBH; the picking pose of each unit of MTBH relative to the other units of MTBH; and coordinated movement of the at least two articulated robots; determining a picking sequence based at least in part on executing the picking sequence; Robot system.

13. 13. The robotic system of claim 12, further comprising a ground-contact base pivot rotationally coupling the ground-contact base to the base support, the ground-contact base pivot configured to allow the ground-contact base wheels or tracks to remain in contact with a ground surface, and the telescoping transport conveyor configured to apply a downward force to the base support.

14. 13. The robotic system of claim 12, wherein attributes of the scoring algorithm further include at least one of minimizing torque for the robot motors, minimizing motion path length, position, velocity and acceleration, jerk, trajectory normalized distance, and attributes related to configuration space, free space, target space, and obstacle space.

15. and an automated palletizing system, the automated palletizing system comprising: at least one palletizing robot support gantry configured to receive at least one palletizing robot; said at least one palletizing robot mounted on said at least one palletizing robot support gantry; and at least one pallet sleeve assembly beneath the at least one palletizing robot, the at least one pallet sleeve assembly comprising: A palette and at least one pallet sleeve on the pallet, the at least one pallet sleeve including four structural walls; The at least one palletizing robot includes an articulated arm having a palletizing robot end effector, the at least one palletizing robot comprising: receiving, at said palletizing robot end effector, a unit of MTBH picked by executing said picking sequence; stacking each unit of the MTBH into at least one pallet sleeve assembly until an at least partially stacked pallet sleeve assembly is formed; It is configured as follows: The robotic system of claim 12.

16. and an automated pallet load stabilization system, said automated pallet load stabilization system comprising:

1. A rotating pallet wrapping mechanism comprising: A support structure; a rotating ring having a diameter larger than the at least partially stacked pallet sleeve assemblies to provide clearance for an actuator system around the at least partially stacked pallet sleeve assemblies; the actuator system having at least two degrees of freedom configured to traverse vertically an interior of the rotating ring; a packaging material spool pin attached to the actuator system, the packaging material spool pin configured to hold a spool of packaging material and configured to dispense packaging material from the spool of packaging material. A rotating pallet wrapping mechanism; a pallet sleeve lifter configured to lift the at least one pallet sleeve from the pallet; The logic, positioning an at least partially stacked pallet sleeve assembly in a position suitable for interfacing with said rotating ring; actuating the pallet sleeve lifter to interface with the pallet sleeves on the at least partially stacked pallet sleeve assemblies and incrementally lift the pallet sleeves upwardly, thereby incrementally exposing the MTBH units for stretch wrapping by the rotating pallet wrap mechanism; actuating the actuator system with the packaging material spool pin to travel in a spiral motion around the incrementally exposed MTBH unit as the pallet sleeve lifter lifts the pallet sleeve, resulting in a wrapped palletized unit load and a fully lifted pallet sleeve. Logic, The robotic system of claim 15.

17. and an automated pallet load stabilization system, said automated pallet load stabilization system comprising:

1. A rotating pallet wrapping mechanism comprising: A support structure; a rotating base for receiving a pallet or said at least partially stacked pallet sleeve assemblies; an actuator system having at least two degrees of freedom configured to traverse vertically along the support structure inside the rotating ring; a packaging material spool pin attached to the actuator system and configured to hold a spool of packaging material and to dispense packaging material from the spool of packaging material. A rotating pallet wrapping mechanism; a pallet sleeve lifter configured to lift the at least one pallet sleeve from the pallet; The logic, positioning the at least partially stacked pallet sleeve assembly on the rotating base in a position suitable for interfacing with the actuator system and the pallet sleeve lifter; actuating the pallet sleeve lifter to interface with the pallet sleeves on the at least partially stacked pallet sleeve assemblies and incrementally lift the pallet sleeves upwardly, thereby incrementally exposing the MTBH units for stretch wrapping by the rotating pallet wrap mechanism; and actuating the actuation system with the packaging material spool pin to dispense packaging material in a spiral motion around the incrementally exposed MTBH unit as the pallet sleeve lifter lifts the pallet sleeve, resulting in a wrapped palletized unit load and a fully lifted pallet sleeve. the logic; The robotic system of claim 15.

18. The logic is as follows: the number of wraps around the exposed MTBH unit; the percent elongation relative to the length of the packaging material; and Offset between layers of packaging material and dispensing the packaging material according to at least one of the following: The robotic system of claim 16.

19. 17. The robotic system of claim 16, further comprising an autonomous mobile robot (AMR) configured to transport the at least partially stacked pallet sleeve assemblies or the packaged palletized unit load.

20. further comprising an automated forklift configured to transport the empty pallets; The logic is as follows: placing the empty pallet under the fully elevated pallet sleeve using the automated forklift; and lowering the fully elevated pallet sleeve onto the empty pallet to form an empty pallet sleeve assembly.

20. The robotic system of claim 19.

Images