Visual support type robotic depalletizer

Abstract

To provide a visual support type robotic depalletizing of products.SOLUTION: A visual support type robotic depalletizer includes: a pallet station for receiving pallet loads of cases arranged in pallet layers; a robot with a grip for grasping and picking at least one of the layers, and a grip engagement interface that defines a predetermined layer engagement position and orientation for the layer relative to the depalletizing end effector; a vision system for imaging the pallet loads of cases and generating at least one image of the upper part of the layer independent of the robot's movement; and a control device that receives the image and determines the layer position and orientation relative to the predetermined layer engagement position and orientation of the grip engagement interface.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application is a nonprovisional application of and claims the benefit of U.S. Provisional Patent Application No. 62 / 916,080, filed October 16, 2019, the entire disclosure of which is incorporated herein by reference.

[0002] [Technical field] The present disclosure relates generally to depalletizing, and more particularly to vision-assisted, robotized depalletizing of products. [Background technology]

[0003] The retail distribution of products (whether through traditional "brick and mortar" stores, online stores, or mixed retail channels) demands improved storage, sorting, and transportation efficiencies, particularly for the distribution of what are known as mixed or heterogeneous cases (within a given shipment), whether for store replenishment or individual orders. The application of intelligent / adaptive automation to these is increasingly driving efficiency improvements at many levels of distribution, including storage, sorting, and transportation.

[0004] Distribution centers and warehouses typically receive products such as cases, boxes, open trays, and stretch-wrap trays on structured pallets, e.g., neatly positioned with no gaps. Depalletizing systems for removing products from pallets are known in the art. Conventional pallet unloaders (e.g., depalletizers) are known that have electromagnetic radiation and optical mapping sensors (e.g., laser scanners, 3D cameras, etc.) to map 3D pallet loads to improve automated positioning for the pallet load. For example, one conventional method and system for detecting and reconstructing an environment to facilitate robot interaction with the environment includes determining a three-dimensional (3D) virtual environment, where the 3D virtual environment represents the physical environment of a robotic manipulator, including a plurality of 3D virtual objects corresponding to respective physical objects in the physical environment. The method then includes determining a two-dimensional (2D) image of the virtual environment, including a 2D depth map. The method may then include determining portions of the 2D image corresponding to one or more given physical objects. The method may then include determining a 3D model corresponding to a portion of the 2D image that corresponds to the given one or more physical objects based on the portion and the 2D depth map. The method may then include selecting a physics object from the given one or more physics objects based on the 3D model. The method may then include providing instructions to a robotic manipulator to move the object. Summary of the Invention

[0005] The foregoing aspects and other features of the present disclosure are explained in the following description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a schematic diagram of a flow facility according to an aspect of the present disclosure. [Figure 2] FIG. 1 is a schematic diagram of a pallet load according to an aspect of the present disclosure. [Figure 3] FIG. 2 is a schematic perspective view of a palletizer / depalletizer cell of the distribution facility of FIG. 1, the palletizer / depalletizer cell having a robot with a vision system and a layer depalletizing tool, according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a top perspective view of the layer depalletizing tool of FIG. 3. [Figure 5] FIG. 4 is a top plan view of the layer depalletizing tool of FIG. 3. [Figure 6] 4 is an exploded perspective view of the layer depalletizing tool of FIG. 3, the layer depalletizing tool being shown without most of its frame. [Figure 7] FIG. 4 is a top perspective view of a portion of the frame and curtain actuation assembly of the layer depalletizing tool of FIG. 3. [Figure 8A] FIG. 4 is a top perspective view of the layer depalletizing tool of FIG. 3 shown in position to grip a complete pallet layer. [Figure 8B] FIG. 4 is a bottom perspective view of the layer depalletizing tool of FIG. 3 shown in position to grip a complete pallet layer. [Figure 9A] 8B is a perspective view similar to FIG. 8A showing the clamps of the layer depalletizing tool of FIG. 3 applying pressure to the pallet layer and with the curtain partially closed. [Figure 9B] 8C is a perspective view similar to FIG. 8B showing the clamps of the layer depalletizing tool of FIG. 3 applying pressure to the pallet layer and with the curtain partially closed. [Figure 10A] 8B is a perspective view similar to FIG. 8A showing the clamps of the layer depalletizing tool of FIG. 3 with some of the pressure on the pallet layers released and the curtains fully closed. [Figure 10B] 8C is a perspective view similar to FIG. 8B showing the clamps of the layer depalletizing tool of FIG. 3 with some of the pressure on the pallet layers released and the curtains fully closed. [Figure 11A] FIG. 4 is a schematic perspective view of a camera field of view for a vision system of the palletizer cell of FIG. 3 according to an embodiment of the present disclosure. [Figure 11B]FIG. 4 is a schematic perspective view of a camera field of view for a vision system of the palletizer cell of FIG. 3 according to an embodiment of the present disclosure. [Figure 11C] FIG. 4 is a schematic perspective view of a camera field of view for a vision system of the palletizer cell of FIG. 3 according to an embodiment of the present disclosure. [Figure 11D] FIG. 4 is a schematic perspective view of a camera field of view for a vision system of the palletizer cell of FIG. 3 according to an embodiment of the present disclosure. [Figure 11E] FIG. 4 is a schematic perspective view of a camera field of view for a vision system of the palletizer cell of FIG. 3 according to an embodiment of the present disclosure. [Figure 11F] FIG. 4 is a schematic perspective view of a camera field of view for a vision system of the palletizer cell of FIG. 3 according to an embodiment of the present disclosure. [Figure 12] 4 is an exemplary image captured by a vision system of the palletizer cell of FIG. 3 according to an embodiment of the present disclosure. [Figure 13A] 4 is an exemplary calibration jig / fixture for the vision system of FIG. 3 according to an embodiment of the present disclosure. [Figure 13B] 13B is an exemplary image of the calibration jig / fixture of FIG. 13A captured by a vision system according to an embodiment of the present disclosure. [Figure 14] 1 is an exemplary flow diagram of a method according to an aspect of the present disclosure. [Figure 15A] 4 illustrates an example position of the layer depalletizing tool of FIG. 3 relative to a pallet layer being picked according to an embodiment of the present disclosure. [Figure 15B] 4 illustrates an example position of the layer depalletizing tool of FIG. 3 relative to a pallet layer being picked according to an embodiment of the present disclosure. [Figure 15C] 4 illustrates an example position of the layer depalletizing tool of FIG. 3 relative to a pallet layer being picked according to an embodiment of the present disclosure. [Figure 15D] 4 illustrates an example position of the layer depalletizing tool of FIG. 3 relative to a pallet layer being picked according to an embodiment of the present disclosure. [Figure 16A] 4 is an exemplary pallet layer configuration that can be picked by the layer depalletizing tool of FIG. 3 according to an embodiment of the present disclosure. [Figure 16B] 4 is an exemplary pallet layer configuration that can be picked by the layer depalletizing tool of FIG. 3 according to an embodiment of the present disclosure. [Figure 16C] 4 is an exemplary pallet layer configuration that can be picked by the layer depalletizing tool of FIG. 3 according to an embodiment of the present disclosure. [Figure 17] 1 is an exemplary flow diagram of a method according to an aspect of the present disclosure. [Figure 18] 1 is an exemplary flow diagram of a method according to an aspect of the present disclosure. [Figure 19A] FIG. 1 is a schematic diagram of slipsheet removal according to aspects of the present disclosure. [Figure 19B] FIG. 1 is a schematic diagram of slipsheet removal according to aspects of the present disclosure. [Figure 19C] FIG. 1 is a schematic diagram of slipsheet removal according to aspects of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0007] FIG. 1 is a schematic diagram of a warehouse system or distribution facility 100WS (herein referred to as warehouse system 100WS) according to embodiments of the present disclosure. While embodiments of the present disclosure are described with reference to the drawings, it should be understood that they may be embodied in many forms. Furthermore, any suitable size, shape, or type of elements or materials may be used. While distribution facility 100WS is described herein as an automated distribution facility, it should be understood that embodiments of the present disclosure are applicable to distribution facilities having any suitable conveying system, such as both automated and manual conveying systems, or even to entirely manual conveying systems.

[0008] 1 and 2, according to an embodiment of the present disclosure, a warehouse system 100WS includes at least one palletizer / depalletizer cell 10A, 10B (collectively referred to herein as palletizer cell 10). Palletizer cell 10 has one or more robotic case manipulators 14 (also referred to herein as articulated robots, adaptive real-time robots, robots, or product picking devices) that, with a vision system, place (individually or with fabricated pick faces) item units CU (also referred to herein as case units or cases or products 18) of a mixed pallet load into stacks SL1-Sn and / or layers PL1-PL4 that build the mixed case pallet load PAL. A suitable example of a palletizer / depalletizer is described in U.S. Patent No. 10,343,857, entitled "Vision-Assisted Robotized Depalletizer," issued July 9, 2019, the entire disclosure of which is incorporated herein by reference.

[0009] The palletizer cell 10 is provided with one or more three-dimensional (3D) time-of-flight (TOF) camera vision systems 310 (referred to herein as vision systems 310) that generate three-dimensional (3D) images of each pallet layer (also referred to herein as a pallet load layer) and its case units CU removed by the robot 14. The vision systems 310 are disposed in the palletizer cell 10 to image the pallet load PAL of cases CU at the pallet unloading / loading station 301, and are configured to generate at least one image (see FIGS. 2, 8A, and 11A-11F) of at least a top / surface 148 of at least one pallet load layer 816 (FIG. 8B—representing pallet layers PL1, PL2, PL3, PL4, PL5) independent of the movement of the robot 14. The three-dimensional image information is generated and provided by the vision system 310 in real time coincidence with the cyclical motion of the robot 14 picking pallet layers to depalletize goods from the pallet load PAL, and indicates in real time (within the pick / place motion cyclical frame of the robot 14) at least the layer pose for each pallet layer in the pallet build from the first case layer PL1 to the last case layer PL5 placed on the pallet support SPAL.

[0010] Three-dimensional image information of the layer pose of each layer identifies deviations from plane (e.g., tilt of the layer relative to the pallet support SPAL and / or other layers, open case unit CU, etc.), which, for example, informs the robot 14 to compensate for the deviations, thereby allowing the robot 14 to compensate in real time for its positioning relative to the layer being picked, facilitating substantially continuous and adaptive depalletizing (fully automated or in cooperation / collaboration with user assistance) with adaptive real-time robot 14 positioning, while simultaneously resolving pallet quality / control and depalletizing by the robot 14.

[0011] A controller (such as robot controller 16 and / or cell controller 10C) is operatively coupled to vision system 310 to receive at least one image from vision system 310. The controller is configured to provide a determination of a layer position and pose / orientation (in the robot's coordinate system or reference frame X, Y, Z, RX, RY, RZ, etc. (FIG. 3)) of at least one pallet layer 816 (FIG. 8B) relative to a predetermined layer-engaging position and orientation of a grip engagement interface 810 (FIG. 8B—as described herein) of a grip 800 (FIG. 8) of the robot 14 based on the at least one image, where the controller is operatively coupled to robot 14 to position grip 800 relative to each top pallet layer (e.g., based on a determined relationship between grip interface 810 and at least one top pallet layer of the pallet load layer) and grasp and hold at least one pallet layer 816 with grip 800 at grip engagement interface 810. The control device determines the respective layer position and orientation of each top layer based on at least one image, and provides a determination of the determined relationship by comparing the respective layer position and orientation with a predetermined reference frame (as described herein) of the robot 14.

[0012] In one aspect, the vision system 310 incorporated into the automated palletizer cell 10 informs and enables the cell controller 10C to provide real-time (or near real-time) command inputs (to automation such as the robot(s) 14) that respond in real time to pallet load variances (corresponding to commanded transaction times, as further described), thereby causing the robot(s) 14 to adapt in real time to resolve pallet load build variances and effect depalletizing in a time-optimal manner (automatically and / or in cooperation / collaboration with user assistance).

[0013] Referring again to FIG. 1 , according to aspects of the present disclosure, distribution facility 100WS includes storage and retrieval system 100 that may operate, for example, at a retail distribution center or warehouse to fulfill orders received from retailers for case units. In one example, a case unit may be a case or unit of goods that is not stored (e.g., not contained) on a tray, on a tote, or on a pallet. In other examples, a case unit may be a case or unit of goods that is contained in any suitable manner, such as on a tray, on a tote, or on a pallet. Note that a case unit may include cased units of goods (e.g., cases of soup cans, boxes of cereal, etc.) or individual items that are adapted to be removed from or placed on a pallet. According to embodiments, shipping cases for case units (e.g., cartons, barrels, boxes, crates, jugs, or any other suitable device for holding case units) may have variable sizes, may be used to hold case units during transport, and may be configured to be palletized for transport. It should be noted that, for example, when a stack or pallet of case units arrives at a storage and retrieval system, the contents of each pallet may be uniform (e.g., each pallet holds a predetermined number of the same items, i.e., one pallet holds soup and another pallet holds cereal), and when the pallets exit the storage and retrieval system, the pallets may contain any suitable number and combination of different case units (e.g., each pallet may hold different types of case units, i.e., a pallet holds a combination of soup and cereal). In embodiments, the storage and retrieval systems described herein may be applied to any environment in which case units are stored and retrieved.

[0014] The storage and retrieval system 100 may be configured, for example, for installation in an existing warehouse structure or adapted to a new warehouse structure. In embodiments of the present disclosure, the storage and retrieval system may include one or more infeed transfer stations 170 and one or more outfeed transfer stations 160, in / out case conveyors 150A, 150B, and 150C (collectively referred to as in / out case conveyors 150), a storage structure array 130, and a number of autonomous vehicle-transport robots 110 (referred to herein as “bots”). In embodiments of the present disclosure, the storage and retrieval system may also include a robot or bot transfer station, as described in U.S. Patent No. 9,096,375, issued August 4, 2015, the entire disclosure of which is incorporated herein by reference. In embodiments of the present disclosure, the bot transfer station may provide an interface between the bot 110 and the in / out case conveyor 150 such that case units can be indirectly transferred between the bot 110 and the in / out case conveyor 150 via the bot transfer station. In an embodiment of the present disclosure, case units can be transferred directly between the bot 110 and the in / out case conveyor 150.

[0015] The storage structure array 130 may include multiple levels of storage rack modules forming a storage array of storage locations 130SL for case units, each storage location 130SL arranged to store at least one case unit at each storage location 130SL. In one embodiment, each level of the storage structure array 130 includes a respective storage / picking aisle 130A and a transfer deck 130B for transporting case units between any of the storage areas of the storage structure array 130 and any shelf of any of the in / out case conveyors 150. The storage aisles 130A and transfer decks 130B are also configured to allow the bot 110 to traverse the storage aisles 130A and transfer decks 130B and retrieve ordered case units to place the case units in picking stock, where the case units are stored or otherwise held in the storage locations 130SL, within the storage aisles 130A and / or on the transfer decks 130B. Bot 110 may be any suitable bot capable of carrying and transporting case units throughout storage and retrieval system 100 . Suitable examples of bots can be found in U.S. Pat. No. 8,425,173, issued April 23, 2013; U.S. Pat. No. 9,561,905, issued February 7, 2017; U.S. Pat. No. 8,965,619, issued February 24, 2015; U.S. Pat. No. 8,696,010, issued April 15, 2014; U.S. Pat. No. 9,187,244, issued November 17, 2015; U.S. Pregrant Publication No. 2012 / 0189416 (U.S. Serial No. 13 / 326,952), entitled "Automated Bot with Transfer Arm," filed December 15, 2011; and U.S. Pat. No. 9,499,338, issued November 22, 2016, the entire disclosures of which are incorporated herein by reference for illustrative purposes only.The bot 110 may be configured to place case units, such as the retail items described above, in picking stock at one or more levels of the storage structure array 130, and then selectively retrieve ordered case units and ship the ordered case units, for example, to a store or other suitable location.

[0016] Infeed transfer station 170 and outfeed transfer station 160 may operate in conjunction with respective in / out case conveyors 150A, 150B to transfer case units bidirectionally to / from one or more levels of storage structure array 130, effecting the infeed and output of case units to and from storage structure array 130. Note that although infeed transfer station 170 and outfeed transfer station 160 (and their respective in / out case conveyors 150A, 150B and palletizer / depalletizer cells 10A, 10B) are described as being dedicated inbound (e.g., infeed) transfer station 170 and dedicated outbound (e.g., outfeed) transfer station 160, in aspects of the present disclosure, each of transfer stations 170, 160 may be used for both inbound and outbound transfer of case units from the storage and retrieval system. It should be noted that although an in / out case conveyor is described herein, the conveyor may be any suitable conveyor (including any suitable conveying path orientation, such as vertical and / or horizontal conveyor paths) or a transfer / picking device having any suitable conveying path orientation.

[0017] In one embodiment, as described above, the infeed transfer station 170 and the outfeed transfer station 160 each include a respective in / out case conveyor 150A, 150B and a respective palletizer / depalletizer cell 10A, 10B (collectively referred to herein as palletizer cells 10). In one embodiment, the palletizer / depalletizer cells 10 are automated cells, each of which is configured to receive load pallets (such as uniform or mixed case units or products) from an area within the pallet load 175, which may include, for example, an in-out load pallet conveyor 175C (illustrated in FIG. 1 as an input conveyor), and / or to build load pallets (such as uniform or mixed case units or products) for transport to an area outside the pallet load 180, which may include, for example, an in-out load pallet conveyor 180C (illustrated in FIG. 1 as an output conveyor). In one embodiment, conveyors 175C, 180C are each connected to storage structure array 130 and configured to transport load pallets bidirectionally in an input direction toward storage structure array 130 and in different output directions away from storage structure array 130. In one embodiment, conveyors 175C, 180C may each include a conveyor arrangement with distribution conveyor beds arranged to form a transport path, or in other embodiments, conveyors 175C, 180C may be individual transport units such as, for example, forklifts / pallet trucks. Suitable examples of automated palletizer / depalletizer cells 10A, 10B may be found in U.S. Patent Application No. 15 / 235,254, filed August 12, 2016, and U.S. Patent No. 8,965,559, issued February 24, 2015, the entire disclosures of which are incorporated herein by reference. Each palletizer cell includes one or more robot case manipulators 14, which may also be referred to as articulated robots or robots.The one or more robotic case manipulators 14 are configured to sequentially transport and place the item units CU of the pallet load onto the pallet supports, as described herein, to build (or in other embodiments, disassemble or release, as described herein) the pallet load 250 on the pallet unloading / loading station 301 (see FIG. 3).

[0018] When the palletizer cell 10 functions in an output role as a palletizer, palletized loads of item units CU, which may be of various sizes, arrive at the palletizer cell 10 via the in / out case conveyor 150B and are picked by one of the robotic case manipulators 14 and placed onto the palletized loads PAL, as described herein. When the palletizer cell 10 functions in an output role as a palletizer, full palletized loads PAL (see FIG. 2) created from the various case units are ready to be picked up by a forklift from the palletizer cell 10 for transport to the off-palletized load area 180. When the palletizer / depalletizer cell 10 functions in an input role as a depalletizer, full pallet loads of cases (similar to pallet loads PAL, which may be formed of homogenous or mixed cases) made up of various pallet load item units CU arranged in pallet load layers are transferred from the area within the pallet load 175 to the pallet unloading / loading station 301 of the palletizer cell 10 by any suitable method, such as a forklift or other transport. Each of the pallet load layers PL1, PL2, PL3, PL4, PL5 is formed of multiple cases CU side-by-side at a common level across the area of ​​the pallet load PAL. In one embodiment, as illustrated in FIG. 2, the pallet layer may be a mixed pallet layer including case CUs of different sizes, while in other embodiments, as illustrated in FIG. 3, the pallet layer may be a homogenous layer including case CUs having approximately the same size throughout the pallet layer. One or more robotic case manipulators 14 pick the pallet load item units CU from the pallet PAL for transfer into the storage structure array 130.

[0019] In one aspect, each infeed transfer station 170 forms a case input path Ip, in which palletizer / depalletizer cells 10A depalletize case units layer by layer or otherwise depalletize case units from standard pallets (e.g., homogenous pallets having stability suitable for automatic engagement of pallet layers by an automated layer interface unit, such as a product picking device or robot 14) into single case units. Palletizer / depalletizer cells 10A are in communication with a transport system of automated storage and retrieval system 100, such as in / out case conveyor 150A, to form an integrated input system (e.g., infeed transfer station 170) that supplies case units to automated storage and retrieval system 100. Each infeed transfer station 170 defines a case input path Ip that is integrated with the automated storage and retrieval system 100 and the warehouse management system 199, where the warehouse management system 199 includes any suitable controller 199C configured to manage, using any suitable non-transitory program code and memory, at least the input of case units into the storage structure array 130, the storage distribution of case units within and the retrieval of case units from the storage structure array 130, the inventory / replenishment of case units, and the output of case units.

[0020] In one aspect, each case unit input path Ip includes at least one corresponding case unit inspection cell 142 in communication with the warehouse management system 199. In one aspect, the at least one corresponding case unit inspection cell 142 may be any suitable inspection cell, including any suitable volumetric inspection, such as with a multi-dimensional light curtain, an imaging system, and / or any other suitable sensing / sensor arrangement configured to detect defects in the case units and identify the case units, for example, for inventory, transport sequencing, storage distribution, and sequencing of the case units for output from the storage structure array 130.

[0021] In one embodiment, as described above, the palletizer / depalletizer cell 10A can be fully automatic to disassemble or release layer(s) from pallets being unloaded by the palletizer / depalletizer cell 10A. Still referring to FIG. 2 , the term decommissioning refers to the removal of pallet layers PL1, PL2, PL3, PL4 (full or partial) from a pallet PAL such that the item units CU of each pallet load are removed from layers PL1, PL2, PL3, PL4 at a predetermined level 200 of the pallet PAL (which may correspond to a commissioning level or transfer surface), whereby, in some embodiments, the pallet PAL is indexed (by any suitable pallet lifting device of the palletizer cell 10) to the next level of the pallet PAL for removal of the next layer PL2, PL3 (full or partial) corresponding to the next level of the pallet PAL.

[0022] In one embodiment, the palletizer / depalletizer cell 10A is configured to release layers PL1, PL2, PL3, PL4 such that the release is synchronized or otherwise coordinated (e.g., matched) by the warehouse management system 199 with a predetermined percentage or feed rate of case units in the automated storage and retrieval system 100 established by the warehouse management system 199. For example, in one embodiment, the warehouse management system 199 is configured to set and / or monitor the predetermined percentage of the supply of case units in the automated storage and retrieval system 100. For example, the warehouse management system 199 monitors and manages the automated systems of the automated storage and retrieval system 100 (e.g., in / out case conveyors 150A, 150B, bot 110, and palletizer / depalletizer cells 10A, 10B, etc.), where each of the automated systems, or one or more of the automated systems, may, in fact, be controlled, alone or in combination, by the warehouse management system 199 or any other suitable controller of the automated storage and retrieval system 100 (e.g., bot controller, conveyor controller, palletizer / depalletizer controller, etc.). Under the control of a controller (e.g., a controller or controllers) there are given transaction times (e.g., time / periods to effect a basic unit of case transport or transfer, such as transferring a case unit on / off an in / out case conveyor to a picking / placement station, or lifting a case unit a predetermined distance, or picking / placing a bot transfer at a storage location, time to transfer a pallet layer to / from a pallet, etc.) that define a predetermined percentage of the supply of case units in the automated storage and retrieval system 100 established by the warehouse management system 199. For example, the controller 199C of the warehouse management system 199 is communicatively connected to the in / out case conveyors 150A, 150B such that the in / out case conveyors 150A, 150B bidirectionally transport case units to / from the storage structure array 130 at a predetermined case feed rate.The controller 199C may also be communicatively connected to the corresponding palletizer / depalletizer cells 10A, 10B of the in-out case conveyors 150A, 150B so that the loading and unloading of layers of the palletizer / depalletizer cells 10A, 10B, respectively, which are substantially continuous, matches a predetermined case feed rate. Although embodiments of the present disclosure are described herein with respect to a distribution facility 100WS having an automated storage and retrieval system 100 with an automated conveying system, embodiments of the present disclosure are also applicable to a distribution facility having any suitable conveying system, such as both an automated and a manual conveying system, or a fully manual conveying system, where both the automated and manual conveying transactions each have a respective transaction time, and where the loading and unloading of case units onto / from pallets may be matched to the transaction times in a manner generally similar to that described herein.

[0023] In one aspect, each outfeed transfer station 160 forms a case output path Op where a palletizer / depalletizer cell 10B palletizes case units layer by layer onto pallets PAL using, for example, an automated layer interface unit, such as one or more robotic case manipulators 14. In one aspect, the pallets PAL may be formed as standard pallets (e.g., homogenous case units) or as mixed pallets, such as those described in U.S. Patent Application No. 14 / 997,920, filed January 18, 2016, the entire disclosure of which is incorporated herein by reference. In one aspect, the warehouse management system 199 is configured to establish a pallet solution using mixed case units that provides a stable pallet load stack suitable for transport as a layer by the end effectors of one or more robotic case manipulators 14. As noted above, a suitable example of a palletizer / depalletizer cell 10B can be found in U.S. patent application Ser. No. 15 / 235,254, filed Aug. 12, 2016, the entire disclosure of which was previously incorporated herein by reference.

[0024] In one aspect, the palletizer / depalletizer cell 10B is in communication with the transport system of the automated storage and retrieval system 100, such as the in / out case conveyor 150B, to form an integrated output system (e.g., outfeed transfer station 160) that receives case units from the automated storage and retrieval system 100 for placement on pallets according to any suitable case-out order sequence. For example, as described above, the item units CU of a pallet load routed to one or more robotic case manipulators 14 are transferred to pallets PAL by the end effectors of the one or more robotic case manipulators 14, and the item units CU of the pallet load (output case units) are arranged in a predetermined sequence established by the warehouse management system 199, layer by layer (note that a layer may cover all or part of a pallet) to form a standard output pallet load.

[0025] Each outfeed transfer station 160 defines a case output path Op that is integrated with the automated storage and retrieval system 100 and the warehouse management system 199, where the warehouse management system 199 includes any suitable controller 199C configured to manage the operation of the distribution facility 100WS, including the output of case units from the storage structure array 130, using any suitable non-transitory program code and memory, as described herein. In one embodiment, each case unit output path Op includes at least one corresponding case unit inspection cell 142 (as described above) in communication with the warehouse management system 199. In one embodiment, as described above, the palletizer / depalletizer cell 10B can be fully automated to build or attach the layer(s) onto pallets for loading into the palletizer / depalletizer cell 10B. Still referring to FIG. 2, the term commissioning refers to the construction of pallet layers PL1, PL2, PL3, PL4 (in whole or in part) onto a pallet PAL, such that item units CU of each pallet load are inserted into layers PL1, PL2, PL3, PL4 at a predetermined level 200 of the pallet PAL (which may correspond to a release / commission level or a transfer surface) until pallet layers PL1, PL2, PL3, PL4, PL5 are formed, whereby, in some embodiments, the pallet PAL is indexed (by any suitable pallet lifting device of the palletizer cell 10) to the next level of the pallet PAL for the construction of the next layer PL1, PL2 (in whole or in part) corresponding to the next level of the pallet PAL.In one aspect, the palletizer / depalletizer cell 10B is configured to load layers PL1, PL2, PL3, PL4, PL4 in a manner generally similar to that described above with respect to the release of layers PL1, PL2, PL3, PL4, such that the loading is synchronized or otherwise coordinated (e.g., matched) by the warehouse management system 199 with a predetermined rate or feed rate of case units in the automated storage and retrieval system 100 established by the warehouse management system 199, where the warehouse management system 199 manages other related aspects of the output, such as the pick-up sequence of case units, the sequence of mixed case units output to the pallet load of mixed case units, and inventory adjustments.

[0026] According to an embodiment of the present disclosure, and with reference to FIG. 3 , a palletizer / depalletizer cell 10A configured to release layers PL1, PL2, PL3, PL4, and PL5 includes at least one robot 14 having a robot arm 12 coupled to a robot controller 316 (which is coupled to or forms part of the cell controller 10C). The robot arm 12 is in the form of a standard industrial articulated robot arm suitable for releasing layers as described herein. In one embodiment, the robot arm 12 includes six degrees of freedom of movement, while in other embodiments, it may have more or less than six degrees of freedom of movement. As used herein, the terms “robot” and “robot arm” are used interchangeably to refer to a programmable system including articulated and / or movable members capable of receiving, controlling, and moving tools. As illustrated in FIG. 3 , a layer depalletizing tool or end effector 99 is coupled to the robot arm 12 and configured to release layers PL1, PL2, PL3, PL4, and PL5 as described herein. In one embodiment, the layer depalletizing tool 99 may be substantially similar to the tool described in U.S. Patent Application No. 14 / 720,089, filed May 22, 2015, entitled "Tool and Method for Layer Depalletizing," the entire disclosure of which is incorporated herein by reference.

[0027] 3-5 and 8B, the layer depalletizing tool 99 has grippers 800 configured to grasp and pick at least one pallet load layer 816 (representing any one of pallet layers PL1, PL2, PL3, PL4, PL5) for transporting the at least one pallet load layer 816 from the pallet load PAL at the pallet unloading / loading station 301 to an output station 333 (which in one embodiment includes any suitable conveyor, such as conveyor 150). The grippers 800 have grip engagement interfaces 810 that define a predetermined layer engagement position and orientation (e.g., in the robot coordinate system or space X, Y, Z, RX, Ry, RZ—see FIG. 3 , referred to herein as the robot's reference frame) for the at least one pallet layer 816 relative to the layer depalletizing tool 99 to repeatedly effect capture and stable holding of the at least one pallet load layer 816 with the grippers 800. While the grip 800 configuration described herein is one suitable example of a pallet layer gripping mechanism that may be utilized in embodiments of the present disclosure, in other embodiments, the grip may include any suitable differential pressure grip, bladder grip, or other capture mechanism that defines an engagement / capture system that interfaces with the pallet layer for transfer of the pallet layer by the grip. The layer depalletizing tool 99 includes a frame 20, four side clamps 22-24 movably mounted to the frame 20 for gripping and releasing pallet layers PL1, PL2, PL3, PL4, and PL5, two curtains 26 mounted to the frame below the clamps 22-24 and inserted below the pallet layer 816 gripped by the clamps 22-24, and a top pad 28 mounted to the frame 20 above the clamps 22-24, where one or more of the four side clamps 22-24, the two curtains 26, and the top pad 28 form a grip engagement interface 810. The predetermined layer engagement position and orientation provides an engagement planar orientation (as defined at least in part by the upper pad 28) of the grip engagement interface 810.The layer position and orientation represent the planarity of an engagement surface 1210 (which may coincide with the top surface 148) of at least one (top) pallet layer 816 positioned to interface with a grip engagement interface 810 that spans substantially across the at least one pallet layer 816 (see plane 1200 in FIG. 12 ), and the layer position and orientation represent a planar misalignment in at least two orthogonal directions (e.g., of a robot coordinate system or frame of reference) between the engagement surface 1210 of the at least one pallet layer 816 and the planar orientation of the grip engagement interface 810. A controller (such as robot controller 16 and / or cell controller 10C) is configured to overcome at least one of the planar misalignment and center point misalignment, respectively, for optimal grip engagement with each top layer based on robot motion boundary conditions defined by at least one of the robot architecture and structure bounding the depalletizer 10, expressed in a predetermined frame of reference of the robot 14.

[0028] Frame 20 includes two pairs of parallel walls 30-32 assembled to define a generally rectangular perimeter. Each wall 30 and 32 includes a bottom rectangular portion 34 and 36, respectively, and an integral triangular portion 38 and 40, respectively. The two triangular portions 40 are slightly curved toward each other. Frame 20 further includes two transverse rectangular hollow tubes 42 and 44, respectively, extending parallel to wall 30 between walls 32 and parallel to wall 30. Near the tops of triangular portions 38-40 of walls 30-32, holes 45 (see FIG. 6A) are provided, in which hollow tubes 42 and 44 are secured and which allow the passage of connectors and cables (not shown) through walls 30-32 and thereafter through hollow tubes 42 and 44. Mounting brackets 46 (see FIG. 8A ) are secured to both hollow tubes 42 and 44 at their intersections, allowing the layer depalletizing tool 99 to be attached to the robot arm 12. The frame components 30-44 are assembled using fasteners and / or welding, and similarly, other portions of the layer depalletizing tool 99 are attached to the frame 20. The frame 20 is not limited to the above, and other members may be provided for attaching the layer depalletizing tool 99 to the robot arm 12 and for operatively receiving other components. It should be noted that the description of the frame 20 is for illustrative purposes only, and in other embodiments, the frame may have any suitable configuration and / or the layer depalletizing tool may be coupled to the robot arm in any suitable manner.

[0029] Referring also to FIG. 6 , each opposing pair of side clamps 22 and 24 is slidably mounted in a respective track 43 and 45, each of which is secured beneath hollow tubes 42 and 44 for movement along the tubes via mounting assemblies 48 and 50, respectively. Because mounting assemblies 48 and 50 are very similar, only one mounting of clamps 22 to frame 20 will be described in greater detail herein. Mounting assembly 48 includes a bracket 52 that is slidably mounted beneath hollow tube 42 via track 43 and attached to the distal ends of rods 60 of two actuators 56. A bar 54 fixedly connects clamps 22 to bracket 52. Bar 54 is attached to bracket 52 so as to extend perpendicularly therefrom and is attached to clamps 22. Clamps 22 are perpendicular to hollow tube 42 in a plane that includes bar 54.

[0030] Two pneumatic actuators 56 are provided between the hollow tubes 42 and the brackets 52 to cause their movement, and thus the movement of the clamps 22 along the hollow tubes 42. More specifically, the body 58 of each actuator 56 is fixedly attached to the hollow tubes 42 on its respective side, and the distal end of the actuator's rod 60 is fixedly attached to the brackets 52. The clamps 22 and 24 are moved in parallel pairs to grip the pallet layer from two opposite sides thereof and then from their other sides. Thus, four of the actuators 56 (fixed to the same hollow tube 42 or 44) are initially operated, and then the other four are simultaneously operated. According to another aspect of the present disclosure, all of the clamps 22 and 24 are actuated simultaneously.

[0031] Given the dimensions and pose of pallet layer 816 (representing any one of pallet layers PL1, PL2, PL3, PL4, PL5) determined by vision system 310 as described herein, each pair of clamps 22 and 24 is movable between an extended position and a retracted position, where clamp 22 or 24 applies pressure to pallet layer 816 from corresponding opposing sides. Clamps 22 and 24 have different widths, although clamps according to another embodiment of the present disclosure may have the same width. Clamps according to another embodiment are pivotally attached to the frame. According to still other embodiments, the clamps have other configurations other than those illustrated herein and are attached to the frame in any suitable manner so as to be differently movable relative to the frame.

[0032] 7, 9A, and 10B, both curtains 26 are defined by a series of metal rolls 62 slidably and rotatably mounted to the rectangular portion 34 of the frame 20. Together, both curtains 26 extend along the entire length of the portion 34, each spanning half of its length. More specifically, each roll 62 includes a hub portion 63 (see FIG. 10B) at its longitudinal end that is received in parallel tracks 64 mounted on the inner surface of the frame 20. Each of the four tracks 64 extends from its center along the lower edge 66 of the rectangular portion 34 to the end of an arcuate portion 68 of the track 64 that rises toward the triangular portion 38 near the longitudinal end of the wall 30. The arcuate portion 68 defines the curtain-receiving portion of the curtain 26 when it is unfolded. Note that in some drawings, portions of the rolls 62 are not shown for simplicity.

[0033] Each curtain 26 includes a rotating head 70 at its leading end, which is slidably mounted in and rotatably mounted to the track 64 along with the roll 62. The roll 62, along with the head 70 of each curtain 26, is moved along its respective track 64 by an actuator 72. The body 74 of the actuator 72 is fixed to the exterior of one rectangular portion 34 of the wall 30, and its rod 76 is fixedly attached to the head 70 via a mounting bracket 78 (FIG. 7B) attached to the exterior of the rectangular portion 34 for unified translation of the rod 76 and head 70 along a track 79. When the curtain 26 opens, the actuator 72 pushes the head 70 along the track 64, forcing the roll 62 in the same direction. When the curtain 26 closes, the actuator 72 pulls the head 70 along the track 64 in the same direction. The head 70 is in the form of a plurality of rotatable friction elements 80 extending along the leading edge of each curtain 26. The friction elements 80 are in the form of rubber O-rings mounted for infinite rotation on two parallel rolls 82. The two rolls 82 are rotatably mounted between them on the mounting bracket 78.

[0034] As can be better seen in FIGS. 5 and 7B, the two rolls 82 of each curtain 26 are driven by respective drive assemblies including one side chain 84 and a drive unit 90. The side chains 84 extend along the straight portions of the respective tracks 64 and are attached to respective drive sprockets 86 and respective driven sprockets 87. The driven sprockets 87 are fixed to the ends of each respective roll 82. The drive unit 90 is fixed to the wall portion 34 and operably connected to the two side chains 84 via respective ones of the drive sprockets 86. The side chains 84 are positioned relative to the two rolls 82 via the sprockets 87 so that the rolls 82 and associated chains (or belts) 84 rotate in unison. Thus, rotation of each drive unit 90 causes rotation of the friction elements 80. The heads 70 can simultaneously and independently rotate and slide along the tracks 64. End rolls 92 are provided between the two driven sprockets 88 of each curtain 26 to further facilitate displacement of head 70 along track 64 by connecting side chains 85 located at each end of head 70, thereby ensuring that head 70 remains perpendicular to track 64. Note that elements of the drive assembly, including side chains 84, have been omitted from some of the drawings for simplicity.

[0035] More specifically, with reference to Figures 4-6, upper pad 28 and its actuation mechanism will now be described. Upper pad 28 is in the form of a plate that is movable toward and away from pallet layer 816 gripped by clamps 22-24. Upper pad 28 is made movable by its attachment to frame 20 via pad actuation assembly 5100. Upper pad 28 (in the form of a plate) is made of a flexible, resilient material, such as rubber or plastic, which is reinforced on its non-contact surface with metal tubing. According to other embodiments, upper pad 28 is made of another material and / or is not reinforced. The pad actuation assembly 5100 includes four pad holder shafts 102-108 rotatably mounted to the frame 20, eight pad holder wheels 6110 mounted in parallel pairs on each of the shafts 102-108, four link elements 112, each fixed between and to both the upper pad 28 and the wheel 6110 of a respective shaft 102-108, four shaft pulleys 114-120, one fixed near one end of each shaft 102-108, an upper pad linear actuator 122 fixed to the frame 20, a pulley assembly 124 fixedly mounted to the frame 20, and four cables 126-132, each operably connecting a respective shaft pulley 114-120 to a movable end of the actuator 122. The four pad holder shafts 102-108 are rotatably mounted to the hollow tubes 42-44 via four support brackets 134. The four shafts 102-108 are positioned end-to-end relative to one another in a square configuration. Each of the link elements 112 is secured to a respective pair of wheels 6110 for partial wrapping thereon and is rigidly secured to the upper pad 28 via attachments 136. The pulley assembly 124 includes a support 138 fixed to the hollow tube 42 so as to be positioned at the movable end of the actuator 122, and three intermediate pulleys 140-144 rotatably mounted to the support 138 below.As can be better seen in FIG. 5, the position and orientation of the intermediate pulleys 140-144 and the actuators allows cables 126-132 to be used to connect the four shaft pulleys 114-120 to the rod 146 of the actuator.

[0036] In operation, the upper pad 28 is raised by causing the actuator 122 to retract its rod 146, which simultaneously pulls on the cables 126-132. This creates a pulling force on the link element 112 which simultaneously rotates the shafts 102-108 and raises the upper pad 28. The opposite effect is achieved by extending the actuator rod 146.

[0037] 1, 3, and 11A-11F, the palletizer cell 10A includes a vision system 310. The vision system 310 includes at least one camera 310C mounted to the palletizer cell 10 independently of the robot 14 to maximize the throughput of the robot 14 (i.e., the next layer of pallets is acquired and analyzed by the vision system 310 while the robot 14 places the previously picked layer of pallets onto the conveyor 150). The at least one camera 310C is (or includes) any suitable three-dimensional image sensor configured to generate one or more of a two-dimensional image, a two-dimensional depth map, and a three-dimensional point cloud. In one embodiment, the vision system 310 includes one camera 310C, in another embodiment, the vision system 310 includes three cameras 310C1, 310C2, and 310C3, and in yet another embodiment, the vision system 310 includes six cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6. In still other embodiments, the vision system 310 includes any suitable number of cameras. When multiple cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6 are provided, two or more of the cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6 may be positioned at multiple vertical levels 387L1 and 387L2 (FIGS. 3 and 11A-11F). For example, in one embodiment, cameras 301C1, 31C2, 301C3 are positioned on a common level 387L1, and cameras 301C4, 301C5, 301C6 are positioned on another common level 387L2 at a different height (e.g., relative to pallet load PAL) than cameras 301C1, 31C2, 301C3, such that multiple cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 are positioned at different heights / levels to image pallet load PAL. In other embodiments, multiple cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 may be positioned on a single level, or on any suitable number of levels.In other embodiments, cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 may also be mounted on a movable platform (independent of robot arm 14) to raise and / or lower cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 to image each pallet layer with substantially full 360° coverage of the pallet load build structure RPAL (as described herein).

[0038] In embodiments described herein, at least one camera 310C is positioned such that three corners of pallet load PAL are directly within the fields of view FOV1-FOV6 of at least one camera 310C. For example, in one embodiment, cameras 310C1-310C3 on level 387L1 are pointed at three corners, but there is no corresponding camera at the fourth corner; in another embodiment, cameras 310C1-310C3 on level 387L1 are pointed at three corners, and cameras 310C4-310C6 are also pointed at three corners, but there is no corresponding camera at the fourth corner; and in yet other embodiments, cameras 310C1-310C6 may be positioned such that all four corners of pallet load PAL are directly within the fields of view FOV1-FOV6. In one embodiment, the at least one camera 310C is positioned such that a respective field of view FOV1-FOV6 of the at least one camera 310C covers the pallet load PAL from the top upward-facing surface 148 (FIG. 12) of the pallet load PAL to the bottom of the pallet load. In one embodiment, the at least one camera 310C is positioned such that the top upward-facing surface 148 (FIG. 12) of each of the pallet layers falls within a respective field of view FOV1-FOV6 of the at least one camera 310C. As described above, the at least one camera 310C may be positioned on a single level, multiple levels, and / or a movable platform such that the top upward-facing surface 148 (FIG. 12) of each of the pallet layers falls within a respective field of view FOV1-FOV6 and / or such that the respective fields of view FOV1-FOV6 cover the pallet load PAL from the top upward-facing surface 148 (FIG. 12) of the pallet load PAL to the bottom of the pallet load.

[0039] It should be noted that while each camera 310C1 to 310C6 is positioned at a vertical angle of approximately 45° (or other suitable vertical angle greater than or less than approximately 45°) and a horizontal angle of approximately 45° (or other suitable horizontal angle greater than or less than approximately 45°) relative to the pallet load PAL so as to view within its field of view FOV1 to FOV6 not only the pallet top surface 148 but also all four outermost vertical surfaces (sides) of the pallet load PAL (or at least three or more vertices / corners formed by the intersection of the sides) (e.g., even in embodiments where only three corners are directly within the field of view FOV1 to FOV6), in other embodiments, each camera of at least one camera 310C may be positioned at any suitable vertical and / or horizontal angle relative to the pallet load PAL. The cameras may be positioned at different heights (e.g., different levels 387L1, 387L2) to image different portions of the pallet load at each level of camera (e.g., lower level 387L2 may image the bottom half of the pallet load PAL and upper level 387L1 may image the top half of the pallet load PAL), while in other embodiments a single level of camera may image the entire pallet load PAL from top to bottom. In one embodiment, at least one camera 310C (e.g., cameras 310C1-310C6, etc.) may have any suitable focal length for a given image intensity. In one embodiment, the respective fields of view FOV1-FOV6 of each of the at least one camera (e.g., cameras 310C1-310C6) (see FIGS. 11A-11F, which illustrate each field of view highlighted relative to the other fields of view) may be approximately 45°, while in other embodiments, the respective fields of view FOV of the at least one camera 310C may be greater than or less than approximately 45°, so long as three corners of the pallet load PAL fall directly within the camera's fields of view FOV1-FOV6. In some embodiments, the attributes of the respective fields of view of each of the at least one camera 310C are the same (e.g., each camera has a field of view of approximately 45°), while in other embodiments, the attributes of the respective fields of view may be different for one or more of the at least one camera 310C (e.g., one or more cameras may have a field of view different from approximately 45°).

[0040] At least one camera 310C is coupled to, informs, and enables the cell controller 10C to issue movement commands to the robot arm 12 (or collectively, the robot 14) so ​​as to, for example, guide the robot arm 12 using real-time (or near real-time) command inputs (based on commanded transaction times as described herein) that respond in real time to pallet load variances to accommodate real-time resolution of pallet load build variances that affect depalletizing. For example, the at least one camera 310C enables the cell controller 10C to issue commands to the robot arm 12 to move and guide the layer depalletizing tool 99 to pallet layers PL1, PL2, PL3, PL4, PL5. As described herein, the vision system 310 provides (or otherwise results in a determination of) the position (X, Y, Z in the robot coordinate system or frame of reference) and / or orientation (RX, RY, RZ in the robot coordinate system or frame of reference) of the top pallet layer (e.g., layer 816). In one embodiment, the vision system 310 provides (or otherwise provides a determination of) the length L and width W of the top layer (see FIG. 12).

[0041] Commands issued by the cell control device 10C (based on image data from at least one camera 310C) result in positional / spatial adjustments of the layer depalletizing tool 99 in multiple degrees of freedom (e.g., planar as well as rotational degrees of freedom) for the pallet layers PL1, PL2, PL3, PL4, PL5 to accommodate tilted pallet layers (e.g., pallet layers that are rotated relative to the pallet support SPAL and / or other pallet layers), offset pallet layers (e.g., pallet layers having edges that overhang the peripheral edges of the pallet support SPAL into the established virtual vertical pallet plane / boundary), open case units CU (e.g., case units such as cardboard boxes with one or more flaps of the box open), out-of-tolerance pallet loads / layers, incomplete layers (e.g., missing case units), etc. The vision system 310 is configured to provide data to the cell control device 10C to provide guidance and positioning of the layer depalletizing tool 99 to an optimal picking position and orientation (e.g., compared to other possible picking positions and orientations) for picking the top layer of the pallet based on the actual position and orientation of the top layer in the coordinate system / reference frame of the robot 14 (the terms coordinate system and reference frame are used interchangeably herein).

[0042] Commands issued by the cell control device 10C (based on image data from at least one camera 31C) also provide obstacle avoidance, for example, when an unexpected object is positioned within or otherwise enters the palletizer cell 10A, when an edge or side of the palletized load PAL is approximately flush with other parts of the layer depalletizing tool 99 or robot 14, when the palletized load PAL is adjacent to a robot exclusion zone (e.g., a predefined area within the palletizer cell 10A where the robot 14 is not allowed to enter), etc. The vision system 310 is configured to detect unexpected objects, sides / edges of the pallet load PAL, the distance between the pallet load PAL and the robot exclusion zone, etc., and send data signals to the cell controller 10C, which in turn commands the robot arm 12 to move the layer depalletizing tool 99 around the unexpected object, around the sides of the pallet load, between the pallet load PAL and the robot exclusion zone (or other obstacle adjacent to the pallet load PAL), and / or in any other suitable manner for picking pallet layers PL1, PL2, PL3, PL4, PL5, or in other embodiments, to command the robot arm 12 to stop moving.

[0043] As described above, cell controller 10C (or other suitable controller, such as robot controller 316) receives image data (e.g., from two-dimensional images, two-dimensional depth maps, and / or three-dimensional point clouds) from vision system 310 for analysis of the image data to detect pallets, pallet layers, and / or analyze their characteristics. Also as described above, the image data provided to cell controller 10C is provided in the coordinate system or frame of reference of robot 14, where vision system 310 is calibrated / registered to the coordinate system or frame of reference of robot 14. Note that each camera 310C1-310C6 is inherently calibrated to its own coordinate system (i.e., each camera perceives the depth of an object from its respective image sensor). When multiple cameras 310C1-310C6 are utilized, in one embodiment, calibration of the vision system 310 includes calibrating the cameras 310C1-310C6 to a common base frame of reference and calibrating the common base frame of reference to the robot's frame of reference, while in other embodiments, when one or more cameras are utilized, the frame of reference of one camera or the frame of reference of one or more cameras 310C1-310C6 may be individually calibrated to the robot's frame of reference.

[0044] 3, 13A, and 13B, calibrating cameras 310C1-310C6 to a common base frame of reference involves identifying and applying a transformation between the respective frames of reference of each camera 310C1-310C6 such that the respective frames of reference of each camera 310C1-310C6 are transformed to (or referenced to) a common base frame of reference, which may be the frame of reference of a single camera or any other suitable common base frame of reference to which each camera 310C1-310C6 may be associated to collectively form a common base frame of reference for all cameras of the vision system. For example, the frame of reference for camera 130C1 (although any one of the cameras may be used) represents the common base frame of reference. A transformation (i.e., a robust transformation in six degrees of freedom) is determined for each of the coordinate systems / frames of reference for the other cameras 310C2, 310C3, 310C4, 310C5, 310C6 relative to the frame of reference of camera 310C1 so that image data from cameras 310C2, 310C3, 310C4, 310C5, 310C6 is correlated or transformed to the frame of reference of camera 130C1. This calibration of cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 may be performed using a calibration jig / fixture 1300 (also referred to as a common camera calibration reference structure) located at pallet unloading / loading station 301. As described herein, each of the at least one camera 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 is calibrated to a common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents the positional relationship of the respective camera reference frame of each camera 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 to each other of the at least one camera 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 and to a predetermined reference frame of the robot 14 (as described herein).

[0045] The calibration fixture 1300 includes uniquely identifiable three-dimensional geometric shapes 1310-1319 (in this example, squares, some rotated relative to others) that provide an asymmetric pattern to the calibration fixture 1300 and constrain the determination / transformation of the camera's frame of reference (e.g., from each camera) to a common base frame of reference, and the transformation between the common base frame of reference and the robot's frame of reference, as further described, to determine the relative pose of the pallet layer to the grip interface. The calibration fixture 1300 shown and described herein is exemplary, and any other suitable calibration fixture may be utilized in a manner similar to that described herein. For illustrative purposes, each of the three-dimensional geometric shapes 1310-1319 is a predetermined size that constrains the identification of corners or points C1-C36 of the three-dimensional geometric shapes 1310-1319, and the transformation is such that the distance between corresponding corners C1-C36 is minimized (e.g., the distance between each of the corners C1-C36 in the reference frame of camera 310C1 is minimized for each of the respective corners C1-C36 identified in the reference frames of each of cameras 310C2-310C6).

[0046] Each of the three-dimensional geometric shapes 1310-1319 is imaged simultaneously (i.e., each of the three-dimensional geometric shapes 1310-1319 is at a single location in a common frame of reference while imaged by all cameras whose reference frames are calibrated to a common base frame of reference), and points / corners C1-C36 of the three-dimensional geometric shapes 1310-1319 identified in the image (one exemplary image is illustrated in FIG. 13B) are uniquely identified by each of the cameras 310C1-310C6 at a single location and uniquely determined independently of the orientation of the calibration fixture, such that the points / corners C1-C36 of the three-dimensional geometric shapes 1310-1319 identified in the image (one exemplary image is illustrated in FIG. 13B) are identified by the vision system 310 (in any suitable manner, such as those described herein with respect to determining pallet layer corners PC1-PC4). Note that the uniquely identified corners C1-C36 are located at a single location in the robot reference frame that is common to all cameras at the time the image set (which is a collection of images taken by each of the cameras for a given location on the calibration fixture or pallet load) corresponding to each set of images taken by cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 was taken. The angles C1-C36 identified in each image of the image set are compared between images to determine a transformation for each camera reference frame to a common base reference frame (which, in one example, may correspond to or otherwise be defined by the reference frame of camera 310C1). Note that the calibration fixture 1300 is configured to identify points throughout the working volume (corresponding to the dimensions of the pallet load PAL) of the pallet unloading / loading station 301. For example, the three-dimensional geometries 1310-1319 may span the XY plane of the pallet unloading / loading station 301, and the calibration fixture may be imaged at various heights throughout the working volume of the pallet unloading / loading station 301.

[0047] Upon registration of all cameras 310C1-310C6 to a common base frame of reference, the common base frame of reference (or the frame of reference of one or more cameras individually) is transformed (e.g., registered) to the robot's frame of reference (X, Y, Z, RX, RY, RZ) by attaching a calibration fixture 1300 (or a similar fixture) to the robot 14. The calibration fixture may be attached to the robot 14 such that the three-dimensional geometric shapes 1310-1319 (and their corners C1-C36) have a known, predetermined spatial relationship (e.g., position, planarity, orientation, etc.) with respect to the grip engagement interface of grip 800 (FIG. 8B) (or other suitable reference datum or location on the robot 14). The robot 14 may be commanded to move in a predetermined motion (whereby the calibration fixture 1300 is held) along one or more of the X, Y, and Z axes while being imaged by one or more of the cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6. The cell controller 10C may compare the identified angles C1-C36 in images obtained by the vision system 310 with, for example, encoder data (which describe the motion path of the robot 14, i.e., the motion of the fixture effected by the robot 14) and generate from the images (based on a common base reference frame) a planar orientation and position (pose and orientation, in six degrees of freedom) of the identified angles C1-C36 relative to the grip engagement interface of the grip 800 ( FIG. 8B ) (or other suitable reference datum or position of the robot 14). The generated planar orientations and positions (in six degrees of freedom, pose and orientation) of the identified angles C1-C36 relative to the grip engagement interface of grip 800 (FIG. 8B) (or other suitable reference datum or position of robot 14) characterize the relationship between the image field of the common base reference frame and the reference frame of robot 14 such that the positions of angles C1-C36 in the vision system's image are calibrated to the robot's reference frame.The above calibration may be performed once upon installation of the palletizing cell 10 or at any suitable time interval (or upon each initialization of the robot 14, such as after a power outage), allowing the image data of the vision system 310 to be represented in the same coordinate system / reference frame as the robot 14.

[0048] In one embodiment, at least one camera 310C resolves three-dimensional definitions of case unit features (e.g., case unit edges and corners) from two or more orthogonal planes such that maximum certainty of feature pose (e.g., X, Y, Z, RX, RY, RZ position of a pallet layer or features of the calibration fixture 1300—see FIGS. 11A-11F and 12 ) is obtained from a single image of the item in each of the fields of view FOV1-FOV6 of the at least one camera 310C, where resolution of the three-dimensional definitions of the pallet layer(s) and / or features of the calibration fixture 1300 is independent of the placement of the camera 310C and is performed in real time (e.g., within the pick / place cycle of at least one robot 14).

[0049] Although six cameras 310C1-310C6 are described, it should be understood that more or less than six cameras 310C1-310C6 may be used and positioned such that the fields of view of the cameras 310C1-310C6 of the vision system 310 cover the entire pallet unloading / loading station 301 of the palletizing cell 10, the pallet support SPAL mounted on the pallet unloading / loading station 301, and the expected pallet load building structure RPAL at the pallet unloading / loading station 301, to capture three-dimensional time-of-flight images of the object(s) desirably anywhere on the pallet load building structure RPAL, at any suitable desired resolution. The combined fields of view FOV1-FOV6 result in a nearly complete 360° coverage of the pallet load building structure RPAL with overlap of the fields of view FOV1-FOV6. For example, the combined fields of view FOV1-FOV6 may cover a standard pallet support SPAL (e.g., having dimensions of 48 inches by 48 inches, 48 ​​inches by 40 inches, and / or 36 inches by 36 inches), although it should be understood that the camera(s) 310C1-310C6 and associated fields of view FOV1-FOV6 may cover (e.g., image) a larger field of view as appropriate (e.g., including a truck bed or any field size). Additionally, the fields of view FOV1-FOV6 may cover any suitable palletized build structure RPAL height PH (see FIG. 2), such as, for example, heights of 60 inches, 70 inches, and 80 inches, although in other embodiments, the fields of view FOV1-FOV6 may cover heights less than 60 inches or greater than 80 inches.

[0050] In one embodiment, each of the camera(s) 310C1-310C6 may have a resolution of 176 pixels by 132 pixels, while in other embodiments, each, or one or more, of the camera(s) 310C1-310C4 may have a resolution of approximately 0.5 inches at the outermost boundary of the three-dimensional space of the pallet build (such that the definition of the depth map across the captured images of the entire pallet support / pallet build or a predetermined portion thereof is approximately 0.5 inches or greater).The camera(s) 310C1-310C6 may have a higher resolution (e.g., 320 pixels by 240 pixels or greater) as needed to provide a desired minimum depth map defining 5 inches. In this manner, sufficient resolution is provided by the vision system 310 to resolve features of at least the top surface 148 of the pallet load PAL so that planarity across the top surface 148 of the pallet load PAL can be determined and fully established for releasing pallet layers PL1-PL5 from the pallet load PAL. Sufficient resolution is also provided to resolve case unit features (e.g., case edges, etc.) so that planarity across the top of each layer PL1-PL5 (see FIG. 2) can be determined and fully established for releasing layers PL1-PL5. The resolution of the camera(s) 310C1-310C6 may be such that minimal processing is required to resolve case unit features (e.g., case unit edges and pallet layer corners) so that the case unit features are resolved substantially in real time from images received by the cell control device 10C. For example, the corners PC1 to PC4 of each palette layer PL1 to PL5 can be calculated in any suitable manner, such as by the Ramer-Douglas-Peucker algorithm (see, for example, "An iterative procedure for the polygonal approximation of plane curves," published in Computer Graphics and Image Processing, Volume 1, Issue 3, November 1972, Pages 244-256; "The Contours, Corners and T-Junctions Detection Algorithm," by Buades et al., published in Image Processing Online, February 27, 2018, ISSN 2105-1232, © 2018), or by the method described in Suraya Abu Bakar, Muhammad Suzuri Hitam and Wan Nural Jawahir Hj. Wan Yussof, 2017, the entire disclosure of which is incorporated herein by reference.The corner positions may be determined by imaging at least the top surface 148 of each pallet layer with the vision system 310 and determining the positions of the corners PC1-PC4 of the pallet layers PL1-PL5 in any other suitable manner, such as those described in "Improved Global and Local Curvature Properties for Shape Corner Detection," Journal of Applied Sciences, 17: 458-466; "The Comparison and Application of Corner Detection Algorithms," Jie Chen et al., Journal of Multimedia, Volume 4, No. 6, December 2009; and "Robust Corner Detection by Image-Based Direct Curvature Field Estimation for Mobile Robot Navigation," Sungho Kim, International Journal of Advanced Robotic Systems, 2012, Volume 9, 187:2012, DOI: 10.5772 / 53872.

[0051] 3, 2, 8B, and 12, in one embodiment, the cell controller 10C is configured to determine, in real time, from corresponding real-time three-dimensional imaging data, layer planarity variances PSV1, PSV2 of pallet layers PL1-PL5 being released against grip engagement interfaces 810 (FIG. 8B) of grips 800 (FIG. 8). The cell controller 10C is also configured to generate, in real time, articulated robot motion signals dependent at least on the real-time determined layer planarity variances PSV1, PSV2, where the articulated robot motion signals are generated in real time for real-time execution by the at least one articulated robot 14 between placement of one released pallet layer PL1-PL5 by the at least one articulated robot 14 and placement of sequentially successive released pallet layers PL1-PL5 that enable substantially continuous release of pallet loads PAL. In one embodiment, the at least one articulated robot motion signal generated by the cell controller 10C is a stop motion signal for the at least one articulated robot 14 along a pick / place path 399, 1580, a slow motion signal for the at least one articulated robot 14 along the pick / place path 399, 1580, or a movement of the at least one articulated robot 14 to a safe position along a safety stop path 398, where the safety stop path 398 is different from the pick / place path 399, 1580. In one embodiment, the articulated robot motion signal generated by the cell controller 10C is a pick position signal that sets a pick position for the layer depalletizing tool 99 based on the layer planarity dispersion PSV1, PSV2 of the pallet layers PL1-PL5 being released.

[0052] The cell control device 10C is configured to determine in real time the layer planarity dispersions PSV1, PSV2 and positions of the pallet layers PL1-PL5, as well as the released layer poses PSV3 (RZ) and positions (X,Y) from the corresponding real-time three-dimensional imaging data, where, for example, the vision system 310 images the top surfaces 148 of the layers PL1-PL5 to obtain a three-dimensional image of the top layer of the pallet load PAL with sufficient detail to identify the sides, corners, and planarity of the top surface 148 of the top pallet layer, as described above. Here, the pallet support dispersions PSV1, PSV2 may be one or more of unevenly spaced case units CU (e.g., spaces between case units CU in a pallet layer forming peaks / valleys in the sheet surface of the case units—FIG. 9A), missing case units CU in the pallet layer, height differences (e.g., protrusions and / or depressions—FIG. 9A), or any other defects in the pallet layer that may affect gripping of the pallet layer by the layer depalletizing tool 99. In one aspect, the cell controller 10C is configured to reject the picking of a pallet layer (and send a stop robot signal until it is replaced) if the dispersion of the pallet supports PSV1, PSV2 exceeds a threshold from a predetermined criterion, such as the plane defined by the upper pad 28. For example, if the missing case units CU in a pallet layer are larger than a predetermined area or if the spacing between case units CU in a pallet layer is greater than a predetermined distance, the picking of the pallet layer is rejected and the pallet layer is not picked until the defect in the pallet layer is resolved (e.g., by manual intervention). If the pallet layer is within the predetermined threshold, the cell controller 10C is configured to resolve the planar dispersion of the pallet layer (e.g., the location of the pallet layer in the three-dimensional robot space X, Y, Z, RX, RY, RZ) and confirm or correct (compensate) the planned robot pick / place path based on said dispersion relative to the adaptive pose of the layer depalletizing tool 99 with a resulting higher picking probability for the pallet layer.The controller may also identify a slowdown in the movement speed of the robot 14 or may modify the placement paths 399, 1580 and respective path trajectories (FIG. 3) of the robot 14 to generate a desired layer depalletizing tool 99 picking pose (e.g., position in three-dimensional space X, Y, Z, RX, RY, RZ).

[0053] In one aspect, the cell control device 10C is configured to establish a pallet layer datum DTM (FIG. 12) for the pallet layer, imaged by at least one three-dimensional camera 310C from the dispersion of the pallet supports PSV1, PSV2, where the pallet layer datum DTM resolves the local surface dispersion at the placement location of each different item unit within the pallet layer and defines a real-time position-based reference for picking of the articulated robot 14 for the pallet layer. In one aspect, the pallet layer datum DTM defines the planarity of the top surface 148 of the pallet layer.

[0054] 3, 12, and 14, exemplary operations of the palletizer cell 10 in the depalletizing configuration are described. With reference to FIG. 14, it should be noted that the operational blocks described therein do not necessarily dictate a particular order of operations, and the operations identified by the operational blocks may be performed in any suitable order. For example, blocks 1401 and 1410 may be performed any time before determining the picking path / trajectory of the robot 14.

[0055] In operation of the palletizer cell 10, the palletized loads PAL are transported and positioned on the unloading / loading station 301 in any suitable manner (such as described above). In exemplary operation of the palletizer cell 10, the fields of the image sensors or cameras 310C are optionally registered with the robot's frame of reference (X, Y, X, RX, RY, RZ) as described herein at any suitable time before and / or during operation of the palletizer cell 10 (FIG. 14, block 1401). The palletized loads PAL are imaged by the at least one camera 310C such that an image of the top surface 148 of the upper pallet layer 816 is captured (FIG. 14, block 1405). Image data from each of the at least one camera 310C is transformed from the respective camera frame of reference to the robot's frame of reference (FIG. 14, block 1410), such as in a manner described herein. It should be noted that in one embodiment, registration of the field of view of the camera 310C is optionally performed as part of the operation prior to transformation of the camera image data into the frame of reference of the robot 14. Transformation of the camera image data into the frame of reference of the robot 14 may also be performed at any suitable time prior to determination of the picking trajectory of the robot 14, such as the exemplary times described in FIG. 14 . The pallet layer's reference datum(s) are determined based on the image data in the frame of reference of the camera 310C and / or the frame of reference of the robot 14 ( FIG. 14 , block 1415). The pallet's reference datum(s) are any suitable geometric feature(s) of the pallet that identifies or otherwise defines the pose and position of the pallet in the frame of reference of the camera 310C and / or the frame of reference of the robot 14 (e.g., corners of the pallet layer, corners of case units in the pallet layer, outermost side of the pallet layer, vertices of the outermost side, orthogonality of the outermost side, position of the side, etc.).For example, in one embodiment, the reference datum(s) are corners PC1-PC4 of the pallet layer 816, where the corners PC1-PC4 are determined by the cell controller 10C in one or more of the reference frames of the cameras 310C and the robot 14 in any suitable manner (such as the methods described above using any suitable image analysis corner-finding algorithm) based on image data obtained by the at least one camera 310C. In one embodiment, the pallet layer corners PC1-PC4 are determined separately from the image data of each of the at least one camera 310C, although in other embodiments, the image data from the cameras may optionally be combined to determine the corners of the pallet layer ( FIG. 14 , block 1420). For example, when image data from the at least one camera 310C is combined, a single point cloud 1270 of at least a portion of the pallet load PAL including the pallet layer 816 is generated using the cell controller 10C by combining the image data from each of the at least one camera 310C.

[0056] The cell control device 10C is configured to fit a plane 1200 to the pallet layer's reference datum(s) based on image data from one or more of the at least one camera 310C in any suitable manner, such as using a random sample consensus (RANSAC) algorithm, a structured segmentation algorithm (or other suitable algorithm for structured point cloud data segmentation), or any other suitable algorithm ( FIG. 14 , block 1425). In one embodiment, the plane 1200 corresponds to and defines the top surface 148 of the pallet layer 816 in the frame of reference of the camera 310C and / or the robot 14. In one embodiment, the location of the corners PC1-PC4 of the pallet layer 816 in the robot's frame of reference can optionally be verified using the cell control device 10C by projecting a single point cloud 1270 onto the plane 1200 ( FIG. 14 , block 1430). Using the cell control device 10C, the pose PSV3 and size (length L and width W) of the pallet layer 816 are determined from image data from one or more of the at least one camera 310C in any suitable manner, such as by any suitable blob analysis technique (centroid, major axis, minimum ferret, etc.) (Figure 14, block 1435).

[0057] With the reference datum(s) (in this example, corners PC1-PC4, and therefore length L and width W) precisely known based on the above operations, and plane 1200 established to represent top surface 148 of pallet layer 816, cell control device 10C verifies the planarity of top surface 148 of pallet layer 816 relative to plane 28P (e.g., defined by upper pad 28) of grip engagement interface 810 ( FIG. 14 , block 1440). Verification of the planarity of top surface 148 may be performed by cell control device 10C using any suitable image analysis thresholding technique to determine whether layer depalletizing tool 99 can pick pallet layer 816. For example, deviation of the planarity of plane 1200 relative to plane 28P beyond a predetermined threshold (e.g., in one embodiment, the deviation may be up to about 5° or up to about 10° about the X-axis and / or Y-axis, while in other embodiments, the deviation may be greater than about 10°) prevents picking of pallet layer 816 by layer depalletizing tool 99. If pallet layer 816 cannot be picked by layer depalletizing tool 99, cell control device 10C is configured to issue any suitable audible and / or visual alert to an operator to resolve the picking of the pallet layer.

[0058] 15A-15C, the cell controller 10C is configured to determine, in any suitable manner, a movement trajectory and path for the robot 14 that provides or otherwise determines an optimal picking position for the layer depalletizing tool 99 relative to the pallet layer 816 (FIG. 14, block 1445) for picking / releasing the pallet layer 816 from the pallet load PAL. The optimal picking position may be determined by the cell controller 10C during a placement cycle of a previously picked pallet layer to result in substantially continuous release of the pallet load PAL. The optimal picking position is a position of the layer depalletizing tool 99 relative to the pallet layer 816, where the distance between the center 99C of the layer depalletizing tool 99 and the center 816C of the pallet layer 816 is minimized while satisfying the physical constraints of the layer depalletizing tool 99 and the palletizing cell 10. For illustrative purposes only, physical constraints of the layer depalletizing tool 99 include, but are not limited to, the grip area 1520 of the grip engagement interface (with the pallet layer 816 inserted) defined by the gripper length LT and gripper width WT when the grip 800 is in the open configuration, the planarity deviation between the side clamps 22-24 and the (vertical) side of the pallet layer (this deviation constraint may be similar to the planarity deviation of the plane 1200 relative to the plane 28P), the planarity deviation between the plane 1200 of the top surface 148 and the plane 28P of the top surface pad 28, picking of the pallet layer 816 performed so that the layer depalletizing tool 99 does not contact the pallet load PAL (other than gripping the pallet layer 816), etc. Also, for purposes of illustration, the physical constraints of the palletizing cell 10 include, but are not limited to, exclusion zones 1500-1502 that prevent movement of the robot 14 (i.e., the robot 14 and the layer depalletizing tools 99 carried by the robot 14 are excluded from entering the exclusion zones 1500-1502).

[0059] By way of example only, the optimal pick location may be determined by the cell controller 10C, where the cell controller 10C finds the pallet layer center 816C based on the pallet layer characteristics determined above (e.g., location of corners PC1-PC4, length L, width W, etc.). The cell controller 10C uses the known dimensions of the layer depalletizing tool 99 (e.g., grip length LT and grip width WT, in addition to outer dimensions) to determine a location for the layer depalletizing tool 99 that minimizes the distance between the center 99C and the center 816C while avoiding the exclusion zones 1500-1502 and satisfying the physical constraints of the layer depalletizing tool 99. The cell controller 10C is configured to determine the feasibility of the optimal pick location relative to the constraints mentioned herein ( FIG. 14 , block 1450).

[0060] While Figure 15A is an exemplary illustration of pallet layer 816 having a uniform case distribution such that it forms a complete pallet layer, in other embodiments, pallet layer 816 may have any suitable non-uniform or incomplete case unit distribution, such as those illustrated in Figures 16A-16C. The pallet layer is centered relative to the center 301CN of the pallet unloading / loading station 301 such that the optimal picking position for the layer depalletizing tool 99 is coincident with the center 816C of the pallet layer 816. Here, in addition to there being space between the layer depalletizing tool 99 and the exclusion zones 1500-1502, there is also space between the gripper 800 and the pallet layer 816 so that the pallet layer 816 can be inserted into the gripping area 1520 (e.g., this picking is ensured by the cell control device 10C).

[0061] While FIG. 15B is an exemplary illustration of pallet layer 816 having a uniform case distribution such that it forms a complete pallet layer, in other embodiments, pallet layer 816 may have any suitable non-uniform or incomplete case unit distribution, such as those illustrated in FIGS. 16A-16C. In this example, center 816C of pallet layer 816 is positioned off-center from center 301CN of pallet unloading / loading station 301. In the example illustrated in FIG. 15B, there is space between pallet layer 816 and exclusion zones 1500, 1501, 1502 to allow layer depalletizing tool 99 to be inserted, as well as space between pallet layer 816 and gripper 800 to allow pallet layer 816 to be inserted into gripping area 1520 (e.g., this picking is confirmed to be feasible by cell control device 10C).

[0062] While Figure 15C is an exemplary illustration of pallet layer 816 having a uniform case distribution such that it forms a complete pallet layer, in other embodiments, pallet layer 816 may have any suitable non-uniform or incomplete case unit distribution, such as those illustrated in Figures 16A-16C. In this example, center 816C of pallet layer 816 is positioned off-center from center 301CN of pallet unloading / loading station 301. While there is space between pallet layer 816 and exclusion zones 1500, 1501, 1502 in the example illustrated in Figure 15C to allow for insertion of layer depalletizing tool 99, Figure 15C illustrates an extreme example of placement of pallet layer 816 at pallet unloading / loading station 301 with minimal clearance between two sides of grip 800 (along the Z axis) and pallet layer 816. Again, the optimal picking position for the layer depalletizing tool 99 in the example illustrated in Figure 15C is the position where the centers 99C, 816C are offset but the distance between the centers 99C, 816C is minimized taking into account the space constraints between the pallet layer 816 and each of the exclusion zones 1500, 1501.

[0063] While Figure 15D is an exemplary illustration of pallet layer 816 having a uniform case distribution such that it forms a complete pallet layer, in other embodiments, pallet layer 816 may have any suitable non-uniform or incomplete case unit distribution, such as those illustrated in Figures 16A-16C. The center 816C of pallet layer 816 is positioned off-center from the center 301CN of pallet unloading / loading station 301. In the example illustrated in Figure 15D, there is space between pallet layer 816 and exclusion zones 1500, 1501, 1502, but gripper 800 cannot be positioned to place pallet layer 816 within gripping area 1520 without entering exclusion zone 1500 (e.g., this picking is not feasible by cell control device 10C). In the example illustrated in FIG. 15D, there is no optimal picking position for the layer depalletizing tool 99, the picking of the pallet layer is aborted, and the cell control device 10C provides an audible and / or visual alert to the operator to resolve the layer picking.

[0064] In an embodiment of the present disclosure, the cell controller 10C is configured to perform a characteristic analysis (using any suitable image analysis technique) of the palletizing cell 10 and the objects located therein ( FIG. 14 , block 1450). For example, in one embodiment, the vision system 310 is configured to identify a pallet support SPAL at the bottom of a pallet load PAL. In some instances where a pallet support is defective, another pallet support is placed below the pallet load PAL (i.e., so that the pallet load PAL includes two or more pallet supports stacked one on top of the other) so that the pallet load can be manipulated, for example, with a forklift. The vision system 310 is configured to identify stacked pallet supports SPAL to prevent other pallet supports (such as the bottommost pallet support) from being picked by the robot 14. In another embodiment, the vision system is configured to determine whether a partial pallet layer (where the case units of the layer do not span the entire pallet area (length L and width W)) provides sufficient support to be picked by the layer depalletizing tool 99. 16A-16C , exemplary partial pallet layers 1601, 1602, 1603 (which may replace, and in some embodiments are representative of, pallet layer 816 illustrated in FIGS. 15A-15D ) are illustrated that may be suitable for picking by the layer depalletizing tool 99 such that the gripping forces applied by the clamps 22-24 are applied generally uniformly along each clamp 22-24. Pallet layers that may be unsuitable for picking include pallet layers that cause the gripping forces applied by the clamps 22-24 to be applied in a non-uniform manner, causing moments on the clamps 22-24 that may cause some case units CU to be gripped and other case units CU to be ungriped. In yet other embodiments, the vision system 310 is configured to identify case units CUF that have fallen from the pallet load PAL to a location in the palletizing cell 10 (e.g., on the floor, on the conveyor 150), so as to generate an audible and / or visual alarm for operator intervention.The vision system 310 may also be configured to determine a "retry" trajectory for the robotic arm 14 in the event of a failed pick. For example, if the robotic arm fails to pick (i.e., the intended pick does not occur), the pallet layer may be re-analyzed by the vision system 310 and / or cell control device 10C in the manner described above to re-identify features of the pallet layer and generate a trajectory for the robotic arm 12 to retry the pick, where the retry trajectory may differ from the trajectory originally generated.

[0065] 3, 8A-10B, 15A-15D, and 17, operation of the layer depalletizing tool 99 to release a layer is described according to an embodiment of the present disclosure. In operation, any suitable controller (such as the robot controller 316 and / or the cell controller 10C) generates a robot space map 1599 (see FIGS. 15A-15D) that includes at least the pallet unloading / loading stations 301 and their structures (exclusion zones) (FIG. 17, block 17100). In one embodiment, the robot space map 1599 is generated as part of the release operation and / or the robot space map 1599 may be generated prior to the release operation. The robot space map 1599 may be generated in any suitable manner, such as using data obtained from the vision system 310 and / or data from a computer-aided design (CAD) model of the palletizer cell 10. While the robot space map 1599 in Figures 15A-15D is illustrated in two dimensions for illustrative purposes, it should be understood that the robot space map 1599 may have any suitable configuration, such as a two-dimensional depth map, a three-dimensional map including one or more point clouds representing structures / exclusion zones, a three-dimensional model generated from image and / or CAD data, etc.

[0066] The pallet load PAL is delivered to and placed at the pallet unloading / loading station 301 of the palletizer cell 10 (FIG. 17, block 17200). The pallet load PAL is imaged by the vision system 310 in a manner described herein (FIG. 17, block 17201). The robot controller 316 and / or cell controller 10C integrates the pallet image data into the robot space map 1599 (see FIGS. 15A-15D) such that at least a representation of the pallet layer 816 is generated in the robot space map 1599 to provide identification of the position, pose, etc. of the top pallet layer 816 (FIG. 17, block 17101). While only the pallet layer 816 is illustrated in FIGS. 15A-15D, it should be understood that image data representing the entire pallet may be iteratively integrated into the robot space map 1599 as each layer is released to provide identification of the position, pose, etc. of subsequent top pallet layers. The robot controller 316 and / or cell controller 10C are configured to dynamically define picking boundary conditions and constraints for the pallet layer 816 based on the position / pose of the pallet layer 816 in the robot space map 1599 ( FIG. 17 , block 17102). For example, based on image data from the vision system 310, the robot controller 316 and / or cell controller 10C are configured to dynamically determine the position of objects within the palletizing cell 10 in any suitable manner (e.g., by a suitable image recognition algorithm) and configure / reconfigure the robot space map 1599 based on the determined position of the objects (see, for example purposes only, the reconfiguration of the robot space map 1599 based on the entry of an object 1571 and the redefinition of an area 1570 in which the robot 14 is permitted to move). The object may be a transient object entering and / or exiting the palletizing cell 10, a portion of the palletizing cell 10 structure, a pallet load PAL, etc. (In some embodiments, a transient object is an unexpected / unauthorized object, in which case the controller may issue a command to the robot 14 to stop movement).Boundary conditions and constraints for the movement of the robot 14 are dynamically determined by the robot controller 316 and / or the cell controller 10C based on the position of objects within the palletizing cell 10. Examples of boundary conditions include the area 1570 of the palletizing cell 10 that allows or restricts the movement of the robot 14, while examples of constraints are those mentioned above, the type of object within the palletizing cell 10, the exclusion zones 1500-1502, the position / pose of the pallet load structure PAL, the acceleration of the movement of the robot 14 based on the contents of the pallet layer, etc.

[0067] When the robot controller 316 receives a signal, for example, from the cell controller 10C, indicating that the pallet layer 816 is ready to be picked ( FIG. 17 , block 17202), the robot controller 316 and / or the cell controller 10C determine a path 1580 and a trajectory 1581 for the robot 14 to move the layer depalletizing tool 99 from an initial position (e.g., a placement position of a previous picker pallet layer or any other suitable position) to a picking position (e.g., the position shown in FIGS. 15A-15D ) for picking the top pallet layer 816 ( FIG. 17 , block 17103). For example, the robot controller 316 and / or the cell controller 10C generate the robot path 1580 and the trajectory 1581 in any suitable manner based on the robot space map 1599 (including image data of the pallet layer 816), the determined boundary conditions, and the determined constraints. Determining the path 1580 and trajectory 1581 of the robot 14 may be an iterative procedure such that the path 1580 and trajectory of the robot 14 are optimized (e.g., time-optimized) to result in optimized movement of the robot 14 from an initial position of the layer depalletizing tool 99 (as described herein) to an optimal picking position, independent of the initial pose of the robot 14. The robot controller 316 and / or cell controller 10C commands the robot 14 to position the layer depalletizing tool 99 at the optimal picking position based on the optimized path 1580 and trajectory 1581 ( FIG. 17 , block 17204) (see FIGS. 8A and 8B ).

[0068] It is noted that for most layer depalletizing tools, systems and methods from the prior art, the positioning of the tool relies solely on the nominal vertical position of the layer.

[0069] However, it is known in the material handling industry that products 18 are often crushed within the pallet load PAL by the weight of the pallet layer 816 or layers above it. Of course, this is especially true when the last layer, resting directly on the pallet support SPAL (see FIG. 2), is being depalletized. Thus, the nominal position of each pallet layer 816 used in the initial programming of the robot 14 can lead to incorrect positioning of the layer depalletizing tool 99, limiting the system's ability to properly depalletize the pallet layer 816.

[0070] In an embodiment of the present disclosure, the top pad 28 of the tier depalletizing tool 99 is coupled to a sensor 888 configured to assess the actual height / position of the top surface 148 of the pallet layer 816 being picked, thus enabling more accurate positioning of the tier depalletizing tool 99. This added accuracy makes the depalletizing system, including the robot 14 and the tier depalletizing tool 99, more efficient and reduces the chance of product damage or also avoids cases where improper positioning prevents the depalletization of the products 18. The sensor may take the form of an analog laser distance sensor or any other suitable distance-determining sensor.

[0071] The top pad 28 is lowered and positioned over the top layer on the pallet ( FIG. 17 , block 17206). The robot 14 positions the layer depalletizing tool 99 at the height of the pallet layer 816 to be picked, as determined from the image data of the vision system 310. Upon recognizing that the top pad 28 contacts the top surface 148 of the pallet layer 816, the sensor 888 measures the position of the top pad 28. The robot controller 316 (and / or cell controller 10C) then calculates the actual height of the pallet layer 816 and compares the actual height with the height of the layer 816 determined from the image data of the vision system 310. If there is a difference between the actual height (as determined by the sensor 888) and the height of the layer 816 determined from the image data of the vision system 310, the robot controller 316 adjusts the position of the layer depalletizing tool 99 accordingly. The height difference can also be communicated to the cell control device 10C to recalibrate / fine-tune the vision system 310 to reduce or minimize the difference between the actual height (determined by the sensor 888) and the height of the layer 816 determined from the image data of the vision system 310.

[0072] Using the top pad 28 to determine the actual height of the top surface of the pallet represents a robust method for verifying the position of the top surface 148 of the pallet layer 816. For example, if a flap on the products 18 is lifted or the products 18 are not fully positioned, the overall position of the top pad 28 is not affected, thus providing valuable and accurate information about the actual height of the pallet layer 816. If necessary, the robot 14 adjusts the height of the layer depalletizing tool 99 ( FIG. 17 , block 17210).

[0073] Considering that each pallet layer 816 may be composed of multiple products 18, it often occurs that some products 18 do not have a side facing the exterior of the pallet layer 816. According to one embodiment, a combination of mechanisms is provided that allows the clamps 22-24 to properly grip the products 18. For example, four clamps 22-24 are used to compress the entire pallet layer 816 on each side without horizontally moving the pallet layer 816 by operating and controlling the pneumatic actuators 56 ( FIG. 17 , block 17212). The robot 14 then slightly lifts the layer depalletizing tool 99 to mainly create a gap between the lower surfaces 8150 ( FIG. 8B ) of the products 18 located around the pallet layer 816 and the upper surfaces of the products 18 in the immediately below pallet layer 816 ( FIG. 17 , block 17214). The horizontal curtains 26 then begin to close, inserting them under the pallet layer 816 ( FIG. 17 , block 17216). Closure of the horizontal curtain 26 is illustrated in Figures 9A-10B. The rotating friction head 70 of the curtain 26 contributes to moving the products 18 from the gripped pallet layer 816 onto the horizontal curtain 26. More specifically, this allows products 18 that are not located around the perimeter of the pallet layer 816 and that are generally only partially lifted, or sometimes not lifted at all, by the side clamps 22-24 to move onto the horizontal curtain 26.

[0074] Friction caused by the pressure of the clamps 22-24 on the vertical surfaces of the products 18 sometimes prevents the products 18 from moving upward to allow the horizontal curtain 26 to close underneath without damaging the products 18. When this occurs, the pressure is automatically reduced on the side clamps 22-24, thereby also reducing the aforementioned friction, thus facilitating movement of the curtain to lift and pick the remaining products 18 in the pallet layer 816 to be picked.

[0075] Fluctuations in the pressure on the clamps 22-24 are adjusted by monitoring the linear movement of the horizontal curtain 26 (FIG. 17, block 17218). If an item 18 blocks this movement, the horizontal curtain 26 cannot move forward. When this occurs, it is concluded that at least one item 18 is blocking the movement. Therefore, pressure is reduced on the clamps 22-24 by the actuator 56 (FIG. 17, block 17220), and the process of picking all items 18 on the pallet layer 816 continues.

[0076] According to another embodiment, the linear movement of the curtain 26 is monitored to detect its resistance to movement even as the curtain continues to move.

[0077] Once the curtain 26 is fully closed (see FIGS. 10A-10B ), the pallet layer 816 is fully gripped by the layer depalletizing tool 99 ( FIG. 17 , block 17222), and the robot 14 moves the layer depalletizing tool 99 with the gripped pallet layer 816 to transfer the pallet layer 816 to an outfeed location (e.g., conveyor 150) ( FIG. 17 , block 17232). The vision system 310 images the pallet load PAL during the placement cycle of the robot 14 to determine the pose, location, etc. of the next layer to be picked ( FIG. 17 , block 17233). The robot 14 places the pallet layer 816 on the outfeed conveyor 150 (or other suitable outfeed location) ( FIG. 17 , block 17234), and the pallet release process continues at block 17202.

[0078] In addition to the above function of enabling accurate assessment of the position of the upper pallet layer 816 of the pallet load PAL, the upper pad 28 also prevents small products 18 from "popping out" of the pallet layer 816 as the horizontal curtain 26 moves underneath. Such movement of products 18 would occur, for example, when the pallet layer 816 is composed of small products 18, i.e., products 18 that have been reduced in height. The weight of the upper pad 28 is sufficient to prevent the "popping out" effect of the small products without limiting the ability of the horizontal curtain 26 to move underneath. Similarly, the upper pad 28 prevents small products 18 from "popping out" when the horizontal curtain 26 is removed from underneath the products 18.

[0079] In some embodiments, slip sheets 277 ( FIG. 2 ) are provided between some layers of products 18 in a pallet load PAL. The standard approach is to have a separate device that automatically removes the slip sheets 277 between each layer's depalletizing sequence. While this approach works well, the addition of the separate device is quite costly. According to embodiments of the present disclosure, a layer depalletizing tool is provided with components that remove the slip sheets 277 disposed on the pallet layer 816 of products 18 at the same time the pallet layer 816 is depalletized. Because such a layer depalletizing tool is similar to the layer depalletizing tool 99 and this further release process is similar to that described above, for purposes of brevity, only the differences therebetween will be described with reference to FIG. 17 .

[0080] The tool includes an upper pad 28 including suction cups (not shown) for removing slipsheets on the pallet layer 816 as the pallet layer 816 is depalletized. Once the pallet layer 816 is fully gripped by the layer depalletizing tool ( FIG. 17 , block 17222), the suction cups are activated ( FIG. 17 , block 17224). Another, e.g., slipsheet sensor 999 (e.g., a camera or scanner configured for resolution of the case edges of the middle case as shown in FIGS. 9A and 9B, particularly FIG. 9B), facing upward toward the gripped pallet layer 816 being gripped and lifted by the tool (see also FIG. 9B), or the same mentioned above, is used to determine, for example, whether a slipsheet 277 attached to the upper pad 28 is present. As the robot 14 lifts and transports the pallet layer 816, the upper pad 28 is slightly lifted ( FIG. 17 , block 17226). If the slipsheet sensor 999 still detects the presence of an object ( FIG. 17 , block 17228), this means that there is a slipsheet 277 underneath the pallet layer 816 being depalletized ( FIG. 17 , block 17229) and a vacuum is maintained on the suction cups ( FIG. 17 , block 17230). Still referring to FIGS. 19A, 19B, and 19C , the robot 17 then places the pallet layer 816 (with the slipsheet 277 underneath it) onto a mat top conveyor (such as the outfeed conveyor 150), and after the pallet layer 816 is placed on the outfeed conveyor 150 ( FIG. 17 , block 17232) or elsewhere, the slipsheet is removed and discarded in a bin ( FIG. 17 , block 17236). Conversely, if the slipsheet sensor 999 detects nothing, this means that there is no slipsheet underneath the layer being depalletized. In such a case, once the previous pallet layer 816 is placed on the outfeed conveyor 150, the vacuum is removed from the suction cups and the robot 14 moves directly back to pick the next pallet layer 816 on the pallet.

[0081] 19A, 19B, and 19C, a slip-sheet remover 998 is positioned or otherwise disposed between portions of the conveyor 150. For example, the conveyor 150 includes a mat top or upstream portion 993 (where the layer being depalletized is placed) and a case spread or downstream portion 994. The slip-sheet remover 998 includes any suitable vacuum removal mechanism, such as a vacuum roller 997 (although any suitable adhesive or suction type roller may be used), a limiting plate 996, and a deflector shield 995. The vacuum roller 997 has any suitable configuration for gripping the slip-sheet 227, such as, for example, the vacuum roller rotates about an axis of rotation 989 and is fluidly connected to suction cups or ports 997P arranged on or in an outer surface (e.g., a product support surface) of the vacuum roller 997, and includes suction tubes 997T arranged to apply a suction force to the slip-sheet 227 as it passes over the vacuum roller 997. A limiting plate 996 is positioned below the vacuum roller 997 to peel or otherwise remove the slipsheet and guide it from the vacuum roller 997 to the bin 990. A deflector shield 995 is positioned downstream from the vacuum roller 997 (e.g., with respect to the direction of movement 988 of the pallet layer 816 along the conveyor 150) and has any suitable size and shape to prevent intrusion of case edges between the conveyors (e.g., so that cases transition smoothly from one conveyor portion to another without getting caught or jamming while passing through the slipsheet remover 998) and to at least partially direct the slipsheet 227 (removed from the bottom of the pallet layer 816) into the gap between the conveyor portions 993, 994 (although, in one or more embodiments, the vacuum roller 997 may provide sufficient suction to independently guide the slipsheet into and through the gap between the conveyor portions 993, 994). As described above, slipsheet 227 is detected by slipsheet sensor 999, which sends any suitable signal to controller 10C to activate slipsheet remover 998.Operation of skip sheet remover 998 includes automatically activating vacuum rollers 997 to automatically remove slip sheets from under pallet layer 816, where slip sheets 227 are prevented (via vacuum rollers 997) from adhering to the bottom of pallet layer 816 as pallet layer 816 transitions from conveyor section 993 to conveyor section 994. Vacuum rollers 997 grip slip sheets 227 as they move over them, where they transport slip sheets 997 through the gap between the conveyor sections and to limiting plate 996, where slip sheets 227 are removed from vacuum rollers 997 and disposed of in bin 990.

[0082] 3, 8A-10B, 15A-15D, and 18, a method of depalletizing cases in depalletizer 10 is provided. The method includes receiving, at pallet unloading station 301 of depalletizer 10 (FIG. 18, block 1800), a pallet load PAL of cases CU arranged in pallet load layers 816 (representative of pallet layers PL1-PL5), each pallet load layer 816 formed of a plurality of cases CU side-by-side at a common level across the area of ​​the pallet load PAL. A robot 14 is provided ( FIG. 18 , block 1810) and includes a depalletizing end effector 99 having grippers 800 configured to grasp and pick at least one pallet load layer 816 for transport from a pallet load PAL at a pallet unloading station 301 to an output station 333 (which in one embodiment includes any suitable conveyor 150). The grippers 800 have a grip engagement interface 810 that defines a predetermined layer engagement position and orientation (e.g., in the robot coordinate system or space X, Y, Z, RX, Ry, RZ—see FIG. 3 —also referred to herein as the robot's reference frame) for at least one of the pallet load layers 816 relative to the depalletizing end effector 99 to repeatedly effect capture and stable holding of the at least one pallet load layer 816 with the grippers 800. The vision system 310 images the pallet load PAL of cases CU at the pallet unload station 301 and generates at least one image of the top of at least one of the pallet load layers 816 independent of the robot's motion (FIG. 18, block 1820).A control device (such as the robot control device 16 and / or cell control device 10C) operably coupled to the vision system 310 receives at least one image from the vision system 310 and, based on the at least one image, determines the layer position and orientation of at least one of the pallet load layers 816 relative to a predetermined layer engagement position and orientation of the grip engagement interface 810 (Figure 18, block 1830), and the control device is operably coupled to the robot 14 to position the grip 800 and capture and hold at least one of the pallet load layers 816 using the grip 800 at the grip engagement interface 810.

[0083] According to one or more aspects of the present disclosure, a depalletizer includes:

[0084] a pallet unloading station configured to receive a pallet load of cases arranged in pallet load layers, each pallet load layer being formed of a plurality of cases side-by-side at a common level across an area of ​​the pallet load;

[0085] a robot including a depalletizing end effector having a gripper configured to grasp and pick at least one of the pallet load layers to transport the at least one pallet load layer from the pallet load at the pallet unloading station to an output station, the gripper having a grip engagement interface defining a predetermined layer engagement position and orientation for the at least one of the pallet load layers relative to the depalletizing end effector to repeatedly effect capture and stable retention of the at least one pallet load layer with the gripper;

[0086] a vision system positioned to image the pallet load of cases at the pallet unloading station and configured to generate at least one image of an upper portion of at least one of the layers of the pallet load independent of movement of the robot;

[0087] and a control device operably coupled to the vision system to receive at least one image from the vision system and configured to provide a determination of a layer position and orientation of at least one of the pallet load layers relative to a predetermined layer engagement position and orientation of the grip engagement interface based on the at least one image, the control device being operably coupled to the robot to position a grip and capture and hold at least one of the pallet load layers with the grip at the grip engagement interface.

[0088] According to one or more aspects of the present disclosure, the predetermined layer engagement position and orientation provides an engagement planar orientation of the grip engagement interface, the layer position and orientation representing a planarity of an engagement surface of at least one of the pallet load layers arranged to interface with the grip engagement interface substantially spanning at least one of the pallet load layers, and the layer position and orientation representing a planar misalignment in at least two orthogonal directions between the engagement surface of at least one of the pallet load layers and the planar orientation of the grip engagement interface.

[0089] According to one or more aspects of the present disclosure, the vision system comprises at least one camera mounted independently of the robot.

[0090] According to one or more aspects of the present disclosure, each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents a positional relationship of each respective camera frame of reference of each respective camera to each other of the at least one camera and to a predetermined frame of reference of the robot.

[0091] According to one or more aspects of the present disclosure, the at least one camera is positioned such that the field of view of the at least one camera covers the pallet load from a top upwardly facing surface of the pallet load to a bottom of the pallet load.

[0092] According to one or more aspects of the present disclosure, the at least one camera is positioned such that the topmost upwardly facing surface of each of the pallet load layers is within the field of view of the at least one camera.

[0093] According to one or more aspects of the present disclosure, a depalletizer includes:

[0094] a pallet unloading station configured to receive a pallet load of cases arranged in pallet load layers, each pallet load layer being formed of a plurality of cases side-by-side at a common level across an area of ​​the pallet load;

[0095] a robot including a depalletizing end effector having a gripper configured to grasp and pick at least one of the pallet load layers to transport the at least one pallet load layer from the pallet load at the pallet unloading station to an output station, the gripper having a grip engagement interface defining a predetermined layer engagement position and orientation for the at least one of the pallet load layers relative to the depalletizing end effector to repeatedly effect capture and stable retention of the at least one pallet load layer with the gripper;

[0096] a vision system, separate from the robot, positioned to image the pallet load at the pallet unloading station and configured to generate at least one image of an upper portion of at least one of the layers of the pallet load decoupled from the motion of the robot;

[0097] and a controller configured to effect a determination of a position and orientation relationship between the grip engagement interface and at least one respective top pallet layer of the pallet load layer based on the at least one image.

[0098] According to one or more aspects of the present disclosure, the controller is operatively coupled to the vision system to receive at least one image from the vision system.

[0099] According to one or more aspects of the present disclosure, a controller is operably coupled to the robot to position a gripper relative to each top pallet layer based on the determined relationship and to capture and hold at least one pallet layer with the gripper at the grip engagement interface.

[0100] According to one or more aspects of the present disclosure, the determined relationship represents the layer position and orientation of each uppermost layer relative to a predetermined layer engagement position and orientation of the grip engagement interface relative to a predetermined reference frame of the robot.

[0101] According to one or more aspects of the present disclosure, the control device determines a respective layer position and orientation of each top layer based on at least one image, and compares the respective layer position and orientation with a predetermined reference frame of the robot, thereby resulting in a determination of the determined relationship.

[0102] According to one or more aspects of the present disclosure, each layer position and orientation represents a planarity of an engagement surface of each top layer, each positioned to interface with a grip engagement interface substantially spanning each top layer, and each layer position and orientation represents at least one of planar misalignment and center-point misalignment in at least two orthogonal directions between the engagement surface of the top layer and the grip engagement interface.

[0103] According to one or more aspects of the present disclosure, the controller is configured to eliminate at least one of planar misalignment and center point misalignment for optimal grip engagement with each top layer, respectively, based on robot motion boundary conditions defined by at least one of the robot's architecture and structure bounding the depalletizer expressed in a predetermined reference frame of the robot.

[0104] According to one or more aspects of the present disclosure, the vision system comprises at least one camera mounted independently of the robot.

[0105] According to one or more aspects of the present disclosure, each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents a positional relationship of each respective camera frame of reference of each respective camera to each other of the at least one camera and to a predetermined frame of reference of the robot.

[0106] According to one or more aspects of the present disclosure, the at least one camera is positioned such that the field of view of the at least one camera covers the pallet load from a top upwardly facing surface of the pallet load to a bottom of the pallet load.

[0107] According to one or more aspects of the present disclosure, the at least one camera is positioned such that the topmost upwardly facing surface of each of the pallet load layers is within the field of view of the at least one camera.

[0108] According to one or more aspects of the present disclosure, a method for depalletizing cases in a depalletizer is provided, the method comprising:

[0109] receiving a pallet load of cases arranged in pallet load layers at a pallet unloading station of a depalletizer, each pallet load layer being formed of a plurality of cases side-by-side at a common level across the area of ​​the pallet load;

[0110] providing a robot with a depalletizing end effector having a gripper configured to grasp and pick at least one of the pallet load layers to transport the at least one pallet load layer from the pallet load at the pallet unloading station to an output station, the gripper having a grip engagement interface defining a predetermined layer engagement position and orientation for the at least one of the pallet load layers relative to the depalletizing end effector to repeatedly effect capture and stable retention of the at least one pallet load layer with the gripper;

[0111] imaging the pallet load of cases at the pallet unloading station using a vision system to generate at least one image of an upper portion of at least one of the layers of the pallet load independent of the robot's motion;

[0112] and a control device operably coupled to the vision system, receiving at least one image from the vision system and providing a determination of a layer position and orientation of at least one of the pallet load layers relative to a predetermined layer engagement position and orientation of the grip engagement interface based on the at least one image, the control device being operably coupled to the robot to position a grip and capture and hold at least one of the pallet load layers with the grip at the grip engagement interface.

[0113] According to one or more aspects of the present disclosure, the predetermined layer engagement position and orientation provides an engagement planar orientation of the grip engagement interface, the layer position and orientation representing a planarity of an engagement surface of at least one of the pallet load layers arranged to interface with the grip engagement interface substantially spanning at least one of the pallet load layers, and the layer position and orientation representing a planar misalignment in at least two orthogonal directions between the engagement surface of at least one of the pallet load layers and the planar orientation of the grip engagement interface.

[0114] According to one or more aspects of the present disclosure, the vision system comprises at least one camera mounted independently of the robot.

[0115] According to one or more aspects of the present disclosure, each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents a positional relationship of each respective camera frame of reference of each respective camera to each other of the at least one camera and to a predetermined frame of reference of the robot.

[0116] According to one or more aspects of the present disclosure, the at least one camera is positioned such that the field of view of the at least one camera covers the pallet load from a top upwardly facing surface of the pallet load to a bottom of the pallet load.

[0117] According to one or more aspects of the present disclosure, the at least one camera is positioned such that the topmost upwardly facing surface of each of the pallet load layers is within the field of view of the at least one camera.

[0118] According to one or more aspects of the present disclosure, a method for depalletizing cases in a depalletizer is provided, the method comprising:

[0119] receiving a pallet load of cases arranged in pallet load layers at a pallet unloading station of a depalletizer, each pallet load layer being formed of a plurality of cases side-by-side at a common level across the area of ​​the pallet load;

[0120] providing a robot with a depalletizing end effector having a gripper configured to grasp and pick at least one of the pallet load layers to transport the at least one pallet load layer from the pallet load at the pallet unloading station to an output station, the gripper having a grip engagement interface defining a predetermined layer engagement position and orientation for the at least one of the pallet load layers relative to the depalletizing end effector to repeatedly effect capture and stable retention of the at least one pallet load layer with the gripper;

[0121] imaging the pallet load at the pallet unloading station using a vision system located separately from the robot to generate at least one image of an upper portion of at least one of the layers of the pallet load decoupled from the motion of the robot;

[0122] and using a controller of the depalletizer to provide a determination of a position and orientation relationship between the grip engagement interface and at least one of each of the topmost pallet layers of the pallet load based on the at least one image.

[0123] According to one or more aspects of the present disclosure, the controller is operatively coupled to the vision system to receive at least one image from the vision system.

[0124] According to one or more aspects of the present disclosure, a controller is operably coupled to the robot to position a gripper relative to each top pallet layer based on the determined relationship and to capture and hold at least one pallet layer with the gripper at the grip engagement interface.

[0125] According to one or more aspects of the present disclosure, the determined relationship represents the layer position and orientation of each uppermost layer relative to a predetermined layer engagement position and orientation of the grip engagement interface relative to a predetermined reference frame of the robot.

[0126] According to one or more aspects of the present disclosure, the control device determines a respective layer position and orientation of each top layer based on at least one image, and compares the respective layer position and orientation with a predetermined reference frame of the robot, thereby resulting in a determination of the determined relationship.

[0127] According to one or more aspects of the present disclosure, each layer position and orientation represents a planarity of an engagement surface of each top layer, each positioned to interface with a grip engagement interface substantially spanning each top layer, and each layer position and orientation represents at least one of planar misalignment and center-point misalignment in at least two orthogonal directions between the engagement surface of the top layer and the grip engagement interface.

[0128] According to one or more aspects of the present disclosure, the controller eliminates at least one of planar misalignment and center point misalignment for optimal grip engagement with each top layer, respectively, based on the robot's motion boundary conditions defined by at least one of the robot's architecture and structure bounding the depalletizer expressed in a predetermined reference frame of the robot.

[0129] According to one or more aspects of the present disclosure, the vision system comprises at least one camera mounted independently of the robot.

[0130] According to one or more aspects of the present disclosure, each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents a positional relationship of each respective camera frame of reference of each respective camera to each other of the at least one camera and to a predetermined frame of reference of the robot.

[0131] According to one or more aspects of the present disclosure, the at least one camera is positioned such that the field of view of the at least one camera covers the pallet load from a top upwardly facing surface of the pallet load to a bottom of the pallet load.

[0132] According to one or more aspects of the present disclosure, the at least one camera is positioned such that the topmost upwardly facing surface of each of the pallet load layers is within the field of view of the at least one camera.

[0133] According to one or more aspects of the present disclosure, a slipsheet removal device for removing a slipsheet from a layer of goods moving along a traverse path is provided, the slipsheet removal device comprising: a frame; a roller rotatably coupled to the frame for rotation about an axis of rotation, the roller having a vacuum port extending through a product support surface of the roller; and a vacuum mechanism coupled to the roller to draw a vacuum through the vacuum port, the vacuum port positioned on the roller for engaging a slipsheet disposed between the roller and a layer of goods supported on the product support surface of the roller to grip the slipsheet and separate the slipsheet from the layer of goods.

[0134] According to one or more aspects of the present disclosure, the rollers generally simultaneously rotate the slipsheet, which is gripped by the vacuum mechanism, about an axis of rotation to separate the slipsheet from the layer of merchandise.

[0135] According to one or more embodiments of the present disclosure, the vacuum mechanism includes at least one suction tube extending through the roller.

[0136] According to one or more embodiments of the present disclosure, the vacuum port includes a suction cup coupled to a roller for gripping the slipsheet.

[0137] According to one or more aspects of the present disclosure, the slipsheet removal device further comprises a deflector shield for engaging the slipsheet and separating the slipsheet from the layer of merchandise.

[0138] According to one or more aspects of the present disclosure, the slip sheet removal device further includes a stripper plate coupled to the frame, the stripper plate positioned relative to the roller to remove (or otherwise peel / remove) the slip sheet from the roller.

[0139] According to one or more aspects of the present disclosure, the slipsheet removal device further comprises a collection bin configured to collect slipsheets separated from the layers of goods.

[0140] According to one or more aspects of the present disclosure, the slip-sheet removal apparatus further includes an upstream conveyor and a downstream conveyor, with the rollers disposed between the upstream conveyor and the downstream conveyor.

[0141] According to one or more aspects of the present disclosure, the transverse path extends along the upstream and downstream conveyors.

[0142] According to one or more aspects of the present disclosure, the upstream conveyor is a mat top conveyor and the downstream conveyor is a case spreader conveyor.

[0143] According to one or more aspects of the present disclosure, a depalletizer includes a pallet unloading station for receiving pallets of goods and separating the pallets into layers of goods; a case conveyor configured to transport the layers of goods from the pallet unloading station to a storage array and provide a supply of goods to the storage array, the case conveyor including a plurality of conveyor sections; and a slip-sheet removal system disposed between two adjacent conveyor sections of the plurality of conveyor sections, the slip-sheet removal system including a frame, a roller rotatably coupled to the frame for rotation about an axis of rotation, the roller having a vacuum port extending through a product support surface of the roller, and a vacuum mechanism coupled to the roller to draw a vacuum through the vacuum port, the vacuum port positioned on the roller for engaging a slip-sheet disposed between the roller and the layer of goods supported on the product support surface of the roller to grip the slip-sheet and separate the slip-sheet from the layer of goods.

[0144] According to one or more aspects of the present disclosure, a method of removing a slipsheet from a layer of goods is provided, the method including the steps of: providing a frame for a slipsheet removal apparatus; providing a roller rotatably coupled to the frame about an axis of rotation, the roller configured to support and transport a layer of goods disposed on the slipsheet along a transverse axis; vacuum gripping the slipsheet with the roller, the vacuum being generated by a vacuum mechanism coupled to the roller; and separating the slipsheet from the layer of goods using the roller.

[0145] It should be understood that the foregoing description is merely illustrative of aspects of the present disclosure. Various substitutions and modifications may be contemplated by those skilled in the art without departing from the aspects of the present disclosure. Accordingly, aspects of the present disclosure are intended to embrace all such substitutions, modifications, and variations that fall within the scope of any claims appended hereto. Furthermore, the mere fact that different features are recited in mutually different dependent or independent claims does not indicate that a combination of these features cannot be used to advantage and that such combination remains within the scope of aspects of the present disclosure.

Claims

1. 1. A vision system for a depalletizer, the vision system for the depalletizer comprising: an array of cameras positioned at the pallet unloading station to image pallet loads of cases received at the pallet unloading station disposed in pallet load layers, each pallet load layer being formed of a plurality of cases juxtaposed at a common level across an area of ​​the pallet load, the array of cameras being configured to generate at least one image of an upper portion of at least one of the pallet load layers; a controller operatively coupled to the camera array to receive the at least one image from the camera array and configured to cause a determination of at least one layer position and orientation of the pallet load layers relative to a predetermined layer engagement position and orientation of a robot grip engagement interface of a robot grip based on the at least one image, wherein the robot grip is configured to grasp and pick at least one of the pallet load layers with the robot grip engagement interface to depalletize the at least one pallet load layer from the pallet load of cases at the pallet unloading station, and the controller is configured to send position commands to the robot grip to capture and hold at least one of the pallet load layers with the robot grip engagement interface; 1. A vision system for a depalletizer comprising:

2. 2. The vision system for a depalletizer of claim 1, wherein the predetermined layer engagement position and orientation provides an engagement planar orientation of the robot grip engagement interface, the layer position and orientation represents a planarity of an engagement surface of at least one of the pallet load layers positioned to interface with the robot grip engagement interface substantially spanning at least one of the pallet load layers, and the layer position and orientation represents a planar misalignment in at least two orthogonal directions between the engagement surface of at least one of the pallet load layers and the engagement planar orientation of the robot grip engagement interface.

3. The vision system for a depalletizer of claim 1 , wherein the array of cameras comprises at least one camera mounted independently from the robot gripper.

4. 4. The vision system for a depalletizer of claim 3, wherein each of the at least one camera is calibrated to a common camera calibration reference structure, the calibration to the common camera calibration reference structure representing a positional relationship of a respective camera frame of reference of each respective camera to each other of the at least one camera and to a predetermined frame of reference of the robot gripper.

5. 4. The vision system for a depalletizer of claim 3, wherein the at least one camera is positioned such that the field of view of the at least one camera covers the pallet load from a top upwardly facing surface of the pallet load to a bottom of the pallet load.

6. 4. The vision system for a depalletizer of claim 3, wherein said at least one camera is positioned such that the topmost upwardly facing surface of each of said pallet load layers is within the field of view of said at least one camera.

7. 1. A vision system for a depalletizer, the vision system for the depalletizer comprising: an array of cameras disposed at the pallet unloading station, separate from the robot, for imaging pallet loads received at the pallet unloading station disposed in pallet load layers, each pallet load layer being formed of a plurality of cases arranged side by side at a common level across an area of ​​the pallet load, the camera array being configured to generate at least one image of an upper portion of at least one of the pallet load layers; a controller configured to cause a determination of a position and orientation relationship between a robot grip engagement interface of a robot grip and a respective top pallet layer of at least one of the pallet load layers based on the at least one image, the robot grip being configured to grasp and pick at least one of the pallet load layers with the robot grip engagement interface to transport the at least one pallet load layer from the pallet load at the pallet unloading station; 1. A vision system for a depalletizer comprising:

8. The vision system for a depalletizer of claim 7 , wherein the controller is operatively coupled to the array of cameras to receive the at least one image from the array of cameras.

9. 8. The vision system for a depalletizer of claim 7, wherein the controller is operatively coupled to the robot gripper to position the robot gripper relative to each top pallet layer based on the determined relationship and to capture and hold the at least one pallet load layer with the robot gripper at the robot gripper engagement interface.

10. 8. The vision system for a depalletizer of claim 7, wherein the determined relationship represents a layer position and orientation of each top layer relative to a predetermined layer engagement position and orientation of the robot grip engagement interface relative to a predetermined frame of reference of the robot grip.

11. 8. The vision system for a depalletizer of claim 7, wherein the controller determines a respective layer position and orientation of each top layer based on the at least one image and compares the respective layer position and orientation to a predetermined frame of reference of the robot gripper, thereby resulting in a determination of the determined relationship.

12. 12. The vision system for a depalletizer of claim 11, wherein the respective layer positions and orientations represent planarity of an engagement surface of each top layer positioned to interface with the robot grip engagement interface substantially spanning each top layer, and the respective layer positions and orientations represent at least one of planarity misalignment and center point misalignment in at least two orthogonal directions between the engagement surface of the top layer and the robot grip engagement interface.

13. 13. The vision system for a depalletizer of claim 12, wherein the controller is configured to resolve at least one of the planar misalignment and the center point misalignment for optimal grip engagement with each top layer, respectively, based on robot motion boundary conditions defined by at least one of a robot architecture and structure that bounds the depalletizer expressed in the predetermined frame of reference of the robot gripper.

14. The vision system for a depalletizer of claim 7 , wherein the array of cameras comprises at least one camera mounted independently from the robot gripper.

15. 15. The vision system for a depalletizer of claim 14, wherein each of the at least one camera is calibrated to a common camera calibration reference structure, the calibration to the common camera calibration reference structure representing a positional relationship of a respective camera frame of reference of each respective camera to each other of the at least one camera and to a predetermined frame of reference of the robot gripper.

16. 15. The vision system for a depalletizer of claim 14, wherein the at least one camera is positioned such that a field of view of the at least one camera covers the pallet load from a top upwardly facing surface of the pallet load to a bottom of the pallet load.

17. 15. The vision system for a depalletizer of claim 14, wherein the at least one camera is positioned such that a top upwardly facing surface of each of the pallet load layers is within the field of view of the at least one camera.

18. 1. A method of depalletizing cases using a vision system of a depalletizer, the method comprising: imaging, with an array of cameras disposed at a pallet unloading station, a pallet load of cases received at the pallet unloading station arranged in pallet load layers, each pallet load layer being formed of a plurality of cases arranged side-by-side at a common level across an area of ​​the pallet load, the camera array generating at least one image of an upper portion of at least one of the pallet load layers; receiving, in a controller operatively coupled to the camera array, the at least one image from the camera array and providing a determination based on the at least one image of a layer position and orientation of at least one of the pallet load layers relative to a predetermined layer engagement position and orientation of a robot grip engagement interface of a robot grip, the robot grip configured to grasp and pick at least one of the pallet load layers with the robot grip engagement interface to depalletize the at least one pallet load layer from the pallet load of cases at the pallet unloading station, the controller configured to send position commands to the robot grip to capture and hold at least one of the pallet load layers with the robot grip engagement interface; A method comprising:

19. 20. The method of claim 18, wherein the predetermined layer engagement position and orientation provides an engagement planar orientation of the robot grip engagement interface, the layer position and orientation representing a planarity of an engagement surface of at least one of the pallet load layers arranged to interface with the robot grip engagement interface substantially spanning at least one of the pallet load layers, and the layer position and orientation representing a planar misalignment in at least two orthogonal directions between the engagement surface of at least one of the pallet load layers and the engagement planar orientation of the robot grip engagement interface.

20. The method of claim 18 , wherein the array of cameras comprises at least one camera mounted independently from the robot grip.

21. 21. The method of claim 20, wherein each of the at least one camera is calibrated to a common camera calibration reference structure, the calibration to the common camera calibration reference structure representing a positional relationship of a respective camera frame of reference of each respective camera to each other of the at least one camera and to a predetermined frame of reference of the robot grip.

22. 21. The method of claim 20, wherein the at least one camera is positioned such that the field of view of the at least one camera covers the pallet load from a top upwardly facing surface of the pallet load to a bottom of the pallet load.

23. 21. The method of claim 20, wherein the at least one camera is positioned such that the topmost upwardly facing surface of each of the pallet load layers is within the field of view of the at least one camera.

24. 1. A method of depalletizing cases using a vision system of a depalletizer, the method comprising: imaging pallet loads received at the pallet unloading station arranged in pallet load layers using an array of cameras located at the pallet unloading station separate from the robot, each pallet load layer being formed of a plurality of cases arranged side by side at a common level across the area of ​​the pallet load, the camera array generating at least one image of an upper portion of at least one of the pallet load layers; using a controller of the depalletizer to cause a determination of a position and orientation relationship between a robot grip engagement interface of a robot grip and each topmost pallet layer of at least one of the pallet load layers based on the at least one image, the robot grip being configured to grasp and pick at least one of the pallet load layers with the robot grip engagement interface to transport the at least one pallet load layer from the pallet load at the pallet unloading station; A method comprising:

25. 25. The method of claim 24, wherein the controller is operatively coupled to the array of cameras to receive the at least one image from the array of cameras.

26. 25. The method of claim 24, wherein the controller is operably coupled to the robot gripper to position the robot gripper relative to each top pallet layer based on the determined relationship and to capture and hold the at least one pallet layer with the robot gripper at the robot gripper engagement interface.

27. 25. The method of claim 24, wherein the determined relationship represents a layer position and orientation of each uppermost layer relative to a predetermined layer engagement position and orientation of the robot grip engagement interface relative to a predetermined frame of reference of the robot grip.

28. 25. The method of claim 24, wherein the control device determines a respective layer position and orientation of each top layer based on the at least one image, and compares the respective layer position and orientation with a predetermined frame of reference of the robot grip, thereby resulting in a determination of the determined relationship.

29. 29. The method of claim 28, wherein the respective layer positions and orientations represent planarity of an engagement surface of each top layer positioned to interface with the robot grip engagement interface substantially spanning each top layer, and the respective layer positions and orientations represent at least one of planar misalignment and center point misalignment in at least two orthogonal directions between the engagement surface of the top layer and the robot grip engagement interface.

30. 30. The method of claim 29, wherein the controller eliminates at least one of the planar misalignment and the center point misalignment for optimal grip engagement with each top layer, respectively, based on robot motion boundary conditions defined by at least one of a robot architecture and structure bounding the depalletizer expressed in the predetermined frame of reference of the robot gripper.

31. The method of claim 24 , wherein the array of cameras comprises at least one camera mounted independently from the robot grip.

32. 32. The method of claim 31 , wherein each of the at least one camera is calibrated to a common camera calibration reference structure, the calibration to the common camera calibration reference structure representing a positional relationship of a respective camera frame of reference of each respective camera to each other of the at least one camera and to a predetermined frame of reference of the robot grip.

33. 32. The method of claim 31, wherein the at least one camera is positioned such that the field of view of the at least one camera covers the pallet load from a top upwardly facing surface of the pallet load to a bottom of the pallet load.

34. 32. The method of claim 31, wherein the at least one camera is positioned such that a top upwardly facing surface of each of the pallet load layers is within the field of view of the at least one camera.

Images