Vision-Assisted Robotic Depalletizer

Abstract

a robot having an end effector with a gripper for grasping and picking at least one of the layers and a grip engagement interface that defines a predetermined layer engagement position and orientation for the layer relative to the depalletizing end effector; a vision system for imaging the pallet load of cases and generating at least one image of an upper portion of the layer independent of movement of the robot; and a controller operatively coupled to the robot to receive the image and effect a determination of the layer position and orientation relative to the predetermined layer engagement position and orientation of the grip engagement interface, the controller positioning the gripper to capture and hold the layer with the gripper at the grip engagement interface.

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application is a non - provisional application of U.S. Provisional Patent Application No. 62 / 916,080, filed on October 16, 2019, the entire disclosure of which is incorporated herein by reference and for which priority is claimed.

[0002] [Technical Field] The present disclosure generally relates to depalletizing, and more specifically, to vision - assisted robotic depalletizing of products.

Background Art

[0003] The retail distribution of products (whether in a traditional "brick - and - mortar" store, an online store, or a hybrid retail channel) requires improvements in storage, sorting, and transportation efficiency, particularly for the distribution of what are known as mixed cases or heterogeneous cases (within a given shipment), whether for store replenishment or individual orders. The application of intelligent / adaptive automation to these has increasingly facilitated improvements in efficiency at many levels of the distribution, including storage, sorting, and transportation.

[0004] In distribution centers and warehouses, products such as cases, boxes, open trays, stretch wrap trays, etc. are typically received on structured pallets, for example, they are neatly positioned without gaps. Depalletizing systems for removing products from pallets are known in the art. Conventional pallet unloaders (such as depalletizers) having electromagnetic radiation and optical mapping sensors (such as laser scanners, 3D cameras, etc.) for mapping 3D pallet loads to improve automatic positioning of pallet loads are known. For example, one conventional method and system for detecting and reconstructing such an environment to facilitate robot interaction with the environment includes the step of determining a three-dimensional (3D) virtual environment, where the 3D virtual environment represents the physical environment of a robot manipulator including a plurality of 3D virtual objects corresponding to respective physical objects within the physical environment. The method then includes the step of determining a 2D image of the virtual environment including a two-dimensional (2D) depth map. The method may then include the step of determining a portion of the 2D image corresponding to a given one or more physical objects. The method may then include the step of determining a 3D model corresponding to the portion of the 2D image corresponding to a given one or more physical objects based on the portion and the 2D depth map. The method may then include the step of selecting one physical object from a given one or more physical objects based on the 3D model. The method may then include the step of providing instructions to the robot manipulator to move the object. SUMMARY OF THE INVENTION

[0005] The foregoing aspects and other features of the present disclosure are described in the following description in connection with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8A

Figure 8B

Figure 9A

Figure 9B

Figure 10A

Figure 10B

Figure 11A

Figure 11B

Figure 11C

Figure 11D

Figure 11E

Figure 11F

Figure 12

Figure 13A

Figure 13B

Figure 14

Figure 15A

Figure 15B

Figure 15C

Figure 15D

Figure 16A

Figure 16B

Figure 16C

Figure 17

Figure 18

Figure 19A

Figure 19B

Figure 19C

DETAILED DESCRIPTION OF THE INVENTION

[0007] FIG. 1 is a schematic diagram of a warehouse system or distribution facility 100WS (referred to herein as warehouse system 100WS) according to an aspect of the present disclosure. Although aspects of the present disclosure are described with reference to the drawings, it should be understood that they can be embodied in many forms. Further, any suitable size, shape, or type of element or material can be used. The distribution facility 100WS is described herein as an automated distribution facility, but it should be understood that aspects of the present disclosure are applicable to distribution facilities having any suitable conveyance system, such as both automated and manual conveyance systems, or a completely manual conveyance system.

[0008] Referring to FIGS. 1 and 2, according to an aspect of the present disclosure, a warehouse system 100WS includes at least one palletizer / depalletizer cell 10A, 10B (collectively referred to herein as palletizer cell 10). The palletizer cell 10 has one or more robotic case manipulators 14 (also referred to herein as articulated robots, adaptive real-time robots, robots, or product picking devices) that place article units CU of mixed pallet loads (also referred to herein as case units or cases or products 18) into stacks SL1-Sn and / or layers PL1-PL4 that build a mixed case pallet load PAL with visual system assistance (individually or at a manufactured pick face). Suitable examples of palletizers / depalletizers are described in U.S. Patent No. 10,343,857, entitled "Vision-Assisted Robotized Depalletizer," issued July 9, 2019, the entire disclosure of which is incorporated herein by reference.

[0009] The palletizer cell 10 is provided with a (one or more) three-dimensional (3D) time-of-flight (TOF) camera vision system 310 (referred to herein as the vision system 310) that generates a three-dimensional (3D) image of each pallet layer (also referred to herein as the pallet load layer) removed by the robot 14 and its case unit CU. The vision system 310 is arranged in the palletizer cell 10 to image the pallet load PAL of the case CU at the pallet loading / unloading station 301 and, independently of the movement of the robot 14, generates at least one image (see FIG. 12) of at least the upper / surface 148 (FIGS. 2, 8A, and 11A-11F) of at least one pallet load layer 816 (representing pallet layers PL1, PL2, PL3, PL4, PL5 in FIG. 8B). The three-dimensional image information is generated and provided by the vision system 310 in real-time in accordance with the periodic movement of the robot 14 that picks the pallet layers to depalletize the goods from the pallet load PAL, and notifies at least the layer pose for each pallet layer in the pallet build from the first case layer PL1 to the last case layer PL5 placed on the pallet support SPAL in real-time (within the picking / placement movement cycle frame of the robot 14).

[0010] The three-dimensional image information of the layer pose of each layer identifies differences from a plane (e.g., the inclination of the layer with respect to the pallet support SPAL and / or other layers, open case unit CU, etc.), for example, to notify compensation for differences with respect to the robot 14, whereby the robot 14 compensates for the positioning of the robot 14 with respect to the layer being picked in real-time, facilitating an adaptive real-time robot 14 placement that is substantially continuous and an adaptive depalletizing (in cooperation / collaboration with full automation or user assistance), while at the same time resolving pallet quality / control and depalletizing by the robot 14.

[0011] A control device (such as the robot control device 16 and / or the cell control device 10C) is operably connected to the vision system 310 so as to receive at least one image from the vision system 310. Based on the at least one image, the control device is configured to determine the layer position and pose / orientation of at least one pallet layer 816 (in FIG. 8B) with respect to a predetermined layer engagement position and orientation of the gripper engagement interface 810 (as described in this specification) of the gripper 800 of the robot 14 (in the coordinate system or reference frame X, Y, Z, RX, RY, RZ, etc. of the robot (in FIG. 3)), where the control device positions the gripper 800 with respect to each top pallet layer (based on the determined relationship, for example, between the gripper interface 810 and at least one respective top pallet layer of the pallet load layer), and is operably connected to the robot 14 to capture and hold at least one pallet layer 816 using the gripper 800 at the gripper engagement interface 810. The control device determines the layer position and orientation of each respective top layer based on the at least one image, and by comparing the respective layer position and orientation with a predetermined reference frame of the robot 14 (as described in this specification), results in the determination of the determined relationship.

[0012] The vision system 310 incorporated in the automated palletizer cell 10, in one aspect, notifies and enables the cell control device 10C to provide real-time (or near real-time) command input for the automation of (one or more) robots 14, etc., which responds in real time to differences in the pallet load (corresponding to the commanded transaction time, as further described), whereby (one or more) robots 14 adapt in real time to eliminate differences in the pallet load build and effect depalletizing in a timely optimal manner (automatically and / or in cooperation / collaboration with user assistance) to effect depalletizing in a timely optimal manner.

[0013] Referring again to FIG. 1, according to aspects of the present disclosure, the distribution facility 100WS includes, for example, a storage and retrieval system 100 that can operate in a retail distribution center or warehouse to fulfill orders received from a retail store for case units. In one example, a case unit can be a case or unit of merchandise that is not stored (e.g., contained) on a tray, tote, or pallet. In other examples, a case unit can be a case or unit of merchandise that is contained in any suitable manner, such as on a tray, tote, or pallet. Note that a case unit can include units (e.g., cases of soup cans, boxes of cereal, etc.) or individual items of merchandise that are placed in cases of merchandise that are adapted to be removed from or placed on a pallet. According to an embodiment, a shipping case for a case unit (e.g., a carton, barrel, box, wooden frame, jug, or any other suitable device for holding a case unit) can have a variable size, can be used to hold a case unit during transportation, and can be configured to be palletized for transportation. Note that, for example, when a bundle or pallet of case units arrives at the storage and retrieval system, the contents of each pallet can be uniform (e.g., each pallet holds a predetermined number of the same items. That is, one pallet holds soup and another pallet holds cereal), and when a pallet exits the storage and retrieval system, the pallet can include any suitable number and combination of various case units (e.g., each pallet can hold various types of case units. That is, the pallet holds a combination of soup and cereal). In an embodiment, the storage and retrieval system described herein can be applied to any environment in which case units are stored and retrieved.

[0014] The storage and retrieval system 100 can be configured, for example, to be installed in an existing warehouse structure or adapted to a new warehouse structure. In aspects of the present disclosure, the storage and retrieval system can include one or more infeed transfer stations 170 and one or more outfeed transfer stations 160, in / out case conveyors 150A, 150B, 150C (collectively referred to as in / out case conveyor 150), an array of storage structures 130, and a number of autonomous vehicle transport robots 110 (referred to herein as "bots"). In aspects of the present disclosure, the storage and retrieval system can also include a robot or bot transfer station, as described in U.S. Patent No. 9,096,375, issued August 4, 2015, the entire disclosure of which is incorporated herein by reference. In aspects of the present disclosure, the bot transfer station can provide an interface between the bot 110 and the in / out case conveyor 150 such that case units can be indirectly transferred between the bot 110 and the in / out case conveyor 150 via the bot transfer station. In aspects of the present disclosure, case units can be directly transferred between the bot 110 and the in / out case conveyor 150.

[0015] The storage structure array 130 may include multiple levels of storage rack modules that form a storage array for storage locations 130SL for case units, each storage location 130SL being arranged to store at least one case unit at each storage location 130SL. In one aspect, each level of the storage structure array 130 includes respective storage / picking aisles 130A and a transfer deck 130B for transferring case units between any of the storage areas of the storage structure array 130 and any shelf of any in / out case conveyor 150. The storage aisles 130A and the transfer deck 130B are also configured to allow a bot 110 to cross the storage aisles 130A and the transfer deck 130B to place case units in picking stock and retrieve the ordered case units, where the case units are stored or otherwise held at the storage locations 130SL, within the storage aisles 130A, and / or on the transfer deck 130B. The bot 110 can be any suitable bot capable of carrying and transferring case units throughout the storage and retrieval system 100. Suitable examples of bots can be seen, for illustrative purposes only, in U.S. Patent No. 8,425,173, issued April 23, 2013; U.S. Patent No. 9,561,905, issued February 7, 2017; U.S. Patent No. 8,965,619, issued February 24, 2015; U.S. Patent No. 8,696,010, issued April 15, 2014; U.S. Patent No. 9,187,244, issued November 17, 2015; U.S. Patent Application Publication No. 2012 / 0189416 (U.S. Serial No. 13 / 326,952), filed December 15, 2011, entitled "Automated Bot with Transfer Arm"; and U.S. Patent No. 9,499,338, issued November 22, 2016, the entire disclosures of which are incorporated herein by reference.Bot 110 places case units such as the above-mentioned retail products in the picking stock at one or more levels of the storage structure array 130, and then selectively retrieves the ordered case units and is configured to ship the ordered case units to, for example, a store or other appropriate location.

[0016] The infeed transfer station 170 and the outfeed transfer station 160 transfer case units bi-directionally to / from one or more levels of the storage structure array 130 and can operate together with their respective in / out case conveyors 150A, 150B to effect infeed of case units to the storage structure array 130 and output of case units from the storage structure array 130. Although the infeed transfer station 170 and the outfeed transfer station 160 (and their respective in / out case conveyors 150A, 150B and palletizer / depalletizer cells 10A, 10B) are described as being dedicated inbound (e.g., infeed) and outbound (e.g., outfeed) transfer stations 170 and 160, in aspects of the present disclosure, each of the transfer stations 170, 160 may be used for both inbound and outbound transfer of case units from the storage and retrieval system. Although in / out case conveyors are described herein, the conveyor may be any suitable conveyor (including any suitable conveyance path orientation such as vertical and / or horizontal conveyor paths) or a transfer / picking device having any suitable conveyance path orientation.

[0017] In one aspect, as described above, each of the infeed transfer station 170 and the outfeed transfer station 160 includes respective in / out case conveyors 150A, 150B and respective palletizer / depalletizer cells 10A, 10B (collectively referred to herein as palletizer cell 10). In one aspect, the palletizer / depalletizer cell 10 is an automated cell, each of which is configured to receive a load pallet from an area of the pallet load 175 that may include, for example, an in / out load pallet conveyor 175C (illustrated in FIG. 1 as an input conveyor) for (uniform or mixed case units or products, etc.), and / or, for example, for constructing a load pallet for conveyance to an area of the pallet offload 180 that may include an in / out load pallet conveyor 180C (illustrated in FIG. 1 as an output conveyor) for (uniform or mixed case units or products, etc.). In one aspect, conveyors 175C, 180C are each connected to the storage structure array 130 and are configured to convey load pallets bidirectionally in an input direction towards the storage structure array 130 and in a different output direction away from the storage structure array 130. In one aspect, conveyors 175C, 180C may each include a conveyor arrangement with a distribution conveyor bed arranged to form a transport path, or in other aspects, conveyors 175C, 180C may be individual transport units such as, for example, forklifts / pallet trucks. Suitable examples of the automated palletizer / depalletizer cells 10A, 10B can be found in U.S. Patent Application No. 15 / 235,254, filed Aug. 12, 2016, and U.S. Patent No. 8,965,559, issued Feb. 24, 2015, the entire disclosures of which are incorporated herein by reference. Each palletizer cell includes one or more robotic case manipulators 14, which may also be referred to as articulated robots or robots.One or more robotic case manipulators 14 are configured to continuously convey and place article units CU of pallet loads onto a pallet support as described herein to build (or in other aspects, disassemble or release as described herein) a pallet load 250 on a pallet load unloading / loading station 301 (see FIG. 3).

[0018] When the palletizer cell 10 functions as an output in the role of a palletizer, article units CU of pallet loads, which can be of various sizes, reach the palletizer cell 10 via an in / out case conveyor 150B as described herein and are picked up by one of the robotic case manipulators 14 and placed on a pallet load PAL. When the palletizer cell 10 functions as an output in the role of a palletizer, a full pallet load PAL (see FIG. 2) created from various case units is ready to be picked up by a forklift from the palletizer cell 10 for transportation to an area outside the pallet load 180. When the palletizer / depalletizer cell 10 functions as an input in the role of a depalletizer, a full pallet load of cases (similar to a pallet load PAL and which can be formed of homogeneous or mixed cases) created from article units CU of various pallet loads arranged in a pallet load layer is transferred from an area within the pallet load 175 to the pallet load unloading / loading station 301 of the palletizer cell 10 by any suitable means such as a forklift or other conveyance. Each of the pallet load layers PL1, PL2, PL3, PL4, PL5 is formed of a plurality of cases CU juxtaposed at a common level across the area of a pallet load PAL. In one aspect, as illustrated in FIG. 2, the pallet layer can be a mixed pallet layer containing cases CU of different sizes, but in other aspects, as illustrated in FIG. 3, it can be a uniform layer containing cases CU having substantially the same size across the entire pallet layer. One or more robotic case manipulators 14 pick article units CU of pallet loads from a pallet PAL for transfer into a storage structure array 130.

[0019] In one aspect, each infeed transfer station 170 forms a case input path Ip, where the palletizer / depalletizer cell 10A depalletizes case units layer by layer or otherwise depalletizes case units from a standard pallet (e.g., a homogeneous pallet having a stability suitable for automatic engagement of pallet layers by an automatic layer interface unit such as a product picking device or robot 14) to a single case unit. The palletizer / depalletizer cell 10A communicates with a conveying system of the automated storage and retrieval system 100, such as an in / out case conveyor 150A, to form an integrated input system (e.g., the infeed transfer station 170) that supplies case units to the automated storage and retrieval system 100. Each infeed transfer station 170 defines a case input path Ip that is integrated with the automated storage and retrieval system 100 and the warehouse management system 199, where the warehouse management system 199 uses any suitable non-transitory program code and memory to manage at least the input of case units to the storage structure array 130, the storage allocation of case units within the storage structure array 130, and the retrieval of case units from the storage structure array 130, the inventory / replenishment of case units, and the case unit output, and includes any suitable control device 199C configured to manage the above aspects.

[0020] In one aspect, the input path Ip for each case unit includes at least one corresponding case unit inspection cell 142 that communicates with the warehouse management system 199. In one aspect, the at least one corresponding case unit inspection cell 142 can be any suitable inspection cell that includes any suitable volume inspection, such as a multi-dimensional light curtain, an imaging system, and / or any other suitable sensing / sensor arrangement configured to detect defects in the case unit and identify the case unit for, e.g., inventory, conveying sequences, storage allocation, and sequences of case units for output from the storage structure array 130.

[0021] In one aspect, as described above, the palletizer / depalletizer cell 10A can be fully automated to disassemble or release (one or more) layers from the pallet being lowered in the palletizer / depalletizer cell 10A. Referring to FIG. 2, the term "decommission" refers to the removal of the article units CU of each pallet load from the layers PL1, PL2, PL3, PL4 at a predetermined level 200 (which may correspond to the decommissioning / commissioning level or transfer surface of the pallet PAL), such that, in some aspects, the pallet PAL is indexed to the next level of the pallet PAL (by any suitable pallet lifting device of the palletizer cell 10) for the removal of the next layers PL2, PL3 (entire or partial) corresponding to the next level of the pallet PAL, which refers to the removal of the (entire or partial) pallet layers PL1, PL2, PL3, PL4 from the pallet PAL.

[0022] In one aspect, the palletizer / depalletizer cell 10A is configured to release the layers PL1, PL2, PL3, PL4 such that the release is synchronized with or otherwise coordinated (e.g., matched) with a predetermined rate or supply speed of the supply of case units established by the warehouse management system 199 in the automated storage and retrieval system 100. For example, in one aspect, the warehouse management system 199 is configured to set and / or monitor a predetermined rate of the supply of case units within the automated storage and retrieval system 100. For example, the warehouse management system 199 monitors and manages the automated systems of the automated storage and retrieval system 100 (e.g., the in / out case conveyors 150A, 150B, the bots 110, and the palletizer / depalletizer cells 10A, 10B, etc.), where each of the automated systems, or one or more of the automated systems, are actually, individually or in combination, under the control of the warehouse management system 199 or any other suitable control device of the automated storage and retrieval system 100 (e.g., a bot control device, a conveyor control device, a palletizer / depalletizer control device, etc.) and define a predetermined rate of the supply of case units in the automated storage and retrieval system 100 established by the warehouse management system 199, with a given transaction time (e.g., the time to transfer case units on / off the in / out case conveyors to / from a picking / placement station, or to lift a case unit a predetermined distance, or the picking / placement of bot transfer at a storage location, etc., the time / period that enables a basic unit of case conveyance or transfer, the time to transfer a pallet layer to / from a pallet, etc.). For example, the control device 199C of the warehouse management system 199 is communicatively connected to the in / out case conveyors 150A, 150B such that the in / out case conveyors 150A, 150B convey case units bidirectionally to / from the storage structure array 130 at a predetermined case supply speed.The control device 199C can also be communicably connected to the palletizer / depalletizer cells 10A, 10B corresponding to the (one or more) in-out case conveyors 150A, 150B such that the attachment and release of the layers of the palletizer / depalletizer cells 10A, 10B, which are each substantially continuous, match a predetermined case supply rate. Aspects of the present disclosure are described herein with respect to a distribution facility 100WS having an automated storage and retrieval system 100 with an automated conveyance system, but aspects of the present disclosure are also applicable to distribution facilities having any suitable conveyance system, such as both an automated conveyance system and a manual conveyance system, or a completely manual conveyance system, where both the automated conveyance transactions and the manual conveyance transactions each have their respective transaction times, and the attachment and release of the case units to / from the pallet can be matched to the transaction times in a manner substantially similar to the methods described herein.

[0023] In one aspect, each outfeed transfer station 160 forms a case output path Op, where the palletizer / depalletizer cell 10B palletizes the case units layer by layer on the pallet PAL using an automatic layer interface unit such as one or more robotic case manipulators 14. In one aspect, the pallet PAL can be formed as a standard pallet (e.g., homogeneous case units), such as those described in U.S. Patent Application No. 14 / 997,920, filed on January 18, 2016, which is hereby incorporated by reference in its entirety, or as a mixed pallet. In one aspect, the warehouse management system 199 is configured to establish a pallet solution using mixed case units that provides a stable pallet load stack suitable for the end effector of one or more robotic case manipulators 14 to transfer as a layer. As described above, a suitable example of the palletizer / depalletizer cell 10B can be found in U.S. Patent Application No. 15 / 235,254, filed on August 12, 2016, which is hereby incorporated by reference in its entirety.

[0024] In one aspect, the palletizer / depalletizer cell 10B communicates with a conveying system of the automated storage and retrieval system 100, such as an in / out case conveyor 150B, to form an integrated output system (e.g., an outfeed transfer station 160) that receives case units from the automated storage and retrieval system 100 for placement on a pallet according to any suitable case out - order sequence. For example, as described above, the case - loaded article units CU destined for one or more robot case manipulators 14 are transferred to the pallet PAL by the end - effector of the one or more robot case manipulators 14, and the case - loaded article units CU (output case units) are arranged in a predetermined sequence established by the warehouse management system 199 layer by layer (note that a layer may cover the whole or part of the pallet) to form a standard output pallet load.

[0025] Each outfeed transfer station 160 defines a case output path Op that is integrated with the automated storage and retrieval system 100 and the warehouse management system 199, where the warehouse management system 199, as described herein, uses any suitable non-transitory program code and memory and includes any suitable control device 199C configured to manage the operation of the distribution facility 100WS, including the output of case units from the storage structure array 130. In one aspect, each case unit output path Op includes at least one corresponding case unit inspection cell 142 (as described above) that communicates with the warehouse management system 199. In one aspect, as described above, the palletizer / depalletizer cell 10B can be fully automated to build or attach (one or more) layers to the pallets loaded into the palletizer / depalletizer cell 10B. Referring to FIG. 2, the term commission refers to the insertion of article units CU of each pallet load at a predetermined level 200 (which may correspond to the release / attachment level or transfer surface of the pallet) into layers PL1, PL2, PL3, PL4 of the pallet PAL until the pallet layers PL1, PL2, PL3, PL4, PL5 are formed, whereby in some aspects the pallet PAL is indexed (by any suitable pallet lifting device of the palletizer cell 10) to the next level of the pallet PAL for the construction of the next layers PL1, PL2 (either in whole or in part) corresponding to the next level of the pallet PAL, referring to the construction of the pallet layers PL1, PL2, PL3, PL4 (either in whole or in part) on the pallet PAL.In one aspect, the palletizer / depalletizer cell 10B is attached in a manner substantially similar to that described above with respect to the release of the layers PL1, PL2, PL3, PL4, in the automated storage and retrieval system 100, in a manner synchronized with or otherwise coordinated (e.g., matched) to a predetermined ratio or supply rate of the supply of case units established by the warehouse management system 199, where the warehouse management system 199 manages the order of removal of case units, the sequence of mixed case units output in the load-out sequence of the pallet load of the mixed case units, and other relevant aspects of the output such as inventory adjustment.

[0026] According to an aspect of the present disclosure, referring to FIG. 3, a palletizer / depalletizer cell 10A configured to release layers PL1, PL2, PL3, PL4, PL5 includes at least one robot 14 having a robot arm 12 coupled to a robot controller 316 (coupled to or forming part of the cell controller 10C). The robot arm 12 is in the form of a standard industrial articulated robot arm suitable for releasing layers as described herein. In one aspect, the robot arm 12 includes six degrees of freedom of movement, while in other aspects, it may have more or less than six degrees of freedom of movement. As used herein, the expressions "robot" and "robot arm" are used interchangeably to mean a programmable system including articulated and / or movable members capable of receiving, controlling, and moving a tool. As illustrated in FIG. 3, a layer depalletizing tool or end effector 99 is coupled to the robot arm 12 and is configured to release layers PL1, PL2, PL3, PL4, PL5 as described herein. In one aspect, the layer depalletizing tool 99 may be substantially similar to the tool described in U.S. Patent Application No. 14 / 720,089, filed May 22, 2015, titled "Tool and Method for Layer Depalletizing," the entire disclosure of which is incorporated herein by reference.

[0027] Referring to FIGS. 3 - 5 and 8B, layer depalletizing tool 99 is configured to grip and pick at least one pallet load layer 816 (representing any one of pallet layers PL1, PL2, PL3, PL4, PL5) and convey at least one pallet load layer 816 from a pallet load PAL at pallet load / unload station 301 to an output station 333 (including in one aspect any suitable conveyor such as conveyor 150). Grip 800 has a grip engagement interface 810 that defines a predetermined layer engagement position and orientation (for example, in a robot coordinate system or space X, Y, Z, RX, Ry, RZ - see FIG. 3, referred to herein as the robot's reference frame) for at least one pallet layer 816 relative to layer depalletizing tool 99, such that repeated capture and stable holding of at least one pallet load layer 816 using grip 800 is achieved. Note that the configuration of grip 800 described herein is one suitable example of a pallet layer gripping mechanism that can be utilized in aspects of the present disclosure, but in other aspects, the grip may have any suitable differential pressure grip, bladder grip, or other capture mechanism that defines an engagement / capture system that interface - connects with the pallet layer for transfer of the pallet layer by the grip. Layer depalletizing tool 99 includes frame 20, four side clamps 22 - 24 movably attached to frame 20 for gripping and releasing pallet layers 1PL1, PL2, PL3, PL4, PL5, two curtains 26 attached to the frame under clamps 22 - 24 and inserted under the pallet load layer 816 gripped by clamps 22 - 24, and an upper pad 28 attached to frame 20 above clamps 22 - 24, where one or more of the four side clamps 22 - 24, two curtains 26, and upper pad 28 form grip engagement interface 810. The predetermined layer engagement position and orientation provide an engagement plane orientation of grip engagement interface 810 (such as at least partially defined by upper pad 28).The layer position and orientation represent the flatness of the engagement surface 1210 (which may coincide with the upper surface 148) of at least one (upper) pallet layer 816, arranged to interface with a grip engagement interface 810 that extends substantially across at least one pallet layer 816 (see plane 1200 of FIG. 12), and the layer position and orientation represent a planar mismatch in at least two orthogonal directions (e.g., in a robot coordinate system or reference frame) between the engagement surface 1210 of at least one pallet layer 816 and the planar orientation of the grip engagement interface 810. A control device (such as the robot control device 16 and / or the cell control device 10C) is configured to resolve at least one of the planar mismatch and the center point mismatch for optimal grip engagement with each top layer, respectively, based on the motion boundary conditions of the robot defined by at least one of the architecture and structure of the robot bounding the depalletizer 10, represented in a predetermined reference frame of the robot 14.

[0028] Frame 20 includes two pairs of parallel walls 30-32 assembled to generally define a rectangular perimeter. Each wall 30 and 32 includes a bottom rectangular portion 34 and 36, respectively, and an integral triangular portion 38 and 40, respectively. The two triangular portions 40 are slightly curved towards each other. Frame 20 further includes two lateral rectangular hollow tubes 42 and 44 extending parallel to wall 30 between walls 32 and parallel to wall 32 between walls 30, respectively. Near the top of the triangular portions 38-40 of walls 30-32, holes 45 (see FIG. 6A) are provided where the hollow tubes 42 and 44 are fixed, passing through walls 30-32, after which connectors and cables (not shown) passing through the hollow tubes 42 and 44 are enabled. Mounting brackets 46 (see FIG. 8A) are fixed at their intersections to both of the hollow tubes 42 and 44, thereby enabling the layer depalletizing tool 99 to be attached to the robot arm 12. The frame components 30-44 are assembled using fasteners and / or welding, and similarly other parts of the layer depalletizing tool 99 are attached to the frame 20. The frame 20 is not limited to the above, and other members can also be provided to attach the layer depalletizing tool 99 to the robot arm 12 and operably receive other components. Note that the description of the frame 20 is for illustrative purposes only, and in other aspects, the frame may have any suitable configuration and / or the layer depalletizing tool may be connected to the robot arm in any suitable manner.

[0029] Referring also to FIG. 6, each pair of opposing side clamps 22 and 24 are slidably attached to their respective tracks 43 and 45, each of which is respectively fixed under the hollow tubes 42 and 44 for movement along the hollow tubes 42 and 44 via attachment assemblies 48 and 50. Since the attachment assemblies 48 and 50 are very similar, only one attachment of the clamp 22 to the frame 20 will be described in more detail herein. The attachment assembly 48 includes a bracket 52 that is slidably attached under the hollow tube 42 via the track 43 and is attached to the distal ends of the rods 60 of the two actuators 56. The bar 54 fixes and connects the clamp 22 to the bracket 52. The bar 54 is attached to the bracket 52 so as to extend vertically therefrom and is attached to the clamp 22. The clamp 22 is perpendicular to the hollow tube 42 within the plane containing the bar 54.

[0030] The provision of two pneumatic actuators 56 between the hollow tube 42 and the bracket 52 causes their movement, and thus also causes the movement of the clamp 22 along the hollow tube 42. More specifically, the body 58 of each actuator 56 is fixed to the hollow tube 42 on its respective side, and the distal end of the rod 60 of the actuator is fixedly attached to the bracket 52. The clamps 22 and 24 are moved in parallel pairs to grip the pallet layer from its two opposing sides and then from its other sides. Thus, the actuators 56 are initially operated in groups of four (fixed to the same hollow tube 42 or 44), and then the other four are operated simultaneously. According to another aspect of the present disclosure, all of the clamps 22 and 24 are operated simultaneously.

[0031] Providing the dimensions and pose of a pallet layer 816 (representing any one of the pallet layers PL1, PL2, PL3, PL4, PL5) as determined by a vision system 310 as described herein, each pair of clamps 22 and 24 is movable between an extended position and a contracted position, where clamp 22 or 24 applies pressure to the pallet layer 816 from the corresponding opposing side. Clamps 22 and 24 have different widths, although clamps according to another aspect of the present disclosure may have the same width. According to another aspect, the clamps are pivotally attached to the frame. According to yet another aspect, the clamps have a configuration other than those illustrated herein and are attached to the frame in any suitable manner such that they are movable differently with respect to the frame.

[0032] Referring to FIGS. 7, 9A, and 10B, both curtains 26 are defined by a series of metal rolls 62 that are slidably and rotatably mounted between them to the rectangular portion 34 of the frame 20. Both curtains 26 together extend along the entire length of portion 34, each covering half of its length. More specifically, each roll 62 includes a hub portion 63 (see FIG. 10B) that is received in parallel tracks 64 attached to the inner surface of the frame 20 at its longitudinal ends. Each of the four tracks 64 extends along the lower end 66 of the rectangular portion 34 from its center to the end of an arcuate portion 68 of the track 64 that rises toward the triangular portion 38 near the longitudinal end side of the wall 30. The arcuate portion 68 defines a curtain receiving portion when the curtain 26 is open. Note that in some drawings, part of the roll 62 is not shown for simplicity.

[0033] Each curtain 26 includes a rotating head 70 at its leading end, which is slidably mounted in track 64 together with roll 62 and is also rotatably mounted in track 64. Roll 62, together with the head 70 of each curtain 26, is moved along its respective track 64 by an actuator 72. The body 74 of the actuator 72 is fixed to one rectangular portion 34 of the wall 30 on its outer surface, and its rod 76 is fixed and attached to the head 70 via an attachment bracket 78 (FIG. 7B) attached to the outer surface of the rectangular portion 34 so that the rod 76 and the head 70 translate together along track 79. When the curtain 26 opens, the actuator 72 pushes the head 70 along track 64 and advances the roll 62 in the same direction. When the curtain 26 closes, the actuator 72 pulls the head 70 along track 64 in the same direction. The head 70 is in the form of a plurality of rotatable friction elements 80 extending along the leading edge of each curtain 26. The friction elements 80 are in the form of rubber O-rings mounted to rotate infinitely on two parallel rolls 82. The two rolls 82 are rotatably mounted between them to the attachment bracket 78.

[0034] As can be better seen in FIGS. 5 and 7B, the two rolls 82 of each curtain 26 are driven by respective drive assemblies including one side chain 84 and a drive portion 90. The side chain 84 extends along the straight portion of each track 64 and is attached to each drive sprocket 86 and each driven sprocket 87. The driven sprocket 87 is fixed to the end of each respective roll 82. The drive portion 90 is fixed to the wall portion 34 and is operably connected to the two side chains 84 via one of each of the drive sprockets 86. The side chain 84 is positioned with respect to the two rolls 82 via the sprocket 87 such that the roll 82 and the associated chain (or belt) 84 rotate together. Thus, the rotation of each drive portion 90 causes the rotation of the friction element 80. The head 70 can rotate and slide simultaneously and independently along the track 64. End rolls 92 are provided between the two driven sprockets 88 of each curtain 26 to further facilitate the displacement of the head 70 along the track 64 by connecting the side chains 85 disposed at each end of the head 70, thereby ensuring that the head 70 remains perpendicular to the track 64. It should be noted that elements of the drive assembly including the side chain 84 are omitted in some drawings for the sake of simplicity.

[0035] More specifically, referring to FIGS. 4 to 6, the upper pad 28 and its operating mechanism will be described here. The upper pad 28 is in the form of a plate that is movable toward and away from the pallet layer 816 gripped by the clamps 22 to 24. The upper pad 28 is made movable by its attachment to the frame 20 via the pad operating assembly 5100. The upper pad 28 (in the form of a plate) is made of a flexible elastic material such as rubber or plastic, which is reinforced with a metal tube on its non-contact surface. According to other aspects, the upper pad 28 is made of another material and / or is not reinforced. The pad operating assembly 5100 includes four pad holder shafts 102 to 108 rotatably attached to the frame 20, eight pad holder wheels 6110 attached in parallel pairs to each of the shafts 102 to 108, four link elements 112, each of which is fixed between the upper pad 28 and the wheels 6110 of the respective shafts 102 to 108, four shaft pulleys 114 to 120, each one fixed near one end of each shaft 102 to 108, an upper pad linear actuator 122 fixed to the frame 20, a pulley assembly 124 fixedly attached to the frame 20, and four cables 126 to 132, each one operably connecting the respective shaft pulleys 114 to 120 to the movable end of the actuator 122. The four pad holder shafts 102 to 108 are rotatably attached to the hollow tubes 42 to 44 via four support brackets 134. The four shafts 102 to 108 are positioned relative to each other end to end in a square configuration. Each of the link elements 112 is fixed to the respective pair of wheels 6110 for partial winding thereon and is firmly fixed to the upper pad 28 via an attachment 136. The pulley assembly 124 includes a support 138 fixed to the hollow tube 42 so as to be positioned at the movable end of the actuator 122, and three intermediate pulleys 140 to 144 rotatably attached to the support 138 thereunder.As can be better seen in FIG. 5, the position and orientation of the intermediate pulleys 140-144 and the actuator enable the four shaft pulleys 114-120 to be connected to the actuator rod 146 using the cables 126-132.

[0036] During operation, the upper pad 28 causes the actuator 122 to retract its rod 146, thereby simultaneously raising the upper pad 28 by being pulled into the cables 126-132. This rotates the shafts 102-108 simultaneously, creating a pulling force on the link element 112 that raises the upper pad 28. The reverse effect is obtained by extending the actuator rod 146.

[0037] As described above, referring to FIGS. 1, 3, and 11A - 11F, the palletizer cell 10A includes a vision system 310. The vision system 310 includes at least one camera 310C attached to the palletizer cell 10 independently of the robot 14 so as to maximize the throughput of the robot 14 (i.e., while the robot 14 places the previously picked pallet layer on the conveyor 150, the next pallet layer is acquired and analyzed by the vision system 310). The at least one camera 310C is any suitable 3D image sensor (or includes one) configured to generate one or more of a 2D image, a 2D depth map, and a 3D point cloud. In one aspect, the vision system 310 includes one camera 310C, in other aspects, the vision system 310 includes three cameras 310C1, 310C2, 310C2, and in yet other aspects, the vision system 310 includes six cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6. In still other aspects, the vision system 310 includes any suitable number of cameras. When multiple cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 are provided, two or more of the cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 can be arranged at multiple vertical levels 387L1, 387L2 (FIGS. 3 and 11A - 11F). For example, in one aspect, cameras 301C1, 31C2, 301C3 are arranged at a common level 387L1, and cameras 301C4, 301C5, 301C6 are arranged at another common level 387L2 so as to be at a different height (e.g., relative to the pallet load PAL) from cameras 301C1, 31C2, 301C3, whereby the multiple cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 are arranged at different heights / levels to image the pallet load PAL. In other aspects, the multiple cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 can be arranged at a single level or at any suitable number of levels.In other aspects, cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 may also be mounted on a movable platform (independent of robotic arm 14) to raise and / or lower cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 in order to image each pallet layer with a substantially complete 360° coverage of the build structure RPAL of the pallet load (as described herein).

[0038] In the aspects described herein, at least one camera 310C is positioned such that three corners of the pallet load PAL are directly within the field of view FOV1 - FOV6 of at least one camera 310C. For example, in one aspect, cameras 310C1 - 310C3 on level 387L1 are directed at three corners, with no corresponding camera for the fourth corner. In another aspect, cameras 310C1 - 310C3 on level 387L1 are directed at three corners, cameras 310C4 - 310C6 are also directed at three corners, with no corresponding camera for the fourth corner. In yet another aspect, cameras 310C1 - 310C6 may be arranged such that all four corners of the pallet load PAL are directly within the field of view FOV1 - FOV6. In one aspect, at least one camera 310C is arranged such that each respective field of view FOV1 - FOV6 of at least one camera 310C covers the pallet load PAL from the uppermost upward-facing surface 148 (FIG. 12) of the pallet load to the bottom of the pallet load. In one aspect, at least one camera 310C is arranged such that the uppermost upward-facing surface 148 (FIG. 12) of each pallet layer is within each respective field of view FOV1 - FOV6 of at least one camera 310C. As described above, at least one camera 310C may be arranged on a single level, multiple levels, and / or a movable platform such that the uppermost upward-facing surface 148 (FIG. 12) of each pallet layer is within the respective fields of view FOV1 - FOV6 and / or such that each respective field of view FOV1 - FOV6 covers the pallet load PAL from the uppermost upward-facing surface 148 (FIG. 12) of the pallet load to the bottom of the pallet load.

[0039] In addition, each of the cameras 310C1 to 310C6 is arranged at a vertical angle of approximately 45° (or other appropriate vertical angle greater than or less than approximately 45°) and a horizontal angle of approximately 45° (or other appropriate horizontal angle greater than or less than approximately 45°) with respect to the pallet load PAL so as to provide not only the upper surface 148 of the pallet but also all four outermost vertical surfaces (side surfaces) of the pallet load PAL (or at least more than three vertices / corners formed by the intersection of the side surfaces) within the fields of view FOV1 to FOV6 (even in a mode where, for example, only three corners directly enter the fields of view FOV1 to FOV6). In other modes, each camera of at least one camera 310C may be arranged at any appropriate vertical and / or horizontal angle with respect to the pallet load PAL. The cameras are arranged at different heights (for example, different levels 387L1, 387L2) to image different parts of the pallet load at each level of the camera (for example, the lower level 387L2 may image the lower half of the pallet load PAL, and the upper level 387L1 may image the upper half of the pallet load PAL), while in other modes, the entire pallet load PAL may be imaged from top to bottom at a single level of the camera. In one mode, at least one camera 310C (for example, cameras 310C1 to 310C6, etc.) may have any appropriate focal length for a given image intensity. In one mode, each of the respective fields of view FOV1 to FOV6 of at least one camera (for example, cameras 310C1 to 310C6; see FIGS. 11A to 11F illustrating each emphasized field of view with respect to other fields of view) may be a field of view of approximately 45°, while in other modes, each field of view FOV of at least one camera 31C may be greater than or less than approximately 45° as long as three corners of the pallet load PAL directly enter the fields of view FOV1 to FOV6 of the camera. In some modes, the attributes of each of the respective fields of view of at least one camera 310C are the same (for example, each camera has a field of view of approximately 45°), while in other modes, the attributes of the respective fields of view may be different for one or more of at least one camera 310C (for example, one or more cameras may have a field of view different from approximately 45°).

[0040] At least one camera 310C is coupled to and notifies the cell control device 10C so that the cell control device 10C can issue a movement command to the robot arm 12 (or the robot 14 as a whole), for example, using real-time (or near real-time) command input (based on the commanded transaction time as described herein) for the robot arm 12 to respond in real time to differences in pallet loads such that the robot arm 12 adapts to resolve differences in pallet builds that affect depalletizing. For example, at least one camera 310C enables the cell control device 10C to issue commands to the robot arm 12 to move and guide the layer depalletizing tool 99 to the pallet layers PL1, PL2, PL3, PL4, PL5. As described herein, the vision system 310 provides (or otherwise results in the determination of) the position (X, Y, Z in the robot coordinate system or reference frame) and / or orientation (RX, RY, RZ in the robot coordinate system or reference frame) of the upper pallet layer (such as layer 816). In one aspect, the vision system 310 provides (or otherwise results in the determination of) the length L and width W of the upper layer (see FIG. 12).

[0041] The commands issued by the cell control device 10C (based on the image data from at least one camera 310C) are for the tilted pallet layer (e.g., the pallet layer rotated relative to the pallet support SPAL and / or other pallet layers), the offset pallet layer (e.g., the pallet layer having an edge that projects beyond the virtual vertical pallet plane / boundary established by the peripheral edge of the pallet support SPAL), the open case unit CU (e.g., a case unit such as a cardboard box with one or more flaps of the box open), the pallet load / layer outside the tolerance range, the incomplete layer (e.g., the case unit is missing), etc. To adapt to these, for the pallet layers PL1, PL2, PL3, PL4, PL5, it brings about a positional / spatial adjustment of the layer depalletizing tool 99 in multiple degrees of freedom (e.g., not only planar degrees of freedom but also rotational degrees of freedom). The vision system 310 is configured to provide data to the cell control device 10C so as to bring about the guidance and positioning of the layer depalletizing tool 99 to the optimal picking position and orientation for picking the upper layer of the pallet (e.g., compared to other possible picking positions and orientations) based on the actual position and orientation of the upper layer in the coordinate system / reference frame of the robot 14 (in this specification, the terms coordinate system and reference frame are used interchangeably).

[0042] Commands issued by the cell control device 10C (based on image data from at least one camera 31C) also result in obstacle avoidance, for example, if an unexpected object is positioned or enters the depalletizer cell 10A, if the edge or side of the pallet load PAL is at approximately the same height as the layer depalletizing tool 99 or other parts of the robot 14, if the pallet load PAL is adjacent to a robot exclusion zone (e.g., a predefined area within the depalletizer cell 10 where the robot 14 is not permitted to enter), etc. The vision system 310 detects an unexpected object, the side / edge of the pallet load PAL, the distance between the pallet load PAL and the robot exclusion zone, etc., and transmits a data signal to the cell control device 10C so that the cell control device 10C can move the layer depalletizing tool 99 around the unexpected object, around the side of the pallet load, between the pallet load PAL and the robot exclusion zone (or other obstacles adjacent to the pallet load PAL), and / or in any other suitable way for picking the pallet layers PL1, PL2, PL3, PL4, PL5, or in other aspects, is configured to command the robot arm 12 to stop moving.

[0043] As described above, the cell control device 10C (or other suitable control device such as the robot control device 316) receives image data (from a two-dimensional image, a two-dimensional depth map, and / or a three-dimensional point cloud, etc.) from the vision system 310 for analysis of the image data so as to detect pallets, pallet layers, and / or analyze their characteristics. Also as described above, the image data provided to the cell control device 10C is provided in the coordinate system or reference frame of the robot 14, where the vision system 310 is calibrated / registered to the coordinate system or reference frame of the robot 14. Note that each camera 310C1 - 310C6 is essentially calibrated to its own coordinate system (i.e., each camera recognizes the depth of an object from its respective image sensor of the camera). When a plurality of cameras 310C1 - 310C6 are used, in one aspect, the calibration of the vision system 310 includes calibration of the cameras 310C1 - 310C6 to a common base reference frame and calibration of the common base reference frame to the robot's reference frame. However, in other aspects, when one or more cameras are used, the reference frame of one camera, or the reference frames of one or more of the cameras 310C1 - 310C6, may be individually calibrated to the robot's reference frame.

[0044] For illustrative purposes only, referring to FIGS. 3, 13A, and 13B, the calibration of cameras 310C1 to 310C6 to a common base reference frame involves identifying and applying the transformation between the respective reference frames of each of cameras 310C1 to 310C6 such that the respective reference frames of each of cameras 310C1 to 310C6 can be the reference frame of a single camera or can be any other suitable common base reference frame to which each of cameras 310C1 to 310C6 can be associated to form a common base reference frame collectively for all cameras of the vision system. For example, the reference frame for camera 130C1 (although any of the cameras can be used) represents the common base reference frame. The transformation (i.e., a rigid transformation in six degrees of freedom) is determined for each of the coordinate systems / reference frames for the other cameras 310C2, 310C3, 310C4, 310C5, 310C6 with respect to the reference frame of camera 310C1 such that the image data from cameras 310C2, 310C3, 310C4, 310C5, 310C6 is correlated or transformed to the reference frame of camera 130C1. This calibration of cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 can be performed using a calibration jig / fixture 1300 (also referred to as a common camera calibration reference structure) disposed at the pallet loading / unloading station 301. As described herein, each of at least one of cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 is calibrated to the common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents the positional relationship between the respective camera reference frames of each of cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 and the other respective cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 and a predetermined reference frame of the robot 14 (as described herein).

[0045] The calibration fixture 1300 provides an asymmetric pattern to the calibration fixture 1300 to determine the relative pose of the pallet layer to the grip interface, and as further described, determines / transforms the reference frames of the cameras (e.g., from each camera) to a common base reference frame, and constrains the transformation between the common base reference frame and the robot's reference frame, including uniquely identifiable three-dimensional geometric shapes 1310-1319 (in this example, squares, some of which are rotated relative to others). The calibration fixture 1300 shown and described herein is exemplary, and any other suitable calibration fixture may be utilized in a manner similar to the methods described herein. For illustrative purposes, each of the three-dimensional geometric shapes 1310-1319 is of a predetermined size that constrains the identification of the corners or points C1-C36 of the three-dimensional geometric shapes 1310-1319, and the transformation is such that the distances between the corresponding corners C1-C36 are minimized (e.g., the distances between each of the respective corners C1-C36 in the reference frame of camera 310C1 are minimized for each of the respective corners C1-C36 identified in the reference frames of each of the cameras 310C2-310C6).

[0046] Each of the three-dimensional geometric shapes 1310 - 1319 is imaged simultaneously (i.e., the three-dimensional geometric shapes 1310 - 1319 are each in a single location in a common reference frame while being imaged by all cameras whose reference frames are calibrated to a common base reference frame), and the points / corners C1 - C36 of the three-dimensional geometric shapes 1310 - 1319 identified in the images (one exemplary image is illustrated in FIG. 13B) are uniquely identified by each of the cameras 310C1 - 310C6 at a single location such that they are identified by the vision system 310 (by any suitable method such as the method described herein with respect to the determination of the corners PC1 - PC4 of the pallet layer), and are uniquely determined independently of the orientation of the calibration fixture. The uniquely identified corners C1 - C36 are located at a single location in a robot reference frame that is common to all cameras when a set of images (a set of images that is a collection of images taken by each of the cameras with respect to a given location of the calibration fixture or the pallet load) corresponding to each set of images taken by the cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 is taken. The corners C1 - C36 identified in each image of the set of images are compared between the images to define a transformation to a common base reference frame (which, in one example, may correspond to or be defined by the reference frame of camera 310C1) for each camera reference frame. The calibration fixture 1300 is configured to identify points across the entire work volume of the pallet loading / unloading station 301 (corresponding to the dimensions of the pallet load PAL). For example, the three-dimensional geometric shapes 1310 - 1319 may span the X - Y plane of the pallet loading / unloading station 301, and the calibration fixture may be imaged at various heights across the entire work volume of the pallet loading / unloading station 301.

[0047] By registering all cameras 310C1 to 310C6 to a common base reference frame, the common base reference frame (or the reference frames of one or more cameras individually) is transformed (e.g., registered) to the reference frame (X, Y, Z, RX, RY, RZ) of the robot 14 by attaching the calibration fixture 1300 (or a similar fixture) to the robot 14. The calibration fixture can be attached to the robot 14 such that the three-dimensional geometric shapes 1310 to 1319 (and their corners C1 to C36) have a known predetermined spatial relationship (e.g., position, planarity, orientation, etc.) with respect to the gripper engagement interface of the gripper 800 (Fig. 8B) (or other suitable reference datum or position of the robot 14). The robot 14 can be commanded to move in a predetermined motion along one or more of the X, Y, Z axes while being imaged by one or more of the cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6 (thereby keeping the calibration fixture 1300 held). The cell controller 10C can compare the identified corners C1 to C36 in the image obtained by the vision system 310 with, for example, encoder data (describing the motion path of the robot 14, i.e., the motion of the fixture brought about by the robot 14), and generate from the image, with respect to the gripper engagement interface of the gripper 800 (Fig. 8B) (or other suitable reference datum or position of the robot 14), the planar orientation and position (pose and orientation in six degrees of freedom) of the identified corners C1 to C36. The generated planar orientation and position (pose and orientation in six degrees of freedom) of the identified corners C1 to C36 with respect to the gripper engagement interface of the gripper 800 (Fig. 8B) (or other suitable reference datum or position of the robot 14) characterize the relationship between the image field of the common base reference frame and the reference frame of the robot 14 such that the positions of the corners C1 to C36 of the image of the vision system are calibrated to the reference frame of the robot.The above calibration can be performed once during the installation of the palletizing cell 10 or at any suitable time interval (or during each initialization of the robot 14 such as after power-off), enabling the image data of the vision system 310 to be represented in the same coordinate system / reference frame as the robot 14.

[0048] In one aspect, at least one camera 310C resolves the three-dimensional definition of the case unit features (e.g., the edges and corners of the case unit) from two or more orthogonal planes such that the maximum certainty of the feature poses (e.g., the X, Y, Z, RX, RY, RZ positions of the pallet layer or the features of the calibration fixture 1300 - see FIGS. 11A-11F and 12) is obtained from a single image of the item in each field of view FOV1-FOV6 of the at least one camera 310C. Here, the resolution of the three-dimensional definition of the feature(s) of the pallet layer and / or the calibration fixture 1300 is independent of the arrangement of the camera 310C and is performed in real time (e.g., within the picking / placement cycle of at least one robot 14).

[0049] Six cameras 310C1 to 310C6 are described, but at any suitable desired resolution, preferably to capture the three-dimensional time-of-flight images of the object(s) at any location on the build structure RPAL of the pallet load, the fields of view of the cameras 310C1 to 310C6 of the vision system 310 cover the pallet load / unload station 301 of the palletizing cell 10, the pallet support SPAL placed on the pallet load / unload station 301, and the entire (or at least a predetermined part) of the predicted build structure RPAL of the pallet load at the pallet load / unload station 301. It should be understood that more or fewer than six cameras 310C1 to 310C6 may be used and arranged. The combined fields of view FOV1 to FOV6 result in a substantially complete 360° coverage of the build structure RPAL of the pallet load with overlap of the fields of view FOV1 to FOV6. For example, the combined fields of view FOV1 to FOV6 may cover a standard pallet support SPAL (having dimensions such as, for example, 48 inches × 48 inches, 48 inches × 40 inches, and / or 36 inches × 36 inches), but it should be understood that the (one or more) cameras 310C1 to 310C6 and the associated (one or more) fields of view FOV1 to FOV6 may cover (for example, image) a larger field as appropriate (including, for example, a truck bed or any field size). Further, the (one or more) fields of view FOV1 to FOV6 may cover any suitable height PH (see FIG. 2) of the build structure RPAL of the pallet load, such as, for example, heights of 60 inches, 70 inches, and 80 inches, but in other embodiments, the (one or more) fields of view FOV1 to FOV6 may cover a height less than 60 inches or greater than 80 inches.

[0050] In one aspect, each of the (one or more) cameras 310C1 to 310C6 may have a resolution of 176 pixels × 132 pixels. However, in other aspects, each of the (one or more) cameras 310C1 to 310C4, or one or more of them, (such that the definition of the depth map of the entire captured image of the pallet support part / pallet build or a predetermined part of the whole is about 0.5 inches or more) at the outermost boundary of the three-dimensional space of the pallet build, about 0.Optionally, it may have a higher resolution (e.g., a resolution of 320 pixels × 240 pixels or greater) so as to provide a desired minimum depth map defining 5 inches. In this way, sufficient resolution is provided by the vision system 310 to resolve the features of at least the upper surface 148 of the pallet load PAL so that the flatness across the upper surface 148 of the pallet load PAL is determined and fully established to release the pallet layers PL1 - PL5 from the pallet load PAL. Also, sufficient resolution is provided to resolve the case unit features (e.g., the edges of the case, etc.) so that the flatness across the upper part of each of the layers PL1 - PL5 (see FIG. 2) is determined and fully established to release the layers PL1 - PL5. The resolution of the camera(s) 310C1 - 310C6 may be such that minimal processing is required to resolve the case unit features (e.g., the edges of the case unit and the corners of the pallet layers) so that the case unit features are resolved in real time from the image substantially received by the cell controller 10C. For example, the corners PC1 - PC4 of each pallet layer PL1 - PL5 can be determined by any suitable method such as, for example, the Ramer - Douglas - Peucker algorithm (see "An iterative procedure for the polygonal approximation of plane curves" published in Computer Graphics and Image Processing, Volume 1, Issue 3, November 1972, Pages 244 - 256, and "The Contours, Corners and T - Junctions Detection Algorithm" by Buades et al. published in Image Processing Online, February 27, 2018, ISSN 2105 - 1232, (Copyright) 2018), or Suraya Abu Bakar, Muhammad Suzuri Hitam and Wan Nural Jawahir Hj. Wan Yussof, 2017, the entire disclosure of which is incorporated herein by reference.It can be determined by imaging at least the upper surface 148 of each pallet layer in the vision system 310 and determining the positions of the corners PC1 to PC4 of the pallet layers PL1 to PL5 by any other suitable method, such as the methods described in 「Improved Global and Local Curvature Properties for Shape Corner Detection」, Journal of Applied Sciences, 17: 458-466; 「The Comparison and Application of Corner Detection Algorithms」, Jie Chen et al., Journal of Multimedia, Volume 4, No. 6, December 2009; and 「Robust Corner Detection by Image-Based Direct Curvature Field Estimation for Mobile Robot Navigation」, Sungho Kim, International Journal of Advanced Robotic Systems, 2012, Volume 9, 187:2012, DOI: 10.5772 / 53872.

[0051] Referring now to FIGS. 3, 2, 8B, and 12, in one aspect, the cell control device 10C is configured to determine in real time, from corresponding real-time three-dimensional imaging data, the layer planar dispersions PSV1, PSV2 of the pallet layers PL1 to PL5 that are released with respect to the gripper engagement interface 810 (FIG. 8B) of the gripper 800 (FIG. 8). The cell control device 10C is also configured to generate in real time a multi-joint robot motion signal that depends at least on the layer planar dispersions PSV1, PSV2 determined in real time, where the multi-joint robot motion signal enables at least one multi-joint robot 14 to perform, in real time, between the arrangement of one released pallet layer PL1 to PL5 and the arrangement of the successively released pallet layers PL1 to PL5 that enables substantially continuous release of the pallet load PAL. In one aspect, at least one multi-joint robot motion signal generated by the cell control device 10C is a stop motion signal along the picking / placement paths 399, 1580 of at least one multi-joint robot 14, a low-speed motion signal along the picking / placement paths 399, 1580 of at least one multi-joint robot 14, or a movement to a safe position along the safety stop path 398 of at least one multi-joint robot 14, where the safety stop path 398 is different from the picking / placement paths 399, 1580. In one aspect, the multi-joint robot motion signal generated by the cell control device 10C is a picking position signal that sets the picking position of the layer depalletizing tool 99 based on the layer planar dispersions PSV1, PSV2 of the released pallet layers PL1 to PL5.

[0052] The cell control device 10C is configured to determine in real time the layer planarity dispersions PSV1, PSV2 and positions of the pallet layers PL1 to PL5, and the released layer pose PSV3 (RZ) and position (X, Y) from the corresponding real-time three-dimensional imaging data. Here, for example, the vision system 310 images the upper surfaces 148 of the layers PL1 to PL5 in order to obtain a three-dimensional image of the upper layer of the pallet load PAL with sufficient fineness to identify the sides, corners, and planarity of the upper surface 148 of the upper pallet layer as described above. Here, the dispersions PSV1, PSV2 of the pallet support may be one or more of a case unit CU with non-uniform spacing (e.g., the space between case units CU in a pallet layer that forms ridges / valleys on the sheet surface of the case unit - Fig. 9A), a missing case unit CU in the pallet layer, a height difference (e.g., protrusions and / or depressions - Fig. 9A), or any other defect in the pallet layer that may affect the gripping of the pallet layer by the layer depalletizing tool 99. In one aspect, the cell control device 10C is configured to reject the picking of the pallet layer (and transmit a stop bot signal until it is replaced) when the dispersions PSV1, PSV2 of the pallet support exceed a threshold from a predetermined reference such as a plane defined by the upper pad 28. For example, if the missing case unit CU in the pallet layer is larger than a predetermined area, or if the spacing between case units CU in the pallet layer is larger than a predetermined distance, the picking of the pallet layer is rejected and the pallet layer is not picked until the defect in the pallet layer is resolved (e.g., by manual intervention). When the pallet layer is within a predetermined threshold, the cell control device 10C resolves the planar dispersion of the pallet layer (e.g., the position of the pallet layer in the three-dimensional robot space X, Y, Z, RX, RY, RZ), and based on the above-described dispersion with respect to the adapted pose of the layer depalletizing tool 99 with a higher picking probability of the resulting pallet layer, is configured to confirm or correct (compensate) the planned robot picking / placement path.The control device can also identify a decrease in the movement speed of the robot 14, or modify the placement paths 399, 1580 of the robot 14 and their respective path trajectories (FIG. 3) to generate the picking pose (e.g., the position in the three-dimensional space X, Y, Z, RX, RY, RZ) of the desired layer depalletizing tool 99.

[0053] In one aspect, the cell control device 10C is configured to set the pallet layer data map DTM (FIG. 12) of the pallet layer imaged by at least one three-dimensional camera 310C from the distributed PSVs 1, 2 of the pallet support. The pallet layer data map DTM resolves the local surface dispersion at the placement positions of each different article unit in the pallet layer and defines a real-time position-based reference for the picking of the articulated robot 14 of the pallet layer. In one aspect, the pallet layer data map DTM defines the flatness of the upper surface 148 of the pallet layer.

[0054] Referring to FIGS. 3, 12, and 14, an exemplary operation of the depalletizer cell 10 in the depalletizing configuration is described. Note that referring to FIG. 14, the operation blocks described therein do not necessarily define a specific operation order, and the operations identified by the operation blocks can be performed in any suitable order. For example, blocks 1401 and 1410 can be performed at any time before determining the picking path / trajectory of the robot 14.

[0055] In the operation of the palletizer cell 10, the pallet load PAL is conveyed and positioned onto the unloading / loading station 301 in any suitable manner (such as the above). In an exemplary operation of the palletizer cell 10, the field of the image sensor or camera 310C is optionally registered in the reference frame (X, Y, X, RX, RY, RZ) of the robot as described herein at any suitable time before and / or during the operation of the palletizer cell 10 (FIG. 14, block 1401). The pallet load PAL is imaged by at least one camera 310C such that an image of the upper surface 148 of the upper pallet layer 816 is captured (FIG. 14, block 1405). The image data from each of the at least one camera 310C is converted from the respective camera reference frame to the reference frame of the robot 14 in a method such as described herein (FIG. 14, block 1410). Note that, in one aspect, the registration of the field of view of the camera 310C is optionally performed as part of the operation before the conversion of the camera image data to the reference frame of the robot 14. Also, the conversion of the camera image data to the reference frame of the robot 14 may be performed at any suitable time before the determination of the picking trajectory of the robot 14, such as the exemplary time described in FIG. 14. The reference datum(s) of the pallet layer is determined based on the image data in the reference frame of the camera 310C and / or the reference frame of the robot reference 14 (FIG. 14, block 1415). The reference datum(s) of the pallet is any suitable geometric feature of the pallet that identifies or otherwise defines the pose and position of the pallet in the reference frame of the camera 310C and / or the reference frame of the robot 14 (e.g., the corners of the pallet layer, the corners of the case units in the pallet layer, the outermost surface of the pallet layer, the vertices of the outermost surface, the orthogonality of the outermost surface, the position of the side surface, etc.).For example, in one aspect, the (one or more) reference datums are the corners PC1 - PC4 of the pallet layer 816, where the corners PC1 - PC4 are determined in one or more of the reference frames of the camera 310C and the reference frame of the robot 14 by any suitable method (such as the above method using any suitable image analysis corner discovery algorithm) based on the image data obtained by at least one camera 310C. In one aspect, the corners PC1 - PC4 of the pallet layer are determined separately from the image data of each of the at least one camera 310C, but in other aspects, the image data from the cameras can be optionally combined to determine the corners of the pallet layer (FIG. 14, block 1420). For example, when the image data from at least one camera 310C is combined, at least a partial single point cloud 1270 of the pallet load PAL including the pallet layer 816 is generated by combining the image data from each of the at least one camera 310C using the cell control device 10C.

[0056] The cell control device 10C is configured to conform the plane 1200 to the (one or more) reference datums of the pallet layer in any suitable manner based on image data from one or more of at least one camera 310C, using, for example, the Random Sample Consensus (RANSAC) algorithm, an organized segmentation algorithm (or any other suitable algorithm for the segmentation of organized point cloud data), or any other suitable algorithm (block 1425 in FIG. 14). In one aspect, the plane 1200 corresponds to and defines the upper surface 148 of the pallet layer 816 in the reference frame of the camera 310C and / or the reference frame of the robot 14. In one aspect, the positions of the corners PC1 to PC4 of the pallet layer 816 in the reference frame of the robot can optionally be verified by determining the positions of the corners PC1 to PC4 by projecting a single point cloud 1270 onto the plane 1200 using the cell control device 10C (block 1430 in FIG. 14). Using the cell control device 10C, the pose PSV3 and size (length L and width W) of the pallet layer 816 are determined from image data from one or more of at least one camera 310C in any suitable manner, such as by any suitable blob analysis technique (center of gravity, principal axis, minimum ferret, etc.) (block 1435 in FIG. 14).

[0057] (One or more) reference datums (in this example, corners PC1 to PC4 and thus lengths L and width W) are accurately known based on the above operations, and a plane 1200 is established to represent the upper surface 148 of the pallet layer 816. Thus, the cell control device 10C verifies the flatness of the upper surface 148 of the pallet layer 816 with respect to a plane 28P (e.g., defined by the upper pad 28) of the grip engagement interface 810 (FIG. 14, block 1440). Verification of the flatness of the upper surface 148 can be performed by the cell control device 10C using any suitable image analysis thresholding technique to determine whether the layer depalletizing tool 99 can pick up the pallet layer 816. For example, a deviation in the flatness of the plane 1200 with respect to the plane 28P exceeding a predetermined threshold (e.g., in one aspect, the deviation can be up to approximately 5° or up to approximately 10° around the X-axis and / or Y-axis, while in other aspects, it can be greater than approximately 10°) prevents the layer depalletizing tool 99 from picking up the pallet layer 816. If the pallet layer 816 cannot be picked up by the layer depalletizing tool 99, the cell control device 10C is configured to issue any suitable audible and / or visual alerts to the operator to resolve the picking of the pallet layer.

[0058] Referring also to FIGS. 15A - 15C, the cell control device 10C is configured to determine the movement trajectories and paths of the robot 14 to provide or otherwise determine the optimal picking position of the layer depalletizing tool 99 with respect to the pallet layer 816 for picking / releasing the pallet layer 816 from the pallet load PAL in any suitable manner (FIG. 14, block 1445). The optimal picking position can be determined by the cell control device 10C during the placement cycle of the previously picked pallet layer so as to effect substantially continuous release of the pallet load PAL. The optimal picking position is the position of the layer depalletizing tool 99 with respect to the pallet layer 816, and the distance between the center 99C of the layer depalletizing tool 99 and the center 816C of the pallet layer 816 is minimized while satisfying the physical constraints of the layer depalletizing tool 99 and the palletizing cell 10. For illustrative purposes only, the physical constraints of the layer depalletizing tool 99 include, but are not limited to, for example, the grip area 1520 of the grip engagement interface defined by the gripper length LT and the gripper width WT in the open configuration of the grip 800 (with the pallet layer 816 inserted), the planarity deviation between the side clamps 22 - 24 and the (vertical) side surface of the pallet layer (this deviation constraint may be similar to the planarity deviation of the plane 1200 with respect to the plane 28P), the planarity deviation between the plane 1200 of the upper surface 148 and the plane 28P of the upper pad 28, the picking of the pallet layer 816 such that the layer depalletizing tool 99 does not contact the pallet load PAL (other than gripping the pallet layer 816), etc. Also, for illustrative purposes, the physical constraints of the palletizing cell 10 include, but are not limited to, the exclusion zones 1500 - 1502 that impede the movement of the robot 14 (i.e., the robot 14 and the layer depalletizing tool 99 carried by the robot 14 are excluded from entering the exclusion zones 1500 - 1502).

[0059] For illustrative purposes only, the optimal picking position can be determined by the cell control device 10C, where the cell control device 10C finds the pallet layer center 816C based on the pallet layer characteristics determined above (e.g., the positions of corners PC1 to PC4, length L, width W, etc.). The cell control device 10C avoids the exclusion zones 1500 to 1502 and uses the known dimensions of the layer depalletizing tool 99 (such as the grip length LT and grip width WT in addition to the outer dimensions) to determine the position of the layer depalletizing tool 99 that minimizes the distance between the center 99C and the center 816C while satisfying the physical constraints of the layer depalletizing tool 99. The cell control device 10C is configured to determine the feasibility of the optimal picking position for the constraints mentioned herein (Figure 14, block 1450).

[0060] Figure 15A is an exemplary diagram having a uniform case distribution such that the pallet layer 816 forms a complete pallet layer. In other embodiments, the pallet layer 816 may have any suitable non-uniform or incomplete case unit distribution such as those illustrated in Figures 16A to 16C. The pallet layer is centered with respect to the center 301CN of the pallet loading / unloading station 301 such that the center 99C is at the optimal picking position of the layer depalletizing tool 99 that coincides with the center 816C of the pallet layer 816. Here, in addition to there being space between the layer depalletizing tool 99 and the exclusion zones 1500 to 1502, there is also space between the grip 800 and the pallet layer 816 such that the pallet layer 816 can be inserted into the grip region 1520 (e.g., it is confirmed that this picking is achievable by the cell control device 10C).

[0061] FIG. 15B is an exemplary diagram having a uniform case distribution such that the pallet layer 816 forms a complete pallet layer. However, in other embodiments, the pallet layer 816 may have any suitable non-uniform or incomplete case unit distribution, such as those illustrated in FIGS. 16A-16C. In this example, the center 816C of the pallet layer 816 is positioned offset from the center 301CN of the pallet loading / unloading station 301. In the example illustrated in FIG. 15B, there is space between the pallet layer 816 and the exclusion zones 1500, 1501, 1502 into which the layer depalletizing tool 99 can be inserted, and there is also space between the pallet layer 816 and the gripper 800 such that the pallet layer 816 can be inserted into the gripper region 1520 (for example, it is confirmed that this picking can be realized by the cell controller 10C).

[0062] FIG. 15C is an exemplary diagram having a uniform case distribution such that the pallet layer 816 forms a complete pallet layer. However, in other embodiments, the pallet layer 816 may have any suitable non-uniform or incomplete case unit distribution, such as those illustrated in FIGS. 16A-16C. In this example, the center 816C of the pallet layer 816 is positioned offset from the center 301CN of the pallet loading / unloading station 301. In the example illustrated in FIG. 15C, there is space between the pallet layer 816 and the exclusion zones 1500, 1501, 1502 into which the layer depalletizing tool 99 can be inserted. However, FIG. 15C illustrates an extreme example of the arrangement of the pallet layer 816 in the pallet loading / unloading station 301 with a minimum clearance between the two sides of the gripper 800 (along the Z-axis) and the pallet layer 816. Again, the optimal picking position of the layer depalletizing tool 99 in the example illustrated in FIG. 15C is such that the centers 99C, 816C are offset, but the distance between the centers 99C, 816C is minimized considering the space constraints between the pallet layer 816 and each of the exclusion zones 1500, 1501.

[0063] FIG. 15D is an exemplary diagram having a uniform case distribution such that the pallet layer 816 forms a complete pallet layer. In other embodiments, however, the pallet layer 816 may have any suitable non-uniform or incomplete case unit distribution, such as those illustrated in FIGS. 16A-16C. The center 816C of the pallet layer 816 is positioned offset from the center 301CN of the pallet loading / unloading station 301. In the example illustrated in FIG. 15D, there is space between the pallet layer 816 and the exclusion zones 1500, 1501, 1502, but the gripper 800 cannot be positioned to place the pallet layer 816 within the gripper region 1520 without entering the exclusion zone 1500 (e.g., it is confirmed that this picking is not achievable by the cell controller 10C). In the example illustrated in FIG. 15D, there is no optimal picking position for the layer depalletizing tool 99, the picking of the pallet layer is aborted, and the cell controller 10C provides an auditory and / or visual alert to the operator to resolve the layer picking.

[0064] In an aspect of the present disclosure, the cell control device 10C is configured to perform feature analysis (Figure 14, block 1450) of the palletizing cell 10 and the objects located therein (using any suitable image analysis technique). For example, in one aspect, the vision system 310 is configured to identify the pallet support SPAL at the bottom of the pallet load PAL. In some examples of defects in the pallet support, another pallet support is placed under the pallet load PAL (i.e., the pallet load PAL includes two or more pallet supports stacked vertically) so that the pallet load can be operated, for example, by a forklift. The vision system 310 is configured to identify the stacked pallet supports SPAL so that other pallet supports (the lowermost pallet support) are not picked by the robot 14. In another aspect, the vision system is configured to determine whether a partial pallet layer (where the case units of the layer do not cover the entire pallet area (length L and width W)) provides sufficient support to be picked by the layer depalletizing tool 99. For example, referring to FIGS. 16A-16C, exemplary partial pallet layers 1601, 1602, 1603 (which may replace the pallet layers 816 illustrated in FIGS. 15A-15D and are typical in some aspects) are illustrated, which may be suitable for picking by the layer depalletizing tool 99 such that the gripping force applied by the clamps 22-24 is applied substantially uniformly along each of the clamps 22-24. A pallet layer not suitable for picking includes a pallet layer that causes a moment on the clamps 22-24 in a non-uniform manner such that some case units CUs are gripped and other case units CUs are not gripped. In yet another aspect, the vision system 310 is configured to identify a case unit CUF that has fallen from the pallet load PAL to a position (such as on the floor, on the conveyor 150, etc.) in the palletizing cell 10 so as to generate an audible and / or visual alarm for operator intervention.The vision system 310 can also be configured to determine a "retry" trajectory for the robotic arm 14 where a picking failure has occurred. For example, if the robotic arm fails to pick (i.e., the intended picking does not occur), the pallet layer re-identifies the features of the pallet layer and can be re-analyzed by the vision system 310 and / or the cell controller 10C in the above-described manner to generate a trajectory for the robotic arm 12 to retry the picking, where the retry trajectory can be different from the initially generated trajectory.

[0065] Referring to FIGS. 3, 8A - 10B, 15A - 15D, and 17, the operation of the layer depalletizing tool 99 for releasing a layer is described in accordance with aspects of the present disclosure. In the operation, any suitable controller (such as the robot controller 316 and / or the cell controller 10C) generates (FIG. 17, block 17100) a robot space map 1599 (see FIGS. 15A - 15D) that includes at least the pallet loading / unloading station 301 and their structures (exclusion zones). In one aspect, the robot space map 1599 is generated as part of the release operation and / or the robot space map 1599 can be generated prior to the release operation. The robot space map 1599 can be generated in any suitable manner using data obtained from the vision system 310 and / or data from a computer-aided design (CAD) model of the palletizer cell 10. The robot space map 1599 of FIGS. 15A - 15D is illustrated two-dimensionally for illustrative purposes, but it should be understood that the robot space map 1599 can have any suitable configuration such as a two-dimensional depth map, a three-dimensional map including one or more point clouds representing structures / exclusion zones, an image, and / or a three-dimensional model generated from CAD data.

[0066] The pallet load PAL is delivered to and placed at the pallet loading / unloading station 301 of the palletizer cell 10 (Figure 17, block 17200). The pallet load PAL is imaged by the vision system 310 in the manner described herein (Figure 17, block 17201). The robot controller 316 and / or the cell controller 10C integrate the pallet image data into the robot space map 1599 (see Figures 15A - 15D) (Figure 17, block 17101) such that at least a typical of the pallet layers 816 is generated within the robot space map 1599 to provide identification of the position, pose, etc. of the upper pallet layer 816. Although only the pallet layer 816 is illustrated in Figures 15A - 15D, it should be understood that the image data representing the entire pallet can be repeatedly integrated into the robot space map 1599 as each layer is released to provide identification of the position, pose, etc. of subsequent upper pallet layers. The robot controller 316 and / or the cell controller 10C are configured to dynamically define the picking boundary conditions and constraints of the pallet layer based on the position / pose of the pallet layer 816 within the robot space map 1599 (Figure 17, block 17102). For example, based on the image data from the vision system 310, the robot controller 316 and / or the cell controller 10C are configured to dynamically determine the position of the objects within the palletizer cell 10 in any suitable manner (such as by means of a suitable image recognition algorithm, etc.), and configure / reconfigure the robot space map 1599 based on the determined position of the objects (see, for illustrative purposes only, the reconfiguration of the robot space map 1599 based on the redefinition of the region 1570 where the intrusion of the object 1571 and the movement of the robot 14 are permitted). The objects can be temporary objects that intrude into and / or exit the palletizer cell 10, a part of the palletizer cell 10 structure, the pallet load PAL, etc. (in some aspects, the temporary objects are unanticipated / unpermitted objects, in which case the controller can issue a command to stop the movement of the robot 14).The boundary conditions and constraints for the movement of the robot 14 are dynamically determined by the robot controller 316 and / or the cell controller 10C based on the positions of the objects within the palletizing cell 10. Examples of boundary conditions include the areas 1570 of the palletizing cell 10 that permit or restrict the movement of the robot 14, while examples of constraints include the above, the types of objects within the palletizing cell 10, the exclusion zones 1500 to 1502, the position / pose of the pallet load structure PAL, the acceleration of the movement of the robot 14 based on the contents of the pallet layers, and the like.

[0067] When the robot control device 316 receives a signal indicating that, for example, the pallet layer 816 is ready to be picked from the cell control device 10C (Figure 17, block 17202), the robot control device 316 and / or the cell control device 10C determines the path 1580 and the trajectory 1581 of the robot 14 for moving the layer depalletizing tool 99 from the initial position (such as the arranged position of the pre-picker pallet layer or any other appropriate position) to the picking position (such as the positions shown in Figures 15A to 15D) for picking the upper pallet layer 816 (Figure 17, block 17103). For example, the robot control device 316 and / or the cell control device 10C generates the path 1580 and the trajectory 1581 of the robot in any appropriate manner based on the robot space map 1599 (including the image data of the pallet layer 816), the determined boundary conditions, and the determined constraints. The determination of the path 1580 and the trajectory 1581 of the robot 14 can be an iterative procedure optimized (such as time-optimized) so that the path 1580 and the trajectory of the robot 14 result in an optimized movement of the robot 14 from the initial position of the layer depalletizing tool 99 (as described herein) to the optimal picking position, independent of the initial pose of the robot 14. The robot control device 316 and / or the cell control device 10C commands the robot 14 to position the layer depalletizing tool 99 at the optimal picking position based on the optimized path 1580 and the trajectory 1581 such that the clamps 22 and 24 surround the pallet layer 816 (Figure 17, block 17204) (see Figures 8A and 8B).

[0068] Note that, for most layer depalletizing tools, systems, and methods from the prior art, the positioning of the tool relies only on the nominal vertical position of the layer.

[0069] However, in the material handling industry, it is known that product 18 is often crushed within pallet load PAL by the weight of pallet layer 816 or the layers located thereon. Of course, this is especially true when the last layer directly placed on pallet support SPAL (see FIG. 2) is depalletized. Thus, the nominal position of each pallet layer 816 used in the initial programming of robot 14 can easily lead to incorrect positioning of layer depalletizing tool 99, limiting the ability of the system to properly depalletize pallet layer 816.

[0070] In an aspect of the present disclosure, upper pad 28 of layer depalletizing tool 99 is coupled to sensor 888 configured to evaluate the actual height / position of upper surface 148 of the pallet layer 816 being picked, thus enabling layer depalletizing tool 99 to be positioned more accurately. This added precision makes the depalletizing system including robot 14 and layer depalletizing tool 99 more efficient, reducing the likelihood of product damage or avoiding cases where improper positioning prevents the depalletizing of product 18. The sensor can take the form of an analog laser distance sensor or any other suitable distance determination sensor.

[0071] The upper pad 28 is lowered and positioned over the upper layer on the pallet (Figure 17, block 17206). The robot 14 positions the layer depalletizing tool 99 at the height of the pallet layer 816 to be picked, as determined from the image data of the vision system 310. When it is recognized that the upper pad 28 contacts the upper surface 148 of the pallet layer 816, the sensor 888 measures the position of the upper pad 28. Thereafter, the robot controller 316 (and / or the cell controller 10C) calculates the actual height of the pallet layer 816 and compares the actual height with the height of the layer 816 determined from the image data of the vision system 310. If there is a difference between the actual height (determined by the sensor 888) and the height of the layer 816 determined from the image data of the vision system 310, the robot controller 316 adjusts the position of the layer depalletizing tool 99 accordingly. The height difference can also be communicated to the cell controller 10C to re-calibrate / fine-tune the vision system 310 to reduce or minimize the difference between the actual height (determined by the sensor 888) and the height of the layer 816 determined from the image data of the vision system 310.

[0072] Using the upper pad 28 to determine the actual height of the upper surface of the pallet represents a robust method for verifying the position of the upper surface 148 of the pallet layer 816. For example, even if the flap of the product 18 is lifted or the product 18 is not properly positioned, the overall position of the upper pad 28 is not affected, and thus valuable and accurate information about the actual height of the pallet layer 816 is provided. If necessary, the robot 14 adjusts the height of the layer depalletizing tool 99 (Figure 17, block 17210).

[0073] Considering that each pallet layer 816 can be composed of a plurality of products 18, it often occurs that some products 18 do not have side faces facing the outside of the pallet layer 816. According to one aspect, a combination of mechanisms is provided that enables the products 18 to be appropriately gripped by the clamps 22 - 24. For example, the four clamps 22 - 24 are used to compress the entire pallet layer 816 on each side without horizontally moving the pallet layer 816 by operating and controlling the pneumatic actuator 56 (FIG. 17, block 17212). Then, the robot 14 slightly lifts the layer depalletizing tool 99 to create a gap mainly between the lower surface 8150 of the product 18 located around the pallet layer 816 (FIG. 8B) and the upper surface of the product 18 of the pallet layer 816 immediately below (FIG. 17, block 17214). Then, the closing of the horizontal curtain 26 is initiated and they are inserted under the pallet layer 816 (FIG. 17, block 17216). The closing of the horizontal curtain 26 is illustrated in FIGS. 9A - 10B. The rotating friction head 70 of the curtain 26 contributes to moving the product 18 from the gripped pallet layer 816 onto the horizontal curtain 26. More specifically, this enables products 18 that are not located around the pallet layer 816 and are generally only partially lifted or sometimes not lifted at all by the side clamps 22 - 24 to move over the horizontal curtain 26.

[0074] The friction generated on the vertical surface of the product 18 by the pressure of the clamps 22 - 24 sometimes prevents the product 18 from moving upward to enable the horizontal curtain 26 to be closed below without damaging the product 18. When this occurs, the pressure is automatically reduced on the side clamps 22 - 24, so the above-mentioned friction is also reduced, thus facilitating the movement of the curtain to lift and pick the remaining products 18 of the pallet layer 816 to be picked.

[0075] The pressure variations of clamps 22-24 are adjusted by monitoring the linear movement of the horizontal curtain 26 (FIG. 17, block 17218). If product 18 obstructs this movement, the horizontal curtain 26 cannot move forward. When this occurs, it is concluded that at least one product 18 is inhibiting the movement. Thus, the pressure is decreased on clamps 22-24 by actuator 56 (FIG. 17, block 17220), and the process of picking all products 18 on pallet layer 816 continues.

[0076] According to another aspect, the linear movement of curtain 26 is monitored to detect its movement resistance even while the curtain is continuing to move.

[0077] When curtain 26 is fully closed (see FIGS. 10A-10B), pallet layer 816 is fully grasped by layer depalletizing tool 99 (FIG. 17, block 17222), and robot 14 moves layer depalletizing tool 99 together with the grasped pallet layer 816 and transfers pallet layer 816 to an outfeed position (such as conveyor 150) (FIG. 17, block 17232). Vision system 310 images pallet load PAL during the placement cycle of robot 14 to determine the pose, position, etc. of the next layer to be picked (FIG. 17, block 17233). Robot 14 places pallet layer 816 on outfeed conveyor 150 (or other suitable outfeed position) (FIG. 17, block 17234), and the pallet release process continues at block 17202.

[0078] In addition to the above-described function that enables accurate evaluation of the position of the upper pallet layer 816 of the pallet load PAL, the upper pad 28 also prevents small products 18 from "popping out" from the pallet layer 816 when the horizontal curtain 26 moves downward. Such movement of the product 18 would occur, for example, when the pallet layer 816 is composed of small products 18, i.e., products 18 with a reduced height. The weight of the upper pad 28 is sufficient to prevent the "popping out" effect of small products without restricting the ability of the horizontal curtain 26 to enter downward. Similarly, the upper pad 28 prevents small products 18 from "popping out" when the horizontal curtain 26 is removed from under the product 18.

[0079] In some aspects, a slip sheet 277 (FIG. 2) is provided between some of the layers of the products 18 within the pallet load PAL. The standard approach is to have an independent device that automatically removes the slip sheet 277 between each layer's depalletizing sequence. This approach works well, but it is quite costly considering the addition of an independent device. According to aspects of the present disclosure, the layer depalletizing tool is provided with components that remove the slip sheet 277 disposed on the pallet layer 816 of the product 18 simultaneously when the pallet layer 816 is depalletized. Such a layer depalletizing tool is similar to the layer depalletizing tool 99, and since this further release process is similar to the above, for the purpose of brevity, only the differences between them will be described with reference to FIG. 17.

[0080] The tool includes an upper pad 28 that includes a suction cup (not shown) for removing the slip sheet on the pallet layer 816 simultaneously when the pallet layer 816 is depalletized. When the pallet layer 816 is fully grasped by the layer depalletizing tool (Figure 17, block 17222), the suction cup is actuated (Figure 17, block 17224). Another, for example, slip sheet sensor 999 (such as a camera or scanner configured for resolution of the case edge of the intermediate case as shown particularly in Figures 9A and 9B, especially Figure 9B), or the same one mentioned above, that faces upwardly toward the grasped pallet layer 816 that is grasped and lifted by the tool (see also Figure 9B) is used to determine whether there is a slip sheet 277, for example, attached to the upper pad 28. When the robot 14 lifts and transfers the pallet layer 816, the upper pad 28 is slightly lifted (Figure 17, block 17226). If still the slip sheet sensor 999 detects the presence of an object (Figure 17, block 17228), this means that there is a slip sheet 277 under the depalletized pallet layer 816 (Figure 17, block 17229) and a vacuum is maintained on the suction cup (Figure 17, block 17230). Referring further to Figures 19A, 19B, and 19C, the robot 17 then places the pallet layer 816 (having the slip sheet 277 under the pallet layer) on a mat upper conveyor (such as the outfeed conveyor 150), and after the pallet layer 816 is placed on the outfeed conveyor 150 (Figure 17, block 17232) or placed elsewhere, the slip sheet is removed and discarded into a bin (Figure 17, block 17236). Conversely, if the slip sheet sensor 999 detects nothing, this means that there is no slip sheet under the depalletized layer. In such a case, when the previous pallet layer 816 is placed on the outfeed conveyor 150, the vacuum is removed from the suction cup and the robot 14 moves directly back to pick the next pallet layer 816 on the pallet.

[0081] Still referring to FIGS. 19A, 19B, and 19C, a slip sheet remover 998 is positioned or otherwise disposed between portions of the conveyor 150. For example, the conveyor 150 includes a mat upper or upstream portion 993 (where the layer being de-palletized is disposed) and a case spreading or downstream portion 994. The slip sheet remover 998 includes any suitable vacuum removal mechanism such as a vacuum roller 997 (however, any suitable adhesion or suction type roller may be used), a restriction plate 996, and a deflection shield 995. The vacuum roller 997 has any suitable configuration for gripping the slip sheet 227, such as, for example, the vacuum roller rotates about a rotation axis 989 and is fluidly connected to suction cups or ports 997P arranged on or within the outer surface (e.g., the product support surface) of the vacuum roller 997, and includes a suction tube 997T arranged to apply a suction force to the slip sheet 227 as the slip sheet 227 passes over the vacuum roller 997. The restriction plate 996 is disposed under the vacuum roller 997 to peel or otherwise remove the slip sheet and guide it from the vacuum roller 997 to the bin 990. The deflection shield 995 is disposed downstream from the vacuum roller 997 (e.g., with respect to the direction of movement 988 of the pallet layer 816 along the conveyor 150) and has any suitable size and shape to prevent the intrusion of the case edge between conveyors (e.g., so that the case smoothly transitions from one conveyor portion to another without getting caught or jammed while passing through the slip sheet remover 998) and direct at least partially the slip sheet 227 (removed from the bottom of the pallet layer 816) into the gap between the conveyor portions 993, 994 (however, in one or more aspects, the vacuum roller 997 may provide sufficient suction force to independently guide the slip sheet within and through the gap between the conveyor portions 993, 994). As described above, the slip sheet 227 is detected by a slip sheet sensor 999, which transmits any suitable signal to the control device 10C to activate the slip sheet remover 998.The operation of the skip sheet remover 998 includes automatically operating the vacuum roller 997 to automatically remove the slip sheet from under the pallet layer 816. Here, when the pallet layer 816 transfers from the conveyor portion 993 to the conveyor portion 994, the slip sheet 227 is prevented from adhering to the bottom of the pallet layer 816 (via the vacuum roller 997). The vacuum roller 997 grips the slip sheet 277 when the slip sheet 227 moves over the vacuum roller 997. Here, the vacuum roller transfers the slip sheet 997 through the gap between the conveyor portions to the restriction plate 996. Here, the slip sheet 227 is removed from the vacuum roller 997 and discarded into the bin 990.

[0082] Referring to FIGS. 3, 8A-10B, 15A-15D, and 18, a method for depalletizing cases in a depalletizer 10 is provided. The method includes receiving, at a pallet unloading station 301 of the depalletizer 10, a pallet load PAL of cases CU disposed in a pallet load layer 816 (typical of pallet layers PL1-PL5), where each of the pallet load layers 816 is formed of a plurality of cases CU juxtaposed at a common level across the region of the pallet load PAL (block 1800 in FIG. 18). A robot 14 is provided (block 1810 in FIG. 18), and the robot 14 is configured to grip and pick at least one of the pallet load layers 816 to convey at least one of the pallet load layers 816 from the pallet load PAL at the pallet unloading station 301 to an output station 333 (including, in one aspect, any suitable conveyor 150) and has a depalletizing end effector 99 having a grip 800 configured to grip the at least one of the pallet load layers 816. The grip 800 has a grip engagement interface 810 that defines a predetermined layer engagement position and orientation for at least one of the pallet load layers 816 (e.g., in a robot coordinate system or space X, Y, Z, RX, Ry, RZ - see FIG. 3, also referred to herein as the robot's reference frame) relative to the depalletizing end effector 99 such that repeated capture and stable retention of at least one of the pallet load layers 816 using the grip 800 is effected. A vision system 310 images the pallet load PAL of cases CU at the pallet unloading station 301 and generates at least one image of the upper portion of at least one of the pallet load layers 816 independent of the movement of the robot (block 1820 in FIG. 18).A control device (such as the robot control device 16 and / or the cell control device 10C) operably connected to the vision system 310 receives at least one image from the vision system 310 and, based on the at least one image, determines at least one layer position and orientation of the pallet load layer 816 relative to a predetermined layer engagement position and orientation of the gripper engagement interface 810 (FIG. 18, block 1830). The control device is operably connected to the robot 14 to position the gripper 800 and capture and hold at least one of the pallet load layers 816 using the gripper 800 at the gripper engagement interface 810.

[0083] According to one or more aspects of the present disclosure, a depalletizer

[0084] is a pallet load unloading station configured to receive a pallet load of cases arranged in a pallet load layer, each of the pallet load layers being formed of a plurality of cases juxtaposed at a common level across the area of the pallet load.

[0085] A robot having a depalletizing end effector with a gripper configured to grip and pick at least one of the pallet load layers to convey at least one of the pallet load layers from the pallet load at the pallet load unloading station to an output station, the gripper having a gripper engagement interface defining a predetermined layer engagement position and orientation for at least one of the pallet load layers relative to the depalletizing end effector such that repeated capture and stable holding of at least one of the pallet load layers using the gripper results.

[0086] A vision system arranged to image the pallet load of cases at the pallet load unloading station and configured to generate at least one image of at least an upper portion of at least one of the pallet load layers independently of the movement of the robot.

[0087] A control device operably coupled to a vision system to receive at least one image from the vision system and configured to determine at least one layer position and orientation of a pallet load layer relative to a predetermined layer engagement position and orientation of a gripper engagement interface based on the at least one image, the control device being operably coupled to a robot to position the gripper and capture and hold at least one of the pallet load layers using the gripper at the gripper engagement interface, and a control device.

[0088] According to one or more aspects of the present disclosure, the predetermined layer engagement position and orientation provide an engagement plane orientation of the gripper engagement interface, and the layer position and orientation represent the planarity of an engagement surface of at least one of the pallet load layers arranged to interface with a gripper engagement interface substantially spanning at least one of the pallet load layers, and the layer position and orientation represent a planar mismatch in at least two orthogonal directions between an engagement surface of at least one of the pallet load layers and a planar orientation of the gripper engagement interface.

[0089] According to one or more aspects of the present disclosure, the vision system includes at least one camera attached independently of the robot.

[0090] According to one or more aspects of the present disclosure, each of the at least one cameras is calibrated to a common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents the positional relationship between the respective camera reference frames of each of the respective cameras, the other each of the at least one cameras, and a predetermined reference frame of the robot.

[0091] According to one or more aspects of the present disclosure, at least one camera is arranged such that the field of view of the at least one camera covers the pallet load from the upward-facing surface at the top of the pallet load to the bottom of the pallet load.

[0092] According to one or more aspects of the present disclosure, at least one camera is arranged such that the upward-facing surface of each of the pallet load layers is within the field of view of the at least one camera.

[0093] According to one or more aspects of the present disclosure, a depalletizer

[0094] is a pallet unloading station configured to receive the pallet load of the cases arranged in the pallet load layer, wherein each of the pallet load layers is formed of a plurality of cases juxtaposed at a common level across the area of the pallet load, the pallet unloading station,

[0095] a robot comprising a depalletizing end effector having a grip configured to grip and pick at least one of the pallet load layers so as to convey at least one of the pallet load layers from the pallet load at the pallet unloading station to an output station, the grip having a grip engagement interface defining a predetermined layer engagement position and orientation for at least one of the pallet load layers with respect to the depalletizing end effector so as to repeatedly effect capture and stable holding of at least one of the pallet load layers using the grip,

[0096] a vision system arranged separately from the robot to image the pallet load at the pallet unloading station and configured to generate at least one image of at least one upper portion of at least one of the pallet load layers, decoupled from the movement of the robot,

[0097] and a control device configured to effect determination of the positional and orientation relationship between the grip engagement interface and each uppermost pallet layer of at least one of the pallet load layers based on the at least one image.

[0098] According to one or more aspects of the present disclosure, the control device is operably connected to the vision system to receive at least one image from the vision system.

[0099] According to one or more aspects of the present disclosure, the control device is operably coupled to the robot to position the gripper relative to each top pallet layer based on the determined relationship and capture and hold at least one pallet load layer using the gripper at the gripper engagement interface.

[0100] According to one or more aspects of the present disclosure, the determined relationship represents the layer position and orientation of each top layer relative to the predetermined layer engagement position and orientation of the gripper engagement interface relative to a predetermined reference frame of the robot.

[0101] According to one or more aspects of the present disclosure, the control device determines the respective layer position and orientation of each top layer based on at least one image and compares the respective layer position and orientation with a predetermined reference frame of the robot to result in a determination of the determined relationship.

[0102] According to one or more aspects of the present disclosure, the respective layer position and orientation represent the planarity of the engagement surface of each top layer, arranged to interface connect with a gripper engagement interface that substantially spans each top layer, and the respective layer position and orientation represent at least one of a planar misalignment and a center point misalignment in at least two orthogonal directions between the engagement surface of the top layer and the gripper engagement interface.

[0103] According to one or more aspects of the present disclosure, the control device is configured to resolve at least one of a planar misalignment and a center point misalignment for optimal gripper engagement with each top layer based on the robot's motion boundary conditions defined by at least one of the robot's architecture and structure bounding the depalletizer represented in the robot's predetermined reference frame.

[0104] According to one or more aspects of the present disclosure, a vision system includes at least one camera mounted independently from a robot.

[0105] According to one or more aspects of the present disclosure, each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents the positional relationship between each respective camera reference frame of each respective camera and each other camera of the at least one camera, and a predetermined reference frame of the robot.

[0106] According to one or more aspects of the present disclosure, the at least one camera is arranged such that the field of view of the at least one camera covers the pallet load from the upward-facing surface at the top of the pallet load to the bottom of the pallet load.

[0107] According to one or more aspects of the present disclosure, the at least one camera is arranged such that the upward-facing surface of each top of the pallet load layers is within the field of view of the at least one camera.

[0108] According to one or more aspects of the present disclosure, a method of depalletizing cases in a depalletizer is provided. The method includes

[0109] receiving, at a pallet load unloading station of the depalletizer, a pallet load of cases arranged in a pallet load layer, wherein each of the pallet load layers is formed of a plurality of cases juxtaposed at a common level across the area of the pallet load;

[0110] A step of providing a robot equipped with a depalletizing end effector having a grip configured to pick at least one pallet load layer by gripping at least one of the pallet load layers so as to convey at least one pallet load layer from the pallet load at the pallet unloading station to the output station, the grip having a grip engagement interface defining a predetermined layer engagement position and orientation for at least one of the pallet load layers with respect to the depalletizing end effector so as to repeatedly effect capture and stable holding of at least one of the pallet load layers using the grip, the step and,

[0111] A step of imaging a pallet load of cases at a pallet unloading station using a vision system and generating at least one image of at least an upper portion of at least one of the pallet load layers independently of the movement of the robot;

[0112] A step of, in a control device operably connected to the vision system, receiving at least one image from the vision system and resulting in determination of the layer position and orientation of at least one of the pallet load layers with respect to the predetermined layer engagement position and orientation of the grip engagement interface, the control device being operably connected to the robot so as to position the grip and capture and hold at least one of the pallet load layers using the grip at the grip engagement interface, the step and, including.

[0113] According to one or more aspects of the present disclosure, the predetermined layer engagement position and orientation provide an engagement plane orientation of the grip engagement interface, and the layer position and orientation represent the planarity of the engagement surface of at least one of the pallet load layers arranged to interface connect with the grip engagement interface substantially spanning at least one of the pallet load layers, and the layer position and orientation represent a plane mismatch in at least two orthogonal directions between the engagement surface of at least one of the pallet load layers and the plane orientation of the grip engagement interface.

[0114] According to one or more aspects of the present disclosure, the vision system comprises at least one camera mounted independently from the robot.

[0115] According to one or more aspects of the present disclosure, each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents the positional relationship between the respective camera reference frames of each of the respective cameras, and each of the other cameras of the at least one camera, and a predetermined reference frame of the robot.

[0116] According to one or more aspects of the present disclosure, the at least one camera is arranged such that the field of view of the at least one camera covers the pallet load from the upward-facing surface at the top of the pallet load to the bottom of the pallet load.

[0117] According to one or more aspects of the present disclosure, the at least one camera is arranged such that the upward-facing surface of the top of each of the pallet load layers is within the field of view of the at least one camera.

[0118] According to one or more aspects of the present disclosure, a method of depalletizing cases in a depalletizer is provided. The method includes

[0119] receiving, at a pallet load unloading station of the depalletizer, a pallet load of cases arranged in a pallet load layer, each of the pallet load layers being formed of a plurality of cases juxtaposed at a common level across the area of the pallet load;

[0120] Providing a robot comprising a depalletizing end effector having a grip configured to pick at least one pallet load layer by gripping at least one of the pallet load layers so as to convey at least one pallet load layer from a pallet load at a pallet unloading station to an output station, the grip having a grip engagement interface defining a predetermined layer engagement position and orientation for at least one of the pallet load layers with respect to the depalletizing end effector so as to repeatedly effect capture and stable holding of at least one of the pallet load layers using the grip, the step of

[0121] Using a vision system arranged separately from the robot to image the pallet load at the pallet unloading station and generate at least one image of at least an upper portion of at least one of the pallet load layers decoupled from the movement of the robot, the step of

[0122] Using a control device of the depalletizer to effect determination of the positional and orientation relationship between the grip engagement interface and each uppermost pallet layer of at least one of the pallet load layers based on the at least one image, the step of

[0123] According to one or more aspects of the present disclosure, the control device is operably coupled to the vision system to receive at least one image from the vision system.

[0124] According to one or more aspects of the present disclosure, the control device is operably coupled to the robot to position the grip with respect to each uppermost pallet layer based on the determined relationship and capture and hold at least one pallet load layer using the grip at the grip engagement interface.

[0125] According to one or more aspects of the present disclosure, the determined relationship represents the layer position and orientation of each uppermost layer with respect to the predetermined layer engagement position and orientation of the grip engagement interface with respect to a predetermined reference frame of the robot.

[0126] According to one or more aspects of the present disclosure, the control device determines the respective layer positions and orientations of each top layer based on at least one image, and by comparing the respective layer positions and orientations with a predetermined reference frame of the robot, a determination of the determined relationship is obtained.

[0127] According to one or more aspects of the present disclosure, the respective layer positions and orientations are arranged to interface connect with a grip engagement interface that substantially spans each top layer, representing the planarity of the engagement surface of each top layer, and the respective layer positions and orientations represent at least one of planar misalignment and center point misalignment in at least two orthogonal directions between the engagement surface of the top layer and the grip engagement interface.

[0128] According to one or more aspects of the present disclosure, the control device resolves at least one of planar misalignment and center point misalignment for optimal grip engagement with each top layer based on the motion boundary conditions of the robot defined by at least one of the architecture and structure of the robot that bounds the depalletizer represented in the predetermined reference frame of the robot.

[0129] According to one or more aspects of the present disclosure, the vision system includes at least one camera attached independently of the robot.

[0130] According to one or more aspects of the present disclosure, each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents the positional relationship between the respective camera reference frames of each of the respective cameras and each of the other at least one camera, and the predetermined reference frame of the robot.

[0131] According to one or more aspects of the present disclosure, at least one camera is arranged such that the field of view of the at least one camera covers the pallet load from the upward-facing surface at the top of the pallet load to the bottom of the pallet load.

[0132] According to one or more aspects of the present disclosure, at least one camera is arranged such that the upward-facing surface of each top of the pallet load layers is within the field of view of the at least one camera.

[0133] According to one or more aspects of the present disclosure, a slip sheet removing device is provided for removing a slip sheet from a layer of goods moving along a transverse path. The slip sheet removing device includes a frame and a roller rotatably coupled to the frame for rotation about a rotation axis, the roller having a vacuum port extending through a goods support surface of the roller, the roller and a vacuum mechanism coupled to the roller to draw a vacuum through the vacuum port, the vacuum port being positioned on the roller for engaging a slip sheet disposed between the roller and a layer of goods supported on the goods support surface of the roller to grip the slip sheet and separate the slip sheet from the layer of goods.

[0134] According to one or more aspects of the present disclosure, the roller rotates the slip sheet gripped by the vacuum mechanism about the rotation axis substantially simultaneously to separate the slip sheet from the layer of goods.

[0135] According to one or more aspects of the present disclosure, the vacuum mechanism includes at least one suction tube extending through the roller.

[0136] According to one or more aspects of the present disclosure, the vacuum port includes a suction cup coupled to the roller for gripping the slip sheet.

[0137] According to one or more aspects of the present disclosure, the slip sheet removing device further includes a deflection shield for engaging the slip sheet and separating the slip sheet from the layer of goods.

[0138] According to one or more aspects of the present disclosure, the slip sheet removal device further comprises a stripper plate coupled to the frame, the stripper plate being positioned relative to the roller to remove (or otherwise peel / remove) the slip sheet from the roller.

[0139] According to one or more aspects of the present disclosure, the slip sheet removal device further comprises a collection bin configured to collect slip sheets separated from a layer of merchandise.

[0140] According to one or more aspects of the present disclosure, the slip sheet removal device further comprises an upstream conveyor and a downstream conveyor, with the roller disposed between the upstream conveyor and the downstream conveyor.

[0141] According to one or more aspects of the present disclosure, the transverse path extends along the upstream and downstream conveyors.

[0142] According to one or more aspects of the present disclosure, the upstream conveyor is a mat top conveyor and the downstream conveyor is a case spread conveyor.

[0143] According to one or more aspects of the present disclosure, a depalletizer includes a pallet unloading station configured to receive a pallet of goods and separate the pallet into layers of goods, and a case conveyor configured to convey the layers of goods from the pallet unloading station to a storage array, thereby providing supply of the goods to the storage array. The case conveyor includes a plurality of conveyor sections, and a slip sheet removal system disposed between two adjacent conveyor sections of the plurality of conveyor sections. The slip sheet removal system includes a frame, and a roller rotatably coupled to the frame for rotation about a rotation axis, the roller having a vacuum port extending through a product support surface of the roller, and a vacuum mechanism coupled to the roller to draw a vacuum through the vacuum port. The vacuum port is positioned on the roller to engage a slip sheet disposed between the roller and a layer of goods supported on the product support surface of the roller to grip the slip sheet and separate the slip sheet from the layer of goods.

[0144] According to one or more aspects of the present disclosure, a method for removing a slip sheet from a layer of goods is provided. The method includes providing a frame of a slip sheet removal device, providing a roller rotatably coupled to the frame about a rotation axis, the roller being configured to support and convey a layer of goods disposed on the slip sheet along a transverse axis, vacuum gripping the slip sheet with the roller, the vacuum being generated by a vacuum mechanism coupled to the roller, and separating the slip sheet from the layer of goods using the roller.

[0145] It should be understood that the foregoing description is merely illustrative of examples of aspects of the present disclosure. Various alternatives and modifications can be contemplated by those skilled in the art without departing from the aspects of the present disclosure. Accordingly, aspects of the present disclosure are intended to embrace all such alternatives, modifications, and variations that fall within the scope of any of the appended claims herein. Further, the mere fact that different features are recited in different dependent or independent claims does not indicate that a combination of these features cannot be advantageously used, nor does it indicate that such a combination remains within the scope of aspects of the present disclosure.

Claims

Claim 1 A depalletizer, wherein the depalletizer is a pallet unloading station configured to receive a pallet load of cases arranged in a pallet load layer, each of the pallet load layers being formed of a plurality of cases juxtaposed at a common level over the area of the pallet load, the pallet unloading station; a robot having a depalletizing end effector with a gripper configured to pick up at least one of the pallet load layers by gripping at least one of the pallet load layers so as to convey at least one of the pallet load layers from the pallet load at the pallet unloading station to an output station, the gripper having a gripper engagement interface defining a predetermined layer engagement position and orientation for at least one of the pallet load layers relative to the depalletizing end effector so as to repeatedly effect capture and stable retention of at least one of the pallet load layers using the gripper; a vision system arranged to image the pallet load of cases at the pallet unloading station and configured to generate at least one image of at least an upper portion of at least one of the pallet load layers independently of the movement of the robot; a control device operably coupled to the vision system to receive at least one image from the vision system and configured to determine at least one layer position and orientation of at least one of the pallet load layers relative to the predetermined layer engagement position and orientation of the gripper engagement interface based on the at least one image, the control device being operably coupled to the robot to position the gripper and capture and hold at least one of the pallet load layers using the gripper at the gripper engagement interface; A depalletizer comprising the above. Claim 2 The predetermined layer engagement position and orientation provide the engagement plane orientation of the grip engagement interface, and the layer position and orientation are arranged to interface and connect with the grip engagement interface that substantially spans at least one of the pallet load layers, representing the planarity of the engagement surface of at least one of the pallet load layers, and the layer position and orientation represent planar misalignments in at least two orthogonal directions between the engagement surface of at least one of the pallet load layers and the planar orientation of the grip engagement interface. The depalletizer according to claim 1.

3. The depalletizer according to claim 1, wherein the vision system comprises at least one camera mounted independently of the robot.

4. Each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents the positional relationship between the respective camera reference frames of each of the respective cameras, each of the other cameras of the at least one camera, and a predetermined reference frame of the robot. The depalletizer according to claim 3.

5. The depalletizer according to claim 3, wherein the at least one camera is arranged such that the field of view of the at least one camera covers the pallet load from the upward-facing surface at the top of the pallet load to the bottom of the pallet load.

6. The depalletizer according to claim 3, wherein the at least one camera is arranged such that the upward-facing surface at the top of each of the pallet load layers is within the field of view of the at least one camera.

7. A depalletizer, wherein the depalletizer A pallet load unloading station configured to receive a pallet load of cases arranged in a pallet load layer, wherein each of the pallet load layers is formed by a plurality of cases juxtaposed at a common level over the area of the pallet load. A pallet load unloading station A robot comprising a depalletizing end effector having a grip configured to pick at least one of the pallet load layers by gripping at least one of the pallet load layers so as to convey at least one pallet load layer from the pallet load at the pallet unloading station to an output station, the grip having a grip engagement interface defining a predetermined layer engagement position and orientation for at least one of the pallet load layers with respect to the depalletizing end effector so as to repeatedly effect capture and stable retention of at least one of the pallet load layers using the grip. A vision system, separate from the robot, arranged to image the pallet load at the pallet unloading station and configured to generate at least one image of at least an upper portion of at least one of the pallet load layers, decoupled from the movement of the robot. A control device configured to result in determination of the positional and orientation relationship between the grip engagement interface and each uppermost pallet layer of at least one of the pallet load layers based on the at least one image. A depalletizer comprising the above.

8. The depalletizer according to claim 7, wherein the control device is operably connected to the vision system so as to receive the at least one image from the vision system.

9. The depalletizer according to claim 7, wherein the control device is operably connected to the robot so as to position the grip with respect to each uppermost pallet layer based on the determined relationship and capture and hold at least one of the pallet load layers using the grip at the grip engagement interface.

10. The depalletizer according to claim 7, wherein the determined relationship represents the layer position and orientation of each uppermost layer with respect to the predetermined layer engagement position and orientation of the grip engagement interface with respect to a predetermined reference frame of the robot.

11. The depalletizer according to claim 7, wherein the control device determines the respective layer position and orientation of each uppermost layer based on the at least one image and compares the respective layer position and orientation with a predetermined reference frame of the robot, thereby resulting in determination of the determined relationship.

12. The flatness of the engagement surface of each top layer, where each of the layer positions and orientations interfaces with the grip engagement interface so as to substantially span across each top layer, and each of the layer positions and orientations represents at least one of planar misalignment and center point misalignment in at least two orthogonal directions between the engagement surface of the top layer and the grip engagement interface; the depalletizer according to claim 11.

13. The control device is configured to eliminate at least one of the planar misalignment and the center point misalignment for optimal grip engagement with each top layer, respectively, based on the robotic motion boundary conditions defined by at least one of the architecture and structure of the robot bounding the depalletizer represented in the predetermined reference frame of the robot; the depalletizer according to claim 12.

14. The depalletizer according to claim 7, wherein the vision system comprises at least one camera attached independently from the robot.

15. Each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents the positional relationship between the respective camera reference frames of each of the cameras, between each of the other cameras of the at least one camera, and the predetermined reference frame of the robot; the depalletizer according to claim 14.

16. The depalletizer according to claim 14, wherein the at least one camera is arranged such that the field of view of the at least one camera covers the pallet load from the upward-facing surface at the top of the pallet load to the bottom of the pallet load.

17. The depalletizer according to claim 14, wherein the at least one camera is arranged such that the upward-facing surface at the top of each of the pallet load layers is within the field of view of the at least one camera.

18. A method of depalletizing cases in a depalletizer, the method comprising: Receiving, at the pallet load unloading station of the depalletizer, a pallet load of cases arranged in a pallet load layer, wherein each of the pallet load layers is formed by a plurality of cases juxtaposed at a common level across the area of the pallet load; a step. A step of providing a robot having a depalletizing end effector having a grip configured to pick at least one of the pallet load layers by gripping at least one of the pallet load layers so as to convey at least one pallet load layer from the pallet load at the pallet unloading station to an output station, wherein the grip has a grip engagement interface defining a predetermined layer engagement position and orientation for at least one of the pallet load layers with respect to the depalletizing end effector so as to repeatedly effect capture and stable holding of at least one of the pallet load layers using the grip, the step; A step of imaging the pallet load of the case at the pallet unloading station using a vision system and generating at least one image of at least an upper portion of at least one of the pallet load layers independently of the movement of the robot; In a control device operably connected to the vision system, receiving the at least one image from the vision system and effecting determination of the layer position and orientation of at least one of the pallet load layers with respect to the predetermined layer engagement position and orientation of the grip engagement interface based on the at least one image, the control device being operably connected to the robot so as to position the grip and capture and hold at least one of the pallet load layers using the grip at the grip engagement interface, the step A method including.

19. The method according to claim 18, wherein the predetermined layer engagement position and orientation provide an engagement plane orientation of the grip engagement interface, the layer position and orientation represent planarity of an engagement surface of at least one of the pallet load layers arranged to interface-connect with the grip engagement interface substantially spanning at least one of the pallet load layers, and the layer position and orientation represent a plane mismatch in at least two orthogonal directions between the engagement surface of at least one of the pallet load layers and the plane orientation of the grip engagement interface.

20. The method according to claim 18, wherein the vision system includes at least one camera attached independently of the robot.

21. A method according to claim 20, wherein each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents the positional relationship between each respective camera reference frame of each camera, each other camera of the at least one camera, and a predetermined reference frame of the robot.

22. The method according to claim 20, wherein the at least one camera is arranged such that the field of view of the at least one camera covers the pallet load from the upward-facing surface of the top of the pallet load to the bottom of the pallet load.

23. The method according to claim 20, wherein the at least one camera is arranged such that the upward-facing surface of the top of each of the pallet load layers is within the field of view of the at least one camera.

24. A method of depalletizing cases in a depalletizer, the method comprising: receiving, at a pallet load unloading station of the depalletizer, a pallet load of cases arranged in a pallet load layer, each of the pallet load layers being formed of a plurality of cases juxtaposed at a common level across the area of the pallet load; providing a robot having a depalletizing end effector having a grip configured to pick up at least one of the pallet load layers by gripping so as to convey at least one of the pallet load layers from the pallet load at the pallet load unloading station to an output station, the grip having a grip engagement interface defining a predetermined layer engagement position and orientation for at least one of the pallet load layers with respect to the depalletizing end effector, such that repeated capture and stable holding of at least one of the pallet load layers using the grip is effected; imaging the pallet load at the pallet load unloading station using a vision system arranged separately from the robot, and generating at least one image of the upper portion of at least one of the pallet load layers decoupled from the movement of the robot; using a control device of the depalletizer to determine the positional and orientation relationship between the grip engagement interface and at least one of the top pallet layers of at least one of the pallet load layers based on the at least one image. A method comprising.

25. The method according to claim 24, wherein the control device is operably coupled to the vision system to receive the at least one image from the vision system.

26. The method according to claim 24, wherein the control device is operably coupled to the robot to position the gripper relative to each top pallet layer based on the determined relationship and capture and hold the at least one pallet load layer using the gripper at the gripper engagement interface.

27. The method according to claim 24, wherein the determined relationship represents the layer position and orientation of each top layer relative to the predetermined layer engagement position and orientation of the gripper engagement interface relative to a predetermined reference frame of the robot.

28. The method according to claim 24, wherein the control device determines the layer position and orientation of each top layer based on the at least one image and compares the respective layer positions and orientations with a predetermined reference frame of the robot to result in a determination of the determined relationship.

29. The respective layer positions and orientations represent the planarity of the engagement surface of each top layer, arranged to interface with the gripper engagement interface substantially spanning each top layer, and the respective layer positions and orientations represent at least one of a planar misalignment and a center point misalignment in at least two orthogonal directions between the engagement surface of the top layer and the gripper engagement interface. The method according to claim 28.

30. The method according to claim 29, wherein the control device resolves at least one of the planar misalignment and the center point misalignment for optimal gripper engagement with each top layer based on at least one of the robot architecture and structure defining the motion boundary conditions of the robot bounding the depalletizer represented in the predetermined reference frame of the robot.

31. The method according to claim 24, wherein the vision system comprises at least one camera mounted independently of the robot.

32. The method according to claim 31, wherein each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration to the common camera calibration reference structure represents the positional relationship between each respective camera reference frame of each of the respective cameras, each of the other cameras of the at least one camera, and a predetermined reference frame of the robot.

33. The method according to claim 31, wherein the at least one camera is arranged such that the field of view of the at least one camera covers the pallet load from the upward-facing surface at the top of the pallet load to the bottom of the pallet load.

34. The method according to claim 31, wherein the at least one camera is arranged such that the upward-facing surface at the top of each of the pallet load layers is within the field of view of the at least one camera.

35. A slip sheet removing device for removing a slip sheet from a layer of goods moving along a transverse path, the slip sheet removing device comprising: a frame; a roller rotatably coupled to the frame for rotation about a rotation axis, the roller having a vacuum port extending through a product support surface of the roller; a vacuum mechanism coupled to the roller to draw a vacuum through the vacuum port, the vacuum port being positioned on the roller to engage a slip sheet disposed between the roller and a layer of goods supported on the product support surface of the roller to grip the slip sheet and separate the slip sheet from the layer of goods; A slip sheet removing device comprising.

Images