Vision-aided robotic depiler

Abstract

A depalletizer having a pallet station for receiving a pallet load of cases placed in a layer of pallet load, a robot having an end effector with a gripper for at least one of gripping and picking up the layer and having a gripper engagement interface defining a predetermined layer engagement position and orientation of the layer relative to the depalletizing end effector, a vision system for imaging the pallet load of cases and generating at least one image of the top of the layer independent of robot motion, and a controller receiving the image and effecting a layer position and orientation of the layer relative to the predetermined layer engagement position and orientation of the gripper engagement interface and operatively coupled to the robot to position the gripper and capture and hold the layer at the gripper engagement interface with the gripper.

Description

[0001] Cross-references to related applications

[0002] This application is a non-provisional application and claims the benefit of U.S. Provisional Patent Application No. 62 / 916,080, filed on October 16, 2019, the entire disclosure of which is incorporated herein by reference. Background Technology 1. Technical Field

[0004] This invention relates generally to depalletizing, and more specifically to vision-assisted robotic depalletizing of products.

[0005] 2. A brief description of related developments

[0006] Retail product delivery (whether through traditional brick-and-mortar stores, online stores, or hybrid retail channels) requires improved storage, sorting, and shipping efficiency, especially for the delivery of so-called mixed or multi-purpose boxes (within a given vehicle)—whether for store replenishment or a single order. Intelligent / adaptive automation applications are increasingly helping to improve efficiency across many delivery levels, including storage, sorting, and shipping.

[0007] Distribution centers and warehouses typically receive their goods on structured pallets, such as boxes, cartons, open trays, stretch-packed pallets, etc., which are arranged in an orderly fashion without gaps between them. Depalletizing systems are known in the art for removing products from pallets. Conventional pallet unloaders (e.g., depalletizers) with electromagnetic radiation and optical mapping sensors (e.g., laser scanners, 3-D cameras, etc.) are known to map the position of the pallet load in 3-D, thereby improving automated positioning relative to the pallet load. For example, a conventional method and system for detecting and reconstructing an environment to facilitate robot interaction with such an environment includes determining a three-dimensional (3-D) virtual environment, where the 3-D virtual environment represents the physical environment of a robot manipulator, the physical environment comprising a plurality of 3-D virtual objects corresponding to corresponding physical objects in the physical environment. The method then involves determining a two-dimensional (2-D) image of the virtual environment, including a 2-D depth map. The method may then include determining portions of the 2-D image corresponding to a given one or more physical objects. The method may then include: determining a 3D model based on the portion and the 2D depth map, the 3D model corresponding to a portion of the 2D image corresponding to a given physical object. The method may then include: selecting a physical object from the given physical objects based on the 3D model. The method may then include providing instructions to a robot manipulator to move the object. Attached Figure Description

[0008] The foregoing aspects and other features of this disclosure are explained in the following description, in conjunction with the accompanying drawings, in which:

[0009] Figure 1 This is a schematic diagram of the distribution facilities based on various aspects of this disclosure;

[0010] Figure 2 This is a schematic diagram of the pallet load based on various aspects of this disclosure;

[0011] Figure 3 Based on all aspects of this disclosure Figure 1 A schematic perspective view of a palletizer / depalletizer unit for a distribution facility, which has a vision system and a robot with layer depalletizing tools;

[0012] Figure 4 yes Figure 3 Top perspective view of the destacking tool;

[0013] Figure 5 yes Figure 3 A top view of the destacking tool;

[0014] Figure 6 yes Figure 3 An exploded perspective view of the destacking tool, showing the destacking tool without most of its frame;

[0015] Figure 7 yes Figure 3 Top perspective view of the frame and part of the curtain actuation assembly of the destacking tool;

[0016] Figure 8A and 8B They are Figure 3 Top and bottom perspective views of the depalletizing tool, showing the position for clamping a full pallet layer;

[0017] Figure 9A and 9B They are similar to Figure 8A and 8B A perspective view showing Figure 3 The clamps of the depalletizing tool apply pressure to the pallet layer, and the curtain section is partially closed;

[0018] Figure 10A and 10B They are similar to Figure 8A and 8B A perspective view showing Figure 3 The clamps of the depalletizing tool have released some of the pressure acting on the pallet layer, and the curtain is completely closed;

[0019] Figure 11A-11F It is intended for use in accordance with all aspects of this disclosure. Figure 3A schematic perspective view of the camera field of view of the vision system of the palletizer unit;

[0020] Figure 12 Based on the various aspects of this disclosure Figure 3 Exemplary images captured by the vision system of the palletizer unit;

[0021] Figure 13A It is based on the various aspects of this disclosure for use Figure 3 An exemplary calibration clamp / fixture for a vision system;

[0022] Figure 13B It is captured by the vision system according to various aspects of this disclosure. Figure 13A Exemplary image of a calibration clamp / fixture;

[0023] Figure 14 This is an exemplary flowchart of methods according to various aspects of this disclosure;

[0024] Figures 15A-15D The various aspects shown in this disclosure are as follows Figure 3 An exemplary position of the depalletizing tool relative to the layer of pallets to be picked up;

[0025] Figures 16A-16C Based on the various aspects of this disclosure, it is possible to... Figure 3 An example pallet layer configuration picked up by the depalletizing tool;

[0026] Figure 17 This is an exemplary flowchart of methods according to various aspects of this disclosure;

[0027] Figure 18 These are exemplary flowcharts of methods according to various aspects of this disclosure; and

[0028] Figure 19A , 19B 19C is a schematic diagram of the removal of thin liner paper according to various aspects of this disclosure. Detailed Implementation

[0029] Figure 1 This is a schematic diagram of a warehouse system or distribution facility 100WS (referred to herein as warehouse system 100WS) according to various aspects of this disclosure. While various aspects of this disclosure will be described with reference to the accompanying drawings, it should be understood that these aspects can be embodied in many forms. Furthermore, elements or materials of any suitable size, shape, or type can be used. It should be understood that although distribution facility 100WS is described herein as an automated distribution facility, aspects of this disclosure are also applicable to distribution facilities with any suitable transportation system, such as automated and manual transportation systems or fully manual transportation systems.

[0030] refer to Figure 1 and Figure 2 According to various aspects of this disclosure, the warehouse system 100WS includes at least one palletizer / depalletizer unit 10A, 10B (generally referred to herein as palletizer unit 10). Palletizer unit 10 has one or more robotic box manipulators 14 (also referred herein as articulated robots, adaptive real-time robots, robots, or product picking devices) that, with the assistance of a vision system, place (individually or fabricating pick faces) mixed-box pallet load units CU (also referred herein as box units or boxes or products 18) in stacks SL1-Sn and / or layers PL1-PL4, thereby constructing a mixed-box pallet load PAL. An example of a suitable palletizer / depalletizer is described in U.S. Patent No. 10,343,857, entitled “Vision-Assisted Robotized Depalletizer,” issued July 9, 2019, the entire disclosure of which is incorporated herein by reference.

[0031] Palletizing unit 10 is equipped with a three-dimensional (3D) time-of-flight (TOF) camera vision system 310 (referred to herein as vision system 310), which generates a three-dimensional (3D) image of each pallet layer (also referred to herein as pallet load layer) and the box unit CU to be removed by robot 14. Vision system 310 is positioned at palletizing unit 10 to image the pallet load PAL of the box CU at pallet unloading / loading station 301 and is configured to generate at least one pallet load layer 816 independently of the movement of robot 14. Figure 8B -It refers to at least one top / surface 148 of pallet layers PL1, PL2, PL3, PL4, PL5. Figure 2 , 8A At least one image of (and 11A-11F) (see) Figure 12 In accordance with the cyclical movement of robot 14 to pick up pallet layers, vision system 310 generates and provides three-dimensional image information in real time to unload goods from pallet load PAL and to notify at least one layer of the pallet structure from the first pallet layer PL1 to the last pallet layer PL5 located on pallet support SPAL in real time (within the frame of robot 14's pick / place motion cycle).

[0032] The difference between the layer pose 3D image information of each layer and the plan (e.g., the skew of the layer relative to the pallet support SPAL and / or other layers, uncovered box unit CU, etc.) is, for example, notified to the robot 14 of the compensation for the difference, so that the robot 14 can use the real-time positioning of the robot 14 relative to the layer to be picked up to make the compensation, thereby facilitating substantially continuous adaptive real-time robot 14 placement and adaptive depalletizing (in a fully automated manner or in a collaborative / cooperative manner with user assistance), and simultaneously solving pallet quality / control and robot 14 depalletizing.

[0033] A controller (e.g., robot controller 16 and / or unit controller 10C) is operatively coupled to vision system 310 to receive at least one image from vision system 310. The controller is configured to determine at least one pallet layer 816 based on the at least one image. Figure 8B ) relative to the gripper 800 of robot 14 ( Figure 8B The clamp engagement interface 810 ( Figure 8B - As described herein) the predetermined layer engagement position and orientation of the layer (e.g., in the robot coordinate system or reference frame X, Y, Z, RX, RY, RZ-) Figure 3 The controller is operatively coupled to robot 14 to position grippers 800 relative to each top pallet layer (e.g., based on a determined relationship between gripper interface 810 and each top pallet layer of at least one pallet load layer), and to capture and hold at least one pallet layer 816 with grippers 800 at gripper engagement interface 810. The controller determines the corresponding layer position and orientation for each top layer based on at least one image, and the determined relationship is achieved by comparing the corresponding layer position and orientation with a predetermined reference frame of robot 14 (as described herein).

[0034] The vision system 310 integrated into the automated palletizer unit 10 notifies and enables the unit controller 10C to provide real-time (or near-real-time) command input to an automated device such as one or more robots 14. This command input responds in real-time (corresponding to the processing time of the command, as will be further described) to changes in pallet load, thereby adapting one or more robots 14 to address changes in pallet load in real time and influence (automatically and / or in a collaborative / cooperative manner with user assistance) depalletizing in a time-optimal manner.

[0035] Refer again Figure 1According to various aspects of this disclosure, the distribution facility 100WS includes a storage and retrieval system 100 that can operate in a retail distribution center or warehouse, for example, to fulfill orders for box units received from retail stores. In one example, a box unit may be a box or unit that does not contain goods stored on a pallet, tote, or pallet (e.g., not included). In other examples, a box unit may be a box or unit containing goods in any suitable manner on a pallet, tote, or pallet. It should be noted that a box unit may include a boxed unit of goods (e.g., a soup can, cereal box, etc.) or a single item adapted to be removed from or placed on a pallet. According to embodiments, shipping containers for the box units (e.g., cardboard boxes, cartons, crates, jars, or any other suitable means for holding the box units) may have variable sizes and may be used to hold the box units during shipment, and may be configured such that they can be stacked for shipment. It is important to note that, for example, when the packages or pallets of the box units arrive at the storage and retrieval system, the contents of each pallet can be identical (e.g., each pallet holds a predetermined number of the same items: one pallet holds soup and another holds cereal), and when the pallet leaves the storage and retrieval system, the pallet can contain any suitable number and combination of different box units (e.g., each pallet can hold different types of box units: one pallet holds a combination of soup and cereal). In embodiments, the storage and retrieval system described herein can be adapted to any environment for storing and retrieving box units.

[0036] The storage and retrieval system 100 can be configured to be installed in, for example, an existing warehouse structure or adapted to a new warehouse structure. In various aspects of this disclosure, the storage and retrieval system may include one or more input transfer stations 170 and one or more output transfer stations 160, input / output box conveyors 150A, 150B, 150C (generally referred to as input / output box conveyor 150), a storage structure array 130, and a plurality of autonomous vehicle transport robots 110 (referred to herein as “robots”). In various aspects of this disclosure, the storage and retrieval system may also include robots or robot transfer stations, such as those described in U.S. Patent No. 9,096,375, issued August 4, 2015, the entire disclosure of which is incorporated herein by reference. In various aspects of this disclosure, the robot transfer station may provide docking between the robot 110 and the input / output box conveyor 150, allowing box units to be indirectly transferred between the robot 110 and the input / output box conveyor 150 via the robot transfer station. In various aspects of this disclosure, the box unit can be directly transferred between the robot 110 and the input / output box conveyor 150.

[0037] The storage structure array 130 may include multiple-level storage rack modules forming a storage array of storage locations 130SL for box units, each storage location 130SL being arranged to store at least one box unit in each storage location 130SL. In one aspect, each level of the storage structure array 130 includes a respective storage / pickup aisle 130A and transfer deck 130B for transferring box units between any storage area of ​​the storage structure array 130 and any rack of any input / output box conveyor 150. The storage aisle 130A and transfer deck 130B are also configured to allow a robot 110 to pass through the storage aisle 130A and transfer deck 130B for placing box units into pick-up inventory and retrieving ordered box units, wherein the box units are stored or otherwise held in the storage aisle 130A and / or on the transfer deck 130B in the storage location 130SL. The robot 110 may be any suitable robot capable of transporting and transferring box units throughout the storage and retrieval system 100. For illustrative purposes only, suitable examples of robots can be found in the following documents: U.S. Patent No. 8,425,173, issued April 23, 2013; U.S. Patent No. 9,561,905, issued February 7, 2017; U.S. Patent No. 8,965,619, issued February 24, 2015; U.S. Patent No. 8,696,010, issued April 15, 2014; U.S. Patent No. 9,187,244, issued November 17, 2014; U.S. Pre-Grant Publication No. 2012 / 0189416 (U.S. Serial No. 13 / 326,952), entitled “Automated Bot with Transfer Arm,” filed December 15, 2011; and U.S. Patent No. 9,499,338, issued November 22, 2016, the entire disclosure of which is incorporated herein by reference. Robot 110 can be configured to place box units (such as retail goods as described above) into pick-up inventory in one or more levels of storage structure array 130, and then selectively retrieve ordered box units for delivery to, for example, a store or other suitable location.

[0038] Feed transfer station 170 and discharge transfer station 160 can operate in conjunction with their respective input / output box conveyors 150A, 150B for bidirectional transfer of box units into and out of one or more levels of storage structure array 130, thereby enabling the feeding of box units into and out of storage structure array 130. It should be noted that although feed transfer station 170 and discharge transfer station 160 (and their respective input / output box conveyors 150A, 150B and palletizer / depalletizer units 10A, 10B) are described as dedicated inbound (e.g., feeding) transfer station 170 and dedicated outbound (e.g., outbound) transfer station 160, in all aspects of this disclosure, each of transfer stations 170, 160 can be used for both inbound and outbound transfer of box units from the storage and retrieval system. It should be noted that although an input / output box conveyor is described here, the conveyor can be any suitable conveyor (including any suitable conveyor path orientation, such as vertical and / or horizontal conveyor paths) or a transfer / pickup device with any suitable conveyor path orientation.

[0039] In one aspect, as described above, each infeed transfer station 170 and outfeed transfer station 160 includes respective input / output box conveyors 150A, 150B and respective palletizer / depalletizer units 10A, 10B (generally referred to herein as palletizer unit 10). In one aspect, the palletizer / depalletizer unit 10 is an automated unit, for example, each unit is configured to receive loaded pallets (e.g., having consistent or mixed box units or products) from a pallet load input 175 area, which may include an input / output loaded pallet conveyor 175C (in... Figure 1 (shown as an input conveyor); and / or constructing loaded pallets (e.g., having consistent or mixed box units or products), for example, to transport to the pallet load output 180 area, which may include input / output loaded pallet conveyors 180C (in... Figure 1(Shown as output conveyors). In one aspect, conveyors 175C and 180C are each connected to storage structure array 130 and configured to bidirectionally transport loaded pallets in an input direction toward storage structure array 130 and in different output directions away from storage structure array 130. In one aspect, conveyors 175C and 180C may each include a conveyor arrangement with a distributed conveyor bed arranged to form a transfer path, or in other aspects, conveyors 175C and 180C may be discrete transport units, such as forklifts / pallet trucks. Suitable examples of automated palletizer / depalletizer units 10A and 10B can be found in the following documents: U.S. Patent Application No. 15 / 235,254, filed August 12, 2016; and U.S. Patent No. 8,965,559, granted February 24, 2015, the entire disclosure of which is incorporated herein by reference. Each palletizer unit includes one or more robotic box manipulators 14, which may also be referred to as articulated robots or robots. As described herein, one or more robotic box manipulators 14 are configured to continuously transport and place pallet-loaded item units CU onto pallet racks for use at pallet unloading / loading station 301 (see Figure 3 ) build (or in other respects described herein, decompose or decommission) pallet load 250.

[0040] When the palletizer unit 10 is used as an output of the palletizer, palletized item units CU of various sizes can arrive at the palletizer unit 10 via the input / output box conveyor 150B, be picked up by one of the robot box manipulators 14 and placed on the palletized load PAL, as will be described herein. When the palletizer unit 10 is used as an output of the palletizer, a full palletized load PAL (see...) formed by various box units... Figure 2 The palletized items are prepared to be picked up by a forklift from palletizer unit 10 for transport to pallet load output area 180. When palletizer / depalletizer unit 10 is used as an input for depalletizer, full palletized boxes (which may resemble pallet load PAL and be formed of similar or mixed boxes) formed by various palletized item units CU arranged in the pallet load layers are transferred from the pallet load in area 175 to the pallet unloading / loading station 301 of palletizer unit 10 in any suitable manner (e.g., by forklift or other transport). Each of the pallet load layers PL1, PL2, PL3, PL4, PL5 is formed by more than one box CU juxtaposed at a common level in the area of ​​the pallet load PAL. In one aspect, as... Figure 2 As shown, the pallet layer can be a mixed pallet layer, which includes boxes (CUs) of different sizes; while in other aspects, such as Figure 3As shown, the pallet layer can be a uniform layer comprising boxes (CUs) of substantially the same size throughout the pallet layer. One or more robotic box manipulators 14 pick up pallet-loaded item units (CUs) from the pallet PAL for transfer into the storage structure array 130.

[0041] In one aspect, each feed transfer station 170 forms a box input path Ip, along which a palletizer / depalletizer unit 10A unloads box units layer by layer, or unloads multiple box units from a standard pallet (e.g., a similar pallet with stability suitable for automatically engaging pallet layers via an automated layer interface unit (e.g., a product picking device or robot 14)) into a single box unit. The palletizer / depalletizer unit 10A communicates with the transport system of the automated storage and retrieval system 100 (e.g., input / output box conveyor 150A) to form an overall input system (e.g., feed transfer station 170) for feeding box units into the automated storage and retrieval system 100. Each feed transfer station 170 defines a box input path Ip in conjunction with the automated storage and retrieval system 100 and the warehouse management system 199, wherein the warehouse management system 199 includes any suitable controller 199C, which is configured with any suitable non-transitory program code and storage to manage at least the box unit input to the storage structure array 130, the box unit storage distribution within the storage structure array 130, the box unit retrieval from the storage structure array 130, the box unit inventory / replenishment, and the box unit output.

[0042] In one aspect, each box unit input path Ip includes at least one corresponding box unit inspection unit 142 communicating with the warehouse management system 199. In one aspect, the at least one corresponding box unit inspection unit 142 may be any suitable inspection unit including any suitable volume inspection, such as utilizing a multi-dimensional light curtain, an imaging system, and / or any other suitable sensing / sensor arrangement configured to detect box unit defects and identify box units, for example, for inventory, transport sequencing, storage allocation, and sequencing of box units output from the storage structure array 130.

[0043] In one aspect, as noted above, the palletizer / depalletizer unit 10A can be fully automated to break down or remove one or more layers from pallets being unloaded at the palletizer / depalletizer unit 10A. It should be noted that, according to reference... Figure 2The term "release" refers to the removal of pallet layers PL1, PL2, PL3, PL4 (whole or partially) from the pallet PAL, such that each pallet-loaded item unit CU is removed from layers PL1, PL2, PL3, PL4 at a predetermined level 200 of the pallet PAL (which may correspond to a release level / loading level or transfer surface). Thus, in some aspects, the pallet PAL (via any suitable pallet lifting device of the palletizer unit 10) is shifted to the next level of the pallet PAL for the purpose of (whole or partially) removing the next level PL2, PL3 corresponding to the next level of the pallet PAL.

[0044] In one aspect, the palletizer / depalletizer unit 10A is configured to release layers PL1, PL2, PL3, PL4 such that this release is synchronized with or coordinated (e.g., matched) by a predetermined rate or feed rate of the box unit flow established by the warehouse management system 199 in the automated storage and retrieval system 100. For example, in one aspect, the warehouse management system 199 is configured to set and / or monitor a predetermined rate of the box unit flow within the automated storage and retrieval system 100. For example, warehouse management system 199 monitors and manages the automation systems of automated storage and retrieval system 100 (e.g., input / output box conveyors 150A, 150B, robot 110, and palletizer / depalletizer units 10A, 10B), wherein each automation system or one or more of the automation systems has a given processing time (e.g., the time / cycle for realizing the transport or transfer of basic units of boxes, such as transferring box units on / off input / output box conveyors to pick / place stations, or lifting box units a predetermined distance, or transferring pick / placement to storage locations; the time for transferring pallet layers to pallets or from pallets, etc.). In fact, under the control of warehouse management system 199 or any other suitable controllers of automated storage and retrieval system 100 (e.g., robot controllers, conveyor controllers, palletizer / depalletizer controllers, etc.), they individually or in combination define a predetermined rate of flow of box units in automated storage and retrieval system 100 established by warehouse management system 199. For example, the controller 199C of the warehouse management system 199 can be communicatively connected to one or more input / output box conveyors 150A, 150B, such that the one or more input / output box conveyors 150A, 150B bidirectionally transfer box units to and from the storage structure array 130 at a predetermined box feed rate. The controller 199C can also be communicatively connected to palletizer / depalletizer units 10A, 10B corresponding to one or more input / output box conveyors 150A, 150B, such that the loading and unloading of layers by the substantially consecutive palletizer / depalletizer units 10A, 10B are matched to a predetermined box feed rate. Although aspects of this disclosure are described herein with respect to a distribution facility 100WS having an automated storage and retrieval system 100 with an automated transport system, aspects of this disclosure are also applicable to distribution facilities having any suitable transport system, such as an automated and manual transport system or a fully manual transport system, wherein both automated and manual transport processes have their own processing times, wherein the loading of box units into and unloading of box units from pallets can be matched with processing times in a manner substantially similar to that described herein.

[0045] In one aspect, each outgoing transfer station 160 forms a box output path Op, on which, for example, a palletizer / depalletizer unit 10B uses an automated layer interface unit (e.g., one or more robotic box manipulators 14) to stack multiple box units layer by layer onto a pallet PAL. In one aspect, the pallet PAL may be formed as a standard pallet (e.g., homogeneous box units) or a mixed pallet, as described in U.S. Patent Application No. 14 / 997,920, filed January 18, 2016, the entire disclosure of which is incorporated herein by reference. In one aspect, a warehouse management system 199 is configured to establish a pallet solution with mixed box units that provides a stable pallet load stack adapted for transfer as a layer by the end effectors of one or more robotic box manipulators 14. As described above, suitable examples of the palletizer / depalletizer unit 10B can be found in U.S. Patent Application No. 15 / 235,254, filed August 12, 2016, the entire disclosure of which is previously incorporated herein by reference.

[0046] In one aspect, the palletizer / depalletizer unit 10B communicates with the transport system (e.g., input / output box conveyor 150B) of the automated storage and retrieval system 100 to form an overall output system (e.g., an outgoing transfer station 160) that receives box units from the automated storage and retrieval system 100 for placement on pallets according to any suitable box output order sequence. For example, as described above, pallet-loaded item units CU sent to one or more robotic box manipulators 14 are transferred to pallet PAL by the end effectors of one or more robotic box manipulators 14. The pallet-loaded item units CU (output box units) are arranged layer by layer (note that this layer may cover the entire pallet or a portion of the pallet) in a predetermined order established by the warehouse management system 199 to form a standard output pallet load.

[0047] Each outgoing transfer station 160 defines a box output path Op in conjunction with the automated storage and retrieval system 100 and the warehouse management system 199, wherein the warehouse management system 199 includes any suitable controller 199C, which is configured with any suitable non-transitory program code and storage to manage the operation of the distribution facility 100WS, including box unit outputs from the storage structure array 130, as described herein. In one aspect, each box unit output path Op includes at least one corresponding box unit checking unit 142 (as described above) communicating with the warehouse management system 199. In one aspect, as described above, the palletizer / depalletizer unit 10B can be fully automated to build (multiple) layers or load (multiple) layers onto pallet loads at the palletizer / depalletizer unit 10B. Note that, referring to Figure 2 The term "loading" refers to constructing pallet layers PL1, PL2, PL3, PL4 (whole or partially) onto pallet PAL, such that each pallet-loaded item unit CU is inserted into layers PL1, PL2, PL3, PL4 at a predetermined level 200 of pallet PAL (which may correspond to a release level / loading level or transfer surface) until pallet layers PL1, PL2, PL3, PL4, PL5 are formed, such that in some aspects, pallet PAL (via any suitable pallet lifting device of palletizer unit 10) is transposed to the next level of pallet PAL for (whole or partially) constructing the next level PL1, PL2 corresponding to the next level of pallet PAL. In one aspect, the palletizer / depalletizer unit 10B is configured to feed layers PL1, PL2, PL3, PL4, and PL5 such that the feeding is synchronized with or coordinated (e.g., matched) by a predetermined rate or feed rate of the box unit flow established by the warehouse management system 199 in the automated storage and retrieval system 100, in a manner substantially similar to that described above regarding the release of layers PL1, PL2, PL3, and PL4, wherein the warehouse management system 199 manages box unit retrieval orders, the sequence of mixed box unit outputs to the unloading sequence of mixed box unit pallet loads, and other related aspects of the outputs (e.g., inventory reconciliation).

[0048] Based on all aspects of this disclosure, and with reference to Figure 3 The palletizer / depalletizer unit 10A, configured to release layers PL1, PL2, PL3, PL4, and PL5, includes at least one robot 14 having a robot arm 12 coupled to a robot controller 316 (which is coupled to or forms part of a unit controller 10C). The robot arm 12 is in the form of a standard industrial articulated robot arm, adapted for releasing layers as described herein. In one aspect, the robot arm 12 includes six degrees of freedom, while in other aspects, the robot arm 12 may have more or fewer than six degrees of freedom. As used herein, the terms "robot" and "robot arm" are used interchangeably to refer to a programmable system comprising articulated and / or movable components capable of receiving, controlling, and moving tools. Figure 3As shown, a layer depalletizing tool or end effector 99 is coupled to the robotic arm 12 and configured to release layers PL1, PL2, PL3, PL4, and PL5 as described herein. In one aspect, the layer depalletizing tool 99 may be substantially similar to the tool described in U.S. Patent Application No. 14 / 720,089, filed May 22, 2015, entitled "Tool and Method for Layer Depalletizing," the entire disclosure of which is incorporated herein by reference.

[0049] refer to Figures 3 to 5 and Figure 8B The depalletizing tool 99 has a gripper 800 configured to grip and pick up at least one of the pallet load layers 816 (representing any one of pallet layers PL1, PL2, PL3, PL4, PL5) to transport at least one pallet load layer 816 from the pallet load PAL at the pallet unloading / loading station 301 to the output station 333 (in one aspect, the output station 333 includes any suitable conveyor, such as conveyor 150). The gripper 800 has a gripper engagement interface 810 defining a predetermined engagement position and orientation of at least one pallet layer 816 relative to the depalletizing tool 99 (e.g., in a robot coordinate system or space X, Y, Z, RX, RY, RZ, see [link]). Figure 3 (and also referred to herein as the robot reference frame), so that at least one pallet load layer 816 can be repeatedly captured and stably held using the gripper 800. It should be noted that the configuration of the gripper 800 described herein is a suitable example of a pallet layer gripping mechanism that can be used in various aspects of this disclosure; however, in other aspects, the gripper may have any suitable differential pressure gripper, airbag gripper, or other capture mechanism defining an engagement / capture system that docks with the pallet layer for transferring the pallet layer via the gripper. The pallet depalletizing tool 99 includes a frame 20, four side clamps 22-24 movably mounted to the frame 20 for clamping and releasing pallet layers 1PL1, PL2, PL3, PL4, PL5, two curtains 26 mounted below the clamps 22-24 to insert under the pallet layer 816 held by the clamps 22-24, and a top pad 28 mounted above the clamps 22-24 to the frame 20, wherein one or more of the four side clamps 22-24, the two curtains 26, and the top pad 28 form a clamp engagement interface 810. A predetermined pallet engagement position and orientation provides the engagement plane orientation of the clamp engagement interface 810 (e.g., defined at least partially by the top pad 28). The pallet position and orientation describe the flatness of the engagement surface 1210 of at least one (top) pallet layer 816 (which may coincide with the top surface 148) (see [link to relevant documentation]). Figure 12The mating surface 1210 is configured to abut against a gripper engagement interface 810 that substantially spans at least one pallet layer 816, and the layer positions and orientations depict a planar misalignment between the mating surface 1210 of at least one pallet layer 816 and the planar orientation of the gripper engagement interface 810 in at least two orthogonal directions (e.g., in a robot coordinate system or reference frame). A controller (e.g., robot controller 16 and / or unit controller 10C) is configured to resolve at least one of the planar misalignment and center point misalignment based on robot motion boundary conditions for optimal gripper engagement with each of the topmost pallet layers, said robot motion boundary conditions being defined by at least one of the robot architecture and structure defining the depalletizer 10 as depicted in a predetermined reference frame of robot 14.

[0050] Frame 20 includes two pairs of parallel walls 30-32, which are assembled to generally define a rectangular perimeter. Each wall 30 and 32 includes bottom rectangular portions 34 and 36 and overall triangular portions 38 and 40, respectively. The two triangular portions 40 are slightly curved toward each other. Frame 20 also includes two transverse rectangular hollow tubes 42 and 44, extending parallel to walls 30 between walls 32 and parallel to walls 30 between walls 32, respectively. Hole 49 (see...) Figure 8A The hollow tubes 42 and 44 are fixed near the top of the triangular portions 38-40 of the walls 30-32, allowing connectors and cables (not shown) to pass through the walls 30-32 and then through the hollow tubes 42 and 44. Mounting bracket 46 (see...) Figure 8A The frame 20 is secured to both hollow tubes 42 and 44 at their intersection, allowing the destacking tool 99 to be attached to the robotic arm 12. Frame components 30-44 are assembled using fasteners and / or welding, as are other parts of the destacking tool 99 mounted to the frame 20. The frame 20 is not limited to the above and may be provided with other components to mount the destacking tool 99 to the robotic arm 12 and to operably receive its additional components. It should be noted that the description of the frame 20 is merely exemplary, and in other respects, the frame may have any suitable construction and / or the destacking tool may be connected to the robotic arm in any suitable manner.

[0051] Also refer to Figure 6Each pair of opposing side clamps 22 and 24 are slidably mounted on their respective tracks 43 and 45, each track being fixed below the hollow tubes 42 and 44 for movement along them via mounting assemblies 48 and 50, respectively. Since mounting assemblies 48 and 50 are very similar, the mounting of one of the clamps 22 to the frame 20 will be described in more detail herein. Mounting assembly 48 includes a bracket 52 slidably mounted to the hollow tube 42 via track 43 below the hollow tube 42 and attached to the distal ends of rods 60 of the two actuators 56. Rod 54 securely connects the clamp 22 to the bracket 52. Rod 54 is thus mounted to the bracket 52 to extend vertically from the bracket 52 to the clamp 22. The clamp 22 is perpendicular to the hollow tube 42 in the plane including rod 54.

[0052] Two pneumatic actuators 56 are positioned between the hollow tube 42 and the support 52 to cause movement of the support 52, and thus also cause the clamps 22 to move along the hollow tube 42. More specifically, the body 58 of each actuator 56 is fixed to the hollow tube 42 on a corresponding lateral side, and the distal end of the actuator rod 60 is fixed to the support 52. The clamps 22 and 24 move in pairs in parallel to clamp the pallet layer from two opposite sides of the pallet layer, and then from the other sides. Thus, four (4) actuators 56 (fixed to the same hollow tube 42 or 44) are operated first, and then another four actuators 56 are operated simultaneously. According to another aspect of this disclosure, all clamps 22 and 24 are actuated simultaneously.

[0053] The dimensions and orientation of pallet layers 816 (representing any one of pallet layers PL1, PL2, PL3, PL4, PL5) are provided by a vision system 310 as described herein. Each pair of grippers 22 and 24 is movable between an extended position and a retracted position, in which gripper 22 or 24 applies pressure to pallet layer 816 from the respective opposite side. Although grippers 22 and 24 have different widths, grippers according to another aspect of this disclosure may have the same width. Grippers according to another aspect are pivotally mounted to a frame. According to other aspects, grippers have configurations other than those shown herein and are mounted to the frame in any suitable manner so that they can move differently relative to the frame.

[0054] Reference Figure 7 , 9A Both curtains 26, like 10B, are defined by a series of metal rollers 62 between rectangular portions 34 of the frame 20. The metal rollers 62 are slidably and rotatably mounted to the rectangular portions 34 of the frame 20. The two curtains 26 extend together along the full length of these portions 34, each curtain spanning half of that length. More specifically, each roller 62 has a hub 63 at its longitudinal end (see...). Figure 10BThe hub 63 is housed parallel to the inner surface of the frame 20 in a track 64. Each of the four (4) tracks 64 extends along the bottom edge 66 of the rectangular portion 34, from the center of the bottom edge 66 to the end of an arcuate portion 68 of the track 64, which rises toward the triangular portion 38 near the longitudinal end side of the wall 30. The arcuate portion 68 defines a curtain receiving portion when the curtain 26 is open. Note that in some figures, some rollers 62 are not shown for the sake of clarity.

[0055] Each curtain 26 includes a rotating front portion 70 at its front end, which is slidably mounted in and rotatably mounted to a track 64 together with a roller 62. The roller 62, together with the front portion 70 of each curtain 26, is moved along their respective tracks 64 by an actuator 72. The body 74 of the actuator 72 is fixed to the outer surface of the rectangular portion 34 of the wall 30, and the rod 76 of the actuator 72 is connected via a mounting bracket 78. Figure 9B and Figure 10B The actuator 72 is fixedly mounted to the front portion 70 so that the front portion 70, together with the rod 76, translates along a track 64 attached to the outer surface of the rectangular portion 34. When the curtain 26 is open, the actuator 72 pushes the front portion 70 along the track 64 and forces the roller 62 to move in the same direction. When the curtain 26 is closed, the actuator 72 pulls the front portion 70 along the track 64 in the same direction. The front portion 70 is in the form of a plurality of rotatable friction elements 80 extending along the front edge of each curtain 26. The friction elements 80 are in the form of rubber O-rings mounted on two parallel rollers 82 for rotating thereon. The two rollers 82 are rotatably mounted on the mounting brackets 78 between the mounting brackets 78.

[0056] from Figure 5 and Figure 7As can be better seen, the two rollers 82 of each curtain 26 are driven by their respective drive assemblies, which include a side chain 84 and a driver 90. The side chain 84 extends along the straight portion of the respective track 64 and is mounted to a respective driving sprocket 86 and a respective driven sprocket 87. The driven sprocket 87 is fixed at the end of each respective roller 82. The driver 90 is fixed to the wall portion 34 and operatively coupled to the two chains 84 via one of the driving sprockets 86. The chains 84 are positioned relative to the two rollers 82 via the sprockets 87 such that the rollers 82 and the associated chains (or belts) 84 rotate together. Thus, rotation of each driver 90 causes rotation of the friction element 80. The front portion 70 can rotate and slide simultaneously and independently along the track 64. An end roller 92 is provided between the two driven sprockets 88 of each curtain 26 to further simplify the movement of the front 70 along the track 64 by connecting the respective chains 85 located at the respective ends of the front 70, thereby ensuring that the front 70 remains perpendicular to the track 64. It should be noted that elements of the drive assembly, including the chains 84, are omitted in some figures for the sake of clarity.

[0057] For more specific reference Figures 4 to 6The top liner 28 and its actuation mechanism will now be described. The top liner 28 is in the form of a plate that is movable toward and away from the pallet layer 816 held by clamps 22-24. The top liner 28 is movable by mounting it to the frame 20 via the liner actuation assembly 5100. The top liner 28 (in the form of a plate) is made of a soft, elastic material (e.g., rubber or plastic) reinforced with metal tubing on its non-contact surfaces. Depending on other aspects, the top liner 28 may be made of a different material and / or unreinforced. The padding actuator assembly 5100 includes: four padding retainer shafts 102-108 rotatably mounted to the frame 20; eight padding retainer wheels 6110 mounted in parallel pairs on each of the shafts 102-108; four connecting elements 112, each connecting element 112 being fixed to the top pad 28 and the wheel 6110 of the corresponding shaft 102-108; four shaft pulleys 114-120, each shaft pulley being fixed near one end of each shaft 102-108; a top padding linear actuator 120 fixed to the frame 20; a pulley assembly 124 fixedly mounted to the frame 20; and four cables 126-132, each cable operably connecting its respective shaft pulley 114-120 to a movable end of the actuator 122. The four padding retainer shafts 102-108 are rotatably mounted to the hollow tubes 42-44 via four support brackets 134. Four shafts 102-108 are positioned relative to each other in a square configuration, connected end to end. Each of the connecting elements 112 is fixed to a corresponding pair of wheels 6110 for partial winding on the wheels and securely fixed to the top liner 28 via attachment 136. The pulley assembly 124 includes: a support 138 fixed to the hollow tube 42 for positioning at a movable end of the actuator 122; and three (3) intermediate pulleys 140-144 rotatably mounted below the support 138. Figure 5 As can be better seen in the diagram, the position and orientation of the intermediate pulleys 140-144 and the actuator allow the use of cables 126-132 to connect the four shaft pulleys 114-120 to the actuator rod 146.

[0058] In operation, the top pad 28 is raised by retracting the actuator 122's rod 146 (which simultaneously pulls cables 126-132). This causes shafts 102-108 to rotate simultaneously, thereby generating a pulling force on the connecting element 112 that raises the top pad 28. The opposite effect is achieved by extending the actuator rod 146.

[0059] As described above, refer to Figure 1 , Figure 3 and Figure 11A-11FPalletizer unit 10A includes a vision system 310. Vision system 310 includes at least one camera 310C mounted to palletizer unit 10 independently of robot 14 to maximize the throughput of robot 14 (i.e., while robot 14 is placing a previously picked pallet layer onto conveyor 150, vision system 310 captures and analyzes the next pallet layer). At least one camera 310C is (or includes) any suitable three-dimensional image sensor configured to generate one or more of two-dimensional images, two-dimensional depth maps, and three-dimensional point clouds. In one aspect, vision system 310 includes one camera 310C; in other aspects, vision system 310 includes three (3) cameras 310C1, 310C2, 310C3; and in still other aspects, vision system 310 includes six (6) cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6. In still other aspects, vision system 310 includes any suitable number of cameras. When multiple cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6 are provided, two or more of cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6 can be arranged in more than one vertical hierarchy 387L1 and 387L2. Figure 3 and 11A -11F). For example, in one aspect, cameras 301C1, 31C2, and 301C3 are positioned on a common level 387L1, and cameras 301C4, 301C5, and 301C6 are positioned on another common level 387L2 so that they are at a different height (e.g., relative to the pallet load PAL) than cameras 301C1, 31C2, and 301C3, thereby arranging multiple cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6 at different heights / levels for imaging the pallet load PAL. In other aspects, multiple cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6 may be arranged at a single level or in any suitable number of levels. In other respects, cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6 can also be mounted on a movable platform (independent of robotic arm 14) to raise and / or lower cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6 in order to image each pallet layer in a manner that provides substantially complete 360° coverage of the pallet load building structure RPAL (as described herein).

[0060] In all aspects described herein, at least one camera 310C is positioned such that the three corners of the pallet load PAL are directly within the fields of view (FOV1-FOV6) ​​of at least one camera 310C. For example, in one aspect, cameras 310C1-310C3 on level 387L1 point to the three corners, while there is no corresponding camera at the fourth corner; in another aspect, cameras 310C1-310C3 on level 387L1 point to the three corners, cameras 310C4-310C6 point to the three corners, while there is no corresponding camera at the fourth corner; and in other aspects, cameras 310C1-310C6 may be arranged such that all four corners of the pallet load PAL are directly within the fields of view (FOV1-FOV6). In one aspect, at least one camera 310C is arranged such that the respective fields of view (FOV1-FOV6) ​​of at least one camera 310C extend from the uppermost surface 148 of the pallet load PAL (…). Figure 12 The pallet load PAL is covered to the bottom of the pallet load. In one aspect, at least one camera 310C is arranged such that the uppermost surface 148 of each pallet layer faces upward. Figure 12 The camera 310C is located within the respective fields of view (FOV1-FOV6) ​​of at least one camera 310C. As described above, at least one camera 310C can be arranged at a single level, multiple levels, and / or on a movable platform, such that the uppermost upward-facing surface 148 of each pallet layer... Figure 12 Within their respective fields of view FOV1-FOV6 and / or from the uppermost face of the pallet load PAL at surface 148 ( ) Figure 12 The bottom of the pallet load is covered by the pallet load PAL.

[0061] It should be noted that each camera 310C1-310C6 is arranged at a vertical angle of approximately 45° (or other suitable vertical angle greater than or less than approximately 45°) and a horizontal angle of approximately 45° (or other suitable horizontal angle greater than or less than approximately 45°) relative to the pallet load PAL, so as to provide the pallet top surface 148 and all four outermost vertical (lateral) sides of the pallet load PAL (or at least more than three vertices / corners formed by the intersection of the lateral sides) within the field of view FOV1-FOV6, for example, even in the case where only three corners fall directly into the field of view FOV1-FOV6; while in other cases, each camera in at least one of the cameras 310C can be arranged at any suitable vertical and / or horizontal angle relative to the pallet load PAL. Placing cameras at different heights (e.g., at different levels 387L1, 387L2) provides imaging of different portions of the pallet load using cameras at each level (e.g., lower level 387L2 can image the lower half of the pallet load PAL, while higher level 387L1 can image the upper half of the pallet load PAL); in other aspects, a single level of camera can image the entire pallet load PAL from top to bottom. In one aspect, at least one camera 310C (e.g., cameras 310C1-310C6) can have any suitable focal length for a predetermined image intensity. In one aspect, the respective fields of view FOV1-FOV6 of each of the at least one camera (e.g., cameras 310C1-310C6) are (see...) Figure 11A-11F In some aspects, the respective fields of view (FOVs) of at least one camera 310C are greater than or less than approximately 45°, provided that the three corners of the pallet load PAL fall directly within the field of view FOV1-FOV6 of the camera. In other aspects, the properties of the respective fields of view(s) of each of the at least one camera 310C are the same (e.g., each camera has a field of view of approximately 45°); in other aspects, the properties of the respective fields of view(s) of one or more of the at least one camera 310C may be different (e.g., one or more cameras may have a field of view other than approximately 45°).

[0062] At least one camera 310C is connected to the unit controller 10C, notifies the unit controller 10C, and enables the unit controller 10C to issue movement commands to the robotic arm 12 (or a general robot 14) to guide the robotic arm 12, for example, using real-time (or near-real-time) command input that responds in real-time to changes in pallet load, so that the robotic arm 12 can address changes in pallet load that affect depalletizing in real time. For example, at least one camera 310C enables the unit controller 10C to issue commands to the robotic arm 12 to move and guide the depalletizing tool 99 to pallet layers PL1, PL2, PL3, PL4, PL5. As described herein, vision system 310 provides the position (X, Y, Z in a robot coordinate system or reference system) and / or orientation (RX, RY, RZ in a robot coordinate system or reference system) of the top pallet layer (e.g., layer 816), or determines the position (X, Y, Z in a robot coordinate system or reference system) and / or orientation (RX, RY, RZ in a robot coordinate system or reference system) of the top pallet layer (e.g., layer 816). In one aspect, vision system 310 provides the length L and width W of the top layer, or determines the length L and width W of the top layer (see [link to documentation]). Figure 12 ).

[0063] Commands issued by the unit controller 10C (based on image data from at least one camera 310C) enable the stack unloading tool 99 to perform positional / spatial adjustments relative to pallet layers PL1, PL2, PL3, PL4, PL5 in multiple degrees of freedom (e.g., planar and rotational degrees of freedom) to accommodate skewed pallet layers (e.g., pallet layers rotating relative to pallet support SPAL and / or other pallet layers), offset pallet layers (e.g., pallet layers with edges extending beyond a virtual vertical pallet plane / boundary established by the outer edge of the pallet support SPAL), uncovered carton units CU (e.g., carton units such as cardboard boxes where one or more flaps of the box are open), pallet loads / layers exceeding tolerances, incomplete layers (e.g., missing carton units), etc. The vision system 310 is configured to provide data to the unit controller 10C in order to guide and position the depalletizing tool 99 to the optimal pick-up position and orientation (e.g., compared to other possible pick-up positions and orientations) based on the actual position and orientation of the top layer in the robot 14 coordinate system / reference system (the terms coordinate system and reference system are used interchangeably herein).

[0064] Commands issued by the unit controller 10C (based on image data from at least one camera 31C) also enable obstacle avoidance, for example, where an unexpected object is located within or otherwise enters the palletizer unit 10A, where the edge or side of the pallet load PAL is at substantially the same height as the depalletizer tool 99 or other parts of the robot 14, where the pallet load PAL is adjacent to a robot restricted area (e.g., a predetermined area within the palletizer unit 10 where the robot 14 is not allowed to enter), etc. The vision system 310 is configured to detect unexpected objects, the sides / edges of the pallet load PAL, the distance between the pallet load PAL and the robot prohibition zone, etc., and send data signals to the unit controller 10C, causing the unit controller 10C to command the robot arm 12 to move the depalletizing tool 99 around the unexpected object, around the side of the pallet load, between the pallet load PAL and the robot prohibition zone (or other obstacles adjacent to the pallet load PAL), and / or in any other suitable manner to pick up pallet layers PL1, PL2, PL3, PL4, PL5, or otherwise command the robot arm 12 to stop moving.

[0065] As described above, the unit controller 10C (or other suitable controller, such as robot controller 316) receives image data (e.g., from 2D images, 2D depth maps, and / or 3D point clouds) from the vision system 310 for analysis to detect pallets, pallet layers, and / or analyze their features. Also as described above, the image data provided to the unit controller 10C is provided in the coordinate system or reference system of the robot 14, wherein the vision system 310 is calibrated to / registered to the coordinate system or reference system of the robot 14. It should be noted that each camera 310C1-310C6 is inherently calibrated to its own coordinate system (i.e., each camera knows the depth of the object from its corresponding image sensor). In the case of employing multiple cameras 310C1-310C6, in one aspect, the calibration of the vision system 310 includes the calibration of the cameras 310C1-310C6 relative to a common base reference frame and the calibration of the common base reference frame relative to a robot reference frame; while in other aspects of employing one or more cameras, the reference frame of one camera or the reference frames of one or more cameras 310C1-310C6 can be individually calibrated to the robot reference frame.

[0066] For illustrative purposes only, see reference. Figure 3 , Figure 13A and Figure 13BThe calibration of cameras 310C1-310C6 relative to a common fundamental reference frame involves identifying and applying transformations between the respective reference frames of each camera 310C1-310C6, such that the respective reference frame of each camera 310C1-310C6 is transformed (or associated) with a common fundamental reference frame. This common fundamental reference frame can be the reference frame of a single camera or any other suitable fundamental reference frame that each camera 310C1-310C6 can be associated with, so as to form a common fundamental reference frame for all cameras in the vision. For example, the reference frame of camera 310C1 (although any camera can be used) represents the common fundamental reference frame. A transformation (i.e., a rigid transformation over six degrees of freedom) relative to the reference system of camera 310C1 is determined for each of the coordinate systems / reference systems of other cameras 310C2, 310C3, 310C4, 310C5, and 310C6, such that image data from cameras 310C2, 310C3, 310C4, 310C5, and 310C6 are associated with or transformed into the reference system of camera 130C1. This calibration of cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6 can be performed using a calibration holder / fixture 1300 (also known as a common camera calibration reference structure) placed at the pallet unloading / loading station 301. As described herein, at least one of the cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6 is calibrated to a common camera calibration reference structure, and the calibration of the common camera calibration reference structure depicts the positional relationship between the respective camera reference frame of each of the corresponding cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6 and each of the other cameras 310C1, 310C2, 310C3, 310C4, and 310C6.

[0067] The calibration fixture 1300 includes uniquely identifiable three-dimensional geometries 1310-1319 (in this example, squares, some of which are rotated relative to other squares) that provide an asymmetric pattern for the calibration fixture 1300 and constrain the determination / transformation of the camera's reference frame (e.g., from each camera) to a common base reference frame, as well as the transformation between the common base reference frame and the robot's reference frame (as will be described further), in order to determine the orientation of the pallet layer relative to the gripper interface. The calibration fixture 1300 shown and described herein is exemplary, and any other suitable calibration fixture can be employed in a similar manner to that described herein. For illustrative purposes, each of the three-dimensional geometries 1310-1319 has a predetermined size that constrains the identification of corners or points C1-C36 of the three-dimensional geometries 1310-1319, and the transformation minimizes the distance between the corresponding corners C1-C36 (e.g., the distance between the corners C1-C36 in the reference frame of camera 310C1 is minimized relative to each of the corners C1-C36 identified in the reference frame of each camera 310C2-310C6).

[0068] Each of the three-dimensional geometries 1310-1319 is imaged simultaneously (i.e., each of the three-dimensional geometries 1310-1319 is at a single location in the common base reference frame during imaging by all the cameras (whose reference frames will be calibrated to a common base reference frame), and is uniquely identified by each of the cameras 310C1-310C6 at that single location, such that the points / corners C1-C36 of the three-dimensional geometries 1310-1319 identified in the image (an exemplary image in...) Figure 13BThe corners C1-C36 (shown in the diagram) are identified by vision system 310 (in any suitable manner, such as the manner described herein regarding the determination of the pallet layer corners PC1-PC4) and are uniquely determined independently of the orientation of the calibration fixture. It is important to note that the uniquely identified corners C1-C36 are located at a single position in the robot reference frame, which is common to all cameras when the image set is captured. This robot reference frame corresponds to each set of images captured by cameras 310C1, 310C2, 310C3, 310C4, 310C5, and 310C6 (the image set is a collection of images captured by each camera at a given position of the calibration fixture or pallet load). The corners C1-C36 identified in each image of the image set are compared between these images to define the transformation of each camera reference frame to a common fundamental reference frame (in one example, which may correspond to or be defined by the reference frame of camera 310C1). It is important to note that the calibration fixture 1300 is configured to identify points within the entire working volume of the pallet unloading / loading station 301 (corresponding to the dimensions of the pallet load PAL). For example, the three-dimensional geometry 1310-1319 can span the XY plane of the pallet unloading / loading station 301, and the calibration fixture can image at various varying heights throughout the entire working volume of the pallet unloading / loading station 301.

[0069] When registering all cameras 310C1-310C6 to a common base reference system, the common base reference system (or the individual reference systems of one or more cameras) is transformed (e.g., registered) to the robot reference system (X, Y, Z, RX, RY, RZ) by mounting the calibration fixture 1300 (or a similar fixture) to the robot 14. The calibration fixture can be mounted to the robot 14 such that the three-dimensional geometry 1310-1319 (and its corners C1-C36) is aligned with the gripper engagement interface of the gripper 800. Figure 8B (or other suitable reference planes or positions of robot 14) have known predetermined spatial relationships (e.g., position, flatness, orientation, etc.). Robot 14 can be commanded to move along one or more of the X, Y, and Z axes in a predetermined motion (where the calibration fixture 1300 is held), while being imaged by one or more cameras 310C1, 310C2, 310C3, 310C4, 310C5, 310C6. Unit controller 10C can compare the identified corners C1-C36 in the images obtained using vision system 310 with, for example, encoder data (which describes the motion path of robot 14, i.e., the motion of the fixture implemented by robot 14), and generate from these images (based on a common base reference frame) the gripper engagement interface of the identified corners C1-C36 relative to gripper 800. Figure 8B(or other suitable reference plane or position of robot 14) planar orientation and position (in six degrees of freedom - attitude and orientation). The generated identified corners C1-C36 relative to the gripper engagement interface of gripper 800 ( Figure 8B The planar orientation and position (in six degrees of freedom – pose and orientation) of the image field of the common base reference frame (or other suitable reference plane or position of robot 14) characterize the relationship between the image field of robot 14 and the reference frame of the common base reference frame, such that the positions of corners C1-C36 in the vision system image are calibrated to the robot reference frame. This calibration can be performed once during palletizing unit 10 loading or at any suitable time interval (or each initialization of robot 14, e.g., after power-off), and allows the image data of vision system 310 to be represented in the same coordinate system / reference frame as robot 14.

[0070] In one aspect, at least one camera 310C resolves the three-dimensional boundary of box unit features (e.g., edges and corners of box units) from two or more orthogonal planes, such that maximum certainty of the feature pose (e.g., X, Y, Z, RX, RY, RZ positions of the pallet layer or features of the calibration fixture 1300) is obtained from a single image of an item in the respective fields of view(s) FOV1-FOV6 of at least one camera 310C. Here, the resolution of the three-dimensional boundary of the pallet layer and / or the features of the calibration fixture 1300 are independent of the placement of the camera 310C and are performed in real time (e.g., within a pick-up / placement cycle of at least one robot 14).

[0071] Although six (6) cameras 310C1-310C6 are described, it should be understood that more or fewer than six (6) cameras 310C1-310C6 may be used or placed such that the field of view of the vision system 310 (multiple) cameras 310C1-310C6 covers the pallet unloading / loading station 301 of the pallet unit 10, the pallet support SPAL located on the pallet unloading / loading station 301, and the entirety (or at least a predetermined portion) of the intended pallet load building structure RPA at the pallet unloading / loading station 301, thereby capturing three-dimensional time-of-flight images of (multiple) objects at any desired location on the pallet load building structure RPA at any suitable desired resolution. The combined field of view (multiple) FOV1-FOV6 results in substantially complete 360° coverage of the pallet load building structure RPA by utilizing the overlap of the field of view (multiple) FOV1-FOV6. For example, the combined field of view (FOV1-FOV6) ​​can cover a standard pallet support SPAL (with dimensions such as 48 inches × 48 inches, 48 ​​inches × 40 inches, and / or 36 inches × 36 inches). It should be understood that the combined cameras (310C1-310C6) and associated field of view (FOV1-FOV6) ​​can appropriately cover (e.g., image it) a larger field of view (including, for example, a truck floor or any desired field size). Furthermore, the combined field of view (FOV1-FOV6) ​​can cover any suitable pallet load construction structure RPAL height PH (see [link to documentation]). Figure 2 For example, heights of 60 inches, 70 inches, and 80 inches; while in other respects, the fields of view FOV1-FOV6 can cover heights less than 60 inches or greater than 80 inches.

[0072] In one aspect, each of cameras 310C1-310C6 may have a resolution of 176 pixels × 132 pixels; while in other aspects, each or more of cameras 310C1-310C4 may have a higher resolution (e.g., 320 pixels × 240 pixels or higher) to provide a desired minimum depth map of approximately 0.5 inches at the outermost boundary of the pallet structure's three-dimensional space as needed (so that the depth map sharpness in the captured image of the entire or predetermined portion of the pallet support / pallet structure is not less than approximately 0.5 inches). Thus, the vision system 310 provides sufficient resolution to resolve features of at least the top surface 148 of the pallet load PAL, allowing the flatness of the top surface 148 across the pallet load PAL to be determined and fully established for removing pallet layers PL1-PL5 from the pallet load PAL. Sufficient resolution may also be provided to resolve box unit features (e.g., such as box edges), allowing the flatness of each layer PL1-PL5 (see...) to be determined. Figure 2The flatness of the top is determined and fully established in order to release layers PL1-PL5. The resolution of the (multiple) cameras 310C1-310C6 can be such that minimal processing is required to distinguish box unit features (e.g., the edges and corners of box units in the pallet layer), so that the box unit features are essentially distinguished in real time from the images received by the unit controller 10C.For example, the corners PC1-PC4 of each pallet layer PL1-PL5 can be determined as follows: by imaging at least the top surface 148 of the corresponding pallet layer using the vision system 310 and determining the positions of the corners PC1-PC4 of the pallet layers PL1-PL5 in any suitable manner, such as by using the Ramer-Douglas-Peucker algorithm, see “An iterative procedure for the polygonal approximation of plane curves” published in Computer Graphics and Image Processing, Vol. 1, No. 3, November 1972, pp. 244-256; and “The Contours, Corners and T-Junctions Detection” by Buades et al. published in Online Image Processing (©2018), ISS No. 2105-1232, February 27, 2018. Algorithm; or determine the positions of the corners PC1-PC4 of pallet layers PL1-PL5 in any other suitable manner, such as as described in the following documents: "Improved Global and Local Curvature Properties for Shape Corner Detection" by Suraya Abu Bakar, Muhammad Suzuri Hitam, and Wan NuralJawahir Hj. Wan Yusso, Journal of Applied Sciences, Vol. 17, pp. 458-466, December 2017; "The Comparison and Application of Corner Detection Algorithms" by Chen Jie et al., Journal of Multimedia, Vol. 4, No. 6, December 2009; and "Robust Corner Detection by Image-Based Direct Curvature Field Estimation for Mobile Robot" by Sungho Kim, International Journal of Advanced Robotics Systems, Vol. 9, No. 187, 2012. Navigation (DOI:10.5772 / 53872); all of their public content is incorporated herein by reference.

[0073] Now for reference Figure 3 , Figure 2 , Figure 8B and Figure 12In one aspect, the unit controller 10C is configured to determine in real time, based on corresponding real-time three-dimensional imaging data, the pallet layers PL1-PL5 being released relative to the gripper 800. Figure 8B The clamp engagement interface 810 ( Figure 8B The unit controller 10C is further configured to generate articulated robot motion signals in real time based at least on the real-time determined layer flatness differences PSV1, PSV2. These articulated robot motion signals are generated in real time so that the at least one articulated robot 14 can execute the articulated robot motion signals in real time between placing a released pallet layer PL1-PL5 and placing consecutively released pallet layers PL1-PL5, thereby achieving substantially continuous release of the pallet load PAL. In one aspect, the at least one articulated robot motion signal generated by the unit controller 10C is: a stop motion signal along the pick / placement paths 399, 1580 of the at least one articulated robot 14; a slow motion signal along the pick / placement paths 399, 1580 of the at least one articulated robot 14; or a movement to a safe position along a safe stop path 398 of the at least one articulated robot 14, wherein the safe stop path 398 is different from the pick / placement paths 399, 1580. In one aspect, the articulated robot motion signal generated by the unit controller 10C is a pick-up position signal, which sets the pick-up position of the depalletizing tool 99 based on the planarity differences PSV1 and PSV2 of the released pallet layers PL1-PL5.

[0074] The unit controller 10C is configured to determine, in real time, the positions of the layer flatness differences PSV1, PSV2 and pallet layers PL1-PL5, as well as the released layer pose PSV3 (RZ) and position (X, Y), based on corresponding real-time 3D imaging data. For example, the vision system 310 images the top surface 148 of layers PL1-PL5 to obtain a 3D image of the top layer of the pallet load PAL, which has sufficient sharpness to distinguish the sides, corners, and flatness of the top surface 148, as described above. Here, the pallet support differences PSV1, PSV2 can be one or more unevenly spaced box units CU (e.g., the space between box units CU in a pallet layer forming peaks / valleys in the box unit seat surface). Figure 9A Missing container units (CUs) in the pallet layer, height differences (e.g., protrusions and / or depressions) Figure 9AThis includes any other defects in the pallet layer that may affect the pallet depalletizing tool 99's ability to hold the pallet layer. In one aspect, the unit controller 10C is configured to reject pallet layer picking (and send a stop robot signal until it is replaced) if the pallet support differences PSV1, PSV2 exceed a threshold from a predetermined reference (e.g., the plane defined by the top pad 28). For example, if the missing box unit CU of the pallet layer is larger than a predetermined area, or if the spacing between box units CU in the pallet layer is greater than a predetermined distance, pallet layer picking is rejected, and the pallet layer will not be picked up until the defects in the pallet layer are resolved (e.g., by human intervention). If the pallet layer is within the predetermined threshold, the unit controller 10C is configured to solve for the pallet layer planar differences (e.g., the position of the pallet layer in the three-dimensional robot space X, Y, Z, RX, RY, RZ) and, based on these differences, confirm or modify (compensate) the planned robot picking / placement path for the adaptive posture of the pallet depalletizing tool 99 with a higher probability of successful pallet layer picking. The controller can also identify a decrease in the moving speed of robot 14 or modify the placement paths 399 and 1580 of robot 14 and the corresponding path trajectories. Figure 3 ), to generate the desired layer unpacking tool 99 pick pose (e.g., position in three-dimensional space X, Y, Z, RX, RY, RZ).

[0075] In one aspect, the unit controller 10C is configured to set the pallet layer reference plane DTM (Difference Model of Pallet Layer) imaged by at least one three-dimensional camera 310C based on pallet support differences PSV1, PSV2. Figure 12 The pallet layer reference plane (DTM) distinguishes local surface differences at the placement positions of each different item unit in the pallet layer and defines a real-time position reference for the articulated robot 14 to pick up the pallet layer. On the one hand, the pallet layer reference plane (DTM) defines the flatness of the top surface 148 of the pallet layer.

[0076] refer to Figure 3 , Figure 12 and Figure 14 This section will describe an exemplary operation of the palletizer unit 10 in the depalletizing configuration. Note that reference... Figure 14 The operation boxes described therein do not necessarily specify a particular order of operations, and the operations identified by the operation boxes can be performed in any suitable order. For example, boxes 1401 and 1410 can be performed at any time before the robot 14 picks up the path / trajectory.

[0077] During the operation of palletizer unit 10, the pallet load PAL is transported and positioned on unloading / loading station 301 in any suitable manner (e.g., as described above). In exemplary operation of palletizer unit 10, as described herein, at any suitable time before and / or during operation of palletizer unit 10, image sensors or cameras 310C are optionally registered with a robot reference frame (X, Y, X, RX, RY, RZ). Figure 14 (Frame 1401). The pallet load PAL is imaged by at least one camera 310C, such that an image of the top surface 148 of the top pallet layer 816 is captured. Figure 14 (See box 1405). Image data from each of at least one camera 310C is converted from the respective camera reference frame to the robot 14 reference frame, for example, in a manner described herein. Figure 14 (See box 1410). It should be noted that, in one aspect, registration of the camera 310C's field of view is optionally performed as part of the operation before converting the camera image data to the robot 14 reference frame. It should also be noted that the conversion of the camera image data to the robot 14 reference frame can be performed at any suitable time before determining the robot 14's pickup trajectory, for example, in... Figure 14 The exemplary time shown. The pallet layer(s) reference reference planes are based on image data in the camera 310C reference frame and / or robot 14 reference frame ( Figure 14 The pallet(s) are determined by (frames 1415). The pallet(s) reference planes are any suitable geometric features of the pallet (e.g., corners of pallet layers, corners of box units within pallet layers, the outermost surface of the pallet layer, the vertices of the outermost surface, the orthogonality of the outermost surface, the position of the sides, etc.) that identify or otherwise define the pallet's attitude and position in the camera 310C reference frame and / or the robot 14 reference frame. For example, in one aspect, the reference planes are the corners PC1-PC4 of pallet layer 816, wherein the corners PC1-PC4 are determined by the unit controller 10C in any suitable manner (e.g., any suitable image analysis angle discovery algorithm as described above) based on image data obtained from at least one camera 310C in one or more of the camera 310C reference frame and the robot 14 reference frame. In one aspect, the corners PC1-PC4 of the pallet layer are determined from image data of each of at least one camera 310C; while in other aspects, the image data from the cameras may optionally be combined (frames 1415). Figure 14 (Box 1420) to determine the corners of the pallet layer. For example, in the case of combining image data from at least one camera 310C, a single point cloud 1270 including at least a portion of the pallet load PAL of the pallet layer 816 is generated by combining the image data from each of the at least one camera 310C with the unit controller 10C.

[0078] The unit controller 10C is configured to fit plane 1200 to (multiple) reference planes of the pallet layer based on image data from one or more of at least one camera 310C, in any suitable manner (e.g., using the Random Sample Consensus (RANSAC) algorithm, an organized segmentation algorithm (or other suitable algorithm for segmenting organized point cloud data), or any other suitable algorithm). Figure 14 (frame 1425). In one aspect, plane 1200 corresponds to and defines the top surface 148 of pallet layer 816 in the reference frame of camera 310C and / or robot 14. In one aspect, the positions of corners PC1-PC4 are determined by projecting a single point cloud 1270 onto plane 1200 using unit controller 10C. Figure 14 (Box 1430), the positions of the corners PC1-PC4 of the pallet layer 816 in the robot reference frame can be optionally verified. Using the unit controller 10C, the positions are determined based on image data from one or more of at least one camera 310C in any suitable manner (e.g., by any suitable blob analysis technique (center of gravity, principal axis, minimum Freret, etc.)). Figure 14 The orientation of the PSV3 and dimensions (length L and width W) of the pallet layer 816 (frame 1435) are as follows.

[0079] Using the (multiple) reference planes accurately known based on the above operations (in this example, corners PC1-PC4, and therefore length L and width W) and the plane 1200 established and characterizing the top surface 148 of the pallet layer 816, the unit controller 10C confirms the flatness of the top surface 148 of the pallet layer 816 relative to the plane 28P of the gripper engagement interface 810 (e.g., defined by the top pad 28). Figure 14 (See box 1440). Confirmation of the flatness of the top surface 148 can be performed by the unit controller 10C using any suitable image analysis thresholding technique to determine whether the depalletizing tool 99 is capable of picking up pallet layer 816. For example, a flatness deviation of plane 1200 relative to plane 28P exceeding a predetermined threshold (e.g., in one aspect, the deviation about the X-axis and / or Y-axis can be as high as about 5° or as high as about 10°; in other aspects, the deviation can be greater than about 10°) prevents the depalletizing tool 99 from picking up pallet layer 816. In the event that the depalletizing tool 99 is unable to pick up pallet layer 816, the unit controller 10C is configured to issue any suitable audible and / or visual alarm to the operator to resolve the pallet layer picking problem.

[0080] Also refer to Figures 15A to 15CThe unit controller 10C is configured to determine the movement trajectory and path of the robot 14 in any suitable manner, which provides or otherwise determines the optimal pick-up position of the depalletizing tool 99 relative to the pallet layer 816. Figure 14 (Box 1445) is used for picking up / removing pallet layers 816 from pallet load PAL. The optimal pick-up position can be determined by the unit controller 10C during the placement cycle of previously picked pallet layers to achieve substantially continuous removal of pallet load PAL. The optimal pick-up position is the position of the depalletizer 99 relative to the pallet layer 816, where the distance between the center 99C of the depalletizer 99 and the center 816C of the pallet layer 816 is minimized, while satisfying the physical constraints of the depalletizer 99 and the palletizing unit 10. For illustrative purposes only, the physical constraints of the depalletizing tool 99 include, but are not limited to: the gripping area 1520 of the gripper interface (in which the pallet layer 816 is inserted) defined here by, for example, the gripper length LT and the gripper width WT, wherein the gripper 800 is in an open configuration; the flatness deviation between the side grippers 22-24 and the (vertical) sides of the pallet layer (this deviation constraint may be similar to the flatness deviation of plane 1200 relative to plane 28P); the flatness deviation between the top surface 148 plane 1200 and the top pad 28 plane 28P; pallet layer 816 picking will be performed such that the depalletizing tool 99 does not contact the pallet load PAL (except for gripping the pallet layer 816), etc. Also for illustrative purposes, the physical constraints of the palletizing unit 10 include, but are not limited to, prohibited areas 1500-1502 that impede the movement of the robot 14 (i.e., the robot 14 and the depalletizing tool 99 carried by the robot 14 are not allowed to enter prohibited areas 1500-1502).

[0081] For illustrative purposes only, the optimal pickup location may be determined by the unit controller 10C, which finds the pallet layer center 816C based on the pallet layer characteristics determined above (e.g., the positions of corners PC1-PC4; length L; width W, etc.). The unit controller 10C uses known dimensions of the depalletizing tool 99 (e.g., external dimensions and clamping length LT and clamping width WT) to determine the position of the depalletizing tool 99 that minimizes the distance between the center 99C and 816C, while avoiding forbidden zones 1500-1502 and satisfying the physical constraints of the depalletizing tool 99. The unit controller 10C is configured to determine the feasibility of the optimal pickup location relative to the constraints mentioned herein. Figure 14 (Frame 1450).

[0082] Figure 15AThis is an exemplary illustration in which pallet layer 816 has a uniform box distribution to form a complete pallet layer; however, in other respects, pallet layer 816 may have any suitable non-uniform or incomplete box unit distribution, such as Figures 16A-16C As shown. The pallet layer is centered relative to the center 301CN of the pallet unloading / loading station 301, such that the optimal pickup position of the pallet depalletizing tool 99 is when its center 99C coincides with the center 816C of the pallet layer 816. Here, there is space between the pallet depalletizing tool 99 and the restricted areas 1500-1502, and space between the gripper 800 and the pallet layer 816, so that the pallet layer 816 can be inserted into the gripping area 1520 (e.g., this pickup is confirmed as feasible by the unit controller 10C).

[0083] Figure 15B This is an exemplary illustration in which pallet layer 816 has a uniform box distribution to form a complete pallet layer; however, in other respects, pallet layer 816 may have any suitable non-uniform or incomplete box unit distribution, such as Figures 16A-16C As shown. In this example, the center 816C of pallet layer 816 is offset from the center 301CN of pallet unloading / loading station 301. Figure 15B In the example shown, there is space between pallet layer 816 and restricted areas 1500, 1501, 1502, into which the pallet depalletizing tool 99 can be inserted, and there is space between pallet layer 816 and gripper 800, so that pallet layer 816 can be inserted into gripping area 1520 (e.g., the pickup is confirmed as feasible by unit controller 10C).

[0084] Figure 15C This is an exemplary illustration in which pallet layer 816 has a uniform box distribution to form a complete pallet layer; however, in other respects, pallet layer 816 may have any suitable non-uniform or incomplete box unit distribution, such as Figures 16A-16C As shown. In this example, the center 816C of pallet layer 816 is offset from the center 301CN of pallet unloading / loading station 301. Figure 15C In the example shown, there is space between pallet layer 816 and restricted areas 1500, 1501, and 1502, into which the pallet unloading tool 99 can be inserted; however, Figure 15C An extreme example is shown where pallet layer 816 is positioned at pallet loading / unloading station 301, where the clearance between the two sides of gripper 800 and pallet layer 816 is minimal (along the Z-axis). Similarly, in Figure 15C In the example shown, the optimal pick-up position for the depalletizing tool 99 is such that, considering the spatial constraints between pallet layer 816 and each restricted area 1500, 1501, the centers 99C, 816C are offset but the distance between the centers 99C, 816C is minimized.

[0085] Figure 15D This is an exemplary illustration in which pallet layer 816 has a uniform box distribution to form a complete pallet layer; however, in other respects, pallet layer 816 may have any suitable non-uniform or incomplete box unit distribution, such as Figures 16A-16C As shown. The center 816C of pallet layer 816 is offset from the center 301CN of pallet unloading / loading station 301. Figure 15D In the example shown, there is space between pallet layer 816 and restricted areas 1500, 1501, 1502; however, gripper 800 cannot be positioned to place pallet layer 816 within gripping area 1520 without entering restricted area 1500 (e.g., this pickup is deemed infeasible by unit controller 10C). Figure 15D In the example shown, there is no optimal pick-up position for the depalletizing tool 99, pallet picking is aborted, and the unit controller 10C provides an audible and / or visual alarm to the operator to resolve the pallet picking problem.

[0086] In various aspects of this disclosure, the unit controller 10C is configured to perform feature analysis on the palletizing unit 10 and the objects therein. Figure 14 (frame 1450) (using any suitable image analysis technique). For example, in one aspect, vision system 310 is configured to identify pallet racks SPAL at the bottom of a pallet load PAL. In some cases, when a pallet rack is defective, another pallet rack is placed under the pallet load PAL (i.e., such that the pallet load PAL comprises two or more pallet racks stacked on top of one), allowing the pallet load to be handled, for example, by a forklift. Vision system 310 is configured to identify stacked pallet racks SPAL such that the other pallet rack (the bottommost pallet rack) is not picked up by robot 14. In another aspect, vision system is configured to determine whether a portion of a pallet layer (where the box units do not span the entire pallet area – length L and width W) provides sufficient support for being picked up by the layer depalletizing tool 99. For example, see reference 1450. Figures 16A-16C Exemplary partial pallet layers 1601, 1602, and 1603 are shown (which can be replaced) Figures 15A-15D The pallet layer 816 shown in the figure, and in some respects indicated Figures 15A-15DThe pallet layer 816 shown is adapted to be picked up by the pallet depalletizing tool 99, such that the clamping force applied by the grippers 22-24 is applied substantially uniformly along the respective grippers 22-24. Pallet layers that may be unsuitable for picking include those that cause the clamping force applied by the grippers 22-24 to be applied inconsistently, thereby generating torque on the grippers 22-24 that may cause some box units CU to be clamped while others are not. In other aspects, the vision system 310 is configured to identify box units CUF that have fallen from the pallet load PAL at the depalletizing unit 10 to a location (on the floor, on the conveyor 150, etc.) to generate audible and / or visual alarms for operator intervention. The vision system 310 may also be configured to determine the “retry” trajectory of the robotic arm 14 where a missed pick-up has occurred. For example, in the event that the robotic arm misses a pickup (i.e., the expected pickup does not occur), the vision system 310 and / or the unit controller 10C can reanalyze the pallet layer in the manner described above to re-identify the features of the pallet layer and generate a robotic arm 12 trajectory for retrying the pickup, wherein the retry trajectory may be different from the initially generated trajectory.

[0087] refer to Figure 3 , Figures 8A to 10B , Figures 15A to 15D and Figure 17 The operation of the layer removal tool 99 for removing layers will be described in accordance with various aspects of this disclosure. During operation, any suitable controller (e.g., robot controller 316 and / or unit controller 10C) generates a robot space map 1599 (see...). Figures 15A-15D () Figure 17 (Box 17100), which includes at least a pallet unloading / loading station 301 and its structure (restricted area). In one aspect, robot space diagram 1599 is generated as part of a release operation, and / or robot space diagram 1599 may be generated prior to the release operation. Robot space diagram 1599 may be generated in any suitable manner, for example, using data obtained from vision system 310 and / or data from a computer-aided design (CAD) model of palletizer unit 10. Although for illustrative purposes, Figures 15A-15D The robot space diagram 1599 is shown in two dimensions, but it should be understood that the robot space diagram 1599 can have any suitable construction, such as a two-dimensional depth map, a three-dimensional map including one or more point clouds representing structures / forbidden areas, a three-dimensional model generated based on images and / or CAD data, etc.

[0088] The pallet load PAL is delivered and placed at the pallet unloading / loading station 301 of the palletizer unit 10. Figure 17(Frame 17200). Vision system 310 images the pallet load PAL in the manner described herein ( Figure 17 (See box 17201). The robot controller 316 and / or the unit controller 10C integrate the pallet image data into the robot space diagram 1599 (see box 17201). Figures 15A-15D () Figure 17 (Box 17101), so that at least a representation of pallet layer 816 is generated in robot space diagram 1599 to achieve recognition of the position, orientation, etc. of the top pallet layer 816. Although in Figures 15A-15D Only pallet layer 816 is shown, but it should be understood that as each layer is deactivated to enable the identification of the position, orientation, etc., of the next top pallet layer, the image data representing the entire pallet can be iteratively integrated into the robot space diagram 1599. The robot controller 316 and / or the unit controller 10C are configured to dynamically define pallet layer picking boundary conditions and constraints based on the position / or orientation of pallet layer 816 within the robot space diagram 1599. Figure 17 (See box 17102). For example, based on image data from vision system 310, robot controller 316 and / or unit controller 10C are configured to dynamically determine the position of objects within palletizing unit 10 in any suitable manner (e.g., using a suitable image recognition algorithm), and to construct / reconstruct robot space map 1599 based on the determined object positions (for illustrative purposes, see only the reconstruction of robot space map 1599 based on the input of object 1571 and the redefinition of the area 1570 in which robot 14 is allowed to move). Objects can be transient objects entering and / or leaving palletizing unit 10, part of the structure of palletizing unit 10, pallet load PAL, etc. (In some respects, transient objects are accidental / unauthorized objects, in which case the controller can issue a stop command to robot 14). The boundary conditions and constraints of robot 14's movement are dynamically determined by robot controller 316 and / or unit controller 10C based on the position of objects within palletizing unit 10. Examples of boundary conditions include the area 1570 of the palletizing unit 10 that allows or restricts the movement of the robot 14; while examples of constraints are those described above: the type of object within the palletizing unit 10, prohibited areas 1500-1502, the position / attitude of the pallet load structure PAL, the acceleration of the robot 14 based on the contents of the pallet layer, etc.

[0089] When the robot controller 316 receives a signal from, for example, the unit controller 10C, indicating that the pallet layer 816 is ready to be picked up ( Figure 17 (Box 17202), robot controller 316 and / or unit controller 10C determine the path 1580 and trajectory 1581 of robot 14 ( Figure 17(frame 17103) for moving the depalletizing tool 99 from its initial position (e.g., the placement location of the previously picked pallet layer or any other suitable location) to a pick-up position (e.g., frame 17103). Figures 15A-15D Those shown herein) are used to pick up the top pallet layer 816. For example, robot controller 316 and / or unit controller 10C generate robot path 1580 and trajectory 1581 in any suitable manner based on robot space map 1599 (which includes image data of pallet layer 816), determined boundary conditions, and determined constraints. The determination of path 1580 and trajectory 1581 of robot 14 can be an iterative process, such that path 1580 and trajectory of robot 14 are optimized (e.g., time-optimized) to achieve optimized movement of robot 14 from its initial position to the optimized pick-up position of depalletizing tool 99, independent of the initial posture of robot 14 (as described herein). Robot controller 316 and / or unit controller 10C command robot 14 to position depalletizing tool 99 to the optimized pick-up position based on optimized path 1580 and trajectory 1581, such that grippers 22 and 24 surround pallet layer 816 ( Figure 17 (See box 17204) Figure 8A and 8B ).

[0090] It should be noted that for most layer destacking tools, systems and methods in the prior art, the positioning of the tool depends only on the nominal vertical position of the layer.

[0091] However, it is known that in the materials handling industry, product 18 is often crushed within the pallet load PAL by the weight of pallet layer 816 or the layers above it. For products located directly on the pallet support SPAL (see... Figure 2 This is especially true for the last few pallets to be unloaded on the pallet. Therefore, the nominal position of each pallet layer 816 used in the initial programming of robot 14 will cause the pallet unloading tool 99 to be mispositioned, thereby limiting the system's ability to fully unpile pallet layers 816.

[0092] In various aspects of this disclosure, the top pad 28 of the depalletizer 99 is coupled to a sensor 888 configured to assess the actual height / position of the top surface 148 of the picked pallet layer 816, thus enabling the depalletizer 99 to be positioned more precisely. This increased precision makes the depalletizing system, including the robot 14 and the depalletizer 99, more efficient and reduces the likelihood of product damage, or even prevents poor positioning from hindering the depalletizing of the product 18. The sensor may be in the form of an analog laser distance sensor or any other suitable distance-determining sensor.

[0093] The top liner 28 is lowered and positioned on the top layer of the pallet. Figure 17(See frame 17206). Robot 14 positions the depalletizer 99 at the height of the pallet layer 816 to be picked up, determined based on image data from vision system 310. Knowing that top pad 28 contacts the upper surface 148 of pallet layer 816, sensor 888 measures the position of top pad 28. Robot controller 316 (and / or unit controller 10C) then calculates the actual height of pallet layer 816 and compares it to the height of layer 816 determined based on image data from vision system 310. If a difference exists between the actual height (determined by sensor 888) and the height of layer 816 determined by image data from vision system 310, robot controller 316 adjusts the position of depalletizer 99 accordingly. Height differences can also be communicated to unit controller 10C for recalibrating / fine-tuning vision system 310 to reduce or minimize any difference between the actual height (determined by sensor 888) and the height of layer 816 determined by image data from vision system 310.

[0094] Using the top pad 28 to determine the actual height of the pallet top surface provides a reliable way to verify the position of the top surface 148 of the pallet layer 816. For example, even if the flap of product 18 is lifted or product 18 is not properly positioned, the overall position of the top pad 28 remains unaffected, thus providing valuable and accurate information about the actual height of the pallet layer 816. When needed, the robot 14 adjusts the height of the pallet depalletizing tool 99. Figure 17 (Box 17210).

[0095] Given that each pallet layer 816 may include multiple products 18, it is common for some products 18 to not have a side facing the outside of the pallet layer 816. According to one aspect, a combination of mechanisms is provided that allows the clamps 22-24 to fully clamp the products 18. For example, by activating and controlling the pneumatic actuator 56, four clamps 22-24 are used to press the entire pallet layer 816 on each side without horizontally moving the pallet layer 816. Figure 17 (Frame 17212). Then, robot 14 slightly lifts the depalletizing tool 99, mainly on the lower surface 8150 of the product 18 located around the periphery of pallet layer 816 (frame 17212). Figure 8B A gap is created between the product 18 on the pallet layer 816 directly below and the product 18 on the pallet layer 816 below. Figure 17 (Frame 17214). Then begin closing the horizontal curtains 26 so that these horizontal curtains 26 are inserted below the pallet layer 816 ( Figure 17 (Frame 17216). The horizontal curtain 26 is closed in Figure 9A-10BAs shown in the diagram, the rotating friction front 70 of the curtain 26 facilitates the movement of the product 18 from the clamped pallet layer 816 on the horizontal curtain 26. More specifically, this allows the product 18, which is not located on the periphery of the pallet layer 816 and is typically only partially lifted by the side clamps 22-24 or sometimes not lifted at all, to move on the horizontal curtain 26.

[0096] The frictional force generated by the pressure of the clamps 22-24 on the vertical surface of the product 18 may sometimes hinder the upward movement of the product 18 so that the horizontal curtain 26 can close from below without damaging the product 18. When this happens, the pressure on the side clamps 22-24 automatically decreases, and therefore the aforementioned frictional force also decreases, thus facilitating the movement of the curtain 26 to lift and pick up the remaining product 18 to be picked up from the pallet layer 816.

[0097] The pressure changes of clamps 22-24 are adjusted by monitoring the linear movement of the horizontal curtain 26. Figure 17 (frame 17218). If product 18 obstructs this movement, horizontal curtain 26 cannot move forward. When this occurs, it is inferred that at least one product 18 is obstructing this movement. Therefore, the pressure on clamps 22-24 is reduced by actuator 56 ( Figure 17 (frame 17220) to continue the process of picking up all products 18 on pallet layer 816.

[0098] According to another aspect, the linear movement of the curtain 26 is monitored so that its motion resistance is detected even as the curtain continues to move.

[0099] When the curtain 26 is fully closed (see...) Figures 10A-10B Pallet layer 816 was completely clamped by pallet unloading tool 99. Figure 17 (frame 17222), and robot 14 moves the depalletizing tool 99 with the clamped pallet layer 816 and transfers the pallet layer 816 to the discharge position (e.g., conveyor 150). Figure 17 (Frame 17232). The vision system 310 images the pallet load PAL during the placement cycle of the robot 14. Figure 17 (Frame 17233) to determine the orientation, position, etc. of the next layer to be picked up. Robot 14 places pallet layer 816 on discharge conveyor 150 (or other suitable discharge location) ( Figure 17 (Box 17234), and the pallet release process continues to Box 17202.

[0100] In addition to the function described above that allows for precise assessment of the position of the top pallet layer 816 in assessing pallet load PAL, the top pad 28 also prevents small products 18 from "popping" out of the pallet layer 816 as the horizontal curtain 26 moves beneath the products. This movement of products 18 might occur, for example, when the pallet layer 816 consists of small products 18 (i.e., products 18 with a reduced height). The weight of the top pad 28 is sufficient to prevent the small products from "popping out" without restricting the ability of the horizontal curtain 26 to move beneath the products. Similarly, the top pad 28 prevents small products 18 from "popping out" as the horizontal curtain 26 moves away from beneath the products.

[0101] In some respects, thin liner paper 277 ( Figure 2 The pallet depallet 18 is positioned between several layers of product 18 in the pallet load PAL. A standard method is to have a separate device that automatically removes the thin liner 277 between each depalletization sequence. This method works well, but it is very expensive considering the addition of a separate device. According to various aspects of this disclosure, the pallet depallet tool is provided with a component that retrieves the thin liner 277 located above the pallet layer 816 of product 18 while the pallet layer 816 is being depalletized. Since this pallet depallet tool is similar to the pallet depallet tool 99, and since this further release process is similar to the process described above, for the sake of simplicity, reference will be made to... Figure 17 Only describe the differences between them.

[0102] The tool includes a top liner 28, which includes a suction cup (not shown) to remove any thin liner paper on the pallet layer 816 while it is being unloaded. This is done when the pallet layer 816 is fully clamped by the depalletizing tool. Figure 17 (Box 17222), the suction cup is activated ( Figure 17 (Frame 17224). For example, another sensor, such as the thin-layer paper sensor 999 (e.g., such as a camera or scanner, which is constructed for use as...). Figure 9A and 9B And especially Figure 9B The same sensor (distinguishing the edge of the intermediate box shown) or mentioned above is used to determine, for example, whether thin liner 277 is attached to the top liner 28, with the thin liner sensor 999 facing upward toward the clamped pallet layer 816 being held and lifted by the tool (see also...). Figure 9B As robot 14 lifts and transfers pallet layer 816, top liner 28 is slightly raised. Figure 17 (Box 17226). If the thin liner sensor 999 still detects the presence of an object ( Figure 17 (frame 17228), which means that there is thin liner 277 under the pallet layer 816 being unstacking. Figure 17(frame 17229), and the vacuum is maintained on the suction cup ( Figure 17 (See box 17230). Further reference... Figure 19A , 19B And 19C, then, robot 17 places pallet layer 816 (with thin liner 277 underneath) on a pad top conveyor (e.g., discharge conveyor 150), and pallet layer 816 is placed on discharge conveyor 150 ( Figure 17 Afterwards (frame 17232) or elsewhere, the thin liner is removed and disposed of in the trash can. Figure 17 (See box 17236). Conversely, if the thin liner sensor 999 does not detect anything, it means that there is no thin liner under the depalletized layer. In this case, the vacuum is removed from the suction cup, and once the previous pallet layer 816 is placed on the discharge conveyor 150, the robot 14 moves directly backward to pick up the next pallet layer 816 on the pallet.

[0103] Still refer to Figure 19A , 19BLike 19C, the thin liner remover 998 is located or otherwise disposed between the various portions of the conveyor 150. For example, the conveyor 150 includes a pad or upstream portion 993 (where the unstacked layer is placed) and a box spreading or downstream portion 994. The thin liner remover 998 includes any suitable vacuum removal mechanism, such as a vacuum roller 997 (although any suitable adhesive or suction roller may be used), a limiting plate 996, and a deflector 995. The vacuum roller 997 has any suitable configuration for attracting the thin liner 227, for example, the vacuum roller rotates about a rolling axis 989, and includes a suction tube 997T fluidly coupled to a suction cup or suction port 997P arranged on or within the outer surface (e.g., cargo support surface) of the vacuum roller 997, and is configured to apply suction to the thin liner 227 as it passes over the vacuum roller 997. A limiting plate 996 is positioned to the side of the vacuum roller 997 to peel or remove the thin liner from the vacuum roller 997 and guide the thin liner to the bin 990. A deflector shroud 995 is located downstream of the vacuum roller 997 (e.g., relative to the pallet layer 816 along the travel direction 988 of the conveyor 150) and has any suitable size and shape to prevent the edge of the bin from entering between the conveyors (e.g., to allow the bin to be smoothly transferred from one conveyor section to another without jamming or clogging, while passing through the thin liner remover 998), and at least partially guides the thin liner 227 (removed from the bottom of the pallet layer 816) into the gap between the conveyor sections 993, 994 (although in one or more respects, the vacuum roller 997 may provide sufficient suction to independently guide the thin liner into and through the gap between the conveyor sections 993, 994). As described above, the thin liner 227 is detected by the thin liner sensor 999, which sends any suitable signal to the controller 10C to activate the thin liner remover 998. Activation of the thin liner remover 998 involves automatically activating the vacuum roller 997 and automatically removing the thin liner from under the pallet layer 816. Here, as the pallet layer 816 is transferred from the conveyor section 993 to the conveyor section 994, the thin liner 227 is prevented from adhering to the bottom of the pallet layer 816 (via the vacuum roller 997). As the thin liner 227 is transferred on the vacuum roller 997, the vacuum roller 997 attracts the thin liner 227, and at the vacuum roller 997, the vacuum roller guides and transfers the thin liner 997 through the gap between the conveyor sections to the limiting plate 996, where the thin liner 227 is removed from the vacuum roller 997 and disposed of in the waste bin 990.

[0104] Reference Figure 3 , Figure 8A-10B , Figures 15A-15D and Figure 18This provides a method for depalletizing cartons in a depalletizer 10. The method includes receiving, at the pallet unloading station 301 of the depalletizer 10, the pallet load PAL of cartons CU placed in pallet load layers 816 (representing pallet layers PL1-PL5). Figure 18 (Frame 1800), each pallet load layer 816 is formed by more than one box CU juxtaposed at a common level on the pallet load PAL area. A robot 14 is provided ( Figure 18 (See frame 1810), wherein robot 14 is equipped with a depalletizing end effector 99 having a gripper 800 configured to grip and pick up at least one of pallet load layers 816 to transport at least one pallet load layer 816 from pallet unloading station 301 to output station 333 (in one aspect, output station 333 includes any suitable conveyor 150). The gripper 800 has a gripper engagement interface 810 defining a predetermined layer engagement position and orientation of at least one of the pallet load layers 816 relative to the depalletizing end effector 99 (e.g., in robot coordinate system or space X, Y, Z, RX, RY, RZ, see [link to robot coordinate system]). Figure 3 (and also referred to herein as the robot reference frame), so as to repeatedly capture and stably hold at least one pallet load layer 816 using the gripper 800. The vision system 310 images the pallet load PAL of the box CU at the pallet unloading station 301 and generates at least one image of the top of at least one pallet load layer 816 independent of robot motion. Figure 18 (frame 1820). A controller (e.g., robot controller 16 and / or unit controller 10C) operably coupled to vision system 310 receives at least one image from vision system 310 and, based on at least one image, performs layer position and orientation determination of at least one of the pallet load layers 816 relative to the gripper engagement interface 810, a predetermined layer engagement position and orientation. Figure 18 (frame 1830), wherein the controller is operatively coupled to the robot 14 to position the gripper 800 and to capture and hold at least one of the pallet load layers 816 at the gripper engagement interface 810.

[0105] According to one or more aspects of this disclosure, the depalletizer includes:

[0106] A pallet unloading station is constructed to receive pallet loads of boxes arranged in pallet load layers, each of which is formed by more than one box juxtaposed at a common level in the area of ​​the pallet load.

[0107] A robot equipped with a depalletizing end effector having a gripper configured to grip and pick up at least one pallet load layer to transport at least one pallet load layer from a pallet unloading station to an output station. The gripper has a gripper engagement interface defining a predetermined layer engagement position and orientation of at least one pallet load layer relative to the depalletizing end effector, so as to repeatably capture and stably hold at least one pallet load layer using the gripper.

[0108] A vision system configured to image the pallet load of boxes at a pallet unloading station and constructed to generate at least one image of the top of at least one of the pallet load layers, independent of robot motion; and

[0109] A controller, operably coupled to a vision system to receive at least one image from the vision system, and configured to determine, based on the at least one image, the layer position and orientation of at least one of the pallet load layers relative to the predetermined layer engagement position and orientation of the gripper engagement interface, wherein the controller is operably coupled to a robot to position the gripper and use the gripper to capture and hold at least one of the pallet load layers at the gripper engagement interface.

[0110] According to one or more aspects of this disclosure, the predetermined layer engagement position and orientation provides an engagement plane orientation for the gripper engagement interface, and the layer position and orientation describes the flatness of the engagement surface of at least one of the pallet load layers, the engagement surface being configured to abut against the gripper engagement interface substantially across the pallet load layer, and the layer position and orientation describes a planar misalignment between the planar orientation of the engagement surface of at least one of the pallet load layers and the gripper engagement interface in at least two orthogonal directions.

[0111] According to one or more aspects of this disclosure, the vision system includes at least one camera mounted independently of the robot.

[0112] According to one or more aspects of this disclosure, each of the at least one camera is calibrated to a common camera calibration reference structure, and wherein the calibration of the common camera calibration reference structure describes the positional relationship between a respective camera reference frame of each corresponding camera and each other camera among the at least one camera and a predetermined reference frame of the robot.

[0113] According to one or more aspects of this disclosure, the at least one camera is configured such that the field of view of the at least one camera covers the pallet load from the uppermost upward-facing surface of the pallet load to the bottom of the pallet load.

[0114] According to one or more aspects of this disclosure, the at least one camera is configured such that the uppermost face-up surface of each of the pallet load layers is within the field of view of the at least one camera.

[0115] According to one or more aspects of this disclosure, the depalletizer includes:

[0116] A pallet unloading station is constructed to receive pallet loads of boxes arranged in pallet load layers, each of which is formed by more than one box juxtaposed at a common level in the area of ​​the pallet load.

[0117] A robot equipped with a depalletizing end effector having a gripper configured to grip and pick up at least one pallet load layer to transport at least one pallet load layer from a pallet load at an unloading station to an output station. The gripper has a gripper engagement interface defining a predetermined layer engagement position and orientation of at least one pallet load layer relative to the end effector, so as to repeatably capture and stably hold at least one pallet load layer using the gripper.

[0118] A vision system, separately configured from the robot, is used to image pallet loads at the pallet unloading station and is configured to generate at least one image of the top of at least one of the pallet load layers, independent of robot movement; and

[0119] A controller is configured to determine the positional and orientational relationship between the gripper interface and each of the topmost pallet layers of at least one of the pallet load layers based on the at least one image.

[0120] According to one or more aspects of this disclosure, the controller is operatively coupled to the vision system in order to receive the at least one image from the vision system.

[0121] According to one or more aspects of this disclosure, a controller is operatively coupled to a robot to position the gripper relative to each top pallet layer based on a determined relationship, and to capture and hold at least one layer at an engagement interface using the gripper.

[0122] According to one or more aspects of this disclosure, the determined relationships depict the layer position and orientation of each topmost layer relative to the predetermined layer engagement position and the orientation of the gripper interface relative to a predetermined reference frame of the robot.

[0123] According to one or more aspects of this disclosure, the controller determines the corresponding layer position and orientation of each topmost layer based on the at least one image, and the determined relationship is achieved by comparing the corresponding layer position and orientation with a predetermined reference frame of the robot.

[0124] According to one or more aspects of this disclosure, the corresponding layer positions and orientations describe the flatness of the mating surface of each topmost layer, the mating surface being configured to abut with a clamping engagement interface substantially correspondingly spanning each topmost layer, and the corresponding layer positions and orientations describe at least one of planar misalignment and center point misalignment between the mating surface of the topmost layer and the clamping engagement in at least two orthogonal directions.

[0125] According to one or more aspects of this disclosure, the controller is configured to resolve at least one of planar misalignment and center point misalignment based on robot motion boundary conditions for optimal gripping engagement with each topmost layer, the robot motion boundary conditions being defined by at least one of robot architectures and structures defining the depalletizer as depicted in the predetermined reference frame of the robot.

[0126] According to one or more aspects of this disclosure, the vision system includes at least one camera mounted independently of the robot.

[0127] According to one or more aspects of this disclosure, each of the at least one camera is calibrated to a common camera calibration reference structure, and wherein the calibration of the common camera calibration reference structure describes the positional relationship between a respective camera reference frame of each corresponding camera and each other camera among the at least one camera and a predetermined reference frame of the robot.

[0128] According to one or more aspects of this disclosure, the at least one camera is configured such that the field of view of the at least one camera covers the pallet load from the uppermost upward-facing surface of the pallet load to the bottom of the pallet load.

[0129] According to one or more aspects of this disclosure, the at least one camera is configured such that the uppermost face-up surface of each of the pallet load layers is within the field of view of the at least one camera.

[0130] According to one or more aspects of this disclosure, a method for depalletizing boxes in a depalletizer is provided. The method includes:

[0131] At the pallet unloading station of the depalletizer, the pallet load of boxes set in the pallet load layer is received, each of which is formed by more than one box placed side by side in a common level on the pallet load area.

[0132] A robot is provided equipped with a depalletizing end effector having a gripper configured to grip and pick up at least one pallet load layer to transport at least one pallet load layer from a pallet unloading station to an output station. The gripper has a gripper engagement interface defining a predetermined layer engagement position and orientation of at least one pallet load layer relative to the depalletizing end effector, so as to repeatably capture and stably hold at least one pallet load layer using the gripper.

[0133] A vision system is used to image the pallet load of boxes at the pallet unloading station, and at least one image of the top of at least one of the pallet load layers is generated independently of robot movement; and

[0134] The controller is operatively coupled to the vision system to receive at least one image from the vision system and to determine, based on the at least one image, the layer position and orientation of at least one of the pallet load layers relative to the predetermined layer engagement position and orientation of the gripper engagement interface, wherein the controller is operatively coupled to the robot to position the gripper and to capture and hold at least one of the pallet load layers at the gripper engagement interface using the gripper.

[0135] According to one or more aspects of this disclosure, the predetermined layer engagement position and orientation provides an engagement plane orientation for the gripper engagement interface, and the layer position and orientation describes the flatness of the engagement surface of at least one of the pallet load layers, the engagement surface being configured to abut against the gripper engagement interface substantially across the pallet load layer, and the layer position and orientation describes a planar misalignment between the planar orientation of the engagement surface of at least one of the pallet load layers and the gripper engagement interface in at least two orthogonal directions.

[0136] According to one or more aspects of this disclosure, the vision system includes at least one camera mounted independently of the robot.

[0137] According to one or more aspects of this disclosure, each of the at least one camera is calibrated to a common camera calibration reference structure, and wherein the calibration of the common camera calibration reference structure describes the positional relationship between a respective camera reference frame of each corresponding camera and each other camera among the at least one camera and a predetermined reference frame of the robot.

[0138] According to one or more aspects of this disclosure, the at least one camera is configured such that the field of view of the at least one camera covers the pallet load from the uppermost upward-facing surface of the pallet load to the bottom of the pallet load.

[0139] According to one or more aspects of this disclosure, the at least one camera is configured such that the uppermost face-up surface of each of the pallet load layers is within the field of view of the at least one camera.

[0140] According to one or more aspects of this disclosure, a method for depalletizing boxes in a depalletizer is provided. The method includes:

[0141] At the pallet unloading station of the depalletizer, the pallet load of boxes set in the pallet load layer is received, each of which is formed by more than one box placed side by side in a common level on the pallet load area.

[0142] A robot is provided equipped with a depalletizing end effector having a gripper configured to grip and pick up at least one pallet load layer to transport at least one pallet load layer from a pallet load at an unloading station to an output station. The gripper has a gripper engagement interface defining a predetermined layer engagement position and orientation of at least one pallet load layer relative to the end effector, so as to repeatably capture and stably hold at least one pallet load layer using the gripper.

[0143] The pallet load at the pallet unloading station is imaged using a vision system separate from the robot, and at least one image of the top of at least one of the pallet load layers is generated independently of robot movement; and

[0144] Using the controller of the depalletizer, the positional and orientational relationships between the gripper interface and each topmost pallet layer of at least one of the pallet load layers are determined based on the at least one image.

[0145] According to one or more aspects of this disclosure, the controller is operatively coupled to the vision system in order to receive the at least one image from the vision system.

[0146] According to one or more aspects of this disclosure, a controller is operatively coupled to a robot to position the gripper relative to each top pallet layer based on a determined relationship, and to capture and hold at least one layer at an engagement interface using the gripper.

[0147] According to one or more aspects of this disclosure, the determined relationships depict the layer position and orientation of each topmost layer relative to the predetermined layer engagement position and the orientation of the gripper interface relative to a predetermined reference frame of the robot.

[0148] According to one or more aspects of this disclosure, the controller determines the corresponding layer position and orientation of each topmost layer based on the at least one image, and the determined relationship is achieved by comparing the corresponding layer position and orientation with a predetermined reference frame of the robot.

[0149] According to one or more aspects of this disclosure, the corresponding layer positions and orientations describe the flatness of the mating surface of each topmost layer, the mating surface being configured to abut with a clamping engagement interface substantially correspondingly spanning each topmost layer, and the corresponding layer positions and orientations describe at least one of planar misalignment and center point misalignment between the mating surface of the topmost layer and the clamping engagement in at least two orthogonal directions.

[0150] According to one or more aspects of this disclosure, the controller is configured to resolve at least one of planar misalignment and center point misalignment based on robot motion boundary conditions for optimal gripping engagement with each topmost layer, the robot motion boundary conditions being defined by at least one of robot architectures and structures defining the depalletizer as depicted in the predetermined reference frame of the robot.

[0151] According to one or more aspects of this disclosure, the vision system includes at least one camera mounted independently of the robot.

[0152] According to one or more aspects of this disclosure, each of the at least one camera is calibrated to a common camera calibration reference structure, and wherein the calibration of the common camera calibration reference structure describes the positional relationship between a respective camera reference frame of each corresponding camera and each other camera among the at least one camera and a predetermined reference frame of the robot.

[0153] According to one or more aspects of this disclosure, the at least one camera is configured such that the field of view of the at least one camera covers the pallet load from the uppermost upward-facing surface of the pallet load to the bottom of the pallet load.

[0154] According to one or more aspects of this disclosure, the at least one camera is configured such that the uppermost face-up surface of each of the pallet load layers is within the field of view of the at least one camera.

[0155] According to one or more aspects of this disclosure, a thin liner removal apparatus is provided for removing thin liner from a cargo layer traveling along a transverse path. The thin liner removal apparatus includes: a frame; a roller rotatably coupled to the frame for rotating about a rolling axis, the roller having a vacuum port extending through a cargo support surface of the roller; and a vacuum mechanism coupled to the roller to evacuate through the vacuum port, wherein the vacuum port is positioned on the roller to constrain the thin liner disposed between the roller and the cargo layer supported on the cargo support surface of the roller, thereby attracting and separating the thin liner from the cargo layer.

[0156] According to one or more aspects of this disclosure, the rollers substantially simultaneously rotate the thin liner paper attracted by the vacuum mechanism about the rolling axis in order to separate the thin liner paper from the cargo layer.

[0157] According to one or more aspects of this disclosure, the vacuum mechanism includes at least one suction tube extending through the roller.

[0158] According to one or more aspects of this disclosure, the vacuum port includes a suction cup coupled to the roller to attract the thin liner paper.

[0159] According to one or more aspects of this disclosure, the thin liner removal device further includes a deflector shroud to constrain the thin liner and separate it from the cargo layer.

[0160] According to one or more aspects of this disclosure, the thin liner removal device further includes a peeling plate coupled to a frame, the peeling plate being positioned relative to the roller to peel (or otherwise strip / remove) the thin liner from the roller.

[0161] According to one or more aspects of this disclosure, the thin liner removal device further includes a collection box configured to collect thin liner separated from the cargo layer.

[0162] According to one or more aspects of this disclosure, the thin liner removal apparatus further includes an upstream conveyor and a downstream conveyor, with the roller disposed between the upstream conveyor and the downstream conveyor.

[0163] According to one or more aspects of this disclosure, the lateral path extends the upstream conveyor and the downstream conveyor.

[0164] According to one or more aspects of this disclosure, the upstream conveyor is a padding conveyor, and the downstream conveyor is a box spreading conveyor.

[0165] According to one or more aspects of this disclosure, a depalletizer includes: a pallet unloading station for receiving cargo pallets and dividing the pallets into cargo layers; a box conveyor configured to transport the cargo layers from the pallet unloading station to a storage array, thereby enabling the feeding of cargo into the storage array, the box conveyor including more than one conveyor section; and a thin liner removal system disposed between two adjacent conveyor sections in the more than one conveyor section, the thin liner removal system including: a frame; a roller rotatably coupled to the frame for rotating about a rolling axis, the roller having a vacuum port extending through a cargo support surface of the roller; and a vacuum mechanism coupled to the roller to evacuate through the vacuum port, wherein the vacuum port is positioned on the roller to constrain the thin liner disposed between the roller and the cargo layers supported on the cargo support surface of the roller, so as to attract the thin liner and separate the thin liner from the cargo layers.

[0166] According to one or more aspects of this disclosure, a method for removing thin liner paper from a cargo layer is provided. The method includes: providing a frame for a thin liner paper removal device; providing a roller rotatably coupled to the frame about a rolling axis, the roller being configured to support and transport a cargo layer placed on the thin liner paper along a transverse axis; using the roller to vacuum-attract the thin liner paper, wherein the vacuum is generated by a vacuum mechanism coupled to the roller; and using the roller to separate the thin liner paper from the cargo layer.

[0167] It should be understood that the foregoing description is merely illustrative of various aspects of this disclosure. Various alternatives and modifications will arise for those skilled in the art without departing from these aspects. Accordingly, the various aspects of this disclosure are intended to cover all such alternatives, modifications, and variations falling within the scope of the appended claims. Furthermore, the undisputed fact that different features are recited in different dependent or independent claims does not imply that combinations of these features cannot be advantageously used, and such combinations are still included within the scope of various aspects of the invention.

Claims

1. A depalletizer, comprising: A pallet unloading station is configured to receive pallet loads of boxes arranged in pallet load layers, each of which is formed by more than one box juxtaposed at a common level in the area of ​​the pallet load. A robot equipped with a depalletizing end effector having a gripper configured to grip and pick up at least one of the pallet load layers to transport at least one pallet load layer from the pallet load at the pallet unloading station to an output station. The gripper has a gripper engagement interface defining a predetermined layer engagement position and orientation of at least one of the pallet load layers relative to the depalletizing end effector, so as to repeatably capture and stably hold at least one of the pallet load layers using the gripper. A vision system configured to image the pallet load of the container at the pallet unloading station and configured to generate at least one image of the top of at least one of the pallet load layers, independent of robot motion. and A controller, operably coupled to the vision system, for receiving at least one image from the vision system, and configured to determine, based on the at least one image, the layer position and orientation of at least one of the pallet load layers relative to the predetermined layer engagement position and orientation of the gripper engagement interface, wherein the controller is operably coupled to the robot for positioning the gripper and using the gripper to capture and hold at least one of the pallet load layers at the gripper engagement interface.

2. The depalletizer according to claim 1, characterized in that, The predetermined layer engagement position and orientation provide the engagement plane orientation of the gripper engagement interface, and the layer position and orientation describe the flatness of the engagement surface of at least one of the pallet load layers, the engagement surface being configured to mate with the gripper engagement interface spanning at least one of the pallet load layers, and the layer position and orientation describe the planar misalignment between the planar orientation of the engagement surface of at least one of the pallet load layers and the gripper engagement interface in at least two orthogonal directions.

3. The depalletizer according to claim 1, characterized in that, The vision system includes at least one camera installed independently of the robot.

4. The depalletizer according to claim 3, characterized in that, Each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration of the common camera calibration reference structure describes the positional relationship between the respective camera reference frame of each corresponding camera and each other camera among the at least one camera and the predetermined reference frame of the robot.

5. The depalletizer according to claim 3, characterized in that, The at least one camera is configured such that the field of view of the at least one camera covers the pallet load from the uppermost face up to the bottom of the pallet load.

6. The depalletizer according to claim 3, characterized in that, The at least one camera is configured such that the uppermost face of each of the pallet load layers is within the field of view of the at least one camera.

7. A depalletizer, comprising: A pallet unloading station is configured to receive pallet loads of boxes arranged in pallet load layers, each of which is formed by more than one box juxtaposed at a common level in the area of ​​the pallet load. A robot equipped with a depalletizing end effector having a gripper configured to grip and pick up at least one of the pallet load layers to transport at least one pallet load layer from the pallet load at an unloading station to an output station. The gripper has a gripper engagement interface defining a predetermined layer engagement position and orientation of at least one of the pallet load layers relative to the end effector, so as to repeatably capture and stably hold at least one of the pallet load layers using the gripper. A vision system, which is separately configured from the robot, is used to image the pallet load at the pallet unloading station and is configured to generate at least one image of the top of at least one of the pallet load layers, independent of robot movement. and A controller is configured to determine the positional and orientational relationship between the gripper interface and each of the topmost pallet layers in at least one of the pallet load layers based on the at least one image.

8. The depalletizer according to claim 7, characterized in that, The controller is operatively coupled to the vision system in order to receive the at least one image from the vision system.

9. The depalletizer according to claim 7, characterized in that, The controller is operatively coupled to the robot to position the gripper relative to each top pallet layer based on a determined relationship, and to capture and hold at least one layer at the engagement interface using the gripper.

10. The depalletizer according to claim 7, characterized in that, The determined relationships describe the layer position and orientation of each topmost layer relative to the predetermined layer engagement position, and the orientation of the gripper interface relative to the predetermined reference frame of the robot.

11. The depalletizer according to claim 7, characterized in that, The controller determines the position and orientation of each topmost layer based on the at least one image, and the determined relationship is achieved by comparing the position and orientation of the corresponding layer with a predetermined reference frame of the robot.

12. The depalletizer according to claim 11, characterized in that, The corresponding layer positions and orientations describe the flatness of the mating surface of each topmost layer, which is configured to mate with the clamping engagement interface correspondingly spanning each topmost layer, and the corresponding layer positions and orientations describe at least one of planar misalignment and center point misalignment between the mating surface of the topmost layer and the clamping engagement in at least two orthogonal directions.

13. The depalletizer according to claim 12, characterized in that, The controller is configured to resolve at least one of planar misalignment and center point misalignment based on robot motion boundary conditions for optimal gripping engagement with each topmost layer, the robot motion boundary conditions being defined by at least one of the robot architecture and structure defining the depalletizer as depicted in the robot's predetermined reference frame.

14. The depalletizer according to claim 7, characterized in that, The vision system includes at least one camera installed independently of the robot.

15. The depalletizer according to claim 14, characterized in that, Each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration of the common camera calibration reference structure describes the positional relationship between the respective camera reference frame of each corresponding camera and each other camera among the at least one camera and the predetermined reference frame of the robot.

16. The depalletizer according to claim 14, characterized in that, The at least one camera is configured such that the field of view of the at least one camera covers the pallet load from the uppermost face up to the bottom of the pallet load.

17. The depalletizer according to claim 14, characterized in that, The at least one camera is configured such that the uppermost face of each of the pallet load layers is within the field of view of the at least one camera.

18. A method for destacking boxes in a destacking machine, the method comprising: At the pallet unloading station of the depalletizer, pallet loads of boxes arranged in the pallet load layer are received, each of the pallet load layers being formed by more than one box juxtaposed at a common level in the area of ​​the pallet load. A robot is provided equipped with a depalletizing end effector having a gripper configured to grip and pick up at least one of the pallet load layers to transport at least one pallet load layer from the pallet load at the pallet unloading station to an output station. The gripper has a gripper engagement interface defining a predetermined layer engagement position and orientation of at least one of the pallet load layers relative to the depalletizing end effector, so as to repeatably capture and stably hold at least one of the pallet load layers using the gripper. The pallet load of the container at the pallet unloading station is imaged using a vision system, and at least one image of the top of at least one of the pallet load layers is generated independently of robot motion. and A controller operatively coupled to the vision system receives at least one image from the vision system and, based on the at least one image, determines the layer position and orientation of at least one of the pallet load layers relative to the predetermined layer engagement position and orientation of the gripper engagement interface, wherein the controller is operatively coupled to the robot to position the gripper and to capture and hold at least one of the pallet load layers at the gripper engagement interface using the gripper.

19. The method according to claim 18, characterized in that, The predetermined layer engagement position and orientation provide the engagement plane orientation of the gripper engagement interface, and the layer position and orientation describe the flatness of the engagement surface of at least one of the pallet load layers, the engagement surface being configured to mate with the gripper engagement interface spanning at least one of the pallet load layers, and the layer position and orientation describe the planar misalignment between the planar orientation of the engagement surface of at least one of the pallet load layers and the gripper engagement interface in at least two orthogonal directions.

20. The method according to claim 18, wherein, The vision system includes at least one camera installed independently of the robot.

21. The method according to claim 20, wherein, Each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration of the common camera calibration reference structure describes the positional relationship between the respective camera reference frame of each corresponding camera and each other camera among the at least one camera and the predetermined reference frame of the robot.

22. The method according to claim 20, characterized in that, The at least one camera is configured such that the field of view of the at least one camera covers the pallet load from the uppermost face up to the bottom of the pallet load.

23. The method according to claim 20, characterized in that, The at least one camera is configured such that the uppermost face of each of the pallet load layers is within the field of view of the at least one camera.

24. A method for destacking boxes in a destacking machine, the method comprising: At the pallet unloading station of the depalletizer, pallet loads of boxes arranged in the pallet load layer are received, each of the pallet load layers being formed by more than one box juxtaposed at a common level in the area of ​​the pallet load. A robot is provided equipped with a depalletizing end effector having a gripper configured to grip and pick up at least one pallet load layer to transport at least one pallet load layer from the pallet load at an unloading station to an output station. The gripper has a gripper engagement interface defining a predetermined layer engagement position and orientation of at least one pallet load layer relative to the end effector, so as to repeatably capture and stably hold at least one pallet load layer using the gripper. The pallet load at the pallet unloading station is imaged using a vision system separately from the robot, and at least one image of the top of at least one of the pallet load layers is generated independently of the robot's movement. and Using the controller of the depalletizer, the positional and orientational relationship between the gripper interface and each topmost pallet layer of at least one of the pallet load layers is determined based on the at least one image.

25. The method according to claim 24, characterized in that, The controller is operatively coupled to the vision system in order to receive the at least one image from the vision system.

26. The method according to claim 24, characterized in that, The controller is operatively coupled to the robot to position the gripper relative to each top pallet layer based on a determined relationship, and to capture and hold at least one layer at the engagement interface using the gripper.

27. The method according to claim 24, characterized in that, The determined relationships describe the layer position and orientation of each topmost layer relative to the predetermined layer engagement position, and the orientation of the gripper interface relative to the predetermined reference frame of the robot.

28. The method according to claim 24, characterized in that, The controller determines the position and orientation of each topmost layer based on the at least one image, and the determined relationship is achieved by comparing the position and orientation of the corresponding layer with a predetermined reference frame of the robot.

29. The method according to claim 28, characterized in that, The corresponding layer positions and orientations describe the flatness of the mating surface of each topmost layer, which is configured to mate with the clamping engagement interface correspondingly spanning each topmost layer, and the corresponding layer positions and orientations describe at least one of planar misalignment and center point misalignment between the mating surface of the topmost layer and the clamping engagement in at least two orthogonal directions.

30. The method according to claim 29, characterized in that, The controller resolves at least one of planar misalignment and center point misalignment based on robot motion boundary conditions for optimal gripping engagement with each topmost layer, the robot motion boundary conditions being defined by at least one of the robot architecture and structure defining the depalletizer as depicted in the robot's predetermined reference frame.

31. The method according to claim 24, wherein, The vision system includes at least one camera installed independently of the robot.

32. The method according to claim 31, wherein, Each of the at least one camera is calibrated to a common camera calibration reference structure, and the calibration of the common camera calibration reference structure describes the positional relationship between the respective camera reference frame of each corresponding camera and each other camera among the at least one camera and the predetermined reference frame of the robot.

33. The method according to claim 31, characterized in that, The at least one camera is configured such that the field of view of the at least one camera covers the pallet load from the uppermost face up to the bottom of the pallet load.

34. The method according to claim 31, characterized in that, The at least one camera is configured such that the uppermost face of each of the pallet load layers is within the field of view of the at least one camera.

Images