Method, apparatus and computer program product for de-palletizing mixed objects

Abstract

Methods, apparatuses, systems, computing devices, and / or the like are provided. An example method can include receiving, from a perception subsystem associated with a object unstacking system, first imaging data associated with a plurality of objects disposed on a pallet; calculating, based at least in part on the first imaging data, one or more comparative size metrics associated with the plurality of objects; determining whether the one or more comparative size metrics satisfy a comparative size threshold range; and causing, in response to determining that the one or more comparative size metrics satisfy the comparative size threshold range, an execution subsystem associated with the object unstacking system to operate in a constant pallet mode.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority and benefit to U.S. Provisional Patent Application No. 63 / 263,223, filed October 28, 2021, the contents of which are incorporated herein by reference. Background Technology

[0003] Palletizing refers to the operation of loading or transferring objects (such as, but not limited to, cartons, boxes, articles, etc.) onto pallets or similar devices. Depalletizing refers to the operation of removing or removing objects (such as, but not limited to, cartons, boxes, articles, etc.) from pallets or similar devices.

[0004] Many palletizing and / or depalletizing solutions are plagued by technical challenges and difficulties. For example, if one or more objects on a pallet have different sizes, many depalletizing solutions cannot properly unload or remove these objects from the pallet, and / or may take longer to unload or remove these objects from the pallet compared to the time required when unloading or removing objects of the same size from the pallet. Summary of the Invention

[0005] Generally speaking, the embodiments disclosed herein provide methods, equipment, systems, computing devices, computing entities, etc.

[0006] According to various embodiments of this disclosure, an apparatus is provided. The apparatus may include at least one processor and at least one non-transitory memory including computer program code. The at least one non-transitory memory and the computer program code may be configured to utilize the at least one processor to cause the apparatus to receive first imaging data associated with a plurality of objects disposed on a pallet from a sensing subsystem associated with an object depalletizing system; calculate one or more comparative size measures associated with the plurality of objects based at least in part on the first imaging data; determine whether the one or more comparative size measures satisfy a comparative size threshold range; and, in response to determining that the one or more comparative size measures satisfy the comparative size threshold range, cause an execution subsystem associated with the object depalletizing system to operate in a constant pallet mode.

[0007] In some embodiments, the sensing subsystem includes a two-dimensional (2-D) image capture device. In some embodiments, the first imaging data includes 2-D image data associated with multiple objects and captured by the 2-D image capture device.

[0008] In some embodiments, the sensing subsystem includes a three-dimensional (3-D) image capture device. In some embodiments, the first imaging data includes 3-D image data associated with multiple objects and captured by the 3-D image capture device.

[0009] In some implementations, prior to calculating one or more comparative size metrics associated with multiple objects, at least one non-transitory memory and computer program code are configured to utilize at least one processor to cause the device to: input first imaging data into an object segmentation machine learning model, wherein the first imaging data includes at least one image data from 2-D image data associated with multiple objects; and receive multiple object segmentation instructions associated with at least one image data from the 2-D image data from the object segmentation machine learning model.

[0010] In some implementations, one or more comparative size metrics are calculated based at least in part on multiple object segmentation indicators.

[0011] In some implementations, the multiple objects include a first object and a second object disposed on the same tray layer of the tray.

[0012] In some implementations, when calculating one or more comparative size measures associated with multiple objects, at least one non-transitory memory and computer program code are configured to utilize at least one processor to cause the device to: determine a first image size measure associated with a first object based at least in part on 2-D or 3-D image data from a sensing subsystem; determine a second image size measure associated with a second object based at least in part on 2-D or 3-D image data from a sensing subsystem; and determine a comparative size measure in one or more comparative size measures based at least in part on the first and second image size measures.

[0013] In some embodiments, the first image size metric associated with the first object is a first image length metric associated with the first object. In some embodiments, the second image size metric associated with the second object is a second image length metric associated with the second object. In some embodiments, the comparison size metric is a comparison length metric. In some embodiments, the comparison size threshold range is a comparison length threshold range.

[0014] In some embodiments, the first image size metric associated with the first object is the first image width metric associated with the first object. In some embodiments, the second image size metric associated with the second object is the second image width metric associated with the second object. In some embodiments, the comparison size metric is a comparison width metric. In some embodiments, the comparison size threshold range is a comparison width threshold range.

[0015] In some embodiments, the first image size metric associated with the first object is the first image height metric associated with the first object. In some embodiments, the second image size metric associated with the second object is the second image height metric associated with the second object. In some embodiments, the comparison size metric is a comparison height metric. In some embodiments, the comparison size threshold range is a comparison height threshold range.

[0016] In some implementations, the execution subsystem includes a height sensing device. In some implementations, the execution subsystem includes a depalletizer device.

[0017] In some implementations, the depalletizer is configured to deliver each of a plurality of objects above a height sensing device.

[0018] In some implementations, at least one non-transitory memory and computer program code are configured to utilize at least one processor to enable the device to: enable the depalletizer to deliver each of a plurality of objects above the height sensing device.

[0019] In some embodiments, at least one non-transitory memory and computer program code are configured to utilize at least one processor to cause the device to: transport a first object from a plurality of objects to above a height sensing device; receive first height data associated with the first object from the height sensing device; and determine, at least in part, a lifting height parameter associated with the depalletizer in a constant pallet mode based on the first height data.

[0020] In some implementations, at least one non-transitory memory and computer program code are configured to utilize at least one processor to cause the device, when the execution subsystem is in constant pallet mode, to cause the depalletizer to lift one or more objects, excluding the first object, to the same lifting height, at least in part, based on a lifting height parameter.

[0021] In some implementations, when the execution subsystem is in constant pallet mode, the depalletizer is configured to lift one or more objects, excluding the first object, to the same lifting height, at least in part based on a lifting height parameter.

[0022] In some implementations, at least one non-transitory memory and computer program code are configured to utilize at least one processor to cause the device to: when the execution subsystem is in a constant pallet mode, cause the depalletizer to lift a second object from a plurality of objects to a lifting height based at least in part on a lifting height parameter; and cause the depalletizer to transfer the second object above a height sensing device.

[0023] In some implementations, at least one non-transitory memory and computer program code are configured to utilize at least one processor to cause the device to: receive second height data associated with a second object from a height sensing device; calculate an object height difference metric between the first object and the second object; and determine whether the object height difference metric meets an object height difference metric threshold range.

[0024] In some implementations, at least one non-transitory memory and computer program code are configured to utilize at least one processor to cause the device to: in response to determining that the object height difference metric does not meet the object height difference metric threshold range, cause the execution subsystem associated with the object depalletizing system to exit constant pallet mode.

[0025] In some implementations, at least one non-transitory memory and computer program code are configured to utilize at least one processor to cause the device to: in response to determining that the depalletizer has delivered each of a plurality of objects on the same pallet layer, cause the execution subsystem associated with the depalletizer to exit constant pallet mode.

[0026] In some embodiments, the first imaging data is associated with a first acquisition time point. In some embodiments, at least one non-transitory memory and computer program code are configured to utilize at least one processor to cause the device to: determine a first average object height metric associated with a plurality of objects; and store the first average object height metric in a memory element.

[0027] In some implementations, at least one non-transitory memory and computer program code are configured to utilize at least one processor to cause the device to: determine a second average object height metric; calculate an average height difference metric based on the first average object height metric and the second average object height metric; and determine whether the average height difference metric meets an average height difference metric threshold range.

[0028] In some implementations, at least one non-transitory memory and computer program code are configured to utilize at least one processor to cause the device to: replace a first average object height measurement in the memory element with a second average object height measurement in response to determining that the average height difference measurement meets an average height difference measurement threshold range.

[0029] According to various embodiments of this disclosure, a computer-implemented method is provided. The computer-implemented method may include receiving first imaging data associated with a plurality of objects disposed on a pallet from a sensing subsystem associated with an object depalletizing system; calculating one or more comparative size measures associated with the plurality of objects based at least in part on the first imaging data; determining whether the one or more comparative size measures satisfy a comparative size threshold range; and, in response to determining that the one or more comparative size measures satisfy the comparative size threshold range, causing an execution subsystem associated with the object depalletizing system to operate in a constant pallet mode.

[0030] According to various embodiments of this disclosure, a computer program product is provided. The computer program product may include at least one non-transitory computer-readable storage medium in which a portion of computer-readable program code is stored. The computer-readable program code portion may include an executable portion configured to receive first imaging data associated with a plurality of objects disposed on a pallet from a sensing subsystem associated with an object depalletizing system; calculate one or more comparative size measures associated with the plurality of objects based at least in part on the first imaging data; determine whether the one or more comparative size measures satisfy a comparative size threshold range; and, in response to determining that the one or more comparative size measures satisfy the comparative size threshold range, cause an execution subsystem associated with the object depalletizing system to operate in a constant pallet mode.

[0031] The above description of the invention is provided merely to outline some exemplary embodiments and to provide a basic understanding of some aspects of this disclosure. Therefore, it should be understood that the above embodiments are merely examples. It should be understood that, in addition to those summarized herein, the scope of this disclosure covers many possible embodiments, some of which will be further described below. Attached Figure Description

[0032] Therefore, this disclosure has been described in general terms, and reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and in which:

[0033] Figure 1A This is an example perspective view of an example depalletizing system that can be used according to various embodiments of this disclosure;

[0034] Figure 1B This is an example diagram illustrating an example depalletizing system that can be used according to various embodiments of the present disclosure;

[0035] Figure 2 These are example schematic diagrams of example components in example depalletizing systems according to various embodiments of this disclosure;

[0036] Figure 3This is a schematic diagram of an example component in an example control subsystem according to various embodiments of the present disclosure;

[0037] Figure 4 , Figure 5 and Figure 6 This is an example diagram illustrating an example tray layer on an example tray according to various embodiments of the present disclosure;

[0038] Figure 7A and Figure 7B This is an example flowchart illustrating an example method for determining, according to various embodiments of the present disclosure, whether to cause the execution subsystem associated with the object depalletizing system to operate in a constant pallet mode;

[0039] Figure 8 This is an example flowchart illustrating an example method for generating object segmentation instructions according to various embodiments of the present disclosure;

[0040] Figure 9 This is an example image illustrating example object segmentation instructions according to various embodiments of the present disclosure;

[0041] Figure 10 This is an example flowchart illustrating an example method for determining comparative size measures according to various embodiments of the present disclosure;

[0042] Figure 11A and Figure 11B This is an example flowchart illustrating an example method for determining whether one or more comparison size metrics meet a comparison size threshold range according to various embodiments of the present disclosure;

[0043] Figure 12 This is an example flowchart illustrating an example method for operating an execution subsystem associated with an object depalletizing system in a constant pallet mode according to various embodiments of the present disclosure;

[0044] Figure 13 This is an example image illustrating the example operation of an execution subsystem in constant tray mode according to various embodiments of the present disclosure;

[0045] Figure 14A and Figure 14B This is an example flowchart illustrating an example method for determining whether to cause the execution subsystem associated with the object depalletizing system to exit a constant pallet mode according to various embodiments of the present disclosure;

[0046] Figure 15 This is an example flowchart illustrating an example method for determining whether to cause the execution subsystem associated with the object depalletizing system to exit a constant pallet mode according to various embodiments of the present disclosure;

[0047] Figure 16A and Figure 16B This is an example flowchart illustrating an example method for determining, according to various embodiments of the present disclosure, whether to allow the execution subsystem associated with the object depalletizing system to continue operating in a constant pallet mode; and

[0048] Figure 17 This is an example flowchart illustrating example methods associated with an object depalletizing system according to various embodiments of the present disclosure. Detailed Implementation

[0049] Various embodiments of this disclosure will now be described more fully below with reference to the accompanying drawings, which illustrate some, but not all, of the embodiments of this disclosure. In fact, this disclosure may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided to enable this disclosure to meet applicable legal requirements. Unless otherwise specified, the term “or” (also indicated as “ / ”) is used herein in both alternative and combined senses. The terms “illustrative” and “exemplary” are used for examples without an indication of quality level. Throughout the text, similar reference numerals may refer to similar elements. Phrases such as “in one embodiment,” “according to one embodiment,” etc., generally mean that the particular feature, structure, or characteristic following the phrase may be included in at least one embodiment of this disclosure, and may be included in more than one embodiment of this disclosure (importantly, such phrases do not necessarily refer to the same embodiment).

[0050] Embodiments of this disclosure can be implemented as computer program products including an article of art. Such computer program products may include one or more software components, including, for example, applications, software objects, methods, data structures, etc. Software components may be encoded in any of a variety of programming languages. An exemplary programming language may be a low-level programming language, such as assembly language associated with a specific hardware architecture and / or operating system platform / system. Software components including assembly language instructions may need to be converted into executable machine code by an assembler before execution by the hardware architecture and / or platform / system. Another exemplary programming language may be a higher-level programming language that is portable across multiple architectures. Software components including higher-level programming language instructions may need to be converted into an intermediate representation by an interpreter or compiler before execution.

[0051] Other examples of programming languages ​​include, but are not limited to, macro languages, shell or command languages, job control languages, scripting languages, database query or search languages, and / or report writing languages. In one or more example implementations, software components including instructions from one of the foregoing examples of programming languages ​​can be executed directly by an operating system or other software components without first being translated into another form. Software components can be stored as files or other data storage constructs. Software components of similar type or related functionality can be stored together, such as in a specific directory, folder, or library. Software components can be static (e.g., pre-built or fixed) or dynamic (e.g., created or modified at runtime).

[0052] Alternatively or concurrently, embodiments of this disclosure may be implemented as non-transitory computer-readable storage media storing applications, programs, program modules, scripts, source code, program code, object code, bytecode, compiled code, interpreted code, machine code, executable instructions, etc. (also referred to herein as executable instructions, instructions for execution, computer program products, program code, and / or similar terms used interchangeably herein). Such non-transitory computer-readable storage media include all computer-readable media (including volatile and non-volatile media).

[0053] In one embodiment, non-volatile computer-readable storage media may include floppy disks, floppy disks, hard disks, solid-state storage (SSS) (e.g., solid-state drives (SSDs), solid-state cards (SSCs), solid-state modules (SSMs), enterprise flash drives, magnetic tape, or any other non-transitory magnetic media). Non-volatile computer-readable storage media may also include punched cards, paper tape, optical marking sheets (or any other physical media with perforated patterns or other optically identifiable markings), optical disc read-only memory (CD-ROM), optical disc rewritable (CD-RW), digital versatile disc (DVD), Blu-ray disc (BD), and any other non-transitory optical media. Such non-volatile computer-readable storage media may also include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory (e.g., serial, NAND, NOR, etc.), multimedia memory cards (MMC), secure digital storage (SD) cards, smart media cards, compact flash memory (CF) cards, memory sticks, etc. In addition, non-volatile computer-readable storage media may also include conductive bridged random access memory (CBRAM), phase change random access memory (PRAM), ferroelectric random access memory (FeRAM), non-volatile random access memory (NVRAM), magnetoresistive random access memory (MRAM), resistive random access memory (RRAM), silicon-oxide-nitride-oxide-silicon memory (SONOS), floating junction gate random access memory (FJG RAM), millipede memory, racetrack memory, etc.

[0054] In one embodiment, the volatile computer-readable storage medium may include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), fast page mode dynamic random access memory (FPM DRAM), extended data output dynamic random access memory (EDO DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), double data rate type 2 synchronous dynamic random access memory (DDR2 SDRAM), double data rate type 3 synchronous dynamic random access memory (DDR3 SDRAM), Rambus dynamic random access memory (RDRAM), dual transistor RAM (TTRAM), dual resistor RAM (T-RAM), zero capacitor (Z-RAM), Rambus embedded memory module (RIMM), dual in-line memory module (DIMM), single in-line memory module (SIMM), video random access memory (VRAM), cache memory (including various levels), flash memory, register memory, etc. It will be understood that, where the implementation is described as using a computer-readable storage medium, other types of computer-readable storage media may be used in place of the aforementioned computer-readable storage media, or other than the aforementioned computer-readable storage media.

[0055] It should be understood that various embodiments of this disclosure can also be implemented as methods, apparatus, systems, computing devices, computing entities, etc. Therefore, embodiments of this disclosure can take the form of data structures, apparatus, systems, computing devices, computing entities, etc., that execute similar execution instructions stored on a computer-readable storage medium to perform certain steps or operations. Consequently, embodiments of this disclosure can also take the form of entirely hardware embodiments, entirely computer program product embodiments, and / or embodiments including a combination of computer program products and hardware that perform certain steps or operations.

[0056] The embodiments of this disclosure are described below with reference to block diagrams and flowcharts. Therefore, it should be understood that each block in the block diagrams and flowcharts can be implemented as a computer program product, a completely hardware embodiment, a combination of hardware and computer program products, and / or an apparatus, system, computing device, computing entity, etc., that executes instructions, operations, steps, and interchangeable similar terms (e.g., executable instructions, instructions for execution, program code, etc.) on a computer-readable storage medium. For example, code retrieval, loading, and execution can be performed sequentially such that one instruction is retrieved, loaded, and executed at a time. In some exemplary embodiments, retrieval, loading, and / or execution can be performed in parallel such that multiple instructions are retrieved, loaded, and / or executed together. Thus, such embodiments can produce machines with specific configurations for performing the steps or operations specified in the block diagrams and flowcharts. Therefore, the block diagrams and flowcharts support various combinations of embodiments for performing specified instructions, operations, or steps.

[0057] In environments such as distribution centers and warehouses, objects can be transported on pallets and unpacked from pallet stacks using one or more robotic depalletizing units (e.g., boxes, items, products).

[0058] In some implementations, objects in a stack have a single, identical stock unit (SKU), indicating that all objects in the stack are of the same type (e.g., have the same size and shape). In such examples, because the size and shape are determined based on the same SKU, the example depalletizing system may only need to detect the location of the objects to depalletize them.

[0059] In some implementations, at least some of the objects in the stack have mixed, different SKUs, indicating that the objects in the stack have different types (e.g., different sizes and / or shapes). In such examples, the example depalletizing system may rely on a vision system (such as a perception subsystem) to determine the size and / or shape of the objects, such that the depalletizing device can depalletize the objects from the pallet (e.g., calculating the pose of the depalletizing device based at least in part on the size and / or shape of the objects).

[0060] While most such perception systems rely on machine learning models to perform this time-consuming operation, one of the main bottlenecks in depalletizing systems is perception. For example, in mixed SKU scenarios, many depalletizing systems need to use vision systems (and machine learning models) to determine the size and shape of each object in order to determine the pose of the depalletizer unit when unloading the object, resulting in significant time constraints and reduced operational efficiency.

[0061] For example, many perception systems in depalletizing robots rely on machine learning models to determine the size and shape of products using the robot's pose. This is an expensive and time-consuming process because the perception system detects the position of products, the size, and shape of each product during the depalletizing operation. In contrast, various embodiments of this disclosure overcome these technical challenges and difficulties. For example, various embodiments of this disclosure provide a depalletizing robot operating in a constant pallet mode. Based on images received from the vision system, when the control subsystem determines that items on the top layer of the pallet have similar dimensions, the depalletizing device is switched to a constant pallet mode. In this mode, the control subsystem uses the same settings, speed profiles, etc., for the depalletizing device as long as it is in constant pallet mode. Furthermore, no new images are provided in constant pallet mode. When the vision system sends a new image indicating that the next item to be picked up does not have a similar size, the control subsystem causes the depalletizing device to switch out of constant pallet mode and begin operating in the default mode.

[0062] The various embodiments disclosed herein overcome the aforementioned technical challenges and difficulties, and provide a variety of technical benefits and advantages.

[0063] For example, continuing with this mixed SKU scenario, although there are different SKUs, some products may have the same or similar shape and size.

[0064] For example, in a mixed SKU scenario, a first object, a second object, and a third object are placed on a pallet. While the first object, the second object, and the third object may have different SKUs and different sizes and / or shapes, the differences in size and / or shape are within acceptable limits. In such an example, the depalletizing system can reduce operation time by minimizing the time required to trigger the sensing system to capture image data of similar products. For example, the depalletizing system can trigger the sensing system once to determine the pick-up pose of the depalletizer (e.g., the lifting height of the depalletizer) for all products of similar size and pick them up incrementally.

[0065] Therefore, the various embodiments of this disclosure provide well-defined and proven solutions for improving the productivity and efficiency of depalletizing systems. For example, solutions according to embodiments of this disclosure can be implemented in multiple group pick systems where the system attempts to unload mixed SKU products. Thus, the example depalletizing system according to embodiments of this disclosure can provide a unified solution to multiple problems and can significantly reduce operation cycles.

[0066] In various embodiments of this disclosure, the depalletizing system is highly dependent on the sensing system because the sensing system detects the size and pose of the object / packaging / product.

[0067] In some implementations, the perception system first uses a machine learning segmentation model to detect all fully visible object / package / product segments on the top layer of the pallet. Then, the control subsystem calculates the length, width, and height of each object / package / product from the pallet base. Subsequently, the control subsystem compares the mentioned attributes of all visible objects / packages / products on that layer, and if all of these fall within a certain threshold range, a constant pallet pattern is triggered, which identifies all products on the visible layer as having the same attributes.

[0068] In some implementations, the average packaging height from the pallet base of the currently detected constant pallet is stored. For any subsequent constant pallet detection, its average packaging height is compared with the stored value. If the difference is large enough, within a certain tolerance range, the pallet scenario is again identified as a constant pallet mode, and the stored value is updated to the average packaging height of the current scenario.

[0069] In some implementations, after the sensing system sends a detection signal of a constant pallet pattern at all gripping points of the depalletizer unit located on a pallet layer, the system stores all gripping poses of the depalletizer unit and attempts to perform gripping sequentially. In some implementations, when picking up the first object, the depalletizer system uses laser-based hardware to measure the product's height and raises the remaining objects / packages / products in the stored gripping poses to this determined height. In some implementations, when the constant pallet pattern ends, the sensing subsystem triggers sensing again to obtain new sensing output.

[0070] In some implementations, the two main conditions for the depalletizer to exit constant pallet mode are (1) if the depalletizer has performed all grips, and / or (2) if the currently detected box height is not within a defined threshold of the first box height (greater than or less than the defined threshold).

[0071] Figure 1A This is an example perspective view of an example depalletizing system 100A that can be used according to various embodiments of the present disclosure.

[0072] exist Figure 1A In the example shown, the example depalletizing system 100A includes a depalletizer device 103. In some embodiments, the depalletizer device 103 is part of the execution subsystem of the depalletizing system 100A.

[0073] exist Figure 1AIn the example shown, the depalletizer device 103 may take the form of a robotic depalletizer. For example, the depalletizer device 103 may include an end effector 113 and a robotic arm 115 connected to the end effector 113. In some embodiments, the depalletizer device 103 may include one or more controllers, sensors, and / or actuators to arouse and control the operation of the end effector 113 and / or the robotic arm 115.

[0074] In some embodiments, the depalletizer assembly 103 is located near the pallet 101. In some embodiments, one or more gripping poses and gripping points are calculated for the depalletizer assembly 103 in order to depalletize objects from the pallet 101. For example, the robotic arm 115 may move and / or rotate such that the end effector 113 can be positioned on the objects on the pallet 101. In some embodiments, the end effector 113 may retrieve and / or grip the objects (e.g., via a suction mechanism, etc.), and the robotic arm 115 may move to lift the objects to a lifting height (and the height of the end effector 113 is referred to as the gripping pose height). After lifting the objects, the robotic arm 115 may move and / or rotate such that the objects gripped by the end effector 113 are positioned above the conveyor 111. Subsequently, the end effector 113 releases the objects onto the conveyor 111, thereby completing the operation of depalletizing objects from the pallet 101 onto the conveyor 111.

[0075] As described above, a depalletizing system can rely on a sensing subsystem to capture data associated with objects, enabling the determination of the depalletizer's gripping pose and gripping point. Now refer to... Figure 1A As shown in the example, the example depalletizing system 100A includes a vision structure 105. In some embodiments, the vision structure 105 is part of the sensing subsystem of the depalletizing system 100A.

[0076] In some embodiments, the vision structure 105 is located near the tray 101. For example, the vision structure 105 includes a vertical beam 109 connected to the horizontal beam 107. In some embodiments, one or more image capturing devices may be disposed on the horizontal beam 107 and / or the vertical beam 109.

[0077] For example, a 2-D image capture device may be positioned near the end of the horizontal beam 107 and facing the tray 101 to capture a 2-D image, which is a top view of the tray 101 (e.g., showing an object on the top tray layer). Examples of 2-D image capture devices may include, but are not limited to, cameras, 2-D image sensors, etc. In some embodiments, the 2-D image can be used to determine the width and / or length of an object.

[0078] Alternatively or additionally, a 3D image capture device may be positioned on the horizontal beam 107 and facing the tray 101 to capture a 3D image of the tray 101. In some embodiments, the 3D image capture device may be movably positioned along the vertical beam 109 and / or the horizontal beam 107. Examples of the 3D image capture device may include, but are not limited to, time-of-flight image sensors, stereo imaging sensors, etc. In some embodiments, the 3D image may be used to determine the width and / or height of an object.

[0079] In some embodiments, a height sensing device may be located at the end of conveyor 111 near pallet 101. In some embodiments, the height sensing device may be part of depalletizing system 100A and configured to sense height data, as described in detail herein. Examples of height sensing devices include, but are not limited to, LiDAR sensors.

[0080] Figure 1B This is an example diagram illustrating a usable example depalletizing system 100B according to various embodiments of the present disclosure.

[0081] exist Figure 1B In the example shown, the example depalletizing system 100B includes a depalletizing device 131, which includes an end effector 133 connected to a robotic arm 135. In some embodiments, the end effector 133 can unpile objects from a pallet 137 onto a conveyor 139, similar to the above-described combination. Figure 1A Those described.

[0082] Figure 2 This is an example schematic diagram of example components in an example depalletizing system 200 according to various embodiments of the present disclosure. Specifically, Figure 2 Example data communication between various components of the example depalletizing system 200 is shown.

[0083] exist Figure 2 In the example shown, the example depalletizing system 200 includes a sensing subsystem 202, an execution subsystem 210, and a control subsystem 218 that can exchange data and / or information via a system bus 216.

[0084] In some implementations, the sensing subsystem may generate imaging data and transmit the imaging data to the control subsystem 218 via system bus 216. Specifically, the sensing subsystem 202 may include a 2-D image capture device 204 (similar to at least combined with the above). Figure 1AThe described 2D image capture device. In some embodiments, the 2D image capture device 204 may generate 2D image data and transmit the 2D image data to the control subsystem 218 via system bus 216. Alternatively, the sensing subsystem 202 may include a 3D image capture device 206 (similar to at least in combination with the above description). Figure 1A (The described 3-D image capture device). In some embodiments, the 3-D image capture device 206 can generate 3-D image data and transmit the 3-D image data to the control subsystem 218 via the system bus 216.

[0085] In some implementations, the control subsystem 218 may send control commands to the execution subsystem 210 via the system bus 216 in order to control the operation of the devices associated with the execution subsystem 210.

[0086] In some embodiments, the execution subsystem 210 may include a height sensing device 208. In some embodiments, the height sensing device 208 may generate height data and send the height data to the control subsystem 218.

[0087] For example, execution subsystem 210 may include depalletizer device 212. In such an example, control subsystem 218 may send control commands to depalletizer device 212 to control the operation of depalletizer device 212 and / or cause depalletizer device 212 to operate in a certain manner, as described in detail herein.

[0088] Alternatively or concurrently, the execution subsystem 210 may include a conveyor 214. In such an example, the control subsystem 218 may send control commands to the conveyor 214 to control the operation of the conveyor 214 and / or to make the conveyor 214 operate in a certain manner.

[0089] In some implementations, system bus 216 may take many forms. For example, system bus 216 may be implemented using wired data transmission protocols such as Fiber Distributed Data Interface (FDDI), Digital Subscriber Line (DSL), Ethernet, Asynchronous Transfer Mode (ATM), Frame Repeater, Cable Service Interface Data Specification (DOCSIS), or any other wired transmission protocol. Similarly, system bus 216 can be configured to communicate via a wireless external communication network using any of a variety of protocols, such as General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), Code Division Multiple Access 1900 (CDMA1900), CDMA1900 1X (1xRTT), Wideband Code Division Multiple Access (WCDMA), Global System for Mobile Communications (GSM), Enhanced Data Rate Evolution of GSM (EDGE), Time Division Synchronous Code Division Multiple Access (TD-SCDMA), Network Standard (LTE), Evolved Universal Terrestrial Radio Access Network (E-UTRAN), Evolved Data Optimization (EVDO), High Speed ​​Packet Access (HSPA), High Speed ​​Downlink Packet Access (HSDPA), Institute of Electrical and Electronics Engineers (IEEE) 802.11 (Wi-Fi), Wi-Fi Direct, 802.16 (WiMAX), Ultra Wideband (UWB), Infrared (IR) protocol, Near Field Communication (NFC) protocol, Wibree, Bluetooth protocol, Universal Serial Bus (USB) protocol, and / or any other wireless protocol. Control subsystem 218 may use such protocols and standards to communicate using the following: Border Gateway Protocol (BGP), Dynamic Host Configuration Protocol (DHCP), Domain Name System (DNS), File Transfer Protocol (FTP), Hypertext Transfer Protocol (HTTP), TLS / SSL / secure HTTP, Interactive Mail Access Protocol (IMAP), Network Time Protocol (NTP), Simple Mail Transfer Protocol (SMTP), Remote Login, Transport Layer Security (TLS), Secure Sockets Layer (SSL), Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Datagram Congestion Control Protocol (DCCP), Stream Control Transmission Protocol (SCTP), Hypertext Markup Language (HTML), etc.

[0090] Figure 3A schematic diagram of a control subsystem 218 according to one embodiment of this disclosure is provided. Generally, the terms computing entity, entity, apparatus, system, and / or similar terms used interchangeably herein can refer to, for example, one or more computers, computing entities, desktop computers, mobile phones, tablets, phablets, laptops, distributed systems, articles / apparatus, terminals, servers or server networks, blade servers, gateways, switches, processing devices, processing entities, set-top boxes, relays, routers, network access points, base stations, etc., and / or any combination of apparatus or entities suitable for performing the functions, operations, and / or processes described herein. Such functions, operations, and / or processes may include, for example, transmitting, receiving, operating, processing, displaying, storing, determining, creating / generating, monitoring, evaluating, comparing, and / or similar terms used herein. In one embodiment, these functions, operations, and / or processes may be performed on data, content, information, and / or similar terms used herein.

[0091] As noted, in one embodiment, the control subsystem 218 may also include one or more network and / or communication interfaces 308 for communicating with various computing entities, such as by conveying data, content, information, and / or similar terms as may be used herein, that can be transmitted, received, manipulated, processed, displayed, stored, etc. For example, the control subsystem 218 may communicate with other components of the depalletizing system.

[0092] like Figure 3As shown, in one embodiment, the control subsystem 218 may include or communicate with one or more processing elements (e.g., processing element 305) (also referred to herein as processor, processing circuitry, and / or similar interchangeable terms), which communicate with other elements within the control subsystem 218 via, for example, a bus or network connection. It should be understood that processing element 305 can be embodied in a variety of different ways. For example, processing element 305 may be embodied as one or more complex programmable logic devices (CPLDs), microprocessors, multi-core processors, coprocessor entities, application-specific instruction set processors (ASIPs), and / or controllers. Furthermore, processing element 305 may be embodied as one or more other processing devices or circuitry. The term "circuitry" can refer to a completely hardware implementation or a combination of hardware and computer program products. Thus, processing element 305 may be embodied as an integrated circuit, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), programmable logic array (PLA), hardware accelerator, other circuitry, etc. It should be understood that processing element 305 may be configured for a particular purpose or configured to execute instructions stored in volatile or non-volatile media or otherwise accessible by processing element 305. Therefore, whether configured by hardware, computer program products, or a combination thereof, when configured accordingly, the processing element 305 may be able to perform the steps or operations according to the embodiments of this disclosure.

[0093] In one embodiment, the control subsystem 218 may also include volatile media (also referred to as non-volatile storage devices, memory, memory storage devices, memory circuitry, and / or similar terms used interchangeably herein) or communicate with volatile media. In one embodiment, the volatile storage device or memory may also include one or more memory elements 306 as described above, such as RAM, DRAM, SRAM, FPM DRAM, EDO DRAM, SDRAM, DDR SDRAM, DDR2 SDRAM, DDR3 SDRAM, RDRAM, RIMM, DIMM, SIMM, VRAM, cache memory, register memory, etc. As will be appreciated, the volatile storage device or memory element 306 can be used to store, for example, memory... Figure 3The processing element 305 shown executes at least a portion of a database, database example, database management system entity, data, application, program, program module, script, source code, object code, bytecode, compiled code, interpreted code, machine code, executable instructions, etc. Therefore, the database, database example, database management system entity, data, application, program, program module, script, source code, object code, bytecode, compiled code, interpreted code, machine code, executable instructions, etc., can be used to control certain aspects of the operation of the control subsystem 218 with the help of the processing element 305 and the operating system.

[0094] In one embodiment, the control subsystem 218 may also include or communicate with a non-volatile medium (also referred to as a non-volatile storage device, memory, memory storage device, memory circuit, and / or similar terms used interchangeably herein). In one embodiment, the non-volatile storage device or memory may include one or more non-volatile storage devices or storage media 307 as described above, such as hard disks, ROMs, PROMs, EPROMs, EEPROMs, flash memory, MMC, SD memory cards, memory sticks, CBRAMs, PRAMs, FeRAMs, RRAMs, SONOS, racetrack memory, etc. As will be appreciated, the non-volatile storage device or storage media 307 may store databases, database examples, database management system entities, data, applications, programs, program modules, scripts, source code, object code, bytecode, compiled code, interpreted code, machine code, executable instructions, etc. The terms database, database example, database management system entity, and / or similar terms are used interchangeably herein and may generally refer to a structured or unstructured collection of information / data stored in a computer-readable storage medium.

[0095] Storage medium 307 may also be embodied as one or more data storage devices, one or more separate database servers, or a combination of a data storage device and a separate database server. Furthermore, in some embodiments, storage medium 307 may be embodied as a distributed repository, such that some of the stored information / data is centrally stored in a location within the system, and other information / data is stored in one or more remote locations. Alternatively, in some embodiments, the distributed repository may be distributed across multiple remote storage locations. Examples of embodiments contemplated herein would include a cloud data storage system maintained by a third-party provider, where some or all of the information / data required to restore system operation may be stored. Additionally, the information / data required to restore system operation may also be partially stored in the cloud data storage system and partially stored in a locally maintained data storage system. More specifically, storage medium 307 may include one or more data storage areas configured to store information / data available in some embodiments.

[0096] As noted, in one embodiment, the control subsystem 218 may also include one or more network and / or communication interfaces 308 for communicating with various computing entities, such as by delivering data, content, information, and / or similar terms used interchangeably herein, that can be transmitted, received, manipulated, processed, displayed, stored, etc. Such communication may be performed using wired data transmission protocols, such as Fiber Distributed Data Interface (FDDI), Digital Subscriber Line (DSL), Ethernet, Asynchronous Transfer Mode (ATM), Frame Repeater, Cable Service Interface Data Specification (DOCSIS), or any other wired transmission protocol. Similarly, the control subsystem 218 can be configured to communicate via a wireless external communication network using any of a variety of protocols, such as General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), Code Division Multiple Access 1900 (CDMA1900), CDMA1900 1X (1xRTT), Wideband Code Division Multiple Access (WCDMA), Global System for Mobile Communications (GSM), Enhanced Data Rate Evolution of GSM (EDGE), Time Division Synchronous Code Division Multiple Access (TD-SCDMA), Network Standard (LTE), Evolved Universal Terrestrial Radio Access Network (E-UTRAN), Evolved Data Optimization (EVDO), High Speed ​​Packet Access (HSPA), High Speed ​​Downlink Packet Access (HSDPA), Institute of Electrical and Electronics Engineers (IEEE) 802.11 (Wi-Fi), Wi-Fi Direct, 802.16 (WiMAX), Ultra Wideband (UWB), Infrared (IR) protocol, Near Field Communication (NFC) protocol, Wibree, Bluetooth protocol, Universal Serial Bus (USB) protocol, and / or any other wireless protocol. Control subsystem 218 may use such protocols and standards to communicate using the following: Border Gateway Protocol (BGP), Dynamic Host Configuration Protocol (DHCP), Domain Name System (DNS), File Transfer Protocol (FTP), Hypertext Transfer Protocol (HTTP), TLS / SSL / secure HTTP, Interactive Mail Access Protocol (IMAP), Network Time Protocol (NTP), Simple Mail Transfer Protocol (SMTP), Remote Login, Transport Layer Security (TLS), Secure Sockets Layer (SSL), Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Datagram Congestion Control Protocol (DCCP), Stream Control Transmission Protocol (SCTP), Hypertext Markup Language (HTML), etc.

[0097] Figure 4 , Figure 5 and Figure 6 This is an example diagram illustrating an example object on an example tray according to various embodiments of the present disclosure. Specifically, Figure 4 An example tray with the same SKU is shown. Figure 5 and Figure 6 An example tray with different SKUs is shown.

[0098] exist Figure 4 In the example diagram 400 shown, objects are stacked on example tray 402 into tray layers, such as tray layer 404A, tray layer 404B, tray layer 404C, and tray layer 404D. Each tray layer may include one or more objects. Figure 4 In the example shown, objects in each tray layer may have the same SKU (e.g., the same size and the same shape), or may have similar sizes and shapes, and objects in different tray layers may have the same SKU (e.g., the same size and the same shape) or may have similar sizes and shapes. Therefore, Figure 4 Example Figure 400 illustrates an example scenario with the same SKU. In some implementations, when unloading objects from each of pallet layers 404A, 404B, 404C, and 404D, the control subsystem can cause the execution subsystem to operate in a constant pallet mode, as described in detail herein.

[0099] exist Figure 5 In the example diagram 500 shown, objects are stacked on example pallets 501 into pallet layers, such as pallet layers 503A, 503B, 503C, and 503D. Each pallet layer may include one or more objects. Figure 5 In the example shown, objects in each tray layer may have the same SKU (e.g., the same size and the same shape), or may have similar sizes and shapes, but objects in different tray layers may have different SKUs (e.g., different sizes and different shapes). Therefore, Figure 5 Example Figure 500 illustrates an example mixed SKU scenario. In some implementations, when unloading objects from each of pallet layers 503A, 503B, 503C, and 503D, the control subsystem can cause the execution subsystem to operate in a constant pallet mode, as described in detail herein.

[0100] exist Figure 6 In the example diagram 600 shown, objects are stacked on example pallets 602 into pallet layers, such as pallet layer 604A, pallet layer 604B, and pallet layer 604C. Each pallet layer may include one or more objects. Figure 6 In the example shown, the objects in each tray layer may have different SKUs (e.g., different sizes and / or different shapes), and the objects in different tray layers may have different SKUs (e.g., different sizes and / or different shapes). Therefore, Figure 6The example figure 600 illustrates an example mixed SKU scenario. In some implementations, the control subsystem can cause the execution subsystem to operate in a mode other than the constant tray mode.

[0101] Figure 7A and Figure 7B This is an example flowchart illustrating an example method 700 for determining whether to cause the execution subsystem associated with the object depalletizing system to operate in a constant pallet mode, according to various embodiments of the present disclosure.

[0102] like Figure 7A As shown, example method 700 begins at step / operation 701. After and / or in response to step / operation 701, example method 700 proceeds to step / operation 703. At step / operation 703, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can receive first imaging data associated with multiple objects set on a pallet from the sensing subsystem associated with the object depalletizing system.

[0103] For example, the first imaging data could be 2D image data captured by the 2D image capture device of the sensing subsystem (similar to the above at least combined) Figure 1A (Those described above). Alternatively or concurrently, the first imaging data may be 3D image data captured by the 3D image capture device of the sensing subsystem (similar to at least those described above). Figure 1A Those described).

[0104] In some embodiments, the 2D image capture device and / or 3D image capture device may be located on top of the pallet, and the first imaging data can provide a view of the top visible pallet layer, including the objects to be unloaded. Figure 9 Example images based on the first image data are shown and described.

[0105] Following / in response to step / operation 703, example method 700 proceeds to step / operation 705. At step / operation 705, processing elements (such as, but not limited to, those described above, at least in conjunction with...) are... Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can calculate one or more comparative dimensional measures associated with multiple objects, at least in part, based on the first imaging data.

[0106] In some implementations, based on the first imaging data, the processing element may calculate the number of pixels for the length, width, and / or height of each object (e.g., in a 2-D image and / or a 3-D image). To calculate a comparative dimensional metric between two objects, the processing element may calculate the difference between the number of pixels for the length of one object and the number of pixels for the length of another object, the difference between the number of pixels for the width of one object and the number of pixels for the width of another object, and / or the difference between the number of pixels for the height of one object and the number of pixels for the height of another object.

[0107] In some implementations, the processing element can calculate a comparative dimension metric between every two objects on the top tray layer so that one or more comparative dimension metrics associated with multiple objects are calculated at step / operation 705.

[0108] Following / in response to step / operation 705, example method 700 proceeds to box A, which connects... Figures 7A to 7B Now for reference. Figure 7B Following / in response to step / operation 705, example method 700 proceeds to step / operation 707. At step / operation 707, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can determine whether one or more comparison size metrics meet a comparison size threshold range. For example, the processing element can determine whether each of the one or more comparison size metrics is within the comparison size threshold range.

[0109] As an example, if the comparison size metric between two objects is 2 and the comparison size threshold range is 4, then the comparison size metric meets the comparison size threshold. As another example, if the comparison size metric between two objects is 4 and the comparison size threshold range is 2, then the comparison size metric does not meet the comparison size threshold.

[0110] In some implementations, the comparison size threshold range corresponds to the system's tolerance in determining whether two objects are considered sufficiently similar in size and / or shape.

[0111] If, at step / operation 707, the processing element determines that one or more comparison size metrics satisfy a comparison size threshold range, then example method 700 proceeds to step / operation 709. At step / operation 709, the processing element (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3The processing element 305 of the control subsystem 218 of the described example depalletizing system can cause the execution subsystem associated with the object depalletizing system to operate in a constant pallet mode in response to determining that one or more comparison size metrics meet a comparison size threshold range.

[0112] For example, as described above, the processing element calculates the length, width, and height of each object / package / product from the tray base and compares the mentioned attributes of all visible objects / packages / products on that layer. If they are all within a certain threshold range, the processing element will identify that all objects on the visible layer have the same attributes and trigger a constant tray mode for the execution subsystem.

[0113] Therefore, once it is determined that the size and shape of the objects on the tray layer are sufficiently similar, the processing element will trigger a constant tray mode.

[0114] Return to reference Figure 7B If, at step / operation 707, the processing element determines that one or more comparison size metrics do not meet the comparison size threshold range, then example method 700 proceeds to step / operation 711. At step / operation 711, the processing element (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system may, in response to determining that one or more comparison size metrics do not meet the comparison size threshold range, cause the execution subsystem associated with the object depalletizing system to operate in a mode other than the constant pallet mode.

[0115] For example, if a comparison size metric indicates that the size difference (e.g., length, width, and / or height) between one object on the top visible layer and another object on the top visible layer does not meet a comparison size threshold range, the processing element determines that the objects are not similar enough in size and shape, and may cause the execution subsystem associated with the object depalletizing system to operate in a mode other than the constant pallet mode (e.g., in the default mode).

[0116] Return to reference Figure 7B After step / operation 709 and / or step / operation 711 and / or in response to that step / operation, example method 700 proceeds to step / operation 713 and ends.

[0117] Figure 8 This is an example flowchart illustrating an example method 800 for generating object segmentation instructions according to various embodiments of the present disclosure.

[0118] like Figure 8As shown, example method 800 begins at step / operation 802. After and / or in response to step / operation 802, example method 800 proceeds to step / operation 804. At step / operation 804, processing elements (such as, but not limited to, those described above at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can receive 2-D image data associated with multiple objects and captured by a 2-D image capture device.

[0119] In some implementations, the sensing subsystem includes a 2-D image capture device. In some implementations, the first imaging data (e.g., in conjunction with the above) Figure 7A and Figure 7B Those described include 2D image data associated with multiple objects on the top tray layer and captured by a 2D image capture device.

[0120] Following / in response to step / operation 804, example method 800 proceeds to step / operation 808. At step / operation 808, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can at least input 2-D image data into an object segmentation machine learning model.

[0121] In some implementations, the processing element can calculate one or more comparative dimensional measures associated with multiple objects (e.g., similar to at least the combination described above). Figure 7A Before the described step / operation 705), the imaging data (such as those described above, at least in combination with...) Figure 7A and Figure 7B The first imaging data (described above) is input into the object segmentation machine learning model. As described above, the first imaging data includes at least one of the 2-D image data associated with multiple objects on the top tray layer.

[0122] In some implementations, an object segmentation machine learning model is a machine learning model trained to identify / segment different objects in 2-D image data. Examples of object segmentation machine learning models may include, but are not limited to, image classification models, artificial neural networks, etc.

[0123] Following / in response to step / operation 808, example method 800 proceeds to step / operation 810. At step / operation 810, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can receive multiple object segmentation instructions from the object segmentation machine learning model.

[0124] In some implementations, multiple object segmentation indicators are associated with at least one piece of data in the 2-D image data. For example, an object segmentation machine learning model can identify / segment different objects from 2-D image data and can generate an object segmentation indicator for each of the identified objects. At least in combination Figure 9 Example 2-D images with object segmentation indicators are shown and described.

[0125] In some implementations, one or more comparative size metrics are calculated based at least in part on multiple object segmentation indicators (e.g., similar to the above, at least in combination). Figure 7A (The steps / operations described in 705). For example, the processing element may first identify all objects in the image data, at least in part, based on object segmentation indicators, and then calculate comparative size metrics, similar to those described herein.

[0126] Return to reference Figure 8 After step / operation 810 and / or in response to that step / operation, example method 800 proceeds to step / operation 812 and ends.

[0127] Figure 9 This is an example image illustrating example object segmentation instructions according to various embodiments of the present disclosure.

[0128] Figure 9 Example 2-D image 901 is shown, which shows an object on the top visible layer of a tray. Figure 9 A height sensing device 905 disposed on the edge of the conveyor is also shown.

[0129] Similar to the above combination Figure 8 The described processing elements can feed a 2-D image 901 to an object segmentation machine learning model. An object segmentation machine learning model is a machine learning model trained to identify / segment different objects in 2-D or 3-D image data. Figure 9 In the example shown, the object segmentation machine learning model can generate one or more object segmentation indicators (such as object segmentation indicator 903A, object segmentation indicator 903B, and object segmentation indicator 903C) that correspond to one or more objects on the top visible layer of the tray of the 2-D image capture.

[0130] In some implementations, each indicator in the object segmentation indicator indicates the length edge, width edge, and / or height edge associated with the object. For example, in Figure 9In the example shown, one or more object segmentation indicators (such as object segmentation indicator 903A, object segmentation indicator 903B, and object segmentation indicator 903C) can be the shape of a rectangular box that marks the length and width edges of an object.

[0131] As described above, one or more comparative size metrics are calculated based at least in part on multiple object segmentation indicators (e.g., similar to the above, at least in combination). Figure 7A (as described in step / operation 705). For example, the processing element may use the length edge, width edge, and / or height edge associated with the object indicated by the object segmentation indicator to calculate one or more comparative dimensional measures, as described in detail herein.

[0132] Figure 10 This is an example flowchart illustrating an example method 1000 for determining comparative dimensional metrics according to various embodiments of the present disclosure. In some embodiments, Figure 10 The example steps / operations shown can be implemented as (e.g., at least in combination) Figure 7A Step / operation 705) Calculates a portion of one or more comparative dimensional metrics associated with multiple objects. For example, multiple objects set on a tray include a first object and a second object, which are set on the same tray layer of the tray (e.g., the top visible layer captured by 2-D image data and / or 3-D image data).

[0133] like Figure 10 As shown, example method 1000 begins at step / operation 1002. After and / or in response to step / operation 1002, example method 1000 proceeds to step / operation 1004. At step / operation 1004, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can determine a first image size metric associated with the first object.

[0134] In some implementations, the processing element determines a first image size metric associated with the first object based at least in part on 2-D or 3-D image data from the sensing subsystem.

[0135] For example, based on object segmentation indications generated by object segmentation machine learning models (e.g., at least in part based on the above). Figure 8 and Figure 9 The processing element can determine the length edge, width edge, and / or height edge of the first object. Based on 2D or 3D image data, the processing element can calculate the number of pixels along the length edge, width edge, and / or height edge of the first object and use it as a measure of the first image size.

[0136] In some implementations, the processing element can provide 2-D or 3-D image data to one or more machine learning models, which are trained to generate image size metrics.

[0137] Return to reference Figure 10 Following / in response to step / operation 1002, example method 1000 proceeds to step / operation 1006. At step / operation 1006, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can determine a second image size metric associated with the second object.

[0138] In some implementations, the processing element determines a second image size metric associated with the second object based at least in part on 2-D or 3-D image data from the sensing subsystem.

[0139] For example, based on object segmentation indications generated by object segmentation machine learning models (e.g., at least in part based on the above). Figure 8 and Figure 9 The processing element can determine the length edge, width edge, and / or height edge of the second object. Based on 2D or 3D image data, the processing element can calculate the number of pixels along the length edge, width edge, and / or height edge of the second object and use it as a measure of the second image size.

[0140] In some implementations, the processing element can provide 2-D or 3-D image data to one or more machine learning models, which are trained to generate image size metrics.

[0141] Return to reference Figure 10 Following / in response to step / operation 1004 and / or step / operation 1006, example method 1000 proceeds to step / operation 1008. At step / operation 1008, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can determine the comparison size metric based at least in part on the first image size metric and the second image size metric.

[0142] In some implementations, the processing element determines a comparison size measure among one or more comparison size measures based at least in part on a first image size measure and a second image size measure.

[0143] Continuing with the example above, the processing element can calculate the difference between the number of pixels along the length and width edges of the first object (e.g., a first image size metric) and the number of pixels along the length and width edges of the second object (e.g., a second image size metric).

[0144] In some implementations, the height of a first object (e.g., a first image size metric) and the height of a second object (e.g., a second image size metric) can be calculated based on point clouds from a 3-D image capture device.

[0145] In some implementations, after determining the comparison size metric, similar to the various examples described herein, the processing element can compare the comparison size metric with a comparison size threshold range.

[0146] Return to reference Figure 10 After step / operation 1008 and / or in response to that step / operation, example method 1000 proceeds to step / operation 1010 and ends.

[0147] Figure 11A and Figure 11B This is an example flowchart illustrating an example method 1100 for determining whether one or more comparison size metrics meet a comparison size threshold range according to various embodiments of the present disclosure.

[0148] like Figure 11A As shown, example method 1100 begins with step / operation 1101. After and / or in response to step / operation 1101, example method 1100 proceeds to step / operation 1103. At step / operation 1103, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can calculate and compare length measurements.

[0149] In some implementations, the processing element may calculate a first image size metric associated with the first object, similar to at least the combination described above. Figure 10 The steps / operations described in step 1004. In some implementations, the first image size metric associated with the first object is the first image length metric associated with the first object.

[0150] In some implementations, the processing element may calculate a second image size metric associated with the second object, similar to the above at least in combination with Figure 10 The steps / operations described in step 1006. In some embodiments, the second image size metric associated with the second object is the second image length metric associated with the second object.

[0151] In some implementations, the processing element calculates a comparative size metric, which is expressed as a comparative length metric, based on a first image length metric and a second image length metric. For example, the processing element calculates the difference between the first image length metric and the second image length metric as the comparative length metric. In some implementations, this difference corresponds to the difference in the length of the object.

[0152] Return to reference Figure 11A Following / in response to step / operation 1103, example method 1100 proceeds to step / operation 1105. At step / operation 1105, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can determine whether the comparison length metric meets the comparison length threshold range.

[0153] As mentioned above, at least combine Figure 7B As described in step / operation 707, the processing element determines whether one or more comparison size metrics meet a comparison size threshold range. Figure 11A In the example shown at step / operation 1105, the comparison size threshold range is the comparison length threshold range.

[0154] If, at step / operation 1105, the processing element determines that the comparison length metric does not meet the comparison length threshold range, then example method 1100 proceeds to step / operation 1107. At step / operation 1107, the processing element (such as, but not limited to, at least in conjunction with the above)... Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can enable the execution subsystem associated with the object depalletizing system to operate in a mode other than the constant pallet mode (e.g., in the default mode).

[0155] After step / operation 1107 and / or in response to that step / operation, example method 1100 proceeds to step / operation 1109 and ends.

[0156] If, at step / operation 1105, the processing element determines that the comparison length metric meets the comparison length threshold range, then example method 1100 proceeds to step / operation 1111. At step / operation 1111, the processing element (such as, but not limited to, the above at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can calculate the comparison width metric.

[0157] In some implementations, the processing element may calculate a first image size metric associated with the first object, similar to at least the combination described above. Figure 10The steps / operations described in step 1004. In some implementations, the first image size metric associated with the first object is the first image width metric associated with the first object.

[0158] In some implementations, the processing element may calculate a second image size metric associated with the second object, similar to the above at least in combination with Figure 10 The steps / operations described in step 1006. In some embodiments, the second image size metric associated with the second object is the second image width metric associated with the second object.

[0159] In some implementations, the processing element calculates a comparative size metric, expressed as a comparative width metric, based on a first image width metric and a second image width metric. For example, the processing element calculates the difference between the first image width metric and the second image width metric as the comparative width metric. In some implementations, the difference corresponds to the difference in the width of the object.

[0160] Return to reference Figure 11A After and / or in response to step / operation 1111, example method 1100 proceeds to box A, which connects... Figures 11A to 11B Now for reference. Figure 11B Following / in response to step / operation 1111, example method 1100 proceeds to step / operation 1113. At step / operation 1113, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can determine whether the comparison width metric meets the comparison width threshold range.

[0161] As mentioned above, at least combine Figure 7B As described in step / operation 707, the processing element determines whether one or more comparison size metrics meet a comparison size threshold range. Figure 11B In the example shown at step / operation 1113, the comparison size threshold range is the comparison width threshold range.

[0162] If, at step / operation 1113, the processing element determines that the comparison width metric does not meet the comparison width threshold range, then example method 1100 proceeds to step / operation 1115. At step / operation 1115, the processing element (such as, but not limited to, at least in conjunction with the above)... Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can enable the execution subsystem associated with the object depalletizing system to operate in a mode other than the constant pallet mode (e.g., in the default mode).

[0163] After step / operation 1115 and / or in response to that step / operation, example method 1100 proceeds to step / operation 1117 and ends.

[0164] If, at step / operation 1113, the processing element determines that the comparison width metric meets the comparison width threshold range, then example method 1100 proceeds to step / operation 1119. At step / operation 1119, the processing element (such as, but not limited to, the above at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can calculate and compare height measurements.

[0165] In some implementations, the processing element may calculate a first image size metric associated with the first object, similar to at least the combination described above. Figure 10 The steps / operations described in step / operation 1004. In some embodiments, the first image size metric associated with the first object is a first image height metric associated with the first object. In some embodiments, the first image height metric is calculated based on a point cloud from a 3-D image capture device.

[0166] In some implementations, the processing element may calculate a second image size metric associated with the second object, similar to the above at least in combination with Figure 10 The steps / operations described in step / operation 1006. In some embodiments, the second image size metric associated with the second object is a second image height metric associated with the second object. In some embodiments, the second image height metric is calculated based on a point cloud from a 3-D image capture device.

[0167] In some implementations, the processing element calculates a comparative size metric, expressed as a comparative height metric, based on a first image height metric and a second image height metric. For example, the processing element calculates the difference between the first image height metric and the second image height metric as the comparative height metric. In some implementations, the difference corresponds to the difference in the height of the objects.

[0168] Return to reference Figure 11B Following / in response to step / operation 1119, example method 1100 proceeds to step / operation 1121. At step / operation 1121, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can determine whether the comparison height metric meets the comparison height threshold range.

[0169] As mentioned above, at least combine Figure 7B As described in step / operation 707, the processing element determines whether one or more comparison size metrics meet a comparison size threshold range. Figure 11A In the example shown at step / operation 1121, the comparison of size threshold ranges is the comparison of height threshold ranges.

[0170] If, at step / operation 1121, the processing element determines that the comparison height metric meets the comparison height threshold range, then example method 1100 proceeds to step / operation 1123. At step / operation 1123, the processing element (such as, but not limited to, the above at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can enable the execution subsystem associated with the object depalletizing system to operate in a constant pallet mode.

[0171] If, at step / operation 1121, the processing element determines that the comparison height metric does not meet the comparison height threshold range, then example method 1100 proceeds to step / operation 1125. At step / operation 1125, the processing element (such as, but not limited to, the above at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can enable the execution subsystem associated with the object depalletizing system to operate in a mode other than the constant pallet mode (e.g., in the default mode).

[0172] After step / operation 1123 and / or step / operation 1125 and / or in response to that step / operation, example method 1100 proceeds to step / operation 1117 and ends.

[0173] therefore, Figure 10 , Figure 11A and Figure 11B The example shown illustrates an example method for determining whether a constant tray mode is triggered. For instance, a perception subsystem may capture a 2-D or 3-D image showing objects on the top visible layer of a tray. Based on the 2-D and / or 3-D images, a control subsystem may implement an object segmentation machine learning model to identify objects on the top visible layer from the 2-D and / or 3-D images, and may calculate the length, width, and height of each object. In some implementations, the control subsystem calculates the length difference between every two objects, the width difference between every two objects, and the height difference between every two objects. The control subsystem then compares these length differences, width differences, and height differences with their corresponding comparison threshold ranges (which may be the same or different). If the length differences, width differences, and height differences all meet their corresponding comparison threshold ranges, the control subsystem causes the execution subsystem to operate in a constant tray mode. If any of the length differences, width differences, and height differences do not meet their corresponding comparison threshold ranges, the control subsystem causes the execution subsystem to operate in a mode other than the constant tray mode.

[0174] Figure 12This is an example flowchart illustrating an example method 1200 for operating an execution subsystem associated with an object depalletizing system in a constant pallet mode according to various embodiments of the present disclosure.

[0175] In some implementations, similar to the various examples described herein, the execution subsystem includes a depalletizer device. In some implementations, the execution subsystem includes a height sensing device.

[0176] like Figure 12 As shown, example method 1200 begins at step / operation 1202. After and / or in response to step / operation 1202, example method 1200 proceeds to step / operation 1204. At step / operation 1204, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can enable the depalletizing device to transfer a first object from a plurality of objects to above the height sensing device.

[0177] In some embodiments, the processing element enables the depalletizer to convey each of a plurality of objects above the height sensing device. In some embodiments, the depalletizer is configured to convey each of a plurality of objects above the height sensing device.

[0178] For example, such as Figure 9 As shown, the height sensing device 905 is disposed on the edge of the conveyor. When the depalletizer unplacing an object from a pallet onto the conveyor, the depalletizer moves the object above the height sensing device 905 and then onto the conveyor, and the height sensing device 905 can detect the height of the object.

[0179] Return to reference Figure 12 Following / in response to step / operation 1204, example method 1200 proceeds to step / operation 1206. At step / operation 1206, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can receive first height data associated with the first object from the height sensing device.

[0180] For example, because the depalletizer moves the first object above the height sensing device at step / operation 1204, the first height data can indicate the height of the first object. For example, the first height data can indicate the height from the bottom of the object to the ground.

[0181] Return to reference Figure 12Following / in response to step / operation 1206, example method 1200 proceeds to step / operation 1208. At step / operation 1208, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can determine, at least in part, the lifting height parameter associated with the depalletizing device in constant pallet mode based on the first height data.

[0182] In some implementations, the lift height parameter may define or indicate the height to which the depalletizer should lift one or more other objects on the same pallet layer as the first object when the depalletizer unpalletizes one or more objects from a pallet.

[0183] In some implementations, when the execution subsystem is in constant pallet mode, the processing element can determine the lifting height parameter for unstacking objects in the pallet layer based on the height to which the destacking device lifts the first object in the pallet layer.

[0184] In some implementations, the sensing subsystem can determine, at least in part, the height at which an object remains on a tray and / or the height at which a box is picked up from a tray, based on image data.

[0185] Return to reference Figure 12 Following / in response to step / operation 1208, example method 1200 proceeds to step / operation 1210. At step / operation 1210, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can enable the depalletizing device to lift one or more objects, excluding the first object, to the same lifting height, at least in part, based on the lifting height parameter.

[0186] In some implementations, when the execution subsystem is in constant pallet mode, the processing element causes the depalletizer to lift one or more objects, excluding the first object, to the same lifting height, at least in part, based on the lifting height parameter.

[0187] In some implementations, when the execution subsystem is in constant pallet mode, the processing element causes the depalletizer to lift one or more objects (other than the first object) on the same pallet layer to the same lift height, at least in part, based on the lift height parameter.

[0188] In some implementations, the lifting height (e.g., how high the depalletizer should lift the object) can be determined based on the object height detected by the sensing subsystem, plus the maximum lifting height parameter or object height detected during constant pallet mode.

[0189] Therefore, the constant pallet pattern described according to various embodiments of this disclosure reduces the height at which objects are subsequently moved from the pallet to the conveyor while still maintaining operational safety, thus providing various technical improvements and advantages as described herein. In some embodiments, the lifting height is not determined based on data from the sensing subsystem, but rather the gripping pose height of the depalletizer device can be determined based on data from the sensing subsystem.

[0190] Return to reference Figure 12 After step / operation 1210 and / or in response to that step / operation, example method 1200 proceeds to step / operation 1212 and ends.

[0191] Figure 13 This is an example image illustrating the example operation of an execution subsystem in constant tray mode according to various embodiments of the present disclosure.

[0192] Specifically, Figure 13 Example image 1301 illustrates example gripping poses (such as, but not limited to, gripping poses 1303A and 1303B) associated with a depalletizer assembly of the execution subsystem. For example, each of the example gripping poses (such as, but not limited to, gripping poses 1303A and 1303B) defines the position of the end effector of the depalletizer assembly to pick up one or more objects from the top visible layer of the pallet. In some embodiments, the example gripping poses (such as, but not limited to, gripping poses 1303A and 1303B) may be generated at least in part based on one or more machine learning models.

[0193] like Figure 13 As shown in the example, the depalletizer can progressively lift objects and move them from the pallet to the conveyor. Furthermore, as mentioned above, when the depalletizer is in constant pallet mode, it can lift each object to the same height.

[0194] Figure 14A and Figure 14B This is an example flowchart illustrating an example method 1400 for determining, according to various embodiments of the present disclosure, whether to cause the execution subsystem associated with the object depalletizing system to exit a constant pallet mode.

[0195] like Figure 14A As shown, example method 1400 begins at step / operation 1402. After and / or in response to step / operation 1402, example method 1400 proceeds to step / operation 1404. At step / operation 1404, when the execution subsystem is in constant tray mode, processing elements (such as, but not limited to, those described above at least in conjunction with...) Figures 1A to 3The processing element 305 of the control subsystem 218 of the described example depalletizing system enables the depalletizing device to lift a second object from a plurality of objects to a lifting height, at least in part, based on a lifting height parameter.

[0196] For example, prior to step / operation 1404, the processing element has already caused the depalletizer to lift the first object from among multiple objects (similar to at least combining...). Figure 12 The steps / operations described in 1202), and the lift height parameters have been determined (similar to at least combining) Figure 12 The steps / operations described in step / operation 1208).

[0197] Furthermore, as described above, when the execution subsystem is in constant pallet mode, the depalletizer can lift multiple objects on the pallet layer to the same height while unloading the remaining portions of the objects from the pallet (e.g., onto a conveyor). Therefore, the depalletizer can lift a second object from among multiple objects to the same height based on a lifting height parameter.

[0198] Return to reference Figure 14A Following / in response to step / operation 1404, example method 1400 proceeds to step / operation 1406. At step / operation 1406, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system enables the depalletizing device to transport a second object above the height sensing device.

[0199] For example, a height sensing device is located on the edge of the conveyor. When the depalletizer unpalletizes a second object from a pallet onto the conveyor, the depalletizer moves the object above the height sensing device and then onto the conveyor, and the height sensing device can sense the height of the second object, similar to at least the combination described above. Figure 12 The steps / operations described in step 1204.

[0200] Return to reference Figure 14A Following / in response to step / operation 1406, example method 1400 proceeds to step / operation 1408. At step / operation 1408, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can receive second height data associated with the second object from the height sensing device.

[0201] For example, because the depalletizer moves the second object above the height sensor at step / operation 1204, the second height data can indicate the height of the second object.

[0202] Return to reference Figure 14A After and / or in response to step / operation 1408, example method 1400 proceeds to box A, which connects... Figures 14A to 14B Now for reference. Figure 14B Following / in response to step / operation 1408, example method 1400 proceeds to step / operation 1410. At step / operation 1410, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can calculate the object height difference metric based at least in part on the first height data and the second height data.

[0203] In some implementations, the object height difference metric can indicate the difference between the heights of objects. For example, the object height difference is measured between the heights of objects.

[0204] In some implementations, various embodiments of this disclosure may calculate the height of an object and then determine a measure of the height difference between the objects.

[0205] In some implementations, the height difference between objects in a constant pallet pattern can be determined at least in part based on height data from a height sensing device (e.g., a LiDAR sensor). For example, the height of a box can be determined based on the height of the end effector of a depalletizer from the ground minus the height of the box's bottom from the ground, as determined using a height sensing device. The height difference between the objects can then be determined.

[0206] In some implementations, a depalletizer is used to lift the first object to a predetermined height. In some implementations, objects following the first object are lifted to a height equal to or slightly greater than the height to which the first object was lifted.

[0207] Return to reference Figure 14B Following / in response to step / operation 1410, example method 1400 proceeds to step / operation 1412. At step / operation 1412, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can determine whether the object height difference metric meets the object height difference metric threshold range.

[0208] For example, the processing element can determine whether the object height difference metric is within the object height difference metric threshold range. If it is, the processing element can determine that the object height difference metric meets the object height difference metric threshold range. If not, the processing element can determine that the object height difference metric does not meet the object height difference metric threshold range.

[0209] If, at step / operation 1412, the processing element determines that the object height difference metric does not meet the object height difference metric threshold range, then example method 1400 proceeds to step / operation 1414. At step / operation 1414, the processing element (such as, but not limited to, at least in conjunction with the above)... Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system may, in response to determining that the object height difference metric does not meet the object height difference metric threshold range, cause the execution subsystem associated with the object depalletizing system to exit constant pallet mode.

[0210] For example, if the processing element determines that the height associated with the second object is significantly different from (e.g., outside the object height difference metric threshold) the height of the first object, the processing element can cause the execution subsystem to exit constant tray mode. Subsequently, the execution subsystem can recalibrate and calculate the lift height of subsequent objects based on image data from the perception subsystem.

[0211] If, at step / operation 1412, the processing element determines that the object height difference metric meets the object height difference metric threshold range, then example method 1400 proceeds to step / operation 1416. In step / operation 1416, the processing element (such as, but not limited to, at least in conjunction with the above)... Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can keep the execution subsystem associated with the object depalletizing system in a constant pallet mode in response to determining that the object height difference metric meets the object height difference metric threshold range.

[0212] Return to reference Figure 14B After step / operation 1414 and / or step / operation 1416 and / or in response to that step / operation, example method 1400 proceeds to step / operation 1418 and ends.

[0213] Figure 15 This is an example flowchart illustrating an example method 1500 for determining whether to cause an execution subsystem associated with an object depalletizing system to exit a constant pallet mode, according to various embodiments of the present disclosure.

[0214] like Figure 15 As shown, example method 1500 begins with step / operation 1501. After and / or in response to step / operation 1501, example method 1500 proceeds to step / operation 1503. At step / operation 1503, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can determine whether the depalletizing device has depalletized each of the multiple objects on the same pallet layer.

[0215] For example, as described above, the processing element can determine the total number of objects on the top visible layer of the pallet based at least in part on 2-D image data and / or object segmentation indications generated by an object segmentation machine learning model. The processing element can then determine whether the depalletizing unit has depalletized all objects based on this total number.

[0216] Alternatively, the processing element may determine the gripping posture of the depalletizer device, and the processing element may determine whether the depalletizer device has performed all gripping postures in order to determine whether the depalletizer device has depalletized each of the multiple objects on the same pallet layer.

[0217] If, at step / operation 1503, the processing element determines that the depalletizer has delivered each of the multiple objects on the same pallet layer, then example method 1500 proceeds to step / operation 1505. At step / operation 1505, the processing element (such as, but not limited to, the above at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can, in response to determining that the depalletizing device has delivered each of the multiple objects on the same pallet layer, cause the execution subsystem associated with the object depalletizing system to exit the constant pallet mode.

[0218] For example, once the processing element determines that the depalletizer has unloaded all objects from the top visible layer of the pallet, the processing element can cause the execution subsystem to exit constant pallet mode. Subsequently, according to the various examples described herein, the processing element can determine whether to cause the execution subsystem to enter constant pallet mode when unloading the next layer of objects.

[0219] If, at step / operation 1503, the processing element determines that the depalletizer has not yet delivered each of the multiple objects on the same pallet layer, then example method 1500 proceeds to step / operation 1507. At step / operation 1507, the processing element (such as, but not limited to, the above at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can, in response to determining that the depalletizing device has not yet delivered each of the multiple objects on the same pallet layer, keep the execution subsystem associated with the object depalletizing system in a constant pallet mode.

[0220] Return to reference Figure 15 After step / operation 1505 and / or step / operation 1507 and / or in response to that step / operation, example method 1500 proceeds to step / operation 1509 and ends.

[0221] Figure 16A and Figure 16BThis is an example flowchart illustrating an example method 1600 for determining, according to various embodiments of the present disclosure, whether to allow the execution subsystem associated with the object depalletizing system to continue operating in a constant pallet mode.

[0222] like Figure 16A As shown, example method 1600 begins at step / operation 1602. After and / or in response to step / operation 1602, example method 1600 proceeds to step / operation 1604. At step / operation 1604, processing elements (such as, but not limited to, those at least combined with the above) are processed. Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can determine a first average object height metric associated with multiple objects.

[0223] For example, a first average object height metric can be determined based on the average of height metrics, where each height metric is the height metric of an object on the top layer of the pallet from the pallet base / ground.

[0224] Return to reference Figure 16A Following / in response to step / operation 1604, example method 1600 proceeds to step / operation 1606. At step / operation 1606, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can store a first average object height metric in a memory element.

[0225] Return to reference Figure 16A Following / in response to step / operation 1606, example method 1600 proceeds to step / operation 1610. At step / operation 1610, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can determine a second average object height metric.

[0226] Similar to those described above in conjunction with step / operation 1604, the processing element can calculate a second average object height metric associated with multiple objects (e.g., objects on the top visible layer). Similar to those described above, the second average object height metric can be determined based on the average of the height metrics, where each height metric is the height of an object on the top layer of the tray from the tray base / ground.

[0227] In some implementations, the second average object height measurement may be determined at a time point after the time point at which the first average object height measurement is determined.

[0228] Return to reference Figure 16AAfter and / or in response to step / operation 1610, example method 1600 proceeds to box A, which connects... Figures 16A to 16B Now for reference. Figure 16B Following / in response to step / operation 1610, example method 1600 proceeds to step / operation 1612. At step / operation 1612, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can calculate the average height difference metric based on a first average object height metric and a second average object height metric.

[0229] For example, the processing element can calculate the average height difference metric by subtracting the second average object height metric from the first average object height metric.

[0230] Return to reference Figure 16B Following / in response to step / operation 1612, example method 1600 proceeds to step / operation 1614. At step / operation 1614, processing elements (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system can determine whether the average height difference metric meets the average height difference metric threshold range.

[0231] For example, if the average height difference measurement is within the average height difference measurement threshold range, the processing element determines that the average height difference meets the average height difference measurement threshold range. If the average height difference measurement exceeds the average height difference measurement threshold range, the processing element determines that the average height difference does not meet the average height difference measurement threshold range.

[0232] If, at step / operation 1614, the processing element determines that the average height difference metric meets the average height difference metric threshold range, then example method 1600 proceeds to step / operation 1616. At step / operation 1616, the processing element (such as, but not limited to, those described above, at least in conjunction with...) Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system may, in response to determining that the average height difference metric meets the average height difference metric threshold range, replace the first average object height metric in the memory element with the second average object height metric.

[0233] For example, the processing element can update the average object height metric stored in memory so that it reflects an accurate height metric based on the remaining objects that have not yet been unpacked.

[0234] If, at step / operation 1614, the processing element determines that the average height difference metric does not meet the average height difference metric threshold range, then example method 1600 proceeds to step / operation 1618. At step / operation 1618, the processing element (such as, but not limited to, at least in conjunction with the above)... Figures 1A to 3 The processing element 305 of the control subsystem 218 of the described example depalletizing system may, in response to determining that the average height difference metric does not meet the average height difference metric threshold range, cause the execution subsystem associated with the object depalletizing system to exit constant pallet mode.

[0235] For example, if the average height difference metric does not meet the average height difference metric threshold range, the processing element determines that the average height associated with the remaining objects that have not yet been destacking is significantly different from the average height of the objects when the constant pallet mode was first triggered. In such an example, the processing element causes the execution subsystem to exit the constant pallet mode, allowing the execution subsystem to recalibrate and calculate the lifting height of subsequent objects based on image data from the perception subsystem.

[0236] Return to reference Figure 16B After step / operation 1616 and / or step / operation 1618 and / or in response to that step / operation, example method 1600 proceeds to step / operation 1620 and ends.

[0237] Figure 17 This is an example flowchart illustrating an example method 1700 associated with an object depalletizing system according to various embodiments of the present disclosure.

[0238] At step / operation 1701, the perception subsystem of the object depalletizing system captures image data (such as, but not limited to, the 2-D image data and / or 3-D image data described above). At step / operation 1703, the control subsystem of the object depalletizing system determines, based on the various example methods described herein, whether to put the execution subsystem into a constant pallet mode. If so, at step / operation 1707, the control subsystem generates and stores all gripping poses for the depalletizing device. In some embodiments, the lift height is determined after the first pickup of the object from the pallet.

[0239] At step / operation 1710, the control subsystem determines whether all gripping poses associated with the depalletizer device have been executed. For example, if all gripping poses have been executed, it means that all objects on the top pallet layer have been depalletized, and the control subsystem triggers the sensing subsystem to generate image data from the next layer of objects, and determines whether to put the execution subsystem into constant pallet mode when depalletizing the next layer of objects.

[0240] If at step / operation 1710 the control subsystem determines that not all gripping poses have been executed, the control subsystem will progressively send the gripping poses to the depalletizer device for execution at step / operation 1709.

[0241] If the control subsystem determines that the execution subsystem will not enter a constant pallet mode at step / operation 1703, and / or subsequently progressively send gripping poses to the depalletizer device for execution at step / operation 1709, the depalletizer device may check the feasibility of each gripping pose at step / operation 1705, and may execute the gripping trajectory defined by the gripping pose at step / operation 1712.

[0242] Many modifications and other embodiments of the present disclosure will occur to those skilled in the art upon benefiting from the teachings presented in the foregoing description and the accompanying drawings. Therefore, it should be understood that the present disclosure is not limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terminology is used herein, it is used only in a general and descriptive sense and not for purposes of limitation.

Claims

1. An apparatus associated with an object depalletizing system, the apparatus comprising at least one processor and at least one non-transitory memory including computer program code, the at least one non-transitory memory and the computer program code being configured to utilize the at least one processor to cause the apparatus to: Receive first imaging data associated with multiple objects placed on a pallet from a sensing subsystem associated with the object depalletizing system; One or more comparative size measures associated with the plurality of objects are calculated, at least in part, based on the first imaging data; Determine whether the one or more comparison size metrics meet the comparison size threshold range; as well as In response to determining that the one or more comparative size metrics satisfy the comparative size threshold range, a control command is sent to cause the execution subsystem associated with the object depalletizing system, including a depalletizer device, to operate in a constant pallet mode, in which all objects on the top layer of the pallet are depalletized using the device with the same settings applied to the depalletizer device. as well as Send a control command to cause the execution subsystem to terminate the constant pallet mode when at least one of the following occurs: all objects on the same layer of the pallet are unstacking, and the height difference between the first and second objects among a plurality of objects on the same layer of the pallet does not meet the height difference threshold range. The heights of the first and second objects among the plurality of objects are received from the execution subsystem associated with the object unloading system.

2. The device of claim 1, wherein the sensing subsystem includes a two-dimensional (2-D) image capture device, wherein the first imaging data includes 2-D image data associated with the plurality of objects and captured by the 2-D image capture device.

3. The device of claim 1, wherein the sensing subsystem includes a three-dimensional (3-D) image capture device, wherein the first imaging data includes 3-D image data associated with the plurality of objects and captured by the 3-D image capture device.

4. The device according to claim 1, wherein, Before calculating the one or more comparative dimensional measures associated with the plurality of objects, the at least one non-transitory memory and the computer program code are configured to utilize the at least one processor to cause the device to: The first imaging data is input into an object segmentation machine learning model, wherein the first imaging data includes at least one image data of 2-D image data associated with the plurality of objects; as well as The object segmentation machine learning model receives multiple object segmentation instructions associated with at least one image data in the 2-D image data.

5. The device of claim 4, wherein the one or more comparative size metrics are calculated based at least in part on the plurality of object segmentation indications.

6. The device of claim 1, wherein the plurality of objects includes a first object and a second object disposed on the same tray layer of the tray.

7. The device according to claim 6, wherein, When calculating the one or more comparative dimensional measures associated with the plurality of objects, the at least one non-transitory memory and the computer program code are configured to utilize the at least one processor to cause the device to: The first image size metric associated with the first object is determined at least in part based on 2-D or 3-D image data from the perception subsystem. The second image size metric associated with the second object is determined at least in part based on the 2-D image data or the 3-D image data from the perception subsystem; as well as The comparison size measure is determined at least in part based on the first image size measure and the second image size measure.

8. A computer-implemented method, the computer-implemented method comprising: Receive first imaging data associated with multiple objects placed on a pallet from a sensing subsystem associated with the object depalletizing system; One or more comparative size measures associated with the plurality of objects are calculated, at least in part, based on the first imaging data; Determine whether the one or more comparison size metrics meet the comparison size threshold range; as well as In response to determining that the one or more comparative size metrics satisfy the comparative size threshold range, a control command is sent to cause the execution subsystem associated with the object depalletizing system, including a depalletizer device, to operate in a constant pallet mode, in which all objects on the top layer of the pallet are depalletized using the same settings for the depalletizer device. as well as Send a control command to cause the execution subsystem to terminate the constant pallet mode when at least one of the following occurs: all objects on the same layer of the pallet are unstacking, and the height difference between the first and second objects among a plurality of objects on the same layer of the pallet does not meet the height difference threshold range. The heights of the first and second objects among the plurality of objects are received from the execution subsystem associated with the object unloading system.

9. The computer-implemented method of claim 8, wherein the perception subsystem includes a two-dimensional (2-D) image capture device, wherein the first imaging data includes 2-D image data associated with the plurality of objects and captured by the 2-D image capture device.

10. The computer-implemented method of claim 8, wherein the perception subsystem includes a three-dimensional (3-D) image capture device, wherein the first imaging data includes 3-D image data associated with the plurality of objects and captured by the 3-D image capture device.

11. The computer-implemented method according to claim 8, wherein, Before calculating the one or more comparative size measures associated with the plurality of objects, the computer-implemented method further includes: The first imaging data is input into an object segmentation machine learning model, wherein the first imaging data includes at least one image data from 2-D image data associated with the plurality of objects; and The object segmentation machine learning model receives multiple object segmentation instructions associated with at least one image data in the 2-D image data.

12. The computer-implemented method of claim 11, wherein the one or more comparative size metrics are calculated based at least in part on the plurality of object segmentation indications.

13. The computer-implemented method of claim 8, wherein the plurality of objects includes a first object and a second object disposed on the same tray layer of the tray.

14. The computer-implemented method according to claim 13, wherein, When calculating the one or more comparative size measures associated with the plurality of objects, the computer-implemented method further includes: The first image size metric associated with the first object is determined at least in part based on 2-D or 3-D image data from the perception subsystem. The second image size metric associated with the second object is determined at least in part based on the 2-D image data or the 3-D image data from the perception subsystem; The comparison size measure is determined at least in part based on the first image size measure and the second image size measure.

15. A computer program product comprising at least one non-transitory computer-readable storage medium having a computer-readable program code portion stored therein, the computer-readable program code portion including an executable portion configured as follows: Receive first imaging data associated with multiple objects placed on a pallet from a sensing subsystem associated with the object depalletizing system; One or more comparative size measures associated with the plurality of objects are calculated, at least in part, based on the first imaging data; Determine whether the one or more comparison size metrics meet the comparison size threshold range; as well as In response to determining that the one or more comparative size metrics satisfy the comparative size threshold range, a control command is sent to cause the execution subsystem associated with the object depalletizing system, including a depalletizer device, to operate in a constant pallet mode, in which all objects on the top layer of the pallet are depalletized using the same settings for the depalletizer device. as well as Send a control command to cause the execution subsystem to terminate the constant pallet mode when at least one of the following occurs: all objects on the same layer of the pallet are unstacking, and the height difference between the first and second objects among a plurality of objects on the same layer of the pallet does not meet the height difference threshold range. The heights of the first and second objects among the plurality of objects are received from the execution subsystem associated with the object unloading system.

16. The computer program product of claim 15, wherein the sensing subsystem includes a two-dimensional (2-D) image capture device, wherein the first imaging data includes 2-D image data associated with the plurality of objects and captured by the 2-D image capture device.

17. The computer program product of claim 15, wherein the sensing subsystem includes a three-dimensional (3-D) image capture device, wherein the first imaging data includes 3-D image data associated with the plurality of objects and captured by the 3-D image capture device.

18. The computer program product according to claim 15, wherein, Before calculating the one or more comparative size measures associated with the plurality of objects, the computer-readable code portion includes an executable portion configured as follows: The first imaging data is input into an object segmentation machine learning model, wherein the first imaging data includes at least one image data of 2-D image data associated with the plurality of objects; as well as The object segmentation machine learning model receives multiple object segmentation instructions associated with at least one image data in the 2-D image data.

19. The computer program product of claim 18, wherein the one or more comparative size measures are calculated at least in part based on the plurality of object segmentation indications.

20. The computer program product of claim 15, wherein the plurality of objects includes a first object and a second object disposed on the same tray layer of the tray.

Images