Methods, apparatus, and computer program products are provided for a dynamic gap system for de-palletizing objects

Abstract

Methods, apparatuses, systems, computing devices, and / or the like are provided. An example method can include determining a height value associated with a highest object on a pallet, causing a light detection and ranging (LiDAR) sensor to travel to a certain height based at least in part on the height value, determining whether a detection signal from the LiDAR sensor indicates a first clear state, in response to determining that the detection signal indicates the first clear state, causing a robotic arm to lift the object until the detection signal indicates a blocked state and subsequently indicates a second clear state.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority and benefit to U.S. Provisional Patent Application No. 63 / 263,759, filed November 8, 2021, the entire contents of which are incorporated herein by reference. Background Technology

[0003] Palletizing refers to the operation of loading or transferring objects (such as, but not limited to, boxes, cartons, containers, articles, etc.) onto pallets or similar devices. Depalletizing refers to the operation of removing or removing objects (such as, but not limited to, boxes, cartons, containers, articles, etc.) from pallets or similar devices.

[0004] Many palletizing and / or depalletizing solutions are plagued by technical challenges and difficulties. For example, if one or more objects on a pallet have different sizes, many depalletizing solutions cannot properly unload or remove these objects from the pallet, and / or may take longer to unload or remove these objects from the pallet compared to the time required when unloading or removing objects of the same size from the pallet. Summary of the Invention

[0005] Generally speaking, the embodiments disclosed herein provide methods, equipment, systems, computing devices, computing entities, etc.

[0006] According to various embodiments of this disclosure, an apparatus is provided. The apparatus may include at least one processor and at least one non-transitory memory including computer program code. The at least one non-transitory memory and the computer program code may be configured to utilize the at least one processor to: cause the apparatus to determine a height value associated with the highest object on a tray; cause a light detection and ranging (LiDAR) sensor to travel to a certain height at least partially based on the height value; determine whether a detection signal from the LiDAR sensor indicates a first unobstructed state; and in response to determining that the detection signal indicates a first unobstructed state, cause a robotic arm to lift the object until the detection signal indicates an obstructed state and subsequently indicates a second unobstructed state.

[0007] According to various embodiments of this disclosure, a computer-implemented method is provided. This computer-implemented method may include: determining a height value associated with the highest object on a tray; causing a LiDAR sensor to travel to a height at least partially based on the height value; determining whether a detection signal from the LiDAR sensor indicates a first unobstructed state; and, in response to determining that the detection signal indicates the first unobstructed state, causing a robotic arm to lift the object until the detection signal indicates an obstructed state and subsequently indicates a second unobstructed state.

[0008] According to various embodiments of this disclosure, a computer program product is provided. The computer program product may include at least one non-transitory computer-readable storage medium in which a portion of computer-readable program code is stored. The computer-readable program code portion may include an executable portion configured to determine a height value associated with the highest object on a tray; cause a LiDAR sensor to travel to a height at least partially based on the height value; determine whether a detection signal from the LiDAR sensor indicates a first unobstructed state; and, in response to determining that the detection signal indicates a first unobstructed state, cause a robotic arm to lift the object until the detection signal indicates an obstructed state and subsequently indicates a second unobstructed state.

[0009] The above description of the invention is provided merely to outline some exemplary embodiments and to provide a basic understanding of some aspects of this disclosure. Therefore, it should be understood that the above embodiments are merely examples. It should be understood that, in addition to those summarized herein, the scope of this disclosure covers many possible embodiments, some of which will be further described below. Attached Figure Description

[0010] Therefore, this disclosure has been described in general terms, and reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and in which:

[0011] Figure 1 This is an exemplary perspective view of an exemplary depalletizing system that can be used according to various embodiments of the present disclosure;

[0012] Figure 2 This is an exemplary schematic diagram of exemplary components in an exemplary depalletizing system according to various embodiments of the present disclosure;

[0013] Figure 3 This is a schematic diagram of exemplary components in an exemplary control system according to various embodiments of the present disclosure;

[0014] Figure 4 , Figure 5 and Figure 6 This is an example diagram illustrating an exemplary tray layer on an exemplary tray according to various embodiments of the present disclosure;

[0015] Figure 7A and Figure 7B This is an exemplary perspective view of an exemplary depalletizing system, which includes an exemplary dynamic clearance system that can be used according to various embodiments of the present disclosure;

[0016] Figure 8 This is an exemplary flowchart illustrating an exemplary method of operating an exemplary dynamic gap system according to various embodiments of the present disclosure;

[0017] Figure 9Aand Figure 9B This is an exemplary flowchart illustrating an exemplary method of operating an exemplary dynamic gap system according to various embodiments of the present disclosure; and

[0018] Figure 10A and Figure 10B This is an example diagram illustrating an exemplary situation according to the present disclosure. Detailed Implementation

[0019] Various embodiments of this disclosure will now be described more fully below with reference to the accompanying drawings, which illustrate some, but not all, of the embodiments of this disclosure. In fact, this disclosure may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided to enable this disclosure to meet applicable legal requirements. Unless otherwise specified, the term “or” (also indicated as “ / ”) is used herein in both alternative and combined senses. The terms “illustrative” and “exemplary” are used for examples without an indication of quality level. Throughout the text, similar reference numerals may refer to similar elements. Phrases such as “in one embodiment,” “according to one embodiment,” etc., generally mean that the particular feature, structure, or characteristic following the phrase may be included in at least one embodiment of this disclosure, and may be included in more than one embodiment of this disclosure (importantly, such phrases do not necessarily refer to the same embodiment).

[0020] Embodiments of this disclosure can be implemented as computer program products including an article of art. Such computer program products may include one or more software components, including, for example, applications, software objects, methods, data structures, etc. Software components may be encoded in any of a variety of programming languages. An exemplary programming language may be a low-level programming language, such as assembly language associated with a specific hardware architecture and / or operating system platform / system. Software components including assembly language instructions may need to be converted into executable machine code by an assembler before execution by the hardware architecture and / or platform / system. Another example programming language may be a higher-level programming language that is portable across multiple architectures. Software components including higher-level programming language instructions may need to be converted into an intermediate representation by an interpreter or compiler before execution.

[0021] Other examples of programming languages ​​include, but are not limited to, macro languages, shell or command languages, job control languages, scripting languages, database query or search languages, and / or report writing languages. In one or more example implementations, software components including instructions from one of the foregoing examples of programming languages ​​can be executed directly by an operating system or other software components without first being translated into another form. Software components can be stored as files or other data storage constructs. Software components of similar type or related functionality can be stored together, such as in a specific directory, folder, or library. Software components can be static (e.g., pre-built or fixed) or dynamic (e.g., created or modified at runtime).

[0022] Additionally or alternatively, embodiments of this disclosure may be implemented as a non-transitory computer-readable storage medium storing applications, programs, program modules, scripts, source code, program code, object code, bytecode, compiled code, interpreted code, machine code, executable instructions, etc. (also referred to herein as executable instructions, instructions for execution, computer program products, program code, and / or similar terms used interchangeably herein). Such non-transitory computer-readable storage media include all computer-readable media (including volatile and non-volatile media).

[0023] In one embodiment, non-volatile computer-readable storage media may include floppy disks, floppy disks, hard disks, solid-state storage (SSS) (e.g., solid-state drives (SSDs), solid-state cards (SSCs), solid-state modules (SSMs), enterprise flash drives, magnetic tape, or any other non-transitory magnetic media). Non-volatile computer-readable storage media may also include punched cards, paper tape, optical marking sheets (or any other physical media with perforated patterns or other optically identifiable markings), optical disc read-only memory (CD-ROM), optical disc rewritable (CD-RW), digital versatile disc (DVD), Blu-ray disc (BD), and any other non-transitory optical media. Such non-volatile computer-readable storage media may also include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory (e.g., serial, NAND, NOR, etc.), multimedia memory cards (MMC), secure digital storage (SD) cards, smart media cards, compact flash memory (CF) cards, memory sticks, etc. In addition, non-volatile computer-readable storage media may also include conductive bridged random access memory (CBRAM), phase change random access memory (PRAM), ferroelectric random access memory (FeRAM), non-volatile random access memory (NVRAM), magnetoresistive random access memory (MRAM), resistive random access memory (RRAM), silicon-oxide-nitride-oxide-silicon memory (SONOS), floating junction gate random access memory (FJG RAM), millipede memory, racetrack memory, etc.

[0024] In one embodiment, the volatile computer-readable storage medium may include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), fast page mode dynamic random access memory (FPM DRAM), extended data output dynamic random access memory (EDO DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), double data rate type 2 synchronous dynamic random access memory (DDR2 SDRAM), double data rate type 3 synchronous dynamic random access memory (DDR3 SDRAM), Rambus dynamic random access memory (RDRAM), dual transistor RAM (TTRAM), dual resistor RAM (T-RAM), zero capacitor (Z-RAM), Rambus embedded memory module (RIMM), dual in-line memory module (DIMM), single in-line memory module (SIMM), video random access memory (VRAM), cache memory (including various levels), flash memory, register memory, etc. It will be understood that, where the implementation is described as using a computer-readable storage medium, other types of computer-readable storage media may be used in place of the aforementioned computer-readable storage media, or other than the aforementioned computer-readable storage media.

[0025] It should be understood that various embodiments of this disclosure can also be implemented as methods, apparatus, systems, computing devices, computing entities, etc. Therefore, embodiments of this disclosure can take the form of data structures, apparatus, systems, computing devices, computing entities, etc., that execute similar execution instructions stored on a computer-readable storage medium to perform certain steps or operations. Consequently, embodiments of this disclosure can also take the form of entirely hardware embodiments, entirely computer program product embodiments, and / or embodiments including a combination of computer program products and hardware that perform certain steps or operations.

[0026] The embodiments of this disclosure are described below with reference to block diagrams and flowcharts. Therefore, it should be understood that each block in the block diagrams and flowcharts can be implemented as a computer program product, a completely hardware embodiment, a combination of hardware and computer program products, and / or an apparatus, system, computing device, computing entity, etc., that executes instructions, operations, steps, and interchangeable similar terms (e.g., executable instructions, instructions for execution, program code, etc.) on a computer-readable storage medium. For example, code retrieval, loading, and execution can be performed sequentially such that one instruction is retrieved, loaded, and executed at a time. In some exemplary embodiments, retrieval, loading, and / or execution can be performed in parallel such that multiple instructions are retrieved, loaded, and / or executed together. Thus, such embodiments can produce machines with specific configurations for performing the steps or operations specified in the block diagrams and flowcharts. Therefore, the block diagrams and flowcharts support various combinations of embodiments for performing specified instructions, operations, or steps.

[0027] As mentioned above, many depalletizing solutions, such as robotic depalletizing units, are plagued by technical challenges and difficulties. For example, when depalletizing mixed stock unit (SKU) objects on a pallet, the physical dimensions of the picked-up boxes (e.g., the height of the boxes themselves) are unknown. Many depalletizing solutions assume that any given object picked up is at its maximum specification height for the robotic depalletizing unit and that the object can be lifted based on that maximum specification height. This directly impacts the number of picks per hour, as it may take longer to lift the object than is required.

[0028] Various embodiments of this disclosure overcome these technical challenges and difficulties and provide various technical benefits and improvements. For example, various examples of this disclosure provide a light detection and ranging (LiDAR) sensor positioned on the side of a pallet to detect when an object lifted by a robotic arm passes over the top of goods on the pallet. Various examples of this disclosure send signals to the robotic arm to stop lifting the object and move it to a placement position (e.g., a conveyor) when the object has been lifted to a sufficient height. This can increase the hourly pickup rate because the robotic arm does not need to lift the object to its maximum height (e.g., does not need to travel the full vertical distance) to depalletize the object. Therefore, various embodiments of this disclosure combine a LiDAR sensor on a mechanical guide with a vision system, resulting in a significant increase in the number of pickups per hour due to the reduced travel distance of the robot.

[0029] Figure 1 This is an exemplary perspective view of an exemplary depalletizing system 100A that can be used according to various embodiments of the present disclosure.

[0030] exist Figure 1In the example shown, the exemplary depalletizing system 100A includes a robotic arm 103.

[0031] exist Figure 1 In the example shown, robot arm 103 may include an end effector 113 and a robot arm body 115 connected to the end effector 113. In some embodiments, robot arm 103 may include one or more controllers, sensors, and / or actuators to arouse and control the operation of the end effector 113 and / or the robot arm body 115. For example, robot arm 103 may communicate with a robot controller, exemplary details of which are described herein.

[0032] In some embodiments, the robotic arm 103 is positioned near the pallet 101. In some embodiments, one or more gripping poses and gripping points of the robotic arm 103 are calculated in order to depalletize objects from the pallet 101. For example, the robotic arm body 115 may move and / or rotate such that the end effector 113 can be positioned on the object in the pallet 101. In some embodiments, the end effector 113 may retrieve and / or grip the object (e.g., via a suction mechanism, etc.), and the robotic arm body 115 may move to lift the object to a lifting height (and the height of the end effector 113 is referred to as the gripping pose height). After lifting the object, the robotic arm body 115 may move and / or rotate such that the object gripped by the end effector 113 is positioned above the conveyor 111. The end effector 113 then releases the object onto the conveyor 111, thereby completing the operation of depalletizing the object from the pallet 101 onto the conveyor 111.

[0033] In some implementations, the depalletizing system may rely on a vision system to capture object-related data, enabling the determination of the robotic arm's gripping pose and gripping point. (See now for reference.) Figure 1 As shown in the example, the exemplary depalletizing system 100A includes a support 105.

[0034] In some embodiments, the support 105 is positioned near the tray 101. For example, the support 105 includes a vertical beam 109 connected to a horizontal beam 107. In some embodiments, one or more image capturing devices may be mounted on the horizontal beam 107.

[0035] For example, a 2D image capture device may be positioned near the end of the horizontal beam 107 and facing the tray 101 to capture a 2D image, which is a top view of the tray 101 (e.g., showing an object on the top tray layer). Examples of 2D image capture devices may include, but are not limited to, cameras, 2D image sensors, etc.

[0036] Additionally or alternatively, the 3D image capture device may be mounted on the horizontal beam 107 and facing the tray 101 to capture 3D images of the tray 101. In some embodiments, the 3D image capture device may be movably mounted along the vertical beam 109 and / or the horizontal beam 107. Examples of the 3D image capture device may include, but are not limited to, time-of-flight image sensors, stereo imaging sensors, etc.

[0037] In some implementations, the height from the tallest object on the pallet to the ground (or to the top surface of the pallet) can be determined based on a 2D and / or 3D image of the pallet 101. However, as mentioned above, the dimensions of the object (e.g., the height of the object itself) may not be determined by the 2D and / or 3D image, thus causing the robotic arm 103 to lift the object to a default maximum height to avoid collisions with other objects on the pallet when depalletizing the object (which affects the hourly pickup rate).

[0038] In some embodiments, a height sensing device may be located at the end of conveyor 111 near pallet 101. In some embodiments, the height sensing device can sense height data, the details of which are described herein.

[0039] Figure 2 This is an exemplary schematic diagram of exemplary components in an exemplary depalletizing system 200 according to various embodiments of the present disclosure. Specifically, Figure 2 Exemplary communication between various components of an exemplary depalletizing system 200 is shown.

[0040] exist Figure 2 In the example shown, the exemplary depalletizing system 200 includes a vision system 202, a clearance system 216, and a programmable logic controller (PLC) 214, which can exchange data and / or information via devices such as, but not limited to, a system bus. In some embodiments, the exemplary depalletizing system 200 includes a robot controller (e.g., but not limited to, a robot controller). Universal Robot Controller (HURC) 210 and Robot Arm 212.

[0041] In some implementations, the vision system 202 can generate imaging data and transmit the imaging data to the robot controller 210 via a device such as, but not limited to, a system bus. Specifically, the vision system 202 may include a 2D image capture device 204 (similar to the one described above combined with at least...). Figure 1 The 2D image capture device described herein. In some embodiments, the 2D image capture device 204 can generate 2D image data and transmit the 2D image data to the robot controller 210 via a device such as, but not limited to, a system bus. Additionally or alternatively, the vision system 202 may include a 3D image capture device 206 (similar to the one described above combined with at least...). Figure 1(The 3D image capture device described). In some embodiments, the 3D image capture device 206 can generate 3D image data and transmit the 3D image data to a robot controller via a device such as, but not limited to, a system bus.

[0042] While the above description provides examples of data communication between the vision system 202 and other systems, it should be noted that the scope of this disclosure is not limited to the above description. In some embodiments, the exemplary vision system 202 may transmit data and / or information (such as, but not limited to, 2D images and / or 3D images) with the PLC 214. In some embodiments, the example vision system 202 may transmit data and / or information (such as, but not limited to, 2D images and / or 3D images) with the gap system 216.

[0043] In some implementations, the robot controller 210 may receive 2D and / or 3D images from the vision system 202 and provide the 2D and / or 3D images to the PLC 214. In some implementations, based on the 2D and / or 3D images, the vision system 202, the robot controller 210, and / or the PLC 214 may determine the height between the tallest object on the pallet and the ground.

[0044] exist Figure 2 In the example shown, the exemplary depalletizing system 200 also includes a gap system 216. The gap system 216 includes a LiDAR sensor 218 attached to the pallet holder assembly 220, the exemplary details of which are described herein. In some embodiments, after determining the height between the highest object on the pallet and the ground, the PLC 214 may transmit control commands to cause the pallet holder assembly 220 to raise the LiDAR sensor 218 at least in part based on the height between the highest object on the pallet and the ground, the details of which are described herein.

[0045] In some implementations, the exemplary depalletizing system 200 also includes a robotic arm 212 configured to lift objects (vertically) upwards from a pallet and (horizontally) away, similar to the various examples described herein. In some implementations, a PLC 214 may send control commands to the robotic arm 212, exemplary details of which are described herein.

[0046] In some implementations, PLC 214 transmits information such as, but not limited to, an early end-of-lift signal (e.g., a signal to stop the vertical lifting of the robot arm) to robot controller 210. In some implementations, robot controller 210 transmits information such as, but not limited to, the top layer of the pallet (e.g., the height value of the highest object on the pallet) to PLC 214.

[0047] In some implementations, the system bus that enables communication between the various components of the exemplary depalletizing system 200 can take various forms. For example, the system bus can be implemented using a wired data transmission protocol such as Fiber Distributed Data Interface (FDDI), Digital Subscriber Line (DSL), Ethernet, Asynchronous Transfer Mode (ATM), Frame Relay, Cable Data Service Interface Specification (DOCSIS), or any other wired transmission protocol. Similarly, the system bus can be configured to communicate via a wireless external communication network using any of a variety of protocols, such as General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), Code Division Multiple Access 1900 (CDMA1900), CDMA1900 1X (1xRTT), Wideband Code Division Multiple Access (WCDMA), Global System for Mobile Communications (GSM), Enhanced Data Rate GSM Evolution Technology (EDGE), Time Division Synchronous Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), Evolved Universal Terrestrial Radio Access Network (E-UTRAN), Evolved Data Optimized (EVDO), High Speed ​​Packet Access (HSPA), High Speed ​​Downlink Packet Access (HSDPA), IEEE 802.11 (Wi-Fi), Wi-Fi Direct, 802.16 (WiMAX), Ultra Wideband (UWB), Infrared (IR) protocol, Near Field Communication (NFC) protocol, Wibree, Bluetooth protocol, Wireless Universal Serial Bus (USB) protocol, and / or any other wireless protocol. In addition, the following protocols and standards may be used to transmit data and / or information, such as, but not limited to, Border Gateway Protocol (BGP), Dynamic Host Configuration Protocol (DHCP), Domain Name System (DNS), File Transfer Protocol (FTP), Hypertext Transfer Protocol (HTTP), TLS / SSL / secure HTTP, Internet Message Access Protocol (IMAP), Network Time Protocol (NTP), Simple Mail Transfer Protocol (SMTP), Remote Login, Transport Layer Security (TLS), Secure Sockets Layer (SSL), Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Datagram Congestion Control Protocol (DCCP), Stream Control Transfer Protocol (SCTP), Hypertext Markup Language (HTML), etc.

[0048] Figure 3 A schematic diagram of an exemplary control system 300 according to one embodiment of the present disclosure is provided (such as, but not limited to, those described above). Figure 2The exemplary robot controller 210 or exemplary PLC 214 described herein. Generally, the terms computing entity, entity, device, system, and / or similar terms used interchangeably herein can refer to, for example, one or more computers, computing entities, desktop computers, mobile phones, tablets, phablets, laptops, distributed systems, items / devices, terminals, servers or server networks, blade servers, gateways, switches, processing devices, processing entities, set-top boxes, relays, routers, network access points, base stations, etc., and / or any combination of devices or entities suitable for performing the functions, operations, and / or processes described herein. Such functions, operations, and / or processes may include, for example, transmitting, receiving, operating, processing, displaying, storing, determining, creating / generating, monitoring, evaluating, comparing, and / or similar terms used herein. In one embodiment, these functions, operations, and / or processes may be performed on data, content, information, and / or similar terms used herein.

[0049] As noted, in one embodiment, the control system 300 may also include one or more network and / or communication interfaces 308 for communicating with various computing entities, such as by transmitting data, content, information, and / or similar terms used herein that can be sent, received, manipulated, processed, displayed, stored, etc. For example, the control system 300 may communicate with other components of the depalletizing system.

[0050] like Figure 3As shown, in one embodiment, the control system 300 may include one or more processing elements (e.g., processing element 305) (also referred to as processor, processing circuitry, and / or similar terms used interchangeably herein) or communicate with such processing elements, which communicate with other elements within the control system 300 via, for example, a bus or network connection. It should be understood that processing element 305 can be embodied in a variety of different ways. For example, processing element 305 may be embodied as one or more complex programmable logic devices (CPLDs), microprocessors, multi-core processors, coprocessor entities, application-specific instruction set processors (ASIPs), and / or controllers. Furthermore, processing element 305 may be embodied as one or more other processing devices or circuitry. The term "circuit" can refer to a completely hardware implementation or a combination of hardware and computer program products. Thus, processing element 305 may be embodied as an integrated circuit, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), programmable logic array (PLA), hardware accelerator, other circuitry, etc. It should be understood that processing element 305 may be configured for a particular purpose or configured to execute instructions stored in volatile or non-volatile media or otherwise accessible by processing element 305. Therefore, whether configured by hardware, computer program products, or a combination thereof, when configured accordingly, the processing element 305 may be able to perform the steps or operations according to the embodiments of this disclosure.

[0051] In one embodiment, the control system 300 may further include or communicate with volatile media (also referred to as volatile storage device, memory, memory storage device, memory circuitry, and / or similar terms used interchangeably herein). In one embodiment, the volatile storage device or memory may also include one or more memory elements 306 as described above, such as RAM, DRAM, SRAM, FPM DRAM, EDO DRAM, SDRAM, DDR SDRAM, DDR2 SDRAM, DDR3 SDRAM, RDRAM, RIMM, DIMM, SIMM, VRAM, cache memory, register memory, etc. As will be appreciated, the volatile storage device or memory element 306 can be used to store, for example, memory... Figure 3The processing element 305 shown executes at least a portion of a database, database instance, database management system entity, data, application, program, program module, script, source code, object code, bytecode, compiled code, interpreted code, machine code, executable instructions, etc. Therefore, the database, database instance, database management system entity, data, application, program, program module, script, source code, object code, bytecode, compiled code, interpreted code, machine code, executable instructions, etc., can be used to control certain aspects of the operation of the control system 300 with the help of the processing element 305 and the operating system.

[0052] In one embodiment, the control system 300 may further include or communicate with a non-volatile medium (also referred to as a non-volatile storage device, memory, memory storage device, memory circuit, and / or similar terms used interchangeably herein). In one embodiment, the non-volatile storage device or memory may include one or more non-volatile storage devices or storage media 307 as described above, such as hard disks, ROMs, PROMs, EPROMs, EEPROMs, flash memory, MMC, SD memory cards, memory sticks, CBRAMs, PRAMs, FeRAMs, RRAMs, SONOS, racetrack memory, etc. As will be appreciated, the non-volatile storage device or storage medium 307 may store databases, database examples, database management system entities, data, applications, programs, program modules, scripts, source code, object code, bytecode, compiled code, interpreted code, machine code, executable instructions, etc. The terms database, database example, database management system entity, and / or similar terms are used interchangeably herein and may generally refer to a structured or unstructured collection of information / data stored in a computer-readable storage medium.

[0053] Storage medium 307 may also be embodied as one or more data storage devices, one or more separate database servers, or a combination of a data storage device and a separate database server. Furthermore, in some embodiments, storage medium 307 may be embodied as a distributed repository, such that some of the stored information / data is centrally stored in a location within the system, and other information / data is stored in one or more remote locations. Alternatively, in some embodiments, the distributed repository may be distributed across multiple remote storage locations. Examples of embodiments contemplated herein would include a cloud data storage system maintained by a third-party provider, where some or all of the information / data required to restore system operation may be stored. Additionally, the information / data required to restore system operation may also be partially stored in the cloud data storage system and partially stored in a locally maintained data storage system. More specifically, storage medium 307 may include one or more data storage areas configured to store information / data available in some embodiments.

[0054] As noted, in one embodiment, the control system 300 may also include one or more network and / or communication interfaces 308 for communicating with various computing entities, such as by transmitting data, content, information, and / or similar terms used interchangeably herein, that can be sent, received, manipulated, processed, displayed, stored, etc. Such communication may be performed using wired data transmission protocols, such as Fiber Distributed Data Interface (FDDI), Digital Subscriber Line (DSL), Ethernet, Asynchronous Transfer Mode (ATM), Frame Repeater, Cable Service Interface Data Specification (DOCSIS), or any other wired transmission protocol. Similarly, the control system 300 can be configured to communicate via a wireless external communication network using any of a variety of protocols, such as General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), Code Division Multiple Access 1900 (CDMA1900), CDMA1900 1X (1xRTT), Wideband Code Division Multiple Access (WCDMA), Global System for Mobile Communications (GSM), Enhanced Data Rate GSM Evolution (EDGE), Time Division Synchronous Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), Evolved Universal Terrestrial Radio Access Network (E-UTRAN), Evolved Data Optimized (EVDO), High Speed ​​Packet Access (HSPA), High Speed ​​Downlink Packet Access (HSDPA), IEEE 802.11 (Wi-Fi), Wi-Fi Direct, 802.16 (WiMAX), Ultra Wideband (UWB), Infrared (IR) protocol, Near Field Communication (NFC) protocol, Wibree, Bluetooth protocol, Wireless Universal Serial Bus (USB) protocol, and / or any other wireless protocol. The control system 300 can use such protocols and standards to communicate using the following: Border Gateway Protocol (BGP), Dynamic Host Configuration Protocol (DHCP), Domain Name System (DNS), File Transfer Protocol (FTP), Hypertext Transfer Protocol (HTTP), TLS / SSL / secure HTTP, Internet Message Access Protocol (IMAP), Network Time Protocol (NTP), Simple Mail Transfer Protocol (SMTP), Remote Login, Transport Layer Security (TLS), Secure Sockets Layer (SSL), Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Datagram Congestion Control Protocol (DCCP), Stream Control Transmission Protocol (SCTP), Hypertext Markup Language (HTML), etc.

[0055] Figure 4 , Figure 5 and Figure 6 This is an example diagram illustrating an example object on an example tray according to various embodiments of the present disclosure. Specifically, Figure 4 An example tray with the same SKU is shown. Figure 5 and Figure 6 An example tray with different SKUs is shown. Various embodiments of this disclosure can be implemented in, but are not limited to, […]. Figure 4 , Figure 5 and Figure 6 The example shown is implemented on an exemplary tray.

[0056] exist Figure 4 In the example diagram 400 shown, objects are stacked on example tray 402 into tray layers, such as tray layer 404A, tray layer 404B, tray layer 404C, and tray layer 404D. Each tray layer may include one or more objects. Figure 4 In the example shown, objects in each tray layer may have the same SKU (e.g., the same size and the same shape), or may have similar sizes and shapes, and objects in different tray layers may have the same SKU (e.g., the same size and the same shape) or may have similar sizes and shapes. Therefore, Figure 4 The example figure 400 shown illustrates a scenario with the same SKU.

[0057] exist Figure 5 In the example diagram 500 shown, objects are stacked on example pallets 501 into pallet layers, such as pallet layers 503A, 503B, 503C, and 503D. Each pallet layer may include one or more objects. Figure 5 In the example shown, objects in each tray layer may have the same SKU (e.g., the same size and the same shape), or may have similar sizes and shapes, but objects in different tray layers may have different SKUs (e.g., different sizes and different shapes). Therefore, Figure 5 The example figure 500 shown illustrates an example mixed SKU scenario.

[0058] exist Figure 6 In the example diagram 600 shown, objects are stacked on example pallets 602 into pallet layers, such as pallet layer 604A, pallet layer 604B, and pallet layer 604C. Each pallet layer may include one or more objects. Figure 6 In the example shown, the objects in each tray layer may have different SKUs (e.g., different sizes and / or different shapes), and the objects in different tray layers may have different SKUs (e.g., different sizes and / or different shapes). Therefore, Figure 6 The example figure 600 shown illustrates an exemplary mixed SKU scenario.

[0059] Figure 7A and Figure 7BThis is an exemplary perspective view of an exemplary dynamic gap system 700 (also known as a dynamic box gap system) that can be used according to various embodiments of the present disclosure.

[0060] exist Figure 7A In the example shown, the exemplary dynamic clearance system 700 is positioned near the tray 701. In some embodiments, the exemplary dynamic clearance system 700 includes a support 704. In some embodiments, the support 704 includes a vertical beam 705 and a horizontal beam 706.

[0061] In some embodiments, the vertical beam 705 includes a guide rail 707. In some embodiments, the guide rail 707 is a linear guide rail. In some embodiments, the guide rail 707 provides a vertical travel path for the bracket assembly 711. For example, the bracket assembly 711 can travel vertically along the guide rail 707. For example, the guide rail 707 may include a belt actuator with a belt 709 (e.g., (etc.). In some examples, the bracket assembly 711 may be connected to one end of the belt 709, and the other end of the belt 709 may be fixed to the guide rail 707 and / or the vertical beam 705. In some embodiments, the belt actuator may include a motor and / or other actuator components that enable the bracket assembly 711 to travel vertically along the guide rail 707. For example, a motor with a gearbox may drive the entire traveling mechanism.

[0062] In some embodiments, the LiDAR sensor 713 is fixed to the bracket assembly 711. For example, when the bracket assembly 711 travels vertically along the guide rail 707, the LiDAR sensor 713 also travels vertically along the guide rail 707. Therefore, in various embodiments of this disclosure, the guide rail 707 is vertically mounted to the bracket 704, wherein the LiDAR sensor 713 is mounted to the bracket assembly 711 provided by the guide rail 707. In some embodiments, the LiDAR sensor 713 is configured to emit a horizontally traveling laser (e.g., toward or above the direction in which the object / tray is placed). Details of this are described herein.

[0063] In some implementations, two proximity sensors are mounted at the far end of the guide rail 707 to prevent the bracket assembly 711 from overtraveling. For example, the top proximity sensor is fixed to the guide rail 707 and located at the top of the travel path of the bracket assembly 711, and the bottom proximity sensor is fixed to the guide rail 707 and located at the bottom of the travel path of the bracket assembly 711.

[0064] like Figure 7A and Figure 7BAs shown, the dynamic gap system 700 may also include a mounting bracket 715 for fixing image capturing devices (such as, but not limited to, 2D and / or 3D image capturing devices). Similar to those described above, the image capturing devices (such as, but not limited to, 2D and / or 3D image capturing devices) can capture 2D and / or 3D images of an object 703 located on a tray 701 from the top of the object 703.

[0065] In some embodiments, the height value of the tallest box on tray 701 can be provided and / or determined based on 2D and / or 3D images. In some embodiments, a controller (such as a PLC) can send control commands to the carrier assembly 711, wherein the LiDAR sensor 713 rises to a certain height at least in part based on the height value of the tallest box on tray 701. In some embodiments, the LiDAR sensor 713 communicates with the controller (such as a PLC) and can determine whether the laser light from the LiDAR sensor 713 is blocked (e.g., in a blocked state) or unblocked (e.g., in a clear state). In some embodiments, when the laser light from the LiDAR sensor 713 is blocked and subsequently unblocked (e.g., clear state), the controller (such as a PLC) sends a control signal to the robot arm to stop the lifting motion.

[0066] Figure 8 This is an exemplary flowchart illustrating an exemplary method 800 of an exemplary dynamic gap system operating according to various embodiments of the present disclosure.

[0067] like Figure 8 As shown, exemplary method 800 begins at step / operation 802. After and / or in response to step / operation 802, exemplary method 800 proceeds to step / operation 804. At step / operation 804, the vision system provides the height value of the tallest object on the tray.

[0068] In this disclosure, the term "highest object on a pallet" refers to the object on the pallet whose top surface is the highest point from the ground. As stated above, in conjunction with at least... Figures 4 to 6 As shown, objects on the tray can be stacked in layers.

[0069] In some implementations, the height value between the ground and the top surface of each object on the top layer of the pallet can be determined, at least in part, based on 2D and / or 3D images captured by a vision system. In some implementations, the maximum height value can be selected from these height values ​​as the height value of the tallest object on the pallet.

[0070] For example, a vision system and / or control system (e.g., one of the control systems described herein) can calculate the height value between the ground and the top surface of each object on the top layer by utilizing one or more dimensioning techniques, and can select the maximum height value from these height values ​​as the height value of the tallest object on the tray. As shown in the figure, the vision system and / or control system (e.g., one of the control systems described herein) can determine the height value associated with the tallest object on the tray.

[0071] For example, the top layer of a pallet may include objects A, B, and C. The height between the top surface of object A and the ground may be 1 meter. The height between the top surface of object B and the ground may be 0.5 meters. The height between the top surface of object C and the ground may be 0.8 meters. In this example, the height of the tallest object on the pallet is 1 meter (e.g., object A is the tallest object on the pallet).

[0072] Re-reference Figure 8 Following and / or in response to step / operation 802, exemplary method 800 proceeds to step / operation 806. At step / operation 806, the vision system provides the pickup point for the robotic arm.

[0073] In some implementations, the pickup point may take the form of, but is not limited to, the pose of the robotic arm used to pick up the object (including, but not limited to, its position on the x and y axes). In some implementations, the pickup point of the robotic arm may be determined at least in part based on 2D and / or 3D images captured by a vision system.

[0074] For example, based on 2D and / or 3D images, a vision system and / or control system (e.g., one of the control systems described herein) can determine the position of each object on the top layer and can calculate the pose of the robot arm (e.g., the position of the robot arm's end effector in the x and y axes) for picking up objects from the top layer.

[0075] Re-reference Figure 8 Following / in response to step / operation 804 and / or step / operation 806, exemplary method 800 proceeds to step / operation 808. At step / operation 808, the PLC raises the LiDAR sensor to a height 50 mm higher than the height value determined at step / operation 804.

[0076] In some implementations, the vision system can provide the height value at step / operation 804 to the PLC. In some implementations, the PLC can transmit control commands to the bracket assembly (where the LiDAR sensor is mounted). In some implementations, the control commands can include a height command indicating the appropriate height position of the bracket assembly / LiDAR sensor. In some implementations, upon receiving the control command, the bracket assembly can activate a motor (along with a belt) to raise the bracket assembly / LiDAR sensor to a certain height according to the height command. Therefore, the PLC causes the LiDAR sensor to travel to a certain height, at least partially based on that height value.

[0077] In the example, the height command instructs the bracket device / LiDAR sensor to be raised to a height equal to the height value determined at step / operation 804 plus a predetermined height buffer value (such as, but not limited to, 50 mm). For example, if the height value provided at step / operation 804 is 1 meter, the PLC can send a command to the bracket device to raise the LiDAR sensor from the ground to 1 meter plus 50 mm.

[0078] While the above description provides a 50 mm example as an exemplary predetermined height buffer value, it should be noted that the scope of this disclosure is not limited to the above description. In some examples, the exemplary method can implement other predetermined height buffer values ​​(which may be less than or greater than 50 mm).

[0079] Re-reference Figure 8 Following and / or in response to step / operation 808, exemplary method 800 proceeds to step / operation 810. At step / operation 810, the LiDAR sensor sends a detection signal to the PLC.

[0080] In some implementations, a LiDAR sensor may include a laser diode configured to emit a laser beam and a photodiode configured to detect the laser beam and generate a detection signal. For example, the laser diode may emit a horizontally traveling laser beam.

[0081] In some implementations, when the laser travels horizontally along its path, if anything (e.g., an object, a robotic arm) obstructs its path, the laser (e.g., by the object, by the robotic arm) is reflected back to the LiDAR sensor and detected by a photodiode. In this example, the photodiode detects the reflected laser and generates an obstruction signal (or a detection signal associated with an “obstruction state”) indicating the presence of something (e.g., an object, a robotic arm) at the height of the LiDAR sensor (e.g., the height determined at step / operation 808). For simplicity, this situation is also referred to as a LiDAR sensor obstruction or unobstructed condition.

[0082] In some implementations, when the laser travels horizontally along a path, if nothing (e.g., no object or robotic arm) obstructs its path, the laser is not reflected back to the LiDAR sensor. In this example, the photodiode does not detect the reflected laser and generates a clear signal (or a detection signal associated with a "clear state") indicating that nothing (e.g., no object or robotic arm) is present at the height of the LiDAR sensor (e.g., the height determined at step / operation 808). For simplicity, this situation is also referred to as the LiDAR sensor being clear or unobstructed.

[0083] Therefore, the above example illustrates that the exemplary LiDAR sensor can detect the presence of any object or any robotic arm at the height of the LiDAR sensor.

[0084] Re-reference Figure 8 Following and / or in response to step / operation 810, exemplary method 800 proceeds to step / operation 812. At step / operation 812, exemplary method 800 determines whether the LiDAR sensor is unobstructed (e.g., whether the detection signal generated by the LiDAR sensor is an unobstructed signal). For example, the PLC determines whether the detection signal from the LiDAR sensor indicates a first unobstructed state.

[0085] For example, a control system (such as, but not limited to, a PLC) can determine whether the detection signal generated by the LiDAR sensor indicates an unobstructed state or an obstructed state. If the detection signal indicates an obstructed state, it indicates that the laser emitted by the LiDAR sensor is blocked (e.g., there are other objects present, which may be at a height higher than the height value determined at step / operation 804).

[0086] For example, Figure 10A An exemplary case is shown where a laser generated by the LiDAR sensor 1002 is blocked by an object on a tray (e.g., when a detection signal indicates a blocking state). In contrast, Figure 10B An exemplary case is shown where the laser generated by the LiDAR sensor 1002 is not blocked by an object on the tray (e.g., when the detection signal indicates an unobstructed state).

[0087] Re-reference Figure 8 If, at step / operation 812, the exemplary method 800 determines that the detection signal generated by the LiDAR sensor indicates that the laser is blocked by an object (e.g., a non-open state), then the exemplary method 800 proceeds to step / operation 816. At step / operation 816, the exemplary method 800 determines whether to continue searching for the tallest object on the tray.

[0088] For example, if the laser is blocked, the control system (such as, but not limited to, a PLC) can determine that there are one or more objects on the tray that are taller than the tallest object determined at step / operation 804, and block the laser so that a detection signal indicates a blocking state. In some embodiments, the control system (such as, but not limited to, a PLC) determines that the height value calculated at step / operation 804 is incorrect, and the height value between the ground and the top surface of the object blocking the laser is greater than the height value determined at step / operation 804.

[0089] In some implementations, at step / operation 816, the control system (such as, but not limited to, a PLC) can determine whether to identify the height value of the tallest object on the tray.

[0090] If, at step / operation 816, exemplary method 800 determines that the tallest object on the tray is not being searched, then exemplary method 800 proceeds to step / operation 824. At step / operation 824, exemplary method 800 includes the PLC sending control commands to the robotic arm to lift the object normally (e.g., lift the object to a height corresponding to the default maximum height).

[0091] As described above, if the PLC determines that the LiDAR sensor is not functioning properly, then the PLC determines that the height value determined at step / operation 804 is incorrect. If the PLC determines that it is not searching for the tallest object, then the PLC reverts to the default lifting height setting used for lifting objects.

[0092] After step / operation 824, exemplary method 800 proceeds to step / operation 826 and ends.

[0093] Re-reference Figure 8 If method 800 determines at step / operation 816 to continue searching for the highest object, then exemplary method 800 proceeds to step / operation 818. At step / operation 818, exemplary method 800 includes pushing (e.g., lifting) the LiDAR sensor up until a detection signal from the LiDAR sensor indicates an open state.

[0094] As described above, if the PLC determines that the LiDAR sensor is not functioning properly, the PLC determines that the height value determined at step / operation 804 is incorrect (e.g., there are other objects taller than the object determined at step / operation 804). If the PLC determines that the tallest object is on the search tray, the PLC can utilize the LiDAR sensor.

[0095] For example, such as Figure 8As shown, the PLC can send control commands to the bracket assembly (in which the LiDAR sensor is installed) to cause the LiDAR sensor to be raised step by step. Each time the bracket assembly / LiDAR sensor is raised, it can only be raised by a predetermined increment (for example, the predetermined increment can be less than the predetermined height buffer value described above in conjunction with step / operation 808).

[0096] In some implementations, each time the bracket assembly / LiDAR sensor is raised, the PLC can determine whether the detection signal from the LiDAR sensor indicates a clear signal. If so, the PLC can send a control command to the bracket assembly to stop raising it. If not, the PLC can cause the bracket assembly / LiDAR sensor to continue raising by a predetermined increment.

[0097] In some implementations, when a detection signal from the LiDAR sensor indicates a clear signal, it indicates that no object is located at that height. In such examples, the LiDAR sensor stops at the height corresponding to the highest object on the tray.

[0098] refer to Figure 8 If, at step / operation 812, the detection signal from the LiDAR sensor indicates a clear state, or at step / operation 818, the LiDAR sensor has moved upward until the detection signal indicates a clear state, then the exemplary method 800 proceeds to step / operation 814. At step / operation 814, when the detection signal from the LiDAR sensor indicates an obstruction state and subsequently indicates a clear state, the LiDAR sensor sends a notification to the PLC.

[0099] As described above, if the detection signal from the LiDAR sensor indicates a clear state (as determined in step / operation 812 or step / operation 818), it indicates that there is no object at the height of the LiDAR sensor. For simplicity, this clear state is also referred to as the first clear state.

[0100] In some implementations, after a detection signal from a LiDAR sensor indicates a first unobstructed state (as determined at step / operation 812 or step / operation 818), the PLC sends control commands to the robotic arm to lift the object from the tray. In some implementations, the term "lift height increment" refers to the height difference between the object's original height before it is lifted and its current height when it is lifted. For example, if the original height between the ground and the top surface of the object is 1 meter when the object is placed on the tray, and the current height between the ground and the top surface of the object is 1.5 meters when the object is lifted, then the lift height increment is 0.5 meters (e.g., the robotic arm has lifted the object by 0.5 meters).

[0101] As the robotic arm lifts the object, the laser diodes from the LiDAR sensor continue to emit lasers that travel horizontally at the height of the LiDAR sensor, and the photodiodes continue to generate detection signals indicating whether the object is open or blocked.

[0102] In some implementations, when a detection signal indicates that the laser is blocked (e.g., in a blocked state), it indicates that the robotic arm has lifted the object by an increment of less than the object's height (e.g., the laser is blocked by the object being lifted). For example, if the box is 10 cm tall and the detection signal indicates that the laser is blocked, the robotic arm has lifted the object by less than 10 cm.

[0103] In some implementations, when the detection signal indicates that the laser is unobstructed (e.g., in an unobstructed state), it indicates that the robotic arm has lifted the object by an increment equal to or greater than the object's own height (so that the object no longer blocks the laser). For example, if the box's height is 10 centimeters and the detection signal indicates that the laser is unobstructed, then the robotic arm has lifted the object by more than 10 centimeters. For simplicity, this unobstructed state is also referred to as the second unobstructed state.

[0104] As illustrated in the examples described above, the LiDAR sensor can be raised to a certain height until it generates a detection signal indicating a first unobstructed state. Subsequently, the robotic arm begins picking up an object, causing the detection signal to indicate an obstructed state. As the robotic arm continues to pick up objects, the detection signal continues to indicate an obstructed state until the robotic arm has raised the object beyond its height, causing the detection signal to change to a second unobstructed state.

[0105] In some implementations, once the robotic arm has lifted an object beyond its height, it can move the object horizontally without colliding with other objects. Therefore, the LiDAR sensor can notify the PLC when the detection signal changes to indicate an obstructed state and subsequently indicates an unobstructed state.

[0106] Now for reference Figure 8 Following step / operation 814, exemplary method 800 proceeds to step / operation 820. At step / operation 820, the PLC stops the mechanical lifting motion of the robot arm in response to a notification received from a LiDAR sensor indicating an obstruction state and subsequently an unobstructed state.

[0107] In some implementations, in response to a detection signal indicating a first unobstructed state, the PLC causes the robotic arm to lift the object until the detection signal indicates an obstructed state and subsequently a second unobstructed state. As described above, if the detection signal from the LiDAR sensor is obstructed and subsequently unobstructed, it indicates that the object has been lifted to a height equal to or greater than the object's actual height. In other words, the object has been lifted to a sufficient height so that it can be transported horizontally without causing a collision with other objects.

[0108] For example, if the PLC determines that the detection signal has changed from a first open state to a blocked state and then back to an open state, the PLC sends a command to the robot arm to stop lifting the object to an unnecessary height and begin horizontal transport of the object.

[0109] Re-reference Figure 8 After step / operation 820, exemplary method 800 proceeds to step / operation 822. At step / operation 822, exemplary method 800 can cause the PLC to plan laser movement.

[0110] For example, as described above, the height sensor is positioned on the conveyor. In some embodiments, the robot arm moves the object horizontally above the height sensor as it passes by. In some embodiments, the height sensor may be a laser-based height sensor (e.g., LiDAR). In some embodiments, the PLC may cause the height sensor to emit a laser to determine the height of the object.

[0111] For example, a height sensor can determine the height from the ground to the bottom surface of an object, and a robotic arm can determine the height from the ground to the top surface of the object (e.g., the robotic arm can track the height the object is lifted). Based on the heights from the ground to the bottom surface and from the ground to the top surface, the object's actual height can be calculated.

[0112] Re-reference Figure 8 After step / operation 822, exemplary method 800 proceeds to step / operation 826 and ends.

[0113] In some implementations, exemplary method 800 can be repeated each time a new object is lifted. For example, after step / operation 822 and / or step / operation 824, exemplary method 800 restarts at step / operation 802 to lift the next object.

[0114] Figure 9A and Figure 9B This is an exemplary flowchart illustrating an exemplary method 900 of an exemplary dynamic gap system operating according to various embodiments of the present disclosure.

[0115] Now for reference Figure 9A The exemplary method 900 begins at step / operation 901. After step / operation 901, the exemplary method 900 proceeds to step / operation 903. At step / operation 903, the vision system provides the height value of the tallest object on the tray, similar to the combination above. Figure 8 The described steps / operations are 804.

[0116] Now for reference Figure 9A Following step / operation 901, exemplary method 900 proceeds to step / operation 905. At step / operation 905, the vision system provides a pickup point, similar to the combination described above. Figure 8 The described steps / operations are 806.

[0117] Now for reference Figure 9A Following step / operation 903 and / or step / operation 905, exemplary method 900 proceeds to step / operation 907. At step / operation 907, the PLC raises the LiDAR sensor to a height 50 mm higher than the height value determined at step / operation 903, similar to the above-described combination. Figure 8 The described steps / operations are 808.

[0118] Now for reference Figure 9A Following step / operation 907, exemplary method 900 proceeds to step / operation 909. At step / operation 909, the LiDAR sensor sends a detection signal to the PLC, similar to the above-described combination. Figure 8 The described steps / operations are 810.

[0119] Now for reference Figure 9A Following step / operation 909, exemplary method 900 proceeds to step / operation 911. At step / operation 911, exemplary method 900 determines whether the LiDAR sensor is unobstructed (e.g., whether the detection signal generated by the LiDAR sensor is an unobstructed signal), similar to the above-described combination. Figure 8 The described steps / operations are 812.

[0120] If, at step / operation 911, the LiDAR sensor is open (e.g., the detection signal generated by the LiDAR sensor is an open signal), then exemplary method 900 proceeds to block A, which will Figure 9A Connected to Figure 9B .

[0121] If, at step / operation 911, the LiDAR sensor is obstructed (e.g., the detection signal generated by the LiDAR sensor is a blocking signal), then exemplary method 900 proceeds to step / operation 913. At step / operation 913, exemplary method 900 determines whether to continue searching for the tallest object on the tray, similar to the above combination. Figure 8 The described steps / operations are 816.

[0122] If, at step / operation 913, exemplary method 900 determines that the tallest object on the tray is not being searched, then exemplary method 900 proceeds to step / operation 917. At step / operation 917, exemplary method 900 includes the PLC sending control commands to the robotic arm to properly lift the object (e.g., lift the object to a height corresponding to the default maximum height), similar to the above-described combination. Figure 8 Step / operation 824 is described. Subsequently, exemplary method 900 proceeds to step / operation 919 and ends.

[0123] Re-reference Figure 9A If method 900 determines at step / operation 913 to continue searching for the highest object, then exemplary method 900 proceeds to step / operation 915. At step / operation 915, exemplary method 900 includes progressively pushing (e.g., lifting) the LiDAR sensor until a detection signal from the LiDAR sensor indicates a clear state, similar to the above-described combination. Figure 8 The described steps / operations are 818.

[0124] Re-reference Figure 9A After step / operation 915, exemplary method 900 proceeds to box A, which will... Figure 9A Connected to Figure 9B .

[0125] Now for reference Figure 9B If, at step / operation 911, the detection signal from the LiDAR sensor indicates a clear state, or at step / operation 915, the LiDAR sensor has moved upward until the detection signal indicates a clear state, then the exemplary method 900 proceeds to step / operation 921. At step / operation 921, when the detection signal from the LiDAR sensor indicates an obstruction state and subsequently indicates a clear state, the LiDAR sensor sends a notification to the PLC, similar to the above-described combination. Figure 8 The described steps / operations are 814.

[0126] Re-reference Figure 9BFollowing step / operation 921, exemplary method 900 proceeds to step / operation 923. At step / operation 923, the PLC, in response to receiving a notification from the LiDAR sensor indicating an obstruction state and subsequently an unobstructed state, stops the mechanical lifting motion of the robot arm, similar to the above-described combination. Figure 8 The described steps / operations are 820.

[0127] Re-reference Figure 9B Following step / operation 921, exemplary method 900 proceeds to step / operation 925. In step / operation 925, the PLC communicates with the robot controller.

[0128] For example, the PLC can transmit and / or transfer data and / or information to the robot controller, indicating that the detection signal from the laser source indicates an obstruction state and subsequently an unobstructed state. As described above, if the detection signal from the LiDAR sensor indicates obstruction and subsequently unobstructed, it indicates that the object has been lifted to a height equal to or greater than the object's height. In other words, the robot arm has lifted the object to a sufficient height so that it can be transported horizontally without causing collisions with other objects.

[0129] Re-reference Figure 9B Following step / operation 925, exemplary method 900 proceeds to step / operation 927. As in step / operation 927, the robot controller determines the current robot position associated with the robot arm.

[0130] For example, a robot controller can determine the x, y, and z positions of a robot arm. In this example, the z position of the robot arm refers to the height of the robot arm in the vertical dimension at which it has sufficiently lifted the object to allow it to be transported horizontally without colliding with other objects.

[0131] Re-reference Figure 9B Following step / operation 927, exemplary method 900 proceeds to step / operation 929. As in step / operation 929, the robot controller calculates the actual height the object has been lifted by the robot arm, changes the starting position of the pre-placement motion, and / or adjusts the collision map of the robot arm to lift the object from the tray.

[0132] In some implementations, the robot controller may calculate the actual lift height based at least in part on the current robot position determined at step / operation 927, which is the actual height the robot arm has lifted the object. For example, the actual lift height may be based on the robot arm's z-position, at which the robot arm has sufficiently lifted the object in the vertical dimension so that the object can be transported horizontally without causing collisions with other objects.

[0133] In some implementations, the robot controller can adjust the pre-placement motion associated with the robot arm. In some implementations, the pre-placement motion defines the lifting information (such as the lifting height) used to lift objects on a tray. In some implementations, the robot controller can set a maximum default lifting height for other objects based at least in part on the actual lifting height. In other words, by default (e.g., in conjunction with step / operation 917), the robot controller can lift objects at least in part based on the pre-placement motion to reduce the likelihood of lifting objects to unnecessarily high heights.

[0134] In some implementations, the robot controller can update the collision map associated with the object (e.g., update the height of the object to avoid collisions). For example, the collision map can be adjusted based on the z-position of the robot arm, at which the robot arm has sufficiently lifted the object in the vertical dimension so that the object can be transported horizontally without causing collisions with other objects.

[0135] In some implementations, after step / operation 923 and / or step / operation 929, exemplary method 900 proceeds to step / operation 931 and ends.

[0136] In some implementations, exemplary method 900 can be repeated each time a new object is lifted. For example, after step / operation 923, step / operation 929 and / or step / operation 917, exemplary method 900 restarts at step / operation 901 to lift the next object.

[0137] Therefore, various embodiments of this disclosure can use information obtained from a vision system to guide a pallet device (which is equipped with LiDAR sensors) to a specific height position on a guide rail before the robotic arm unpacks the object from the pallet.

[0138] For example, an exemplary method may include combining a LiDAR sensor on a mechanical guide rail with a vision system, thereby significantly increasing the number of picks per hour due to the reduced travel distance of the robot. The guide rail itself is a belt actuator, vertically mounted to a support, with the LiDAR sensor mounted on a bracket assembly provided by the guide rail. In some embodiments, two proximity sensors are mounted at the far end of the guide rail to prevent the bracket assembly from overtraveling. In some embodiments, a motor with a gearbox drives the entire mechanism. In some embodiments, the vision system provides the height value of the highest box on the tray. In some embodiments, a PLC guides the bracket assembly with the LiDAR sensor to that height position. In some embodiments, the LiDAR sensor communicates with the PLC and signals whether it is unobstructed. When the LiDAR sensor is obstructed and subsequently unobstructed, a signal is sent to the robot to stop the lifting motion and move to the placement position.

[0139] Many modifications and other embodiments of the present disclosure will occur to those skilled in the art upon benefiting from the teachings presented in the foregoing description and the accompanying drawings. Therefore, it should be understood that the present disclosure is not limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terminology is used herein, it is used only in a general and descriptive sense and not for purposes of limitation.

Claims

1. An apparatus, the apparatus comprising: A support, positioned near a tray of multiple objects, wherein the support includes a vertical beam connected to a horizontal beam; A light detection and ranging LiDAR sensor is fixed to a bracket assembly, wherein the bracket assembly travels along a guide rail mounted on the vertical beam; One or more image capturing devices configured to capture one or more images, wherein the one or more image capturing devices are mounted on a horizontal beam; The device comprises at least one processor and at least one non-transitory memory including computer program code, the at least one non-transitory memory and the computer program code being configured to utilize the at least one processor to cause the device to: Determine the height value associated with the tallest of the plurality of objects on the tray; The light-detecting and ranging LiDAR sensor is allowed to travel to a certain height, at least in part, based on the height value determined based on one or more images captured by the one or more image-capturing devices; and Determine whether the detection signal from the LiDAR sensor indicates a first unobstructed state, wherein the first unobstructed state indicates that the detection signal is not blocked by the plurality of objects on the tray; and In response to determining that the detection signal indicates a first unobstructed state, the robotic arm lifts one of the plurality of objects until the detection signal indicates an obstructed state, and subsequently indicates a second unobstructed state, wherein the second unobstructed state indicates that the object has been lifted to a height equal to or greater than the object's height.

2. The device of claim 1, wherein the at least one non-transitory memory and the computer program code are configured to utilize the at least one processor to cause the device to: The LiDAR sensor is moved to a height equal to the height value plus a predetermined height buffer value.

3. The device according to claim 2, wherein the predetermined height buffer value is 50 mm.

4. The device of claim 1, wherein the at least one non-transitory memory and the computer program code are configured to utilize the at least one processor to cause the device to: Communicate with the robot controller.

5. The device of claim 1, wherein the at least one non-transitory memory and the computer program code are configured to utilize the at least one processor to cause the device to: In response to determining that the detection signal does not indicate the first unobstructed state, determine whether to search for the tallest object on the tray.

6. The device of claim 5, wherein the at least one non-transitory memory and the computer program code are configured to utilize the at least one processor to cause the device to: In response to determining the highest object on the search tray, the LiDAR sensor is moved upwards incrementally.

7. A computer-implemented method, the computer-implemented method comprising: Based on one or more images captured by one or more image capture devices, determine the height value associated with the tallest object among a plurality of objects on a tray; The light detection and ranging LiDAR sensor is made to travel to a certain height at least in part based on the height value, which is determined based on one or more images captured by the one or more image capture devices; Determine whether the detection signal from the LiDAR sensor indicates a first unobstructed state, wherein the first unobstructed state indicates that the detection signal is not blocked by the plurality of objects on the tray; as well as In response to determining that the detection signal indicates a first unobstructed state, the robotic arm lifts one of the plurality of objects until the detection signal indicates an obstructed state, and subsequently indicates a second unobstructed state, wherein the second unobstructed state indicates that the object has been lifted to a height equal to or greater than the object's height.

8. The computer-implemented method according to claim 7, wherein the computer-implemented method comprises: The LiDAR sensor is moved to a height equal to the height value plus a predetermined height buffer value.

9. The computer-implemented method of claim 8, wherein the predetermined height buffer value is 50 mm.

10. The computer-implemented method of claim 7, comprising communicating with a robot controller.

11. The computer-implemented method according to claim 7, comprising: In response to determining that the detection signal does not indicate a first unobstructed state, it is determined whether to search for the tallest object on the tray.

12. The computer-implemented method according to claim 11, comprising: In response to determining the highest object on the search tray, the LiDAR sensor is moved upwards incrementally.

13. A computer program product comprising at least one non-transitory computer-readable storage medium storing a computer-readable program code portion, the computer-readable program code portion including an executable portion configured to: Based on one or more images captured by one or more image capture devices, determine the height value associated with the tallest object among multiple objects on a tray; The light detection and ranging LiDAR sensor is made to travel to a certain height at least in part based on the height value, which is determined based on one or more images captured by the one or more image capture devices; Determine whether the detection signal from the LiDAR sensor indicates a first unobstructed state, wherein the first unobstructed state indicates that the detection signal is not blocked by the plurality of objects on the tray; as well as In response to determining that the detection signal indicates a first unobstructed state, the robotic arm lifts one of the plurality of objects until the detection signal indicates an obstructed state, and subsequently indicates a second unobstructed state, wherein the second unobstructed state indicates that the object has been lifted to a height equal to or greater than the object's height.

14. The computer program product of claim 13, wherein the computer-readable program code portion includes an executable portion, the executable portion being configured to: The LiDAR sensor is moved to a height equal to the height value plus a predetermined height buffer value.

15. The computer program product of claim 14, wherein the predetermined height buffer value is 50 mm.

16. The computer program product of claim 13, wherein the computer-readable program code portion includes an executable portion, the executable portion being configured to: Communicate with the robot controller.

17. The computer program product of claim 13, wherein the computer-readable program code portion includes an executable portion, the executable portion being configured to: In response to determining that the detection signal does not indicate the first unobstructed state, it is determined whether to search for the tallest object on the tray.

Images