System for profile analysis of a tray

Abstract

The present application relates to a system for profiling a tray, the system comprising: a turntable configured to rotate the tray; a light source positioned adjacent to the turntable, the light source configured to illuminate the tray when the tray is rotated on the turntable; a camera positioned relative to the light source, the camera configured to capture an image of the tray when the tray is rotated on the turntable; and a photographing compartment enclosing an area containing the turntable, the light source, and the camera, where the photographing compartment includes sides and a top to reduce an amount of ambient light entering the enclosed area.

Description

[0001] Divisional Application Information

[0002] This application is a divisional application of the invention patent application with application date of April 19, 2022, application number 202280036890.4, and invention name “Automated Pallet Profile Analysis”. Technical Field

[0003] This document describes the automatic profiling of items, such as pallets, as they are received at a warehouse environment or other storage facility. Background Art

[0004] Warehouses or similar storage facilities may receive thousands of items for storage each day. Such items may include pallets, crates, bins, or other items that may be stored in a warehouse. Items entering a warehouse may have unique information that may affect where the item is stored, how long the item is stored, etc. Items may also have unique identification information that may be used throughout the supply chain.

[0005] Items may be stored in a warehouse for a short or temporary period of time. Items may also be stored in a warehouse for a longer period of time. Depending on the characteristics of the items, the items may be stored in different warehouse locations. For example, a storage location may be selected for an item based on its ability to accommodate the height, weight, and / or size of the item. As another example, a storage location may be designated for all items of a particular customer. As yet another example, a storage location may be selected for items that have temperature storage requirements (e.g., the item needs to be in a refrigerated warehouse).

[0006] Storage and other item characteristics can be identified for each item in a variety of ways. For example, storage characteristics can be received from previous supply chain users (such as suppliers who deliver storage items to warehouses for storage). Sometimes, this information may contain human errors. Sometimes, once the items arrive at the warehouse, this information may be inaccurate because unexpected events may occur during the transportation of the items. For example, a pallet may be damaged, tilted, and / or lose containers or other contents when it arrives at the warehouse. Therefore, information about the pallet determined by the supplier before shipping to the warehouse may be inaccurate. Inaccurate information can be used to find less efficient or suboptimal storage locations. In addition, relying on inaccurate information can lead to unnoticed damage, which can have a negative impact on the supply chain and customer satisfaction. Once the items are in the warehouse and are ready to be moved to the storage location, changes to the items during shipping or delivery (e.g., dents, broken pallets, lost boxes, etc.) may not be taken into account. Once the items enter the warehouse, they may or may not be inspected again.

[0007] As another example, storage or other article characteristics may be determined by a warehouse worker. A warehouse worker may inspect the article as it enters the warehouse and manually input information about the article into a computing system such as a warehouse management system. A warehouse worker may also use a handheld scanner, camera, or similar device to inspect the article and collect information about the storage characteristics of the article. Manual inspection may result in delays in warehouse efficiency and delays in moving incoming articles for storage. A warehouse worker may make mistakes when inspecting and / or recording information about the article. Some characteristics may not be easily noticed by the human eye. A warehouse worker may also work quickly so that the article queue can be inspected, so the warehouse worker may not see or record the article and storage characteristics. In addition, if all the articles in the inspection queue are hurriedly inspected within a given time period, the warehouse worker may input an erroneous value into the computing system.

[0008] Item characteristics may also be identified by cameras or other scanning devices positioned throughout the warehouse. For example, as items move through the warehouse, cameras may capture images of the items and send those images to a computing system for processing. The cameras may not be able to capture clear images of the items and / or may not be able to capture all sides / angles of the items. The computing system may not be able to fully identify the storage and other item characteristics of the items, which may lead to warehouse inefficiencies. Summary of the invention

[0009] This document generally describes systems, methods, apparatus, devices, and techniques for profiling items, such as pallets in a warehouse. The disclosed techniques may include a combination of warehouse components and scanning devices configured to capture image data of pallets from multiple angles. For example, the disclosed techniques may include an on conveyor belt and an off conveyor belt, a rotating turntable, and a vertical pole with one or more fixed cameras. The vertical pole may be positioned at a threshold distance away from the rotating turntable to allow each camera to have sufficient field of view to capture an image of the pallet (i.e., capture an image of the entire width of the pallet). The vertical pole may remain stationary so that one or more cameras configured thereon capture an image of the pallet from the same distance, angle, and field of view (FOV). Photo booths or similar light neutralizing structures may be positioned around the rotating turntable to provide consistent and uniform lighting to capture images of the pallet, regardless of lighting conditions in the surrounding environment (i.e., a docking area in a warehouse). One or more lights (e.g., red lights, strobe lights) may be positioned inside the photo booth and / or on the vertical pole to illuminate the rotating turntable and the trays above, and specific lighting configurations (e.g., wavelength of light emitted from the light source, polarizing filter, strobe duration, light intensity) and / or camera configurations (e.g., shutter speed, polarizing lens filter) may be used to achieve consistent and reliable images despite highly variable tray conditions (e.g., frost, transparent and translucent plastic packaging layers). For example, the lighting may be offset and angled (e.g., approximately 45 degree offset) from the camera's vantage point to reduce glare in the captured image. The photo booth and lighting may provide brighter, clearer images of the trays to be captured by the camera.

[0010] The computing system may control the operation of the components (e.g., cameras, turntables, upper and lower conveyor belts, lights, etc.) used in the disclosed technology for profiling a pallet. For example, a pallet may be loaded onto the upper conveyor belt (e.g., when first entering a warehouse). The pallet may be automatically aligned as it moves down the upper conveyor belt, and may then be automatically moved onto the turntable. The computing system may control the turntable to center the pallet on it, which may permit images of the pallet to be captured from the same / similar distance around the pallet as the pallet rotates on the turntable. The turntable may rotate at a predetermined speed, and may include pauses at various rotation angles (e.g., pauses every 15, 30, 45, 60, or 90 degrees of rotation) to capture images of the pallet at those orientations relative to the camera. Alternatively and / or in addition, the pallet may rotate continuously and images may be captured at various orientations without pausing the rotation of the turntable. The computing system may receive image data from the camera, and use image analysis techniques and / or machine learning models to identify information about the pallet. For example, the computing system may identify one or more tags or other unique identifiers associated with a pallet and / or its components (i.e., boxes on the pallet). In another example, the computing system may determine the dimensions of the pallet (e.g., height, width, depth), the types of items contained in the pallet, the quantity of those items, information about those specific items (e.g., date, origin, expiration date), and / or other details.

[0011] The computing system may use the determined pallet information to guide subsequent processing of the pallet, such as inbounding the pallet into a warehouse to identify a specific storage location for the pallet in the warehouse. For example, the specific storage location identified and used to store the pallet may depend on the type of items contained in the pallet, the specific environmental requirements of those items (e.g., temperature, humidity), and for perishable goods, information indicating the age / expiration date of the goods. In instances where the computer system may not be able to discern some or all of those details with at least a threshold confidence level, the computer system may direct the pallet to a temporary storage location within the warehouse to await manual profiling of the pallet, which may be performed by a remote user via images and other captured information of the pallet. For example, for pallets that cannot be automatically identified, such as in instances where there are significant visual obstacles (i.e., frost, layers of plastic packaging) in identifying the contents of the pallet, the pallet may be directed to a temporary storage area and may be added to a queue for manual review. A manual reviewer may then use a computing device to review the image and other captured details of the pallet and may identify the pallet and its contents, which a computing system may use to redirect the pallet to a specific organized storage location, such as a long-term storage area.

[0012] In some embodiments, instead of routing the pallet to a temporary storage location, additional images of the pallet may be captured and analyzed (e.g., rotating the turntable again and taking images at different orientations). A camera may capture images of the pallet, which may be processed by a computing system to identify the pallet label. If the label is not recognizable, the pallet may be routed to a lower conveyor belt for temporary storage. If the label is recognizable, the pallet may be routed to a lower conveyor belt for organized storage.

[0013] Image data associated with the pallet may be stored for subsequent processing. Image data may also be processed in real time. This additional processing may be performed to determine pallet information. Some of the information may include, but is not limited to, damage, tilt, box count, number of layers, contents, and pallet size (e.g., height, weight, overall dimensions). The pallet information identified by the computing system may be compared to pallet information manually entered into a warehouse management system (WMS) and / or warehouse control system (WCS) by a warehouse worker or otherwise stored by the WMS / WCS. Comparing the pallet information may be useful in determining the accuracy of the information identified by the computing system.

[0014] One embodiment may include a system for profiling a pallet in a warehouse. The system may include a turntable that can rotate the pallet, and a first conveyor belt and a second conveyor belt for moving the pallet in the warehouse. The first conveyor belt can automatically route the pallet from an initial location in the warehouse to the turntable, and the second conveyor belt can automatically route the pallet from the turntable to a destination location in the warehouse. The system may also include a vertical profiling structure located in a fixed position proximate a side of the turntable, the vertical profiling structure having a plurality of cameras mounted at a plurality of different locations along the vertical profiling structure. As the pallet rotates on the turntable, the plurality of cameras can capture images of the pallet, the images providing a plurality of different views of the pallet from a plurality of different vantage points provided by the cameras mounted to the vertical profiling structure at the plurality of different locations. The system may also include a computing system that can profile the pallet and control the operation of the turntable, the first and second conveyor belts, and the plurality of cameras. The computing system may: instruct the first conveyor belt to automatically route the pallet to the turntable; instruct the plurality of cameras to capture image data of the pallet on the turntable as the turntable rotates until a rotation cycle is completed; receive the captured image data of the pallet from the plurality of cameras; and retrieve an image-based model of the pallet from a data storage device. The image-based model may be trained using images of pallets with unique identifiers. The computing system may also determine whether the unique identifier on the pallet is recognizable based on applying the image-based model of the pallet to the captured image data of the pallet, and transmit a notification to a warehouse management system indicating whether the unique identifier on the pallet is recognizable or not. The notification may cause the warehouse management system to route the pallet to a different storage location based on the identification of the unique identifier.

[0015] This implementation may optionally include one or more of the following features. For example, the plurality of cameras may be arranged along the same vertical plane, with each camera having a substantially horizontal angle. As another example, the vertical profiling structure may be a pole, the vertical profiling structure may be laterally spaced from the first conveyor belt, the second conveyor belt, and the turntable, such that the vertical profiling structure is not suspended above any of the first conveyor belt, the second conveyor belt, and the turntable, and the vertical profiling structure may be positioned at a working distance of between 1.7 meters and 1.9 meters from the turntable. In some implementations, the vertical profiling structure may include at least one light source that is vertically aligned with and offset from each of the plurality of cameras. The at least one light source may be a red light. One or more of the plurality of cameras may include at least one of a polarizer, a red light filter, and a bandpass filter. In some implementations, the vertical profiling structure may include a light source mounted at the top of the vertical profiling structure and angled downward to illuminate the top surface of the tray as the tray rotates on the turntable.

[0016] As another example, the system may also include a photo booth that may enclose an area including the turntable and the vertical profiling structure. The photo booth may include (i) sides and top that may prevent ambient light from entering the enclosed area, and (ii) a light source for projecting light onto the tray as the tray rotates on the turntable. The light source may be laterally offset from the vertical profiling structure and angled relative to the turntable so that the light source reflects light from the top surface of the tray at an acute angle to the multiple cameras. The light source may emit light of a specific intensity and duration associated with a predetermined shutter speed of the multiple cameras. In some implementations, the photo booth may include a bar positioned along the top surface of the photo booth, the bar being aligned in the direction of the first conveyor belt. The bar may include a light source and a camera. The light source may project light onto the top surface of the tray as the tray rotates on the turntable. The camera may capture an image of the top surface of the tray. In some implementations, the photo booth may be made of a dark fabric that may prevent ambient light from entering the enclosed area.

[0017] In some embodiments, the photo booth may further include (i) a first opening in a first side of the photo booth through which the tray may be received from the first conveyor belt onto the turntable, and (ii) a second opening in a second side of the photo booth opposite the first side through which the tray may be moved from the turntable to the second conveyor belt. In some embodiments, the system may further include at least one light source attached to a vertical lighting structure, wherein the vertical lighting structure may be positioned within the enclosed area. The vertical lighting structure may be offset from the vertical profiling structure.

[0018] As another example, the system may include an overhead camera configured to capture images of a top surface of the tray as the tray rotates on the turntable. In some embodiments, the system may include one or more cameras mounted near a top portion of the vertical profiling structure and angled downward to capture images of the top surface of the tray. In some embodiments, one or more of the plurality of cameras may be mounted along a middle portion of the vertical profiling structure and parallel to a surface of the turntable to capture images of a side surface of the tray as the tray rotates on the turntable. One or more of the plurality of cameras may be mounted along a bottom portion of the vertical profiling structure and angled upward to capture images of a bottom surface of the tray as the tray rotates on the turntable.

[0019] As another example, the computing system may also stitch together one or more image data of the side surface, the top surface, and the bottom surface of the pallet and determine the height of the pallet, damage to the pallet, the tilt of the pallet, the contents of the pallet, the number of rows on the pallet, and the number of items on the pallet based on applying the image-based model to the stitched together image data. As yet another example, the computing system may also center the pallet at the midpoint of the turntable and instruct the turntable to rotate in a first direction at a constant predetermined speed. In some embodiments, the predetermined speed may be based on the size of the pallet, the number of the cameras on the vertical profiling structure, and one or more angles of the cameras.

[0020] As another example, the computing system may generate a placeholder label for the pallet based on determining that the unique identifier on the pallet is not recognizable, generate a pallet profile for the pallet based on using the placeholder label as an identifier for the pallet, and transmit a notification with the placeholder label to an automated warehouse device. The notification may cause the automated warehouse device to print the placeholder label and apply the printed placeholder label to a designated surface of the pallet.

[0021] As another example, while determining whether the unique identifier on the pallet is identifiable, the computing system may: instruct the first conveyor belt to automatically send the second pallet to the turntable along a specific route; instruct the multiple cameras to capture image data of the second pallet on the turntable as the turntable rotates; receive the captured image data of the second pallet from the multiple cameras; and determine whether the unique identifier on the second pallet is identifiable based on applying an image-based model of the second pallet to the captured image data of the second pallet.

[0022] As another example, based on determining that the unique identifier on the pallet is not recognizable, the computing system may also mark the pallet as unrecognizable. Based on determining that the unique identifier on the pallet is recognizable, the computing system may also mark the pallet as recognizable. As another example, the destination location may be at least one of an organized storage location and a temporary storage location in the warehouse. The computing system may determine the organized storage location of the pallet based at least in part on owner, item, and code data associated with the pallet. In some embodiments, the unique identifier may include a barcode, a label, a product identifier, a QR code, and a SKU. In some embodiments, the plurality of cameras may include three cameras. In some embodiments, each time the pallet rotates 30 degrees on the turntable, the plurality of cameras may be instructed to capture image data of the pallet once. In some embodiments, each time the pallet rotates 90 degrees on the turntable, the plurality of cameras may be instructed to capture image data of the pallet once. In some embodiments, once the pallet rotates 30 degrees for the first time, the plurality of cameras may be instructed to capture image data of the pallet, and then subsequently each time the pallet rotates 90 degrees on the turntable, capture once. In some embodiments, the computing system may also identify information about the pallet based on applying the image-based model to the captured image data of the pallet, access pre-identified information about the pallet from a data storage device, compare the identified information with the pre-identified information about the pallet, determine a match between the identified information and the pre-identified information about the pallet, and update a pallet profile associated with the pallet with the identified information based on determining a match between the identified information and the pre-identified information.

[0023] Certain embodiments described herein may also include a computer-implemented method for profiling a pallet. The method may include: instructing, by a computing system, a first conveyor belt to automatically route a pallet onto a turntable; and instructing, by the computing system, a plurality of cameras to capture image data of the pallet on the turntable as the turntable rotates until a rotation cycle is completed. The plurality of cameras may be mounted at a plurality of different locations along a vertical profiling structure, the vertical profiling structure being located in a fixed position proximate a side of the turntable. The method may also include receiving, by the computing system, the captured image data of the pallet from the plurality of cameras, the image data providing a plurality of different views of the pallet from a plurality of different vantage points, the plurality of different vantage points provided by the cameras mounted to the vertical profiling structure at the plurality of different locations. The method may include: retrieving, by the computing system, an image-based model of the pallet from a data storage device, the image-based model being trained using images of pallets having unique identifiers; determining, by the computing system, whether the unique identifier on the pallet is recognizable based on applying the image-based model of the pallet to the captured image data of the pallet; and transmitting, by the computing system, a notification to a warehouse management system indicating whether the unique identifier on the pallet is recognizable or not recognizable. The notification may cause the warehouse management system to route the pallet to a different storage location based on identification of the unique identifier.

[0024] In some implementations, the method may include one or more of the following features. For example, the method may further include stitching together, by the computing system, one or more image data of a side surface, a top surface, and a bottom surface of the pallet, and determining, by the computing system, the height of the pallet, damage to the pallet, the tilt of the pallet, the contents of the pallet, the number of rows on the pallet, and the number of items on the pallet based on applying the image-based model to the stitched-together image data. The method may also include centering, by the computing system, the pallet at a midpoint of the turntable, and instructing, by the computing system, the turntable to rotate in a first direction at a constant predetermined speed.

[0025] In some embodiments, the vertical profiling structure may be a pole, the vertical profiling structure may be laterally spaced apart from the first conveyor belt and the turntable so that the vertical profiling structure is not suspended above the first conveyor belt or the turntable, and the vertical profiling structure may be positioned at a working distance of between 1.7 meters and 1.9 meters from the turntable.

[0026] As another example, the method may also include configuring, by the computing system, a light source to emit light of a specific intensity and duration associated with a predetermined shutter speed of the plurality of cameras. The light source may be included in a photo booth, which may enclose an area including the turntable and the vertical profiling structure. The photo booth may include the light source that (i) prevents ambient light from entering the sides and top of the enclosed area, and (ii) projects light onto the tray as the tray rotates on the turntable.

[0027] In some embodiments, the method may also include: generating, by the computing system, a placeholder label for the pallet based on determining that the unique identifier on the pallet is not recognizable; generating, by the computing system, a pallet profile for the pallet using the placeholder label as an identifier for the pallet; and transmitting, by the computing system, a notification with the placeholder label to an automated warehouse device. The notification may cause the automated warehouse device to print the placeholder label and apply the printed placeholder label to a designated surface of the pallet.

[0028] One or more advantages can be achieved from the disclosed technology. For example, the combination of a rotating turntable and a fixed camera attached to a vertical rod near the turntable provides continuous capture of images from a uniform working distance. The vertical rod can be located at a specific working distance from the rotating turntable, and the camera can be arranged at a specific angle and a specific FOV. The tray can be centered on the turntable and rotated at a constant speed. Rotating the tray in a central position with a fixed camera instead of rotating the camera around the tray can provide a more uniform image to be captured. Images can be uniformly captured from the same distance, FOV and angle, which can make it easier, more efficient and more accurate to stitch images together. Images of each side of the tray can also be captured, thereby providing a more complete view of the tray for overview analysis via image analysis techniques.

[0029] Additionally, these images can provide more comprehensive image processing. Because images of each side of the pallet can be captured, labels, text, damage, missing content, skew, and other characteristics can be more easily identified using image processing techniques. When performing manual inspections, warehouse workers may monitor a portion of any side of the pallet, thereby missing information that can be critical to profiling and storing the pallet. For example, a warehouse worker may not be able to reach under the pallet to inspect for damage to the wood frame. As a result, the pallet may be stored improperly and the damage may be irreparable. This can lead to warehouse inefficiencies and increased costs throughout the supply chain.

[0030] The disclosed technology can also more accurately and efficiently identify pallet information using automation. Manual manual inspection of each pallet entering the warehouse can be time-consuming, inefficient, and can lead to errors in identifying pallet characteristics. The disclosed technology can provide automatic profiling of each pallet as it enters the warehouse and at least identify the label associated with the pallet. Once the label is identified, the pallet can be sent on a specific route for organized storage in the warehouse. Additional pallet information and characteristics can be identified from the image data of the pallet at a later time (for example, when the pallet is sent to an organized storage location on a specific route). If at least the label cannot be identified from the image data, the pallet can be sent on a specific route to a temporary storage location, where the warehouse worker can manually inspect the pallet and / or analyze the image data. Therefore, the warehouse worker can only inspect the pallet whose label is not identified by the disclosed technology. Compared with the warehouse worker who must manually inspect each pallet entering the warehouse, the warehouse worker can have a smaller work queue. Therefore, the warehouse worker can spend an appropriate amount of time to manually inspect the pallet, identify the label and determine where the pallet can be stored in the warehouse.

[0031] Compared with existing profiling systems, the disclosed technology can also be implemented at a lower cost in warehouse environments or other storage facilities. The vertical bar described herein may include very few cameras. The bar may be positioned anywhere in the warehouse and may be easily moved around in the warehouse. In some embodiments, the bar may include one to three cameras. These cameras can be easily repaired and maintained because the camera is in one position. It is not necessary to purchase independent cameras and position them throughout the warehouse and / or fix them to the components of the warehouse to perform the technology, methods and processes described herein. Existing profiling systems can use many cameras positioned throughout the warehouse. When the pallet moves throughout the warehouse, these cameras may not fully capture every angle and side of the pallet, resulting in inefficiency and poor pallet profiling results in the warehouse. In addition, setting, maintaining and / or fixing the cameras positioned throughout the warehouse may be timely and expensive. On the other hand, the disclosed technology can provide a space-efficient, centrally located profiling system, i.e., a vertical bar. Setting the vertical bar with a camera may be fast and efficient. Maintenance may also be performed quickly and easily. Finally, the central location of the camera at the vertical pole can take up less floor space in the warehouse, which can optimize space efficiency in the warehouse.

[0032] As another example, the disclosed technology may provide lights offset from the camera at the vertical pole to reduce glare. Thus, a clearer image may be captured. One or more red lights may also be used to eliminate wavelengths of light other than red light. By eliminating those wavelengths, image analysis techniques may be performed more accurately. The clearer the image, the easier, faster, and more accurately the computing system can identify labels and other pallet characteristics in the image. Similarly, the disclosed technology may provide for uniform lighting using photo booths positioned around a rotating turntable. The photo booths may help reduce or otherwise eliminate ambient lighting, which may facilitate capturing clearer images of the pallet with minimal or no glare. As described above, the clearer the image, the more accurately and quickly the computing system can identify the label of the pallet and route the pallet for storage.

[0033] As another example, the disclosed techniques may provide for maximizing throughput. Image processing and analysis may be performed at staggered time intervals. At a minimum, a determination of whether a pallet's tag is identifiable may be performed and completed during or before a first time goal. If time remains, additional processing and analysis may be performed to identify additional information about the pallet. Otherwise, a pallet may be routed to a storage location in a warehouse while another pallet is ingested and profiled. Thus, the disclosed techniques may provide for quickly identifying at least a pallet's tag and moving the ingested pallet through a profiling process and into storage. In doing so, warehouse efficiency may not be compromised by the profiling of the pallets described herein.

[0034] In addition, the disclosed technology can provide improved overall warehouse efficiency and optimize warehouse space. As described, the camera can be positioned on a single vertical pole, which can take up little space in the warehouse's footprint. The vertical pole can be easily moved around the warehouse to scan in different locations. Therefore, warehouse space can be optimized. The single vertical pole with a camera can also provide more accurate images of incoming pallets to capture. The more accurate the captured image, the more accurate and efficient the computing system's profiling of such pallets. When pallets are accurately and quickly profiled, they can be sent on a specific route for storage in the warehouse without causing delays or blockages in the incoming pallet queues and throughout the supply chain. Therefore, overall warehouse efficiency can be improved.

[0035] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1A is a conceptual diagram of a system for profiling items in a warehouse environment.

[0037] Figure 1BDepicting an example pallet of items that may be profiled in a warehouse environment.

[0038] Figure 1C Depicts example product identifiers identified on a box under different lighting conditions.

[0039] Figure 2 A perspective view of a photographing bay depicting a system for profiling items in a warehouse environment.

[0040] Figure 3A is a top view of an example configuration of a system for profiling items in a warehouse environment.

[0041] Figure 3B is a top view of another example configuration of a system for profiling items in a warehouse environment.

[0042] Figure 3C is a top view of another example configuration of a system for profiling items in a warehouse environment.

[0043] Figure 4A Depicts the camera configuration used to profile an item.

[0044] Figure 4B Depicting another camera configuration for profiling an item.

[0045] Figure 4C 1 through E depict another camera and lighting configuration for profiling an item.

[0046] Figure 5A is a block diagram of a process for profiling an item using the techniques described herein.

[0047] Figure 5B It is a reference Figure 5A Timeline of the process used to profile an item.

[0048] Figure 6 is a flow chart of a process for profiling pallets entering a warehouse environment.

[0049] Figure 7 is a flow chart of a process for identifying pallet information from image data of a pallet.

[0050] Figure 8 is a swim lane diagram of the process used to profile and store pallets in a warehouse environment.

[0051] Fig. 9 is a swim lane diagram of the process used to determine the storage location of profiled pallets.

[0052] Fig.10 is a system diagram that depicts system components that may be configured to perform any of the techniques described herein.

[0053] Fig.11A A through B are swim lane diagrams of an example process for profiling a tray.

[0054] Fig.12 is a block diagram of a computing device that may be used to implement the systems and methods described herein. DETAILED DESCRIPTION

[0055] This document describes systems, methods, and techniques for profiling items such as pallets as they enter a warehouse environment. The disclosed technology can provide a combination of a vertical pole with a fixed camera and a rotating turntable to provide continuous and uniform image capture of inbound pallets. The vertical pole can occupy minimal space in the warehouse and can be moved around the warehouse to a desired location. Light, such as and / or including red light, can be offset from the fixed camera and provide uniform illumination, which can produce a clearer image without glare. The captured image can be received and processed by a computing system to identify a unique identifier, such as a label, of the pallet in the image. Once the label is identified, the computing system can route the pallet for storage in the warehouse environment. If the label cannot be identified, the computing system can route the pallet to a temporary storage area and add the captured image to a queue for review and inspection by a warehouse worker. The processes, methods, and techniques described herein can be repeated for each inbound pallet.

[0056] Referring to the drawings, FIG1 is a conceptual diagram of a system 100 for profiling items in a warehouse environment 128. The warehouse environment 128 may be any type of storage facility or other environment for moving, routing, and / or storing items for varying lengths of time. Exemplary warehouse environments include cold storage facilities, distribution centers, and general warehouses. The system 100 includes an upper conveyor belt 102, a lower conveyor belt 106, a turntable 104, a vertical rod 108 (e.g., a profiling structure), and a photo booth 126. As shown in FIG. Figure 2 As further described in FIG. 3 , the photo booth 126 may be used to provide uniform lighting conditions for better and clearer image capture. In some implementations, the system 100 may not include the photo booth 126 .

[0057] The conveyor belts 102 and 106 may be any type of conveyor belt, roller or other mechanism used in a warehouse environment to move (e.g., automatically, semi-automatically and / or manually) throughout the warehouse environment 128 and route items. The turntable 104 may be a rotatable platform. The upper conveyor belt 102 may be configured / attached to the turntable 104 so that a pallet 120 with items 122 (e.g., boxes, containers, etc.) positioned on the belt 102 may be automatically moved onto the turntable 104. When the pallet 120 arrives at the warehouse 128, the upper conveyor belt 102 may receive the pallet. For example, the upper conveyor belt 102 may be in or near a docking station area of ​​the warehouse 128. The pallet 120 may be unloaded from a truck at the docking station and automatically or manually placed (e.g., injected) onto the conveyor belt 102 (e.g., by a warehouse worker, a forklift, etc.). The conveyor belt 102 may automatically move the pallet 120 toward the turntable 104.

[0058] Before the pallet 120 is moved onto the turntable 104, an adjustment device 138 configured to the upper conveyor belt 102 may align and / or center the pallet 120 on the belt 102. Thus, the pallet 120 may be delivered onto the turntable 104 at the center or midpoint of the turntable 104. In some implementations, the pallet 120 may also be automatically centered at the midpoint of the turntable 104. Centering and aligning the pallet 120 may facilitate capturing a uniform image of the pallet 120, as further described below.

[0059] Still referring to FIG. 1 , the lower conveyor belt 106 may be configured / attached to the turntable 104 so that once an image of the pallet 120 is captured, the pallet 120 may be automatically moved onto the belt 106, as further described below. The lower conveyor belt 106 may include one or more branches for routing the pallet 120 to an organized storage location 132 and / or a temporary storage location 130 in a warehouse 128. The pallet 120 may move along the conveyor belts 102 and 106 and / or the turntable 104 at a constant speed. In some embodiments, the speed may be 2 in / sec. Based on the configuration and performance of the components in the system 100 described herein, the pallet 120 may move at a faster or slower speed.

[0060] The turntable 104 may be configured to rotate the tray 120 in a clockwise and / or counterclockwise direction about the z-axis. In some embodiments, the turntable 104 may rotate the tray 120 in one direction during a first scan (e.g., pass) of the tray 120 and then rotate the tray in the opposite direction during a second scan of the tray 120. The turntable 104 may have a slow but smooth rotation so that images of the tray 120 can be captured from every side / angle. For example, the turntable may rotate at a speed of 2in / sec. One or more other rotation speeds may be used. In some embodiments, the speed and / or direction of rotation may be adjusted based on the tray being scanned. For example, for a large tray having a larger surface area, the turntable 104 may rotate at a faster speed than a tray that is half the size of the large tray and thus has a smaller surface area.

[0061] The vertical rod 108 may include one or more cameras 110A to N. Optionally, the vertical rod 108 may also include one or more light sources, such as those described in reference Figure 2 As described in 4. Figure 1A , camera 110N may also be positioned within photo booth 126 above turntable 104. Camera 110N may be configured to capture images of the top surface of tray 120. In general, cameras 110A-N may be configured to continuously capture image data of tray 120 as it rotates on turntable 104. One or more of cameras 110A-N may be offset 90 degrees from turntable 104. Cameras 110A-N may also be offset at other angles and / or positioned relative to turntable 104.

[0062] The pole 108 may be a uniform height to accommodate any size pallet that may be placed on the turntable 104. In some embodiments, the height of the pole 108 may be adjusted depending on the size of the pallet being scanned. The pole 108 may be fixed and placed at a working distance away from the turntable 104. Since the pole 108 is fixed, the cameras 110A-N attached thereto may capture images of the pallet 120 from all sides based on a constant distance, field of view (FOV), and angle.

[0063] In some embodiments, the pole 108 may include a camera positioned at the top of the pole 108 to capture images of the top of the pallet 120. The pole 108 may also include a camera positioned at the bottom of the pole 108 to capture images of the bottom of the pallet 120. The placement of such cameras may be advantageous in providing the capture of images of portions of the pallet 120 that may otherwise be difficult for a warehouse worker to inspect or see. Thus, the image may provide a more complete view of the entire pallet 120 for overview analysis.

[0064] The cameras 110A-N may be attached to the pole 108 at predetermined angles so that images of each side of the pallet 120 may be captured from multiple angles. Thus, the images may be stitched together to generate a comprehensive image model of the pallet 120 for further processing (e.g., identifying characteristics of the pallet 120). In some embodiments, one or more of the cameras 110A-N may be other types of sensing or imaging devices, including but not limited to LIDAR, 3D stereo cameras, thermal sensors, and / or scanners (e.g., barcodes, RFID, QR codes). In other embodiments, the pole 108 may include one or more cameras 110A-N in addition to one or more LIDAR sensors, thermal sensors, and scanners. Positioning all cameras, sensors, and / or scanners on a single pole 108 may provide for more efficient use of space in the warehouse environment 128, as well as more time-saving, efficient, and accurate scanning and profiling of pallets.

[0065] In some embodiments, the vertical rod 108 may include one camera 110A. For example, the camera 110A may have a larger FOV to capture the tray 120 from top to bottom. The rod 108 with one camera 110A may be positioned away from the turntable 104 (e.g., a larger working distance) so as to capture the tray 120 from top to bottom. This configuration may be advantageous because it may be less expensive to implement than multiple cameras. In addition, one camera 110A may be a high-resolution camera. Profiling the tray 120 will not require stitching together images from multiple cameras, which may reduce processing time and use less computing resources. Therefore, profiling by image processing may become more efficient.

[0066] In some embodiments, vertical bar 108 may include two cameras 110A to B. Cameras 110A to B may have larger FOV and may be high-resolution cameras. Profiling pallet 120 may require stitching images from two cameras 110A to B together, which uses less processing power and computing resources than stitching images from more than two cameras together. As an exemplary configuration, bar 108 may be positioned at a working distance of 1.9m away from turntable 104. Bar 108 may have a maximum height of 2.75m (e.g., 108 inches). First camera 110A may be positioned at 1.6m (e.g., 63 inches) from the bottom of bar 108. Second camera 110B may be positioned at 1.6m from the top of bar 108. Therefore, cameras 110A to B may have a FOV overlap of approximately 0.225m (e.g., 8.9 inches). This FOV overlap may facilitate easier, faster, and more accurate stitching together of images captured by cameras 110A-B.

[0067] In other embodiments, the vertical pole 108 may include three cameras 110A to C, as shown in FIG. Figure 4A Pole 108 may also include additional cameras 110D-N or other sensors or imaging devices.

[0068] The vertical rod 108 configuration may be advantageous because it uses less space in the warehouse environment 128 and also prevents potential imaging and profiling differences that may occur when the camera rotates around the pallet. When the camera rotates around a fixed pallet, each image captured may be taken from a different angle and / or at an unequal distance from the pallet. For example, a rotating camera may move up and down around each side of the pallet. By moving up and down, the rotating camera may inherently be repositioned at a different distance from the pallet to capture each image. Once the images are stitched together, this may result in image distortion. Stitching such images together may require additional processing power and computing resources, and may result in inaccurate imaging and pixelation. Therefore, pallet characteristics may be inaccurately identified or otherwise not identified.

[0069] On the other hand, the rod 108 configuration ensures that all images are captured from an equal distance (e.g., working distance) from the tray 120 as the tray 120 rotates on the turntable 104. Images of equal distance can be more easily stitched together to produce a composite image of the tray 120 for profiling and feature identification.

[0070] The system 100 may optionally include one or more sensors, cameras, and / or scanners configured to the turntable 104. For example, one or more sensors may be configured to the turntable 104 to detect and determine the weight of the pallet 120. As another example, one or more cameras, scanners, or other imaging devices may be attached to the top surface of the turntable 104 to capture one or more images or other information located on the bottom of the pallet 120 (e.g., damage to the bottom of the wood frame). As yet another example, the turntable 104 may have one or more motion sensors to determine the speed and / or angle at which the turntable 104 rotates.

[0071] Still refer to Figure 1A, the system 100 includes a computing system 152, a warehouse management system (WMS) 154, a user device 156, and a pallet profile data storage device 158 that communicate (e.g., wired and / or wirelessly) via a network 150. The computing system 152 can control one or more components of the system 100 to profile the pallet 120. In some embodiments, the computing system 152 can be a combination of one or more computing systems including the WMS 154. The computing system 152 can also perform image processing techniques to identify information about the pallet 120. For example, the computing system 152 can receive image data from the cameras 110A to N (step A, 160). In some embodiments, the computing system 152 can indicate when the cameras 110A to N capture an image of the pallet 120 (e.g., once the pallet 120 is sensed or otherwise detected on the turntable 104). The computing system 152 can receive the image data when the image is captured by the cameras 110A to N. The computing system 152 may also receive image data after the tray 120 is rotated a full 360 degrees by the turntable 104 .

[0072] The computing system 152 may profile the tray 120 based on the image data (step B, 162). As described throughout this disclosure, the computing system 152 may perform image processing and analysis techniques to identify a tag or other unique identifier associated with the tray 120. As an example, the computing system 152 may be trained to identify tags having an x-dimension ranging from 0.495 mm to 1.016 mm, a height of 31.75, a left quiet zone of at least 10 times the x-dimension, and a right quiet zone of at least 10 times the x-dimension. The computing system 152 may also be trained to recognize tags of different sizes. The computing system 152 may also identify information or characteristics about the tray 120 from the image data. The computing system 152 may profile the tray 120 as the image data is received. The computing system 152 may also profile the tray 120 once all of the image data is received (e.g., the tray 120 has rotated a full 360 degrees on the turntable 104).

[0073] The computing system 152 may generate control instructions based on profiling the pallet 120 (step C, 164). The control instructions may be implemented by the computing system 152 to control one or more components of the system 100. For example, the control instructions may cause the pallet to automatically move from the turntable 104 to the lower conveyor belt 106. The pallet 120 may then be routed to an appropriate storage location (step D, 166). The control instructions may indicate an appropriate storage location. As further described below, an appropriate storage location may be determined based on whether the computing system can accurately identify a label of the pallet 120 when profiling the pallet 120 (step B, 162). If the label can be accurately identified, the pallet 120 may be routed (step D, 166) to the organized storage location 132. If the label cannot be accurately identified, the pallet 120 may be routed (step D, 166) to the temporary storage location 130.

[0074] The computing system 152 may also transmit control instructions to the WMS 154 (step C, 164). Instructions may be sent to the WMS 154 when the label of the pallet 120 cannot be identified from the image data. When the instructions are sent to the WMS 154, the pallet 120 may also be automatically routed to the temporary storage area 130 (step D, 166). The WMS 154 may communicate with the user device 156 to identify the pallet 120 placed in the temporary storage area 130 (step E, 168). As further described below, the warehouse worker may inspect the image data of the pallet 120 in the temporary storage area 130 to identify the label of the pallet 120 and the organized storage location of the pallet 120. The warehouse worker may also manually inspect the pallets in the temporary storage location 130. The user device 156 may identify or otherwise profile pallets that were previously unable to be identified. In other words, the user device 156 may receive image data of the pallet 122, where this image data may be provided to the WMS 154 by the computing system 152. The WMS 154 may use the image data to determine the organized storage location 132 of the pallet 122. The user device 156 may present the image data to the warehouse worker in a graphical user interface. The warehouse worker may view this image data and use the image data to perform manual and / or visual inspection of the pallet 122.

[0075] Once the pallet 120 is identified by the warehouse worker and / or WMS 154 (step F, 170), the pallet 120 may be moved to the organized storage location 132 for the pallet 120 (step F, 170). For example, the pallet 120 may be automatically moved from the temporary storage area 130 to the organized storage location 132 by an automated mechanical transport system (e.g., a conveyor belt), a crane, or other autonomous vehicle. The pallet 120 may also be moved manually, such as by a warehouse worker and / or a forklift.

[0076] Figure 1B An example pallet 172 of items that may be profiled in a warehouse environment is depicted. Boxes 174A-N may be stacked on the pallet 172. The boxes 174A-N may contain a large number of products (e.g., items). Each of the boxes 174A-N may have a corresponding label 176A-N (e.g., barcode, SKU). The labels 176A-N may identify the box 174A-N and the contents of the box 174A-N. Each of the boxes 174A-N may also have text identifying the contents of the box 174A-N, the customer, and other information related to the box. As shown, the boxes 174A-N may be stacked on the pallet 172 in multiple layers. Each layer on the pallet 172 may have a different number, type, and / or size of boxes 174A-N. Thus, each layer of boxes 174A-N may be different from each other. For example, the bottom layer may include 4 large boxes 174A-N containing product Y. The top layer may contain 6 small boxes 174A to N containing product X.

[0077] The boxes 174A to N may then be packaged together on a pallet 172 using plastic packaging 180 (e.g., transparent, translucent). The pallet 172 may have its own identifier 178 (e.g., barcode, label, SKU). Figure 1B , when boxes 174A-N are packed together, packaging 180 may obscure one or more labels 176A-N and / or markings / text on boxes 174A-B. Environmental obstructions 182 (cross-hatched areas) such as frost, dust, abrasion, etc. may also occur, further hindering clear viewing of pallet identifiers 178 and / or any of the labels 176A-N of boxes 174A-N. Packaging 180 and / or environmental obstructions 182 may present challenges to unambiguously identifying boxes 174A-N on pallet 172, particularly when plastic packaging 180 may reflect light to obscure boxes 174A-N below. Therefore, computing system 152 may be adapted to handle such obstructions when profiling a pallet, as described herein.

[0078] Using the techniques described herein, computing system 152 may determine Figure 1C172 . The pallet profile 184 may include information determined by the computing system 152 when profiling the pallet 172. For example, the profile 184 may include information such as a system ID, the name and / or quantity of each type of product and / or box on the profile, the customer pallet ID, the height, width, depth, weight, and a timestamp of when the pallet 172 was profiled. One or more additional or other pieces of information may be determined for the pallet 172 and stored in the pallet profile 184. As described throughout this document, the computer system 152 may use any of a variety of image analysis techniques to identify and determine this information from the pallet, such as OCR techniques, machine learning models, other techniques, and / or combinations thereof.

[0079] Figure 1C Depicted are example product identifiers identified on a box under different lighting conditions. Box 186 may include product identifier 190. Box 186 may also be packaged with plastic wrap 188, which may cover a portion or all of product identifier 190. An image 191 of product identifier 190 may be captured by one or more cameras described herein (e.g., reference 184). Figure 1A captured by cameras 110A to N).

[0080] The camera may include a red light filter. In addition, one or more red strobe lights may be positioned near the box 186 to help illuminate the product identifier 190 and reduce glare from the plastic packaging 188. The camera may be set to capture a 10 decibel gain (dB) using a 5 millisecond (ms) exposure and a red strobe light. The camera may also be configured with a 630 nanometer (nm) bandpass filter. An image 191 is generated that shows a clear image of the product identifier 190. The plastic packaging 188 may not cause glare or otherwise hinder the ability to read the product identifier in the image 191 using the techniques described herein.

[0081] On the other hand, box 192 is set under different lighting and exposure conditions. Box 192 includes product identifier 196, which may be below a layer of plastic wrap 194. The camera may be set to capture 10dB gain using 2ms exposure and red strobe light. The camera may also have a 630nm bandpass filter. As shown, image 196' is not very clear in depicting product identifier 196. In addition, part of the product identifier is blocked by plastic wrap 194 in image 196'. In some embodiments, the computing system described herein may still profile pallet 192 based on image 196' of product identifier 196. However, the camera and lighting conditions may be adjusted so that the camera may be exposed for a longer period of time. Therefore, using the technology described herein, a clearer, fully illuminated image of the product identifier may be captured, as shown with respect to image 191. This image may be used by the computing system to more accurately profile the pallet and identify information about the pallet.

[0082] In some alternative embodiments, the camera lens may be configured with a polarizing filter. In addition, the LED light source may be positioned offset from the camera lens to reduce glare in the lens, as described throughout this disclosure. The LED light source may emit red light on the tray. In some embodiments, the LED light source may emit white light and a polarizing film may be applied to the light source to filter out all wavelengths except those corresponding to red light. Therefore, a polarizing film may be configured to the LED light source. By using a polarizing filter on the camera lens, an LED light source, and a polarizing film, wavelengths of light other than red light may be filtered out. In the red light setting, the camera may produce a clearer image of the tray with reduced or no glare. The computing system may more easily analyze the image, as described herein.

[0083] The use of polarizing filters, LED light sources, and polarizing films may cause the camera to capture light having wavelengths between 620nm and 650nm. These wavelengths may correspond to red light. Thus, the tray may be imaged under a red light setting, which may provide for easier and more accurate image analysis performed by the computing system.

[0084] Figure 2 A perspective view of a photographing booth 126 of a system 100 for profiling items in a warehouse environment 128 is depicted. Figure 2 The photo booth 126 depicted in FIG. 1 is an example configuration. Various other configurations may be utilized, as described throughout this disclosure. The photo booth 126 may be made of a dark material (e.g., cloth) that may be used in other photographic backgrounds. The dark material may be useful in preventing ambient light from entering the enclosed area and / or preventing the cameras 110A to N from capturing clear, evenly illuminated images of the tray.

[0085] like Figure 2104. The photo booth 126 may be formed by two walls 126A and 126B. Wall 126A may include an opening 127 through which a tray may pass when it moves from the conveyor belt 102 to the turntable 104. Wall 126B may include an opening 129 through which a tray may pass when it moves from the turntable 104 to the lower conveyor belt 106. The openings 127 and 129 may not be covered by materials such as fabric, plastic, baffles, or doors. In fact, in some embodiments, the openings 127 and 129 may be sized to match the vertical height and width of the largest tray that can pass through the photo booth 126. The openings 127 and 129 may be sized in other ways to match the size of the tray entering the photo booth 126. Thus, when the tray rotates on the turntable 104, the size of the tray may block any light that may pass through the openings 127 and 129. In some implementations, the compartment 126 can also include a top portion connecting the walls 126A and 126B. The compartment 126 can also include side walls, such that the compartment 126 can form a completely enclosed area around the turntable 104, thereby not allowing ambient light to enter the enclosed area.

[0086] exist Figure 2 In an example of a vertical rod 108, a vertical rod 108 may be positioned between walls 126A and 126B at a set working distance away from the turntable 104. The cameras 110A-N may be vertically aligned along the rod 108. Additionally, the lights 112A-N may be vertically aligned along the rod 108. The lights 112A-N may be offset from the cameras 110A-N to reduce or otherwise avoid glare in the lenses of the cameras 110A-N and to provide uniform lighting on the turntable 104.

[0087] Figure 3A 1 is a top view of an example configuration of a system 300 for profiling items in a warehouse environment 128. In this example, the lower conveyor includes belts 106A and 106B. Belt 106A can route pallets to an organized storage location in the warehouse 128. Belt 106B can route pallets to a temporary storage location in the warehouse 128. Further, in this example system 300, the photo booth 126 is a complete enclosure around the turntable 104. The booth 126 has sides and a top to prevent ambient light from entering the complete enclosure.

[0088] The compartment includes a bar 134 positioned above and across the center of the turntable 104. The bar may include one or more overhead cameras 110A and one or more overhead lights 112A. The overhead camera 110A may be configured to capture images of the top of the tray as it rotates on the turntable 104. The lights 112A may also provide uniform illumination of the top of the tray. A vertical rod 108 (e.g., a frame, a vertically oriented support structure) may include one or more cameras 110B to N and one or more lights 112B to N for capturing images of the sides of the tray as it rotates on the turntable 104. As depicted in this example, the cameras 110B to N and lights 112B to N may be mounted to the rod 108 and oriented toward the turntable 104 (and the tray to be illuminated and imaged when on top of the turntable 104), resulting in the cameras 110B to N and lights 112B to N having the same or similar orientation and field of view / illumination 302. The lights 112B-N may be mounted to the pole 108 in an arrangement that allows them to be spaced vertically and / or horizontally from the cameras 110B-N, and may be disposed in any of a variety of ratios relative to the cameras 110B-N (e.g., there may be two lights around each camera to provide uniform lighting from the perspective of each camera). As described above, the lights 112A-N may also be offset from the cameras 110A-N to minimize or otherwise avoid glare in the lenses of the cameras 110A-N, as described below. Figure 3B to the example described in C.

[0089] Figure 3B 1 is a top view of another example configuration of a system 310 for profiling items in a warehouse environment 128. As with the other examples discussed above, the photo booth 126 may enclose the entire area around the turntable 104 and may include a top surface that provides a complete enclosure for the area around the turntable 104 (and the cameras 110A-N and the lights 112A-N). Here, the vertical pole 108 includes the cameras 110A-C. The lights 112A-N may be attached to separate vertical light support structures 109A-B (e.g., poles, columns, structures) and laterally offset from the vertical pole 108 with the cameras 110A-N. The light support structures 109A-B may also be arranged at an angle and / or working distance that is different from the angle and / or working distance of the pole 108, which may cause the cameras 110A-N to have a field of view 312 that is different from the illumination field 314A-B of the lights 112A-N. The example system 310 may include the same configuration as described above with respect to FIG. Figure 3AThe top-down cameras and lights (not depicted) are similarly described as being located above the turntable 104. By offsetting the lights 112A-N so that the illumination fields 314A-B have different angles relative to the turntable 104 than the field of view 312 of the cameras 110A-N, the system 310 illuminates the turntable 104 while minimizing glare in the lenses of the cameras 110A-N, which can increase the fidelity and clarity of the pallet images captured by the cameras 110A-N and thus increase the accuracy of the pallet profiling performed by the system 310.

[0090] Figure 3C 1 is a top view of another exemplary configuration of a system 330 for profiling items in a warehouse environment. The system 330 is depicted as having a photo booth 126, but a system 330 without a photo booth may be used. The system 330 further includes cameras 110A to N mounted on the vertical rod 108. The cameras 110A to N may be modularly mounted to the vertical rod 108 so that the cameras 110A to N may be moved up and down and / or left and right on the vertical rod 108. Changing the camera positioning may be advantageous for accommodating different configurations (i.e., different fields of view) of the cameras 110A to N and capturing image data of pallets of different sizes. For example, when a shorter pallet rotates on the turntable 104, the cameras 110A to N may move closer together on the vertical rod 108 and / or move lower on the vertical rod 108. As another example, when a taller pallet rotates on the turntable 104, the cameras 110A-N may be moved along the rod 108 to have a greater vertical distance between the cameras so that the top to the bottom of the pallet may be captured by the cameras 110A-N. Other variations and / or configurations of the cameras 110A-N mounted on the rod 108 are also possible.

[0091] The lighting elements 112A and 112B are positioned at an angle offset from the cameras 110A to N. Each of the lighting elements 112A and 112B may be mounted to a light pole 109A and 109B, respectively, each lighting element may include one or more individual lighting units (e.g., one or more light bars). For example, two vertically oriented light bars may be mounted to each of the light poles 109A and 109B. In some implementations, one or more of the light poles 109A and 109B may have additional or fewer light bars mounted thereon. The number of light elements for each pole 109A and 109B may be based on any of a variety of factors, such as camera settings, the size of the tray 122, one or more distances between the light poles 109A and 109B and the tray 122, each other, and / or the cameras 110A to N, and / or one or more angles of the light poles 109A and 109B relative to the tray 122 (discussed further below).

[0092] Although the lighting elements 112A-B are shown as being mounted to the poles 109A-B, the lighting elements may alternatively be mounted to the frame of the picture taking booth 126 .

[0093] The cameras 110A-N on the vertical pole 108 may be positioned at a distance D1 (342) from a front plane 334 of the tray 122 that faces the cameras 110A-N and is imaged by the cameras. The distance D1 (342) may be selected based on a variety of factors, such as the expected size range of the trays 122 to be imaged (e.g., height, width), the number of cameras 110A-N on the vertical pole 108, the type of camera lenses used with the cameras 110A-N, and / or camera settings, such as the field of view (FOV) 336 of the cameras 110A-N. The cameras 110A-N may be oriented toward the tray 122 along a vertical plane 350 that bisects the turntable 104 and / or the tray 122 at or near a midpoint and extends from the turntable 104 and / or the tray 122 in a substantially orthogonal direction. The FOV 336 may be oriented along the plane 350 and may have an angle θ a , which angle may be selected in conjunction with the distance D1 to ensure that the entire tray 122 can be captured by the cameras 110A to N. For example, the vertical rod 108 may be moved to a different distance D1 to accommodate one or more changes to the system 330, such as the cameras 110A to N being changed and the resulting angle θ of the FOV 336. a Change.

[0094] Each of lighting elements 112A and 112B may be positioned at a distance D2 (344) from plane 334 of tray 122. Lights 112A and 112B may also be positioned at a distance D3 (346) in front of plane 332 corresponding to the lenses of cameras 110A-N. In some examples, lighting elements 112A-B may be positioned behind plane 332, resulting in distance D2 being greater than distance D1. Each of lighting elements 112A-B may also be positioned at a distance D4 (348) away from and offset from cameras 110A-N and plane 350. Lights 112A and 112B may be oriented along a plane 352A-B that is different from plane 350 of cameras 110A-N and is oriented at an angle θ b and θ c (340A-B) is offset from plane 350, which angle may be the same or different for each of lighting elements 112A-B. For example, angle θ b and θ c The illumination elements 112A-B may be selected to provide sufficient illumination for the cameras 110A-N to reliably capture a clear image of the tray 122 within the photographing compartment 126 while minimizing glare. d and θe Directed illumination field 338A-B, light is emitted / projected by illumination elements 112A-B across the angle. The positioning and orientation angles of illumination elements 112A-B may be offset from the position and orientation of cameras 110A-N (e.g., offset distances D3 and D4, and offset angle θ b and θ c ), which may help improve the quality of the images captured by the cameras 110A-N (e.g., reduce glare in the images). The lighting elements 112A-B may be positioned outside of the FOV 336 of the cameras 110A-N and away from and oriented relative to the tray 122 so that the illumination fields 338A-B individually or collectively fully illuminate the entire width of the tray 122. For example, each lighting element 112A-B may be positioned and oriented so that they each project light across the entire width of the tray 122 (based on the illumination fields 338A-B of the lighting element). In another example, each lighting element 112A-B may be positioned and oriented to illuminate only a portion of the width of the tray 122 so that collectively they provide complete illumination of the entire width of the tray 122. The offset angle θ based on the orientation 352A-B of the lighting elements 112A-B b and θ c (340A to B) may be any of a variety of angles between 0° and 90°, such as 15°, 30°, 45°, 60°, 75°, less than 45°, greater than 45°, any of a variety of ranges (e.g., 1° to 15°, 15° to 30°, 30° to 45°, 45° to 60°, 60° to 75°, 75° to 89°, or any combination thereof), and / or other angles or ranges between 0° and 90°.

[0095] The lighting elements 112A-B may have the same illumination field 338A-B, and the same positioning and orientation across the plane 350 and reflected on opposite sides of the plane. For example, the distance D4 may be equidistant for both lights 112A and 112B. In other examples, one of the lights 112A may have a different illumination field 338A, a different orientation 352A with a different angle 340A, and / or may have a shorter distance D3 and / or D4 away from and offset from the cameras 110A-N than the other light 112B.

[0096] Distances D2 and D3 may be equal. For example, lights 112A and 112B may be equidistant from plane 332 of cameras 110A-N and plane 334 of tray 122. In some examples, distance D2 may be greater than distance D3. In other examples, distance D3 may be greater than distance D2. Furthermore, as described throughout, lights 112A and 112B may be easily moved (e.g., backward or forward) and / or rotated at different angles.

[0097] The lighting elements 112A to 112B are Figure 3C 122) as being positioned outside the width of the tray 122 (i.e., the distance D4 is greater than half the width of the tray 122) (as indicated by plane 358 on one side), which may help increase the achieved angles 340A-B and help reduce potential glare in images captured by cameras 110A-N. Alternatively, the lighting elements 112A-B may be positioned at or on the perimeter / width of the tray 122 (i.e., the distance D4 is set to approximately half the width of the tray 122), and / or may be positioned inside the perimeter / width of the tray 122 (i.e., the distance D4 is less than half the width of the tray 122). For example, such adjustment of the lighting elements 112A-B at or inside the plane 358 may permit the photo booth 126 to be reduced in size and more compact.

[0098] The photo booth 126 may include openings 360 and 362 through which the tray 122 may be received into the photo booth 126 via the conveyor 102 for imaging, and transported out of the photo booth 126 via the conveyor 106 after imaging is completed. The openings 360 and 362 may be sized and shaped to accommodate the maximum size of the tray 122 within a threshold margin (e.g., 5% size margin, 10% size margin, 20% size margin) to minimize the amount of ambient light received into the photo booth 126 through the openings 360 and 362. Additionally, the size and configuration of the photo booth 126 relative to the turntable 104, conveyor belts 102 / 106, and the openings 360 / 362 may be configured to minimize the distance D5 between the edge of the tray 122 (plane 358) and the openings 360 / 362. By minimizing this distance D5, the sides of the tray 122 facing the openings 360 and 362 can be used to block most of the ambient light that would otherwise be received into the picture taking compartment 126 and could interfere with the imaging of the tray 122 by the cameras 110A-N. In some examples, the picture taking compartment 126 may additionally include opaque light shields 366 and 368 adjacent to the openings 360 and 362 that may substantially fill the space between the tray 122 and the openings 360 and 362 (i.e., span most, all, or more of the distance D5) to further block ambient light that is not blocked by the tray 122 itself. The light shields 366 / 368 may extend a vertical span within the picture taking compartment 126, such as the height of the tray 122, the full height of the picture taking compartment 126, and / or other vertical spans. The light shields 366 / 368 may be configured to block ambient light without interfering with or otherwise contacting the trays 122 on the conveyors 102 / 106 and / or the turntable 104. The light shields 366 / 368 may additionally be made and / or coated with a non-reflective material or color, which may reduce unintentional light reflections during the image capture process.

[0099] Although the openings 360 and 362 are shown overlapping the conveyors 102 and 106, respectively, such that the entirety of the turntable 104 is contained within the photo booth 126, other configurations are possible, such as portions of the openings 360 and 362 overlapping portions of the turntable 104 while still permitting a full rotation of the pallet 122 without obstruction / contact between the walls of the openings 360 / 362 and the pallet 122. This configuration can further reduce the distance D5 and also reduce the ambient light received into the photo booth 126 via the openings 360 / 362. The photo booth 126 is configured in the depicted example to have no doors at the openings 360 and 362, which can help increase the throughput of pallets through the system 330 by not having to wait for delays associated with opening and closing doors, and can also make the system 330 more robust by reducing potential points of failure (e.g., failures due to malfunctioning / damaged automated doors). However, in some examples, the picture taking booth 126 may be configured to include automated doors at the openings 360 and 362 , which may additionally shield ambient light from entering the picture taking booth 126 .

[0100] Figure 3C 1 and 2. Also depicted in FIG. 1 are exemplary overhead cameras 370 and overhead lighting elements 372A-B, which may be mounted to and suspended from the top surface / ceiling of the picture taking compartment 126. The overhead camera 370 may be considered a portion of the cameras 110A-N and may be centered above the tray 122 and / or the turntable 104. Although one overhead camera 370 is depicted, more than one overhead camera may be used and spaced apart so as to cover the top surface of the tray 122, for example, along with the cameras 110A-N mounted to the uprights 108. Similar to the cameras 112A-B relative to the cameras 110A-N, the overhead lighting elements 372A-B may be spaced apart from and offset from the camera 370. The lighting elements 372A-B may be mounted to a common mounting structure with the camera 370 and may have the same distance from the top surface of the tray 122 and the same orientation as the cameras 370. Alternatively, the above discussion regarding positioning and orienting lighting elements 112A-B to be offset from cameras 110A-N may apply to overhead lighting elements 372A-B relative to overhead camera 370 .

[0101] Any of a variety of lighting elements and lighting arrangements may be used for lamps 112A-B and 372A-B. For example, lamps 112A-B and 372A-B may be configured to emit light within the visible spectrum, such as monochromatic light within the visible spectrum (i.e., red light having a wavelength between 610nm and 750nm, orange light having a wavelength between 595nm and 610nm, yellow light having a wavelength between 580nm and 595nm, and / or others) and / or polychromatic light combining multiple different wavelengths (e.g., "white" light combining light from the visible spectrum, other polychromatic light combinations). For example, lamps 112A-B and 372A-B may be configured to emit red light having a wavelength of approximately 610nm, 615nm, 625nm, 650nm, 675nm, 700nm, 725nm, 750nm, and / or other variations of red light. Lamps 112A-B and 372A-B may additionally and / or alternatively be configured to emit light outside the visible spectrum, such as infrared light (e.g., wavelengths above 750 nm), ultraviolet light (e.g., between 100 nm and 400 nm), and / or other portions of the invisible spectrum. Lamps 112A-B and 372A-B may emit light in pulses and / or strobes, consistent with images captured by cameras 110A-N and 370. For example, pulse durations may include, for example, 50 μs, 100 μs, 200 μs, 250 μs, 500 μs, 750 μs, 1 ms, 2 ms, 5 ms, 25 ms, 50 ms, 100 ms, 500 ms, 750 ms, 0.01 s, 0.05 s, 0.10 s, 0.5 s, 0.75 s, 1.0 s, 1.5 s, and / or other durations. The shutter speed used by cameras 110A-N and 370 may correspond to (or be shorter or longer than) the pulse duration used for lights 112A-B and 372A-B, e.g., be the same as the pulse duration. Cameras 110A-N and 370 may use any of a variety of aperture settings, such as f / 2, f / 4, f / 6, f / 8, f / 12, f / 16, f / 22, and / or other aperture settings. The larger the f-stop (i.e., "f / 8"), the smaller the aperture, the less light enters the image sensor of the camera, and the greater the depth of field of the captured image (i.e., more items at different depths are captured in focus). The opposite is true for smaller f-stops. The aperture setting and / or shutter speed may be selected based on, for example, any one or more of the distances D1, D2, D3, and D4 and the angles 340A and 340B of the lights 112A and 112B to reliably capture a clear image of the tray 122 (i.e., an image in which the contents of the tray 122 are in focus and without blur / glare).

[0102] The cameras 110A-N may be positioned at least a threshold distance away from the pallets 122 and the turntable 104, such as such that the distance D1 is at least 0.5 m, 0.75 m, 1.0 m, 1.2 m, 1.4 m, 1.5 m, 1.7 m, 1.8 m, 2 m, and / or other threshold distances. For example, such a larger distance may permit the cameras 110A-N to capture images of the entire surface of the pallet 122 facing the cameras 110A-N with camera settings (i.e., aperture, shutter speed) that provide greater image reliability and availability across a group of pallets that will vary in size, shape, composition, condition, material, and / or other details. For example, intermediate apertures (e.g., f / 6, f / 8, f / 11) can be used to permit the use of faster shutter speeds (e.g., 50μs, 100μs, 200μs, 250μs, 500μs) while also capturing images with a greater depth of field (i.e., a greater range of depths away from the camera is captured in focus).

[0103] For high contrast imaging that can provide better machine vision and automated analysis, the lights 112A-B and 372A-B can illuminate the turntable 104 and the tray 122 with a single wavelength (e.g., red light rather than white light). For example, red lighting elements and / or red light filters can be applied to the lights 112A-B and 372A-B to reject all other wavelengths except red, thereby emitting monochromatic light that can focus on details of interest, such as labels, bar codes, and other product identifiers. For example, such red light can be useful for avoiding glare that can occur due to light reflected on plastic packaging or other parts of the tray and items thereon, and / or removing ambient light that would otherwise interfere with details in the image. In one example, the lights 112A-B and 372A-B may be red light emitting diodes (LEDs) having any of a variety of configurations, such as 225 mm in length, positioned approximately 1.7 m (e.g., 67 inches) away from the turntable 104, and having a 45 degree emission angle, having a light coverage (illumination field) of approximately 1.4 m. When the emission angle is sufficiently sharp relative to the FOV of the cameras 110A-N and 370, glare in the lenses of the cameras 110A-C may be avoided or otherwise mitigated. Although polarizers may be used for the lights 112A-B and 372A-B and / or the cameras 110A-N and / or 370, they may not be needed if the lights 112A-B and 372A-B have a larger angle relative to the orientation of the cameras 110A-N and / or 370. With the configurations described herein, the cameras 110A-C may use any of a variety of lens lengths (e.g., 35 mm lens length) and / or any of a variety of focal lengths (e.g., f / 4). The lights 112A-B and 372A-B may also be floodlights, which may be, for example, approximately 330 mm in length. In some configurations, the lights 112A-B and 372A-B may be strobe lights placed in front of the tray 120. The cameras 110A-C may also have polarizers and / or wide angle lenses. The lights 112A-B and 372A-B may have a 10 dB gain and / or other dB gain. The cameras 110A-N and 370 may be set to any of a variety of exposure lengths (e.g., 0.5 s to 5.0 ms) and / or utilize bandpass filters (e.g., 600 nm bandpass filter, 630 nm bandpass filter, 800 nm bandpass filter).

[0104] Bandpass filters may also be applied to the cameras 110A-N and / or 370 to reduce different types of wavelengths captured by the cameras 110A-N and / or 370. For example, bandpass filters may be advantageous for high contrast imaging techniques, such as black and white imaging techniques. Such filters may provide illumination for the sensors of the cameras 110A-N and / or 370 to capture clearer, sharper images of the pallets on the turntable 104. Polarized optics may also be applied to the lights 112A-B and / or 372A-B, and / or the cameras 110A-N and / or 370. Polarized optics may help reduce or eliminate glare from packages on the pallets. Polarized optics may also improve the contrast of labels, such as bar codes.

[0105] Figure 3A Any one or more of the systems 300, 310, and 330 in C may be combined to perform the techniques described herein. Figure 3A 4 through 5 C depict example configurations of the picture taking compartment 126. The picture taking compartment 126 may include one or more other alternative implementations, as described throughout this disclosure.

[0106] Figure 4A Depicting a camera configuration for profiling an item. In this example, structure 400 may have cameras 110A-C aligned on horizontal bars 136A, 136B, and 136C, respectively. Two lights may be positioned horizontally below each of cameras 110A-C. In some implementations, one light may be positioned horizontally above the camera and one light may be positioned horizontally below the camera. As shown, lights 112A-B are positioned horizontally below camera 110A, lights 112C-D are positioned horizontally below camera 110B, and lights 112E-F are positioned horizontally below camera 110C. In some implementations, lights 112A-F may not be on structure 400. Alternatively, as shown in FIG. Figure 4C As depicted in FIGS. 4A to 4B , lights 112A to F may be positioned on a vertical pole structure separate from structure 400. Additionally, in some embodiments, lights 112A to F may be attached to a frame of a photographing booth structure, which may enclose the area where the tray is imaged, as described herein.

[0107] The cameras 110A-C may be configured in one or more different configurations. For example, a 65MP camera may be used. The 65MP camera may have a 65MP sensor with a resolution of 9344x7000. Using an 18mm lens, the camera may be positioned 1m away from the turntable (e.g., working distance). The camera may have a FOV of 1.661m x 1.244m, thereby capturing 2.81 pixels of the pallet label every 0.5mm. If the camera is positioned 1.5m away, the FOV may be 2.491m x 1.244m. If the camera is positioned 1.7m away, as described throughout this disclosure, the camera may have a FOV of 2.823m x.2.115m, thereby capturing 1.65 pixels of the pallet label every 0.5mm. Using a 21mm lens, the camera may be positioned 2m away from the turntable. The FOV may be 2.847m x 2.133m, thereby capturing 1.65 pixels of the pallet label every 0.5mm.

[0108] As another example, a 127MP camera may be used. A 127MP camera may have a 127.6MP sensor and a resolution of 13400x9528. Therefore, such a camera may produce high-resolution images and / or be placed at a greater working distance from the turntable than a 65MP camera. Using a 55mm lens, the camera may be positioned 3.5m away from the turntable. The camera may have a FOV of 2.941m x2.091m, thereby capturing 2.28 pixels of the product label every 0.5mm. When the camera is positioned 1.7m away, the FOV may be 1.47mx 1.05m, thereby capturing 4.56 pixels of the product label every 0.5mm. Using a 21mm lens, the camera may be positioned 1.5m away from the turntable. Therefore, the camera may have a FOV of 2.135mx 1.6m. When the camera is positioned 2m away, the FOV may be 2.847mx 2.133m. Therefore, the camera can capture 1.65 pixels of the product label every 0.5mm.

[0109] Figure 4BDescribe the camera configuration for profiling items. In this exemplary configuration of rod 108, three cameras 110A, 110B and 110C can be positioned vertically along rod 108. As described herein, cameras 110A to C can move up and down and / or left and right on rod 108. The placement of cameras 110A to C can depend on the working distance of rod 108 from the turntable described herein. The placement of cameras 110A to C can also depend on the size of the tray being imaged and / or the placement, positioning and / or distance of one or more separate lighting structures, as described throughout this disclosure. Rod 108 can also include light bars (e.g., strobe lights) 112A to B vertically along vertical rod 108. As shown, light bar 112A can be positioned between camera 110A and camera 110B. Light bar 112B can be positioned between camera 110B and camera 110C. One or more additional or fewer light bars can be positioned along vertical rod 108. In addition, as referenced Figure 4C As described in FIGS. 10A to 10E, one or more light bars may be positioned along a separate vertical rod that may be offset from cameras 110A to C and positioned a particular distance away from the cameras. Figure 4B , lights 112A-B may be positioned around cameras 110A-C to provide uniform lighting for the FOV of cameras 110A-C.

[0110] As reference Figure 3C As described, one or more of the lights 112A-F may be red lights. Additionally, as described herein, a red filter or a bandpass filter may be applied to the cameras 110A-C.

[0111] In some embodiments, one or more of the cameras 110A-C can be tilted at a specific angle. For example, one of the cameras 110A-C can be tilted at a 30 degree angle. One or more or all of the cameras 110A-C can be tilted at a 90 degree angle. In some embodiments, the cameras 110A-C can also be positioned at different heights along the vertical pole 108.

[0112] In some embodiments, the pole may have only two cameras 110A-B. The cameras 110A-B may have 65MP sensors. The cameras 110A-B may have a FOV of 1.375m x 1.22m with a 33mm lens. When the pole 108 is placed at a working distance of 1.9m from the turntable, the cameras 110A-B may have a FOV of 1.6m x 1.22m. The pallet label size from this working distance may be 2.86 pixels per 0.5mm. When the pole 108 is placed at a working distance of 1.7m from the turntable, the cameras 110A-B may have a FOV of 1.45m x 1.09m.

[0113] Figure 4Cto E depict another camera and lighting configuration for profiling an item, wherein Figure 4C Display perspective, Figure 4D Shows a front view (from the perspective of the turntable and / or tray being imaged) and Figure 4E A side view is shown. In this example, the vertical profiling structure 400 (e.g., the vertical pole 108 described throughout) includes three cameras 110A-C. The cameras 110A-C may be mounted along the horizontal mounting structures 136A-C, as shown in FIG. Figure 4A In this example, the vertical light mounting structures 410A-B (e.g., light poles 108 described throughout) may be positioned and offset at distances D3 (346) and D4 (348) from the vertical profiling structure 400, as described above with respect to Figure 3C 106. The vertical structures 410A-B may each include two light bars 112A-112B and 112C-D that may be vertically aligned along the vertical structures 410A-B. In some implementations, the light bars 112A-D may be fixed at several locations along the vertical structures 410A-B so that when a pallet is positioned on top of the turntable 104 and / or conveyor 102 / 106, the light bars span the height of the pallet. Thus, the light bars 112A-D may provide light along the entire vertical height of the tallest pallet. As an illustrative example, if the largest pallet is 49 inches tall, the vertical structures may be 48 inches to 50 inches tall so that the light bars 112A-D positioned thereon may provide uniform illumination from the top to the bottom of the pallet.

[0114] Figure 5A is a block diagram of a process 500 for profiling an item using the techniques described herein. Figure 5B 5 is a timeline 501 of process 500. One or more blocks of process 500 may be performed by computing system 152. Process 500 may also be performed by one or more other computing systems, computers, servers, remote servers, and / or computing devices. For simplicity and illustrative purposes, process 500 is described from the perspective of computing system 152.

[0115] refer to Figure 5A Continuing to process 500 in B, a tray (502) may be ingested at ingest location 504. As described throughout this disclosure, the tray may be placed on upper conveyor belt 102. Once the tray (502) is ingested, computing system 152 may start a clock or timer. The clock or timer may operate in seconds. In some embodiments, the clock or timer may operate in milliseconds.

[0116] For a predetermined length of conveyor belt 102, the pallet may be in buffer position 506. Buffer position 506 may provide a time window for one or more components described herein to reset and / or reallocate computing resources so that the components may be ready to profile the pallet.

[0117] At the alignment position 508, the pallet may be aligned or otherwise straightened on the conveyor belt 102. Thus, the pallet may enter the turntable 104 at the turntable position 510 without being angled in a particular direction. Alignment devices (e.g., with reference to FIG. 1 ) may be positioned along the side of the conveyor belt 102 at the alignment position 508. Such devices may align or straighten the pallet on the conveyor belt 102. Alignment may begin after the buffer position 506 at a time of t=10 seconds.

[0118] Once aligned, the tray can be routed onto the turntable 104. In the turntable position 510, the camera on the vertical rod 108 can capture images of the tray as it rotates. The tray can reach the turntable position at t=12 seconds. At this time, a top photo of the tray can be taken (522). As the tray rotates, additional photos of different sides of the tray can be captured. For example, at time=14, the tray can rotate 30 degrees and a first side photo of the tray can be taken (524). At t=17.6, the tray can rotate 120 degrees and a second side photo can be taken (526). At t=21.2, the tray can rotate 210 degrees and a third side photo can be taken (528). At t=24.8, the tray can rotate 300 degrees and a fourth side photo can be taken (530). By t=27 seconds, the tray can rotate a full 360 degrees (532). The captured photos may be transmitted to the computing system 152 and processed as they are received by the computing system 152 (534). In some implementations, once all images of the tray are captured (e.g., by t=27 seconds, once the tray has been fully rotated and images of each side of the tray have been captured), the images may be processed. The point at which the first photo of the tray is processed may start a timer indicating the processing time of the photo. For example, as depicted in timeline 501, the first photo may begin processing at t=12 seconds and may be completed by t=80 seconds. As another example, the last photo may begin processing at t=24.8 seconds (e.g., when the photo was captured) and may be completed by t=90 seconds. Processing of the photos may be completed individually and / or collectively in one or more different time frames based on available computing resources and processing power.

[0119] It should be appreciated that additional photos of the tray may be taken during a complete rotation. It should also be appreciated that the photos may be taken at different degree intervals. For example, the first photo may be taken at 30 degree intervals. Subsequent photos may then be captured at 90 degree offsets until the tray completes a complete rotation. As another example, each of the images may be captured at 90 degree offsets until the tray completes a complete rotation.

[0120] Still refer to Figure 5A Continuing to process 500 in B, after the photo of the pallet is captured and processed, a physical inspection and printing and applying (512) may be performed. For example, a physical inspection of the pallet may begin at t=32 seconds. The physical inspection may be performed by one or more automated warehouse devices. The devices may be configured to inspect the pallet and ensure that the pallet can be safely placed in a certain location in the warehouse. For example, the devices may inspect the pallet for damage to the wood planks and / or for tilt. Damage and / or tilt may include the ability to safely move the pallet throughout the warehouse and store the pallet therein.

[0121] In some embodiments, the physical inspection may be complete by t=39 seconds. The physical inspection may take longer or shorter time depending on the size of the pallet and / or items being inspected during the physical inspection. Additionally, at t=39 seconds, the results of processing the first photo may begin to be anticipated (536). Results may also be anticipated at any time before or after t=39 seconds.

[0122] Another tray may be ingested at approximately t=60 seconds (538). On average, one new tray may be ingested every minute (502). In process 500, one or more other time frames may be used to ingest trays and profile such trays.

[0123] The computing system 152 may also perform a print and apply process at 512. This process may occur at a later time (e.g., t=80 seconds). For example, the print and apply process may be performed only if the computing system 152 determines that a pallet label cannot be identified from the processed photos. The computing system 152 may therefore receive or determine a barcode result (540). This may include applying image processing techniques to the photos to determine whether a barcode or other unique identifier, such as a label described herein, can be identified from any of the photos.

[0124] The printing and applying process may include checking whether the label is in the correct position on the pallet, which is preferred or otherwise required for storage by the warehouse. Therefore, even if the computing system 152 can identify the label from the image, the label may still be in an undesirable location / position on the pallet. Therefore, the computing system may print a copy of the identified label and may apply the printed copy to the desired correct position on the pallet. The label may need to be in the correct position to facilitate faster and more accurate scanning of the label to identify the pallet and move the pallet around in the warehouse.

[0125] If the label is recognized and it is in the correct desired location, the computing system 152 may not have to generate, print, and apply a new label. On the other hand, if the label cannot be recognized, the computing system 152 may automatically generate a label as a placeholder. The generated label may be assigned to the pallet, printed, and then automatically affixed to the pallet in the correct desired location. The label may be affixed to the pallet by an automated warehouse device.

[0126] At t=90 seconds, the pallet may move to decision point 514. At decision point 514, computing system 152 may determine where in the warehouse the pallet may be routed. Accordingly, computing system 152 may determine a storage location for the pallet (546). Computing system 152 may determine the storage location (544) based on receiving information associated with the pallet (e.g., from a WMS) or retrieving information (e.g., from a pallet profile data storage device). The information may include owner information, item information, and code data.

[0127] At this point, the computing system 152 may determine whether the tag was accurately identified in the photo. For example, if the tag is accurately identified and is in the correct desired location or the tag is accurately identified but not in the correct desired location, the computing system 152 may still determine that the tag is accurately identified. The pallet may be routed to continue to be stored along the conveyor belt 106A (516). Therefore, the pallet may be stored (518) in an organized storage location. At this point, the computing system 152 may or may not have identified additional information about the pallet from the photo. The computing system 152 may identify additional information at any time between receiving the photo of the pallet, routing the pallet to the organized storage location, and / or once the pallet is stored at the organized storage location. Therefore, even if additional characteristics about the pallet have not yet been identified, the pallet may be routed to the organized storage location for storage (516). Therefore, warehouse efficiency may be improved and the computing system 152 may continue to identify other pallets as they are ingested (502).

[0128] An organized storage location may be selected by the computing system 152 based on the received, accessed, or retrieved pallet information (544). For example, the pallet information may indicate a specific storage location where all pallets of the same owner are to be stored. The pallet information may also indicate storage conditions that the pallet must meet (e.g., the pallet must be stored in a refrigerated area). The computing system 152 may also use the pallet information to automatically determine the optimal storage location for the pallet (e.g., height information about the pallet may limit the locations where the pallet can be stored while optimizing the available space in the warehouse).

[0129] As another example, if the label is not accurately identified (e.g., the label is on the pallet, but the label cannot be read from the photo of the pallet) or the label has been generated for the pallet as a placeholder (e.g., no photo indicates that the label is present on the pallet), then the computing system 152 may determine at decision point 514 that the pallet is rejected (520). A rejected pallet is a pallet that cannot be routed to be deposited (516) to an organized storage location. After all, it is uncertain where the pallet should be stored in the warehouse. The pallet may be routed down the lower conveyor belt 106B to a temporary storage location. As further described below, once in the temporary storage area, the pallet may be manually inspected by a warehouse worker to identify a label or other characteristic about the pallet, which may then be used to determine the organized storage location of the pallet. Once in the temporary storage area, the warehouse worker may also determine the organized storage location of the pallet based on analyzing the photos captured and previously processed by the computing system 152. In some embodiments, once in the temporary storage area, the pallet may be automatically inspected by one or more warehouse devices.

[0130] At time = 90 seconds, the second pallet captured at t = 60 seconds may complete a full rotation on turntable 104 (542). At some later time in timeline 501, computing system 152 may determine when to no longer capture images of a pallet and / or to no longer capture a pallet (548). Computing system 152 may also stop the clock or timer at this time. As described throughout this disclosure, process 500 may be performed for each pallet that enters the warehouse.

[0131] Figure 6 6 is a flow chart of a process 600 for profiling pallets entering a warehouse environment. One or more blocks in the process 600 may be performed by the computing system 152. The process 600 may also be performed by one or more other computing systems, computers, servers, remote servers, and / or computing devices. For simplicity and illustrative purposes, the process 600 is described from the perspective of the computing system 152.

[0132] Referring to process 600, at 602, a pallet can be loaded onto a conveyor belt. As described herein, the pallet can be automatically or manually moved onto the conveyor belt. The pallet can be placed onto the conveyor belt by a human worker, a forklift, or an automated device in the warehouse that is configured to automatically move the pallet around the warehouse. Then, at 604, the pallet can be automatically moved to a turntable. Automatically moving the pallet to the turntable can include re-adjusting or straightening the pallet on the conveyor belt.

[0133] Once the tray is on the turntable, the tray can be centered (606). To center the tray, the computing system 152 can receive image data of the tray as it is loaded onto the turntable. Using image analysis and processing techniques, the computing system 152 can identify the midpoint of the turntable and determine whether the tray is positioned at the midpoint or offset from the midpoint. Using machine learning and / or trained models, the computing system 152 can identify how far the tray is from the midpoint of the turntable and how much the tray needs to move to be centered at the midpoint. The computing system 152 can then instruct components of the turntable (e.g., rollers) to automatically readjust the position of the tray on the turntable.

[0134] Centering the pallet 120 at 606 facilitates ensuring that the cameras described herein capture images of the pallet from a uniform distance, angle, and FOV. Thus, the images can be more easily and accurately stitched together for use in identifying the pallet's label and profiling the pallet (e.g., identifying characteristics about the pallet, such as damage, tilt, and size).

[0135] After centering the tray, the turntable may be rotated (608). For example, the computing system 152 may instruct the turntable to rotate at a desired speed and / or direction. The speed and / or direction of rotation may be predetermined and applied to each tray ingested. In some implementations, the speed and / or direction may be adjusted and based at least in part on the configuration of the tray or other tray identification information (e.g., size, weight, etc.).

[0136] As the turntable rotates, image data (610) of the tray may be captured. In some embodiments, the turntable may rotate continuously without stopping, so that images may be captured at various predetermined points along the continuous rotation. In some embodiments, the turntable may rotate the tray (608) incrementally and stop, so that image data (610) may be captured, and then the tray (608) may be rotated again. 608 to 610 may be cycled until all sides of the tray are captured by the camera. The turntable may rotate in multiple increments, such as every 15 degrees, 45 degrees, or 90 degrees. In some embodiments, the increments of rotation may be consistent or they may be variable. Therefore, as described herein, images may be captured at predetermined rotation intervals (e.g., every 30 degrees, every 90 degrees, first 30 degrees and then every 90 degrees) until a complete rotation of the tray is completed. Images may also be captured at predetermined time intervals (e.g., every 2 seconds, every 5 seconds, etc.). The time to capture the image may also be determined based on the speed at which the tray is rotated on the turntable. Images may also be captured at consistent or variable rotation increments. For example, the turntable may initially be rotated 45 degrees, images may be captured, and then the turntable may be rotated 90 degrees. Sometimes, the rotation may be determined dynamically. For example, the increment of rotation may be determined based on an analysis of each captured image (e.g., reference 612). As an illustrative example, if a particular increment of rotation causes glare to appear in the image data, the computing system 152 may determine that the tray should be rotated in either direction (e.g., clockwise or counterclockwise) by 5 degree increments or some other nominal amount. Additional images may then be captured in 5 degree rotation increments and / or until a sufficiently good image of a particular side or portion of the tray is captured in the image data.

[0137] In some embodiments, some cameras may be configured to capture images of the pallet at a first time, and other cameras may be configured to capture image data of the pallet at a second time, where the first time and the second time are different. Images of each side of the pallet as well as the top and bottom of the pallet may be captured. Multiple images of each side of the pallet may also be captured. In some embodiments, only one image of each side of the pallet may be captured.

[0138] When the images are captured, they can be transmitted to the computing system 152 for processing. Once the tray has rotated a full 360 degrees and / or all images of the tray have been captured, the images can also be transmitted to the computing system 152 together.

[0139] At 612, the computing system 152 may identify the pallet information from the image. Image processing techniques may be used to identify or otherwise detect information about the pallet. For example, optical character recognition (OCR) techniques may be used to identify a label or other unique identifier of the pallet. Once the label is identified, the computing system 152 may determine other information about the pallet, such as owner information, the contents of the pallet, the storage location of the pallet, storage requirements, etc. Barcode detection techniques may also be used to identify the pallet. Machine learning algorithms, methods, and trained models may also be used to identify information about the pallet. As described throughout this disclosure, at 612, the computing system 152 may identify at least the label of the pallet. Once identified, the label may be used to determine the storage location of the pallet. The computing system 152 may identify additional information about the pallet at the same time and / or later.

[0140] In 614, the computing system 152 may access pre-identified information about the pallet. This information may be retrieved from a manifest filled out by a shipper or other relevant stakeholder. For example, this information may be included in the manifest of a truck that recently arrived at the warehouse. In some embodiments, this information may be received from a WMS and / or a data storage device, as described throughout this disclosure. The pre-identified pallet information may include information provided to the warehouse before the pallet arrives / is delivered to the warehouse. For example, the information may be received from a supplier, a customer, a shipper, or other relevant stakeholder in the supply chain. In some embodiments, the pre-identified pallet information may be specific to the particular pallet being identified. In some embodiments, the pre-identified pallet information may be universal and applicable to a variety of different pallets. A variety of different pallets may include some of the same features. For example, a pallet may contain different types of products, but the pallet may be associated with the same customer code. Therefore, the pre-identified pallet information may include a customer code.

[0141] The computing system 152 may then compare the identified pallet information to the pre-identified pallet information (616). Thus, the computing system 152 may use the pre-identified information to filter out possible identifications or other information about the pallet. The computing system 152 may match the tag identified for the pallet in 612 with the pre-identified pallet information (which may include metadata). By matching the information, the computing system 152 may link or otherwise associate the pallet with a pallet profile that is stored and used to identify and move pallets in the warehouse. The computing system 152 may optionally determine whether the pre-identified pallet information is missing any of the information identified in 612. If so, the computing system 152 may update the pre-identified pallet information accordingly. Comparing the information may be beneficial in efficiently generating a more accurate, up-to-date profile of the pallet without the need for human intervention or manual inspection.

[0142] At 618, the computing system 152 may determine a match between the pallet information and the pre-identified information. This may include verifying one or more pieces of information about the pallet. For example, using a machine learning trained model, the computing system 152 may verify that the contents of the pallet are what the pre-identified pallet information says. The computing system 152 may also verify information such as an owner, storage condition, and / or storage location associated with the pallet.

[0143] In some implementations, the computing system 152 may also determine the accuracy of the information identified in 612. Upon matching the information, the computing system 152 may determine whether the level of accuracy in identifying the pallet information exceeds a threshold (e.g., 90% accuracy). For example, in the event that the computing system 152 is unable to identify the label of the pallet from the image, the level of accuracy may be 0% or a value less than a predetermined threshold. The computing system 152 may also use the level of accuracy to determine whether the pallet may be routed to an organized storage location or a temporary storage location. If the pallet information is accurate above a predetermined threshold, the pallet may be automatically moved to an organized storage location. On the other hand, if the pallet information accuracy does not exceed a predetermined threshold (e.g., the label cannot be identified from the image data), the pallet may be routed to a temporary storage location for further inspection and review.

[0144] In addition, in some embodiments, if the accuracy is low or the tray cannot be matched with the pre-identified tray information, the computing system 152 may optionally instruct the turntable to rotate the tray a second time. The turntable may rotate in the opposite direction and / or at different speeds. The turntable may also rotate in the same direction and / or at the same speed. One or more different images of the tray may be captured. Blocks 612 to 618 may be repeated based on the newly captured image data. In some embodiments, in 612, the tray information may be identified by combining and / or comparing the image data received from the first profiling process with the image data received from the second profiling process. If after the second profiling process, the tray information accuracy exceeds a threshold, the tray may be moved to the lower conveyor belt to reach an organized storage location identified by the tray information. If after the second profiling process, the tray information accuracy does not exceed a threshold, the tray may be routed to the lower conveyor belt to reach a temporary storage location.

[0145] Figure 7 1 is a flow chart of a process 700 for identifying pallet information from image data of a pallet. One or more blocks in process 700 may be performed by computing system 152. Process 700 may also be performed by one or more other computing systems, computers, servers, remote servers, and / or computing devices. For simplicity and illustrative purposes, process 700 is described from the perspective of computing system 152.

[0146] Referring to process 700, at 702, computing system 152 may receive image data of a pallet. In some embodiments, process 700 may be performed during process 600, while an image of the pallet is being captured and transmitted to computing system 152 as it is captured. In some embodiments, such as referring to Figure 5A As described in B, process 700 may be performed during different stages of profiling the pallet (e.g., once all images are captured and transmitted to computing system 152, during physical inspection and printing and applying 512, decision point 514, between time t=12 seconds and t=90 seconds, etc.).

[0147] The computing system 152 may retrieve an image-based model of the pallet (704). As described herein, the image-based model may be stored in a pallet profile data storage device. The model may also be stored in one or more other data storage devices, databases, cloud servers, or other remote data storage devices and / or servers. The model may be trained using machine learning techniques. The model may be trained to recognize different characteristics of the pallet from the image data. The model may be generated using different training data sets. The model may also be trained and / or improved using image data and analysis performed in real time to profile different pallets.

[0148] At 706, the computing system 152 may apply one or more of the models to the image data of the pallet. By applying the models to the image data of the pallet, the computing system 152 may determine information about the pallet at 708. For example, a label of the pallet may be identified (710). The label may be identified using a machine learning trained model and / or optical character recognition (OCR) technology. OCR technology may also be used to detect any text that may appear on the pallet. The text and / or label may be further processed (e.g., using a machine learning trained model) to identify additional information about the pallet, such as the pallet name, owner information, and an organized storage location of the pallet.

[0149] At 712, the size of the pallet may also be determined. Because the image data is captured from a uniform offset degree (e.g., 90 degrees), height, and working distance, the computing system 152 may more efficiently and accurately identify information such as the height, width, length, and / or depth of the pallet. For example, a machine learning model may be trained to distinguish and identify pixels that indicate background from pixels that indicate a pallet. The computing system 152 may use known distance information (e.g., working distance) and pixels that indicate a pallet to infer the size of the pallet. For example, the height may be inferred by triangulating pixels from multiple images, where those pixels represent the sides and top of the pallet.

[0150] In 714, the computing system 152 may also identify damage to the pallet. Machine learning may be used to train a model to identify damage. For example, the model may receive images of pallets with damage and images of pallets without damage. Machine learning techniques and algorithms may be used on the images to determine the probability that a particular feature in the test image data set indicates damage. This trained model may then be applied to the image data received in 702 to identify whether damage is present in any of the image data of the pallet.

[0151] Similarly, the tilt of the pallet may be identified at 716. The tilt of the pallet may be identified using a model that is trained using machine learning techniques as described with reference to a model for detecting damage.

[0152] The contents of the tray may be identified 718. This may include retrieving or otherwise accessing pre-identified information about the tray that may indicate what the contents of the particular tray should be. Machine learning techniques may also be used to determine the contents of the tray.

[0153] In addition, the computing system 152 may determine the number of rows on the pallet at 720 and the number of boxes (e.g., cases) on the pallet at 722. As described herein, a machine learning trained model may be used. For example, the model may be used to count the number of rows on the pallet and the total number of boxes on the pallet.

[0154] Any of 712 to 722 may be performed in combination. In some examples, only one or less than all of 712 to 722 may be performed. Furthermore, 712 to 722 may be performed in any order.

[0155] Still referring to process 700, computing system 152 may store the determined information about the pallet (724). The information may be stored in a profile associated with the pallet. The association may be made based on an identified tag or other unique identifier of the pallet. Figure 5A As mentioned in B, if the label of a pallet cannot be identified, a placeholder label can be generated for the pallet. The placeholder label can then be linked or associated with a new profile for the pallet, which can be populated with pallet information automatically determined by a computing system and / or manually determined by a warehouse worker.

[0156] The determined pallet information may also be sent to a WMS described herein (726). The WMS may accordingly update the pallet profile with the information determined by the computing system 152 in 708 through 722. The WMS may also use the determined pallet information to move a pallet in a warehouse or make other modifications to the movement, storage, or other aspects of the supply chain associated with the pallet. In some embodiments, the computing system 152 may update the pallet profile information and then transmit the updated pallet profile to the WMS.

[0157] Figure 8 1 is a swim lane diagram of a process 800 for profiling and storing pallets in a warehouse environment. The process 800 is described with reference to the system 100 in FIG1 . One or more blocks of the process 800 may be performed by the computing system 152. One or more blocks of the process 800 may also be performed by other computing systems, servers, and / or computing devices (e.g., WMS 154).

[0158] Referring to process 800, at 802, the computing system 152 may receive image data of a pallet, as described throughout this disclosure. Then, at 804, the computing system 152 may perform a profile analysis on the pallet (e.g., referring to Figures 6 to 7 Based on the pallet profile analysis, computing system 152 may determine whether the pallet label is identifiable (806). For example, computing system 152 may determine that the label cannot be clearly identified from the image data. As another example, computing system 152 may determine that the label can be clearly identified.

[0159] If the tag is identifiable in 806, then in 808, the computing system 152 may transmit a notification to the WMS 154 to determine the storage location of the pallet. The WMS 154 may use the techniques described herein to determine the storage location. For example, the WMS 154 may access a data storage device to determine where the identified pallet may be stored (e.g., based on a customer code, an identified owner, etc.). The WMS 154 may generate control instructions to automatically move the pallet to the identified storage location. In some embodiments, the computing system 152 may determine the storage location of the pallet. In addition, the computing system 152 and / or the WMS 154 may determine control instructions that can be used to automatically route the pallet along one or more conveyor belts in the warehouse to the determined storage location.

[0160] If the tag is not recognizable in 806, computing system 152 may transmit a notification that further identification is required to WMS 154 (810). As described herein, additional identification may be required because the tag of the pallet is not recognizable via image analysis and processing in 804. For example, WMS 154 may identify the pallet using stored information, pre-identified pallet information, manifests, and / or information from relevant stakeholders in the supply chain.

[0161] Based on the notification in 810, WMS154 may also generate control instructions that can be used to automatically route the pallet to a temporary storage location along one or more conveyor belts in the warehouse. The pallet may be kept in the temporary storage area until it can be identified by WMS154 and / or by a warehouse worker. For example, as described herein, computing system 152 and / or WMS154 may provide image data of the pallet to a user device (e.g., user device 156) of a warehouse worker. The user device may output the image data on a graphical user interface (GUI) display. The warehouse worker may then review the image data and / or manually inspect the pallet in the temporary storage location to identify one or more features or information about the pallet.

[0162] In some embodiments and as referenced Figure 5A As described in B, when the pallet cannot be identified in 806, the computing system 152 and / or WMS154 may generate a placeholder tag for the pallet. In 810, the placeholder tag may be transmitted to the WMS154 and / or the device of the warehouse worker. The warehouse worker may use the placeholder tag to locate the pallet in a temporary storage location. The warehouse worker may then review the pallet, identify information about the pallet based on manual inspection, and provide the identified information as input to the user device. The identified information may then be stored by the computing system 152 and / or WMS154 in association with the placeholder tag. Thus, the placeholder tag may become a unique identifier for the pallet in the warehouse. The information collected or determined about the pallet may be linked or otherwise associated to the placeholder tag.

[0163] As described throughout this disclosure, process 800 may be repeated for each pallet that enters the warehouse and / or each pallet that is ingested and profiled. Furthermore, in some embodiments, process 800 may not be completed for a first pallet before process 800 for a second pallet begins (e.g., see FIG. 800 ). Figure 5B). Starting the process 800 for the second pallet while the process 800 for the first pallet is still being executed can be beneficial to maximizing throughput and maintaining warehouse efficiency. As an example, during the process 800, a warehouse worker at the user device 156 can be identifying the first pallet while the computing system 152 can be performing a profile analysis on the second pallet in 804. As described above, performing the process 800 on multiple pallets simultaneously can be beneficial to improving warehouse efficiency and moving the pallets to their designated organized storage locations to avoid bottlenecks or other obstructions in the warehouse.

[0164] Fig. 9 1 is a swim lane diagram of a process 900 for determining a storage location for a profiled pallet. The process 900 is described with reference to the system 100 in FIG1 . One or more blocks of the process 900 may be performed by the computing system 152, the WMS 154, and the user device 156. One or more blocks of the process 900 may be performed by other computing systems, servers, and / or computing devices.

[0165] Referring to process 900, at 902, the computing system 152 may receive image data of a pallet, as described throughout this disclosure. The computing system 152 may then use the image data to profile the pallet (904) (e.g., referring to Figures 6 to 8 Once the profile analysis is performed, the computing system 152 may determine whether the pallet can be accurately identified (906). As described herein, at least, the computing system 152 may determine whether the tag on the pallet is recognized. The computing system 152 may also determine whether any identifying information about the pallet that can be used to store the pallet can be recognized.

[0166] If the pallet can be accurately identified, information about the pallet identified during the profiling may be transmitted to the WMS 154 (908). For example, the computing system 152 may determine that the image data includes a clear capture of the label. Using image processing techniques and analysis, the computing system 152 may thus detect the label and determine information about the pallet based on the label, such as the owner and storage requirements / information. The information transmitted to the WMS 154 may include the image data, the clear capture of the label, and the information identified from the computing system 152 analyzing the label in the image data. In some implementations, the computing system 152 may only determine that the image data includes a clear capture of the label, but the computing system 152 may not perform image analysis techniques on the clear capture of the label. Instead, the computing system 152 may transmit the clear capture of the label to the WMS 154 and the WMS 154 may perform image analysis techniques on the clear capture to identify information about the pallet.

[0167] In 910, WMS154 may receive pallet information. WMS154 may determine an optimal storage location for the pallet based on the pallet information (912). The pallet information may include an owner, an item type / name, and a tag. WMS154 may also determine an optimal organized storage location based on other information. As an example, computing system 152 may identify a pallet by its owner by analyzing a clear capture of a tag. The owner name / identifier may be passed to WMS154. WMS154 may look up the owner in a data storage device that may store information about warehouse customers. The stored information about the owner may indicate where the owner's pallets are stored and / or what storage conditions must be met. Using this information, WMS154 may determine to which organized storage location the pallet should be routed.

[0168] Referring back to 906, if the pallet cannot be accurately identified, the computing system 152 may identify the pallet as unidentifiable (914). The computing system 152 may mark the image data associated with the pallet. As described herein, the computing system 152 may mark the pallet as unidentifiable when the computing system 152 cannot identify any labels on the pallet and / or the pallet has a label but the image data does not include a clear capture of the label that can be used for accurate image analysis. When the pallet is labeled, the computing system 152 may also indicate whether the pallet has a label. This indication may make it easier and faster for warehouse workers to determine what they need to look for during manual inspection of the pallet.

[0169] When the pallet cannot be identified by the computing system 152, in 916, the computing system 152 may transmit the pallet image data to the WMS 154. In 918, the WMS 154 may receive the pallet image data. In 920, the WMS 154 may route the pallet to a temporary storage location. For example, the computing system 152 may instruct the WMS 154 to route the pallet to a temporary storage area. In some embodiments, the WMS 154 may determine that because the pallet is marked as unidentified, the pallet may be automatically routed to a temporary storage location. In some embodiments, the computing system 152 may instruct the pallet to be routed to a temporary storage location. Once in the temporary storage area, the pallet may be placed in a queue for review by a warehouse worker.

[0170] The WMS 154 may then transmit the pallet image data to the user device 156 (922). At 924, the user device 156 may receive the pallet image data. At 926, the user device 156 may then output the image data. At 926, the user device 156 may also receive user input identifying the pallet. In some embodiments, as described throughout, a warehouse worker may manually inspect and analyze the image data at the user device 156 (e.g., referring to FIG. 1 ). Figure 8). In addition, the warehouse worker may manually inspect the pallet and provide input to the user device 156 regarding one or more features that the warehouse worker identified on the pallet. For example, if the label in the image is blurry, the warehouse worker may view the actual pallet and identify the barcode (e.g., by scanning the label, taking another photo of the label, and / or entering the label information into the user device 156).

[0171] At 928, the user device 156 may then transmit the user identification of the pallet to the WMS 154. Using the user identification of the pallet, the WMS 154 may determine a storage location (e.g., an organized storage location) for the pallet (928), as described throughout this disclosure. Furthermore, as described throughout, the WMS 154 and / or the computing system 152 may then execute instructions that cause the pallet to be moved to the determined organized storage location.

[0172] Fig.10 is a system diagram depicting system components that may be configured to perform any of the techniques described herein. The system may include a computing system 152, a WMS 154, a user device 156, and a pallet profile data storage device 158 in data communication via a network 150. In some implementations, one or more of such system components may be one computer, computing system, server, remote server, server node, and / or device. For example, the computing system 152 and the WMS 154 may be part of the same computing system.

[0173] The computing system 152 may be configured to profile the pallets as they enter the warehouse, as described throughout this disclosure. The computing system 152 may include a pallet profiling controller 1002, an image analysis module 1004, a pallet route determiner 1006, a pallet match determiner 1008, a pallet profile verification module 1010, a communication interface 1012, a pallet profile analyzer 1014, and a storage location determination engine 1074. In some embodiments, the computing system 152 may be a combination of one or more other components and / or systems, such as the WMS 154, a warehouse control system (WCS) 1080, and / or a cloud computing system 1082. In some embodiments, one or more of the components of the computing system 152 may be part of other systems such as the WMS 154, the WCS 1080, and / or the cloud computing system 1082. For example, the pallet profiling controller 1002 may be part of the WCS 1080. The WCS 1080 may be configured to control physical components in the warehouse, such as conveyor belts, cameras, and turntables, and determine which components in the warehouse perform specific functions. As another example, the pallet profiler 1014 may be part of the cloud computing system 1082. The cloud computing system 1082 may be configured to perform processing of the image data to determine information about the scanned pallet.

[0174] Referring to the computing system 152, the pallet profiling controller 1002 may be configured to control the operation of components for routing pallets and capturing images of pallets. The controller 1002 may include one or more cameras 1016 (e.g., cameras 110A-N described throughout), a controller 1018, a turntable 1020, one or more conveyor belts 1024 (e.g., upper conveyor belt 102 and lower conveyor belt 106 described throughout), one or more lights 1026 (e.g., lights 112A-N described throughout), a communication interface 1028, a pallet centering module 1030, and a turntable rotator module 1032. The communication interface 1028 may provide the controller 1002 with communication with one or more of the systems, devices, and components described herein. The pallet profiling controller 1002 may be used to control the cameras 1016, the turntable 1020, the conveyor belts 1024, and the lights 1026.

[0175] Instructions for controlling these components may be executed by the controller 1018 of the pallet profiling controller 1002. For example, the instructions executed by the controller 1018 may cause automatic rotation of the turntable 1020, turn lights 1026 on / off, slow down / speed up the conveyor belt 1024, and prompt the camera 1016 to capture images of the pallet as it rotates on the turntable 1020. As an illustrative example, the controller 1002 may selectively turn one or more of the cameras 1016 on or off. This determination may be based on whether features of the pallet can be clearly identified from one or more images of the pallet. As another illustrative example, the controller 1002 may turn off the overhead lights 1026 and / or increase the brightness of the lights 1026 that illuminate the sides of the pallet instead of the top of the pallet. The controller 1002 may also turn on and off a red light that may or may not be used for each pallet being profiled (e.g., a red light may be used for pallets that are wrapped in cellophane or another material that may cause glare in the camera 1016 lens). As yet another illustrative example, controller 1002 may control conveyor belt 1024 to route pallets to an organized storage location or a temporary storage location.

[0176] Still referring to the controller 1002, the pallet centering module 1030 can be configured to center the pallet at the midpoint of the turntable 1020, as described throughout this disclosure. The pallet centering module 1030 can also be configured to instruct the alignment device of the belt 1024 to align and adjust the pallet when the pallet is automatically moved to the turntable 1020 on the upper conveyor belt 1024.

[0177] As described herein, the turntable rotator module 1032 can be configured to determine the degree of rotation, speed of rotation, and / or direction of rotation of the turntable 1020. The module 1032 can determine when to begin the rotation of the turntable 1020. For example, the module 1032 can determine that the turntable 1020 can remain stationary until a tray is detected to be centered on and / or otherwise located on the turntable 1020. One or more sensors positioned on the surface of the turntable 1020 can sense the weight of the tray to identify when the tray is located on the turntable 1020. In some implementations, the module 1032 can determine that the turntable 1020 can continue to rotate regardless of whether the tray is located on the turntable 1020. The module 1032 can send instructions to the controller 1018 to adjust the movement / rotation of the turntable 1020.

[0178] The module 1032 may also determine the degree of rotation based at least in part on the size of the pallet and any pre-identified information about the pallet. For example, the module 1032 may determine that the turntable 1020 should be rotated at a smaller degree of rotation (e.g., a slower rotation) so that the camera 1016 can more accurately capture images of the entire surface area of ​​a larger pallet. For example, for a pallet with a smaller surface area, a larger degree of rotation (e.g., a faster rotation) may be used.

[0179] Module 1032 can adjust the rotation speed of turntable 1020 based at least in part on receiving an indication of the weight of the pallet, the size of the pallet, and pre-identified information from one or more sensors. For example, a larger pallet may require turntable 1020 to move at a slower speed than a pallet that is half the size of the large pallet because there is more surface area to be captured by camera 1016. In some embodiments, rotating the pallet at a slower speed can be beneficial to capture more pallet images that can be stitched together. When camera 1016 is a low-resolution camera, a slower speed can be beneficial to ensure that the image is not blurred. When camera 1016 is a high-resolution camera, module 1032 can determine the faster speed of rotating turntable 1020. After all, despite the faster speed, the higher-resolution camera 1016 can still capture sharper images. Faster rotation speeds can also be beneficial to increase efficiency and reduce the time required when performing profile analysis on pallets entering the warehouse.

[0180] The image analysis module 1004 may be configured to receive image data captured by the camera 1016. The module 1004 may perform image analysis techniques on this image data to identify information about the pallet. As described herein, during the profiling process, the module 1004 may detect at least a product label or other unique identifier from the image data. The module 1004 may be trained using one or more machine learning models to identify different types of labels from the image data. The module 1004 may also be configured to perform object detection techniques and optical character recognition (OCR) to not only detect the label but also determine the information associated with the label. Using similar techniques, the module 1004 may determine the text on the pallet and any other identifying characteristics about the pallet. For example, as described herein, the module 1004 may use a machine learning trained model to detect different features of the pallet from the image data. One or more of such models may be used to detect damage to the contents, tilt, height, weight, size, contents, damage to the pallet, temperature of the contents, text, etc. The information identified by the image analysis module 1004 may be transmitted to the pallet profile data storage device 158 and stored in a profile associated with the pallet, as further described below.

[0181] The pallet route determiner 1006 may determine where the pallet may be moved after the image is captured. The determiner 1006 may receive the identified information from the image analysis module 1004. Using the information, the determiner 1006 may determine whether the pallet should be moved to an organized storage location or a temporary storage area. If the pallet should be moved to an organized storage area, the determiner 1006 may determine a specific location in the organized storage area based on the information from the image analysis module 1004. The determiner 1006 may also access pre-identified information about the pallet from the pallet profile data storage device 158 to determine where to store the pallet in the organized storage area. Once the determiner 1006 identifies where to store the pallet, the determiner 1006 may generate instructions for sending the pallet along a specific route. The instructions may then be transmitted to the pallet profile analysis controller 1002 and executed by the controller 1018.

[0182] The pallet match determiner 1008 may be configured to match the information identified by the image analysis module 1004 with pre-identified information associated with the pallet. For example, the determiner 1008 may receive information indicating characteristics about the pallet from a supplier before the pallet arrives at the warehouse. The determiner 1008 may associate this information with the pallet profile in the pallet profile data storage device 158. The determiner 1008 may also associate this information with the information determined by the image analysis module 1004.

[0183] In some embodiments, the determiner 1008 may use the information identified by the module 1004 to identify a profile for a pallet already stored in the pallet profile data storage device 158. Once the profile is identified, the determiner 1008 may update any information in the profile with the information identified by the image analysis module 1004.

[0184] In some embodiments, the determiner 1008 may identify that there is currently no profile for the tray or that a profile cannot be matched to the tray. In such cases, the determiner 1008 may generate a profile for the tray and store the identified information in the profile (e.g., with reference to the placeholder tags described throughout this disclosure, FIGS. 5 to 8 ).

[0185] The pallet profile verification module 1010 may be configured to determine the accuracy of the information identified by the image analysis module 1004. The pallet profile verification module 1010 may receive pre-identified pallet information (e.g., received from an owner, supplier, or other relevant stakeholder in the supply chain), information identified by a warehouse worker during a manual inspection of the pallet, and information identified by the image analysis module 1004. Using this information, the module 1010 may determine whether accurate / up-to-date information is stored in a profile associated with the pallet. The module 1010 may compare the information and determine and assign an accuracy confidence value to the information. If the pallet profile contains information whose accuracy confidence value is below a predetermined threshold (e.g., the supplier information indicates no damage, but the image analysis module 1004 identifies damage), the module 1010 may flag the pallet for review by a warehouse worker.

[0186] Communications interface 1012 may be configured to provide communications between computing system 152 and other systems, devices, and components via network 150 .

[0187] Still referring to the computing system 152, the pallet profile analyzer 1014 can be configured to determine one or more characteristics about the pallet that can affect the storage of the pallet. The pallet profile analyzer 1014 can use image data captured by the camera 1016 to make such determinations. In addition, the pallet profile analyzer 1014 can use a machine learning trained model to make such determinations, as described throughout this disclosure. The analyzer 1014 can include a damage detector 1034, a configuration detector 1036, and a content detector 1038.

[0188] The damage detector 1034 may be configured to identify any damage indications from the image data using the techniques, methods, and processes described throughout this disclosure. The detector 1034 may also be configured to identify any gaps and / or voids where boxes, containers, or other items are expected to be located on the pallet. Such gaps / voids may indicate missing and / or damaged items. During transportation to the warehouse, damage may occur to the pallet or its contents. The pallet itself may be damaged (e.g., broken decks, twisted wooden frames, etc.). One or more items stacked on the pallet may also be damaged. The detector 1034 may be trained to identify features in the image data that indicate different types of damage. Such detection techniques may be beneficial in reducing human error, because it is challenging and / or time-consuming for a person to accurately identify all types of damage that may occur to the pallet and / or its contents.

[0189] The configuration detector 1036 may be configured to determine configuration information about the pallet from the image data and / or information determined by the image analysis module 1004. The detector 1036 may determine the size (e.g., length, height, and width) of the pallet using the techniques described herein. The detector 1036 may also be trained to determine the tilt of the pallet, the number of layers or rows of items on the pallet, and the number of items on the pallet. The detector 1036 may apply a machine learning trained model to make each of these determinations.

[0190] The content detector 1038 may be configured to determine content information about the pallet from the image data and / or information determined by the image analysis module 1004. In some embodiments, the detector 1038 may determine how many boxes or items are on the pallet (e.g., box count). The detector 1038 may make this determination based at least in part on information identified by the damage detector 1034 and / or the configuration detector 1036. In some embodiments, the content generator 1038 may determine what kind of items / objects are stored on the pallet. The generator 1038 may also use the identified information from the module 1004 and the pre-identified information about the pallet to confirm, verify or otherwise determine what kind of items are on the pallet and / or what kind of items should be on the pallet. In some embodiments, the content generator 1038 may also identify and translate text, barcodes, QR codes, and / or other unique identifiers associated with the pallet and / or individual items on the pallet.

[0191] The storage location determination engine 1074 may be configured to determine where to store the pallets in the warehouse. The engine 1074 may make this determination automatically using the techniques described herein. The determined storage location may then be transmitted to the WMS 154. In some embodiments, the computing system 152 may not have a storage location determination engine 1074. Instead, the storage location may be determined by the WMS 154, as described below.

[0192] The pallet information determined and / or identified by the computing system 152 may then be stored in the pallet profile data storage device 158. The data storage device 158 may include data records for each of the pallets 1040A-N that enter the warehouse.

[0193] Example pallet profiles 1040A-N may include image data 1042, damage 1044, box count 1046, skew 1048, size 1050, layers 1052, content 1054, text 1056, and labels 1058. Each of the information 1042-1058 may be determined by one or more components of the computing system 152. One or more of the information 1042-1058 may also be determined by a warehouse worker at a user device 156, the WMS 154, and / or any other relevant stakeholder in the supply chain (e.g., supplier, owner, customer, etc.).

[0194] In some embodiments, the computing system 152 may store less than all images captured by the camera 1016 in the pallet profiles 1040A to N. For example, an image that clearly captures the label of the pallet may be stored in the image data 1042. Other images of the pallet may be discarded. Sometimes, images of the pallet may be captured by one or more stakeholders in the supply chain. Such images may be stored in the image data 1042. The image analysis module 1004 of the computing system 152 may then access such images and perform image analysis techniques to identify information about the pallet. When the label of the pallet cannot be clearly identified from the image data 1042, all captured images may be stored in the image data 1042 so that the warehouse worker can analyze the image to identify information about the pallet. If the worker can accurately identify information about the pallet from the image, one or more of the images may be saved in the image data 1042 and other images (e.g., blurred images) may be discarded. In some embodiments, if the worker cannot identify information from the image but can identify information through manual inspection of the pallet, all images may be discarded from the image data 1042.

[0195] The WMS 154 may include a storage location determiner 1060 and a communication interface 1062. The WMS 154 may be configured to manage information about the warehouse and warehouse components and their operations. In some embodiments, as described above, the WMS 154 may be part of the computing system 152, the WCS 1080, and / or the cloud computing system 1082. In some embodiments, the WMS 154 may include the WCS 1080, the cloud computing system 1082, and / or an automation application programming interface (API) 1061. The communication interface 1062 may be configured to provide communication between the components described herein.

[0196] The storage location determiner 1060 may be configured to identify a storage location for a pallet. As described throughout this disclosure, the determiner 1060 may identify whether the pallet may be stored in a temporary storage location or in an organized storage location. The determiner 1060 may also identify which storage location in the organized storage location to store the pallet. In some embodiments, the determiner 1060 may also generate instructions for routing the pallet to the identified storage location. The determiner 1060 may communicate the identified storage location and / or instructions to the pallet profiling controller 1002 for execution by the controller 1018.

[0197] The determiner 1060 may receive pallet information 1064A-N that may be used to determine an optimal storage location for a pallet. The received pallet information 1064A-N may include a storage location 1066 , a tag 1068 , and an owner 1070 .

[0198] The pallet profiles 1040A-N may sometimes indicate where pallets of a particular type and / or owner should be stored (e.g., storage location 1066). The determiner 1060 may use this information to designate or assign a particular organized storage location in the warehouse to the pallet. The determiner 1060 may also update the storage location 1066 to the organized storage location selected for the pallet. In addition, the tag 1068 and / or owner 1070 may include requirements or other specifications for storing a particular pallet. This information may also be used to select a particular organized storage location in the warehouse for the pallet.

[0199] The user device 156 may be used by a warehouse worker to review the identified information about the pallet and / or identify information about the pallet. The user device 156 may be a computing system, computing device, computer, laptop, tablet computer, smart phone, mobile phone, or other similar mobile device. Using the user device 156, a user may view information about the pallet identified by the computing system 152. Using the user device 156, a warehouse worker may, for example, verify the information identified by the computing system 152. The warehouse worker may also match the information identified by the computing system 152 with information stored in the pallet profiles 1040A to N. The warehouse worker may identify information that the computing system 152 did not identify (e.g., the computing system 152 omitted or otherwise did not attempt to identify damage or box counts) and / or information that the computing system 152 could not identify (e.g., the computing system 152 attempted but failed to identify the label of the pallet because the image data was blurred).

[0200] The user device 156 may include a tray image data analysis interface 1072 and a communication interface 1076. The communication interface 1076 may provide for communications between any of the components described herein.

[0201] The pallet image data analysis interface 1072 may be configured to present image data captured by the camera 1016. The interface 1072 may also present information about the pallet identified by the computing system 152 and / or any known information about the pallet (e.g., information stored in the pallet profile data storage device 158 and / or determined by the WMS 154, WCS 1080, and / or cloud computing system 1082). A warehouse worker may review the image data and attempt to identify information about the pallet from the data. For example, a warehouse worker may provide input to the interface 1072 that includes visual observations made by the warehouse worker while manually surveying the pallet and / or by viewing the image data presented in the interface 1072.

[0202] In some embodiments, the interface 1072 may include an identification or flag on the image data to help warehouse workers more efficiently inspect the pallet (e.g., via manual inspection of the pallet). For example, the interface 1072 may present a flag on a blurred image of the side of the pallet where the label is located. The warehouse worker can then more quickly determine which side of the pallet to manually inspect based on the marked image.

[0203] Fig.11A B is a swim lane diagram of an exemplary process 1100 for profiling a tray. Reference is made to the systems described throughout this document (e.g., the system 100 described above with respect to FIG. 1 and the system 100 described above with respect to FIG. 1). Fig.10 Process 1100 is described with reference to system 1000 described herein. One or more blocks of process 1100 may be performed by computing system 152, WMS 154, and pallet profiling controller 1002. One or more blocks of process 1100 may be performed by other computing systems, servers, and / or computing devices, such as those described throughout this document.

[0204] refer to Fig.11A, the WMS 154 directs profiling of a pallet (e.g., a pallet received at the warehouse and not yet profiled) (1102). To do so, the WMS 154 may direct the pallet profiling controller 1002 to control a conveyor to transport the pallet to the pallet profiler (1104), which may include centering the pallet on the conveyor, moving the pallet to a turntable, and centering the pallet on the turntable in preparation for scanning. The WMS 154 may obtain a transaction ID (1106) for the pallet generated (1108) by the computing system 152, which the computing system 152 may use to temporarily identify the pallet and information determined for the pallet, which the WMS 154 may use to ultimately identify and profile the pallet. The WMS 154 may retrieve candidate pallet information (1110), such as a pallet manifest from an inbound truck that has been unloaded at the warehouse. The information provided on the manifest for a pallet may vary significantly across different providers and carriers, from simply listing the number of pallets provided for individual SKUs to more detailed information regarding a unique identifier associated with each pallet, identification of specific items contained on the pallet (e.g., a list of the number of boxes for each SKU contained in the pallet), date information for the contents of the pallet (e.g., the date of each box, the date of the batch of boxes contained in the pallet), and / or other details. Pallet profiling may attempt to pair individual pallets with pre-existing information for the pallet (e.g., manifest information) to supplement or further enrich the pre-existing information (e.g., fill in unknown details such as expiration information for boxes contained in the pallet), and correct any inaccurate or otherwise incorrect information contained in the pre-existing pallet information.

[0205] The WMS 145 may initiate a tray scan of the tray (1112), which may include instructing the tray profiling controller 1002 to initiate the scan. The tray profiling controller 1002 may control the turntable to position and orient the tray for the first scan (1114). As discussed throughout this document, this may include centering the tray on the turntable 104 and positioning the tray so that a first side of the tray is perpendicular to the orientation of the cameras 110A-D attached to the vertical rod 108. Other initial orientations are also possible, such as tilting the first side from vertical by a certain degree of rotation, such as 15°, 20°, 30°, 45°, 60°, 85°, and / or other degrees of rotation. As the tray is positioned, the controller 1002 may control the operation of the lights and cameras (1116), such as sending coordinated and synchronized control instructions to the lights and cameras to initiate the capture of a first set of images. For example, a first set of images may include multiple images of the front of the pallet captured by cameras 110A-D and one or more images of the top surface of the pallet captured by overhead camera 110N. The images may be captured by the cameras and received by computing system 152 (1118), which may then determine whether more scans are needed (1120). The determination regarding more scans may be a static determination, which may involve advancing through a static number of pallet positions and image set captures (e.g., one set of images for each of the four sides of the pallet and one set of images for the top of the pallet). For a static determination, the determination may involve simply determining whether each of the predetermined pallet positions has been imaged (e.g., one set of images for each side, one set of images for each 45 degree rotation of the pallet), and if not all positions have been imaged, the next position in the process may be directed to controller 1002, which may then cause steps 1114-1120 to be repeated.

[0206] In some examples, the determination (1120) regarding more scans may be dynamic, which may involve, for example, performing an initial and quick assessment of the most recently captured image to determine whether it is useful for image analysis. In the event that the image does not have at least a threshold quality according to one or more image quality metrics, the computing system 152 may direct one or more modifications to re-image the portion of the tray, such as adjusting the lighting configuration (e.g., strobe light duration, light intensity, light wavelength, position and / or angle of the light), camera settings (e.g., adjusting aperture, shutter speed, focus, camera position), and / or tray position (e.g., rotating forward or backward by small increments, such as 5°, 10°, 15°, and / or other amounts). Such modifications may be transmitted to the controller 1002, which may prompt the performance of steps 1114 through 1120 with the modified settings. Further modifications may be performed, up to a threshold number (e.g., 2, 3, 4, or more), after which the best set of images for a portion of the tray may be selected and the process 1100 may continue to image other portions of the tray.

[0207] Once all images of the pallet have been captured (no more scans are performed at step 1120), the controller 1002 may be notified that the scan is complete, which may prompt the controller 1002 to control the turntable and conveyor to further move the pallet (1122), such as transferring the pallet from the turntable 104 to the conveyor 106 and moving the pallet out of the photo booth 126. Once the scan is complete, the computing system 152 may also initialize first and second timers for performing specific image processing operations on the captured images, which may affect the physical movement, handling, and labeling of the pallet. Instead of simply performing these operations until they are completed, the timers may be set to require execution within a threshold amount of time to ensure that the physical handling of the pallet is not delayed by the image processing operations (i.e., the physical advancement of the pallet is suspended while waiting for the image processing to complete), which may increase the throughput of the pallet through the profiler and system. A first image processing operation (1126) may be initiated by the computing system 152, which may correspond to the first timer. The first image processing operation may be a first pass through the captured image of the pallet to quickly determine whether a pallet identifier (e.g., a barcode, QR code, alphanumeric code) of the pallet can be detected from the image. This pallet identifier can be used to identify the corresponding pallet information from the pre-existing pallet information. If the pallet identifier is detected (and / or detected with at least a threshold confidence level) before the first timer expires, the first image processing operation may end and the result (e.g., the pallet identifier) ​​may be reported to the WMS 154 (1128). If the first timer expires before the first image processing operation is completed, a result indicating that the pallet identifier was not detected and / or one or more candidate pallet identifiers with a threshold confidence level may be returned. The first timer may be a static value (e.g., 0.5 seconds, 1 second, 5 seconds, 10 seconds, 20 seconds, and / or other) and / or a dynamic value (e.g., the time from the completion of the scan until the pallet passes / arrives at one or more physical target locations (e.g., completely exits the photo booth)). After completing the first image processing operation, the computing system 152 may initiate a second image processing operation (1130).

[0208] The WMS 154 may use the results to identify a corresponding candidate pallet from pre-existing candidate pallets (1132). The identifier of the candidate pallet may be used as, associated with, and / or used to generate a unique warehouse identifier for the pallet within the warehouse, which the WMS 154 may use to track, manage, and / or process warehousing operations regarding the pallet. The unique warehouse identifier may be associated with various details about the pallet, such as the contents of the pallet (e.g., product sku, number of boxes per sku, date per box, size of the pallet, weight of the pallet), which may be used to select a long-term storage location for the pallet and to select a specific pallet to fulfill an order. If an identifier is not identified and / or a candidate pallet is not identified, a new unique warehouse identifier may be generated for the pallet.

[0209] The WMS 154 may direct a physical inspection of the pallet (1134), which may be performed by, for example, one or more devices configured to perform a safety inspection of the pallet (e.g., determine size, detect tilt of the pallet). Examples are described above with respect to 512 and 536. The controller 1002 may receive instructions from the WMS 154 and may control the movement of the pallet via a conveyor and / or other pallet movement components to the physical inspection unit (1136).

[0210] refer to Fig. 11B , the controller returns (1138) physical inspection information (e.g., height, width, safety information) to the WMS 154, which adds the physical inspection information to the pallet's profile (1140). The WMS 154 directs labeling of the pallet (1142), and the controller 1002 can control a conveyor to move the pallet to an automatic labeler (1144), which can apply a label using the pallet's unique warehouse identifier. The controller 1002 can verify that the label applied to the pallet is readable and located in one or more specific locations on the pallet (1146), and can return a confirmation to the WMS 154, which can add the label information and the confirmation to the pallet's profile (1148).

[0211] Upon expiration of the second timer or completion of the second image processing operation, whichever occurs first, the computing system 152 may return the results of the second image processing operation to the WMS 154 (1150), which may add those results to the pallet profile (1152). For example, the second image processing operation may include identifying details of the contents and boxes contained on the pallet, such as text, markings, codes, and / or other labels on the boxes. The second image processing operation may be intended to specifically identify the contents contained in the pallet, such as the specific sku contained in the pallet, the number of boxes per sku, date information associated with each box, and / or other significant information to better understand and track the contents of the goods contained in the warehouse. The second image processing operation may process all captured images of the pallet a second time, but using a different image processing technique, such as using an OCR and / or machine learning model, and / or other text / object detection algorithm. The second timer may be a static value (e.g., 0.5 seconds, 1 second, 5 seconds, 10 seconds, 20 seconds, and / or other) that is started simultaneously with the first timer or after the first operation / timer is completed. Alternatively, the second timer may be a dynamic value (eg, the time until confirmation of labeling of the pallet is complete).

[0212] The WMS 154 may analyze the accumulated pallet information and determine which information to retain and use for the pallet for any conflicting or inconsistent information (e.g., a conflict between pre-existing pallet information from a manifest and the detected pallet information). In some instances, the detected information may be selected instead of the pre-existing information. In other instances, the conflict may result in the pallet being marked for manual review and placed in a temporary storage area. The WMS 154 may use the pallet information to determine whether to route the pallet to a long-term storage area and / or a temporary storage area (1154). If the accumulated pallet information does not contain enough details and / or confidence associated therewith to determine the contents of the pallet, the pallet may be marked for temporary storage. The WMS 154 may direct a storage determination (1156), which the controller 1002 may use to control the conveyor to route the pallet along a path to a long-term storage area or a temporary storage area (1158).

[0213] After completing the second image processing operation, the computing system 152 may initiate a third image processing operation (1152), which may perform additional processing of the image to determine and / or detect additional details about the pallet that were cut off via the timers for the first and second image processing operations, and / or perform additional / alternative image processing (i.e., process the image using one or more machine learning models to more accurately and / or consistently detect the contents of the pallet, perform image manipulation / modification operations to improve image quality, which may then be used to reprocess the first and / or second image processing operations). The third image processing operation may not be time-limited, and may be permitted to process until it is concluded. The third image processing result (1160) may be returned and added to the pallet's profile (1162). In instances where the pallet is routed to a temporary storage area, the pallet may be added to a queue of accumulated pallet profiles containing captured images to undergo manual review (1164), which may result in manual identification and / or correction of the pallet information (1166) and may permit a long-term storage location to be determined and used to store the pallet (1168).

[0214] Fig.12 1 is a block diagram of computing devices 1200, 1250 that can be used to implement the systems and methods described in this document as a client or as a server or multiple servers. Computing device 1200 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Computing device 1250 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, and other similar computing devices. The components shown here, their connections and relationships, and their functions are intended to be exemplary only, and are not intended to limit the implementation of the inventions described and / or claimed in this document.

[0215] The computing device 1200 includes a processor 1202, a memory 1204, a storage device 1206, a high-speed interface 1208 connected to the memory 1204 and a high-speed expansion port 1210, and a low-speed interface 1212 connected to a low-speed bus 1214 and the storage device 1206. Each of the components 1202, 1204, 1206, 1208, 1210, and 1212 are interconnected using various buses and may be mounted on a common motherboard or otherwise as appropriate. The processor 1202 may process instructions for execution within the computing device 1200, including instructions stored in the memory 1204 or on the storage device 1206 to display graphical information for a GUI on an external input / output device such as a display 1216 coupled to the high-speed interface 1208. In other implementations, multiple processors and / or multiple buses may be used as appropriate along with multiple memories and types of memories. Furthermore, multiple computing devices 1200 may be connected, with each device providing portions of the necessary operations (eg, as a server bank, a group of blade servers, or a multi-processor system).

[0216] Memory 1204 stores information within computing device 1200. In one implementation, memory 1204 is a computer-readable medium. In another implementation, memory 1204 is one or more volatile memory units. In another implementation, memory 1204 is one or more non-volatile memory units.

[0217] The storage device 1206 is capable of providing mass storage for the computing device 1200. In one embodiment, the storage device 1206 is a computer-readable medium. In various embodiments, the storage device 1206 may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid-state memory device or device array, including devices in a storage area network or other configuration. In one embodiment, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods such as those described above. The information carrier is a computer-readable medium or a machine-readable medium, such as the memory 1204, the storage device 1206, or a memory on the processor 1202.

[0218] The high-speed controller 1208 manages bandwidth-intensive operations of the computing device 1200, while the low-speed controller 1212 manages less bandwidth-intensive operations. This distribution of work is exemplary only. In one embodiment, the high-speed controller 1208 is coupled to the memory 1204, the display 1216 (e.g., through a graphics processor or accelerator), and to the high-speed expansion port 1210 that can accept various expansion cards (not shown). In an embodiment, the low-speed controller 1212 is coupled to the storage device 1206 and the low-speed expansion port 1214. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), may be coupled to one or more input / output devices, such as a keyboard, a pointing device, a scanner, or a networking device (e.g., a switch or router), for example, through a network adapter.

[0219] As shown in the figure, computing device 1200 can be implemented in a number of different forms. For example, the computing device can be implemented as a standard server 1220 or multiple times in a group of such servers. The computing device can also be implemented as part of a rack-mounted server system 1224. In addition, the computing device can be implemented in a personal computer such as a laptop computer 1222. Alternatively, components from computing device 1200 can be combined with other components in a mobile device (not shown), such as device 1250. Each of such devices can contain one or more of computing devices 1200, 1250, and the entire system can be composed of multiple computing devices 1200, 1250 communicating with each other.

[0220] Computing device 1250 includes a processor 1252, a memory 1264, an input / output device (e.g., a display 1254), a communication interface 1266, and a transceiver 1268, among other components. Device 1250 may also be provided with a storage device (e.g., a microdrive) or other device to provide additional storage. Each of components 1250, 1252, 1264, 1254, 1266, and 1268 are interconnected using various buses, and several of the components may be mounted on a common motherboard or otherwise appropriately mounted.

[0221] The processor 1252 can process instructions for execution within the computing device 850, including instructions stored in the memory 1264. The processor can also include separate analog and digital processors. For example, the processor can provide cooperation with other components of the device 1250, such as control of a user interface, applications run by the device 1250, and wireless communications performed by the device 1250.

[0222] The processor 1252 may communicate with a user through a control interface 1258 and a display interface 1256 coupled to the display 1254. For example, the display 1254 may be a TFT LCD display or an OLED display, or other suitable display technology. The display interface 1256 may include appropriate circuits for driving the display 1254 to present graphics and other information to the user. The control interface 1258 may receive commands from the user and convert them for submission to the processor 1252. In addition, an external interface 1262 may be provided to communicate with the processor 1252 so as to enable near-area communication of the device 1250 with other devices. For example, the external interface 1262 may provide wired communication (e.g., via a docking program) or wireless communication (e.g., via Bluetooth or other such technologies).

[0223] Memory 1264 stores information within computing device 1250. In one embodiment, memory 1264 is a computer-readable medium. In one embodiment, memory 1264 is one or more volatile memory units. In another embodiment, memory 1264 is one or more non-volatile memory units. Expansion memory 1274 may also be provided and connected to device 1250 via expansion interface 1272, which may include a SIMM card interface, for example. This expansion memory 1274 may provide additional storage space for device 1250, or may also store applications or other information for device 1250. Specifically, expansion memory 1274 may include instructions to perform or supplement the processes described above, and may also include security information. Thus, for example, expansion memory 1274 may be provided as a security module for device 1250, and may be programmed with instructions that permit secure use of device 1250. In addition, security applications may be provided via a SIMM card along with additional information, such as placing identification information on a SIMM card in an unbreakable manner.

[0224] For example, the memory may include flash memory and / or MRAM memory as discussed below. In one embodiment, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods such as those described above. The information carrier is a computer-readable medium or a machine-readable medium, such as the memory 1264, the expansion memory 1274, or a memory on the processor 1252.

[0225] Device 1250 may communicate wirelessly through a communication interface 1266, which may include digital signal processing circuitry, if necessary. Communication interface 1266 may provide for communication under various modes or protocols, such as GSM voice calls, SMS, EMS or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. For example, such communication may occur through a radio frequency transceiver 1268. Additionally, short-range communications may occur, for example, using Bluetooth, WiFi, or other such transceivers (not shown). Additionally, GPS receiver module 1270 may provide additional wireless data to device 1250, which may be used as appropriate by applications running on device 1250.

[0226] Device 1250 may also communicate audibly using audio codec 1260, which may receive verbal information from a user and convert it into usable digital information. Audio codec 1260 may likewise produce audible sound for the user, such as through a speaker in, for example, an earpiece of device 1250. Such sound may include sound from a voice phone call, may include recorded sound (e.g., voice messages, music files, etc.), and may also include sound produced by applications operating on device 1250.

[0227] As shown in the figure, the computing device 1250 can be implemented in a number of different forms. For example, the computing device can be implemented as a cellular telephone 1280. The computing device can also be implemented as part of a smart phone 1282, a personal digital assistant, or other similar mobile device.

[0228] Various implementations of the systems and techniques described herein may be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that may be executed and / or interpreted on a programmable system that includes at least one programmable processor that may be special or general purpose (coupled to receive data and instructions from a storage system and to transmit data and instructions to a storage system), at least one input device, and at least one output device.

[0229] These computer programs (also referred to as programs, software, software applications or code) contain machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages ​​and / or in assembly / machine language. As used herein, the terms "machine-readable medium", "computer-readable medium" refer to any computer program product, apparatus and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor that includes a machine-readable medium that receives machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0230] To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including auditory input, voice input, or tactile input.

[0231] The systems and techniques described herein may be implemented in a computing system that includes a back-end component (e.g., as a data server); or includes a middleware component (e.g., an application server); or includes a front-end component (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back-end, middleware, or front-end components. The components of the system may be interconnected by any digital data communication form or medium, such as a communication network. Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

[0232] A computing system may include clients and servers. A client and server are generally remote from each other and generally interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0233] Although this document contains many specific implementation details, these details should not be interpreted as limitations on the scope of what can be claimed but should be interpreted as descriptions of features that can be specific to a specific embodiment of a specific disclosed technology. The specific features described in this specification in the context of a separate embodiment may also be implemented in a single embodiment in part or in whole in combination. Conversely, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although features may be described herein as acting in a specific combination and / or initially claimed, in some cases, one or more features from the combination may be removed from the claimed combination, and the claimed combination may be directed to a sub-combination or a variation of a sub-combination. Similarly, although operations are described in a specific order, this should not be understood as requiring such operations to be performed in a specific order or in a sequential order or performing all operations to achieve the desired result. Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims.

Claims

1. A system for profiling a pallet, the system comprising: a turntable configured to rotate the tray; a light source positioned adjacent the turntable, the light source configured to illuminate the tray as the tray rotates on the turntable; a camera positioned relative to the light source, the camera being configured to capture an image of the tray as the tray rotates on the turntable; and A picture taking booth encloses an area containing the turntable, the light source, and the camera, wherein the picture taking booth includes sides and a top to reduce the amount of ambient light entering the enclosed area.

2. The system of claim 1 , wherein the system further comprises a vertical profiling structure located in a fixed position inside the photo booth near a side of the turntable, wherein the vertical profiling structure comprises the light source and the camera.

3. The system of claim 1 , wherein the system comprises a plurality of cameras positioned at a plurality of different locations proximate to the turntable, wherein the plurality of cameras comprises the camera and is configured to capture images of the pallet from a plurality of different views based on the plurality of cameras positioned at the plurality of different locations.

4. The system of claim 1, wherein the system further comprises a computing system configured to profile the pallet, the computing system performing operations comprising: receiving image data of the captured image of the pallet from the camera; and A machine learning model is applied to the image data to identify damage to the pallet, wherein the model is trained using test images of pallets with damage and test images of pallets without damage to determine a probability that a feature in the test image indicates damage to the pallet.

5. The system of claim 4, wherein the damage comprises at least one of a broken deck of the pallet, a twisted frame of the pallet, or damage to one or more items on the pallet.

6. The system of claim 4, wherein the computing system is further configured to perform operations comprising: applying a machine learning model to the image data to identify the tilt of the pallet, wherein the model is trained using test images of pallets having different amounts of tilt to determine probabilities that features in the test images indicate different amounts of tilt.

7. The system of claim 1, wherein the light source is red light.

8. The system of claim 1, wherein the light source includes a polarizing filter that causes the camera to capture light having a wavelength between 620 nm and 650 nm.

9. The system of claim 1, wherein the light source is configured to emit red light having a wavelength between 610 nm and 750 nm, orange light having a wavelength between 595 nm and 610 nm, and yellow light having a wavelength between 580 nm and 595 nm.

10. The system of claim 1, wherein the light source is configured to emit infrared light having a wavelength above 750 nm.

11. The system of claim 1, wherein the light source is configured to emit ultraviolet light having a wavelength between 100 nm and 400 nm.

12. A system for profiling a pallet, the system comprising: conveyors for automatically rotating pallets to image and move them throughout the facility; a light source positioned adjacent the conveyor, the light source being configured to illuminate the pallet as the pallet is automatically rotated or moved by the conveyor; a camera positioned relative to the light source, the camera being configured to capture an image of the pallet as the pallet is automatically rotated or moved by the conveyor belt; and A photo booth encloses an area containing a portion of the conveyor, the light source, and the camera, wherein the photo booth includes sides and a top to reduce the amount of ambient light entering the enclosed area.

13. The system of claim 12, wherein the system further comprises a computing system, the operations performed by the computing system comprising: receiving image data of the captured image of the pallet from the camera; and A machine learning model is applied to the image data to identify damage to the pallet, wherein the model is trained using test images of pallets with damage and test images of pallets without damage to determine a probability that a feature in the test image indicates damage to the pallet.

14. The system of claim 12, wherein the light source is configured to emit red light having a wavelength between 610 nm and 750 nm, orange light having a wavelength between 595 nm and 610 nm, and yellow light having a wavelength between 580 nm and 595 nm.

15. A system for profiling a pallet, the system comprising: A light-controlled environment encloses an area, wherein the light-controlled environment includes: a conveyor configured to automatically move a pallet within the light-controlled environment to view the pallet from a plurality of different angles for imaging; a light source positioned adjacent the conveyor, the light source configured to illuminate the pallet as the pallet is moved by the conveyor; and A camera is positioned relative to the light source, the camera being configured to capture images of the pallet as the pallet is moved by the conveyor.

16. The system of claim 15, wherein the light-controlled environment comprises a photo booth.

17. The system of claim 15, wherein the camera is offset from and adjacent to the light source.

18. The system of claim 15, wherein the light-controlled environment comprises a room in a facility.

19. The system of claim 15, wherein the system further comprises a computing system, the operations performed by the computing system comprising: receiving image data of the captured image of the pallet from the camera; and A machine learning model is applied to the image data to identify damage to the pallet, wherein the model is trained using test images of pallets with damage and test images of pallets without damage to determine a probability that a feature in the test image indicates damage to the pallet.

20. The system of claim 15, wherein the light source is configured to emit red light having a wavelength between 610 nm and 750 nm, orange light having a wavelength between 595 nm and 610 nm, and yellow light having a wavelength between 580 nm and 595 nm.

Images