Warehouse management system and method for operating same

Abstract

A warehouse management system and a method of using such a system are disclosed. A data collection device including a lidar and a visual image capture device is attached to a vehicle, such as a forklift. A data collection device collects lidar and visual image data while moving around a warehouse having storage locations therein, such as shelves. Lidar and visual image data are processed, producing processed data in the form of a digital twin representing a storage area and accessible using a computer device to provide information about storage locations as well as information about products stored on the shelf. In particular, the information includes product information, a visual image, a volume percentage of the stored location fill, and a percentage of product presence compared to a previous amount.

Description

Technical Field

[0001] The present invention relates to a warehouse management system and method of operating such a system, and in particular but not limited to a system which collects data using the movement of a vehicle (eg, a forklift) around a warehouse. Background Art

[0002] As part of logistics operations, warehouse management systems are often used to track the contents of a warehouse and perform inventory checks on that content. Such systems are important for tracking the inventory being stored in a warehouse. This is not only to ensure that incoming and outgoing goods are recorded, but also to ensure that goods are not misplaced within the warehouse. However, such systems rarely provide up-to-date information. In addition, in warehouses where large quantities of identical goods arrive and are stored together, but are picked out for separate distribution, recording individual goods is not easily verified because they are picked and sorted from those larger quantities. If a pallet of goods containing dozens or hundreds of a particular item arrives, traditional warehouse management systems rely on whether the number of items on that pallet has been accurately entered into the system. This in turn relies on the goods supplier accurately representing the number of items on the pallet. Inaccuracies in this number can cause difficulties for the warehouse management system, resulting in delays in the distribution of goods and pallets becoming empty before the management system expects.

[0003] Preferred embodiments of the present invention seek to overcome or alleviate the above-identified shortcomings of the prior art. Summary of the invention

[0004] According to one aspect of the present invention, there is provided a method for using a warehouse management system, comprising:

[0005] providing a data collection device including a laser radar and a visual image capture device;

[0006] moving the data collection device around a storage area having a plurality of storage locations therein and collecting lidar and visual image data while moving;

[0007] Processing the data to generate processed data in the form of a digital twin, which represents the storage area and can be accessed using a computer device to provide information about the storage location and information about the products stored in the storage location and the storage area, wherein the information about the storage location includes multiple product information, visual images, the volume percentage of the stored location filled, and the percentage of product present compared to a previous quantity.

[0008] By collecting lidar and visual image data while moving around a warehouse and processing that data to generate a digital twin model representing the warehouse, the advantage of maintaining up-to-date, nearly real-time information related to the warehouse in a warehouse management system is provided. In particular, the combined data is able to produce useful analytics beyond a standard warehouse management system that indicates which items are stored in which locations. For example, data related to the percentage of products still present at a location can be compared to the expected quantity based on the quantity that has arrived and the quantity that has been instructed to be picked from the storage location. In addition, the efficient and effective use of the warehouse can be monitored, including ensuring that the different volumes of space in different storage locations are effectively utilized (large items are stored in large spaces and small items are stored in small spaces). Similarly, if the same products are stored in different locations, it can be ensured that the picking of those items effectively utilizes the warehouse space (two pallets that are more than half full may reflect inefficient use of warehouse space).

[0009] By operating this approach across multiple devices working in parallel in the warehouse, it is possible to cover the entire warehouse faster and perform operations in a distributed manner across the entire fleet. This ensures that the digital twin is in its most up-to-date form. A digital twin platform can aggregate data and consolidate information from multiple sources onto a single platform, making it available to the entire company, not just a single warehouse.

[0010] In a preferred embodiment, the data collection device is attached to a warehouse vehicle and is moved by the movement of the vehicle.

[0011] Use warehouse vehicles, such as forklifts, to take advantage of the movement of equipment around the warehouse.

[0012] In another preferred embodiment, the vision capture device comprises a 3D camera.

[0013] The method may further include scanning the product prior to placing the product in the storage location to determine a volume occupied by the product.

[0014] By scanning a pallet of product when it arrives at the warehouse, this improves the accuracy of determining metrics such as the percentage volume of product stored versus the original volume. Since product is picked from the front of a pallet and this is the side scanned by the device as it passes along the warehouse aisles, between the shelves of the storage locations, this means that oddly shaped pallets (those that don't stack neatly as a cuboid) can still be factored into the percentage figure for product stored.

[0015] The method may also include transmitting at least some of the above-mentioned process data to another warehouse management system.

[0016] According to another aspect of the present invention, there is provided a warehouse management system, comprising:

[0017] at least one data collection device including a lidar and visual image capture device movable about a storage area having a plurality of storage locations therein for collecting lidar and visual image data while moving;

[0018] At least one processor for processing the data to generate processed data in the form of a digital twin, which represents the storage area and can be accessed using a computer device to provide information about the storage location and information about the products stored in the storage location and the storage area, wherein the information about the storage location includes multiple product information, visual images, the volume percentage of the stored location filled, and the percentage of product present compared to a previous quantity.

[0019] In a preferred embodiment, the data collection device is attached to a warehouse vehicle and is moved by the movement of the vehicle.

[0020] In another preferred embodiment, the vision capture device comprises a 3D camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Preferred embodiments of the present invention will now be described by way of example only and not in any limiting sense with reference to the accompanying drawings, in which:

[0022] Figure 1 is a schematic representation of the system of the present invention;

[0023] Figure 2 is a digital representation of a storage area displayed on a computer device using the system of the present invention;

[0024] Figure 3 is an image of a visual representation of processed data collected using the system of the present invention;

[0025] Figure 4 is an alternative visual representation of processed data collected using the system of the present invention; and

[0026] Figure 5 and Figure 6 yes Figure 3 A close-up portion of a visual representation displayed on a computer device. DETAILED DESCRIPTION

[0027] First refer to Figure 1, a data collection device 10 is provided to collect data associated with a storage area, such as a warehouse 12, which includes a plurality (typically a majority) of storage locations 14 on shelves 16 of storage racks 18. Ideally (although not necessarily), the data collection device is a single unit that includes at least one lidar device and at least one visual camera. The data collection device 10 may also include other sensors, including but not limited to radio frequency identification (RFID) readers, motion sensors (such as accelerometers), and other environmental monitoring sensors that measure parameters, including but not limited to temperature, humidity, light intensity, and noise levels. Sensors may also be provided to measure any different gas levels, such as volatile organic compounds (VOCs), particularly where chemicals are stored, and to measure other air quality or pollution factors. Radio frequency levels may also be measured around the warehouse, particularly in the 2.4 GHz to 5 GHz range.

[0028] The data collection device 10 is mounted on equipment that enables the device 10 to be moved around the warehouse 12. Figure 1 In the example shown, the data collection device 10 is mounted on a forklift 20. Alternatively, the data collection device may be mounted on the warehouse and moved around the warehouse by other equipment, including but not limited to other vehicles, autonomous and controlled robots, drones, and mounted on backpacks or other items that may be carried by people walking around the warehouse. However, the data collection device is preferably mounted on a vehicle or the like that is already operating within the warehouse 12 so as not to increase traffic or create obstacles in the aisles between the shelves 18.

[0029] The data collection device 10 also includes a processor and a transmitter that enable the device to process data directly (indicated by arrow 24) or indirectly (indicated by arrows 26 and 28) via a server memory (schematically indicated at 30) and send processed data and raw data to the computer device 22.

[0030] The operation of the system of the present invention will now be described with additional reference to the remaining figures. A forklift 22 is mounted with a data collection device 10 and the vehicle operates in a normal manner, moving around a warehouse 12, loading items or products onto pallets and placing them in storage locations 14 on shelves 16 of racks 18 or on the floor below the racks 18. As the forklift 20 moves around the warehouse, the data collection device 10 collects lidar and visual image data. The data collection device 10 can be programmed to collect data continuously, or alternatively, can collect data periodically when not moving and continuously collect data when the vehicle 20 is in motion (determined by an accelerometer located in the device 10). For the lidar and visual image data being collected, the data collection device 10 is preferably capable of collecting data in all directions around 360°.

[0031] The lidar data represents the distance from the data collection device 10 to an object within the field of view of the device. The data is used to create a 3D point cloud representation of the warehouse 12 and its contents, including the shelves 18 and the goods or products stored in multiple storage locations 14. The device also collects visual image data. Any form of such visual image data is suitable, which can include simple standard images taken from one or more directions of the device 10. It can also include 360° images, which are most preferably taken using a 3D vision camera or stereo vision system in addition to the lidar and monocular camera modules. This is particularly useful because 3D images can be used in conjunction with lidar data to help interpret those data, thereby producing the most accurate representation of the structure within the warehouse. In this context, the term structure is used to include non-movable or nearly non-movable aspects of the warehouse, such as exterior walls, floors, ceilings, accessories such as light fixtures and walkways, as well as shelves 18. The data is also used to account for non-permanent fixtures, such as products stored on shelf 16 of shelf 18 in storage location 14, as well as objects of no interest as part of a warehouse management system, such as movable and mobile things within warehouse 12 (e.g., other warehouse vehicles and people moving around the warehouse).

[0032] Visual data is also used to identify items or products stored on the shelves 18 of the storage location 14. For example, individual products can be provided with visually readable data, such as bar codes, QR codes, readable alphanumeric codes, and readable words. Such readable codes and materials can be identified on individual items or associated with a group of items by being attached to the pallet where the item is located. Alternatively, product identity data can be stored on an RFID tag and read by an RFID reader installed on the device 10. In this way, the processing data can identify one or more items stored in a specific location and allocate those items to that location within the warehouse management system.

[0033] A particularly important function of the present invention is to provide continuous monitoring of the contents of a warehouse. In large warehouses, it is common for multiple forklifts to operate. If each of those trucks 20 is provided with a data collection device 10, and each of those devices is continuously collecting data as the forklift moves, then a large amount of data related to the warehouse can be generated. Therefore, the warehouse management system can be provided with data that accurately represents the contents of the warehouse, which has real-time or up-to-date or at least very close to up-to-date accuracy. For example, when a forklift receives a new pallet of products when it arrives inside the warehouse, the forklift will transport the pallet to one of the storage locations. When the forklift approaches the pallet, the camera in the data collection device 10 generates image data related to the pallet, and these images can be interpreted to determine the products located on the pallet using bar codes, etc., as described above. The laser radar of the data collection device 10 can also generate data related to the volume of the pallet and the products on it. This is more preferably achieved by a scanning process that can involve the forklift traversing a circular path around the pallet, or by some other pallet scanning process. However, even without a formal scanning process, important volume information can be inferred from the laser radar data collected when the forklift approaches the pallet. This volume information can be used to determine the most appropriate place to position the pallet to efficiently utilize space within the warehouse. For example, some storage locations allow larger volumes to be stored, while other storage locations are only suitable for products with a smaller height on the pallet.

[0034] Once the pallet is in place, warehouse staff will typically pick the goods on the pallet for redistribution. As the forklift 20 continues to operate around the warehouse and continues to use its lidar and visual image data collection devices, it can be determined that the volume of products located on the pallet is decreasing. This in turn allows the utilization rate of the product to be estimated, so that the time when the pallet will be depleted of all inventory can be estimated. This data can also be used to cross-reference with other aspects of the warehouse management system or another warehouse management system, which instructs the picker to the pallet on which it is located, which is usually provided with a known number of items. This information is usually provided to the warehouse management system when the products arrive from the supplier. However, if the quantity data is incorrect, this may cause significant difficulties in the warehouse because the warehouse management system is instructing the picker to retrieve the product from the location where the product does not exist. By maintaining a real-time or near real-time estimate of the volume of products located at any storage location, it can be cross-referenced with existing data to ensure accuracy in the event that the product quantity is entered inaccurately, or to identify suspected theft of items. When the system utilizes multiple devices working in parallel on multiple vehicles (or drones or carried on backpacks), the entire warehouse can be monitored quickly by performing operations in a distributed manner across the fleet of vehicles already operating in the warehouse. The data therefore represents a digital twin model (i.e. a digitally accessible version of the warehouse) that is used to monitor the goods within the warehouse. The digital twin platform aggregates data and integrates information from multiple sources onto a single platform, making it available to the entire company, not just a single warehouse. In addition to mapping the location of items within the warehouse, other data collected, such as temperature, humidity, light intensity, noise levels, and other environmental variables (such as pollution and gas levels), can also be mapped onto the digital twin. These additional map layers added to the digital twin support the construction of a more accurate picture of the warehouse environment, ensuring that optimal or necessary storage conditions are maintained throughout the warehouse, thereby building a more complete picture than using a series of static sensors. It also helps to strictly control energy use in the warehouse and can maintain a continuous, real-time audit of warehouse conditions. Monitoring and scanning of wireless radio frequencies can identify communication black spots and ensure that items such as handheld scanners are always working throughout the warehouse.

[0035] As part of a warehouse management system, displays are generated on computer equipment that provide easily accessible information related to each storage location within the warehouse. Examples of these displays are Figure 4 , Figure 5 and Figure 6 The data includes, but is not limited to, a plurality of product information, visual images, the volume percentage of the stored location filled, and the percentage of product present compared to the previous quantity. Figure 4A simplified representation of a storage rack is shown, wherein the status of the storage locations is indicated by contrasting colors. Typically, the colors red and green are used as follows: red indicates an occupied storage location (containing goods), and green indicates an empty storage location. Figure 4 In the black and white image, the lighter grey shades are red occupied locations and the darker grey shades are green empty locations. This visual representation is accompanied by numerical data, including the number of occupied locations as a percentage of the total number of locations and the percentage of empty space. Figure 5 and Figure 6 In the example, images are generated from 3D point cloud data that has been collected by the LiDAR device and combined with visual image data and interpreted data to produce a more realistic representation of the storage location. Figure 3 Shows a data overview of a single shelf. Figure 2 An overview of multiple racks that make up all or part of a warehouse is shown. The initial interaction with the warehouse management system begins with an overview of the warehouse, such as Figure 2 As shown, selecting the shelf of interest will produce Figure 3 Use standard image navigation techniques to create images such as Figure 5 and Figure 6 The image shown, and upon selection of a storage area, as visually represented, data 32 is generated as displayed, which includes an indication at 34 as to whether the storage location 14 is occupied or empty ("used" equals occupied). Displayed at 36 is the storage location code used by warehouse personnel. Also displayed is the boundary information (i.e., the size of the space) of the storage location, as indicated at 38, and the occupancy percentage, as indicated at 40. With particular reference to Figure 6 , it can be seen that when the image is zoomed in closer, it is possible to view the representation of the 3D point cloud data as a visualization of the remaining product on the pallet, such as the storage location indicated at 42. In addition, an example of a visual image of a particular storage location of interest that has been extracted from the collected visual image data can also be displayed. If more information is needed, a history of visual images can be displayed and linked together to generate a time-lapse video of that storage location.

[0036] Those skilled in the art will appreciate that the above embodiments have been described by way of example only and not in any limiting sense, and that various changes and modifications are possible without departing from the scope of protection defined by the appended claims. The processing of the data collected by the data collection device may be performed on the data collection device before transmission, or may be performed on a server or a dedicated computer after transmission, or both.

Claims

1. A method of using a warehouse management system, comprising: providing a data collection device including a laser radar and a visual image capture device; moving the data collection device around a storage area having a plurality of storage locations therein and collecting lidar and visual image data while moving; Processing the data to generate processed data in the form of a digital twin, the processed data representing the storage area and being accessible using a computer device to provide information about the storage locations and about the products stored in the storage locations and the storage area, wherein the information for the storage locations includes multiple product information, visual images, the volume percentage of the stored locations filled, and the percentage of product present compared to a previous quantity.

2. The method according to claim 1, wherein: The data collection device is attached to a warehouse vehicle and is moved by movement of the vehicle.

3. The method according to claim 1 or 2, wherein: The visual capture device includes a 3D camera.

4. The method of any preceding claim, further comprising scanning the product prior to placing the product in the storage location to determine the volume occupied by the product.

5. A method according to any preceding claim, further comprising transmitting at least some of the process data to another warehouse management system.

6. A warehouse management system comprising: at least one data collection device including being movable about a storage area having a plurality of storage locations therein for collecting lidar and visual image data while moving; At least one processor for processing the data to generate processed data in the form of a digital twin, the processed data representing the storage area and being accessible using a computer device to provide information about the storage locations and information about the products stored in the storage locations and the storage area, wherein the information for the storage locations includes multiple product information, visual images, the volume percentage of the stored locations filled, and the percentage of product present compared to a previous quantity.

7. The warehouse management system according to claim 6, wherein: The data collection device is attached to a warehouse vehicle and is moved by movement of the vehicle.

8. The warehouse management system according to claim 6 or 7, wherein: The visual capture device includes a 3D camera.

Images