Multi-source fusion unmanned warehouse management system
By using cameras and lidar in the warehouse to collect data, combining cloud processing and AI technology, a three-dimensional model is established, and the cargo information is automatically identified and recorded, the high cost problems caused by the complex equipment in the existing technology are solved, and efficient and accurate Warehousing management.
Patent Information
- Application Number
- CN202411087353.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-29
AI Technical Summary
The existing warehousing management technology has high costs due to the complex front-end equipment, which has affected the development of the warehousing industry.
Adopt a multi-source fusion unmanned warehousing management system, using cameras and lidar to collect real-time color and distance data, combine cloud processing and AI technology to establish a three-dimensional model, automatically identify and record the types and quantities of goods, and realize fully automated management.
It greatly reduces the front-end cost of warehousing management, improves management efficiency, and improves the accuracy of goods identification, realizing automated management.
Smart Images

Figure CN119047972A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of warehouse management, and in particular to a multi-source integrated unmanned warehouse management system. Background Art
[0002] Warehousing is an important way to store goods. Storing goods in a warehouse not only ensures the safety of the goods, but also facilitates the centralized and efficient management of the goods. Usually, there are many types and quantities of goods stored in a warehouse. The premise for achieving efficient warehouse management is to understand the warehouse entry and exit of goods, including the time, type and quantity of goods entering / exiting the warehouse, as well as the personnel who perform the entry / exit operations. In traditional management methods, the acquisition and management of this information are all done manually. With the development of modern technology, manual management can no longer meet the increasingly high usage requirements in terms of efficiency and accuracy.
[0003] In order to improve the efficiency of warehouse management, many automated management methods have been proposed. These methods are mainly based on computer vision or radar. For example, CN118154105A discloses an intelligent warehouse management system. After the patent establishes a 3D (three-dimensional) model based on the images collected by the image acquisition unit, the goods can be viewed in the 3D model. However, the patent requires personnel to wear VR (virtual reality) glasses and move in the warehouse, which is not only costly but also unable to achieve truly automated management. CN118037176A discloses an automated unmanned warehouse system and its cargo management method. The patent uses robots to move in the warehouse to collect environmental information through cameras and laser radars, and then perceive the situation of the goods. Although the patent realizes automated management, it still has the problem of high cost.
[0004] It can be seen that although the current warehouse management technology has achieved automated management to a certain extent and improved the efficiency of warehouse management, it uses a variety of complex equipment at the front end, resulting in high costs, which invisibly raises the threshold for warehousing and affects the development of the warehousing industry. Summary of the invention
[0005] The embodiment of the present application provides a multi-source integrated unmanned warehouse management system to solve the problem of high cost caused by complex front-end equipment in the prior art.
[0006] The embodiment of the present application provides a multi-source integrated unmanned warehouse management system, including: Cameras are installed at multiple locations in the warehouse. The cameras are used to collect real-time color data in the warehouse. LiDAR, which is installed at multiple locations in the warehouse. LiDAR is used to collect real-time distance data in the warehouse; The cloud processing device is communicatively connected with both the camera and the lidar. The cloud processing device stores a three-dimensional model of the warehouse. After acquiring real-time distance data, the cloud processing device establishes a corresponding real-time target model, extracts personnel targets and cargo targets in the real-time target model, and determines the type and quantity of cargo taken by the personnel according to the shape and size of the cargo targets. After acquiring real-time color data, the cloud processing device adds color to the real-time target model, corrects the type of cargo taken by the personnel according to the color of the cargo target, and generates corresponding inbound / outbound records according to the type and quantity of cargo taken by the personnel.
[0007] The multi-source integrated unmanned warehouse management system in this application has the following advantages: By using cloud technology and AI (artificial intelligence) technology, only cameras and laser radars need to be installed in the warehouse, which greatly reduces the cost of warehouse management at the front end, realizes fully automated management, and improves management efficiency. In addition, the data collected by the camera and laser radar are integrated to improve the accuracy of cargo identification. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0009] Figure 1 A schematic diagram of the usage status of a multi-source fusion unmanned warehouse management system in a warehouse provided in an embodiment of the present application.
[0010] Explanation of the accompanying drawings: 100, warehouse; 110, shelf; 120, goods; 200, camera; 300, lidar; 400, entrance and exit gates; 410, entrance collection unit; 420, exit collection unit. DETAILED DESCRIPTION
[0011] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0012] Figure 1 A schematic diagram of the composition of a multi-source integrated unmanned warehouse management system in a warehouse provided in an embodiment of the present application. The present application embodiment provides a multi-source integrated unmanned warehouse management system, including: Cameras 200 are arranged at multiple locations in the warehouse 100, and the cameras 200 are used to collect real-time color data in the warehouse 100; Laser radars 300 are arranged at multiple locations in the warehouse 100, and the laser radars 300 are used to collect real-time distance data in the warehouse 100; The cloud processing device is communicatively connected to the camera 200 and the laser radar 300. The cloud processing device stores a three-dimensional model of the warehouse 100. After acquiring the real-time distance data, the cloud processing device establishes a corresponding real-time target model, extracts the personnel target and the cargo target in the real-time target model, and determines the type and quantity of the cargo 120 taken by the personnel according to the shape and size of the cargo target. After acquiring the real-time color data, the cloud processing device adds color to the real-time target model, corrects the type of the cargo 120 taken by the personnel according to the color of the cargo target, and generates corresponding inbound / outbound records according to the type and quantity of the cargo 120 taken by the personnel.
[0013] Exemplarily, a plurality of shelves 110 are placed in the warehouse 100 , and a plurality of goods 120 are placed on the shelves 110 . After a person enters the warehouse 100 , the incoming goods 120 will be placed on the shelves 110 , and the outgoing goods 120 will be taken from the shelves 110 .
[0014] In order to ensure that the data collected by the camera 200 and the laser radar 300 have a corresponding relationship, multiple groups of cameras 200 and laser radars 300 are installed in the warehouse 100, each group includes a camera 200 and a laser radar 300, and the cameras 200 and laser radars 300 in a group are installed in very close positions, and the cameras 200 and laser radars 300 in the same group face the same direction, so that the cameras 200 and laser radars 300 in the same group collect data in the same area in the same direction. Moreover, installing multiple groups of cameras 200 and laser radars 300 in the warehouse 100 can enable any person to collect the action of taking and placing any goods 120 in the warehouse 100. In order to achieve this goal, the shelves 110 are arranged in multiple rows and columns in the warehouse 100, and there are passages for people to pass between two adjacent rows or columns of shelves 110, and each group of cameras 200 and laser radars 300 will be installed at the top of the warehouse 100 corresponding to each passage.
[0015] The camera 200 and the laser radar 300 can be connected to the cloud processing device through the Internet. The cloud processing device can be set up in a cloud processing center far away from the warehouse 100. The real-time color data and real-time distance data collected in multiple warehouses 100 at different locations within a certain area will be uniformly transmitted to the cloud processing device in the cloud processing center to realize centralized warehouse management.
[0016] In an embodiment of the present application, when establishing a three-dimensional model, a collection device collects initial distance data at multiple locations in a closed warehouse 100, and a cloud processing device establishes a three-dimensional model of the warehouse 100 based on the initial distance data.
[0017] Specifically, the acquisition device can be a device that integrates multiple laser radars. When collecting the initial distance data, the acquisition device can be placed at multiple locations in the channel. Before the acquisition, it is necessary to ensure that the camera 200 and the laser radar 300 have been installed, and the shelf 110 is also placed at a fixed position. Goods 120 are also placed on the shelf 110. During the acquisition process, no personnel can be present inside the warehouse 100. After collecting the initial distance data of a position each time, personnel need to enter the warehouse 100 to adjust the position of the acquisition device, and then collect again until the initial distance data of all positions are collected. After the initial distance data of multiple locations in the warehouse 100 are collected, these initial distance data will be presented in the form of point clouds. The points in the point cloud represent the distance between the acquisition device and these objects after the acquisition device emits lasers and passes through objects, such as walls, shelves 110, goods 120, etc., and then are received by the acquisition device after reflection. In the process of multiple acquisitions, it is necessary to ensure that there is a certain overlap in the areas of two adjacent acquisitions. When the cloud processing device establishes a three-dimensional model, it will splice the point clouds collected multiple times according to the point clouds of the overlapping parts to form complete point cloud data of the warehouse 100. After obtaining the complete point cloud data, the cloud processing device will connect the discrete point clouds to form a large number of triangular planes, which will form a three-dimensional model.
[0018] The result collected by the laser radar 300 is also the point cloud data of a fixed area in the warehouse. After the three-dimensional model is established, the three-dimensional model is represented in a three-dimensional coordinate system, and any point in the three-dimensional model can be represented in the form of coordinates. Similarly, after the three-dimensional model is established, the backstage staff can mark the camera 200, the laser radar 300 and the shelf 110 in the three-dimensional model. The marked items will have certain coordinates, that is, the coordinates of the laser radar 300 in the three-dimensional model are known. After the cloud processing device obtains the real-time distance data, it determines the coordinates of the target point according to the coordinates of the laser radar 300. Multiple target points with certain coordinates constitute a real-time target model.
[0019] Specifically, the specific manifestation of real-time distance data is also a point cloud. After the laser radar 300 at each position collects real-time distance data, the cloud processing device calculates the coordinates of each target point in the point cloud in the three-dimensional coordinate system based on the coordinates of the laser radar 300 and the distance represented in the point cloud. Based on these coordinates, a surface of the area corresponding to the position will be established. Multiple surfaces can form a three-dimensional space model, which is the real-time target model. The real-time target model not only includes information about personnel and taken goods 120, but also includes information about irrelevant items such as shelves 110, other goods 120 and walls in the warehouse 100. Therefore, after obtaining the real-time target model, the cloud processing device also deletes the target points in the real-time target model that overlap with the three-dimensional model to extract personnel targets and cargo targets.
[0020] Specifically, after the personnel takes the goods 120, it can be assumed that the shelves 110, other goods 120, walls and other irrelevant items in the warehouse 100 have not changed. Therefore, the target points with the same coordinates in the real-time target model can be deleted according to the coordinates of each target point in the three-dimensional model, and the extraction of personnel targets and goods targets can be completed.
[0021] Furthermore, although the three-dimensional model and the real-time target model have a large number of target points, there is still a certain distance between adjacent points. Therefore, the three-dimensional model and the real-time target model may use target points with different coordinates to represent the same position in the warehouse 100, that is, there may be a situation where the target points in the three-dimensional model and the real-time target model cannot be aligned. In view of this situation, the three-dimensional model and the real-time target model can be placed in the same three-dimensional coordinate system, and then the distance between each target point in the three-dimensional model and the real-time target model is calculated, and the two target points with the closest distance are used as matching point pairs, and a matching point pair will be considered to be the same position in the warehouse 100. When deleting the target point in the real-time target model, the coordinates of each target point in the three-dimensional model can be obtained first, and then based on the corresponding relationship of the matching point pairs, the target points in the real-time target model that have a corresponding relationship with the target points in the three-dimensional model can be deleted.
[0022] In a possible embodiment, after extracting the personnel target and the cargo target, the cloud processing device uses a neural network model to identify the area of the personnel target in the real-time target model, deletes the area of the personnel target in the real-time target model, obtains the area of the cargo target, and determines the shape and size of the cargo target based on the area of the cargo target.
[0023] Exemplarily, the neural network model preferably adopts a convolutional neural network model, which needs to be trained with a large number of training samples in the form of three-dimensional point clouds before use to improve the accuracy of the neural network model in identifying people. After recognition using the neural network model, the area of the person target can be marked in the merged target of the person target and the cargo target, and after removing the target points contained in the person target from the merged target, the area of the cargo target can be obtained.
[0024] In an embodiment of the present application, the cloud processing device stores shape data and size data of all goods 120 in the warehouse 100. After obtaining the shape and size of the cargo target, the cloud processing device generates a combination model of the cargo 120 in multiple quantity and position combinations according to the shape data and size data of the cargo 120, compares each combination model with the cargo target, determines the similarity between each combination model and the cargo target, and uses the type and quantity of the cargo 120 in the combination model with the highest similarity in shape and size to the cargo target as the type and quantity of the cargo 120 in the cargo target.
[0025] Specifically, the shape data and size data can be input by the backstage personnel after measuring and observing the goods 120. For example, the shape data and size data of a cube-shaped good 120 can be expressed as: a cube, with a length of 1 cm, a width of 1 cm, and a height of 1 cm. The shape data and size data are the data of the smallest unit of the goods 120 that the personnel can take during the process of entering or leaving the warehouse.
[0026] After obtaining the area of the cargo target, the cloud processing device can obtain a characteristic description of the area, that is, the cargo target in terms of shape and size, for example, it can be described as: a cube, length a2 cm, width b2 cm, height c2 cm. The cargo 120 is combined in multiple quantities and multiple positional relationships to obtain a combination model under multiple quantity and positional relationships. The combination model is a model formed by stacking the cargo models of multiple quantity and positional relationships after the cloud processing device models the cargo 120 according to the shape data and size data. When a person takes multiple cargoes 120, one of the multiple combination models will be used for stacking. Therefore, by comparing the cargo target with each combination model, the type and quantity of the cargo in the cargo target can be obtained.
[0027] When calculating the similarity between the combined model and the cargo target, the minimum distance between each target point in the cargo target and any target point in the combined model can be calculated. The sum of the minimum distances of all target points in the cargo target can be used to represent the similarity. The smaller the sum of the minimum distances, the higher the similarity.
[0028] Furthermore, after obtaining the shape and size of the cargo target, the cloud processing device also extracts the shape features of the cargo target, selects cargo 120 having the same features as the shape features as similar cargo, and generates a combined model according to the shape data and size data of the similar cargo.
[0029] Specifically, there may be many types of goods stored in the warehouse 100, so it is necessary to screen the goods 120 participating in the combination of the goods model before establishing the combination model to minimize the number of combination models that need to be established. During the screening, the feature description of the goods target can be extracted, specifically the feature description of the shape. Then, among the multiple types of goods, the shape data of each type of goods is obtained in turn, and the goods with the same shape data and shape feature description of the goods target are regarded as similar goods, and multiple combination models can be established based on the similar goods. In the subsequent comparison, it is only necessary to calculate the similarity between the goods target and each combination model established by similar goods, which greatly reduces the amount of data.
[0030] In a possible embodiment, after obtaining the area of the cargo target, the cloud processing device determines the coordinate area of the cargo target, extracts the regional color data corresponding to the coordinate area of the cargo target in the real-time color data, and adds the regional color data to the cargo target.
[0031] Exemplarily, since the camera 200 and the laser radar 300 have almost the same direction and acquisition range, they can share the same coordinate system. After the area of the cargo target is determined, the area surrounded by the coordinates of its edge target points is the coordinate area of the cargo target. After the coordinate area is determined, the color data in the same coordinate area can be extracted from the real-time color data as the regional color data. The regional color data is the color data of the cargo target. Therefore, by adding the regional color data to the cargo target, the cargo target can be colored, so that the cargo target has color information.
[0032] Furthermore, the cloud processing device also stores the initial color data of all goods 120 in the warehouse 100. After adding the regional color data to the cargo target, the cloud processing device obtains the initial color data corresponding to the type of the cargo 120 determined according to the shape and size of the cargo target, compares the obtained initial color data with the regional color data added to the cargo target, and corrects the type of the cargo 120 according to the comparison result.
[0033] Specifically, since there may be a lot of goods 120 with the same or similar shapes but different types, color is the most effective means to distinguish these goods 120 with similar shapes. The color mentioned in the embodiment of the present application includes information such as color, text, and graphics, which is reflected in the form of RGB (red, green, and blue) values of each target point in the cargo target. The initial color data can be obtained by the warehouse manager taking photos of each type of cargo 120 and uploading them to the cloud processing device. After adding the regional color data to the cargo target, a series of RGB values of the target points are obtained, which also includes the positional relationship of these target points. For the same type of goods 120, the RGB values of all target points on it will be the same or at least have a high degree of similarity. Therefore, after the cloud processing device determines the initial color data according to the type of goods in the cargo target, it obtains the RGB values of a series of target points corresponding to the cargo target, and after aligning these target points with the target points of the regional color data added to the cargo target, calculates the RGB difference of each group of aligned target point pairs. If the sum of the RGB difference values of all target point pairs is small enough, it can be considered that the type of goods determined by the size and shape of the cargo target and the type of goods determined by the regional color data are the same, and the type of goods is also the accurate type of the goods 120 taken by the person. If the sum of the calculated RGB difference values exceeds a certain threshold, it is considered that the cargo type is incorrectly identified, and the cloud processing device will re-determine the cargo type according to the shape and size of the cargo target until the cargo type is correctly identified.
[0034] In a possible embodiment, it also includes an entry collection unit 410 and an exit collection unit 420, which are respectively arranged on the outside and inside of the exit door 400. The entry collection unit 410 and the exit collection unit 420 are respectively used to collect facial images of people entering and leaving the warehouse 100. The cloud processing device determines the identity information of the person after recognizing the facial image, and adds the identity information to the entry / exit record.
[0035] Exemplarily, the warehouse 100 in the present application is a closed area, and personnel need to pass through the access door 400 to enter and exit. The entry collection unit 410 and the exit collection unit 420 face the outside and inside of the access door 400 respectively. Before a person enters the warehouse 100, the entry collection unit 410 will collect the facial image of the person. The cloud processing device will first identify the key positions of the facial image, including eyes, mouth, etc., and then calculate the distance between these key positions to obtain facial feature information, and then query the database to see whether there is personnel information that meets these facial feature information. If so, it means that the identity of the person is legal. At this time, the access door 400 will open to allow the person to enter. When a person leaves, it is no longer necessary to authenticate the legitimacy of the identity. It is only necessary to determine the identity information of the person based on the facial image, and then combine the identity information with the type and quantity of the goods taken and the current time to form a warehouse entry / exit record.
[0036] It should be understood that the judgment of entering or leaving the warehouse is determined based on the situation of the same person taking the goods 120 when entering the warehouse 100 and the situation of the same person taking the goods 120 when leaving the warehouse 100. When a person enters the warehouse 100 but does not enter the area where the shelf 110 is located, it is necessary to use the camera 200 and the laser radar 300 to collect real-time color data and real-time distance data respectively, and the cloud processing device determines the type and quantity of the goods. When the person leaves the warehouse 100, the type and quantity of the goods are judged again, and the type and quantity of the goods when the person enters and leaves are compared, so that the type and quantity of the goods 120 entering the warehouse and the type and quantity of the goods 120 leaving the warehouse can be obtained.
[0037] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0038] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A multi-source integrated unmanned warehouse management system, characterized in that: include: Cameras (200) are arranged at multiple locations in the warehouse (100), and the cameras (200) are used to collect real-time color data in the warehouse (100); Laser radars (300) are arranged at multiple locations in the warehouse (100), and the laser radars (300) are used to collect real-time distance data in the warehouse (100); A cloud processing device is communicatively connected to the camera (200) and the laser radar (300), wherein the cloud processing device stores a three-dimensional model of the warehouse (100). After acquiring the real-time distance data, the cloud processing device establishes a corresponding real-time target model, extracts the personnel target and the cargo target in the real-time target model, and determines the type and quantity of the cargo (120) taken by the personnel according to the shape and size of the cargo target. After acquiring the real-time color data, the cloud processing device adds color to the real-time target model, corrects the type of the cargo (120) taken by the personnel according to the color of the cargo target, and generates corresponding storage / exit records according to the type and quantity of the cargo (120) taken by the personnel.
2. The multi-source fusion unmanned warehouse management system according to claim 1 is characterized in that: The coordinates of the laser radar (300) in the three-dimensional model are known. After acquiring the real-time distance data, the cloud processing device determines the coordinates of the target point according to the coordinates of the laser radar (300). A plurality of target points with determined coordinates constitute the real-time target model.
3. The multi-source fusion unmanned warehouse management system according to claim 2 is characterized in that: After establishing the real-time target model, the cloud processing device also deletes the target points in the real-time target model that overlap with the three-dimensional model to extract the personnel target and the cargo target.
4. The multi-source fusion unmanned warehouse management system according to claim 1 is characterized in that: After extracting the personnel target and the cargo target, the cloud processing device uses a neural network model to identify the area of the personnel target in the real-time target model, deletes the area of the personnel target in the real-time target model, obtains the area of the cargo target, and determines the shape and size of the cargo target according to the area of the cargo target.
5. The multi-source integrated unmanned warehouse management system according to claim 4 is characterized in that: The cloud processing device stores shape data and size data of all goods (120) in the warehouse (100). After obtaining the shape and size of the cargo target, the cloud processing device generates a combination model of the cargo (120) in multiple quantity and position combinations according to the shape data and the size data of the cargo (120), compares each of the combination models with the cargo target, determines the similarity between each of the combination models and the cargo target, and uses the type and quantity of the cargo (120) in the combination model with the highest similarity in shape and size to the cargo target as the type and quantity of the cargo (120) in the cargo target.
6. The multi-source fusion unmanned warehouse management system according to claim 5 is characterized in that: After obtaining the shape and size of the cargo target, the cloud processing device further extracts the shape features of the cargo target, selects cargo (120) having the same features as the shape features as similar cargo, and generates the combined model according to the shape data and the size data of the similar cargo.
7. The multi-source integrated unmanned warehouse management system according to claim 4 is characterized in that: After obtaining the area of the cargo target, the cloud processing device determines the coordinate area of the cargo target, extracts the regional color data corresponding to the coordinate area of the cargo target in the real-time color data, and adds the regional color data to the cargo target.
8. The multi-source integrated unmanned warehouse management system according to claim 7 is characterized in that: The cloud processing device also stores initial color data of all goods (120) in the warehouse (100). After the regional color data is added to the cargo target, the cloud processing device obtains the initial color data corresponding to the type of the cargo (120) determined according to the shape and size of the cargo target, compares the obtained initial color data with the regional color data added to the cargo target, and corrects the type of the cargo (120) according to the comparison result.
9. The multi-source fusion unmanned warehouse management system according to claim 1 is characterized in that: When establishing the three-dimensional model, a collection device collects initial distance data at multiple locations in a closed warehouse (100), and the cloud processing device establishes the three-dimensional model of the warehouse (100) based on the initial distance data.
10. The multi-source fusion unmanned warehouse management system according to claim 1 is characterized in that: It also includes an entry collection unit (410) and an exit collection unit (420), the entry collection unit (410) and the exit collection unit (420) being respectively arranged on the outside and the inside of the exit door (400), the entry collection unit (410) and the exit collection unit (420) being respectively used to collect facial images of persons entering and leaving the warehouse (100), the cloud processing device identifying the facial images to determine the identity information of the persons, and adding the identity information to the entry / exit records.
