Cargo size detection device and detection method thereof

Abstract

The application belongs to the technical field of cargo conveying, and specifically discloses a cargo size detection device and a detection method thereof. The cargo size detection device comprises a cargo loading platform, a tray for supporting cargo is arranged on the cargo loading platform, a mounting bracket is arranged on the upper side of the tray, a depth camera for detecting the size of the cargo is mounted on the mounting bracket, the depth camera is outwardly connected with a computer for real-time processing of RGB image and depth image data, and the computer is further used for data transmission and control. The present application can realize accurate detection of three-dimensional size by using only one device, simplifies the installation and debugging process, reduces the failure rate, and significantly reduces the hardware cost and maintenance workload. Through fusion processing of the depth map and the RGB image, the actual maximum length and width of irregular cargo can be accurately measured, and the limitation of the traditional method in cargo shape adaptability is solved. The detection accuracy is improved, and various shapes and sizes of cargo can be flexibly dealt with.

Description

Technical Field

[0001] This invention belongs to the field of cargo transportation technology, and specifically relates to a cargo size detection device and its detection method. Background Technology

[0002] In automated warehouses, cargo size detection is a core component ensuring compliance with storage regulations and the safety of stacker crane operations. Traditional solutions employ multiple sets of photoelectric sensors (one pair each in the length, width, and height directions) for detection: when cargo on the forks enters the loading platform, if a corner of the cargo triggers a photoelectric sensor, the sensor activates an alarm. This approach has the following drawbacks: The large number of sensors necessitates independent deployment in three dimensions, resulting in complex installation and debugging, and a high failure rate. Relying on position detection requires the additional installation of a "goods position sensor" to determine whether the forks are in position, increasing hardware costs; It cannot adapt to irregular cargo and cannot accurately measure the actual maximum length and width of irregular cargo. Summary of the Invention

[0003] To address the aforementioned problems in the prior art, a cargo size detection device and method are proposed. The present invention provides the following technical solution: A cargo size detection device includes a loading platform with a pallet for supporting cargo. A mounting bracket is provided on the upper side of the pallet, and a depth camera for detecting cargo size is mounted on the mounting bracket. The depth camera is externally connected to a computer for real-time processing of RGB images and depth image data, and for transmitting and controlling the data transmission.

[0004] Preferably, a light source is mounted on the mounting bracket.

[0005] Preferably, an alarm light is installed on the mounting bracket.

[0006] Preferably, the depth camera is centrally positioned above the tray.

[0007] A method for inspecting the dimensions of goods, comprising the following steps: S1. The depth camera captures images and transmits the image information to the computer; S2, Computer processing of image data; S21. Based on the RGB image, use Mask-RCNN to identify and segment all pixels of the goods and all pixels of the pallet, and use an edge detection algorithm to extract the edge contour pixels of the goods and the edge contour pixels of the pallet respectively. S22. Determine the maximum height of the cargo based on the depth map; S23. Based on the cargo outline pixels and pallet edge outline pixels extracted in S21, remove cargo outline pixels that are within the pallet outline pixels and retain cargo outline pixels that are outside the pallet outline pixels as p1. Calculate the three-dimensional coordinates of each pixel of p1 and pallet (2) by using the distances between the pixels of p1 and the pixels of the pallet and the corresponding distances in the depth map. Obtain the mathematical expression of the pallet by fitting the three-dimensional coordinates of the pallet using the least squares method. Compare the three-dimensional coordinates of all P1 with the mathematical expression of the pallet (2).

[0008] Preferably, in step S23, if any dimension exceeds the limit, the information is transmitted to the stacker crane control system, which then triggers an alarm light.

[0009] Preferably, step S21 includes the following steps: S211. A depth camera acquires RGB images of goods and pallets, labels them using a labeling tool, trains an image segmentation model using Mask-RCNN, and deploys it to the computer. S212. The depth camera captures images of the goods in real time and transmits them to the computer. The captured images are processed using the Mask-RCNN model to obtain the pixel set p2 of the goods in the image and the pixel set p3 of the pallet in the image. S213. The edge contour pixel set p4 of the goods and the edge contour pixel set p5 of the pallet are extracted from p2 and p3 respectively using the Canny edge detection algorithm.

[0010] Preferably, in step S21 or step S23, if p2 is empty, an empty load prompt is issued when the forks pick up the goods onto the loading platform.

[0011] Preferably, step S22 includes the following steps: S221. While acquiring RGB images, the depth camera obtains the corresponding depth map and traverses all pixels in the depth map to find the pixel with the smallest depth value. S222, Maximum height of cargo = Installation height of depth camera - Minimum value of depth map.

[0012] Preferably, step S23 includes the following steps: S231. Based on S213, p5 is extracted, and the least squares method is first used to fit the mathematical expression f1 of the tray in the image. S232. Compare p4 with the mathematical expression f1 obtained in S231, and retain the point set p1 outside the tray range; S233. Convert the two-dimensional pixel coordinates of p1, p5, and the depth images acquired by 221 into three-dimensional coordinates P1 and P5 in the depth camera coordinate system using Formula 1; Formula 1:

[0013] Where Xs, Ys, and Zs are the three-dimensional coordinates in the depth camera coordinate system, (u0, v0) represents the intersection of the optical axis and the imaging plane, that is, the coordinates of the origin of the image coordinate system in the pixel coordinate system; (fu, fv) are the focal lengths along the X and Y axes, (u0, v0) and (fu, fv) are the fixed parameters of the depth camera, which can be obtained through the calibration of the depth camera, and Z represents the distance from a certain point of the cargo to the depth camera; S234. Based on S233, P5 is calculated. First, the least squares method is used to fit the mathematical expression f2 of the tray in the depth camera coordinate system. S235. Traverse all three-dimensional coordinates of P1 and compare them with the mathematical expression f2 obtained by the preset S234. If any dimension exceeds the limit, it is transmitted to the stacker crane control system, and the stacker crane control system triggers the alarm light.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. By integrating a depth camera with advanced image processing technology, this solution can achieve accurate detection of three-dimensional dimensions with only one device, eliminating the need to deploy multiple sensors independently in three dimensions. This simplifies the installation and debugging process, reduces the failure rate, and significantly reduces hardware costs and maintenance workload. 2. This solution utilizes the real-time image acquisition and processing capabilities of a depth camera, combined with algorithms to determine the loading status of the forks, eliminating the need for additional "cargo positioning sensors." This not only saves on hardware costs but also improves the overall reliability and flexibility of the system. 3. Traditional methods often only detect whether limits are exceeded in a fixed direction, and are inadequate for irregular goods. This solution, however, through the fusion of depth maps and RGB images, can accurately measure the actual maximum length and width of irregular goods, overcoming the limitations of traditional methods in adapting to different goods shapes. It not only improves detection accuracy but also flexibly handles goods of various shapes and sizes. Regardless of whether the goods are regular or irregular, the system can quickly and accurately provide dimensional information, providing strong support for warehouse management and making this solution highly valuable in complex and ever-changing warehouse environments. Attached Figure Description

[0015] Figure 1 This is a structural schematic diagram in a specific embodiment of the present invention; In the attached diagram: 1. Loading platform; 2. Pallet; 3. Mounting bracket; 4. Depth camera; 5. Light source; 6. Alarm light; 7. Forks. Detailed Implementation

[0016] To enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. The directional terms mentioned in the following embodiments, such as "up", "down", "left", "right", etc., are only for reference to the directions in the accompanying drawings. Therefore, the directional terms used are for illustration and not for limiting the invention.

[0017] like Figure 1 As shown, a cargo size detection device includes a loading platform 1, on which a pallet 2 for supporting cargo is provided. A mounting bracket 3 is provided on the upper side of the pallet 2, and a depth camera 4 for detecting cargo size is mounted on the mounting bracket 3. The depth camera 4 is externally connected to a computer for real-time processing of RGB images and depth image data, and for transmitting and controlling the data transmission.

[0018] Furthermore, a light source 5 is installed on the mounting bracket 3 to illuminate vertically downwards, ensuring that the depth camera 4 acquires clear images of the goods.

[0019] Furthermore, an alarm light 6 is installed on the mounting bracket 3.

[0020] Furthermore, the depth camera 4 is centrally positioned above the tray 2, which is preferably rectangular.

[0021] Only one device needs to be set up, eliminating the need for multi-directional calibration and alignment; adaptable to complex scenarios: unaffected by cargo color or reflectivity, capable of detecting the true dimensions of irregularly shaped goods; high degree of functional integration. A method for inspecting the dimensions of goods, comprising the following steps: S1, Depth Camera 4 captures images and transmits the image information to the computer; S2, Computer processing of image data; S21. Based on the RGB image, use Mask-RCNN to identify and segment all pixels of the goods (p2) and all pixels of the pallet 2 (p3), and use an edge detection algorithm to extract the edge contour pixels of the goods and the edge contour pixels of the pallet 2 respectively. Furthermore, step S21 includes the following steps: S211 and depth camera 4 acquire RGB images of goods and pallet 2, use annotation tools to complete the annotation, use Mask-RCNN to train the image segmentation model, and deploy it to the computer; S212. The depth camera 4 captures images of the goods in real time and transmits them to the computer. The captured images are processed using the Mask-RCNN model to obtain the pixel set p2 of the goods in the image and the pixel set p3 of the pallet 2 in the image. S213. The edge contour pixel set p4 of the goods and the edge contour pixel set p5 of the pallet 2 are extracted from p2 and p3 respectively using the Canny edge detection algorithm.

[0022] If p2 is empty, an empty load message will be displayed.

[0023] S22. Determine the maximum height of the cargo based on the depth map; Furthermore, step S22 includes the following steps: S221. While acquiring RGB images, the depth camera 4 obtains the corresponding depth map and traverses all pixels in the depth map to find the pixel with the smallest depth value. S222, Maximum height of cargo = Installation height of depth camera 4 - Minimum value of depth map.

[0024] S23. Based on the cargo outline pixel points p4 and pallet 2 edge outline pixel points p5 extracted in S213, remove the cargo outline pixel points within the pallet 2 outline pixel points, and retain the cargo outline pixel point set outside the pallet 2 outline pixel points as p1. Calculate the three-dimensional coordinates P5 of each pixel point of pallet 2 outline and the three-dimensional coordinates P1 of each pixel point of p1 using the pixel sets p5 and p1 outline pixel points of pallet 2 and the corresponding distances in the depth map. Obtain the mathematical expression f2 by fitting the point set P5 using the least squares method. Compare the three-dimensional coordinates of all P1 with the mathematical expression of pallet 2. If any dimension exceeds the limit, transmit the data to the stacker crane control system, and the stacker crane control system triggers the alarm light 6.

[0025] Furthermore, step S23 includes the following steps: S231. Based on S213, p5 is extracted. First, the least squares method is used to fit the mathematical expression f1 of the tray 2 in the image. Specifically, the mathematical expression f1 is a mathematical expression that fits the outline of the tray 2 in the image into four straight lines that form a rectangle. S232. Compare p4 with the mathematical expression f1 obtained in S231, and retain the point set p1 outside the range of tray 2; S233. Convert the two-dimensional pixel coordinates of p1, p5, and the depth images acquired by 221 into three-dimensional coordinates P1 and P5 in the coordinate system of the depth camera 4 using Formula 1; Formula 1:

[0026] Where Xs, Ys, and Zs are the three-dimensional coordinates in the coordinate system of the depth camera 4, (u0, v0) represents the intersection of the optical axis and the imaging plane, that is, the coordinates of the origin of the image coordinate system in the pixel coordinate system; (fu, fv) are the focal lengths along the X and Y axes, (u0, v0) and (fu, fv) are the fixed parameters of the depth camera 4, which can be obtained through the calibration of the depth camera 4, and Z represents the distance from a certain point of the cargo to the depth camera 4.

[0027] S234. Based on S233, P5 is calculated. First, the least squares method is used to fit the contour of the tray 2 in three-dimensional space to obtain the mathematical expression f2 in the coordinate system of the depth camera 4. S235. Traverse all three-dimensional coordinates of P1 and compare them with the mathematical expression f2 (threshold) obtained by the preset S234. If any dimension exceeds the limit, it is transmitted to the stacker crane control system, and the stacker crane control system triggers alarm light 6.

[0028] Furthermore, the edge computer obtains the dimensions (length, width, and height) of the goods, records them, and sends them to the stacker crane control system, facilitating subsequent dimension verification or optimization of the warehouse layout.

[0029] This method can detect the true size of irregular goods and determine the loading status of the forks 7 through an algorithm, without the need for additional positioning sensors; the data is traceable and the size of the goods can be stored for subsequent size verification or optimization of warehouse layout.

Claims

1. A method for detecting the dimensions of goods, characterized in that, The steps include: S1, Depth camera (4) captures images and transmits image information to computer; S2, Computer processing of image data; S21. Based on the RGB image, use Mask-RCNN to identify and segment all the pixels of the goods and all the pixels of the pallet (2). Use the edge detection algorithm to extract the edge contour pixels of the goods and the edge contour pixels of the pallet (2) respectively. S22. Determine the maximum height of the cargo based on the depth map; S23. Based on the set of cargo outline pixels extracted in S21 and the set of edge outline pixels of pallet (2), remove cargo outline pixels within the outline pixels of pallet (2) and retain the set of cargo outline pixels outside the outline pixels of pallet (2) as p1. Calculate the three-dimensional coordinates P1 of each pixel of p1 and pallet (2) using the pixels of p1, the pixels of pallet (2) and the corresponding distances in the depth map. Fit the three-dimensional coordinates of pallet using the least squares method to obtain the mathematical expression of pallet. Compare the three-dimensional coordinates of all P1 with the mathematical expression of pallet (2).

2. The cargo size detection method according to claim 1, characterized in that, In step S23, if any dimension exceeds the limit, the information is transmitted to the stacker crane control system, and the stacker crane control system triggers the alarm light (6) to sound an alarm.

3. The cargo size detection method according to claim 1, characterized in that, Step S21 includes the following steps: S211. The depth camera (4) acquires RGB images of the goods and pallet (2), uses a labeling tool to complete the labeling, and uses Mask-RCNN to train the image segmentation model and deploy it to the computer. S212, The depth camera (4) acquires cargo images in real time and transmits them to the computer. The acquired images are processed using the Mask-RCNN model to obtain the pixel set p2 of the cargo in the image and the pixel set p3 of the pallet (2) in the image. S213. The edge contour pixel set p4 of the goods and the edge contour pixel set p5 of the pallet (2) are extracted from p2 and p3 respectively using the Canny edge detection algorithm.

4. The cargo size detection method according to claim 3, characterized in that, The fork (7) picks up the goods and places them on the loading platform (1). In step S21, if p2 is empty, an empty load prompt is given.

5. The cargo size detection method according to claim 3, characterized in that, Step S22 includes the following steps: S221. While acquiring the RGB image, the depth camera (4) obtains the corresponding depth map and traverses all pixels in the depth map to find the pixel with the smallest depth value. S222, Maximum height of cargo = depth camera (4) installation height - minimum value of depth map.

6. The cargo size detection method according to claim 5, characterized in that, Step S23 includes the following steps: S231. Based on S213, p5 is extracted. First, the least squares method is used to fit the mathematical expression f1 of the tray (2) in the image. S232. Compare p4 with the mathematical expression f1 obtained from S231, and retain the point set p1 outside the range of tray (2); S233. Convert the two-dimensional pixel coordinates of p1, p5 and the depth image acquired in step S221 into three-dimensional coordinates P1 and P5 in the coordinate system of the depth camera (4) using Formula 1; Formula 1: Where Xs, Ys, and Zs are the three-dimensional coordinates in the coordinate system of the depth camera (4), (u0, v0) represents the intersection of the optical axis and the imaging plane, that is, the coordinates of the origin of the image coordinate system in the pixel coordinate system; (fu, fv) are the focal lengths along the X-axis and Y-axis, (u0, v0) and (fu, fv) are the fixed parameters of the depth camera (4), which can be obtained by calibrating the depth camera (4), and Z represents the distance from a certain point of the cargo to the depth camera (4); S234. Based on S233, P5 is calculated. First, the least squares method is used to fit the mathematical expression f2 of the tray (2) in the coordinate system of the depth camera (4). S235. Traverse all three-dimensional coordinates of P1 and compare them with the mathematical expression f2 obtained by the preset S234. If any dimension exceeds the limit, it is transmitted to the stacker control system, and the stacker control system triggers the alarm light (6) to alarm.

7. A cargo size detection device, characterized in that, The cargo size detection method as described in claim 1 includes a loading platform (1), on which a pallet (2) for supporting cargo is provided, and a mounting bracket (3) is provided on the upper side of the pallet (2). A depth camera (4) for detecting cargo size is mounted on the mounting bracket (3), and the depth camera (4) is externally connected to a computer for real-time processing of RGB images and depth image data, and for transmitting and further controlling the data.

8. The cargo size detection device according to claim 7, characterized in that, A light source (5) is mounted on the mounting bracket (3).

9. The cargo size detection device according to claim 7, characterized in that, An alarm light (6) is installed on the mounting bracket (3).

10. The cargo size detection device according to claim 7, characterized in that, The depth camera (4) is centrally positioned above the tray (2).

Images