Cargo size detection device and detection method thereof

Abstract

The invention belongs to the technical field of goods conveying, and particularly discloses a goods size detection device and a detection method thereof.The goods size detection device comprises a goods carrying table, a tray used for bearing goods is arranged on the goods carrying table, and a mounting support is arranged on the upper side of the tray; a depth camera used for detecting the size of goods is installed on the installation support, and the depth camera is outwards connected with a computer used for processing RGB images and depth image data in real time and conducting transmission and progress control on the data. According to the scheme, accurate detection of the three-dimensional size can be achieved only through one device, the installation and debugging process is simplified, the fault rate is reduced, and the hardware cost and the maintenance workload are remarkably reduced; through fusion processing of the depth map and the RGB image, the actual maximum length and width of the irregular cargo can be accurately measured, and the limitation of a traditional method on cargo shape adaptability is solved. The detection precision is improved, and goods of various shapes and sizes can be flexibly handled.

Description

Technical Field

[0001] The present invention belongs to the technical field of cargo transportation, and in particular relates to a cargo size detection device and a detection method thereof. Background Art

[0002] In intelligent high-bay warehouses, cargo size detection is a key component in ensuring storage compliance and stacker crane operational safety. Traditional solutions use multiple sets of photoelectric sensors (one pair each for length, width, and height). When cargo on forks enters the loading platform, if the corners of the cargo hit a photoelectric sensor, the sensor triggers an alarm. This approach has the following drawbacks: There are many sensors, and they need to be deployed independently in three dimensions, which makes installation and debugging complex and has a high failure rate. Relying on in-place detection, an additional "cargo in-place sensor" needs to be installed to determine whether the forks are in place, which increases hardware costs; It cannot adapt to irregular goods and cannot accurately measure the actual maximum length and width of irregular goods. Summary of the Invention

[0003] In response to the above problems in the prior art, a cargo size detection device and detection method are proposed. The present invention provides the following technical solutions: A cargo size detection device includes a cargo platform, a pallet for supporting cargo is provided on the cargo platform, a mounting bracket is provided on the upper side of the pallet, a depth camera for detecting cargo size is installed on the mounting bracket, and the depth camera is externally connected to a computer for real-time processing of RGB images and depth image data, and transmitting and controlling the data transmission.

[0004] Preferably, a light source is mounted on the mounting bracket.

[0005] Preferably, an alarm light is installed on the mounting bracket.

[0006] Preferably, the depth camera is centrally arranged above the tray.

[0007] A method for detecting cargo size, comprising the steps of: S1, the depth camera captures images and transmits image information to the computer; S2, computer processing image data; S21. Use Mask-RCNN to identify and segment all pixels of the cargo and all pixels of the pallet in the RGB image, and use an edge detection algorithm to extract edge contour pixels of the cargo and the pallet respectively; S22. Calculate the maximum height of the cargo based on the depth map; S23. Based on the cargo contour pixel points and the edge contour pixel points of the pallet extracted in S21, the cargo contour pixel points within the pallet contour pixel points are eliminated, and the cargo contour pixel points outside the pallet contour pixel points are retained as p1. The three-dimensional coordinates of p1 and each pixel point of the pallet (2) are calculated through the corresponding distances between the pixel points of p1 and the pixel points of the pallet and the depth map. The mathematical expression of the pallet is obtained by fitting the three-dimensional coordinates of the pallet using the least squares method. All three-dimensional coordinates of P1 are traversed and compared with the mathematical expression of the pallet (2).

[0008] Preferably, in step S23, if any dimension exceeds the limit, it is transmitted to the stacker control system, and the stacker control system triggers an alarm light to alarm.

[0009] Preferably, step S21 includes the following steps: S211. The depth camera collects RGB images of the goods and pallets, annotates them using an annotation tool, and uses Mask-RCNN training to obtain an image segmentation model, which is then deployed on the computer. S212, the depth camera collects cargo images in real time and transmits them to the computer, and uses the Mask-RCNN model to process the collected images to obtain a pixel point set p2 where the cargo is located in the image and a pixel point set p3 where the pallet is located in the image; S213 , using the Canny edge detection algorithm to extract the edge contour pixel point set p4 of the goods and the edge contour pixel point set p5 of the pallet from the p2 and p3 respectively.

[0010] Preferably, the fork picks up the goods to the loading platform. In step S21 or step S23, if p2 is empty, an empty load prompt is given.

[0011] Preferably, step S22 includes the following steps: S221: The depth camera acquires the corresponding depth map while capturing the RGB image, and traverses all pixels in the depth map to find the pixel with the minimum depth value; S222. Maximum cargo height = depth camera installation height - minimum depth map value.

[0012] Preferably, step S23 includes the following steps: S231, based on p5 extracted in S213, first use the least squares method to fit the mathematical expression f1 of the tray in the image; S232, compare p4 with the mathematical expression f1 obtained in S231, and retain the point set p1 outside the pallet range; S233, using formula 1 to convert the two-dimensional pixel coordinates of p1, p5 and the depth image collected by 221 into three-dimensional coordinates P1 and P5 in the depth camera coordinate system; Formula 1:

[0013] Wherein, Xs, Ys, and Zs are the three-dimensional coordinates in the depth camera coordinate system, (u0, v0) represents the intersection of the optical axis and the imaging plane, that is, the coordinates of the origin of the image coordinate system in the pixel coordinate system; (fu, fv) are the focal lengths along the X and Y axes, (u0, v0) and (fu, fv) are fixed parameters of the depth camera, which can be obtained through the depth camera calibration, and Z represents the distance from a certain point on the cargo to the depth camera; S234, according to the calculation of S233 to obtain P5, first use the least squares method to fit the mathematical expression f2 of the tray in the depth camera coordinate system; S235. Compare all three-dimensional coordinates of P1 with the mathematical expression f2 obtained in the preset S234. If any dimension exceeds the limit, it is transmitted to the stacker control system, and the stacker control system triggers an alarm light to alarm.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. By integrating a depth camera with advanced image processing technology, this solution can achieve accurate three-dimensional measurement with just one device, eliminating the need to deploy multiple sensors in three dimensions. This simplifies the installation and commissioning process, reduces failure rates, and significantly reduces hardware costs and maintenance workload. 2. This solution utilizes the real-time image acquisition and processing capabilities of the depth camera, combined with an algorithm to determine the loading status of the forks. This eliminates the need for additional "cargo arrival sensors," saving hardware costs while also improving the overall reliability and flexibility of the system. 3. Traditional methods can often only detect whether a fixed direction is out of limit, and are unable to cope with irregular goods. However, this solution can accurately measure the actual maximum length and width of irregular goods through the fusion processing of depth maps and RGB images, which overcomes the limitations of traditional methods in adaptability to goods shapes. It not only improves detection accuracy, but also can flexibly cope with goods of various shapes and sizes. Regardless of whether the goods are regular or irregular, the system can quickly and accurately provide dimensional information, providing strong support for warehouse management, making this solution highly applicable in complex and changing warehouse environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a structural diagram of a specific embodiment of the present invention; In the attached figure, 1. cargo platform; 2. pallet; 3. mounting bracket; 4. depth camera; 5. light source; 6. warning light; 7. cargo fork. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention is clearly and completely described below in conjunction with the accompanying drawings of the present invention. The directional words mentioned in the following embodiments, such as "up", "down", "left" and "right", etc., are only referenced to the directions of the accompanying drawings. Therefore, the directional words used are used to illustrate rather than limit the invention.

[0017] like Figure 1 As shown, a cargo size detection device includes a cargo platform 1, on which a pallet 2 for supporting cargo is provided, and a mounting bracket 3 is provided on the upper side of the pallet 2. A depth camera 4 for detecting the size of cargo is installed on the mounting bracket 3, and the depth camera 4 is externally connected to a computer for real-time processing of RGB images and depth image data, and for transmitting and controlling the data transmission.

[0018] Furthermore, a light source 5 is installed on the mounting bracket 3 to illuminate vertically downward to ensure that the depth camera 4 obtains clear pictures of the cargo.

[0019] Furthermore, an alarm light 6 is installed on the mounting bracket 3 .

[0020] Furthermore, the depth camera 4 is centrally arranged above the tray 2, and the tray 2 is preferably rectangular.

[0021] Only one device needs to be set up, and there is no need for multi-directional calibration and alignment; it can adapt to complex scenarios: it is not affected by the color or reflection of the goods and can detect the true size of irregular goods; it has high functional integration: A method for detecting cargo size, comprising the steps of: S1, the depth camera 4 captures images and transmits the image information to the computer; S2, computer processing image data; S21. Use Mask-RCNN to identify and segment all pixels of the cargo (p2) and all pixels of pallet 2 (p3) in the RGB image. Use an edge detection algorithm to extract the edge contour pixels of the cargo and pallet 2, respectively. Furthermore, step S21 includes the following steps: S211, the depth camera 4 collects RGB images of the goods and the pallet 2, annotates them using an annotation tool, and uses Mask-RCNN training to obtain an image segmentation model, and deploys the model on the computer; S212, the depth camera 4 collects pictures of the goods in real time and transmits them to the computer, and uses the Mask-RCNN model to process the collected pictures to obtain a pixel point set p2 where the goods are located in the image and a pixel point set p3 where the pallet 2 is located in the image; S213 , using the Canny edge detection algorithm to extract the edge contour pixel point set p4 of the goods and the edge contour pixel point set p5 of the pallet 2 from the p2 and p3 respectively.

[0022] Among them, if p2 is empty, a no-load prompt will be given.

[0023] S22. Calculate the maximum height of the cargo based on the depth map; Furthermore, step S22 includes the following steps: S221, the depth camera 4 acquires the corresponding depth map while collecting the RGB image, and traverses all pixels in the depth map to find the pixel with the minimum depth value; S222: Maximum height of cargo = installation height of depth camera 4 - minimum value of depth map.

[0024] S23. Based on the cargo contour pixel points p4 and the edge contour pixel points p5 of pallet 2 extracted in S213, the cargo contour pixel points within the pallet 2 contour pixel points are eliminated, and the cargo contour pixel point set outside the pallet 2 contour pixel points is retained as p1. The three-dimensional coordinates P5 of each pixel point of the pallet 2 contour and the three-dimensional coordinates P1 of each pixel point of p1 are calculated through the contour pixel point set p5 of pallet 2 and the pixel point set of p1 and the corresponding distances in the depth map. The mathematical expression f2 is obtained by fitting the point set P5 by the least squares method. The three-dimensional coordinates of all P1 are traversed and compared with the mathematical expression of pallet 2. If any dimension exceeds the limit, it is transmitted to the stacker control system, and the stacker control system triggers the alarm light 6 to alarm.

[0025] Furthermore, step S23 includes the following steps: S231, based on p5 extracted in S213, first use the least squares method to fit the mathematical expression f1 of the tray 2 in the image. Specifically, the mathematical expression f1 is a mathematical expression that fits the outline of the tray 2 in the image into four straight lines that enclose a rectangle; S232, compare p4 with the mathematical expression f1 obtained in S231, and retain the point set p1 outside the range of tray 2; S233, using formula 1 to convert the two-dimensional pixel coordinates of p1, p5 and the depth image collected by 221 into three-dimensional coordinates P1 and P5 in the coordinate system of the depth camera 4; Formula 1:

[0026] Among them, Xs, Ys, Zs are the three-dimensional coordinates in the coordinate system of the depth camera 4, (u0, v0) represents the intersection of the optical axis and the imaging plane, that is, the coordinates of the origin of the image coordinate system in the pixel coordinate system; (fu, fv) are the focal lengths along the X-axis and Y-axis, (u0, v0) and (fu, fv) are fixed parameters of the depth camera 4, which can be obtained by calibration of the depth camera 4, and Z represents the distance from a certain point of the cargo to the depth camera 4.

[0027] S234, according to the calculation of S233 to obtain P5, first use the least squares method to fit the outline of the tray 2 in the three-dimensional space to obtain the mathematical expression f2 in the coordinate system of the depth camera 4; S235 , traverse all three-dimensional coordinates of P1 and compare them with the mathematical expression f2 (threshold) obtained in the preset S234. If any dimension exceeds the limit, it is transmitted to the stacker control system, and the stacker control system triggers the alarm light 6 to alarm.

[0028] Furthermore, the edge computer obtains the size information (length, width and height) of the goods, which can be recorded and sent to the stacker crane control system to facilitate subsequent size review or optimization of the warehouse layout.

[0029] This method can detect the true size of irregular goods and determine the loading status of the fork 7 through an algorithm without the need for additional in-place sensors; data traceability allows the cargo dimensions to be stored for subsequent dimension review or optimization of warehouse layout.

Claims

1. A cargo size detection device, characterized in that: The invention comprises a cargo platform (1), wherein a pallet (2) for supporting cargo is provided on the cargo platform (1), a mounting bracket (3) is provided on the upper side of the pallet (2), a depth camera (4) for detecting the size of cargo is installed on the mounting bracket (3), and the depth camera (4) is externally connected to a computer for processing RGB images and depth image data in real time and transmitting and controlling data transmission.

2. The cargo size detection device according to claim 1, characterized in that: A light source (5) is mounted on the mounting bracket (3).

3. The cargo size detection device according to claim 1, characterized in that: An alarm light (6) is installed on the mounting bracket (3).

4. The cargo size detection device according to claim 1, characterized in that: The depth camera (4) is centrally arranged above the tray (2).

5. A method for detecting cargo size, characterized in that: The steps include: S1, the depth camera (4) captures images and transmits the image information to the computer; S2, computer processing image data; S21, using Mask-RCNN to identify and segment all pixels of the cargo and all pixels of the pallet (2) in the RGB image, and using an edge detection algorithm to extract edge contour pixels of the cargo and edge contour pixels of the pallet (2); S22. Calculate the maximum height of the cargo based on the depth map; S23. Based on the cargo contour pixel point set extracted in S21 and the edge contour pixel point set of the pallet (2), the cargo contour pixel points within the pallet (2) contour pixel points are eliminated, and the cargo contour pixel point set outside the pallet (2) contour pixel points is retained as p1. The three-dimensional coordinates of p1 and each pixel point of the pallet (2) are calculated through the corresponding distances between the pixel points of p1 and the pixel points of the pallet (2) and the depth map. The mathematical expression of the pallet is obtained by fitting the three-dimensional coordinates of the pallet using the least squares method. All three-dimensional coordinates of P1 are traversed and compared with the mathematical expression of the pallet (2).

6. The cargo size detection method according to claim 5, characterized in that: In step S23, if any dimension exceeds the limit, it is transmitted to the stacker control system, and the stacker control system triggers the alarm light (6) to alarm.

7. The cargo size detection method according to claim 5, characterized in that: Step S21 includes the following steps: S211, the depth camera (4) collects RGB images of the goods and the pallet (2), annotates them using an annotation tool, and uses Mask-RCNN training to obtain an image segmentation model, and deploys the model on the computer; S212, the depth camera (4) collects pictures of the goods in real time and transmits them to the computer, and uses the Mask-RCNN model to process the collected pictures to obtain a pixel point set p2 where the goods are located in the image and a pixel point set p3 where the pallet (2) is located in the image; S213, using the Canny edge detection algorithm to extract the edge contour pixel point set p4 of the goods and the edge contour pixel point set p5 of the pallet (2) from p2 and p3 respectively.

8. The cargo size detection method according to claim 7, characterized in that: The cargo fork (7) picks up the cargo and places it on the cargo platform (1). In step S21, if p2 is empty, an empty load prompt is given.

9. The cargo size detection method according to claim 7, characterized in that: Step S22 includes the following steps: S221, the depth camera (4) acquires the corresponding depth map while collecting the RGB image, and traverses all the pixels in the depth map to find the pixel with the minimum depth value; S222. Maximum height of cargo = installation height of depth camera (4) - minimum value of depth map.

10. The cargo size detection method according to claim 9, characterized in that: Step S23 includes the following steps: S231, based on p5 extracted in S213, first use the least squares method to fit the mathematical expression f1 of the tray (2) in the image; S232, compare p4 with the mathematical expression f1 obtained in S231, and retain the point set p1 outside the range of the tray (2); S233, using formula 1 to convert the two-dimensional pixel coordinates of p1, p5 and the depth image collected by 221 into three-dimensional coordinates P1 and P5 in the coordinate system of the depth camera (4); formula 1: Wherein, Xs, Ys, Zs are the three-dimensional coordinates in the coordinate system of the depth camera (4), (u0, v0) represents the intersection of the optical axis and the imaging plane, that is, the coordinates of the origin of the image coordinate system in the pixel coordinate system; (fu, fv) are the focal lengths along the X-axis and Y-axis, (u0, v0) and (fu, fv) are fixed parameters of the depth camera (4), which can be obtained by calibration of the depth camera (4), and Z represents the distance from a certain point of the cargo to the depth camera (4); S234, according to the calculation of S233 to obtain P5, first use the least squares method to fit the mathematical expression f2 of the tray (2) in the coordinate system of the depth camera (4); S235, traverse all three-dimensional coordinates of P1 and compare them with the mathematical expression f2 obtained in the preset S234. If any dimension exceeds the limit, it is transmitted to the stacker control system, and the stacker control system triggers the alarm light (6) to alarm.

Images