Cargo tray detection method and system based on depth camera

Abstract

The invention belongs to the field of industrial robots, and particularly relates to a cargo tray detection method and system based on a depth camera, and the method comprises the following steps: 1) when the cargo tray detection system receives an AGV tray detection instruction, judging whether the working state of the camera is normal, if the camera normally obtains a point cloud, executing the next step, and otherwise, terminating a task; 2) intercepting a point cloud according to a set area, dividing a three-dimensional grid, and intercepting according to the three-dimensional grid to obtain an ROI area; 3) performing coarse segmentation on the three-dimensional grid to obtain point cloud blocks divided into the three-dimensional grid; performing fine segmentation on the segmented point clouds to obtain the width and height of each point cloud; 4) traversing the clustered point cloud blocks, performing planar projection on each obtained point cloud block, and dividing the point cloud blocks into two-dimensional grids; 5) searching a tray in the two-dimensional grid, and resolving the pose of the tray; and 6) detecting a plurality of trays, and reporting the pose of the tray closest to the AGV. The method has the advantages of simple engineering deployment, high precision, low price, strong robustness and the like.

Description

Technical Field

[0001] The present invention belongs to the field of industrial robots, and in particular to a method and system for detecting a cargo pallet based on a depth camera. Background Art

[0002] With the development of modern logistics technology, forklift AGV has become more and more important in intelligent warehousing, and cargo pallet detection is one of the core technologies of forklift AGV. The warehousing environment is complex, with sunlight interference or insufficient light. Accurate, efficient and easy-to-deploy cargo pallet detection is an urgent problem to be solved.

[0003] After searching patents, it is found that the detection of cargo pallets by depth cameras currently uses methods such as feature-based template matching or deep learning. Among them, although the feature-based template matching solution does not have high hardware requirements, it has poor versatility. A single template cannot be applied to pallets of different sizes and shapes; pallet occlusion or damage can also cause missed detection. The detection solution based on deep learning has high hardware requirements and requires a large amount of data for training. There is a risk of false detection when the environmental background changes and training data is insufficient, and the accuracy is insufficient.

[0004] Among them, the authors Zhan Yan et al. used a method of calculating the color features of key points and adaptive neighborhood fast point feature histograms for feature matching in the paper "Pallet Recognition Method Based on Adaptive Color Fast Point Feature Histogram". The versatility of a single template is poor. The author Zhou Hengsen used an improved YOLOv7 deep learning method for pallet detection in the paper "Research and Implementation of Precise Positioning and Driving Road Detection of Pallets in Visual Navigation Unmanned Forklifts". This method is highly complex and has poor generalization for pallets of different styles. Summary of the invention

[0005] In view of the defects of the prior art, the object of the present invention is to provide a cargo pallet detection system and method based on a depth camera, which has the advantages of simple engineering deployment, high precision, low price and strong robustness.

[0006] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: a cargo pallet detection method based on a depth camera, comprising the following steps:

[0007] 1) When the cargo pallet detection system receives the AGV pallet detection instruction, it determines whether the camera is working normally. If the camera obtains the point cloud normally, it executes the next step. Otherwise, it repeats step 1) for a set number of times and terminates the task;

[0008] 2) Intercept the point cloud according to the set area, divide it into three-dimensional grids, and obtain the ROI area according to the three-dimensional grid interception;

[0009] 3) The three-dimensional grid in step 2) is roughly segmented to obtain point cloud blocks divided into the three-dimensional grid; the segmented point cloud is finely segmented to obtain the width and height of each point cloud block;

[0010] 4) Traverse the clustered point cloud blocks, use the edge extraction algorithm to obtain each point cloud block, perform plane projection, and divide it into a two-dimensional grid;

[0011] 5) Search for the pallet in the two-dimensional grid and solve the pallet position;

[0012] 6) If multiple pallets are detected, they are sorted by Euclidean distance and the position of the pallet closest to the AGV is reported.

[0013] The camera is installed at the top of the detection space rotated to 15° relative to the horizontal plane, and the distance between the pallet and the camera is less than 5 meters, and the height difference between the pallet and the camera is less than 2 meters.

[0014] The step 1) is specifically:

[0015] Determine whether the camera is working normally. If the camera can obtain point cloud normally, convert the point cloud to the AGV body center coordinate system according to the camera external parameter calibration parameters. If the camera cannot obtain point cloud, determine that the camera is disconnected or the data acquisition has timed out, try to reconnect the camera or reacquire data, and obtain the camera working status again. If it is still abnormal, terminate the task and issue a warning prompt.

[0016] The step 2) is specifically:

[0017] 2-1) According to the set area, the point cloud is intercepted, and the plane fitting is performed on the intercepted area point cloud. The height Z from the plane to the camera is calculated and compared with the calibrated height to determine whether the pallet is on the ground, and the ground point cloud is filtered. After filtering the ground point cloud, the ROI area is intercepted;

[0018] Step 2-1), specifically:

[0019] According to the configuration of the parameter file, the ground point cloud area parameters xmin, xmax, ymin, ymax, zmin, zmax are intercepted, and the three-dimensional point cloud is intercepted according to these parameters;

[0020] Use the RANSAC algorithm to fit the plane and determine the plane normal vector Is the angle between the plane and the unit vector (0,0,1) less than the threshold? If so, add up the Z coordinates of the points on the plane and divide by the number of points to get the height Z of the plane from the ground. average ;

[0021] Compare Z average And parameter file calibration height Z calibrate Is the difference less than the threshold? If so, the pallet is on the ground and the interception is greater than Z averageThe point cloud and the detection area parameters xmin, xmax, ymin, ymax, zmin, zmax configured in the parameter file are used to obtain the ROI area detection tray;

[0022] 2-2) If the pallet is detected on the ground, the point cloud is divided into three-dimensional grids. The grid size is set according to the actual situation, and the point cloud number and boundary information in the grid are saved. The upper boundary of each grid and the ground height z are compared, and the grids that are less than the threshold are eliminated.

[0023] 3-1) The obtained three-dimensional grid is roughly segmented along the y-axis and x-axis of the point cloud. When the grid point cloud is empty, segmentation is performed to filter out the point cloud blocks that do not meet the size requirements;

[0024] 3-2) The segmented point cloud is re-segmented by the fast Euclidean clustering algorithm. After segmentation, the width and height of each point cloud are calculated, and the point cloud blocks that do not meet the requirements are removed;

[0025] Calculate the point cloud block boundary values ​​xmin, xmax, ymin, ymax, zmin, zmax, and the point cloud block height height = z max -z min , width width = y max -y min If the height is less than 10cm or the width is less than 50cm, the point cloud block will be removed.

[0026] 4-1) Traverse the clustered point cloud blocks, extract the edge of each point cloud block, fit a straight line, roughly estimate the angle of the current point cloud block relative to the y-axis, and reversely rotate the point cloud according to the angle between the front surface of the tray and the YOZ plane;

[0027] 4-2) Project each point cloud block to the YOZ plane and divide the point cloud into two-dimensional grids. According to the set value, divide the point cloud in the point cloud block into two-dimensional grids, and record the boundary information and point cloud number;

[0028] 4-3) Search along the increasing direction of the z-axis and traverse the grid along the decreasing direction of the y-axis. When the grid point cloud is empty, segment it and record the point cloud number and boundary information of each block after segmentation to realize the division of the point cloud into a two-dimensional grid.

[0029] In step 5), the searching for the tray in the two-dimensional grid is specifically as follows:

[0030] Traverse the two-dimensional grid along the increasing direction of the Z axis, and search for the pallet in the two-dimensional grid, and search for the three-leg pallet according to the parameter file configuration; if the conditions are met, mark the starting sequence number of the pallet, and when the number of the current layer block is one, mark the end of the search for a pallet;

[0031] The parameter file is configured to determine the number of pallet legs of the pallet to be searched.

[0032] In step 5), the pallet posture is solved as follows:

[0033] 5-1) The searched pallet point cloud is used to extract the edge through the edge extraction algorithm and then fit the straight line to calculate the pallet angle;

[0034] a. Project the point cloud onto the XOY plane, traverse each point P(x,y), and use the formula atan(x / y) to get the angle theta of point P relative to the X axis and calculate If the angle difference between P0 and P1 is 0.1 degrees, compare the d of the two points. p , keep d p Smaller points get the edge of the point cloud;

[0035] b. Fit the straight line Ax+By=0 by RANSAC algorithm, and the tray angle is atan(B / A);

[0036] 5-2) Calculate the left and right deviations through the edge of the pallet and the hole of the pallet, calculate the depth through the edge, and calculate the height through the upper and lower edges of the pallet;

[0037] Traverse all point clouds to obtain the upper and lower boundary values ​​of the point clouds and the height value of the pallet, that is:

[0038] h=(z min +z max )

[0039] Among them, z min is the lower boundary value, z max is the upper boundary value.

[0040] A cargo pallet detection system based on a depth camera comprises: a memory and a processor; the memory is used to store a computer program; the processor is used to implement a cargo pallet detection method based on a depth camera when executing the computer program.

[0041] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, a method for detecting a cargo pallet based on a depth camera is implemented.

[0042] The present invention has the following beneficial effects and advantages:

[0043] 1. The project deployment of the present invention is simple, and the position of the pallet can be located by only configuring the detection range, and is suitable for pallets of various colors and specifications.

[0044] 2. The present invention has the advantages of high precision, low price, strong robustness, etc., and partial occlusion of the tray does not affect the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A method flow chart of a preferred embodiment of the present invention;

[0046] Figure 2 Obtaining an original point cloud and a point cloud transformed according to camera extrinsic parameters for a camera in a preferred embodiment of the present invention;

[0047] Figure 3 In a preferred embodiment of the present invention, the ground is filtered out and the point cloud of the ROI area is intercepted;

[0048] Figure 4 is a point cloud block after rough segmentation in a preferred embodiment of the present invention;

[0049] Figure 5 It is a clustering fine segmentation in a preferred embodiment of the present invention and filters out point cloud blocks whose sizes do not meet the requirements;

[0050] Figure 6 The point cloud is obtained by extracting the edge fitting straight line and rotating it inversely according to the straight line angle in a preferred embodiment of the present invention;

[0051] Figure 7 Point clouds of four pallets detected in a preferred embodiment of the present invention;

[0052] Figure 8 It is an infrared image of an actual scene image in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0053] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0054] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several modified improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0055] Reference Figure 1 As shown, a flowchart of a cargo pallet detection system and method based on a depth camera includes the following steps:

[0056] 1) When the cargo pallet detection system receives the AGV pallet detection instruction, it determines whether the camera is working normally. If the camera obtains the point cloud normally, it executes the next step. Otherwise, it repeats step 1) for a set number of times and terminates the task;

[0057] In step 1), the camera is installed at the top of the detection space rotated 15° relative to the horizontal plane, and the distance between the pallet and the camera is less than 5 meters, and the height difference between the pallet and the camera is less than 2 meters.

[0058] In step 1), it is necessary to determine whether the camera is working normally. If the camera can obtain point cloud normally, the point cloud will be converted to the AGV body center coordinate system according to the camera external parameter calibration parameters. If the camera cannot obtain point cloud, it is determined that the camera is disconnected or the data acquisition has timed out. Try to reconnect the camera or reacquire data and obtain the camera working status again. If it is still abnormal, terminate the task and issue a warning prompt.

[0059] 2) According to the set area, the point cloud is intercepted and divided into three-dimensional grids, and the ROI area is obtained according to the three-dimensional grid interception, specifically:

[0060] 2-1) According to the set area, the point cloud is intercepted, and the plane fitting is performed on the intercepted area point cloud. The height Z from the plane to the camera is calculated and compared with the calibrated height to determine whether the pallet is on the ground, and the ground point cloud is filtered. After filtering the ground point cloud, the ROI area is intercepted;

[0061] Step 2-1), specifically:

[0062] According to the configuration of the parameter file, the ground point cloud area parameters xmin, xmax, ymin, ymax, zmin, zmax are intercepted, and the three-dimensional point cloud is intercepted according to these parameters;

[0063] Use the RANSAC algorithm to fit the plane and determine the plane normal vector Is the angle between the plane and the unit vector (0,0,1) less than the threshold? If so, add up the Z coordinates of the points on the plane and divide by the number of points to get the height Z of the plane from the ground. average ;

[0064] Compare Z average And parameter file calibration height Z calibrate Is the difference less than the threshold? If so, the pallet is on the ground and the interception is greater than Z average The point cloud and the detection area parameters xmin, xmax, ymin, ymax, zmin, zmax configured in the parameter file are used to obtain the ROI area detection tray;

[0065] 2-2) If the pallet is detected on the ground, the point cloud is divided into three-dimensional grids. The grid size is set according to the actual situation, and the point cloud number and boundary information in the grid are saved. The upper boundary of each grid and the ground height z are compared, and the grids that are less than the threshold are eliminated.

[0066] 3) The three-dimensional grid in step 2) is roughly segmented to obtain point cloud blocks divided into three-dimensional grids; the segmented point cloud is finely segmented to obtain the width and height of each point cloud block, specifically:

[0067] 3-1) The obtained three-dimensional grid is roughly segmented along the y-axis and x-axis of the point cloud. When the grid point cloud is empty, segmentation is performed to filter out the point cloud blocks that do not meet the size requirements;

[0068] 3-2) The segmented point cloud is re-segmented by the fast Euclidean clustering algorithm. After segmentation, the width and height of each point cloud are calculated, and the point cloud blocks that do not meet the requirements are removed;

[0069] Calculate the point cloud block boundary values ​​xmin, xmax, ymin, ymax, zmin, zmax, and the point cloud block height height = z max -z min , width width = y max -y min If the height is less than 10cm or the width is less than 50cm, the point cloud block will be removed.

[0070] 4) Traverse the clustered point cloud blocks, use the edge extraction algorithm to obtain each point cloud block, perform plane projection, and divide it into a two-dimensional grid; specifically:

[0071] 4-1) Traverse the clustered point cloud blocks, extract the edge of each point cloud block, fit a straight line, roughly estimate the angle of the current point cloud block relative to the y-axis, and reversely rotate the point cloud according to the angle between the front surface of the tray and the YOZ plane;

[0072] 4-2) Project each point cloud block to the YOZ plane and divide the point cloud into two-dimensional grids. According to the set value, divide the point cloud in the point cloud block into two-dimensional grids, and record the boundary information and point cloud number;

[0073] 4-3) Search along the increasing direction of the z-axis and traverse the grid along the decreasing direction of the y-axis. When the grid point cloud is empty, segment it and record the point cloud number and boundary information of each block after segmentation to realize the division of the point cloud into a two-dimensional grid.

[0074] 5) Search for the pallet in the two-dimensional grid and solve the pallet position;

[0075] 5-1) Search for the tray in the two-dimensional grid, specifically:

[0076] Traverse the two-dimensional grid along the increasing direction of the Z axis, and search for the pallet in the two-dimensional grid, and search for the three-leg pallet according to the parameter file configuration; if the conditions are met, mark the starting sequence number of the pallet, and when the number of the current layer block is one, mark the end of the search for a pallet;

[0077] The parameter file is configured to determine the number of pallet legs of the pallet to be searched.

[0078] 5-2) Calculate the pallet posture, specifically:

[0079] 5-2-1) The pallet point cloud is searched, and the edge is extracted by edge extraction algorithm, and then the straight line is fitted to calculate the angle of the pallet;

[0080] a. Project the point cloud onto the XOY plane, traverse each point P(x,y), and use the formula atan(x / y) to get the angle theta of point P relative to the X axis and calculate If the angle difference between P0 and P1 is 0.1 degrees, compare the d of the two points. p , keep d p Smaller points get the edge of the point cloud;

[0081] b. Fit the straight line Ax+By=0 by RANSAC algorithm, and the tray angle is atan(B / A);

[0082] 5-2-2) Calculate the left and right deviations through the edge of the pallet and the pallet hole, calculate the depth through the edge, and calculate the height through the upper and lower edges of the pallet;

[0083] Traverse all point clouds to obtain the upper and lower boundary values ​​of the point clouds and the height value of the pallet, that is:

[0084] h=(z min +z max )

[0085] Among them, z min is the lower boundary value, z max is the upper boundary value.

[0086] 6) If multiple pallets are detected, they are sorted by Euclidean distance and the position of the pallet closest to the AGV is reported.

[0087] Embodiment 1:

[0088] Step 1: Specifically, when the system receives the AGV pallet inspection instruction, it first determines whether the camera is working normally. If the camera is disconnected or the data acquisition timeout occurs, it tries to reconnect the camera or reacquire data, and obtain the camera working status again. If it is still abnormal, it terminates the task and issues a warning prompt.

[0089] Step 2: Convert the acquired point cloud to the AGV body center coordinate system (the camera is installed with 15 degrees downward rotation) according to the camera external parameter calibration parameters. Figure 2 As shown, the white point cloud is the original point cloud, and the blue point cloud is the point cloud after the camera extrinsic calibration parameters are installed and rotated;

[0090] Step 3: Intercept the point cloud according to the set area, perform plane fitting on the intercepted area point cloud, calculate the plane to camera height Z and compare it with the calibration height to determine whether the pallet is on the ground, and filter the ground point cloud. After filtering the ground, intercept the ROI area, refer to Figure 3The figure shows the point cloud of the ROI area after filtering the ground. The blue point cloud is the ROI area, and the white point cloud is the filtered point cloud.

[0091] Step 4: After confirming that the detection tray is on the ground in step 3, the point cloud is divided into three-dimensional grids. The grid size is set according to the actual situation, and the point cloud number and boundary information in the grid are saved. The upper boundary of each grid and the ground height z are compared, and the grids with a height less than the threshold are eliminated;

[0092] Step 5: Use the 3D grid obtained in step 4 to roughly segment the point cloud along the y-axis and x-axis. When the grid point cloud is empty, segment it and filter out the point cloud blocks that do not meet the size requirements. Figure 4 As shown, the blue point cloud and the red point cloud are the point cloud blocks after rough segmentation, and the white point cloud is the filtered point cloud;

[0093] Step 6: Traverse the point cloud blocks obtained in step 4 and perform fast Euclidean clustering respectively. After clustering, filter out the point cloud blocks that do not meet the size. Figure 5 As shown;

[0094] Step 7: Traverse the clustered point cloud blocks, extract the edges of each point cloud block, and fit a straight line to roughly estimate the angle of the current point cloud block relative to the y-axis, and reversely rotate the point cloud according to the changed angle. Figure 6 As shown, the blue one is the point cloud after reverse rotation, and the white one is the original point cloud;

[0095] Step 8: Project to the YOZ plane and divide the point cloud into two-dimensional grids. According to the set value, divide the point cloud in the point cloud block into two-dimensional grids, and record the boundary information and point cloud number;

[0096] Step 9: Search along the increasing direction of the z-axis and traverse the grid along the decreasing direction of the y-axis. When the grid point cloud is empty, segment it and record the point cloud number and boundary information of each block after segmentation.

[0097] Step 10: Increase along the Z axis, traverse the two-dimensional grid in step 8, search for the tray, and search for the three-leg tray according to the parameter file configuration. Figure 7 As shown;

[0098] Step 11: Calculate the position and posture of the pallet. Search for the pallet point cloud, extract the edge through the edge extraction algorithm, and then fit the straight line to infer the pallet angle. Infer the left and right deviations through the pallet edge and the pallet hole, infer the depth through the edge, and infer the height through the upper and lower edges of the pallet.

[0099] Step 12: If multiple pallets are detected, they are sorted by Euclidean distance and the position of the pallet closest to the AGV is reported.

[0100] According to the process shown in the embodiment, the engineering deployment of the present invention is simple, and the position of the pallet can be located by only configuring the detection range, and is suitable for pallets of various colors and specifications.

[0101] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A cargo pallet detection method based on a depth camera, characterized in that: The following steps are involved: 1) When the cargo pallet detection system receives the AGV pallet detection instruction, it determines whether the camera is working normally. If the camera obtains the point cloud normally, it executes the next step. Otherwise, it repeats step 1) for a set number of times and terminates the task; 2) Intercept the point cloud according to the set area, divide it into three-dimensional grids, and obtain the ROI area according to the three-dimensional grid interception; 3) The three-dimensional grid in step 2) is roughly segmented to obtain point cloud blocks divided into the three-dimensional grid; the segmented point cloud is finely segmented to obtain the width and height of each point cloud block; 4) Traverse the clustered point cloud blocks, use the edge extraction algorithm to obtain each point cloud block, perform plane projection, and divide it into a two-dimensional grid; 5) Search for the pallet in the two-dimensional grid and solve the pallet position; 6) If multiple pallets are detected, they are sorted by Euclidean distance and the position of the pallet closest to the AGV is reported.

2. The method for detecting a cargo pallet based on a depth camera according to claim 1, characterized in that: The camera is installed at the top of the detection space rotated to 15° relative to the horizontal plane, and the distance between the pallet and the camera is less than 5 meters, and the height difference between the pallet and the camera is less than 2 meters.

3. The method for detecting a cargo pallet based on a depth camera according to claim 1, characterized in that: The step 1) is specifically: Determine whether the camera is working normally. If the camera can obtain point cloud normally, convert the point cloud to the AGV body center coordinate system according to the camera external parameter calibration parameters. If the camera cannot obtain point cloud, determine that the camera is disconnected or the data acquisition has timed out, try to reconnect the camera or reacquire data, and obtain the camera working status again. If it is still abnormal, terminate the task and issue a warning prompt.

4. The method for detecting a cargo pallet based on a depth camera according to claim 1, characterized in that: The step 2) is specifically: 2-1) According to the set area, the point cloud is intercepted, and the plane fitting is performed on the intercepted area point cloud. The height Z from the plane to the camera is calculated and compared with the calibrated height to determine whether the pallet is on the ground, and the ground point cloud is filtered. After filtering the ground point cloud, the ROI area is intercepted; Step 2-1), specifically: According to the configuration of the parameter file, the ground point cloud area parameters xmin, xmax, ymin, ymax, zmin, zmax are intercepted, and the three-dimensional point cloud is intercepted according to these parameters; Use the RANSAC algorithm to fit the plane and determine the plane normal vector Is the angle between the plane and the unit vector (0,0,1) less than the threshold? If so, add up the Z coordinates of the points on the plane and divide by the number of points to get the height Z of the plane from the ground. average ; Compare Z average And parameter file calibration height Z calibrate Is the difference less than the threshold? If so, the pallet is on the ground and the interception is greater than Z average The point cloud and the detection area parameters xmin, xmax, ymin, ymax, zmin, zmax configured in the parameter file are used to obtain the ROI area detection tray; 2-2) If the pallet is detected on the ground, the point cloud is divided into three-dimensional grids. The grid size is set according to the actual situation, and the point cloud number and boundary information in the grid are saved. The upper boundary of each grid and the ground height z are compared, and the grids that are less than the threshold are eliminated.

5. The method for detecting a cargo pallet based on a depth camera according to claim 1, characterized in that: The step 3) is specifically: 3-1) The obtained three-dimensional grid is roughly segmented along the y-axis and x-axis of the point cloud. When the grid point cloud is empty, segmentation is performed to filter out the point cloud blocks that do not meet the size requirements; 3-2) The segmented point cloud is re-segmented by the fast Euclidean clustering algorithm. After segmentation, the width and height of each point cloud are calculated, and the point cloud blocks that do not meet the requirements are removed; Calculate the point cloud block boundary values ​​xmin, xmax, ymin, ymax, zmin, zmax, Point cloud block height height = z max -z min , width width = y max -y min If the height is less than 10cm or the width is less than 50cm, the point cloud block will be removed.

6. The method for detecting a cargo pallet based on a depth camera according to claim 1, characterized in that: The step 4) is specifically: 4-1) Traverse the clustered point cloud blocks, extract the edge of each point cloud block, fit a straight line, roughly estimate the angle of the current point cloud block relative to the y-axis, and reversely rotate the point cloud according to the angle between the front surface of the tray and the YOZ plane; 4-2) Project each point cloud block to the YOZ plane and divide the point cloud into two-dimensional grids. According to the set value, divide the point cloud in the point cloud block into two-dimensional grids, and record the boundary information and point cloud number; 4-3) Search along the increasing direction of the z-axis and traverse the grid along the decreasing direction of the y-axis. When the grid point cloud is empty, segment it and record the point cloud number and boundary information of each block after segmentation to realize the division of the point cloud into a two-dimensional grid.

7. The method for detecting a cargo pallet based on a depth camera according to claim 1, characterized in that: In step 5), the searching for the tray in the two-dimensional grid is specifically as follows: Traverse the two-dimensional grid along the increasing direction of the Z axis, and search for the pallet in the two-dimensional grid, and search for the three-leg pallet according to the parameter file configuration; if the conditions are met, mark the starting sequence number of the pallet, and when the number of the current layer block is one, mark the end of the search for a pallet; The parameter file is configured to determine the number of pallet legs of the pallet to be searched.

8. The method for detecting a cargo pallet based on a depth camera according to claim 1, characterized in that: In step 5), the pallet posture is solved as follows: 5-1) The searched pallet point cloud is used to extract the edge through the edge extraction algorithm and then fit the straight line to calculate the pallet angle; a. Project the point cloud onto the XOY plane, traverse each point P(x,y), and use the formula atan(x / y) to get the angle theta of point P relative to the X axis and calculate If the angle difference between P0 and P1 is 0.1 degrees, compare the d of the two points. p , keep d p Smaller points get the edge of the point cloud; b. Fit the straight line Ax+By=0 by RANSAC algorithm, and the tray angle is atan(B / A); 5-2) Calculate the left and right deviations through the edge of the pallet and the hole of the pallet, calculate the depth through the edge, and calculate the height through the upper and lower edges of the pallet; Traverse all point clouds to obtain the upper and lower boundary values ​​of the point clouds and the height value of the pallet, that is: h=(z min +z max ) Among them, z min is the lower boundary value, z max is the upper boundary value.

9. A cargo pallet detection system based on a depth camera, characterized in that: including memory and processor; The memory is used to store a computer program; the processor is used to implement a cargo pallet detection method based on a depth camera as described in any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method for detecting a cargo pallet based on a depth camera as described in any one of claims 1 to 8 is implemented.

Images