A tray pose recognition method and system based on laser radar

Abstract

The application discloses a tray pose recognition method and system based on a laser radar, generates warehouse position column information to perform more fine point cloud filtering, and also utilizes point cloud clustering, target tracking and a multi-frame accumulation method to perform rough pose judgment based on the warehouse position column information, so that the tray pose is recognized based on the laser radar. The application is not affected by light, can work in strong light, backlight or a light and dark joint scene, does not need training data, can recognize different sizes of storage cages and trays, and has higher pose recognition accuracy and position recognition accuracy. Through use of the application, an ARM handling robot can autonomously pick up goods, and manpower is saved.

Description

Technical Field

[0001] This invention relates to the field of AMR (Automatic Mobile Transport) handling, and in particular to a pallet pose recognition method and system based on lidar. Background Technology

[0002] Autonomous Mobile Robots (AMRs) are equipped with automatic navigation and sensing systems that allow them to travel along pre-planned routes and navigate around obstacles. In industrial applications, AMRs do not require a driver and are typically powered by batteries. They perceive their surroundings through sensors such as lasers and vision devices, and their movement and behavior are controlled by industrial computers and electronic control systems. Compared to other manual logistics systems, AMR systems offer advantages such as strong environmental adaptability, high automation, and unrestricted operating hours, making them significant for improving production efficiency and reducing costs for enterprises.

[0003] Current warehouse scheduling systems tell AMRs (Autonomous Mobile Transporters) where to pick up or unload goods, but they don't tell them the goods' pose (x, y, yaw). To achieve this, additional sensors are usually installed on the AMR to obtain the pallet's pose. Common sensors include RGB-D depth cameras or monocular cameras. For example, using an RGB-D depth camera to locate and identify the pallet's pose is susceptible to lighting conditions, especially in strong light, backlight, or scenes with contrasting light and dark areas. This can affect image or point cloud accuracy, leading to pallet localization failures and incorrect pose recognition. Alternatively, deep learning methods can be used, but these require a large amount of data for model training and a high-performance computing platform on the AMR, resulting in extremely high costs. Encountering untrained pallets or scenarios will also lead to pallet localization failures. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a pallet pose recognition method and system based on lidar.

[0005] The objective of this invention is achieved through the following technical means: a pallet pose recognition method based on lidar, the steps of which are as follows:

[0006] Step 1: Receive external dependency information, including point cloud, localization, warehouse position information, and pallet pose detection signal;

[0007] Step 2: Initialize the storage location column information. Generate the storage location column and vehicle coordinate system based on the storage location column information received from the scheduling system, which includes the storage location start point, end point and column width information; perform system initialization, including initializing target tracking information and initializing pose detection information.

[0008] Step 3: Point cloud filtering, filtering out point clouds that are on the vehicle body, point clouds outside the library position column, and discrete point clouds;

[0009] Step 4: Point cloud clustering. Based on the angular resolution characteristics of the radar, generate three-dimensional voxels; project the point cloud onto the voxels, divide the voxels into different categories, and calculate the contour convex hull of each category.

[0010] Step 5: Track the tray tip in the previous and next frames; the calculation of the tray pose requires information accumulated over multiple frames, so the tray tip needs to be tracked in the previous and next frames; if the contour convex hull of the current frame matches the previous frame, the target is considered to be successfully tracked, the tracking list information is updated, it is recorded as the tray tip, and the position and pose of the target are filtered; if the tracking is unsuccessful, the target is simply added to the tracking list, and the parameters of the extended Kalman tracking model are initialized.

[0011] Step 6: Secondary filtering of invalid point clouds. When calculating the tray pose, the point cloud of the outer end face of the tray is required. However, the point cloud will be projected onto the inner end face of the tray, so the point cloud of the inner end face needs to be filtered out. Based on the horizontal and vertical projection methods, the inner and outer end faces of the tray are found; the point cloud of the inner end face of the tray is filtered out, and the filtered tracking list information is reconstructed, including convex hull information, length and width information;

[0012] Step 7: Coarse pose determination. Set the maximum allowable angle threshold for pallet placement. Determine if the pallet pose meets the requirement that the angle between the line fitted by Ransac and the central axis of the storage location is less than the maximum allowable angle threshold for pallet placement. If it is greater than the threshold, it is determined that the pose requirement is not met, and the detection cycle of the next frame starts from Step 3. Otherwise, it is determined that the requirement is met; proceed to Step 8.

[0013] Step 8: Pose fine calculation. Based on the pallet toe point cloud that meets the requirements, the pose and midpoint of the pallet are calculated using least squares and multi-frame accumulation.

[0014] Furthermore, the method for generating the library position column in step 2 is as follows: establish a vehicle coordinate system, with the y-axis pointing forward along the front of the vehicle and the x-axis pointing to the right in the tangential direction to the front of the vehicle.

[0015] Assume the storage location starts at A(x0, y0), ends at B(x1, y1), and the unit vector from the end point to the start point is v(x, y). The two normal vectors of v are v1(-y, x) and v2(y, -x). The column width is w. A quadrilateral is generated, A0A1B1B0. The x and y coordinates of the four vertices of the quadrilateral are:

[0016] A0·x=x0+(-y*w / 2) A0·y=y0+(x*w / 2)

[0017] A1·x=x0+(y*w / 2) A1·y=x1+(-x*w / 2)

[0018] B0·x=x0+(-y*w / 2) B0·y=y0+(x*w / 2)

[0019] B1·x=x0+(y*w / 2) B1·y=x1+(-x*w / 2).

[0020] Furthermore, the specific steps for classifying voxels into different categories in step 4 are as follows: randomly select a voxel containing point clouds, search the four voxels adjacent to this voxel (front, back, left, and back), and if the adjacent voxels contain point clouds, classify them into the same category; then, starting from the search of the adjacent voxel, search the four voxels in front, back, left, and right, and classify the voxels containing point clouds into the same category, repeat this step until the four adjacent voxels do not contain point clouds.

[0021] Furthermore, the filtering in step 5 is based on the extended Kalman filter model, which filters the position and pose of the target.

[0022] Furthermore, the point cloud filtering module includes: vehicle filtering, library position column filtering, and radius filtering;

[0023] The self-vehicle filtering: filters out the point cloud that hits the self-vehicle.

[0024] The storage location column filtering: filters out point clouds outside of storage locations, that is, filters out point clouds outside of quadrilateral A0A1B1B0.

[0025] The radius filtering is used to filter discrete points; with any point P as the center and r as the radius, if the number of remaining points whose distance to point P is less than r is N, and if N is less than 5, then point P is considered a discrete point and needs to be filtered out.

[0026] Furthermore, the coarse pose determination includes:

[0027] Based on the Ransac algorithm, a preliminary determination is made as to whether the pallet pose meets the requirements and whether the pallet is a storage cage. That is, the angle between the straight line fitted by Ransac and the central axis of the storage location is less than the maximum allowable pose of the pallet.

[0028] The center points of all targets are projected onto the central axis of the storage location and sorted from closest to farthest from the starting point of the storage location.

[0029] Select the point pt closest to the starting point of the storage location;

[0030] Select point clouds from all target objects whose distance from pt is less than the maximum distance maxDist. maxDist is calculated as: storage space width * sin(maximum allowed pallet angle threshold).

[0031] The point cloud that meets the requirements in the above operations is fitted with a Ransac line to obtain the starting point and unit vector of the line that meet the requirements; the number of pallet toes on the line formed by the starting point and unit vector determines whether it is a regular pallet with three pallet toes or a storage cage with two pallet toes.

[0032] Furthermore, the specific steps for calculating the pose and midpoint of the tray using least squares and multi-frame accumulation in the precise pose calculation are as follows:

[0033] Step 1: Use the least squares method to fit the point cloud equation y = kx + b on the tray tip of the straight line formed by the starting point of the straight line and the unit vector in step 6, where k is the pose yaw of the tray, and save the result in the pose list.

[0034] Step 2: Calculate the mean (x', y') of all point clouds on the tray toe along the line formed by the starting point of the line in Step 6 and the unit vector, and save the result to the pose list.

[0035] Step 3: Set a pose list threshold, repeat steps 1 and 2. When the number of poses in the pose list is greater than the pose list threshold, calculate the mean of yaw and (x', y') in the pose list. The calculation result is the final pose and midpoint of the tray.

[0036] The present invention also provides a pallet pose recognition system based on lidar, the system being implemented using the method described in any one of claims 1-7, characterized in that the system comprises: a data receiving module, a data sending module, a warehouse position information initialization module, a point cloud filtering module, a point cloud clustering module, a target tracking module, and a pallet pose detection module;

[0037] The data receiving module receives external dependent information via the network, including point cloud, positioning, warehouse location information, and parking detection signals.

[0038] The data sending module is used to send the cargo pose (x, y, yaw) identified by the network to the control system to execute the cargo retrieval operation;

[0039] The storage location column information initialization module generates storage locations based on column information received from the warehouse scheduling system. The coordinates of the four vertices of the storage location are A0A1B1B0.

[0040] The point cloud filtering module is designed to filter out invalid point clouds, thereby accelerating the processing efficiency of subsequent steps; it includes vehicle filtering, library position filtering, and radius filtering.

[0041] The point cloud clustering module is based on the three-dimensional voxel method. According to the angular resolution characteristics of the radar, three-dimensional voxels are generated. The point cloud is projected into the voxels. Adjacent voxels belong to the same category. Finally, the contour convex hull of each category is calculated for subsequent target tracking modules.

[0042] The target tracking module is used to obtain the accumulated information from multiple frames required for pallet pose calculation and to track the target in the previous and next frames. If the contour convex hull of the current frame matches the clustered target convex hull of the previous frame, the target is considered to be successfully tracked, the tracking list information is updated, it is recorded as the pallet toe, and the position and pose of the target are filtered; if the tracking is unsuccessful, the target is simply added to the tracking list and the parameters of the extended Kalman tracking model are initialized.

[0043] The pallet pose detection module is used to filter out invalid point clouds, perform coarse pose judgment and fine pose calculation, and can detect the pose of the storage cage with two legs and the pallet with three legs based on the lidar installed on the fork tip.

[0044] The pallet pose detection module can be divided into lateral and longitudinal detection based on the distance between the pallet and the vehicle, and sends the detection results to the control system for picking.

[0045] Furthermore, the control system is used to plan driving routes and control vehicle operation.

[0046] Furthermore, the pallet pose detection module determines the lateral and longitudinal poses as follows: when the vehicle is less than 1.5 meters away from the goods, the pallet pose is identified laterally; when the vehicle is more than 1.5 meters away from the goods, the pallet pose is identified longitudinally.

[0047] The beneficial effects of this invention are:

[0048] This invention addresses the shortcomings of current pallet or storage cage pose recognition systems for AMR (Autonomous Mobile Robot) handling robots by proposing a method and system for recognizing cargo pose based on LiDAR (Light Detection and Ranging). This method is unaffected by lighting conditions and requires no training data. The system can recognize storage cages and pallets of different sizes, with pose recognition accuracy ≤1° and position recognition accuracy ≤1cm. By using this invention, AMR handling robots can autonomously retrieve goods, saving manpower. Attached Figure Description

[0049] Figure 1 The flowchart shows a pallet pose recognition method based on LiDAR.

[0050] Figure 2 This is a schematic diagram of the vehicle coordinate system;

[0051] Figure 3 The quadrilateral for the storage location is generated based on column information;

[0052] Figure 4 A voxel diagram;

[0053] Figure 5 This is a schematic diagram of pose recognition based on LiDAR. Detailed Implementation

[0054] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0055] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0056] like Figure 1 As shown, the present invention provides a pallet pose recognition method based on LiDAR, the steps of which are as follows:

[0057] Step 1: Receive external dependency information via network, including LiDAR point cloud, positioning, warehouse position information, and pallet pose detection signal;

[0058] Step 2: Initialize warehouse location column information. Based on the warehouse location information received from the scheduling system, a warehouse location column is generated. The warehouse location information includes the warehouse location start point, end point, and warehouse location width. For example... Figure 2 As shown, the vehicle coordinate system has the y-axis pointing forward from the front of the vehicle and the x-axis pointing to the right in the tangential direction to the front of the vehicle; the four vertices of the storage location column generated based on Formula 1 in the vehicle coordinate system are A0A1B1B0.

[0059] Let the starting point of the storage location be A(x0,y0), the ending point be B(x1,y1), the unit vector from the ending point to the starting point be v(x,y), and the two normal vectors of v be v1(-y,x) and v2(y,-x). Let the column width be w. The following uses Formula 1 to generate the four vertices of the storage location column as A0A1B1B0, as follows: Figure 3 As shown.

[0060]

[0061] Step 3: Perform system initialization, including initializing target tracking information, initializing pose detection information, and point cloud filtering to filter out point clouds hitting the vehicle body, point clouds outside the library positions, and discrete point clouds.

[0062] The self-vehicle filtering: filters out the point cloud that hits the self-vehicle.

[0063] The storage location filtering filters out point clouds other than storage locations, specifically filtering out point clouds other than quadrilateral A0A1B1B0. The coordinates of the four vertices of quadrilateral A0A1B1B0 are distributed as A0(x... 11 y 11 ),A1(x 22 y 22 ),B1(x 33 y 33 ),B0(x 44 y 44 Arbitrary point cloud Pt(x) after self-filtering. 55 y 55 Let vector with vector cross product dot0, vector with vector cross product dot1, vector with vector cross product dot2, vector with vector The cross product of dot3. If dot0*dot1*dot2*dot3>0, then point Pt is considered to be inside quadrilateral A0A1B1B0 and needs to be retained; if dot0*dot1*dot2*dot3≤0, then point Pt is outside quadrilateral A0A1B1B0 and needs to be filtered out. Where the vector... The calculation is shown in Formula 2. The calculation of the cross product dot0, dot1, dot2, and dot3 is shown in Formula 3.

[0064]

[0065]

[0066] The radius filtering method filters discrete points. Using any point P as the center and r as the radius, let N be the number of remaining points whose distance to point P is less than r. If N is less than 5, then point P is considered a discrete point and needs to be filtered out. The specific steps are as follows:

[0067] Assuming that the set of all point clouds retained after vehicle filtering and parking location filtering is PointCloudA, we first construct a binary tree for PointCloudA. This method uses a KD-tree.

[0068] This method uses a front-line LiDAR with only two dimensions, x and y. First, it calculates the variance of the point cloud in PointCloudA along both the x and y directions, and determines the dimension with the largest variance, assuming it's in the x-direction. Then, it finds the median x of all data items along dimension x. a Press x aDivide the dataset into two subsets, Sa and Sb. Create tree nodes to store the details of this partition, specifically recording the dimension x and the median x. a .

[0069] Repeat the above division for Sa and Sb, and set the newly generated tree nodes as the left and right child nodes of the previous division;

[0070] Perform the above two steps recursively until it can no longer be divided.

[0071] Select any point P from PointCloudA. Starting from the root node of the KD-tree, calculate the distance between point P and the nodes in the KD-tree. If the distance is less than r = 0.05 meters, increment the number N of points with a distance less than r from point P until a leaf node of the KD-tree is reached. If N < 5, point P is considered a discrete point and needs to be filtered out. When N ≥ 5, point P is retained. Repeat this step until all point clouds used in PointCloudA are included in the calculation.

[0072] Step 4: The point cloud set filtered in Step 3 is PointCloudB. Point cloud clustering is performed, and 3D voxels are generated based on the radar's angular resolution characteristics. A radar scan is 360°, the radar angular resolution is 0.2°, and voxels are generated at a step size of 0.1 meters. Figure 4 As shown; the point cloud is projected onto voxels, and the voxels are divided into different categories. Neighboring voxels belong to the same category. Finally, the convex hull of the contour is calculated based on the point cloud of each category. The steps for calculating the convex hull are as follows:

[0073] Step ①: Select the point cloud with the smallest x-coordinate within the category as the pole (if the x-coordinates are the same, select the point with the smallest y-coordinate). This point must lie on the convex hull.

[0074] Step 2: Sort the remaining point clouds by polar angle. If the polar angles are the same, compare the distances to the poles, prioritizing those closer to the poles.

[0075] Step 3: Use a queue S to store the points on the convex hull. First, push the two points with the smallest polar angle and pole size onto the stack.

[0076] Step 4: Scan each point as needed and check whether the line segment formed by the first two elements of queue S and this point "turns" to the right (cross product ≤ 0).

[0077] Step 5: If satisfied, pop the first element from queue S and return to step 4 to check again, until satisfied. Store the point in queue S, and continuously perform step 5 on other points.

[0078] Step 6: The final queue S contains a sequence of vertices of the convex hull.

[0079] Step 5: Perform target tracking on the tray tip in the previous and next frames; if the contour convex hull of the current frame intersects with the contour convex hull of the previous frame in the vehicle coordinate system, the target tracking is considered successful, the tracking list information is updated, and the position and pose of the target are filtered based on the extended Kalman filter model; if the tracking is unsuccessful, only the tray tip is added to the tracking list, and the extended Kalman tracking model parameters are initialized.

[0080] Step 6: Filter out invalid point clouds. When calculating the pallet pose, the point cloud of the outer end face of the pallet is needed, but the point cloud will be projected onto the inner end face of the pallet. Therefore, the point cloud of the inner end face needs to be filtered out. This method uses horizontal and vertical projection to filter out the point cloud of the inner end face of the pallet. First, project all the point clouds in the pallet feet from Step 5 onto the straight line formed by the start and end points of the storage location. Find the point Pt0 closest to the start point of the storage location. If the distance between the remaining point and Pt0 is greater than 0.03 meters, it is considered to be the point cloud of the inner end face of the pallet and needs to be filtered out. Reconstruct the pallet foot information based on the filtered point cloud, such as convex hull, length, and width information.

[0081] Step 7: Coarse pose determination. Based on the Ransac algorithm, a preliminary determination is made as to whether the pallet pose meets the requirements. The number of pallet toes on the straight line formed by the starting point and the unit vector determines whether it is a pallet (3 toes) or a storage cage (2 toes). A maximum allowed pallet placement angle threshold θ is set; in this method, it is 25°. The pallet toes from Step 6 are sorted from closest to furthest from the storage location starting point. The point Pt1 closest to the storage location starting point is selected. From all pallet toes, point clouds with a distance less than maxDist from Pt1 are selected and stored in the point cloud set PointCloudC. maxDist is the maximum distance from the point cloud to point Pt1, calculated as: storage location width w * sin(θ).

[0082] The detailed steps for performing Ransac line fitting on the point cloud set PointCloudC are as follows:

[0083] Step ①: Randomly select two points Pt from the point cloud set. a and Pt b

[0084] Step 2: Calculate Pt a and Pt b The unit vector v that makes up the straight line ab =(x ab y ab ), unit vector v ab and point Pt a Let Pt be a point a and Pt b The straight line l formed by two points ab parameter.

[0085] Step 3: Calculate the distance from the remaining point cloud set PointCloudC to the line l. ab If the distance is less than 0.05 meters, the point is considered to be on line l. ab Nearby points, line l ab Count of nearby points ab Add 1.

[0086] Step 4: Repeat steps 1 to 3 until Count is reached. ab When the number of points is greater than half the number of points in the point cloud set PointCloudC, it indicates that the line l ab It is a valid straight line; calculate vector v. ab The angle θ0 between the pallet and the line formed by the start and end points of the pallet location is calculated. If θ0 is less than θ, it means a pallet pose that meets the requirements has been found. If θ0 is greater than θ, exceeding the maximum allowable angle threshold θ for pallet placement, the current detection is considered a failure, and the detection cycle for the next frame begins from step 3.

[0087] Step 8: Pose fine calculation. Based on the pallet toe point cloud that meets the requirements, the pose and midpoint of the pallet are calculated by least squares and multi-frame accumulation.

[0088] Step 1: Use the least squares method to fit the point cloud equation y = kx + b on the tray tip of the line formed by the starting point of the line in step 7 and the unit vector, where k is the pose yaw of the tray and b is a constant. Save the result in the pose list.

[0089] Step 2: Calculate the mean (x', y') of all point clouds on the tray toe along the line formed by the starting point of the line in Step 7 and the unit vector, and save the result to the pose list.

[0090] Step 3: Set the pose list threshold M = 10, repeat steps 1 and 2. When the number of poses in the pose list is greater than the pose list threshold, calculate the mean of yaw and (x', y') in the pose list. The calculation result is the final pose and midpoint of the tray.

[0091] According to an embodiment of the present invention, a pallet pose recognition system based on LiDAR is provided. The system includes: a data receiving module, a data sending module, a pallet position information initialization module, a point cloud filtering module, a point cloud clustering module, a target tracking module, and a pallet pose detection module.

[0092] The data receiving module receives external dependent information via the network, including point cloud, positioning, warehouse location information, and parking detection signals.

[0093] The data sending module is used to send the cargo pose (x, y, yaw) identified by the network to the control system to execute the cargo retrieval operation;

[0094] The storage location column information initialization module generates storage locations based on column information received from the warehouse scheduling system. The coordinates of the four vertices of the storage location are A0A1B1B0.

[0095] The point cloud filtering module is designed to filter out invalid point clouds, thereby accelerating the processing efficiency of subsequent steps; it includes vehicle filtering, library position filtering, and radius filtering.

[0096] The point cloud clustering module is based on the three-dimensional voxel method. According to the angular resolution characteristics of the radar, three-dimensional voxels are generated. The point cloud is projected into the voxels. Adjacent voxels belong to the same category. Finally, the contour convex hull of each category is calculated for subsequent target tracking modules.

[0097] In the target tracking module, the calculation of the tray pose requires information accumulated over multiple frames, so the target needs to be tracked in both the previous and next frames. If the contour convex hull of the current frame matches the clustered target convex hull of the previous frame, the target is considered to be successfully tracked, the tracking list information is updated, it is recorded as the tray toe, and the position and pose of the target are filtered. If the tracking is unsuccessful, the target is simply added to the tracking list and the parameters of the extended Kalman tracking model are initialized.

[0098] The pallet pose detection module is used to filter out invalid point clouds, perform coarse pose judgment and fine pose calculation, and can detect the pose of the storage cage with two legs and the pallet with three legs based on the lidar installed on the fork tip.

[0099] The pallet position detection module can be divided into lateral and longitudinal detection based on the distance between the pallet and the vehicle, and sends the detection results to the control system (used to plan the driving path and control the vehicle operation) for picking up goods.

[0100] This system can be divided into lateral and longitudinal detection based on the distance between the pallet and the vehicle, such as... Figure 5 As shown. When the vehicle is close to the goods (<1.5 meters), the pallet position is identified laterally; when the vehicle is far from the goods, the pallet position is identified longitudinally, and the detection result is sent to the control system for goods retrieval. The planning control then plans the vehicle's driving route to perform the goods retrieval operation.

[0101] The pallet pose detection module has an angle detection accuracy of ≤1° and a position detection accuracy of ≤2cm.

[0102] The above description is merely a preferred embodiment of the present invention. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make many possible variations and modifications to the technical solutions of the present invention using the methods and techniques disclosed above, or modify them into equivalent embodiments with equivalent changes, without departing from the scope of the technical solutions of the present invention. Therefore, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall still fall within the protection scope of the technical solutions of the present invention.

Claims

1. A pallet pose recognition method based on lidar, characterized in that, The steps of this method are as follows: Step 1: Receive external dependency information, including point cloud, localization, warehouse position information, and pallet pose detection signal; Step 2: Initialize the warehouse location column information. Generate the warehouse location column and vehicle coordinate system based on the warehouse location column information received from the scheduling system, which includes the warehouse location start point, end point and column width information; perform system initialization, including initializing target tracking information and initializing pose detection information. Step 3: Point cloud filtering, filtering out point clouds that are on the vehicle body, point clouds outside the library position column, and discrete point clouds; Step 4: Point cloud clustering. Based on the angular resolution characteristics of the radar, generate three-dimensional voxels; project the point cloud onto the voxels, divide the voxels into different categories, and calculate the contour convex hull of each category. Step 5: Perform target tracking on the tray tip in the previous and next frames; if the contour convex hull of the current frame matches the previous frame, the target tracking is considered successful, the tracking list information is updated, it is recorded as the tray tip, and the position and pose of the target are filtered; if the tracking is unsuccessful, the target is simply added to the tracking list, and the parameters of the extended Kalman tracking model are initialized. Step 6: Secondary filtering of invalid point clouds to find the inner and outer end faces of the tray; filter out the point cloud of the inner end face of the tray and reconstruct the filtered tracking list information, including convex hull information and length and width information; Step 7: Coarse pose determination. Set the maximum allowable angle threshold for pallet placement. Determine if the pallet pose meets the requirement that the angle between the line fitted by Ransac and the central axis of the storage location is less than the maximum allowable angle threshold for pallet placement. If it is greater than the threshold, it is determined that the pose requirement is not met, and the detection cycle of the next frame starts from Step 3. Otherwise, it is determined that the requirement is met; proceed to Step 8. Step 8: Pose fine calculation. Based on the pallet toe point cloud that meets the requirements, the pose and midpoint of the pallet are calculated using least squares and multi-frame accumulation.

2. The pallet pose recognition method based on lidar according to claim 1, characterized in that, The method for generating the library location column in step 2 is as follows: establish a vehicle coordinate system, with the y-axis pointing forward along the front of the vehicle and the x-axis pointing to the right in the tangential direction to the front of the vehicle. Assume the starting point of the storage location is A(x0,y0), the ending point is B(x1,y1), the unit vector from the ending point to the starting point is v(x,y), and the two normal vectors of v are v1(-y,x) and v2(y,-x). The column width is w. Generate a quadrilateral storage location column A0A1B1B0. The x and y coordinates of the four vertices of this column are as follows: A 0.x =x0+(-y*w / 2) A 0.y =y0+(x*w / 2) FLUENT 1.x =x0+(y*w / 2) A 1.y =x1+(-x*w / 2) B0.x=x0+(-y*w / 2) B 0.y =y0+(x*w / 2) B 1.x =x0+(y*w / 2) B 1.y =x1+(-x*w / 2)。 3. The pallet pose recognition method based on lidar according to claim 1, characterized in that, The specific steps for classifying voxels into different categories in step 4 are as follows: Randomly select a voxel containing point clouds, and search the four voxels adjacent to this voxel (front, back, left, and back). If the adjacent voxels contain point clouds, classify them into the same category. Then, starting from the voxel that was searched, search the four voxels that were adjacent to it (front, back, left, and right) and classify the voxels containing point clouds into the same category. Repeat this step until the four adjacent voxels no longer contain point clouds.

4. The pallet pose recognition method based on lidar according to claim 1, characterized in that, The filtering in step 5 is based on the extended Kalman filter model, which filters the position and pose of the target.

5. The pallet pose recognition method based on lidar according to claim 1, characterized in that, The point cloud filtering includes: vehicle filtering, library position filtering, and radius filtering; The vehicle-mounted filter: filters out the point cloud that hits the vehicle body; The storage location column filtering: filters out point clouds other than storage locations, that is, filters out point clouds other than quadrilateral A0A1B1B0. The radius filtering is used to filter discrete points; with any point P as the center and r as the radius, if the number of remaining points whose distance to point P is less than r is N, and if N is less than 5, then point P is considered a discrete point and needs to be filtered out.

6. The pallet pose recognition method based on lidar according to claim 1, characterized in that, The coarse pose determination includes: Based on the Ransac algorithm, it is initially determined whether the pallet pose meets the requirements and whether the pallet is a storage cage; that is, the angle between the straight line fitted by Ransac and the central axis of the storage location is less than the maximum allowable pose of the pallet. The center points of all targets are projected onto the central axis of the storage location and sorted from closest to farthest from the starting point of the storage location. Select the point pt closest to the starting point of the storage location; Select point clouds from all targets whose distance from pt is less than the maximum distance maxDist; maxDist is calculated as: warehouse width * sinθ, where θ represents the maximum allowable angle threshold for pallet placement; The point cloud that meets the requirements is fitted with a Ransac line to obtain the starting point and unit vector of the line that meets the requirements; the number of pallet toes on the line formed by the starting point and unit vector determines whether it is a regular pallet with three pallet toes or a storage cage with two pallet toes.

7. A pallet pose recognition method based on lidar according to claim 6, characterized in that, The specific steps for calculating the pose and midpoint of the tray using least squares and multi-frame accumulation in the precise pose calculation are as follows: Step 1: Use the least squares method to fit the point cloud equation y = kx + b on the tray toe on the straight line formed by the starting point of the straight line and the unit vector in step 6, where k is the pose yaw of the tray, and save the result in the pose list. Step 2: Calculate the mean (x', y') of all point clouds on the tray toe along the line formed by the starting point of the line in Step 6 and the unit vector, and save the result to the pose list. Step 3: Set a pose list threshold, repeat steps 1 and 2. When the number of poses in the pose list is greater than the pose list threshold, calculate the mean of yaw and (x', y') in the pose list. The calculation result is the final pose and midpoint of the tray.

8. A pallet pose recognition system based on lidar, the system being implemented using the method described in any one of claims 1-7, characterized in that, The system includes: a data receiving module, a data sending module, a library position information initialization module, a point cloud filtering module, a point cloud clustering module, a target tracking module, and a pallet pose detection module; The data receiving module receives external dependent information via the network, including point cloud, positioning, warehouse location information, and parking detection signals. The data sending module is used to send the cargo pose (x, y, yaw) identified by the network to the control system to execute the cargo retrieval operation; The storage location column information initialization module generates storage locations based on column information received from the warehouse scheduling system. The coordinates of the four vertices of the storage location are A0A1B1B0. The point cloud filtering module is designed to filter out invalid point clouds, thereby accelerating the processing efficiency of subsequent steps; it includes vehicle filtering, library position filtering, and radius filtering. The point cloud clustering module is based on the three-dimensional voxel method. According to the angular resolution characteristics of the radar, three-dimensional voxels are generated. The point cloud is projected into the voxels. Adjacent voxels belong to the same category. Finally, the contour convex hull of each category is calculated for subsequent target tracking modules. In the target tracking module, the calculation of the tray pose requires information accumulated over multiple frames, so the target needs to be tracked in both the previous and next frames. If the contour convex hull of the current frame matches the clustered target convex hull of the previous frame, the target is considered to be successfully tracked, the tracking list information is updated, it is recorded as the tray toe, and the position and pose of the target are filtered. If the tracking is unsuccessful, the target is simply added to the tracking list and the parameters of the extended Kalman tracking model are initialized. The pallet pose detection module is used to filter out invalid point clouds, perform coarse pose judgment and fine pose calculation, and can detect the pose of the storage cage with two legs and the pallet with three legs based on the lidar installed on the fork tip. The pallet position detection module can be divided into lateral and longitudinal detection based on the distance between the pallet and the vehicle, and sends the detection results to the control system for picking up the goods.

9. A pallet pose recognition system based on lidar according to claim 8, characterized in that, The control system is used to plan driving routes and control vehicle operation.

10. A pallet pose recognition system based on lidar according to claim 8, characterized in that, The pallet pose detection module determines the lateral and longitudinal poses as follows: when the vehicle is less than 1.5 meters away from the goods, the pallet pose is identified laterally; when the vehicle is more than 1.5 meters away from the goods, the pallet pose is identified longitudinally.

Images