Palletizing Robot Positioning Method and Palletizing System
By processing LiDAR point cloud data and normal vector clustering, accurate positioning of palletizing robots and quality inspection of container stacking were achieved, solving the problems of positioning and packing quality of palletizing robots in containers and improving the effectiveness and safety of path planning.
Patent Information
- Application Number
- CN202411229833.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2044-09-03
AI Technical Summary
In existing technologies, the positioning of palletizing robots inside containers is inaccurate, affecting the effectiveness and safety of path planning, and the quality of goods stacking after loading is difficult to guarantee.
Point cloud data is collected using LiDAR. Through point cloud stitching and preprocessing, combined with normal vector clustering and optimal random consistency algorithm, the planar position and coordinate system of each face of the container are determined, enabling accurate positioning of the palletizing robot. The container placement quality inspection method is used to determine whether the container is placed correctly.
This improves the positioning accuracy and packing quality of palletizing robots within containers, ensuring that the robots accurately reach the target point and complete high-quality cargo stacking.
Smart Images

Figure CN118877568B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of palletizing system technology, and in particular to a palletizing robot positioning method and palletizing system. Background Technology
[0002] Currently, domestic and international containerized cargo is mainly transported using large box trucks or containers. Due to space limitations in the truck / container compartments, most enterprises, companies, warehouses, and logistics centers now use semi-automated equipment or manual labor for cargo handling. Using palletizing robots to replace manual loading and unloading operations aligns with the future trend of factory intelligence and automation.
[0003] The accuracy of obtaining the palletizing robot's positioning information directly affects the effectiveness and safety of path planning, determining whether the robot can accurately reach the predetermined target point and perform subsequent operations. Therefore, accurately locating the palletizing robot's posture within the container is fundamental for it to complete the loading task. Furthermore, the quality of the stacking of goods after loading affects subsequent goods stacking and is equally important. Summary of the Invention
[0004] This application provides a method and system for positioning a palletizing robot, which aims to accurately locate the position and posture of the palletizing robot inside a container and improve palletizing quality.
[0005] Therefore, according to one aspect of this application, a palletizing robot positioning method is provided, comprising the following steps:
[0006] S1. After receiving the positioning command, the host computer collects point cloud data inside the container using lidar;
[0007] S2. The acquired point cloud is stitched together, the ROI region is extracted, and the extracted point cloud is preprocessed and saved to the cache queue;
[0008] S3. The point clouds of the five regions above, below, left, right and front of the lidar are extracted by direct filtering. The noise of the extracted point clouds is reduced. The point clouds of the top, front, left and right sides of the container are extracted by normal vector clustering. The cluster with the most point clouds is recorded as the point cloud of each container wall.
[0009] S4. During the initial positioning, the top, front, left, and right sides of the container are all wavy walls. Based on the point clouds of each wall obtained in step S3, calculate the centroid and feature vector of each wall point cloud. Using the centroid as a point on the plane and the feature vector pointing into the container as the plane normal vector, determine a plane. Move the plane along the direction of the normal vector until the number of point clouds on the plane is less than a set threshold, then stop moving the plane and use the position of the plane at this time as the position of the wall. If it is not the first positioning and there are stacked containers at the front of the container, then the front plane is not positioned first.
[0010] S5. To facilitate equipment entry and exit, a transition ramp is provided between the container and the ground during loading. Therefore, the inverse vector direction of the normal vector of the top plane of the container obtained in step S4 is used as the fixed direction. The point cloud of the bottom area of the container is clustered, the cluster with the most point cloud is extracted, and the optimal random consistency algorithm is used to fit the plane. The resulting plane position is used as the bottom position of the container.
[0011] S6. If this is not the first time positioning the container, and there are already stacked boxes at the front of the container, since it cannot be guaranteed that the plane formed by stacking the boxes is flat and perpendicular to the bottom surface of the container, special processing is required. That is, the cross product of the normal vectors of the left and bottom sides of the container wall obtained in the previous steps is used as the normal vector of the front face. The projection of the distance from each point on the front face to the origin of the lidar on the normal vector of the front face is calculated, and the average value is taken as the distance from the origin of the lidar to the front face. Then, the plane equation of the front face is determined.
[0012] S7. Using the intersection of the bottom, front, and left sides of the container as the origin of the container's coordinate system, the orientation of the container's coordinate system is determined by the direction of the normal vector of each plane. By calculating the transformation matrix between the container's coordinate system and the lidar's coordinate system, the pose of the lidar in the container's coordinate system is determined. By determining the installation positions of the palletizing robot and the lidar on the equipment, the pose of the palletizing robot inside the container is determined and returned to the palletizing robot, thus ending the positioning method.
[0013] According to another aspect of this application, a palletizing system is provided, including the palletizing robot positioning method described above.
[0014] Optionally, the palletizing system further includes a box placement quality detection method, which is used to detect whether the box has been successfully placed and whether the box is placed correctly after the palletizing robot has completed one placement and left the detection area of the lidar.
[0015] Optionally, the method for detecting the quality of box stacking includes the following steps:
[0016] Step 1: After receiving the detection command, the host computer collects point cloud data inside the container using LiDAR;
[0017] Step 2: Stitch together the acquired point cloud, extract the ROI region, and preprocess the extracted point cloud as the current point cloud;
[0018] Step 3: Read the most recent point cloud record from the cache queue as the target point cloud. Use K-nearest neighbor search, with K set to 1. Traverse the points in the current point cloud. If there is no point in the target point cloud that is less than the set threshold distance from the current point cloud, save the current point cloud as a difference point in the difference point cloud object.
[0019] Step 4: If the number of difference point clouds is less than the set threshold, it is considered that no boxes were placed in this detection, and the result of placement failure is returned, and the detection method ends; otherwise, continue to Step 5.
[0020] Step 5: In the difference point cloud, with the angle between the point cloud and the front face less than 30° as a condition and the distance threshold set to 10mm, perform plane fitting, extract the point cloud that is 30mm away from the plane in the difference point cloud, cluster the extracted point cloud according to the normal vector, and extract the cluster with the most point cloud as the planar point cloud of the box facing the lidar side.
[0021] Step 6: Calculate the centroid and eigenvector of the planar point cloud obtained in Step 5. Using the eigenvector as the coordinate direction of the planar point cloud and the centroid as the origin, calculate the maximum and minimum values of the planar point cloud in this coordinate system. Then, the centroid of the planar point cloud and the length, width, and height of the enclosing rectangle can be obtained.
[0022] Step 7: By calculating the transformation matrix between the planar coordinate system and the lidar coordinate system obtained in Step 6, the pose of the container in the lidar coordinate system is determined. Since the transformation matrix between the container coordinate system and the lidar coordinate system was calculated when positioning the container, the pose of the container in the container coordinate system can be obtained. Based on the pose of the container, it is determined whether the container is placed correctly, and the detection method ends.
[0023] The beneficial effects of the palletizing robot positioning method and palletizing system provided in this application are as follows: Compared with the prior art, the palletizing robot positioning method and palletizing system based on LiDAR in containers in this application organically combine LiDAR vision measurement technology with palletizing robot technology, solving the positioning problem of palletizing robots in containers and the detection problem of container stacking quality. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] in:
[0026] Figure 1 This is a schematic diagram of the container point cloud in a palletizing robot positioning method according to an embodiment of this application;
[0027] Figure 2 This is a schematic diagram of the point cloud of the container wall in a palletizing robot positioning method according to an embodiment of this application;
[0028] Figure 3 This is a point cloud diagram illustrating the positioning results of a palletizing system according to an embodiment of this application;
[0029] Figure 4 This is a schematic diagram of the current point cloud inside the container in a container stacking quality detection method shown in an embodiment of this application;
[0030] Figure 5 This is a schematic diagram of the buffer target point cloud in a box stacking quality detection method according to an embodiment of this application;
[0031] Figure 6 This is a schematic diagram of the difference point cloud in a box stacking quality detection method according to an embodiment of this application;
[0032] Figure 7 This is a schematic diagram of the planar point cloud of the boxes in a box stacking quality inspection method according to an embodiment of this application;
[0033] Figure 8 This is a schematic diagram of the box positioning point cloud in a box stacking quality inspection method shown in one embodiment of this application. Detailed Implementation
[0034] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings. Preferred embodiments of this application are shown in the drawings. However, this application can be implemented in many other different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of this application.
[0035] It should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.
[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
[0037] According to one aspect of this application, embodiments of this application provide a method for positioning a palletizing robot in a container based on LiDAR, comprising the following steps:
[0038] S1. After receiving the positioning command, the host computer collects point cloud data inside the container using lidar;
[0039] S2. The acquired point cloud is stitched together (multi-radar point cloud fusion), ROI regions are extracted, and the extracted point cloud is preprocessed such as downsampling and then saved to the cache queue;
[0040] S3. The point clouds of the five regions above, below, left, right and front of the lidar are extracted by direct filtering. The noise of the extracted point clouds is reduced. The point clouds of the top, front, left and right sides of the container are extracted by normal vector clustering. The cluster with the most point clouds is recorded as the point cloud of each container wall.
[0041] S4. During the initial positioning, the top, front, left, and right sides of the container are all wavy walls. Based on the point clouds of each wall obtained in step S3, calculate the centroid and feature vector of each wall point cloud. Using the centroid as a point on the plane and the feature vector pointing into the container as the plane normal vector, determine a plane. Move the plane along the direction of the normal vector until the number of point clouds on the plane is less than a set threshold, then stop moving the plane and use the position of the plane at this time as the position of the wall. If it is not the first positioning and there are stacked containers at the front of the container, then the front plane is not positioned first.
[0042] S5. To facilitate equipment entry and exit, a transition ramp is provided between the container and the ground during loading. Therefore, the inverse vector direction of the normal vector of the top plane of the container obtained in step S4 is used as the fixed direction to cluster the point cloud of the bottom area of the container. To avoid interference from the transition ramp on the positioning, the cluster with the most point cloud is extracted, and the optimal random consistency algorithm is used to fit the plane. The resulting plane position is used as the bottom position of the container.
[0043] S6. If this is not the first time positioning the container, and there are already stacked boxes at the front of the container, since it cannot be guaranteed that the plane formed by stacking the boxes is flat and perpendicular to the bottom surface of the container, special processing is required. That is, the cross product of the normal vectors of the left and bottom sides of the container wall obtained in the previous steps is used as the normal vector of the front face. The projection of the distance from each point on the front face to the origin of the lidar on the normal vector of the front face is calculated, and the average value is taken as the distance from the origin of the lidar to the front face. Then, the plane equation of the front face is determined.
[0044] S7. Using the intersection of the bottom, front, and left sides of the container as the origin of the container's coordinate system, the orientation of the container's coordinate system is determined by the direction of the normal vector of each plane. By calculating the transformation matrix between the container's coordinate system and the lidar's coordinate system, the pose of the lidar in the container's coordinate system is determined. By determining the installation positions of the palletizing robot and the lidar on the equipment, the pose of the palletizing robot inside the container is determined and returned to the palletizing robot, thus ending the positioning method.
[0045] According to another aspect of this application, embodiments of this application also provide a palletizing system, including the palletizing robot positioning method described in the above embodiments.
[0046] The palletizing system also includes a box placement quality detection method. This method is used to detect whether the box has been successfully placed and whether it is placed correctly after the palletizing robot has completed a placement and left the detection area of the LiDAR.
[0047] The method for inspecting the quality of box and parcel stacking includes the following steps:
[0048] Step 1: After receiving the detection command, the host computer collects point cloud data inside the container using LiDAR;
[0049] Step 2: The acquired point cloud is stitched together (multi-radar point cloud fusion), the ROI region is extracted, and the extracted point cloud is preprocessed such as downsampling before being used as the current point cloud;
[0050] Step 3: Read the most recent point cloud record from the cache queue as the target point cloud. Use K-nearest neighbor search, with K set to 1. Traverse the points in the current point cloud. If there is no point in the target point cloud that is less than the set threshold distance from the current point cloud, save the current point cloud as a difference point in the difference point cloud object.
[0051] Step 4: If the number of difference point clouds is less than the set threshold, it is considered that no boxes were placed in this detection, and the result of placement failure is returned, and the detection method ends; otherwise, continue to Step 5.
[0052] Step 5: In the difference point cloud, with the angle between the point cloud and the front face less than 30° as a condition and the distance threshold set to 10mm, perform plane fitting, extract the point cloud that is 30mm away from the plane in the difference point cloud, cluster the extracted point cloud according to the normal vector, and extract the cluster with the most point cloud as the planar point cloud of the box facing the lidar side.
[0053] Step 6: Calculate the centroid and eigenvector of the planar point cloud obtained in Step 5. Using the eigenvector as the coordinate direction of the planar point cloud and the centroid as the origin, calculate the maximum and minimum values of the planar point cloud in this coordinate system. Then, the centroid of the planar point cloud and the length, width, and height of the enclosing rectangle can be obtained.
[0054] Step 7: By calculating the transformation matrix between the planar coordinate system and the lidar coordinate system obtained in Step 6, the pose of the container in the lidar coordinate system is determined. Since the transformation matrix between the container coordinate system and the lidar coordinate system was calculated when positioning the container, the pose of the container in the container coordinate system can be obtained. Based on the pose of the container, it is determined whether the container is placed correctly, and the detection method ends.
[0055] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to the accompanying drawings.
[0056] Initial positioning of the palletizing system in the scenario of automated loading of box-shaped materials (i.e., boxes):
[0057] Step 1: The palletizing robot acts as the client, sending a positioning command to the server via the TCP / IP protocol. The command number is "1". After receiving the positioning command, the server begins collecting LiDAR point cloud data.
[0058] Step 2: The point clouds from multiple LiDAR sensors are stitched together, ROI regions are extracted, and the extracted point clouds are preprocessed, including downsampling, before being saved to a cache queue. The processed container point cloud is shown below. Figure 1 As shown;
[0059] Step 3: Extract point clouds from the top, bottom, left, right, and front regions of the LiDAR using a direct-pass filtering method, and then perform noise reduction on each extracted region's point cloud. First, for the point clouds of the top, front, left, and right regions of the container, extract the cluster with the most point clouds using normal vector clustering; this cluster represents the container wall's point cloud. The extracted container wall point cloud is shown below. Figure 2 As shown;
[0060] Step 4: Based on the point cloud of the container wall obtained in Step 3, calculate the centroid and feature vector of each point cloud. Taking the centroid as a point on the plane and the feature vector pointing into the container as the plane normal vector, determine a plane. Move the plane along the direction of the normal vector until the number of point clouds on the plane is less than a set threshold, then stop moving the plane and take the position of the plane at this time as the position of the container wall.
[0061] Step 5: Using the inverse direction of the normal vector of the container top plane obtained in Step 4 as a fixed direction, cluster the point cloud of the container bottom area. Extract the cluster with the most point clouds, fit a plane using the optimal random consensus algorithm, and use the resulting plane position as the container bottom position;
[0062] Step Six: Using the intersection of the container's bottom, front, and left sides as the origin of the container coordinate system, determine the container's coordinate system orientation using its normal vector direction. Determine the LiDAR's pose in the container coordinate system by calculating the transformation matrix between the container coordinate system and the radar coordinate system. Determine the palletizing robot's pose inside the container by determining the installation positions of the palletizing robot and the LiDAR on the equipment. The positioning results are as follows: Figure 3 As shown.
[0063] The server returned the data to the palletizing robot as "1 / / 86 / / 01351 / / -0290 / / -6806 / / 00001 / / -0002 / / 00016 / / ".
[0064] Data Explanation:
[0065]
[0066] Inspection of box and material stacking effect:
[0067] Step 1: After completing one stacking operation and leaving the radar detection area, the palletizing robot sends a positioning command to the server via TCP / IP protocol. The command number is "2". Upon receiving the command, the server begins collecting LiDAR point cloud data.
[0068] Step 2: The point clouds from multiple LiDAR sensors are stitched together, ROI regions are extracted, and the extracted point clouds are preprocessed, including downsampling, to obtain the current point cloud. The current point cloud inside the container is shown below. Figure 4 As shown;
[0069] Step 3: Read the most recent point cloud record from the cache queue as the target point cloud. Use K-nearest neighbor search, with K set to 1, to traverse the points in the current point cloud. If no point in the target point cloud is less than 1mm away from the current point, save the current point cloud as a difference point in the difference point cloud object. The target point cloud is as follows: Figure 5 As shown, the difference point cloud is as follows Figure 6 As shown;
[0070] Step 4: The number of difference point clouds obtained in Step 3 is 745. Within these difference point clouds, using an angle of less than 30° with the front face and a distance threshold of 10mm, plane fitting is performed to extract point clouds within a 30mm distance from the plane. The extracted point clouds are then clustered according to their normal vectors. The cluster with the largest number of point clouds is selected as the planar point cloud facing the lidar side of the component. The extracted planar point cloud facing the lidar side is shown below. Figure 7 As shown;
[0071] Step 5: Calculate the centroid and eigenvectors of the planar point cloud obtained in Step 4. Using the eigenvectors as the coordinate directions of the planar point cloud and the centroid as the origin, calculate the maximum and minimum values of the planar point cloud in this coordinate system. This allows us to determine the centroid of the planar point cloud and the length, width, and height of the enclosing rectangle. The minimum enclosing rectangle and eigenvectors are shown below. Figure 8 As shown;
[0072] Step Six: Determine the pose of the container in the LiDAR coordinate system by calculating the transformation matrix between the planar coordinate system and the LiDAR coordinate system obtained in Step Five. Since the transformation matrix between the container coordinate system and the LiDAR coordinate system was already calculated when positioning the container, the pose of the container in the container coordinate system can be obtained. The server returns the result to the palletizing robot as "2 / / 86 / / 00001 / / -0003 / / 00012 / / ".
[0073] Data Explanation:
[0074]
[0075] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0076] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A positioning method for a palletizing robot, characterized in that, Includes the following steps: S1. After receiving the positioning command, the host computer collects point cloud data inside the container using lidar; S2. The acquired point cloud is stitched together, ROI regions are extracted, and the extracted point cloud is preprocessed and saved to the cache queue; S3. The point clouds of the five regions above, below, left, right and front of the lidar are extracted by direct filtering. The noise of the extracted point clouds is reduced. The point clouds of the top, front, left and right sides of the container are extracted by normal vector clustering. The cluster with the most point clouds is recorded as the point cloud of each container wall. S4. During the initial positioning, the top, front, left, and right sides of the container are all wavy walls. Based on the point clouds of each wall obtained in step S3, calculate the centroid and feature vector of each wall point cloud. Using the centroid as a point on the plane and the feature vector pointing into the container as the plane normal vector, determine a plane. Move the plane along the direction of the normal vector until the number of point clouds on the plane is less than a set threshold, then stop moving the plane and use the position of the plane at this time as the position of the wall. If it is not the first positioning and there are stacked containers at the front of the container, then the front plane is not positioned first. S5. To facilitate equipment entry and exit, a transition ramp is provided between the container and the ground during loading. Therefore, the inverse vector direction of the normal vector of the top plane of the container obtained in step S4 is used as the fixed direction. The point cloud of the bottom area of the container is clustered, the cluster with the most point cloud is extracted, and the optimal random consistency algorithm is used to fit the plane. The resulting plane position is used as the bottom position of the container. S6. If this is not the first time positioning the container, and there are already stacked boxes at the front of the container, since it cannot be guaranteed that the plane formed by stacking the boxes is flat and perpendicular to the bottom surface of the container, special processing is required. That is, the cross product of the normal vectors of the left and bottom sides of the container wall obtained in the previous steps is used as the normal vector of the front face. The projection of the distance from each point on the front face to the origin of the lidar on the normal vector of the front face is calculated, and the average value is taken as the distance from the origin of the lidar to the front face. Then, the plane equation of the front face is determined. S7. Using the intersection of the bottom, front, and left sides of the container as the origin of the container's coordinate system, the orientation of the container's coordinate system is determined by the direction of the normal vector of each plane. By calculating the transformation matrix between the container's coordinate system and the lidar's coordinate system, the pose of the lidar in the container's coordinate system is determined. By determining the installation positions of the palletizing robot and the lidar on the equipment, the pose of the palletizing robot inside the container is determined and returned to the palletizing robot, thus ending the positioning method.
2. A palletizing system, characterized in that, This includes the palletizing robot positioning method as described in claim 1.
3. The palletizing system according to claim 2, characterized in that, The palletizing system also includes a box placement quality detection method, which is used to detect whether the box has been successfully placed and whether it is placed correctly after the palletizing robot has completed one placement and left the detection area of the lidar.
4. The palletizing system according to claim 3, characterized in that, The method for inspecting the quality of box stacking includes the following steps: Step 1: After receiving the detection command, the host computer collects point cloud data inside the container using LiDAR; Step 2: Stitch together the acquired point cloud, extract the ROI region, and preprocess the extracted point cloud as the current point cloud; Step 3: Read the most recent point cloud record from the cache queue as the target point cloud. Use K-nearest neighbor search, with K set to 1. Traverse the points in the current point cloud. If there is no point in the target point cloud that is less than the set threshold distance from the current point cloud, save the current point cloud as a difference point in the difference point cloud object. Step 4: If the number of difference point clouds is less than the set threshold, it is considered that no boxes were placed in this detection, and the result of placement failure is returned, and the detection method ends; otherwise, continue to Step 5. Step 5: In the difference point cloud, with the angle between the point cloud and the front face less than 30° as a condition and the distance threshold set to 10mm, perform plane fitting, extract the point cloud that is 30mm away from the plane in the difference point cloud, cluster the extracted point cloud according to the normal vector, and extract the cluster with the most point cloud as the planar point cloud of the box facing the lidar side. Step 6: Calculate the centroid and eigenvector of the planar point cloud obtained in Step 5. Using the eigenvector as the coordinate direction of the planar point cloud and the centroid as the origin, calculate the maximum and minimum values of the planar point cloud in this coordinate system. Then, the centroid of the planar point cloud and the length, width, and height of the enclosing rectangle can be obtained. Step 7: By calculating the transformation matrix between the planar coordinate system and the lidar coordinate system obtained in Step 6, the pose of the container in the lidar coordinate system is determined. Since the transformation matrix between the container coordinate system and the lidar coordinate system was calculated when positioning the container, the pose of the container in the container coordinate system can be obtained. Based on the pose of the container, it is determined whether the container is placed correctly, and the detection method ends.
Citation Information
Patent Citations
Self-adaptive control method and warehousing control system applied to intelligent warehousing
CN112193706A
Online mixed stacking method and system, electronic device and storage medium
CN114862063A
