Pallet pose recognition method for unmanned forklift
By acquiring deep point cloud data and constructing a lightweight convolutional neural network for pallet pose recognition, the problem of balancing cost, accuracy, and speed in pallet pose recognition for unmanned forklifts has been solved, achieving efficient and accurate pallet pose recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGDONG JATEN ROBOT & AUTOMATION
- Filing Date
- 2025-12-30
- Publication Date
- 2026-04-21
AI Technical Summary
Existing technologies struggle to balance cost, accuracy, speed, and robustness in unmanned forklift pallet pose recognition. Manual labeling methods are costly and prone to damage, 2D LiDAR has poor versatility, deep learning-based methods are greatly affected by lighting conditions, and depth camera-based methods are slow and difficult to recognize in densely stacked scenarios.
By acquiring deep point cloud data, defining 3D ROI regions for cropping and filtering, constructing a lightweight convolutional neural network for coarse pallet localization, using 2D image data to identify pallets and holes, and combining the lightweight convolutional neural network to train a model for pallet pose calculation.
It achieves the best balance between speed and accuracy in pallet recognition with strong resistance to appearance interference, reduces costs and improves the generalization ability of the model.
Smart Images
Figure CN121904451A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of pattern recognition technology, and more specifically, to a pallet pose recognition method for unmanned forklifts. Background Technology
[0002] In smart logistics and automated warehousing systems, unmanned forklifts are the core equipment for automating point-to-point material handling. Their most critical and frequently performed task is the automated picking and placing of various pallets. To accomplish this task, the first core technical challenge that unmanned forklifts must address is the rapid and accurate pose recognition of the pallets. Here, "pose" includes the pallet's position in three-dimensional space (X, Y, Z coordinates) and its attitude angles (especially the yaw angle around the vertical axis), particularly the alignment between the centerline of the pallet's insertion holes and the forklift's forks.
[0003] Currently, there are various technical methods for pallet pose recognition, but each has its own limitations, making it difficult to achieve a balance between cost, accuracy, speed, and robustness.
[0004] Manual labeling method: This involves affixing QR codes, AprilTags, or ArUco labels to pallets. The automated forklift uses its onboard camera to recognize these pre-defined labels and calculate the pallet's position. This requires uniform deployment and maintenance of labels on all pallets, making this solution costly, and the labels are prone to wear, dirt, or detachment during repeated handling.
[0005] The geometric feature method based on 2D LiDAR: This method uses a 2D LiDAR to perform a horizontal scan at a fixed height at the pallet placement location, and locates the pallet by identifying the point cloud patterns generated by the bottom legs or side beams of the pallet. However, this method requires the radar scanning plane to be perfectly aligned with the pallet, and necessitates manual pre-entry of the heights of all pallets into the system and raising the forks to the corresponding height upon reaching the designated point. This method has poor versatility.
[0006] Deep learning combined with depth images is a method that trains a deep neural network using a large number of labeled RGB images of the pallet and then uses the corresponding depth images to regress the pallet's pose. However, this type of method is severely affected by changes in lighting conditions (sunlight, artificial light, shadow), reflections, and overexposure or underexposure, all of which can significantly impact the model's recognition performance.
[0007] The method based on 3D point clouds using a depth camera acquires 3D environmental point clouds along the fork arm direction using a depth camera, and identifies the pallet through template matching, point cloud segmentation, and plane detection. This method generates a large amount of point cloud data, resulting in a relatively slow processing speed. Furthermore, in densely stacked scenes, relying solely on point cloud features is insufficient to accurately identify the geometric features of the pallet. Summary of the Invention
[0008] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a pallet pose recognition method for unmanned forklifts, the method comprising:
[0009] Acquire depth point cloud data about the stack, and annotate the stack and holes in the depth point cloud data;
[0010] Define a 3D ROI region, crop the depth point cloud data through the 3D ROI region to obtain cropped point cloud data, calculate the normal vector of each point in the cropped point cloud data in the X-axis / Y-axis direction, and perform filtering operation on the cropped point cloud data based on the normal vector and the preset filtering threshold.
[0011] Define the 2D image size, and project the cropped point cloud data onto the 2D image based on the 2D image size to obtain the 2D image data.
[0012] Preferably, the method further includes:
[0013] The pixel coordinates of the cropped point cloud data projected onto the 2D image are: ;
[0014] in, and These are defined as the scale and offset from pixel coordinates to 3D coordinates, respectively.
[0015] Preferably, a coarse localization model of the stack is constructed, and 2D image data is processed using the coarse localization model of the stack to obtain 2D bounding box data. The stack index matrix is then obtained from the 2D bounding box data, including:
[0016] A coarse localization model for the pallet is constructed based on a lightweight convolutional neural network, and the model is trained using deep point cloud data.
[0017] Input 2D image data into the pallet coarse positioning model, and output 2D bounding boxes of all pallets and all holes in the 2D image data, i.e. 2D bounding box data, where each bounding box contains the center point coordinates and width and height.
[0018] Classify and associate 2D bounding box data, and define a stack index matrix, which includes stack instances and hole instances.
[0019] Preferably, the method further includes:
[0020] Each element in the stack index matrix represents an index of a hole instance for each stack instance. Iterate through the 2D bounding boxes of all holes, when the first hole... The center point coordinates of the hole's bounding box are within the current stack's bounding box. Add the hole instance's index to... In, and the length of 2 Add to the stack index matrix.
[0021] Preferably, the point cloud of board instances is obtained based on the board index matrix, and the point clouds of the left support leg, the middle support leg, and the right support leg are obtained respectively through a classification and clipping strategy and the point cloud of board instances, including:
[0022] For each stack instance and its two corresponding hole instances in the stack index matrix;
[0023] The minimum / maximum values of the X-axis / Y-axis coordinates of the stack are obtained from the 2D bounding box of the stack instance. The ROI region of the stack is obtained based on the minimum / maximum values of the X-axis / Y-axis coordinates. The stack ROI region is used to clip the clipped point cloud data to obtain the stack instance point cloud. Plane fitting is performed on the stack instance point cloud, the normal vector is calculated and the principal plane is extracted to update the stack instance point cloud.
[0024] Obtain the left and right attributes of the holes using the 2D bounding boxes of the two hole instances. Obtain the minimum / maximum X-axis coordinates of the two holes in 3D space using the left and right attributes. Obtain the point clouds of the left leg, the middle leg, and the right leg using the minimum / maximum X-axis coordinates.
[0025] Preferably, the point clouds of the left leg, the middle leg, and the right leg are obtained by using the minimum / maximum values of the X-axis coordinates, including:
[0026] The left and right attributes of the hole include left hole and right hole;
[0027] Based on the minimum value of the left hole in the three-dimensional X-axis coordinate and the minimum value of the three-dimensional X-axis coordinate of the pallet, the point cloud of the left leg is obtained by cropping from the point cloud of the pallet instance.
[0028] Based on the maximum value of the three-dimensional X-axis coordinate of the left hole and the minimum value of the three-dimensional Y-axis coordinate of the right hole, the point cloud of the middle support leg is obtained by cropping from the point cloud of the plate instance.
[0029] Based on the maximum value of the three-dimensional X-axis coordinate of the right hole and the maximum value of the three-dimensional X-axis coordinate of the pallet, the point cloud of the right leg is obtained by cropping from the point cloud of the plate instance.
[0030] Preferably, the pose of the stack is calculated based on the point clouds of the left leg, the middle leg, the right leg, and the 2D bounding box data, including:
[0031] Calculate the two-dimensional Y-axis coordinates of the center points of the 2D borders of the left and right holes and average them to obtain the two-dimensional average data. Update the two-dimensional Y-axis coordinates of the center point of the stack based on the two-dimensional average data.
[0032] The point cloud of the middle leg is extracted from a range above and below the three-dimensional Y-axis coordinate of the center point of the onboard, and a reference point cloud is obtained. The center point coordinate of the reference point cloud is calculated, and the three-dimensional Y-axis coordinate of the center point of the onboard is obtained.
[0033] Calculate the 3D center points of the left and right leg point clouds respectively, and obtain the pallet attitude angle based on the 3D center points of the left and right leg point clouds.
[0034] Preferably, the method further includes:
[0035] The pallet attitude angle is the angle between the line connecting the 3D center points of the left and right outrigger point clouds on the XOZ plane and the X-axis. The specific calculation formula is as follows:
[0036] ;
[0037] Where p is the attitude angle of the stack. These are the three-dimensional center points of the point clouds of the left and right legs, respectively.
[0038] Compared with the prior art, the present invention has the following beneficial effects:
[0039] Compared to coarse localization of pallets on RGB or intensity map images, this scheme identifies pallets on point cloud-converted images, exhibiting strong resistance to appearance interference. The neural network learns the common geometric features of all pallets, rather than color or texture. Furthermore, point cloud data of pallets with different structures can be obtained with the help of simulation and data synthesis systems, which is beneficial for generating model training data. The generalization ability is also significantly better than traditional image recognition methods. By using a lightweight neural network to replace the template matching problem of the pure point cloud scheme, the coarse localization of pallets can be completed, achieving the best balance between speed and accuracy. Attached Figure Description
[0040] Figure 1 This is a flowchart of the steps of a pallet pose recognition method for unmanned forklifts according to the present invention.
[0041] Figure 2 This is a 2D RGB diagram of the unmanned forklift and pallet stacking in this invention.
[0042] Figure 3 This is a schematic diagram of the 2D strength of the unmanned forklift and pallet stacking in this invention.
[0043] Figure 4 This is a 3D depth point cloud diagram of the unmanned forklift and pallet stacking in this invention.
[0044] Figure 5 This is a schematic diagram of the 2D binarized image of the 3D depth point cloud conversion in this invention.
[0045] Figure 6 It is a schematic diagram of the recognition result of the neural network in the 2D binary image in the present invention.
[0046] Figure 7 It is a schematic diagram of the position and attitude angle of the pallet in the present invention. Detailed implementation manners
[0047] The present invention will be specifically described below with reference to the accompanying drawings of the specification. As shown in Figure 1 to Figure 7 , where Figure 6 the green ones are the recognized holes, and the blue ones are the recognized pallets. The method for recognizing the position and orientation of the pallet for an automated guided vehicle will be introduced in detail below.
[0048] Step 1: Obtain depth point cloud data and convert the depth point cloud data into 2D image data.
[0049] As a further solution of the present invention, obtain depth point cloud data of the pallet and label the pallet and the holes in the depth point cloud data.
[0050] As a further solution of the present invention, define a three-dimensional ROI region, crop the depth point cloud data through the three-dimensional ROI region to obtain cropped point cloud data, calculate the normal vectors of each point in the cropped point cloud data in the X-axis / Y-axis directions, and perform a filtering operation on the cropped point cloud data based on the normal vectors and a preset filtering threshold.
[0051] As a further solution of the present invention, define the 2D image size, project the cropped point cloud data onto 2D according to the 2D image size to obtain 2D image data.
[0052] It should be noted that a depth camera fixed on the automated guided vehicle or a scene containing various pallets is built in a simulation system to collect depth point cloud data of the pallet. During the collection, pallets with different structures (channel type, grid type) and different stackings should be covered to construct a small dataset with diversity. Labeling software such as Labelme can be used to complete the labeling of the pallet and the holes in the collected dataset.
[0053] In this embodiment, the 3D depth point cloud data is converted into 2D image data , and the conversion process is as follows:
[0054] Define a three-dimensional region of interest with the X and Y axis coordinates in the camera coordinate system , crop the source point cloud data to obtain point cloud .
[0055] Furthermore, define the size of the 2D image as Before projecting the point cloud onto a 2D image, irrelevant ground and pallet side point cloud data needs to be filtered out. The remaining point cloud consists of the front elevation point cloud of the pallets and stacked goods. Calculation... The normal vector of each point in the X and Y axes Set thresholds respectively The filtered point cloud is obtained. , will dot clouds Projected onto 2D image The pixel coordinates are .
[0056] in, and These are defined as the scale and offset from pixel coordinates to 3D coordinates, respectively. It is a constant.
[0057] Step 2: Construct a coarse localization model for the stack, process the 2D image data using the coarse localization model to obtain 2D bounding box data, and obtain the stack index matrix using the 2D bounding box data.
[0058] As a further aspect of the present invention, a coarse localization model for the pallet is constructed based on a lightweight convolutional neural network, and the coarse localization model for the pallet is trained using deep point cloud data.
[0059] In this embodiment, a common lightweight convolutional neural network can be selected, whose input is the aforementioned 2D image data, and whose output is the 2D bounding box of all stacks and all holes in the image (point cloud). Each bounding box contains the coordinates of its center point. and width and height .
[0060] As a further aspect of the present invention, 2D image data is input into the pallet coarse positioning model, and 2D bounding boxes of all pallets and all holes in the 2D image data are output, i.e., 2D bounding box data, wherein each bounding box contains the center point coordinates and width and height.
[0061] Furthermore, the network is trained using the dataset obtained by the collection and annotation method described in step one. The trained network is then deployed to the embedded computing unit of the unmanned forklift. During on-site operation, the current scene point cloud collected by the depth camera is converted into a binary image according to step one, and the image is input into the network. The network outputs the positions of the pallets and holes, i.e., the 2D bounding boxes of all pallets and all holes.
[0062] As a further aspect of the invention, each element in the stack index matrix represents an index of a hole instance for each stack instance. Iterate through the 2D bounding boxes of all holes, when the first hole... The center point coordinates of the hole's bounding box are within the current stack's bounding box. Add the hole instance's index to... In, and the length of 2 Add to the stack index matrix.
[0063] In this embodiment, all 2D bounding boxes of stacks and holes are categorized and associated, and a stack index matrix is defined. Each element in the index matrix represents an index of a stack instance and its corresponding hole instance. Iterate through all the bounding boxes of the holes, when the first... The center point coordinates of the hole's bounding box are within the bounding box of the current stack. The index of the hole instance will be... Add to In, and the length of 2 Add to This is because a pallet containing two holes is a complete pallet.
[0064] Step 3: Obtain the point cloud of the board instance based on the board index matrix, and obtain the point cloud of the left leg, the point cloud of the middle leg, and the point cloud of the right leg respectively through the classification and clipping strategy and the point cloud of the board instance.
[0065] As a further aspect of the present invention, for each stack instance and its corresponding two hole instances in the stack index matrix.
[0066] Furthermore, the minimum / maximum values of the X-axis / Y-axis coordinates of the stack are obtained in 3D space through the 2D bounding box of the stack instance. The ROI region of the stack is obtained based on the minimum / maximum values of the X-axis / Y-axis coordinates. The clipping point cloud data is clipped through the stack ROI region to obtain the stack instance point cloud. Plane fitting is performed on the stack instance point cloud, the normal vector is calculated, and the principal plane is extracted to update the stack instance point cloud.
[0067] In this embodiment, for Each stack instance in and its two hole instance indexes and First, the minimum and maximum values of the stack's X-axis coordinates in three-dimensional space are calculated using the coordinates of the center point of the stack instance's border and its width and height. The minimum and maximum values of the Y-axis coordinates can be obtained similarly. Then, using the aforementioned calculations, a three-dimensional region is defined, and the point cloud is cropped. Get a stack instance Point clouds Then, plane fitting is performed on the point cloud of the stack instance, the normal vector is calculated, and the most important plane is extracted to update the data. .
[0068] Furthermore, the left and right attributes of the holes are obtained through the 2D bounding boxes of the two hole instances. The minimum and maximum values of the X-axis coordinates of the two holes in three-dimensional space are obtained through the left and right attributes of the holes. The point clouds of the left leg, the middle leg, and the right leg are obtained through the minimum and maximum values of the X-axis coordinates.
[0069] Furthermore, the left and right attributes of the hole include a left hole and a right hole.
[0070] Specifically, based on the minimum value of the left hole in the three-dimensional X-axis coordinate and the minimum value of the three-dimensional X-axis coordinate of the pallet, the point cloud of the left leg is obtained by cropping from the point cloud of the pallet instance.
[0071] Specifically, the point cloud of the middle support leg is obtained by cropping from the point cloud of the plate instance based on the maximum value of the three-dimensional X-axis coordinate of the left hole and the minimum value of the three-dimensional Y-axis coordinate of the right hole.
[0072] Specifically, based on the maximum value of the three-dimensional X-axis coordinate of the right hole and the maximum value of the three-dimensional X-axis coordinate of the pallet, the point cloud of the right leg is obtained by cropping from the point cloud of the plate instance.
[0073] In this embodiment, according to and The positional relationship between the center points of the 2D bounding boxes of the two holes is determined, the left and right attributes of the holes are calculated, and the minimum and maximum values of the 3D X-axis coordinates of the left and right holes are further calculated. and .
[0074] In this embodiment, the point cloud of the left leg of the pallet is extracted from the point cloud of the pallet based on the minimum X-axis coordinate of the pallet and the minimum three-dimensional X-axis coordinate of the left hole. .
[0075] Similarly, based on the maximum 3D X-axis coordinate of the right hole and the maximum 3D X-axis coordinate of the pallet, the point cloud extracted to the right leg of the pallet is cropped. .
[0076] In this embodiment, the point cloud of the middle leg of the pallet is extracted by cropping based on the maximum value of the three-dimensional X-axis coordinate of the left hole and the minimum value of the three-dimensional X-axis coordinate of the right hole. .
[0077] Step 4: Calculate the pose of the stack based on the point cloud of the left leg, the point cloud of the middle leg, the point cloud of the right leg, and the 2D bounding box data.
[0078] As a further aspect of the present invention, the two-dimensional Y-axis coordinates of the center points of the 2D borders of the left and right holes are calculated and averaged to obtain two-dimensional average data, and the two-dimensional Y-axis coordinates of the center point of the stack are updated based on the two-dimensional average data.
[0079] As a further aspect of the present invention, the point cloud of the intermediate support leg is extracted from a certain range above and below the three-dimensional Y-axis coordinate of the center point coordinate of the onboard, a reference point cloud is obtained, the center point coordinate of the reference point cloud is calculated, and the three-dimensional Y-axis coordinate of the center point coordinate of the onboard is obtained.
[0080] In this embodiment, the two-dimensional Y-axis coordinates of the center points of the 2D borders of the left and right holes are calculated and averaged, and the two-dimensional Y-axis coordinates of the center point of the stack are updated accordingly. Finally, the three-dimensional Y-axis coordinates of the center point of the stack are calculated by reverse engineering.
[0081] The center point position is calculated as the three-dimensional Y-axis coordinate of the center point of the pallet, and a certain range above and below is selected. Obtain the reference point cloud of the middle support leg, and calculate its center point coordinates, which are the center point coordinates of the pallet.
[0082] As a further aspect of the present invention, the three-dimensional center points of the left leg point cloud and the right leg point cloud are calculated respectively, and the attitude angle of the pallet is obtained based on the three-dimensional center points of the left leg point cloud and the right leg point cloud.
[0083] The attitude angle of the pallet is the angle between the line connecting the three-dimensional center points of the left and right leg point clouds on the XOZ plane and the X-axis. The specific calculation formula is as follows:
[0084] ;
[0085] Where p is the attitude angle of the stack. These are the three-dimensional center points of the point clouds of the left and right legs, respectively.
[0086] The specific usage and function of this embodiment are explained below:
[0087] This method converts 3D depth point clouds from a depth camera into binary 2D images and uses these images to identify the pose of a pallet. First, a small number of point cloud images of pallets with different structures are collected to train a lightweight neural network. During deployment, this neural network is used to predict the pixel positions of all pallets and holes in the image. The pixel positions are then converted to 3D world coordinates to obtain the boundary coordinates of the left, center, and right legs of the pallet, thus obtaining the point cloud data for the three legs. Finally, the center point coordinates and pose angle of the pallet are obtained by calculating the center point coordinates of the three legs. The core innovation of this solution lies in the fact that the 2D position recognition of the pallet does not directly utilize the 2D RGB image or intensity map reflecting the object's reflection intensity provided by the depth camera. Instead, it converts the 3D depth point cloud into a 2D binary image, transforming a complex 3D point cloud recognition problem into a more efficient and mature 2D image target detection problem. A coarse-to-fine pose calculation strategy is designed, using a neural network to solve the coarse localization problem of identifying the location, and further utilizing more accurate depth point clouds to solve the fine localization problem of determining the exact location.
[0088] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via limited means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.
[0089] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.
[0090] It should be understood that, in the embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0091] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A pallet pose recognition method for unmanned forklifts, characterized in that, It includes the following steps: Acquire depth point cloud data and convert it into 2D image data; Construct a coarse localization model for the stack, process 2D image data using the coarse localization model to obtain 2D bounding box data, and obtain the stack index matrix using the 2D bounding box data. The point cloud of the board instance is obtained based on the board index matrix. The point cloud of the left support leg, the point cloud of the middle support leg, and the point cloud of the right support leg are obtained by using the classification and clipping strategy and the point cloud of the board instance respectively. The pose of the pallet is calculated based on the point clouds of the left leg, the middle leg, the right leg, and the 2D bounding box data.
2. The pallet pose recognition method for unmanned forklifts according to claim 1, characterized in that, Acquire depth point cloud data and convert it into 2D image data, specifically: Acquire depth point cloud data about the stack, and annotate the stack and holes in the depth point cloud data; Define a 3D ROI region, crop the depth point cloud data through the 3D ROI region to obtain cropped point cloud data, calculate the normal vector of each point in the cropped point cloud data in the X-axis / Y-axis direction, and perform filtering operation on the cropped point cloud data based on the normal vector and the preset filtering threshold. Define the 2D image size, and project the cropped point cloud data onto the 2D image based on the 2D image size to obtain the 2D image data.
3. The pallet pose recognition method for unmanned forklifts according to claim 2, characterized in that, The method further includes: The pixel coordinates of the cropped point cloud data projected onto the 2D image are: ; in, and These are defined as the scale and offset from pixel coordinates to 3D coordinates, respectively.
4. The pallet pose recognition method for unmanned forklifts according to claim 1, characterized in that, A coarse localization model of the stack is constructed. The 2D image data is then processed using this model to obtain 2D bounding box data. Finally, the stack index matrix is obtained from the 2D bounding box data, including: A coarse localization model for the pallet is constructed based on a lightweight convolutional neural network, and the model is trained using deep point cloud data. Input 2D image data into the pallet coarse positioning model, and output 2D bounding boxes of all pallets and all holes in the 2D image data, i.e. 2D bounding box data, where each bounding box contains the center point coordinates and width and height. Classify and associate 2D bounding box data, and define a stack index matrix, which includes stack instances and hole instances.
5. A pallet pose recognition method for unmanned forklifts according to claim 4, characterized in that, The method further includes: Each element in the stack index matrix represents an index of a hole instance for each stack instance. Iterate through the 2D bounding boxes of all holes, when the first hole... The center point coordinates of the hole's bounding box are within the current stack's bounding box. Add the hole instance's index to... In, and the length of 2 Add to the stack index matrix.
6. The pallet pose recognition method for unmanned forklifts according to claim 1, characterized in that, Based on the board index matrix, point clouds of board instances are obtained. Then, using a classification and clipping strategy, point clouds of the left leg, middle leg, and right leg are obtained respectively, including: For each stack instance and its two corresponding hole instances in the stack index matrix; The minimum / maximum values of the X-axis / Y-axis coordinates of the stack are obtained from the 2D bounding box of the stack instance. The ROI region of the stack is obtained based on the minimum / maximum values of the X-axis / Y-axis coordinates. The stack ROI region is used to clip the clipped point cloud data to obtain the stack instance point cloud. Plane fitting is performed on the stack instance point cloud, the normal vector is calculated and the principal plane is extracted to update the stack instance point cloud. Obtain the left and right attributes of the holes using the 2D bounding boxes of the two hole instances. Obtain the minimum / maximum X-axis coordinates of the two holes in 3D space using the left and right attributes. Obtain the point clouds of the left leg, the middle leg, and the right leg using the minimum / maximum X-axis coordinates.
7. A pallet pose recognition method for unmanned forklifts according to claim 6, characterized in that, The point clouds of the left leg, the middle leg, and the right leg are obtained by using the minimum / maximum values of the X-axis coordinates, including: The left and right attributes of the hole include left hole and right hole; Based on the minimum value of the left hole in the three-dimensional X-axis coordinate and the minimum value of the three-dimensional X-axis coordinate of the pallet, the point cloud of the left leg is obtained by cropping from the point cloud of the pallet instance. Based on the maximum value of the three-dimensional X-axis coordinate of the left hole and the minimum value of the three-dimensional Y-axis coordinate of the right hole, the point cloud of the middle support leg is obtained by cropping from the point cloud of the plate instance. Based on the maximum value of the three-dimensional X-axis coordinate of the right hole and the maximum value of the three-dimensional X-axis coordinate of the pallet, the point cloud of the right leg is obtained by cropping from the point cloud of the plate instance.
8. A pallet pose recognition method for unmanned forklifts according to claim 1, characterized in that, The pose of the pallet is calculated based on the point clouds of the left leg, the middle leg, the right leg, and the 2D bounding box data, including: Calculate the two-dimensional Y-axis coordinates of the center points of the 2D borders of the left and right holes and average them to obtain the two-dimensional average data. Update the two-dimensional Y-axis coordinates of the center point of the stack based on the two-dimensional average data. The point cloud of the middle leg is extracted from a range above and below the three-dimensional Y-axis coordinate of the center point of the onboard, and a reference point cloud is obtained. The center point coordinate of the reference point cloud is calculated, and the three-dimensional Y-axis coordinate of the center point of the onboard is obtained. Calculate the 3D center points of the left and right leg point clouds respectively, and obtain the pallet attitude angle based on the 3D center points of the left and right leg point clouds.
9. A pallet pose recognition method for unmanned forklifts according to claim 8, characterized in that, The method further includes: The pallet attitude angle is the angle between the line connecting the 3D center points of the left and right leg point clouds on the XOZ plane and the X-axis. The specific calculation formula is as follows: ； Where p is the attitude angle of the stack. These are the three-dimensional center points of the point clouds of the left and right legs, respectively.