Tray identification and pose calculation method

Abstract

The invention discloses a tray identification and pose calculation method, which comprises the following steps of: 1, acquiring a camera initial point cloud by using a TOF (Time of Flight) camera of a forklift robot, and acquiring a world initial point cloud under a world coordinate system of the forklift robot through matrix conversion; 2, preprocessing the world initial point cloud to obtain a world point cloud, and converting the world point cloud into a grey-scale map through projection, matrix change and dimension reduction; 3, performing template matching on the grey-scale map to obtain a tray confidence frame; 4, extracting a tray end surface point cloud and a tray leg column point cloud in the confidence frame; and 5, fitting a tray end face equation through the tray end face point cloud, calculating a tray jack midpoint coordinate through the tray leg column point cloud, and finally obtaining a complete tray pose including a tray yaw angle, the tray midpoint coordinate and a tray jack interval. According to the method, the position and posture of the tray can be accurately and quickly calculated in a complex environment, so that the automatic inserting and taking task of the forklift robot in a working environment is realized.

Description

Technical Field

[0001] The present invention relates to the field of pallet recognition detection and position calculation. Specifically, it is a method for enabling a forklift robot to automatically recognize a pallet and accurately calculate the position of the pallet in different working environments, thereby guiding the forklift robot to accurately insert the pallet. Technical Background

[0002] As the core transfer container in the production process and warehousing and logistics transportation, pallets are the key forklift objects in the intelligent operation of forklift robots. With the increasing development of automated factories, the application scope of forklift robots is becoming wider and wider, and the penetration rate of unmanned operations is also increasing. Forklift robots are widely deployed in scenarios such as repetitive handling tasks, high-intensity handling work, harsh working environments, and high environmental requirements. In order to achieve a high degree of automation of forklift robots, it is crucial to enable them to automatically identify pallets and accurately calculate the position and posture (pose) of pallets, so as to automatically complete the insertion and extraction operations. However, due to complex and changeable environmental factors, the ability of forklift robots to recognize pallets through images in the past was difficult to meet the needs of practical applications in terms of accuracy and speed. Therefore, developing a real-time detection and pose calculation method for pallets with sufficient accuracy and good real-time performance has become a key issue that needs to be solved urgently.

[0003] Traditional pallet recognition methods mainly include the following: one is to recognize the pallet by capturing the color features or geometric features of the pallet in the RGB image; another method is to paste labels such as QR codes on the pallet, and then indirectly recognize the pallet by identifying these labels; and another method is to use a laser sensor to obtain the point cloud data of the pallet, and use point cloud template matching technology based on these point cloud data to complete the pallet recognition.

[0004] The above three traditional pallet recognition methods all have certain limitations. The first method is highly dependent on lighting conditions, and sufficient lighting is often not guaranteed in the working environment of forklift robots, which leads to a significant decrease in recognition accuracy. Although the second method has achieved pallet recognition to a certain extent, it is still guided in essence and has not reached the level of full automation. In the complex environment where forklift robots are actually used, markers such as QR codes are easily damaged, and long-term use is more likely to cause problems in pallet recognition. Although the third method is based on point cloud data obtained by laser sensors for recognition, due to the large amount of point cloud data, the computing resources required for point cloud template matching are very huge. However, in actual applications, forklift robots often lack sufficient computing power resources, resulting in a long recognition process and poor real-time performance, which makes it difficult to meet the needs of actual handling work. Summary of the invention

[0005] The present invention aims to solve the deficiencies of the above-mentioned prior art and proposes a pallet recognition and posture calculation method, so as to achieve high-precision and high-speed pallet recognition and posture detection, thereby realizing the goal of fully automated handling operations of forklift robots.

[0006] To solve the technical problem, the present invention adopts the following technical solution:

[0007] The present invention provides a method for pallet recognition and posture calculation, which is characterized by being applied to forklift robot handling operations and is performed in the following steps:

[0008] Step 1: Establish the camera coordinate system and the world coordinate system of the forklift robot;

[0009] Step 2: The original 3D point cloud containing the pallet features in the world coordinate system of the forklift robot After preprocessing, the preprocessed 3D point cloud containing the pallet features is obtained. , and record The serial numbers corresponding to all points in ;

[0010] Step 3, define K projection planes;

[0011] Step 4: Initialize k=1;

[0012] Step 5: Project to the kth projection plane, then use the dimensionality reduction algorithm to convert the projected points into a two-dimensional point set, and then use the screening coincidence point algorithm to remove all coincidence points in the two-dimensional point set to obtain a two-dimensional point set without overlap. ;

[0013] Step 6: Use the conversion algorithm to convert the two-dimensional point set without overlapping points Convert to target grayscale image ;

[0014] Step 7: According to the parameters of the pallet used by the forklift robot, Make a proportional pallet template M, with a height of H and a width of W. The left and right holes of the pallet inserted by the forklift robot are marked as ;

[0015] Step 8: Use an image template matching method to obtain the target grayscale image If the tray template M fails to match, k+1 is assigned to k. If k≤K, return to step 5. If k>K, determine There is no pallet in the template; if the pallet template M is matched successfully, go to step 9;

[0016] Step 9: Combine the matching result of the pallet template M with the pallet template to obtain the pallet pose.

[0017] The method for pallet identification and posture calculation of the present invention is also characterized in that step 1 comprises:

[0018] The midpoint of the contact surface between the forklift robot and the forklift robot body is taken as the origin. , taking the forklift robot's forward direction as The positive direction of the axis is perpendicular to the ground and upward. Positive axis direction, determined by the right-hand rule Axis, thus establishing the world coordinate system of the forklift robot ;

[0019] Take the TOF camera light source as , the TOF camera is directly in front of Axis positive direction, TOF camera vertically downward Positive direction of axis, determined by right-hand rule Axis, thus establishing the camera coordinate system .

[0020] Furthermore, the specific steps of the pre-processing method in step 2 include:

[0021] Step 2.1: The forklift robot uses its own TOF camera to obtain the original 3D point cloud containing the pallet features. After coordinate transformation, the original 3D point cloud containing the pallet features in the world coordinate system of the forklift robot is obtained. ;

[0022] Step 2.2: Use a straight-through filter to Perform background removal and keep the coordinates with the world coordinate system Axis origin The distance range is The point cloud inside is obtained to obtain the filtered three-dimensional point cloud containing the pallet features. ,in, is the closest point of the pallet placement range, It is the farthest point of the pallet placement range;

[0023] Step 2.3: Use voxel filtering to Sampling is performed to obtain a downsampled 3D point cloud containing the pallet features. ;

[0024] Step 2.4: Use statistical filters to After performing noise reduction to remove outlier point clouds, the preprocessed 3D point cloud containing the pallet features is obtained. ,in, express The nth point in express Total number of midpoints, recorded exist The serial number in is recorded as .

[0025] Furthermore, the step 3 comprises:

[0026] Define perpendicular to K projection planes of the plane is the distance from the TOF camera And with the forklift robot world coordinate system The axis angles are The plane, where Represents the projection plane With the world coordinate system The angle of the axis, Represents the world coordinate system The axis angle is The projection plane of , K represents the total number of projection planes.

[0027] Further, the step 5 comprises:

[0028] Step 5.1: Calculation Each point in Distance ,in, express The nth point in arrive distance;

[0029] Step 5.2: All points in the projection are And rotate and translate all the projected points so that all the projected points move to plane, thus obtaining the kth transition 3D point cloud ;

[0030] Step 5.3: The three-dimensional coordinates of all points in , converted into a two-dimensional point set The two-dimensional coordinate set of ,in, express The three-dimensional coordinates of the nth point in , express The two-dimensional coordinates of the nth point in ;

[0031] Step 5.4: Follow , ,Will The two-dimensional coordinates of all points are proportionally reduced times, and The center of the image is translated to the coordinates , and then translated The two-dimensional coordinates of all points in are rounded to obtain the preprocessed two-dimensional point set , whose coordinates are ,in, Respectively The maximum and minimum values ​​in Respectively The maximum and minimum values ​​in , S is a positive integer, Represents the two-dimensional coordinates of the nth point after preprocessing;

[0032] Step 5.5: If There are coincident points with the same two-dimensional coordinates in , then all the coincident points are Get the serial number in All coincident points with corresponding serial numbers in the plane The distance between the two points is 0. Delete the remaining coincident points to obtain a two-dimensional point set without coincidence The two-dimensional coordinate set of ,in, for Middle The coordinates of the points, express The total number of midpoints.

[0033] Further, the step 6 comprises:

[0034] Step 6.1: The center is located in the two-dimensional coordinate The nth point at The numbers in the matrix Middle Line The element value of the column will The center is located in the two-dimensional coordinate The nth point at The point corresponding to the serial number in The distance matrix Middle Line The element values ​​of the columns are obtained, resulting in two matrices of dimension S×S and the distance matrix ;

[0035] Step 6.2: Calculate using formula (1) Middle Gray value of a point , and the corresponding grayscale image Middle Line The element value of the column, and divide The element values ​​of the remaining positions other than the two-dimensional coordinates are set to 0, so as to obtain a target grayscale image with a dimension of S×S ;

[0036] (1)

[0037] In formula (1), Indicates rounding operation. for Located in the middle Line The element value of the column, For the matrix The minimum value in For the matrix The maximum value in is the maximum gray value.

[0038] Furthermore, the image template matching method in step 8 includes:

[0039] Step 8.1: Combine the tray template M and the target grayscale image , use formula (4) to calculate the upper left corner vertex of M and Coordinates on Similarity score when the ;

[0040] (2)

[0041] In formula (2), is the coordinate in M The gray value at yes The standardized and normalized values, yes Central coordinates The gray value at yes Standardized and normalized values; is the coordinate in R The gray value at Coordinates in M The gray value at is standardized and normalized, yes Central coordinates The gray value at is the standardized and normalized value;

[0042] Step 8.2: If Exceeding the preset value When , it means that on the kth projection plane The pallet on the is successfully identified, and the k-th identification result is output, including: the confidence box of the pallet location The upper left corner is Coordinates on and a confidence box for where the pallet is located The height H and width W of ; otherwise, it represents the kth projection plane Identification failed.

[0043] Further, the step 9 comprises:

[0044] Step 9.1: Extract the confidence box of the tray location based on M The local two-dimensional pixel coordinates of the pallet end surface area corresponding to the , the local two-dimensional pixel coordinates corresponding to the end surface area of ​​the tray are obtained in The corresponding 3D point cloud , then, from Extract the initial point cloud of the pallet end surface ;

[0045] Step 9.2: Use the least squares method to calculate the initial point cloud Perform fitting to obtain the tray end face equation , and thus calculate the yaw angle of the pallet ;

[0046] Step 9.3: Screening Equation of the center and tray end face The distance between the corresponding planes is less than the threshold Point Cloud , extraction tray Point cloud of middle pallet legs The relative position of the left socket of the tray is obtained Local point cloud of the left leg column on both sides 、Local point cloud of right leg column And the right socket of the tray Local point cloud of the left leg column on both sides 、Local point cloud of right leg column ;

[0047] Step 9.4: According to Calculate left jack The midpoint of the left leg column point cloud;

[0048] according to Calculate left jack The midpoint of the right leg column point cloud;

[0049] According to the hole The midpoint of the left leg column point cloud and the midpoint of the right leg column point cloud are used to calculate the left socket The midpoint coordinates of

[0050] according to Calculate right jack The midpoint of the left leg column point cloud;

[0051] according to Calculate right jack The midpoint of the right leg column point cloud;

[0052] According to the hole The midpoint of the left leg column point cloud and the midpoint of the right leg column point cloud are used to calculate the right socket The midpoint coordinates of

[0053] Step 9.5: According to the left jack The midpoint coordinates and right jack The midpoint coordinates of The midpoint distance of

[0054] Step 9.6, from the right jack The midpoint coordinates, left jack The midpoint coordinates of The midpoint distance and the lateral angle of the pallet The posture of the tray.

[0055] The electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the tray identification and posture calculation method, and the processor is configured to execute the program stored in the memory.

[0056] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the tray identification and posture calculation method when the computer program is executed by a processor.

[0057] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0058] 1. Compared with processing image data, the present invention effectively avoids the influence of lighting factors by processing point cloud data, and significantly improves the robustness of pallet recognition.

[0059] 2. The present invention projects the point cloud data onto a set plane and assigns a corresponding grayscale value to the plane according to the distance from the projection point to the plane, thereby significantly enhancing the contrast between the pallet and the surrounding environment and further improving the accuracy of pallet recognition.

[0060] 3. The present invention converts three-dimensional point cloud data into two-dimensional image data and performs template matching. This method effectively reduces the computational complexity and improves the real-time performance of pallet recognition, so that the forklift robot can quickly and accurately complete pallet recognition and posture calculation.

[0061] 4. The present invention recognizes pallets through image template matching, does not require any marking of the pallets, and realizes fully automatic pallet recognition and posture calculation.

[0062] 5. The present invention simplifies the template production so that it is not limited by the type of pallet and is applicable to pallets of different materials, different numbers of sockets and different shapes. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A flowchart of the pallet recognition and posture calculation of the present invention;

[0064] Figure 2 Schematic diagram of coordinate transformation of the present invention;

[0065] Figure 3 A schematic diagram of a practical application of the test scenario of the present invention;

[0066] Figure 4 A schematic diagram of the template matching result of the present invention;

[0067] Figure 5 Schematic diagram of the pallet template of the present invention. DETAILED DESCRIPTION

[0068] In this embodiment, a pallet recognition and posture calculation method is applied to forklift robot handling operations, such as Figure 1 As shown, the following steps are followed:

[0069] Step 1: Take the midpoint of the contact surface between the forklift robot and the forklift robot body as the origin , taking the forklift robot's forward direction as The positive direction of the axis is perpendicular to the ground and upward. Positive axis direction, determined by the right-hand rule Axis, thus establishing the world coordinate system of the forklift robot ;

[0070] Take the TOF camera light source as , the TOF camera is directly in front of Axis positive direction, TOF camera vertically downward Positive direction of axis, determined by right-hand rule Axis, thus establishing the camera coordinate system ; The coordinate relationship is as follows Figure 2 shown.

[0071] Step 2: The forklift robot uses its own TOF camera to obtain the original 3D point cloud containing the pallet features. After coordinate transformation, the original 3D point cloud containing the pallet features in the world coordinate system of the forklift robot is obtained. The coordinate transformation in this embodiment is to calculate the rotation matrix and the translation matrix according to the forklift's own values ​​and the camera installation position to obtain the transformation matrix.

[0072] Step 3: Use a straight-through filter to Perform background removal and keep the coordinates with the world coordinate system Axis origin The distance range is The point cloud inside is obtained to obtain the filtered three-dimensional point cloud containing the pallet features. ,in, is the closest point of the pallet placement range, It is the farthest point of the pallet placement range;

[0073] Step 4: Use voxel filtering to Sampling is performed to obtain a downsampled 3D point cloud containing the pallet features. ;

[0074] Step 5: Use statistical filters to After performing noise reduction to remove outlier point clouds, the processed 3D point cloud containing the pallet features is obtained. ,in, express The nth point in express Total number of midpoints; record exist The serial number in is recorded as ;

[0075] Step 6: Define perpendicular to K projection planes of the plane is the distance from the TOF camera And with the forklift robot world coordinate system The axis angles are of the plane, where Represents the projection plane With the world coordinate system The angle of the axis, Represents the world coordinate system The axis angle is The projection plane, K represents the total number of projection planes;

[0076] In this embodiment, the projection plane and Axis angle setting and sorting: Set the maximum angle and minimum angle according to the possible yaw angle range of the pallet in actual applications, divide the interval of the maximum angle and the minimum angle into K equal angles, and then sort them according to the actual probability.

[0077] Step 7, initialize k=1;

[0078] Step 8. Calculation Each point in Distance ,in, express The nth point in arrive distance;

[0079] Step 9: Use equation (1) and equation (2) to All points in the projection are And rotate and translate all the projected points so that all the projected points move to plane, thus obtaining the kth transition 3D point cloud ;

[0080] (1)

[0081] (2)

[0082] In formula (1) and formula (2), yes The projection coefficient, yes The three-dimensional coordinates of the nth point in , yes The plane equation is yes The nth point in the projection is The coordinates on the plane; A, B, C, D are the four coefficients of the plane equation respectively;

[0083] Rotate and translate the projection to The plane can keep its original shape characteristics unchanged and The axis coordinates are reset to zero and converted into an approximate two-dimensional image.

[0084] Step 10: The three-dimensional coordinates of all points in , converted into a two-dimensional point set The two-dimensional coordinate set of ,in, express The three-dimensional coordinates of the nth point in , express The two-dimensional coordinates of the nth point in ;

[0085] Step 11: Follow , ,Will The two-dimensional coordinates of all points are proportionally reduced times, and The center of the image is translated to the coordinates , and then translated The two-dimensional coordinates of all points in are rounded to obtain the preprocessed two-dimensional point set , whose coordinates are ,in, Respectively The maximum and minimum values ​​in Respectively The maximum and minimum values ​​in , S is a positive integer, Represents the two-dimensional coordinates of the nth point after preprocessing.

[0086] Step 12: If There are coincident points with the same two-dimensional coordinates in , then all the coincident points are Get the serial number in All coincident points with corresponding serial numbers in the plane The distance between the two points is 0. Delete the remaining coincident points to obtain a two-dimensional point set without coincidence 2D coordinate set ,in, for Middle The coordinates of the points, express The total number of midpoints.

[0087] In this embodiment, the principle of filtering out overlapping points is as follows:

[0088] Because in actual pallet recognition work, the obstacles between the pallet end face and the camera have been dealt with in the obstacle detection stage, so when two or more point clouds are projected to the same pixel point, the distance from the pallet end face point cloud to the projection plane must be a larger value.

[0089] Step 13: The center is located in the two-dimensional coordinate The nth point at The numbers in the matrix Middle Line The element value of the column will The center is located in the two-dimensional coordinate The nth point at The point corresponding to the serial number in The distance matrix Middle Line The element values ​​of the columns are obtained, resulting in two matrices of dimension S×S and the distance matrix ;

[0090] Step 14: Calculate using formula (3) Middle Gray value of a point , and the corresponding grayscale image Middle Line The element value of the column, and divide The element values ​​of the remaining positions other than the two-dimensional coordinates are set to 0, thus obtaining a target grayscale image of dimension S×S ;

[0091] (3)

[0092] In formula (3), Indicates rounding operation. for Located in the middle Line The element value of the column, For the matrix The minimum value in For the matrix The maximum value in is the maximum gray value; Indicates rounding down;

[0093] The environmental fusion process in this embodiment is as follows: when the pallet point cloud and the background point cloud are projected on the same plane, if a binary image is generated, the contour will be lost, causing the pallet projection and the background projection to be fused into a group of pixels, thereby greatly reducing the accuracy of subsequent template matching.

[0094] Step 15: According to the parameters of the pallet used by the forklift robot, Make a proportional pallet template M, with a height of H and a width of W. The left and right holes of the pallet inserted by the forklift robot are marked as ;

[0095] The template making process in this embodiment is as follows: measure the width and height of the tray, the width and height of the jack, and the coordinates of the upper left vertex of the jack on the end face of the tray, and scale down the measured values ​​to make a binary image, with the grayscale value of the end face of the tray set to 255 and the grayscale value of the jack set to 0. Figure 5 Black and white part of the image.

[0096] Step 16: Combine the pallet template M and the target grayscale image , use formula (4) to calculate the upper left corner vertex of M and Coordinates on Similarity score when the ;

[0097] (4)

[0098] In formula (4), is the coordinate in M The gray value at yes The standardized and normalized values, yes Central coordinates The gray value at yes Standardized and normalized values; is the coordinate in R The gray value at Coordinates in M The gray value at is standardized and normalized, yes Central coordinates The gray value at is the standardized and normalized value;

[0099] Step 17: If Exceeding the preset value When , it means that on the kth projection plane The pallet on the is successfully identified, and the k-th identification result is output, including: the confidence box of the pallet location The upper left corner is Coordinates on and a confidence box for where the pallet is located The height H and width W of , and execute step 18; otherwise, it means the kth projection plane After the recognition fails and k+1 is assigned to k, return to step 8 and execute sequentially until k>K. If the Kth projection plane If the recognition still fails, then The target tray was not identified.

[0100] Step 18: Extract the confidence box of the tray location based on M The local two-dimensional pixel coordinates corresponding to the end surface area of ​​the tray within , the local two-dimensional pixel coordinates corresponding to the end surface area of ​​the tray are obtained in The corresponding 3D point cloud , then, from Extract the initial point cloud of the pallet end surface ;

[0101] Step 19: Use the least squares method to calculate the initial point cloud Fitting is performed and the tray end surface equation is obtained from formula (5): , and thus calculate the yaw angle of the pallet ;

[0102] (5)

[0103] In formula (5), is the equation of the tray end face The normal vector of is the yaw angle of the tray.

[0104] Step 20: Screening Equation of the center and tray end face The distance between the corresponding planes is less than the threshold Point Cloud , extraction tray Point cloud of middle pallet legs The relative position of the left socket of the tray is obtained Local point cloud of the left leg column on both sides 、Local point cloud of right leg column And the right socket of the tray Local point cloud of the left leg column on both sides 、Local point cloud of right leg column .

[0105] Step 21: According to Calculate left jack The midpoint of the left leg column point cloud;

[0106] according to Calculate left jack The midpoint of the right leg column point cloud;

[0107] According to the hole The midpoint of the left leg column point cloud and the midpoint of the right leg column point cloud are used to calculate the left socket The midpoint coordinates of

[0108] Step 22: According to Calculate right jack The midpoint of the left leg column point cloud;

[0109] according to Calculate right jack The midpoint of the right leg column point cloud;

[0110] According to the hole The midpoint of the left leg column point cloud and the midpoint of the right leg column point cloud are used to calculate the right socket The midpoint coordinates of .

[0111] Step 23: According to the left jack The midpoint coordinates and right jack The midpoint coordinates of The midpoint distance of

[0112] Step 24: Right jack The midpoint coordinates, left jack The midpoint coordinates of The midpoint distance and the lateral angle of the pallet The posture of the tray.

[0113] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0114] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.

[0115] Pallet recognition and posture calculation method test:

[0116] exist Figure 3 In the process shown, first a frame of point cloud captured by the camera is Figure 3 Part (a) of the figure is preprocessed, including removing out-of-range point clouds, implementing point cloud downsampling, and removing outliers to effectively reduce the number of point clouds, thereby obtaining the processed image as shown in FIG. Figure 3 Then, these point clouds are projected onto a specific projection plane to generate images such as Figure 3 In part (c), during this process, it is found that the tray and the background have a high degree of integration. To this end, the grayscale value is assigned to the projection point according to the distance between the projection point and the projection plane, and then the image is generated as follows: Figure 3 Then, template matching was performed and the matching results are shown in Figure 4 As shown. The pose calculation visualization is shown in the figure Figure 3 As shown in part (e) of the figure, the green rectangle is the fitted pallet end face, and the red point set is the local point set of the pallet end face and the local point set of the pallet leg column. A significant advantage of point cloud data is that it is not affected by lighting conditions and exhibits strong lighting robustness. By preprocessing the point cloud, not only the time consumption of the projection process is significantly reduced, but also the key shape features of the pallet are successfully retained. In addition, the method of converting distance information into grayscale values ​​ensures that there is sufficient contrast between the pallet and the background, thereby highlighting the features of the pallet and greatly improving the accuracy of template matching. By extracting the pallet leg column point cloud of the template matching result, the pallet end face parameters and the coordinates of the midpoint of the pallet socket can be effectively and accurately calculated, and then the complete posture of the pallet can be obtained.

[0117] This test fully demonstrated the reliability of the pallet recognition and posture calculation method. Even in a complex and changeable working environment, the forklift robot can achieve accurate pallet recognition and posture calculation with this method, and its performance is not limited by lighting conditions, and it can maintain excellent performance even at night or in low-light environments. In addition, this method has a relatively small amount of calculation and can run at high speed on mobile devices with limited computing resources, ensuring that the forklift robot can complete the handling task quickly and smoothly. The experimental results show that this method can fully meet the needs of the forklift robot in pallet recognition and posture calculation, and has never failed under the preset conditions, further verifying its excellent stability.

Claims

1. A pallet recognition and posture calculation method, characterized in that it is applied to forklift robot handling operations and is performed in the following steps: Step 1: Establish the camera coordinate system and the world coordinate system of the forklift robot; Step 2: The original 3D point cloud containing the pallet features in the world coordinate system of the forklift robot After preprocessing, the preprocessed 3D point cloud containing the pallet features is obtained. , and record The serial numbers corresponding to all points in ; Step 3, define K projection planes; Step 4: Initialize k=1; Step 5: Project to the kth projection plane, then use the dimensionality reduction algorithm to convert the projected points into a two-dimensional point set, and then use the screening coincidence point algorithm to remove all coincidence points in the two-dimensional point set to obtain a two-dimensional point set without overlap. ; Step 6: Use the conversion algorithm to convert the two-dimensional point set without overlapping points Convert to target grayscale image ; Step 7: According to the parameters of the pallet used by the forklift robot, Make a proportional pallet template M, with a height of H and a width of W. The left and right holes of the pallet inserted by the forklift robot are marked as ; Step 8: Use an image template matching method to obtain the target grayscale image If the tray template M fails to match, k+1 is assigned to k. If k≤K, return to step 5. If k>K, determine There is no pallet in the template; if the pallet template M is matched successfully, go to step 9; Step 9: Combine the matching result of the pallet template M with the pallet template to obtain the pallet pose.

2. A pallet recognition and posture calculation method according to claim 1, characterized in that: The step 1 comprises: The midpoint of the contact surface between the forklift robot and the forklift robot body is taken as the origin. , taking the forklift robot's forward direction as The positive direction of the axis is perpendicular to the ground and upward. Positive axis direction, determined by the right-hand rule Axis, thus establishing the world coordinate system of the forklift robot ; Take the TOF camera light source as , the TOF camera is directly in front of Axis positive direction, TOF camera vertically downward Positive direction of axis, determined by right-hand rule Axis, thus establishing the camera coordinate system .

3. A pallet recognition and posture calculation method according to claim 2, characterized in that: The specific steps of the pretreatment method in step 2 include: Step 2.1: The forklift robot uses its own TOF camera to obtain the original 3D point cloud containing the pallet features. After coordinate transformation, the original 3D point cloud containing the pallet features in the world coordinate system of the forklift robot is obtained. ; Step 2.2: Use a straight-through filter to Perform background removal and keep the coordinates with the world coordinate system Axis origin The distance range is The point cloud inside is obtained to obtain the filtered three-dimensional point cloud containing the pallet features. ,in, is the closest point of the pallet placement range, It is the farthest point of the pallet placement range; Step 2.3: Use voxel filtering to Sampling is performed to obtain a downsampled 3D point cloud containing the pallet features. ; Step 2.4: Use statistical filters to After performing noise reduction to remove outlier point clouds, the preprocessed 3D point cloud containing the pallet features is obtained. ,in, express The nth point in express Total number of midpoints, recorded exist The serial number in is recorded as .

4. A pallet recognition and posture calculation method according to claim 4, characterized in that: The step 3 comprises: Define perpendicular to K projection planes of the plane is the distance from the TOF camera And with the forklift robot world coordinate system The axis angles are The plane, where Represents the projection plane With the world coordinate system The angle of the axis, Represents the world coordinate system The axis angle is The projection plane of , K represents the total number of projection planes.

5. A pallet recognition and posture calculation method according to claim 4, characterized in that: The step 5 comprises: Step 5.1: Calculation Each point in Distance ,in, express The nth point in arrive distance; Step 5.2: All points in the projection are And rotate and translate all the projected points so that all the projected points move to plane, thus obtaining the kth transition 3D point cloud ; Step 5.3: The three-dimensional coordinates of all points in , converted into a two-dimensional point set The two-dimensional coordinate set of ,in, express The three-dimensional coordinates of the nth point in , express The two-dimensional coordinates of the nth point in ; Step 5.4: Follow , ,Will The two-dimensional coordinates of all points are proportionally reduced times, and The center of the image is translated to the coordinates , and then translated The two-dimensional coordinates of all points in are rounded to obtain the preprocessed two-dimensional point set , whose coordinates are ,in, Respectively The maximum and minimum values ​​in Respectively The maximum and minimum values ​​in , S is a positive integer, Represents the two-dimensional coordinates of the nth point after preprocessing; Step 5.5: If There are coincident points with the same two-dimensional coordinates in , then all the coincident points are Get the serial number in All coincident points with corresponding serial numbers in the plane The distance between the two points is 0. Delete the remaining coincident points to obtain a two-dimensional point set without coincidence The two-dimensional coordinate set of ,in, for Middle The coordinates of the points, express The total number of midpoints.

6. A pallet recognition and posture calculation method according to claim 5, characterized in that: The step 6 comprises: Step 6.1: The center is located in the two-dimensional coordinate The nth point at The numbers in the matrix Middle Line The element value of the column will The center is located in the two-dimensional coordinate The nth point at The point corresponding to the serial number in The distance matrix Middle Line The element values ​​of the columns are obtained, resulting in two matrices of dimension S×S and the distance matrix ; Step 6.2: Calculate using formula (1) Middle Gray value of a point , and the corresponding grayscale image Middle Line The element value of the column, and divide The element values ​​of the remaining positions other than the two-dimensional coordinates are set to 0, so as to obtain a target grayscale image with a dimension of S×S ; (1) In formula (1), Indicates rounding operation. for Located in the middle Line The element value of the column, For the matrix The minimum value in For the matrix The maximum value in is the maximum gray value.

7. A pallet recognition and posture calculation method according to claim 6, characterized in that: The image template matching method in step 8 includes: Step 8.1: Combine the tray template M and the target grayscale image , use formula (4) to calculate the upper left corner vertex of M and Coordinates on Similarity score when the ; (2) In formula (2), is the coordinate in M The gray value at yes The standardized and normalized values, yes Central coordinates The gray value at yes Standardized and normalized values; is the coordinate in R The gray value at Coordinates in M The gray value at is standardized and normalized, yes Central coordinates The gray value at is the standardized and normalized value; Step 8.2: If Exceeding the preset value When , it means that on the kth projection plane The pallet on the is successfully identified, and the k-th identification result is output, including: the confidence box of the pallet location The upper left corner is Coordinates on and a confidence box for where the pallet is located The height H and width W of ; otherwise, it represents the kth projection plane Identification failed.

8. A pallet recognition and posture calculation method according to claim 7, characterized in that: The step 9 comprises: Step 9.1: Extract the confidence box of the tray location based on M The local two-dimensional pixel coordinates corresponding to the end surface area of ​​the tray within , the local two-dimensional pixel coordinates corresponding to the end surface area of ​​the tray are obtained in The corresponding 3D point cloud , then, from Extract the initial point cloud of the pallet end surface ; Step 9.2: Use the least squares method to calculate the initial point cloud Perform fitting to obtain the tray end face equation , and thus calculate the yaw angle of the pallet ; Step 9.3: Screening Equation of the center and tray end face The distance between the corresponding planes is less than the threshold Point Cloud , extraction tray Point cloud of middle pallet legs The relative position of the left socket of the tray is obtained Local point cloud of the left leg column on both sides 、Local point cloud of right leg column And the right socket of the tray Local point cloud of the left leg column on both sides 、Local point cloud of right leg column ; Step 9.4: According to Calculate left jack The midpoint of the left leg column point cloud; according to Calculate left jack The midpoint of the right leg column point cloud; According to the hole The midpoint of the left leg column point cloud and the midpoint of the right leg column point cloud are used to calculate the left socket The midpoint coordinates of according to Calculate right jack The midpoint of the left leg column point cloud; according to Calculate right jack The midpoint of the right leg column point cloud; According to the hole The midpoint of the left leg column point cloud and the midpoint of the right leg column point cloud are used to calculate the right socket The midpoint coordinates of Step 9.5: According to the left jack The midpoint coordinates and right jack The midpoint coordinates of The midpoint distance of Step 9.6, from the right jack The midpoint coordinates, left jack The midpoint coordinates of The midpoint distance and the lateral angle of the pallet The posture of the tray.

9. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the pallet identification and posture calculation method described in any one of claims 1 to 8, and the processor is configured to execute the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the pallet identification and posture calculation method described in any one of claims 1 to 8 are executed.

Images