Space pose recognition method and system for tire assembly industry robot unstacking

Abstract

The present application belongs to the field of robot target workpiece space pose detection, and specifically discloses a space pose recognition method and system for tire assembly industrial robot unstacking. It comprises: collecting tire assembly point cloud data of a rack; measuring calibration plate image data at the end of the robot in n poses to calibrate the relationship between the point cloud camera coordinate system and the robot end flange coordinate system; performing normal vector filtering processing on the preprocessed point cloud data to determine the circular boundary of the tire assembly; obtaining tire assembly boundary point cloud through boundary detection, and fitting a three-dimensional circle based on the boundary point cloud to obtain the space pose of the tire assembly; and judging whether the space pose is correct. The present application can realize fast and accurate recognition and positioning of tire assembly by integrating advanced visual recognition technology and pose estimation algorithm, thereby significantly improving the automation level of industrial robots in unstacking operations.

Description

Technical Field

[0001] This invention belongs to the field of spatial pose detection of robot target workpieces, and more specifically, relates to a spatial pose recognition method and system for destacking tire assembly industrial robots. Background Technology

[0002] With the booming development of new energy vehicles, the automotive manufacturing industry is facing an automated transformation and upgrade. Related sub-sectors, such as tire and wheel assembly, are also facing challenges related to capacity, efficiency, and safety. Among these, the production scheduling of tire assemblies is located upstream in the entire automotive industry chain, and its efficiency and quality directly affect the production rhythm and cost control of the entire vehicle manufacturing process. Currently, tire assembly line scheduling still faces many challenges, such as the flexibility required for multi-variety, small-batch production, insufficient automation upgrades of production equipment, and safety hazards in the process flow. Therefore, improving the efficiency and reliability of tire assembly production scheduling has become a practical need for the automotive manufacturing industry.

[0003] The current traditional method of production scheduling still relies mainly on manual unpacking, which involves manually handling tire assemblies and using gantry clamps for assistance. This method is prone to scheduling errors during production, leading to tire assembly problems, which in turn cause consumer dissatisfaction and subsequent driving safety issues. Furthermore, because tire assemblies are heavily loaded, manual handling can easily result in workplace injuries, hindering industrial efficiency and making it difficult to meet the industry's automation upgrade needs.

[0004] Furthermore, in response to the shortcomings of manual unpacking during the production scheduling process, automated equipment, represented by heavy-duty industrial robots, is the general trend of its industry development. For the detection and gripping of target workpieces (i.e. tire assemblies), traditional teaching methods cannot be implemented due to the floating position of the tire assembly rack and the floating position of the target workpiece. Detection methods based on two-dimensional images have difficulty obtaining accurate depth values, which can easily lead to interference and collision problems. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a spatial pose recognition method and system for tire assembly industrial robot depalletizing. It acquires tire assembly point cloud data view-by-view using a point cloud camera mounted on the end effector of the industrial robot, and utilizes point cloud preprocessing algorithms such as voxel filtering, as well as model fitting algorithms such as normal vector filtering and RANSAC 3D circle fitting, ultimately achieving pose recognition and depalletizing / loading of the tire assembly. This invention can be applied to automotive assembly line systems for loading and scheduling assembled tire assemblies, improving the automation level of related production lines, enhancing the robustness of the automotive assembly field, and effectively solving many shortcomings of traditional manual handling, such as inefficiency and stability. It currently has significant industrial application value.

[0006] To achieve the above objectives, according to one aspect of the present invention, a spatial pose recognition method for depalletizing a tire assembly industrial robot is proposed, comprising the following steps:

[0007] Step 1: Collect point cloud data of the material rack tire assembly viewpoint by viewpoint according to the configuration file;

[0008] Step 2: Measure the calibration plate image data at n poses of the robot's end effector to calibrate the relationship between the point cloud camera coordinate system and the robot's end effector flange coordinate system. n≥3;

[0009] Step 3, based on the relationship After preprocessing the point cloud data, normal vector filtering is performed to determine the circular boundary of the tire assembly.

[0010] Step 4: Obtain the boundary point cloud of the tire assembly through boundary detection, and use the RANSAC algorithm to fit a three-dimensional circle based on the boundary point cloud to obtain the spatial pose of the tire assembly.

[0011] Step 5: Calculate the estimated pose of the tire assembly based on the position of the tire assembly rack and the distribution pattern of the tire assembly. Determine whether the spatial pose is correct based on the error between the estimated pose and the spatial pose. If correct, output the result; otherwise, return to step 3.

[0012] As a further preferred option, step one also includes the following steps:

[0013] The configuration file records multiple viewpoint positions for each tire layer, and the viewpoints of the current layer are traversed to collect complete point cloud data for the current layer.

[0014] As a further preferred option, step two includes the following steps:

[0015] (21) Traverse the n end flange poses of the robot and record the n poses, measure the calibration plate image data and perform grayscale processing;

[0016] (22) Perform Gaussian blurring and binarization on the grayscale image of the calibration plate;

[0017] (23) Calculate the robot hand-eye relationship based on the binarized calibration plate image and the camera intrinsic parameter matrix and distortion coefficients to obtain the relationship between the camera coordinate system and the robot end flange coordinate system.

[0018] As a further preferred embodiment, step three, which involves preprocessing the point cloud data and then performing normal vector filtering, includes the following steps:

[0019] (31) Based on the robot hand-eye relationship The multiple point cloud images obtained in step one are unified under the robot's base coordinate system to obtain complete point cloud data;

[0020] (32) Based on the XYZ coordinate range of the spatial location of the rack, perform pass-through filtering on the initial point cloud data to filter the point cloud data of the ground and the rack frame.

[0021] (33) Perform voxel filtering sampling on the point cloud data after the pass-through filtering process, construct a point cloud voxel map, calculate the mean of the point cloud in each voxel, and replace all the point clouds in the voxel with the mean.

[0022] (34) After voxel filtering sampling, due to the fluctuation and noise interference of the data collected by the camera each time, the voxel-filtered point cloud is processed by Gaussian filtering.

[0023] (35) Perform normal vector filtering on the point cloud data after Gaussian filtering to determine the circular boundary of the tire assembly.

[0024] As a further preferred option, step (35) includes the following steps:

[0025] (351) Perform PCA principal component analysis on the point cloud data after Gaussian filtering to obtain the normal vector of the point cloud data;

[0026] (352) Since the point cloud coordinate system under the unified coordinate system is the robot's base coordinate system, and the z-axis of the base coordinate system coincides with the z-axis of the material rack coordinate system, in the normal vector filtering process, the angle between the normal vector of point p and the z-axis is used as the filtering object. When the angle is greater than the threshold θ threshold Remove the point when the included angle is less than the threshold θ. threshold If the point is retained, the circular boundary of the tire assembly is obtained after the normal vector filtering process.

[0027] As a further preferred embodiment, the PCA principal component analysis includes:

[0028] Given a point p, find its n nearest neighbors or neighbors within a radius r using a binary tree. Calculate the covariance matrix of these n+1 points, and then calculate their eigenvalues ​​and eigenvectors. The eigenvector corresponding to the smallest eigenvalue is the normal vector of point p.

[0029] As a further preferred embodiment, step four, which involves fitting a three-dimensional circle using the RANSAC algorithm based on the boundary point cloud to obtain the spatial pose of the tire assembly, includes:

[0030] (41) Perform Euclidean clustering on the point cloud data after normal vector filtering, based on the maximum number of tire assemblies n in each layer. max Segment the point clouds of multiple tire assemblies to facilitate subsequent calculation of their poses one tire at a time;

[0031] (42) For the point cloud data of each tire assembly, the boundary point cloud is calculated by the corner clearance algorithm, and the boundary point cloud of each tire assembly is extracted one by one.

[0032] (43) After obtaining n max After obtaining the boundary point cloud of each target workpiece, the RANSAC fitting three-dimensional circle algorithm is used to calculate the center and axis direction of the three-dimensional circle, which is the spatial pose of the tire assembly.

[0033] As a further preferred embodiment, in step (42), the angular gap algorithm includes:

[0034] Traverse the tire assembly point cloud. Assuming the current point is p, find its n nearest neighbors using a binary tree, or all points within a radius r as its neighbors. Using the current point as the origin, map the n neighboring points onto a plane perpendicular to the normal vector of point p, and construct a base coordinate system. At this point, the coordinates of each neighboring point are... n = 1...n, will Sort from largest to smallest As the corner gap between adjacent points, determine the maximum corner gap. When θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When point p is a non-boundary point;

[0035] Preferably, in step (43), the RANSAC three-dimensional circle fitting algorithm process is as follows:

[0036] (431) Defining a mathematical model of a three-dimensional circle requires the spatial coordinates of at least three points;

[0037] (432) In a single loop, n points are randomly selected for the boundary point cloud, n≥3, and the three-dimensional circle model and model parameters are calculated using the least squares method;

[0038] (433) Traverse the boundary point cloud. Assume the current point is p. When the distance between p and the 3D circle model is less than the threshold l threshold Let point p be an interior point. When the distance between p and the 3D circle model is greater than or equal to a threshold l... threshold Let point p be an outer point, and count the number of inner points;

[0039] (434) Repeat steps (432) and (433). When the number of points in the (k+1)th cycle is greater than the number of points in the (k)th cycle, the optimal 3D circle model parameters are updated to the result of the (k+1)th cycle.

[0040] (435) Set a loop termination condition. When the number of loops reaches the loop termination condition, after the loop ends, take the three-dimensional circle model with the largest number of interior points as the final solution model.

[0041] As a further preferred option, step five includes the following steps:

[0042] (51) Under the robot base coordinate system, based on the tire assembly rack position and tire assembly distribution pattern, the (x, y) coordinates of the tire assembly are statistically analyzed. The average value of the (x, y) coordinates of the tire assembly is used as its position estimate. Meanwhile, the tire assembly depth value z is distributed in a discrete equidistant interval. Thus, the above (x, y) coordinates and z coordinates are compiled into a configuration file to obtain the (x, y, z) coordinates of the n tire assemblies in the i-th single layer.

[0043] (52) Based on the i-th layer, the z-axis depth estimate of its configuration file is known. Combined with the estimated coordinate values ​​(x, y) in the configuration file, n estimated tire assembly coordinate values ​​(x, y, z) are obtained. The calculated values ​​are traversed to find the shortest distance between them and the estimated values. The sum of the shortest distances of the n points is counted, which is the sum of the errors between the calculated values ​​and the estimated values. When the sum of the errors is greater than or equal to the preset threshold, the current calculation has an error and returns to step three. When the sum of the errors is less than the preset threshold, the current calculation passes.

[0044] According to another aspect of the present invention, a spatial pose recognition system for depalletizing a tire assembly industrial robot is also provided, comprising:

[0045] The first main control module is used to collect point cloud data of the material rack tire assembly from viewpoint to viewpoint according to the configuration file.

[0046] The second main control module is used to measure calibration board image data in n poses of the robot's end effector to calibrate the relationship between the point cloud camera coordinate system and the robot's end effector flange coordinate system. n≥3;

[0047] The third main control module is used to determine the relationship. After preprocessing the point cloud data, normal vector filtering is performed to determine the circular boundary of the tire assembly.

[0048] The fourth main control module is used to obtain the boundary point cloud of the tire assembly through boundary detection, and to fit a three-dimensional circle based on the boundary point cloud using the RANSAC algorithm to obtain the spatial pose of the tire assembly.

[0049] The fifth main control module is used to calculate the estimated pose of the tire assembly based on the position of the tire assembly rack and the distribution pattern of the tire assembly. It determines whether the spatial pose is correct based on the error between the estimated pose and the spatial pose. If it is correct, it outputs the result; otherwise, it triggers the third main control module to re-run the calculation.

[0050] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages:

[0051] 1. This invention solves the pose recognition problem of tire assemblies using point cloud processing technology. By mounting a point cloud camera on the end effector of an industrial robot to collect point cloud data of the tire assembly view-by-view, and utilizing point cloud preprocessing algorithms such as voxel filtering, as well as model fitting algorithms such as normal vector filtering and RANSAC 3D circle fitting, the pose recognition and destacking / loading of the tire assembly are ultimately achieved. This example can be applied to automotive assembly line systems to load and schedule assembled tire assemblies, improving the automation level of related production lines, enhancing the robustness of automotive assembly, and effectively addressing many shortcomings of traditional manual handling, such as inefficiency and stability. It currently has significant industrial application value.

[0052] 2. By integrating advanced visual recognition technology and pose estimation algorithms, this invention enables rapid and accurate identification and positioning of tire assemblies, thereby significantly improving the automation level of industrial robots in depalletizing operations. This increased automation not only reduces manual intervention and labor intensity but also shortens the work cycle and improves production efficiency.

[0053] 3. This invention utilizes advanced technologies such as PCA principal component analysis, RANSAC algorithm, and angular clearance algorithm. This scheme can accurately extract the boundaries of the tire assembly from complex point cloud data and fit its three-dimensional spatial pose. The combined use of these methods improves the accuracy of pose recognition, and due to the robustness of the algorithm, it can maintain a high recognition accuracy even when the point cloud data contains noise and incompleteness.

[0054] 4. This invention reduces reliance on manual operation through intelligent pose recognition and depalletizing processes, thereby lowering labor costs. Simultaneously, precise pose control and optimized depalletizing strategies reduce material waste and equipment wear caused by misidentification or improper operation, further reducing production costs. Furthermore, the introduction of an automated depalletizing system also helps improve operational safety and reduce workplace accidents. Attached Figure Description

[0055] Figure 1 This is a flowchart of a tire assembly pose recognition method according to an embodiment of the present invention;

[0056] Figure 2 This is a flowchart of robot hand-eye calibration according to an embodiment of the present invention;

[0057] Figure 3 This is a grayscale image of the calibration board according to one embodiment of the present invention;

[0058] Figure 4 This is a binarized image of the calibration board according to an embodiment of the present invention;

[0059] Figure 5 This is an image showing the result of calibration point detection and drawing on a calibration board according to an embodiment of the present invention;

[0060] Figure 6 This is a flowchart of point cloud data preprocessing according to an embodiment of the present invention;

[0061] Figure 7 This is an embodiment of the present invention, which unifies multiple point clouds to an image under a robot base coordinate system.

[0062] Figure 8 This is a point cloud image after pass-through filtering as described in one embodiment of the present invention;

[0063] Figure 9 This is a flowchart of tire assembly pose calculation based on preprocessed point cloud according to an embodiment of the present invention;

[0064] Figure 10 This is a preprocessed point cloud normal vector calculation diagram according to an embodiment of the present invention;

[0065] Figure 11 This is the result image after filtering the point cloud normal vectors according to one embodiment of the present invention;

[0066] Figure 12 This is a point cloud boundary calculation image after normal vector filtering, as described in one embodiment of the present invention;

[0067] Figure 13 This is an image showing the calculation results for a single-layer tire assembly according to an embodiment of the present invention;

[0068] Figure 14 This is a flowchart illustrating the algorithm calculation result verification process according to an embodiment of the present invention. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0070] Example 1

[0071] like Figure 1 As shown in the figure, an embodiment of the present invention provides a spatial pose recognition method for depalletizing a tire assembly industrial robot, comprising:

[0072] Step 1: Collect point cloud data of the material rack tire assembly viewpoint by viewpoint according to the configuration file.

[0073] In this step, a configuration file is used to record multiple viewpoint positions for each tire layer, and the viewpoints of the current layer are traversed to collect complete point cloud data for the current layer.

[0074] More specifically, when a forklift moves a tire assembly rack to a designated area, a point cloud camera is mounted on the robot's end effector. However, due to the limited field of view of the point cloud camera, it is not possible to obtain complete point cloud data of the upper tire assembly on the rack after a single measurement. Therefore, a configuration file (such as txt, json, xml, etc.) is used. The configuration file records multiple viewpoint positions of each tire layer. By traversing the viewpoints of the current layer, complete point cloud data of the current layer can be collected.

[0075] Optionally, in one embodiment of the present invention, the ROI of the target workpiece in the depth map is obtained through a convolutional neural network. The depth map is then converted into raw point cloud data of the target workpiece based on the depth camera intrinsic parameter matrix and the ROI of the target workpiece in the depth map. More specifically,

[0076] (11) Collect tire assembly depth map data under actual working conditions, label the ROI of the tire skin that can be grabbed at the top of the depth map, construct its tire assembly depth map dataset, and use the dataset to train a convolutional neural network model.

[0077] (12) Input the depth map of the target workpiece into the trained convolutional neural network model to obtain the ROI of the target workpiece in the depth map.

[0078] More specifically, in one embodiment, a mask is created based on the ROI of the target workpiece in the depth map, and the depth map is filtered based on the mask. During the filtering process, areas outside the ROI are set to 0. The filtered depth map data is then converted into point cloud data based on the intrinsic parameter matrix coefficients of the depth camera.

[0079] Optional, the above also includes:

[0080] (The intrinsic parameter matrix coefficients for calibrating the depth camera include: the focal length f along the x-axis) x y-axis focal length f y x-axis pixel offset c x y-axis pixel offset c y ;

[0081] Traverse the depth map pixels in a loop to obtain their pixel coordinates (u, v) and the gray value at the pixel, which is the depth value at that point.

[0082] If the depth value of the point is set to 0, skip it; otherwise, calculate the point cloud coordinates corresponding to its pixel and obtain the original point cloud data of the target workpiece. The calculation formula for the point cloud coordinates corresponding to the pixel includes:

[0083]

[0084] Where (X, Y, Z) are the point cloud coordinates corresponding to the pixel.

[0085] More specifically, it also includes:

[0086] Depth maps of the tire assembly are captured from different angles using multiple depth cameras. These depth maps are then fused using a viewpoint fusion weighting formula, which includes:

[0087]

[0088] In the formula, Wi is the weight of the i-th viewpoint, Di is the distance from the i-th viewpoint to the target tire, Dj represents the distance from the j-th viewpoint to the target tire, σ is the standard deviation used to adjust the sensitivity of weight allocation, and n is the total number of viewpoints.

[0089] The fused depth map is as follows:

[0090]

[0091] In the formula, Dfused(u,v) is the depth value of the fused depth map at pixel coordinates (u,v), Wi is the weight of the i-th view, and Di,depth(u,v) is the depth value of the depth map of the i-th view at pixel coordinates (u,v).

[0092] Step 2: Measure the calibration plate image data at n poses of the robot's end effector to calibrate the relationship between the point cloud camera coordinate system and the robot's end effector flange coordinate system. n≥3. Specifically:

[0093] (21) Traverse the n end flange poses of the robot and record the n poses, measure the calibration plate image data and perform grayscale processing;

[0094] (22) Perform Gaussian blurring and binarization on the grayscale image of the calibration plate;

[0095] (23) Calculate the robot hand-eye relationship based on the binarized calibration plate image and the camera intrinsic parameter matrix and distortion coefficients to obtain the relationship between the camera coordinate system and the robot end flange coordinate system.

[0096] In this step, the grayscale image of the calibration board is denoised based on a Gaussian kernel and then binarized to improve the accuracy and efficiency of the calibration board image recognition, thereby enhancing the robustness of subsequent calculation processes.

[0097] Secondly, after detecting the coordinates of the calibration points in the preprocessed binary image of the calibration board, the sub-pixel coordinates are calculated, and the extrinsic parameter matrix is ​​calculated based on the camera intrinsic parameter matrix and distortion coefficients, i.e., the relationship between the calibration board coordinate system and the camera coordinate system. Finally, based on the end-effector pose and the calculated camera extrinsic parameter matrix, the hand-eye relationship between the point cloud camera and the robot's end effector is obtained, i.e., the relationship between the point cloud camera coordinate system and the robot's end-effector coordinate system.

[0098] More specifically, calculating relationships This includes: generating 3D coordinates of calibration points based on the inherent geometric parameters of the calibration board; detecting the coordinates of calibration points in the binarized image of the calibration board and calculating their sub-pixel coordinates; calculating the corresponding camera extrinsic matrix based on the calibration point image coordinates, the camera intrinsic matrix, and the distortion coefficients; and calculating the robot hand-eye relationship (i.e., the relationship between the point cloud camera coordinate system and the robot end-effector coordinate system) based on the n camera extrinsic matrices and the n robot end-effector poses.

[0099] Step 3, based on the relationship The point cloud data is preprocessed and then subjected to normal vector filtering to determine the circular boundary of the tire assembly. The preprocessing includes: unifying the coordinate system, pass-through filtering, voxel sampling, Gaussian filtering, and normal vector calculation.

[0100] Specifically, in one embodiment of the present invention, the preprocessing of the point cloud data followed by normal vector filtering includes the following steps:

[0101] (31) Based on the robot hand-eye relationship The multiple point cloud images obtained in step one are unified under the robot's base coordinate system to obtain complete point cloud data;

[0102] (32) Based on the XYZ coordinate range of the spatial location of the rack, perform pass-through filtering on the initial point cloud data to filter the point cloud data of the ground and the rack frame.

[0103] (33) Perform voxel filtering sampling on the point cloud data after the pass-through filtering process, construct a point cloud voxel map, calculate the mean of the point cloud in each voxel, and replace all the point clouds in the voxel with the mean.

[0104] (34) After voxel filtering sampling, due to the fluctuation and noise interference of the data collected by the camera each time, the voxel-filtered point cloud is processed by Gaussian filtering.

[0105] (35) Perform normal vector filtering on the point cloud data after Gaussian filtering to determine the circular boundary of the tire assembly.

[0106] Step (35) includes the following steps:

[0107] (351) Perform PCA principal component analysis on the point cloud data after Gaussian filtering to obtain the normal vector of the point cloud data;

[0108] (352) Since the point cloud coordinate system under the unified coordinate system is the robot's base coordinate system, and the z-axis of the base coordinate system coincides with the z-axis of the material rack coordinate system, in the normal vector filtering process, the angle between the normal vector of point p and the z-axis is used as the filtering object. When the angle is greater than the threshold θ threshold Remove the point when the included angle is less than the threshold θ. threshold If the point is retained, the circular boundary of the tire assembly is obtained after the normal vector filtering process.

[0109] Alternatively, in one embodiment, the PCA principal component analysis includes: determining a point p, finding its n nearest neighbors or neighboring points within a neighborhood with radius r using a binary tree, calculating the covariance matrix of these n+1 points, thereby calculating its eigenvalues ​​and eigenvectors, and the eigenvector corresponding to the smallest eigenvalue is the normal vector of point p.

[0110] Step 4: Obtain the boundary point cloud of the tire assembly through boundary detection, and use the RANSAC algorithm to fit a three-dimensional circle based on the boundary point cloud to obtain the spatial pose of the tire assembly.

[0111] Optionally, the step of fitting a three-dimensional circle using the RANSAC algorithm based on the boundary point cloud to obtain the spatial pose of the tire assembly includes:

[0112] (41) Perform Euclidean clustering on the point cloud data after normal vector filtering, based on the maximum number of tire assemblies n in each layer. max Segment the point clouds of multiple tire assemblies to facilitate subsequent calculation of their poses one tire at a time;

[0113] (42) For the point cloud data of each tire assembly, the boundary point cloud is calculated by the corner clearance algorithm, and the boundary point cloud of each tire assembly is extracted one by one.

[0114] (43) After obtaining n max After obtaining the boundary point cloud of each target workpiece, the RANSAC fitting three-dimensional circle algorithm is used to calculate the center and axis direction of the three-dimensional circle, which is the spatial pose of the tire assembly.

[0115] In step (42), the angular gap algorithm includes:

[0116] Traverse the tire assembly point cloud. Assuming the current point is p, find its n nearest neighbors using a binary tree, or all points within a radius r as its neighbors. Using the current point as the origin, map the n neighboring points onto a plane perpendicular to the normal vector of point p, and construct a base coordinate system. At this point, the coordinates of each neighboring point are... n = 1…n, will Sort from largest to smallest As the corner gap between adjacent points, determine the maximum corner gap. When θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When p is a non-boundary point.

[0117] Preferably, in step (43), the RANSAC three-dimensional circle fitting algorithm process is as follows:

[0118] (431) Defining a mathematical model of a three-dimensional circle requires the spatial coordinates of at least three points;

[0119] (432) In a single loop, n points are randomly selected for the boundary point cloud, n≥3, and the three-dimensional circle model and model parameters are calculated using the least squares method;

[0120] (433) Traverse the boundary point cloud. Assume the current point is p. When the distance between p and the 3D circle model is less than the threshold l threshold Let point p be an interior point. When the distance between p and the 3D circle model is greater than or equal to a threshold l... threshold Let point p be an outer point, and count the number of inner points;

[0121] (434) Repeat steps (432) and (433). When the number of points in the (k+1)th cycle is greater than the number of points in the (k)th cycle, the optimal 3D circle model parameters are updated to the result of the (k+1)th cycle.

[0122] (435) Set a loop termination condition. When the number of loops reaches the loop termination condition, after the loop ends, take the three-dimensional circle model with the largest number of interior points as the final solution model.

[0123] Step 5: Based on the tire assembly rack position and tire assembly distribution pattern, calculate the estimated pose of the tire assembly. Determine if the spatial pose is correct based on the error between the estimated pose and the spatial pose. If correct, output the result; otherwise, return to Step 3 to redetermine the circular boundary of the tire assembly. Specifically:

[0124] (51) Under the robot base coordinate system, based on the tire assembly rack position and tire assembly distribution pattern, the (x, y) coordinates of the tire assembly are statistically analyzed. The average value of the (x, y) coordinates of the tire assembly is used as its position estimate. Meanwhile, the tire assembly depth value z is distributed in a discrete equidistant interval. Thus, the above (x, y) coordinates and z coordinates are compiled into a configuration file to obtain the (x, y, z) coordinates of the n tire assemblies in the i-th single layer.

[0125] (52) Based on the i-th layer, the z-axis depth estimate of its configuration file is known. Combined with the estimated coordinate values ​​(x, y) in the configuration file, n estimated tire assembly coordinate values ​​(x, y, z) are obtained. The calculated values ​​are traversed to find the shortest distance between them and the estimated values. The sum of the shortest distances of the n points is counted, which is the sum of the errors between the calculated values ​​and the estimated values. When the sum of the errors is greater than or equal to the preset threshold, the current calculation is incorrect and returns to step three. When the sum of the errors is less than the preset threshold, the current calculation is successful, which means that the spatial pose obtained in step four is correct.

[0126] Example 2

[0127] This embodiment provides a spatial pose recognition system for depalletizing tire assembly industrial robots. This system is used to implement the methods of any one or a combination of the above embodiments, including:

[0128] The first main control module is used to collect point cloud data of the material rack tire assembly from viewpoint to viewpoint according to the configuration file.

[0129] The second main control module is used to measure calibration board image data in n poses of the robot's end effector to calibrate the relationship between the point cloud camera coordinate system and the robot's end effector flange coordinate system. n≥3;

[0130] The third main control module is used to determine the relationship. After preprocessing the point cloud data, normal vector filtering is performed to determine the circular boundary of the tire assembly.

[0131] The fourth main control module is used to obtain the boundary point cloud of the tire assembly through boundary detection, and to fit a three-dimensional circle based on the boundary point cloud using the RANSAC algorithm to obtain the spatial pose of the tire assembly.

[0132] The fifth main control module is used to calculate the estimated pose of the tire assembly based on the position of the tire assembly rack and the distribution pattern of the tire assembly. It determines whether the spatial pose is correct based on the error between the estimated pose and the spatial pose. If it is correct, it outputs the result; otherwise, it triggers the third main control module to re-run the calculation.

[0133] Example 3

[0134] This embodiment provides a spatial pose recognition method for depalletizing tire assembly industrial robots, such as... Figure 1 As shown, it includes:

[0135] Step 1: Collect point cloud data of tire assemblies on the rack. The tire assembly rack is moved to the designated area by a forklift. A point cloud camera is mounted on the robot's end effector. However, due to the limited field of view of the point cloud camera, it is impossible to obtain complete point cloud data of the upper layer of tire assemblies on the rack after a single measurement. Therefore, a configuration file (such as txt, json, xml, etc.) is used. The configuration file records multiple viewpoint positions for each layer of tires. By traversing the viewpoints of the current layer, complete point cloud data of the current layer can be collected.

[0136] Step 2: After measuring and preprocessing the calibration board image data at n (n≥3) poses of the robot end effector, determine the relationship between the calibration point cloud camera coordinate system and the robot end effector flange coordinate system. like Figure 2 As shown.

[0137] First, the calibration board image is preprocessed, specifically including:

[0138] Traverse the n (n≥3) end flange poses of the robot and record the n poses. Measure the calibration plate image data and perform grayscale processing, such as... Figure 3 As shown. Traverse the robot's n (n≥3) poses, read the calibration board image data, and convert the three-channel data into single-channel grayscale data to meet the needs of subsequent sub-pixel calculations.

[0139] Secondly, the grayscale image of the calibration board is subjected to Gaussian blurring and binarization, such as... Figure 4 As shown, the grayscale image of the calibration board is denoised using a Gaussian kernel and then binarized, thereby improving the accuracy and efficiency of calibration board image recognition and enhancing the robustness of subsequent computation processes.

[0140] Finally, after detecting the coordinates of the calibration points in the preprocessed binary image of the calibration board, the sub-pixel coordinates are calculated, and the extrinsic matrix is ​​calculated based on the camera intrinsic matrix and distortion coefficients, i.e., the relationship between the calibration board coordinate system and the camera coordinate system. Finally, based on the end-effector pose and the calculated camera extrinsic parameter matrix, the hand-eye relationship between the point cloud camera and the robot's end effector is obtained, i.e., the relationship between the point cloud camera coordinate system and the robot's end-effector coordinate system. Specifically, it includes:

[0141] Generate the 3D coordinates of the calibration points based on the inherent geometric parameters of the calibration plate.

[0142] Detect the coordinates of calibration points in the binarized image of the calibration board, and calculate their sub-pixel coordinates, such as... Figure 5 As shown.

[0143] The corresponding camera extrinsic matrix is ​​calculated based on the coordinates of the calibration point image, the camera intrinsic matrix, and the distortion coefficients.

[0144] The robot's hand-eye relationship is calculated based on n camera extrinsic parameter matrices and n robot end-effector poses, i.e., the relationship between the point cloud camera coordinate system and the robot end-effector coordinate system.

[0145] Step 3: Perform data preprocessing steps on the point cloud data, such as unifying the coordinate system, pass-through filtering, voxel sampling, Gaussian filtering, and normal vector calculation. Figure 6 As shown.

[0146] (31): The multiple point cloud data acquired per viewpoint in step one are not in the same coordinate system, but in their respective camera coordinate systems. Therefore, it is necessary to transform the multiple point cloud data to the robot base coordinate system based on the robot hand-eye relationship and viewpoint pose obtained in step one, so as to unify the point cloud data coordinate system. Figure 7 As shown.

[0147] For complete point cloud data, including the point cloud of the main target workpiece (i.e., the tire assembly), the background point cloud, and the noisy point cloud, the initial point cloud data cannot be directly used to calculate the pose of the target workpiece. Therefore, preprocessing of the initial point cloud is necessary.

[0148] After obtaining complete point cloud data in a unified coordinate system, preprocessing is required to accelerate the pose calculation method and improve the robustness of the pose calculation system. The specific preprocessing steps for the point cloud data include:

[0149] (32): Based on the XYZ coordinate range of the material rack's spatial location, the initial point cloud data is subjected to pass-through filtering to filter the point cloud data of the ground and the material rack frame, laying the groundwork for subsequent Euclidean clustering, such as... Figure 8 As shown.

[0150] (33): Perform voxel filtering sampling on the point cloud after direct filtering to construct a point cloud voxel map. Calculate the mean value of the point cloud within each voxel and replace all point clouds within that voxel with the mean value. This reduces data redundancy while preserving most of the geometric information of the original data, thereby accelerating the subsequent calculation process.

[0151] (34): After voxel filtering sampling, due to the fluctuation and noise interference of the data collected by the camera each time, the voxel-filtered point cloud is processed by Gaussian filtering to reduce the fluctuation and noise interference of the point cloud data collected each time.

[0152] After point cloud data preprocessing, it was observed that no obvious geometric model matched the point cloud data of the target workpiece (i.e., tire assembly). Simultaneously, the edge normals of the target workpiece were observed to be continuously distributed. Therefore, a normal vector filtering function was designed to filter and obtain the circular boundary of the target workpiece, which could then be used as the basis for subsequent pose calculations. The pose calculation process is as follows: Figure 9 As shown.

[0153] As shown in Figure x, the preprocessed point cloud is subjected to normal vector filtering, specifically including:

[0154] The normal vector of the preprocessed point cloud is calculated point by point using a PCA-based principal component analysis method, such as... Figure 10 As shown.

[0155] Principal Component Analysis (PCA) determines a point p and uses a binary tree to find its n nearest neighbors or neighbors within a neighborhood with a radius of r. It then calculates the covariance matrix of these n+1 points, thereby calculating their eigenvalues ​​and eigenvectors. The eigenvector corresponding to the smallest eigenvalue is the normal vector of point p.

[0156] Since the point cloud coordinate system under the unified coordinate system is the robot's base coordinate system, and the z-axis of the base coordinate system coincides with the z-axis of the workpiece (i.e., the rack coordinate system), the angle between the normal vector of point p and the z-axis is used as the filtering object during the normal vector filtering process. When the angle is greater than the threshold θ, the filtering is applied. threshold Remove the point when the included angle is less than the threshold θ. threshold If so, retain that point, and the filtering result is as follows: Figure 11 As shown. In this embodiment, the threshold θ threshold Set it between 20-40°.

[0157] After the normal vector filtering process, the target workpiece point cloud has a relatively clear circular boundary. After obtaining its boundary point cloud through boundary detection, the center coordinates and axis direction of its boundary can be obtained according to the RANSAC three-dimensional circle fitting method, that is, the pose result of the target workpiece (tire assembly).

[0158] The specific methods for calculating the pose of the target workpiece in the point cloud after normal vector filtering include:

[0159] Perform Euclidean clustering on the filtered point cloud data based on the maximum number of tire assemblies n in each layer. max Segment the point clouds of multiple target workpieces to facilitate subsequent pose calculations for each workpiece individually.

[0160] In the loop, for each target workpiece point cloud data, its boundary point cloud is calculated using an angular gap-based algorithm, and the boundary point cloud of each target workpiece is extracted one by one, such as... Figure 12 As shown.

[0161] The angular gap algorithm works as follows: traverse the point cloud. Assuming the current point is p, find its n nearest neighbors using a binary tree, or all points within a neighborhood of radius r. Using the current point as the origin, map the n neighborhood points onto a plane perpendicular to the normal vector of point p, and construct a base coordinate system. At this point, the coordinates of each neighboring point are... n = 1…n, will Sort from largest to smallest As the corner gap between adjacent points, determine the maximum corner gap. When θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When p is a non-boundary point.

[0162] After obtaining n max After obtaining the boundary point cloud of the target workpiece, a RANSAC-fitted 3D circle algorithm is used to calculate the center and axial direction of the 3D circle, which is the pose result of the target workpiece (tire assembly). Figure 13 As shown.

[0163] The RANSAC 3D circle fitting algorithm process is as follows:

[0164] Defining a mathematical model of a three-dimensional circle requires the spatial coordinates of at least three points.

[0165] In a single loop, n (n≥3) points are randomly selected from the boundary point cloud, and the three-dimensional mathematical model of the circle is calculated using the least squares method.

[0166] Traverse the boundary point cloud. Assuming the current point is p, when the distance between p and the 3D circle model is less than the threshold l... threshold Let point p be an interior point. When the distance between p and the 3D circle model is greater than or equal to a threshold l... threshold Let point p be an exterior point, and count the number of interior points.

[0167] Iterate through the above loop. When the number of points in the (k+1)th loop is greater than the number of points in the kth loop, the optimal 3D circle model parameters are updated to the result of the (k+1)th loop. The loop terminates when the number of loops reaches a preset value.

[0168] After the loop ends, the 3D circle model with the largest number of interior points is selected as the final solution model.

[0169] Step 4: After calculating multiple poses of the single-layer target workpiece (i.e., the tire assembly), a method for judging the correctness of the calculation results is still needed. This method checks the correctness of the aforementioned algorithm's calculations to avoid interference and collision problems between the fixture and the environment or the target workpiece, thereby enhancing the stability of the pose recognition algorithm. Figure 14 As shown.

[0170] Since the tire assembly rack is transported to the designated area by forklift each time, and since the tire assemblies are arranged in a certain regularity, there is a potential estimated value for the position of the tire assembly. Therefore, it is only necessary to compare the error between the value calculated by the pose detection algorithm and the estimated value to determine the correctness of the calculation result.

[0171] The specific process is as follows:

[0172] (41): Under the robot base coordinate system, based on the position of the tire assembly rack and the general distribution pattern of the tire assembly, the (x, y) coordinates of the tire assembly are statistically analyzed, and the average value of the tire assembly is used as the estimated value of its position. Meanwhile, the tire assembly depth value z is distributed in a discrete and equidistant manner, so that the above (x, y) coordinates and z coordinates can be compiled into a configuration file.

[0173] (42): After the aforementioned detection algorithm is used to calculate, the (x, y, z) coordinates of the n tire assemblies in the i-th layer are obtained. Based on the i-th layer, the estimated value of the z-axis depth in the configuration file is known. Combined with the estimated coordinate values ​​of (x, y) in the configuration file, n estimated tire assembly coordinate values ​​(x, y, z) can also be obtained. The calculated values ​​are traversed to find the shortest distance between them and the estimated values. The sum of the shortest distances of the n points is counted, which is the sum of the errors between the calculated values ​​and the estimated values. When the sum of the errors is greater than or equal to the preset threshold, the current calculation is incorrect. When the sum of the errors is less than the preset threshold, the current calculation is successful.

[0174] This example demonstrates how point cloud processing technology solves the pose recognition problem for tire assemblies. By mounting a point cloud camera on the end effector of an industrial robot and collecting point cloud data of the tire assembly view-by-view, and utilizing point cloud preprocessing algorithms such as voxel filtering, as well as model fitting algorithms such as normal vector filtering and RANSAC 3D circle fitting, pose recognition and destacking / loading of the tire assembly are ultimately achieved. This example can be applied to automotive assembly line systems for loading and scheduling assembled tire assemblies, improving the automation level of related production lines, enhancing the robustness of automotive assembly, and effectively addressing many shortcomings of traditional manual handling, such as inefficiency and stability. It has significant industrial application value.

[0175] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A spatial pose recognition method for destacking tire assembly industrial robots, characterized in that, Includes the following steps: Step 1: Collect point cloud data of the material rack tire assembly viewpoint by viewpoint according to the configuration file; Step 2: Measure the calibration plate image data at n poses of the robot's end effector to calibrate the relationship between the point cloud camera coordinate system and the robot's end effector flange coordinate system. n≥3; Step 3, based on the relationship After preprocessing the point cloud data, normal vector filtering is performed to determine the circular boundary of the tire assembly. Step 4: Obtain the boundary point cloud of the tire assembly through boundary detection, and use the RANSAC algorithm to fit a three-dimensional circle based on the boundary point cloud to obtain the spatial pose of the tire assembly. Step 5: Calculate the estimated pose of the tire assembly based on the position of the tire assembly rack and the distribution pattern of the tire assembly. Determine whether the spatial pose is correct based on the error between the estimated pose and the spatial pose. If correct, output the result; otherwise, return to step 3. In step four, the step of fitting a three-dimensional circle using the RANSAC algorithm based on the boundary point cloud to obtain the spatial pose of the tire assembly includes: (41) Perform Euclidean clustering on the point cloud data after normal vector filtering, based on the maximum number of tire assemblies n in each layer. max Segment the point clouds of multiple tire assemblies to facilitate subsequent calculation of their poses one tire at a time; (42) For the point cloud data of each tire assembly, the boundary point cloud is calculated by using an angle clearance algorithm, and the boundary point cloud of each tire assembly is extracted one by one; the angle clearance algorithm includes: Traverse the tire assembly point cloud. Let the current point be p. Using a binary tree, find its n nearest neighbors, or all points within a radius r. Using the current point as the origin, map the n neighboring points onto a plane perpendicular to the normal vector of point p, and construct a base coordinate system. At this point, the coordinates of each neighboring point are... Will Sort from largest to smallest As the corner gap between adjacent points, determine the maximum corner gap. When θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When point p is a non-boundary point; (43) After obtaining n max After obtaining the boundary point cloud of each target workpiece, the RANSAC fitting three-dimensional circle algorithm is used to calculate the center and axis direction of the three-dimensional circle, which is the spatial pose of the tire assembly.

2. The spatial pose recognition method for destacking tire assembly industrial robots according to claim 1, characterized in that, Step one also includes the following steps: The configuration file records multiple viewpoint positions for each tire layer, and the viewpoints of the current layer are traversed to collect complete point cloud data for the current layer.

3. The spatial pose recognition method for destacking tire assembly industrial robots according to claim 1, characterized in that, Step two includes the following steps: (21) Traverse the n end flange poses of the robot and record the n poses, measure the calibration plate image data and perform grayscale processing; (22) Perform Gaussian blurring and binarization on the grayscale image of the calibration plate; (23) Calculate the robot hand-eye relationship based on the binarized calibration plate image and the camera intrinsic parameter matrix and distortion coefficients to obtain the relationship between the camera coordinate system and the robot end flange coordinate system.

4. The spatial pose recognition method for destacking tire assembly industrial robots according to claim 1, characterized in that, Step three, the preprocessing of the point cloud data followed by normal vector filtering includes the following steps: (31) Based on the robot hand-eye relationship The multiple point cloud images obtained in step one are unified under the robot's base coordinate system to obtain complete point cloud data; (32) Based on the XYZ coordinate range of the spatial location of the rack, perform pass-through filtering on the initial point cloud data to filter the point cloud data of the ground and the rack frame. (33) Perform voxel filtering sampling on the point cloud data after the pass-through filtering process, construct a point cloud voxel map, calculate the mean of the point cloud in each voxel, and replace all the point clouds in the voxel with the mean. (34) After voxel filtering sampling, due to the fluctuation and noise interference of the data collected by the camera each time, the voxel-filtered point cloud is processed by Gaussian filtering. (35) Perform normal vector filtering on the point cloud data after Gaussian filtering to determine the circular boundary of the tire assembly.

5. The spatial pose recognition method for destacking tire assembly industrial robots according to claim 4, characterized in that, Step (35) includes the following steps: (351) Perform PCA principal component analysis on the point cloud data after Gaussian filtering to obtain the normal vector of the point cloud data; (352) Since the point cloud coordinate system under the unified coordinate system is the robot's base coordinate system, and the z-axis of the base coordinate system coincides with the z-axis of the material rack coordinate system, in the normal vector filtering process, the angle between the normal vector of point p and the z-axis is used as the filtering object. When the angle is greater than the threshold θ threshold Remove the point when the included angle is less than the threshold θ. threshold If the point is retained, the circular boundary of the tire assembly is obtained after the normal vector filtering process.

6. The spatial pose recognition method for destacking tire assembly industrial robots according to claim 5, characterized in that, The PCA principal component analysis includes: Given a point p, find its n nearest neighbors or neighbors within a radius r using a binary tree. Calculate the covariance matrix of these n+1 points, and then calculate their eigenvalues ​​and eigenvectors. The eigenvector corresponding to the smallest eigenvalue is the normal vector of point p.

7. The spatial pose recognition method for destacking tire assembly industrial robots according to claim 1, characterized in that, In step (43), the RANSAC three-dimensional circle fitting algorithm process is as follows: (431) Defining a mathematical model of a three-dimensional circle requires the spatial coordinates of at least three points; (432) In a single loop, n points are randomly selected for the boundary point cloud, n≥3, and the three-dimensional circle model and model parameters are calculated using the least squares method; (433) Traverse the boundary point cloud, with the current point being p. When the distance between p and the 3D circle model is less than the threshold l threshold Let point p be an interior point. When the distance between p and the 3D circle model is greater than or equal to a threshold l... threshold Let point p be an outer point, and count the number of inner points; (434) Repeat steps (432) and (433). When the number of points in the (k+1)th cycle is greater than the number of points in the (k)th cycle, the optimal 3D circle model parameters are updated to the result of the (k+1)th cycle. (435) Set a loop termination condition. When the number of loops reaches the loop termination condition, after the loop ends, take the three-dimensional circle model with the largest number of interior points as the final solution model.

8. The spatial pose recognition method for destacking tire assembly industrial robots according to claim 1, characterized in that, Step five includes the following steps: (51) Under the robot base coordinate system, based on the tire assembly rack position and tire assembly distribution pattern, the (x, y) coordinates of the tire assembly are statistically analyzed. The average value of the (x, y) coordinates of the tire assembly is used as its position estimate. Meanwhile, the tire assembly depth value z is distributed in a discrete equidistant interval. Thus, the above (x, y) coordinates and z coordinates are compiled into a configuration file to obtain the (x, y, z) coordinates of the n tire assemblies in the i-th single layer. (52) Based on the i-th layer, the z-axis depth estimate of its configuration file is known. Combined with the estimated coordinate values ​​(x, y) in the configuration file, n estimated tire assembly coordinate values ​​(x, y, z) are obtained. The calculated values ​​are traversed to find the shortest distance between them and the estimated values. The sum of the shortest distances of the n points is counted, which is the sum of the errors between the calculated values ​​and the estimated values. When the sum of the errors is greater than or equal to the preset threshold, the current calculation has an error and returns to step three. When the sum of the errors is less than the preset threshold, the current calculation passes.

9. A spatial pose recognition system for depalletizing tire assembly industrial robots, characterized in that, include: The first main control module is used to collect point cloud data of the material rack tire assembly from viewpoint to viewpoint according to the configuration file. The second main control module is used to measure calibration board image data in n poses of the robot's end effector to calibrate the relationship between the point cloud camera coordinate system and the robot's end effector flange coordinate system. n≥3; The third main control module is used to determine the relationship. After preprocessing the point cloud data, normal vector filtering is performed to determine the circular boundary of the tire assembly. The fourth main control module is used to obtain the boundary point cloud of the tire assembly through boundary detection, and to fit a three-dimensional circle based on the boundary point cloud using the RANSAC algorithm to obtain the spatial pose of the tire assembly. In the fourth main control module, the step of fitting a three-dimensional circle using the RANSAC algorithm based on the boundary point cloud to obtain the spatial pose of the tire assembly includes: (41) Perform Euclidean clustering on the point cloud data after normal vector filtering, based on the maximum number of tire assemblies n in each layer. max Segment the point clouds of multiple tire assemblies to facilitate subsequent calculation of their poses one tire at a time; (42) For the point cloud data of each tire assembly, the boundary point cloud is calculated by using an angle clearance algorithm, and the boundary point cloud of each tire assembly is extracted one by one; the angle clearance algorithm includes: Traverse the tire assembly point cloud. Let the current point be p. Using a binary tree, find its n nearest neighbors, or all points within a radius r. Using the current point as the origin, map the n neighboring points onto a plane perpendicular to the normal vector of point p, and construct a base coordinate system. At this point, the coordinates of each neighboring point are... Will Sort from largest to smallest As the corner gap between adjacent points, determine the maximum corner gap. When θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When point p is a non-boundary point; (43) After obtaining n max After obtaining the boundary point cloud of each target workpiece, the RANSAC fitting three-dimensional circle algorithm is used to calculate the center and axis direction of the three-dimensional circle, which is the spatial pose of the tire assembly. The fifth main control module is used to calculate the estimated pose of the tire assembly based on the position of the tire assembly rack and the distribution pattern of the tire assembly. It determines whether the spatial pose is correct based on the error between the estimated pose and the spatial pose. If it is correct, it outputs the result; otherwise, it triggers the third main control module to re-run the calculation.

Images