Spatial pose recognition method and system for disorderly stacked tire skins

Abstract

The invention belongs to the field of tire skin space pose detection, and particularly discloses a space pose recognition method and system for disorderly stacked tire skins. Comprising the following steps: obtaining an ROI of a target workpiece in a depth map through a convolutional neural network; converting the depth map into original point cloud data of the target workpiece according to the internal reference matrix of the depth camera and the ROI of the target workpiece in the depth map; preprocessing the original point cloud data, calculating a boundary point cloud of the original point cloud data, and fitting a three-dimensional circle by adopting an RANSAC algorithm based on the boundary point cloud to obtain a space pose of the tire skin; and calculating the proportion of the number of the point clouds in the three-dimensional circle in the boundary point clouds, and judging whether the spatial pose is accurate or not according to the proportion. According to the method, the limitation in traditional image detection and point cloud processing is effectively solved, higher adaptability and robustness are achieved, more accurate data support can be provided in a complex production environment, and the automation level and production efficiency of tire assembly are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of tire skin spatial posture detection, and more specifically, relates to a method and system for spatial posture recognition of disorderly stacked tire skins. Background Art

[0002] As the only part of a car that contacts the ground, tires play a vital role. They are not only related to the traction, stability and comfort of the car, but also directly affect driving safety and fuel efficiency. Therefore, the quality and production efficiency of tires are of great significance to the overall development of the automotive industry. With the continuous development of the automotive industry, the demand for tires continues to grow, and the tire industry has become one of the largest industries in the world. Especially in the production process, the assembly efficiency of tires has a direct impact on improving overall productivity and reducing production costs.

[0003] In the traditional tire production process, the handling and assembly of tire skins mostly rely on manual operations. However, this method not only leads to a significant increase in production costs, but also has low handling efficiency. At the same time, long-term manual operation also increases the risk of worker injury, affecting the improvement of production safety and working environment. Although traditional image detection technology can obtain tire-related information to a certain extent, it cannot effectively reflect the true state of the target object, and is often limited by the resolution of the equipment and environmental interference, resulting in inaccurate data obtained. Although traditional point cloud processing technology can provide certain support for object positioning, due to noise interference, the accuracy is often difficult to meet the actual application requirements, especially in complex production environments, where positioning accuracy poses great challenges.

[0004] Based on the above defects and shortcomings, there is an urgent need in the art to propose a spatial posture recognition method for disorderly stacked tire skins, so as to more accurately and efficiently complete the positioning of the tire skins in a complex production environment. Summary of the invention

[0005] In view of the above defects or improvement needs of the prior art, the present invention provides a spatial posture recognition method and system for disorderly stacked tire skins, which combines the advantages of image data and point cloud data, accurately captures the appearance characteristics of the tire through advanced image processing technology, and then accurately analyzes the three-dimensional spatial information of the tire in combination with point cloud data, thereby achieving more accurate and efficient tire skin spatial posture detection. This method effectively solves the limitations of traditional image detection and point cloud processing, has stronger adaptability and robustness, can provide more accurate data support in complex production environments, significantly improves the automation level and production efficiency of tire assembly, reduces labor costs, and reduces safety hazards caused by improper operation.

[0006] To achieve the above object, according to one aspect of the present invention, a spatial posture recognition method for disorderly stacked tire skins is proposed, comprising the following steps:

[0007] Step 1: Obtain the ROI where the target workpiece is located in the depth map through a convolutional neural network;

[0008] Step 2: convert the depth map into original point cloud data of the target workpiece according to the depth camera intrinsic parameter matrix and the ROI where the target workpiece is located in the depth map;

[0009] Step 3, preprocessing the original point cloud data to calculate its boundary point cloud, and fitting a three-dimensional circle based on the boundary point cloud using a RANSAC algorithm to obtain the spatial position of the tire skin;

[0010] Step 4, calculate the proportion of the number of point clouds within the three-dimensional circle in the boundary point cloud, and judge whether the spatial posture is accurate based on the proportion. If so, output the spatial posture of the tire skin. If not, return to step 1 and recalculate the ROI of the target workpiece in the depth map.

[0011] As further preferred, step one comprises the following steps:

[0012] (11) Collecting the depth map data of disorderly stacked tire skins under actual working conditions, marking the ROI of the tire skin that can be grabbed at the top of the depth map, constructing its disorderly stacked tire skin depth map dataset, and using the dataset to train a convolutional neural network model;

[0013] (12) The depth map of the target workpiece is input into the trained convolutional neural network model to obtain the ROI where the target workpiece is located in the depth map.

[0014] As further preferred, step 2 comprises the following steps:

[0015] (21) creating a mask according to the ROI of the target workpiece in the depth map, filtering the depth map according to the mask, and setting the area outside the ROI to 0 during the filtering process;

[0016] (22) The filtered depth map data is converted into point cloud data according to the intrinsic parameter matrix coefficients of the depth camera.

[0017] As further preferred, step (22) comprises the following steps:

[0018] (221) Calibrate the internal parameter matrix coefficients of the depth camera, the internal parameter matrix coefficients include: x-axis focal length f x , y-axis focal length f y , x-axis pixel offset c x , y-axis pixel offset c y ;

[0019] (222) Loop through the depth map pixel points to obtain their pixel coordinates (u, v) and the grayscale value at the pixel point, that is, the depth value at the point;

[0020] (223) If the depth value of the point is set to 0, skip it; otherwise, calculate the point cloud coordinates corresponding to the pixel point and obtain the original point cloud data of the target workpiece. The calculation formula of the point cloud coordinates corresponding to the pixel point includes:

[0021]

[0022] Among them, (X, Y, Z) is the point cloud coordinate corresponding to the pixel point.

[0023] As further preferred, step three comprises the following steps:

[0024] (31) Perform uniform filtering on the original point cloud data to reduce the number of point clouds without losing its geometric information;

[0025] (32) Performing Euclidean clustering on the point cloud data after uniform filtering to extract the main target workpiece point cloud;

[0026] (33) performing a principal component analysis (PCA) on the main target workpiece point cloud to obtain a normal vector of the target workpiece point cloud;

[0027] (34) calculating a boundary point cloud of the target workpiece by using an angular clearance algorithm based on the target workpiece point cloud and the normal vector of the target workpiece point cloud;

[0028] (35) Perform RANSAC fitting of the three-dimensional circle on the boundary point cloud of the target workpiece to obtain the spatial position of the tire skin.

[0029] As further preferred, in step (33), the PCA principal component analysis includes:

[0030] For any point p, use the binary tree to find its n neighboring points or the neighboring points within a radius of r, and use these n+1 points to calculate its covariance matrix, thereby calculating its eigenvalues ​​and eigenvectors. The eigenvector corresponding to the minimum eigenvalue is the normal vector of point p.

[0031] As further preferred, in step (34), calculating the boundary point cloud of the target workpiece based on the angular clearance algorithm includes:

[0032] Traverse the target workpiece point cloud, assume that the current point is p, and find n neighboring points in its domain or all points in the domain with a radius of r as its neighboring points according to the binary tree. Take the current point as the coordinate origin, map the n domain points to the plane perpendicular to the normal vector of point p, and construct the base coordinate system. At this time, the coordinates of each neighboring point are n=1…n, Sort from largest to smallest, As the angular gap between adjacent points, determine the maximum angular gap When θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When , point p is a non-boundary point.

[0033] As further preferred, in step (35), the step of fitting a three-dimensional circle using the RANSAC algorithm includes:

[0034] (351) In a single cycle, n points are randomly selected from the boundary point cloud, where n ≥ 3, and the three-dimensional circle mathematical model is calculated using the least squares method;

[0035] (352) Traverse the boundary point cloud, assuming that the current point is p, when the distance between p and the three-dimensional circle model is less than the threshold l threshold , point p is recorded as an inner point, when the distance between p and the three-dimensional circle model is greater than or equal to the threshold l threshold , record point p as an external point, and count the number of internal points;

[0036] (353) Repeat steps (351) and (352), and when the number of internal points in the k+1th cycle is greater than the number of internal points in the kth cycle, the optimal three-dimensional circle model parameters are updated to the results of the k+1th cycle;

[0037] (354) Set the loop termination condition. When the number of loops reaches the loop termination condition, after the loop ends, take the three-dimensional circle model with the largest number of internal points as the final solution model.

[0038] As a further preferred embodiment, step 2 further includes: capturing depth maps of disordered stacked tire skins from different angles by multiple depth cameras, and fusing the depth maps of disordered stacked tire skins captured at different angles by using a perspective fusion weight calculation formula, wherein the perspective fusion weight calculation formula includes:

[0039]

[0040] Where Wi is the weight of the i-th view, Di is the distance from the i-th view to the target tire skin, Dj represents the distance from the j-th view to the target tire skin, σ is the standard deviation, which is used to adjust the sensitivity of weight distribution, and n is the total number of view angles;

[0041] The fused depth map is:

[0042]

[0043] Where Dfused(u,v) is the depth value of the fused depth map at the pixel coordinate (u,v), Wi is the weight of the i-th perspective, and Di,depth(u,v) is the depth value of the depth map of the i-th perspective at the pixel coordinate (u,v).

[0044] According to another aspect of the present invention, there is also provided a spatial posture recognition system for disorderly stacked tire skins, comprising:

[0045] The first main control module is used to obtain the ROI of the target workpiece in the depth map through a convolutional neural network;

[0046] The second main control module is used to convert the depth map into original point cloud data of the target workpiece according to the depth camera internal parameter matrix and the ROI where the target workpiece is located in the depth map;

[0047] The third main control module is used to calculate the boundary point cloud after preprocessing the original point cloud data, and fit a three-dimensional circle based on the boundary point cloud using the RANSAC algorithm to obtain the spatial posture of the tire skin;

[0048] The fourth main control module is used to calculate the proportion of the number of point clouds within the three-dimensional circle in the boundary point cloud, and judge whether the spatial posture is accurate based on the proportion. If so, the spatial posture of the tire skin is output; if not, the first main control module is triggered to recalculate the ROI of the target workpiece in the depth map.

[0049] In general, the above technical solution conceived by the present invention has the following technical advantages compared with the prior art:

[0050] 1. The present invention utilizes two data types, depth map and point cloud, as well as convolutional neural network and traditional point cloud algorithm to solve the problem of accurate spatial posture detection of disordered stacked tire skins. The approximate range of the target workpiece is obtained through the depth map and convolutional neural network, and the separate point cloud data of the target workpiece is obtained through the point cloud preprocessing algorithm. The RANSAC three-dimensional circle fitting and angular clearance boundary calculation method are used to finally obtain the accurate posture of the target workpiece. This example can be used for the loading process of disordered stacked tire skins in complex situations, which greatly improves the automation level of the process, effectively saves its labor costs and risks, and can effectively solve the many shortcomings of traditional manual handling methods. It has great application prospects in the future.

[0051] 2. The present invention uses a convolutional neural network to quickly and accurately identify the ROI of the target workpiece in the depth map, then converts the depth map into point cloud data, and uses the RANSAC algorithm to fit a three-dimensional circle to determine the spatial position of the tire skin. This streamlined recognition method reduces human errors and improves recognition efficiency and accuracy.

[0052] 3. The present invention effectively handles the noise and outliers in the original point cloud data by preprocessing the point cloud data, PCA principal component analysis, and calculating the boundary point cloud based on the angular clearance algorithm. In addition, the use of the RANSAC algorithm further improves the robustness of the system to abnormal data, ensuring that the spatial pose can be accurately identified even when some data is damaged or incomplete.

[0053] 4. The present invention determines the accuracy of spatial posture by calculating the proportion of the number of point clouds within the three-dimensional circle in the boundary point cloud, and makes self-adjustments based on this proportion. This self-optimization mechanism enables the system to continuously learn and improve in actual operation, improving the intelligence level and long-term stability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of a method for spatial posture recognition of disordered stacked tire skins according to an embodiment of the present invention;

[0055] Figure 2 A convolutional neural network inputting an unordered tire skin depth map image according to an embodiment of the present invention;

[0056] Figure 3 is a convolutional neural network depth graph reasoning result image according to an embodiment of the present invention;

[0057] Figure 4 is a flowchart of point cloud raw data preprocessing according to an embodiment of the present invention;

[0058] Figure 5 It is a depth map ROI-to-point cloud raw data image according to an embodiment of the present invention;

[0059] Figure 6 is a data image after uniform filtering of a point cloud according to an embodiment of the present invention;

[0060] Figure 7 is a data image after Euclidean clustering of the point cloud according to an embodiment of the present invention;

[0061] Figure 8 is a target workpiece point cloud normal vector calculation result image according to an embodiment of the present invention;

[0062] Fig. 9 is a flow chart of a method for calculating a target workpiece posture according to an embodiment of the present invention;

[0063] Fig.10 is a cloud computing result image of a target workpiece boundary point according to an embodiment of the present invention;

[0064] Fig.11It is a target workpiece spatial posture detection result image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0066] like Figure 1 As shown, a spatial posture recognition method for disorderly stacked tire skins provided by an embodiment of the present invention includes the following steps:

[0067] Step 1: Use a convolutional neural network to obtain the ROI where the target workpiece is located in the depth map.

[0068] In one embodiment, step one includes the following steps:

[0069] (11) Collecting the depth map data of disorderly stacked tire skins under actual working conditions, marking the ROI of the tire skin that can be grabbed at the top of the depth map, constructing its disorderly stacked tire skin depth map dataset, and using the dataset to train a convolutional neural network model;

[0070] (12) The depth map of the target workpiece is input into the trained convolutional neural network model to obtain the ROI where the target workpiece is located in the depth map.

[0071] In one embodiment, the depth map is obtained by:

[0072] The depth map of the disordered stacked tire skin is captured from different angles by multiple depth cameras, and the depth map of the disordered stacked tire skin captured from different angles is fused using a perspective fusion weight calculation formula, and the perspective fusion weight calculation formula includes:

[0073]

[0074] Where Wi is the weight of the i-th view, Di is the distance from the i-th view to the target tire skin, Dj represents the distance from the j-th view to the target tire skin, σ is the standard deviation, which is used to adjust the sensitivity of weight distribution, and n is the total number of view angles;

[0075] The fused depth map is:

[0076]

[0077] Where Dfused(u,v) is the depth value of the fused depth map at the pixel coordinate (u,v), Wi is the weight of the i-th perspective, and Di,depth(u,v) is the depth value of the depth map of the i-th perspective at the pixel coordinate (u,v).

[0078] Step 2: Convert the depth map into original point cloud data of the target workpiece according to the depth camera intrinsic parameter matrix and the ROI where the target workpiece is located in the depth map.

[0079] In one embodiment, step 2 includes the following steps:

[0080] (21) creating a mask according to the ROI of the target workpiece in the depth map, filtering the depth map according to the mask, and setting the area outside the ROI to 0 during the filtering process;

[0081] (22) The filtered depth map data is converted into point cloud data according to the intrinsic parameter matrix coefficients of the depth camera.

[0082] Specifically, in one embodiment, step (22) includes the following steps:

[0083] (221) Calibrate the internal parameter matrix coefficients of the depth camera, the internal parameter matrix coefficients include: x-axis focal length f x , y-axis focal length f y , x-axis pixel offset c x , y-axis pixel offset c y ;

[0084] (222) Loop through the depth map pixel points to obtain their pixel coordinates (u, v) and the grayscale value at the pixel point, that is, the depth value at the point;

[0085] (223) If the depth value of the point is set to 0, skip it; otherwise, calculate the point cloud coordinates corresponding to the pixel point and obtain the original point cloud data of the target workpiece. The calculation formula of the point cloud coordinates corresponding to the pixel point includes:

[0086]

[0087] Among them, (X, Y, Z) is the point cloud coordinate corresponding to the pixel point.

[0088] Step three, preprocessing the original point cloud data to calculate its boundary point cloud, and fitting a three-dimensional circle based on the boundary point cloud using the RANSAC algorithm to obtain the spatial position of the tire skin.

[0089] In one embodiment, step three includes the following steps:

[0090] (31) Perform uniform filtering on the original point cloud data to reduce the number of point clouds without losing its geometric information;

[0091] (32) Performing Euclidean clustering on the point cloud data after uniform filtering to extract the main target workpiece point cloud;

[0092] (33) performing a principal component analysis (PCA) on the main target workpiece point cloud to obtain a normal vector of the target workpiece point cloud;

[0093] (34) calculating a boundary point cloud of the target workpiece by using an angular clearance algorithm based on the target workpiece point cloud and the normal vector of the target workpiece point cloud;

[0094] (35) Perform RANSAC fitting of the three-dimensional circle on the boundary point cloud of the target workpiece to obtain the spatial position of the tire skin.

[0095] Specifically, in one embodiment, in step (33), the PCA principal component analysis includes:

[0096] For any point p, use the binary tree to find its n neighboring points or the neighboring points within a radius of r, and use these n+1 points to calculate its covariance matrix, thereby calculating its eigenvalues ​​and eigenvectors. The eigenvector corresponding to the minimum eigenvalue is the normal vector of point p.

[0097] Specifically, in one embodiment, in step (34), calculating the boundary point cloud of the target workpiece based on the angular clearance algorithm includes:

[0098] Traverse the target workpiece point cloud, assume that the current point is p, and find n neighboring points in its domain or all points in the domain with a radius of r as its neighboring points according to the binary tree. Take the current point as the coordinate origin, map the n domain points to the plane perpendicular to the normal vector of point p, and construct the base coordinate system. At this time, the coordinates of each neighboring point are n=1…n, Sort from largest to smallest, As the angular gap between adjacent points, determine the maximum angular gap When θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When , point p is a non-boundary point.

[0099] Specifically, first, traverse the target workpiece point cloud, assume that the current point is p, and find its n neighboring points in the domain or all points in the domain with a radius of r as its neighboring points according to the binary tree. Take the current point as the coordinate origin, map the n domain points to the plane perpendicular to the normal vector of point p, and construct a polar coordinate system. Secondly, in the polar coordinate system of point p, the coordinates of each domain point are n=1…n, Sort from largest to smallest, As the angular gap between adjacent points, determine the maximum angular gap Finally, the algorithm presets the threshold value as θ threshold , when θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When , point p is a non-boundary point.

[0100] In one embodiment, in step (35), fitting a three-dimensional circle using a RANSAC algorithm comprises:

[0101] (351) In a single cycle, n points are randomly selected from the boundary point cloud, where n ≥ 3, and the three-dimensional circle mathematical model is calculated using the least squares method;

[0102] (352) Traverse the boundary point cloud, assuming that the current point is p, when the distance between p and the three-dimensional circle model is less than the threshold l threshold , point p is recorded as an inner point, when the distance between p and the three-dimensional circle model is greater than or equal to the threshold l threshold , record point p as an external point, and count the number of internal points;

[0103] (353) Repeat steps (351) and (352), and when the number of internal points in the k+1th cycle is greater than the number of internal points in the kth cycle, the optimal three-dimensional circle model parameters are updated to the results of the k+1th cycle;

[0104] (354) Set the loop termination condition. When the number of loops reaches the loop termination condition, after the loop ends, take the three-dimensional circle model with the largest number of internal points as the final solution model.

[0105] Step 4, calculate the proportion of the number of point clouds within the three-dimensional circle in the boundary point cloud, and judge whether the spatial posture is accurate based on the proportion. If so, output the spatial posture of the tire skin. If not, return to step 1 and recalculate the ROI of the target workpiece in the depth map.

[0106] In a specific embodiment, a spatial posture recognition method for disorderly stacked tire skins includes:

[0107] Step 1: Construct a depth image dataset for disordered stacked tire skins. For disordered stacked tire skins in complex scenes, the original depth image and point cloud data are coupled together. How to obtain the original data of a single target workpiece is the primary task of calculating the precise spatial position of the target workpiece. Traditional convolutional neural networks are mainly used for two-dimensional images, while this example uses a depth map as the input layer of the convolutional neural network. Figure 2 As shown, the ROI of the target workpiece located in the depth map is obtained through a convolutional neural network.

[0108] (11): Collect the depth map data of disordered stacked tire skins under actual working conditions, mark the ROI of the tire skin that can be grabbed at the top of the depth map, construct its disordered stacked tire skin depth map dataset, and train the convolutional neural network model based on the dataset.

[0109] (12): Based on the convolutional neural network model trained in step (11), the depth map is input, and the reasoning process is performed to obtain the ROI of the target tire skin that can be grasped, such as Figure 3 shown.

[0110] Step 2: For the target workpiece information in the depth map, it is difficult to efficiently and accurately estimate the exact pose of the target workpiece on the two-dimensional image, so the subsequent steps will convert it into point cloud data for subsequent precise pose calculation. The subsequent steps are to calibrate the depth camera, obtain its intrinsic parameter matrix coefficients, use the mask to obtain the depth map target workpiece data, and convert the depth map data into point cloud data based on the depth camera intrinsic parameter matrix.

[0111] (21): Calibrate its depth camera and obtain its internal parameter matrix coefficients, including the x-axis focal length f x , y-axis focal length f y , x-axis pixel offset c x , y-axis pixel offset c y .

[0112] (22): Create a mask using the target workpiece ROI obtained by the convolutional neural network in S101, filter its depth map, and set the area outside the ROI to 0 to extract the point cloud data of the target workpiece and accelerate the subsequent depth map to point cloud conversion process.

[0113] (23): The filtered depth map data is converted into point cloud data by using the camera intrinsic parameter matrix coefficients obtained by calibration in step (21). The specific conversion process is as follows:

[0114] Loop through the depth map pixels to get their pixel coordinates (u, v) and the grayscale value at the pixel, that is, the depth value at the point.

[0115] If the point is set to 0, skip it. Otherwise, calculate the point cloud coordinates (x, y, z) corresponding to the pixel point according to the following formula:

[0116]

[0117] Finally, the original point cloud data of the target workpiece is obtained.

[0118] Step 3: After obtaining the original point cloud data of the target workpiece, preprocess the original point cloud, calculate its boundary point cloud after preprocessing, and then fit the three-dimensional circle through the RANSAC algorithm to obtain the accurate spatial pose result of the tire skin. After obtaining the original workpiece point cloud, it is difficult to directly calculate its accurate pose through a certain algorithm, so a data preprocessing algorithm is needed to realize the feasibility of pose calculation and accelerate the subsequent calculation process, such as Figure 4 shown.

[0119] (31): The density of the original point cloud is uncertain, such as Figure 5 As shown in , the point cloud algorithm is positively correlated with the number of point clouds, so uniform filtering is performed on the original point cloud to reduce the number of point clouds without losing its geometric information, such as Figure 6 shown.

[0120] (32): For the tire skin point cloud after uniform filtering, due to the inaccuracy of the ROI obtained by the convolutional neural network, some environment and other workpiece point clouds are still left. Through Euclidean clustering, the main target workpiece point cloud is extracted, such as Figure 7 shown.

[0121] (33): For the target workpiece point cloud obtained in the above steps, the normal vector of the target workpiece point cloud is calculated by PCA principal component analysis, as follows: Figure 8 shown.

[0122] PCA principal component analysis determines a point p, finds its n neighboring points or neighboring points within a range with a radius of r through a binary tree, and calculates its covariance matrix with these n+1 points, thereby calculating its eigenvalues ​​and eigenvectors. The eigenvector corresponding to the minimum eigenvalue is the normal vector of point p.

[0123] After preprocessing the point cloud data, the target workpiece point cloud and its normal vector are obtained. The method for calculating its precise spatial pose includes: Fig. 9 As shown:

[0124] (34): For the target workpiece point cloud, its boundary point cloud is calculated based on the angular clearance algorithm, and the target workpiece boundary point cloud is extracted, such as Fig.10 shown.

[0125] The angular clearance algorithm is to traverse the point cloud, assuming that the current point is p, and find n neighboring points in its area or all points in the area with a radius of r as its neighboring points according to the binary tree. With the current point as the coordinate origin, the n area points are mapped to the plane perpendicular to the normal vector of point p, and the base coordinate system is constructed. At this time, the coordinates of each neighboring point are n=1…n, Sort from largest to smallest, As the angular gap between adjacent points, determine the maximum angular gap When θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When , point p is a non-boundary point.

[0126] (35): After obtaining the boundary point cloud of the target workpiece, the RANSAC three-dimensional circle fitting algorithm is used to calculate the center and axis direction of the three-dimensional circle, that is, the position and posture result of the target workpiece, such as Fig.11 shown.

[0127] The RANSAC three-dimensional circle fitting algorithm process is as follows:

[0128] Defining a mathematical model of a three-dimensional circle requires the spatial coordinates of at least three points.

[0129] In a single cycle, n (n≥3) points are randomly selected from the boundary point cloud, and the three-dimensional circle mathematical model is calculated using the least squares method.

[0130] Traverse the boundary point cloud, assuming that the current point is p, when the distance between p and the 3D circle model is less than the threshold l threshold , point p is recorded as an inner point, when the distance between p and the three-dimensional circle model is greater than or equal to the threshold l threshold , denote point p as an external point, and count the number of internal points.

[0131] Traversing the above loop, when the number of internal points in the k+1th loop is greater than the number of internal points in the kth loop, the optimal three-dimensional circle model parameters are updated to the results of the k+1th loop.

[0132] The loop termination condition is that the number of loops reaches the preset value. After the loop ends, the three-dimensional circle model with the largest number of internal points is taken as the final solution model.

[0133] Step 4: After the above calculation process, the precise spatial pose of the target workpiece is obtained. At this time, a method for judging the correctness of the calculation results is still needed. By detecting the calculation correctness of the above algorithm, the interference and collision between the fixture and the environment or the target workpiece can be avoided, thereby enhancing the stability of the pose recognition algorithm.

[0134] The visualization result of the boundary point cloud obtained is the inner circle of the tire skin and its outer irregular edge, and the RANSAC three-dimensional circle fitting algorithm can obtain the number of points inside the three-dimensional circle. By calculating the proportion of the number of points inside the three-dimensional circle in the boundary point cloud, it can be determined whether the calculated three-dimensional circle is the ideal inner edge of the tire skin.

[0135] The specific process is as follows:

[0136] Get the number of interior points and boundary point clouds in the above RANSAC 3D circle fitting algorithm.

[0137] Calculate the proportion of points inside the 3D circle in the boundary point cloud.

[0138] The obtained proportion is compared with the preset threshold. If it is greater than the preset threshold, the calculation result is determined to be correct, otherwise the calculation result is determined to be incorrect.

[0139] So far, this example has solved the problem of accurate spatial pose detection of disordered stacked tire skins by using two data types, depth map and point cloud, as well as convolutional neural network and traditional point cloud algorithm. The approximate range of the target workpiece is obtained through depth map and convolutional neural network, and the separate point cloud data of the target workpiece is obtained through point cloud preprocessing algorithm. Finally, the accurate pose of the target workpiece is obtained by using RANSAC three-dimensional circle fitting and angular clearance boundary calculation method. This example can be used for the loading process of disordered stacked tire skins in complex situations, which greatly improves the automation level of this process, effectively saves its labor cost and risk, and can effectively solve the many shortcomings of traditional manual handling methods. It has great application prospects in the future.

[0140] According to another aspect of the present invention, a system for implementing the method of any of the above embodiments or a combination of multiple embodiments is also provided, including:

[0141] The first main control module is used to obtain the ROI of the target workpiece in the depth map through a convolutional neural network;

[0142] The second main control module is used to convert the depth map into original point cloud data of the target workpiece according to the depth camera internal parameter matrix and the ROI where the target workpiece is located in the depth map;

[0143] The third main control module is used to calculate the boundary point cloud after preprocessing the original point cloud data, and fit a three-dimensional circle based on the boundary point cloud using the RANSAC algorithm to obtain the spatial posture of the tire skin;

[0144] The fourth main control module is used to calculate the proportion of the number of point clouds within the three-dimensional circle in the boundary point cloud, and judge whether the spatial posture is accurate based on the proportion. If so, the spatial posture of the tire skin is output; if not, the first main control module is triggered to recalculate the ROI of the target workpiece in the depth map.

[0145] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for spatial posture recognition of disorderly stacked tire skins, characterized in that: The following steps are involved: Step 1: Obtain the ROI where the target workpiece is located in the depth map through a convolutional neural network; Step 2: convert the depth map into original point cloud data of the target workpiece according to the depth camera intrinsic parameter matrix and the ROI where the target workpiece is located in the depth map; Step 3, preprocessing the original point cloud data to calculate its boundary point cloud, and fitting a three-dimensional circle based on the boundary point cloud using a RANSAC algorithm to obtain the spatial position of the tire skin; Step 4, calculate the proportion of the number of point clouds within the three-dimensional circle in the boundary point cloud, and judge whether the spatial posture is accurate based on the proportion. If so, output the spatial posture of the tire skin. If not, return to step 1 and recalculate the ROI of the target workpiece in the depth map.

2. A method for spatial posture recognition of disorderly stacked tire skins according to claim 1, characterized in that: Step one includes the following steps: (11) Collecting the depth map data of disorderly stacked tire skins under actual working conditions, marking the ROI of the tire skin that can be grabbed at the top of the depth map, constructing its disorderly stacked tire skin depth map dataset, and using the dataset to train a convolutional neural network model; (12) The depth map of the target workpiece is input into the trained convolutional neural network model to obtain the ROI where the target workpiece is located in the depth map.

3. A method for spatial posture recognition of disorderly stacked tire skins according to claim 1, characterized in that: Step 2 includes the following steps: (21) creating a mask according to the ROI of the target workpiece in the depth map, filtering the depth map according to the mask, and during the filtering process, setting the area outside the ROI of the depth map to 0 through the mask; (22) The filtered depth map data is converted into point cloud data according to the intrinsic parameter matrix coefficients of the depth camera.

4. A method for spatial posture recognition of disorderly stacked tire skins according to claim 3, characterized in that: Step (22) comprises the following steps: (221) Calibrate the internal parameter matrix coefficients of the depth camera, the internal parameter matrix coefficients include: x-axis focal length f x , y-axis focal length f y , x-axis pixel offset c x , y-axis pixel offset c y ; (222) Loop through the depth map pixel points to obtain their pixel coordinates (u, v) and the grayscale value at the pixel point, that is, the depth value at the pixel point; (223) If the depth value of the pixel point is set to 0, skip it; otherwise, calculate the point cloud coordinates corresponding to the pixel point; after the loop is completed, the original point cloud data of the target workpiece is obtained. The calculation formula of the point cloud coordinates corresponding to the pixel point includes: Among them, (X, Y, Z) is the point cloud coordinate corresponding to the pixel point.

5. The spatial posture recognition method for disorderly stacked tire skins according to claim 3, characterized in that: Step three includes the following steps: (31) Perform uniform filtering on the original point cloud data to reduce the number of point clouds without losing its geometric information; (32) Performing Euclidean clustering on the point cloud data after uniform filtering to extract the main target workpiece point cloud; (33) performing a principal component analysis (PCA) on the main target workpiece point cloud to obtain a normal vector of the target workpiece point cloud; (34) calculating the boundary point cloud of the target workpiece, including the inner circular boundary and the outer irregular boundary of the tire skin, based on the target workpiece point cloud and the normal vector of the target workpiece point cloud by using an angular clearance algorithm; (35) Perform RANSAC fitting of the three-dimensional circle on the boundary point cloud of the target workpiece to obtain the three-dimensional circle model coefficients of the inner circular boundary of the tire skin to obtain the spatial position of the tire skin.

6. A method for spatial posture recognition of disorderly stacked tire skins according to claim 5, characterized in that: In step (33), the PCA principal component analysis includes: For any point p, use the binary tree to find its n neighboring points or the neighboring points within a radius of r, and use these n+1 points to calculate its covariance matrix, thereby calculating its eigenvalues ​​and eigenvectors. The eigenvector corresponding to the minimum eigenvalue is the normal vector of point p.

7. The method for spatial posture recognition of disorderly stacked tire skins according to claim 5, characterized in that: In step (34), the step of calculating the boundary point cloud of the target workpiece based on the angular clearance algorithm includes the following steps: Traverse the target workpiece point cloud, assume that the current point is p, and find n neighboring points in its domain or all points in the domain with a radius of r as its neighboring points according to the binary tree. Take the current point as the coordinate origin, map the n domain points to the plane perpendicular to the normal vector of point p, and construct the base coordinate system. At this time, the coordinates of each neighboring point are n=1…n, Sort from largest to smallest, As the angular gap between adjacent points, determine the maximum angular gap When θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When , point p is a non-boundary point.

8. The spatial posture recognition method for disorderly stacked tire skins according to claim 5, characterized in that: In step (35), the three-dimensional circle fitting using the RANSAC algorithm includes: (351) In a single cycle, n points are randomly selected from the boundary point cloud, where n ≥ 3, and the three-dimensional circle model and model parameters are calculated using the least squares method; (352) Traverse the boundary point cloud, assuming that the current point is p, when the distance between p and the three-dimensional circle model is less than the threshold l threshold , point p is recorded as an inner point, when the distance between p and the three-dimensional circle model is greater than or equal to the threshold l threshold , record point p as an external point, and count the number of internal points; (353) Repeat steps (351) and (352), and when the number of internal points in the k+1th cycle is greater than the number of internal points in the kth cycle, the optimal three-dimensional circle model parameters are updated to the results of the k+1th cycle; (354) Set the loop termination condition. When the number of loops reaches the loop termination condition, after the loop ends, take the three-dimensional circle model with the largest number of internal points as the final solution model.

9. The spatial posture recognition method for disorderly stacked tire skins according to claim 5, characterized in that: Also includes: The depth map of the disordered stacked tire skin is captured from different angles by multiple depth cameras, and the depth map of the disordered stacked tire skin captured from different angles is fused using a perspective fusion weight calculation formula, and the perspective fusion weight calculation formula includes: Where Wi is the weight of the i-th view, Di is the distance from the i-th view to the target tire skin, Dj represents the distance from the j-th view to the target tire skin, σ is the standard deviation, which is used to adjust the sensitivity of weight distribution, and n is the total number of view angles; The fused depth map is: Where Dfused(u,v) is the depth value of the fused depth map at the pixel coordinate (u,v), Wi is the weight of the i-th perspective, and Di,depth(u,v) is the depth value of the depth map of the i-th perspective at the pixel coordinate (u,v).

10. A spatial position recognition system for disorderly stacked tire skins, characterized in that: include: The first main control module is used to obtain the ROI of the target workpiece in the depth map through a convolutional neural network; The second main control module is used to convert the depth map into original point cloud data of the target workpiece according to the depth camera internal parameter matrix and the ROI where the target workpiece is located in the depth map; The third main control module is used to calculate the boundary point cloud after preprocessing the original point cloud data, and fit a three-dimensional circle based on the boundary point cloud using the RANSAC algorithm to obtain the spatial posture of the tire skin; The fourth main control module is used to calculate the proportion of the number of point clouds within the three-dimensional circle in the boundary point cloud, and judge whether the spatial posture is accurate based on the proportion. If so, the spatial posture of the tire skin is output; if not, the first main control module is triggered to recalculate the ROI of the target workpiece in the depth map.

Images