A spatial pose recognition method and system for disordered stacked tire covers

Abstract

This invention belongs to the field of tire skin spatial pose detection, and specifically discloses a method and system for spatial pose recognition of disordered stacked tire skins. It includes: obtaining the Region of Interest (ROI) of the target workpiece in the depth map using a convolutional neural network; converting the depth map into raw point cloud data of the target workpiece based on the depth camera intrinsic parameter matrix and the ROI of the target workpiece in the depth map; preprocessing the raw point cloud data to calculate its boundary point cloud, and fitting a three-dimensional circle based on the boundary point cloud using the RANSAC algorithm to obtain the spatial pose of the tire skin; calculating the proportion of the number of points in the three-dimensional circle in the boundary point cloud, and judging the accuracy of the spatial pose based on this proportion. This invention effectively solves the limitations of traditional image detection and point cloud processing, has stronger adaptability and robustness, can provide more accurate data support in complex production environments, and significantly improves the automation level and production efficiency of tire assembly.

Description

Technical Field

[0001] This invention belongs to the field of tire skin spatial pose detection, and more specifically, relates to a method and system for spatial pose recognition of disordered stacked tire skins. Background Technology

[0002] As the only component of a car in contact with the ground, tires play a crucial role, affecting not only traction, stability, and comfort, but also driving safety and fuel efficiency. Therefore, tire quality and production efficiency are of great significance to the overall development of the automotive industry. With the continuous development of the automotive industry, the demand for tires continues to grow, making the tire industry one of the world's largest industries. Especially in the production process, tire assembly efficiency has a direct impact on improving overall productivity and reducing production costs.

[0003] In traditional tire production, the handling and assembly of tire casings largely rely on manual labor. However, this method not only significantly increases production costs but also results in low handling efficiency. Furthermore, prolonged manual operation increases the risk of worker injury, impacting production safety and the improvement of the working environment. While traditional image detection technology can acquire tire-related information to some extent, it cannot effectively reflect the true state of the target object and is often limited by equipment resolution and environmental interference, leading to inaccurate data. Although traditional point cloud processing technology can provide some support for object localization, noise interference often limits its accuracy to meet practical application requirements, especially in complex production environments where positioning accuracy faces significant challenges.

[0004] Based on the aforementioned defects and shortcomings, there is an urgent need in this field to propose a spatial pose recognition method for disordered stacked tire casings, so as to more accurately and efficiently locate the tire casings in complex production environments. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a method and system for spatial pose recognition of disordered stacked tire skins. It combines the advantages of image data and point cloud data, using advanced image processing technology to accurately capture the tire's appearance features, and then combining this with point cloud data to precisely analyze the tire's three-dimensional spatial information, thereby achieving more accurate and efficient tire skin spatial pose detection. This method effectively overcomes the limitations of traditional image detection and point cloud processing, exhibiting stronger adaptability and robustness. It can provide more accurate data support in complex production environments, significantly improving the automation level and production efficiency of tire assembly, reducing labor costs, and minimizing safety hazards caused by improper operation.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for spatial pose recognition of disordered stacked tire casings is proposed, comprising the following steps:

[0007] Step 1: Obtain the ROI of the target workpiece in the depth map using a convolutional neural network;

[0008] Step 2: Based on the depth camera intrinsic parameter matrix and the ROI of the target workpiece in the depth map, convert the depth map into the raw point cloud data of the target workpiece.

[0009] Step 3: After preprocessing the original point cloud data, calculate its boundary point cloud, and use the RANSAC algorithm to fit a three-dimensional circle based on the boundary point cloud to obtain the spatial pose of the tire skin.

[0010] Step 4: Calculate the proportion of the number of points in the three-dimensional circle to the number of points in the boundary point cloud, and determine whether the spatial pose is accurate based on the proportion. If it is accurate, output the spatial pose of the tire skin. If not, return to Step 1 and recalculate the ROI of the target workpiece in the depth map.

[0011] As a further preferred option, step one includes the following steps:

[0012] (11) Collect depth map data of disordered stacked tire skin under actual working conditions, mark the ROI of the tire skin that can be grabbed at the top of the depth map, construct its disordered stacked tire skin depth map dataset, and use the dataset to train the convolutional neural network model.

[0013] (12) Input the depth map of the target workpiece into the trained convolutional neural network model to obtain the ROI of the target workpiece in the depth map.

[0014] As a further preferred option, step two includes the following steps:

[0015] (21) Create a mask based on the ROI of the target workpiece in the depth map, and filter the depth map based on the mask. During the filtering process, set the area outside the ROI to 0.

[0016] (22) Convert the filtered depth map data into point cloud data based on the intrinsic parameter matrix coefficients of the depth camera.

[0017] As a further preferred option, step (22) includes the following steps:

[0018] (221) The intrinsic parameter matrix coefficients of the depth camera are calibrated. These coefficients include: the focal length f along the x-axis. x y-axis focal length f y x-axis pixel offset c x y-axis pixel offset c y ;

[0019] (222) Traverse the depth map pixels to obtain their pixel coordinates (u, v) and the gray value at the pixel, which is the depth value at that point.

[0020] (223) If the depth value of the point is set to 0, skip this step; otherwise, calculate the point cloud coordinates corresponding to the pixel and obtain the original point cloud data of the target workpiece. The calculation formula for the point cloud coordinates corresponding to the pixel includes:

[0021]

[0022] Where (X, Y, Z) are the point cloud coordinates corresponding to the pixel.

[0023] As a further preferred option, step three includes the following steps:

[0024] (31) Perform uniform filtering on the original point cloud data to reduce the number of point clouds without losing its geometric information;

[0025] (32) Perform Euclidean clustering on the point cloud data after uniform filtering to extract the point cloud of the main target workpiece;

[0026] (33) Perform PCA principal component analysis on the point cloud of the main target workpiece to obtain the normal vector of the point cloud of the target workpiece;

[0027] (34) Based on the target workpiece point cloud and the normal vector of the target workpiece point cloud, the boundary point cloud of the target workpiece is calculated by the angular gap degree algorithm;

[0028] (35) Perform RANSAC fitting of the boundary point cloud of the target workpiece to calculate a three-dimensional circle, so as to obtain the spatial pose of the tire skin.

[0029] As a further preferred embodiment, in step (33), the PCA principal component analysis includes:

[0030] For any point p, find its n nearest neighbors or neighboring points within a radius r using a binary tree. Calculate the covariance matrix of these n+1 points, and then calculate their eigenvalues ​​and eigenvectors. The eigenvector corresponding to the smallest eigenvalue is the normal vector of point p.

[0031] As a further preferred embodiment, in step (34), the calculation of the boundary point cloud of the target workpiece using the angular gap degree algorithm includes:

[0032] Traverse the point cloud of the target workpiece. Assuming the current point is p, find its n nearest neighbors using a binary tree, or all points within a radius r as its neighbors. Using the current point as the origin, map the n neighborhood points onto a plane perpendicular to the normal vector of point p, and construct a base coordinate system. At this point, the coordinates of each neighboring point are... n = 1…n, will Sort from largest to smallest As the corner gap between adjacent points, determine the maximum corner gap. When θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When p is a non-boundary point.

[0033] As a further preferred embodiment, in step (35), the fitting of the three-dimensional circle using the RANSAC algorithm includes:

[0034] (351) In a single loop, n points are randomly selected for the boundary point cloud, n≥3, and the three-dimensional circle mathematical model is calculated using the least squares method;

[0035] (352) Traverse the boundary point cloud. Assume the current point is p. When the distance between p and the 3D circle model is less than the threshold l threshold Let point p be an interior point. When the distance between p and the 3D circle model is greater than or equal to a threshold l... threshold Let point p be an outer point, and count the number of inner points;

[0036] (353) Repeat steps (351) and (352). When the number of points in the (k+1)th cycle is greater than the number of points in the (k)th cycle, the optimal three-dimensional circle model parameters are updated to the result of the (k+1)th cycle.

[0037] (354) Set a loop termination condition. When the number of loops reaches the loop termination condition, after the loop ends, take the three-dimensional circle model with the largest number of interior points as the final solution model.

[0038] As a further preferred embodiment, step two also includes: capturing depth maps of the disordered stacked tires from different angles using multiple depth cameras, and fusing the depth maps of the disordered stacked tires captured from different angles using a viewpoint fusion weight calculation formula, wherein the viewpoint fusion weight calculation formula includes:

[0039]

[0040] In the formula, Wi is the weight of the i-th viewpoint, Di is the distance from the i-th viewpoint to the target tire skin, Dj represents the distance from the j-th viewpoint to the target tire skin, σ is the standard deviation, used to adjust the sensitivity of weight allocation, and n is the total number of viewpoints.

[0041] The fused depth map is as follows:

[0042]

[0043] In the formula, Dfused(u,v) is the depth value of the fused depth map at pixel coordinates (u,v), Wi is the weight of the i-th view, and Di,depth(u,v) is the depth value of the depth map of the i-th view at pixel coordinates (u,v).

[0044] According to another aspect of the present invention, a spatial pose recognition system for disordered stacked tire casings is also provided, comprising:

[0045] The first main control module is used to obtain the ROI of the target workpiece in the depth map through a convolutional neural network;

[0046] The second main control module is used to convert the depth map into the original point cloud data of the target workpiece based on the depth camera intrinsic parameter matrix and the ROI of the target workpiece in the depth map.

[0047] The third main control module is used to preprocess the original point cloud data and calculate its boundary point cloud, and then use the RANSAC algorithm to fit a three-dimensional circle based on the boundary point cloud to obtain the spatial pose of the tire skin.

[0048] The fourth main control module is used to calculate the proportion of the number of point clouds inside the three-dimensional circle in the boundary point cloud, and to determine whether the spatial pose is accurate based on the proportion. If it is accurate, the spatial pose of the tire skin is output; otherwise, the first main control module is triggered to recalculate the ROI of the target workpiece in the depth map.

[0049] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages:

[0050] 1. This invention utilizes two data types, depth maps and point clouds, along with convolutional neural networks and traditional point cloud algorithms, to solve the problem of accurate spatial pose detection for disordered stacked tire skins. The approximate range of the target workpiece is obtained through depth maps and convolutional neural networks. Individual point cloud data of the target workpiece is obtained through point cloud preprocessing algorithms. Finally, the precise pose of the target workpiece is obtained using RANSAC 3D circle fitting and angular gap boundary calculation methods. This example can be used in the loading process of disordered stacked tire skins under complex conditions, greatly improving the automation level of this process, effectively saving labor costs and risks, and effectively solving many shortcomings of traditional manual handling methods. It has great application prospects in the future.

[0051] 2. This invention uses a convolutional neural network to quickly and accurately identify the Region of Interest (ROI) of the target workpiece in the depth map. The depth map is then converted into point cloud data, and the RANSAC algorithm is used to fit a 3D circle to determine the spatial pose of the tire casing. This streamlined identification method reduces human error and improves the efficiency and accuracy of identification.

[0052] 3. This invention effectively removes noise and outliers from the original point cloud data through steps such as preprocessing point cloud data, PCA principal component analysis, and calculating boundary point clouds based on the angular gap degree algorithm. Furthermore, the use of the RANSAC algorithm further improves the system's robustness to outlier data, ensuring accurate spatial pose identification even when some data is damaged or incomplete.

[0053] 4. This invention determines the accuracy of spatial pose by calculating the proportion of point cloud within a 3D circle to the boundary point cloud, and then self-adjusts based on this proportion. This self-optimization mechanism enables the system to continuously learn and improve in actual operation, enhancing the system's intelligence level and long-term stability. Attached Figure Description

[0054] Figure 1 This is a flowchart of a method for recognizing the spatial pose of disordered stacked tire skins according to an embodiment of the present invention;

[0055] Figure 2 This is an embodiment of the present invention that uses a convolutional neural network to input a disordered fetal depth map image;

[0056] Figure 3 This is an image representing the inference result of a convolutional neural network depth map according to an embodiment of the present invention;

[0057] Figure 4 This is a flowchart of the point cloud raw data preprocessing process according to an embodiment of the present invention;

[0058] Figure 5 This is a source image of the original point cloud data converted from a depth map ROI according to an embodiment of the present invention;

[0059] Figure 6 This is a point cloud uniformly filtered data image according to one embodiment of the present invention;

[0060] Figure 7 This is a point cloud Euclidean clustered data image as described in one embodiment of the present invention;

[0061] Figure 8 This is an image showing the result of calculating the point cloud normal vector of the target workpiece according to an embodiment of the present invention;

[0062] Figure 9 This is a flowchart of a target workpiece pose calculation method according to an embodiment of the present invention;

[0063] Figure 10 This is a cloud computing result image of the target workpiece boundary points according to an embodiment of the present invention;

[0064] Figure 11This is an image showing the spatial pose detection result of the target workpiece according to an embodiment of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0066] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for spatial pose recognition of disordered stacked tire casings, comprising the following steps:

[0067] Step 1: Obtain the ROI of the target workpiece in the depth map using a convolutional neural network.

[0068] In one embodiment, step one includes the following steps:

[0069] (11) Collect depth map data of disordered stacked tire skin under actual working conditions, mark the ROI of the tire skin that can be grabbed at the top of the depth map, construct its disordered stacked tire skin depth map dataset, and use the dataset to train the convolutional neural network model.

[0070] (12) Input the depth map of the target workpiece into the trained convolutional neural network model to obtain the ROI of the target workpiece in the depth map.

[0071] In one embodiment, the depth map is obtained by:

[0072] Depth maps of randomly stacked tires are captured from different angles using multiple depth cameras. These depth maps are then fused using a viewpoint fusion weighting formula, which includes:

[0073]

[0074] In the formula, Wi is the weight of the i-th viewpoint, Di is the distance from the i-th viewpoint to the target tire skin, Dj represents the distance from the j-th viewpoint to the target tire skin, σ is the standard deviation, used to adjust the sensitivity of weight allocation, and n is the total number of viewpoints.

[0075] The fused depth map is as follows:

[0076]

[0077] In the formula, Dfused(u,v) is the depth value of the fused depth map at pixel coordinates (u,v), Wi is the weight of the i-th view, and Di,depth(u,v) is the depth value of the depth map of the i-th view at pixel coordinates (u,v).

[0078] Step 2: Based on the depth camera intrinsic parameter matrix and the ROI of the target workpiece in the depth map, convert the depth map into the raw point cloud data of the target workpiece.

[0079] In one embodiment, step two includes the following steps:

[0080] (21) Create a mask based on the ROI of the target workpiece in the depth map, and filter the depth map based on the mask. During the filtering process, set the area outside the ROI to 0.

[0081] (22) Convert the filtered depth map data into point cloud data based on the intrinsic parameter matrix coefficients of the depth camera.

[0082] Specifically, in one embodiment, step (22) includes the following steps:

[0083] (221) The intrinsic parameter matrix coefficients of the depth camera are calibrated. These coefficients include: the focal length f along the x-axis. x y-axis focal length f y x-axis pixel offset c x y-axis pixel offset c y ;

[0084] (222) Traverse the depth map pixels to obtain their pixel coordinates (u, v) and the gray value at the pixel, which is the depth value at that point.

[0085] (223) If the depth value of the point is set to 0, skip this step; otherwise, calculate the point cloud coordinates corresponding to the pixel and obtain the original point cloud data of the target workpiece. The calculation formula for the point cloud coordinates corresponding to the pixel includes:

[0086]

[0087] Where (X, Y, Z) are the point cloud coordinates corresponding to the pixel.

[0088] Step 3: After preprocessing the original point cloud data, calculate its boundary point cloud, and use the RANSAC algorithm to fit a three-dimensional circle based on the boundary point cloud to obtain the spatial pose of the tire skin.

[0089] In one embodiment, step three includes the following steps:

[0090] (31) Perform uniform filtering on the original point cloud data to reduce the number of point clouds without losing its geometric information;

[0091] (32) Perform Euclidean clustering on the point cloud data after uniform filtering to extract the point cloud of the main target workpiece;

[0092] (33) Perform PCA principal component analysis on the point cloud of the main target workpiece to obtain the normal vector of the point cloud of the target workpiece;

[0093] (34) Based on the target workpiece point cloud and the normal vector of the target workpiece point cloud, the boundary point cloud of the target workpiece is calculated by the angular gap degree algorithm;

[0094] (35) Perform RANSAC fitting of the boundary point cloud of the target workpiece to calculate a three-dimensional circle, so as to obtain the spatial pose of the tire skin.

[0095] Specifically, in one embodiment, step (33) of the PCA principal component analysis includes:

[0096] For any point p, find its n nearest neighbors or neighboring points within a radius r using a binary tree. Calculate the covariance matrix of these n+1 points, and then calculate their eigenvalues ​​and eigenvectors. The eigenvector corresponding to the smallest eigenvalue is the normal vector of point p.

[0097] Specifically, in one embodiment, step (34), calculating the boundary point cloud of the target workpiece using an angular gap-based algorithm, includes:

[0098] Traverse the point cloud of the target workpiece. Assuming the current point is p, find its n nearest neighbors using a binary tree, or all points within a radius r as its neighbors. Using the current point as the origin, map the n neighborhood points onto a plane perpendicular to the normal vector of point p, and construct a base coordinate system. At this point, the coordinates of each neighboring point are... n = 1…n, will Sort from largest to smallest As the corner gap between adjacent points, determine the maximum corner gap. When θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When p is a non-boundary point.

[0099] Specifically, first, traverse the point cloud of the target workpiece. Assuming the current point is p, find its n nearest neighbors using a binary tree, or all points within a radius r as its neighbors. Using the current point as the origin, map the n neighborhood points onto a plane perpendicular to the normal vector of point p, and construct a polar coordinate system. Second, in the polar coordinate system of point p, the coordinates of each neighborhood point are... n = 1…n, will Sort from largest to smallest As the corner gap between adjacent points, determine the maximum corner gap. Finally, the algorithm presets a threshold of θ. threshold When θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When p is a non-boundary point.

[0100] In one embodiment, step (35), fitting a three-dimensional circle using the RANSAC algorithm, includes:

[0101] (351) In a single loop, n points are randomly selected for the boundary point cloud, n≥3, and the three-dimensional circle mathematical model is calculated using the least squares method;

[0102] (352) Traverse the boundary point cloud. Assume the current point is p. When the distance between p and the 3D circle model is less than the threshold l threshold Let point p be an interior point. When the distance between p and the 3D circle model is greater than or equal to a threshold l... threshold Let point p be an outer point, and count the number of inner points;

[0103] (353) Repeat steps (351) and (352). When the number of points in the (k+1)th cycle is greater than the number of points in the (k)th cycle, the optimal three-dimensional circle model parameters are updated to the result of the (k+1)th cycle.

[0104] (354) Set a loop termination condition. When the number of loops reaches the loop termination condition, after the loop ends, take the three-dimensional circle model with the largest number of interior points as the final solution model.

[0105] Step 4: Calculate the proportion of the number of points in the three-dimensional circle to the number of points in the boundary point cloud, and determine whether the spatial pose is accurate based on the proportion. If it is accurate, output the spatial pose of the tire skin. If not, return to Step 1 and recalculate the ROI of the target workpiece in the depth map.

[0106] In one specific embodiment, a method for spatial pose recognition of disordered stacked tire casings includes:

[0107] Step 1: Construct a depth image dataset for disordered stacked tire casings. For disordered stacked tire casings in complex scenarios, the original depth images and point cloud data of each workpiece are coupled together. Obtaining the original data of a single target workpiece is the primary task for calculating its accurate spatial pose. Traditional convolutional neural networks are mainly used for two-dimensional images, but in this example, depth maps are used as the input layer of the convolutional neural network. Figure 2 As shown, the ROI of the target workpiece in the depth map is obtained through a convolutional neural network.

[0108] (11): Collect depth map data of disordered stacked tire skin under actual working conditions, mark the ROI of the tire skin that can be grabbed at the top of the depth map, construct its disordered stacked tire skin depth map dataset, and train a convolutional neural network model based on the dataset.

[0109] (12): Based on the convolutional neural network model trained in step (11), input the depth map, execute the inference process, and obtain the ROI of the target tire skin that it can grasp, such as Figure 3 As shown.

[0110] Step 2: For the target workpiece information in the depth map, it is difficult to estimate the accurate pose of the target workpiece efficiently and accurately on the 2D image. Therefore, the subsequent steps convert it into point cloud data for subsequent accurate pose calculation. The subsequent steps are to calibrate the depth camera, obtain its intrinsic parameter matrix coefficients, use a mask to obtain the target workpiece data in the depth map, and convert the depth map data into point cloud data according to the depth camera intrinsic parameter matrix.

[0111] (21): Calibrate its depth camera and obtain its intrinsic parameter matrix coefficients, including the focal length f along the x-axis. x y-axis focal length f y x-axis pixel offset c x y-axis pixel offset c y .

[0112] (22): A mask is created by the target workpiece ROI obtained by the convolutional neural network in S101, its depth map is filtered, and the area outside the ROI is set to 0 in order to extract the target workpiece point cloud data and accelerate the subsequent depth map to point cloud process.

[0113] (23): Using the camera intrinsic parameter matrix coefficients obtained in step (21), the filtered depth map data is converted into point cloud data. The specific conversion process is as follows:

[0114] Loop through the pixels in the depth map to obtain their pixel coordinates (u, v) and the grayscale value at that pixel, which is the depth value at that point.

[0115] If the point is set to 0, skip it; otherwise, calculate its corresponding point cloud coordinates (x, y, z) according to the following formula.

[0116]

[0117] Finally, the original point cloud data of the target workpiece is obtained.

[0118] Step 3: After obtaining the original point cloud data of the target workpiece, the obtained original point cloud is preprocessed. After preprocessing, its boundary point cloud is calculated, and then the RANSAC algorithm is used to fit a 3D circle to obtain the accurate spatial pose result of its tire skin. After obtaining the original workpiece point cloud, it is difficult to directly calculate its accurate pose using a certain algorithm. Therefore, a data preprocessing algorithm is needed to realize the feasibility of pose calculation and accelerate the subsequent calculation process, such as... Figure 4 As shown.

[0119] (31): The original point cloud density is variable, such as Figure 5 As shown, the point cloud algorithm is positively correlated with the number of points in the point cloud. Therefore, uniform filtering is performed on the original point cloud to reduce the number of points without losing its geometric information, such as... Figure 6 As shown.

[0120] (32): For the tire skin point cloud after uniform filtering, due to the imprecision of the ROI obtained by the convolutional neural network, some environmental and other workpiece point clouds remain. Euclidean clustering is used to extract the point cloud of the main target workpiece, such as Figure 7 As shown.

[0121] (33): For the target workpiece point cloud obtained in the above steps, the normal vector of the target workpiece point cloud is calculated by PCA principal component analysis, such as... Figure 8 As shown.

[0122] Principal Component Analysis (PCA) determines a point p and uses a binary tree to find its n nearest neighbors or neighbors within a neighborhood with a radius of r. It then calculates the covariance matrix of these n+1 points, thereby calculating their eigenvalues ​​and eigenvectors. The eigenvector corresponding to the smallest eigenvalue is the normal vector of point p.

[0123] After point cloud data preprocessing, the target workpiece point cloud and its normal vector are obtained. The specific methods for calculating its precise spatial pose include, for example: Figure 9 As shown:

[0124] (34): For the point cloud of the target workpiece, its boundary point cloud is calculated using an algorithm based on angular gap, and the boundary point cloud of the target workpiece is extracted, such as... Figure 10 As shown.

[0125] The angular gap algorithm works as follows: traverse the point cloud. Assuming the current point is p, find its n nearest neighbors using a binary tree, or all points within a neighborhood of radius r. Using the current point as the origin, map the n neighborhood points onto a plane perpendicular to the normal vector of point p, and construct a base coordinate system. At this point, the coordinates of each neighboring point are... n = 1…n, will Sort from largest to smallest As the corner gap between adjacent points, determine the maximum corner gap. When θ max >θ threshold When point p is a boundary point, when θ max ≤θ threshold When p is a non-boundary point.

[0126] (35): After obtaining the boundary point cloud of the target workpiece, the RANSAC fitting algorithm for a 3D circle is used to calculate the center and axis direction of the 3D circle, which is the pose result of the target workpiece, such as... Figure 11 As shown.

[0127] The RANSAC 3D circle fitting algorithm process is as follows:

[0128] Defining a mathematical model of a three-dimensional circle requires the spatial coordinates of at least three points.

[0129] In a single loop, n (n≥3) points are randomly selected from the boundary point cloud, and the three-dimensional mathematical model of the circle is calculated using the least squares method.

[0130] Traverse the boundary point cloud. Assuming the current point is p, when the distance between p and the 3D circle model is less than the threshold l... threshold Let point p be an interior point. When the distance between p and the 3D circle model is greater than or equal to a threshold l... threshold Let point p be an exterior point, and count the number of interior points.

[0131] Iterate through the above loop. When the number of points in the (k+1)th loop is greater than the number of points in the kth loop, update the optimal 3D circle model parameters to the result of the (k+1)th loop.

[0132] The loop terminates when the number of iterations reaches a preset value. After the loop ends, the 3D circle model with the largest number of interior points is selected as the final solution model.

[0133] Step 4: After the above calculation process, the precise spatial pose of the target workpiece is obtained. At this point, a method for judging the correctness of the calculation results is still needed. By checking whether the calculation of the above algorithm is correct, interference and collision problems between the fixture and the environment or the target workpiece can be avoided, thereby enhancing the stability of the pose recognition algorithm.

[0134] Since the obtained boundary point cloud visualization result is the inner circle of the tire skin and its irregular outer edge, the RANSAC three-dimensional circle fitting algorithm can obtain the number of points inside the three-dimensional circle. By calculating the proportion of the number of points inside the three-dimensional circle in the boundary point cloud, it can be determined whether the calculated three-dimensional circle is its ideal inner edge of the tire skin.

[0135] The specific process is as follows:

[0136] Obtain the number of interior points and the number of boundary point clouds in the above RANSAC 3D circle fitting algorithm.

[0137] Calculate the proportion of points inside the 3D circle in the boundary point cloud.

[0138] The obtained percentage is compared with a preset threshold. If it is greater than the preset threshold, the calculation result is considered correct; otherwise, the calculation result is considered incorrect.

[0139] This example demonstrates how depth maps and point clouds, along with convolutional neural networks and traditional point cloud algorithms, can solve the problem of precise spatial pose detection for disordered stacked tire casings. The approximate range of the target workpiece is obtained through depth maps and convolutional neural networks. Individual point cloud data of the target workpiece is obtained through point cloud preprocessing algorithms. Finally, the precise pose of the target workpiece is obtained using RANSAC 3D circle fitting and angular gap boundary calculation methods. This example can be used in complex situations involving the loading process of disordered stacked tire casings, greatly improving the automation level of this process, effectively saving labor costs and risks, and effectively addressing many shortcomings of traditional manual handling methods. It has great application prospects in the future.

[0140] According to another aspect of the present invention, a system for implementing the method of any of the above embodiments or a combination of embodiments is also provided, comprising:

[0141] The first main control module is used to obtain the ROI of the target workpiece in the depth map through a convolutional neural network;

[0142] The second main control module is used to convert the depth map into the original point cloud data of the target workpiece based on the depth camera intrinsic parameter matrix and the ROI of the target workpiece in the depth map.

[0143] The third main control module is used to preprocess the original point cloud data and calculate its boundary point cloud, and then use the RANSAC algorithm to fit a three-dimensional circle based on the boundary point cloud to obtain the spatial pose of the tire skin.

[0144] The fourth main control module is used to calculate the proportion of the number of point clouds inside the three-dimensional circle in the boundary point cloud, and to determine whether the spatial pose is accurate based on the proportion. If it is accurate, the spatial pose of the tire skin is output; otherwise, the first main control module is triggered to recalculate the ROI of the target workpiece in the depth map.

[0145] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for spatial pose recognition of disordered stacked tire casings, characterized in that, Includes the following steps: Step 1: Obtain the ROI of the target workpiece in the depth map using a convolutional neural network; Step 2: Based on the depth camera intrinsic parameter matrix and the ROI of the target workpiece in the depth map, convert the depth map into the raw point cloud data of the target workpiece. Step 3: After preprocessing the original point cloud data, calculate its boundary point cloud, and use the RANSAC algorithm to fit a three-dimensional circle based on the boundary point cloud to obtain the spatial pose of the tire skin. Step three also includes: capturing depth maps of the disordered stacked tires from different angles using multiple depth cameras, and fusing the depth maps captured from different angles using a viewpoint fusion weight calculation formula, wherein the viewpoint fusion weight calculation formula includes: ， In the formula, It is the first Weight of each perspective It is the first The distance from each angle to the target tire casing. Indicates the first The distance from each angle to the target tire casing. It is the standard deviation, used to adjust the sensitivity of weight allocation. It is the total number of viewpoints; The merged depth map is as follows: ， In the formula, For the merged depth map in pixel coordinates Depth value at that location, The weight of the nth viewpoint, For the first Depth map of each viewpoint in pixel coordinates The depth value at that location; Step 4: Calculate the proportion of the number of points in the three-dimensional circle to the number of points in the boundary point cloud, and determine whether the spatial pose is accurate based on the proportion. If it is accurate, output the spatial pose of the tire skin. If not, return to Step 1 and recalculate the ROI of the target workpiece in the depth map.

2. The method for spatial pose recognition of disordered stacked tire casings according to claim 1, characterized in that, Step one includes the following steps: (11) Collect depth map data of disordered stacked tire skin under actual working conditions, mark the ROI of the tire skin that can be grabbed at the top of the depth map, construct its disordered stacked tire skin depth map dataset, and use the dataset to train the convolutional neural network model. (12) Input the depth map of the target workpiece into the trained convolutional neural network model to obtain the ROI of the target workpiece in the depth map.

3. The method for spatial pose recognition of disordered stacked tire casings according to claim 1, characterized in that, Step two includes the following steps: (21) Create a mask based on the ROI of the target workpiece in the depth map, and filter the depth map based on the mask. During the filtering process, set the area outside the ROI of the depth map to 0 through the mask. (22) Convert the filtered depth map data into point cloud data based on the intrinsic parameter matrix coefficients of the depth camera.

4. The method for spatial pose recognition of disordered stacked tire casings according to claim 3, characterized in that, Step (22) includes the following steps: (221) The intrinsic parameter matrix coefficients of the depth camera are calibrated, including: the focal length of the x-axis. y-axis focal length x-axis pixel offset y-axis pixel offset ; (222) Iterate through the pixels in the depth map and obtain their pixel coordinates. And the grayscale value at the pixel, that is, the depth value at the pixel; (223) If the depth value of the pixel is set to 0, skip this step; otherwise, calculate the point cloud coordinates corresponding to the pixel. After the loop ends, obtain the original point cloud data of the target workpiece. The calculation formula for the point cloud coordinates corresponding to the pixel includes: ， in, These are the point cloud coordinates corresponding to the pixels.

5. The method for spatial pose recognition of disordered stacked tire casings according to claim 3, characterized in that, Step three includes the following steps: (31) Perform uniform filtering on the original point cloud data to reduce the number of point clouds without losing its geometric information; (32) Perform Euclidean clustering on the point cloud data after uniform filtering to extract the point cloud of the main target workpiece; (33) Perform PCA principal component analysis on the point cloud of the main target workpiece to obtain the normal vector of the point cloud of the target workpiece; (34) Based on the point cloud of the target workpiece and the normal vector of the point cloud of the target workpiece, the boundary point cloud of the target workpiece is calculated by the angular clearance algorithm, including the inner circular boundary and the outer irregular boundary of the tire skin. (35) Perform RANSAC fitting of the boundary point cloud of the target workpiece to calculate the three-dimensional circle, obtain the coefficients of the three-dimensional circle model of the inner circular boundary of the tire skin, and obtain the spatial pose of the tire skin.

6. The method for spatial pose recognition of disordered stacked tire casings according to claim 5, characterized in that, In step (33), the PCA principal component analysis includes: For any point p, find its n nearest neighbors or neighboring points within a radius r using a binary tree. Calculate the covariance matrix of these n+1 points, and then calculate their eigenvalues ​​and eigenvectors. The eigenvector corresponding to the smallest eigenvalue is the normal vector of point p.

7. A method for spatial pose recognition of disordered stacked tire casings according to claim 5, characterized in that, In step (34), the calculation of the boundary point cloud of the target workpiece using the angular gap algorithm includes the following steps: Traverse the point cloud of the target workpiece. Assuming the current point is p, find its n nearest neighbors using a binary tree, or all points within a radius r as its neighbors. Using the current point as the origin, map the n neighborhood points onto a plane perpendicular to the normal vector of point p, and construct a base coordinate system. At this point, the coordinates of each neighboring point are... , i=1,…,n, will Sort from largest to smallest As the corner gap between adjacent points, determine the maximum corner gap. ,when When point p is a boundary point, when When p is a non-boundary point.

8. A method for spatial pose recognition of disordered stacked tire casings according to claim 5, characterized in that, In step (35), fitting a three-dimensional circle using the RANSAC algorithm includes: (351) In a single loop, n points are randomly selected for the boundary point cloud, n≥3, and the three-dimensional circle model and model parameters are calculated using the least squares method; (352) Traverse the boundary point cloud. Assume the current point is p. When the distance between p and the 3D circle model is less than the threshold... Let point p be an interior point. When the distance between p and the 3D circle model is greater than or equal to a threshold... Let point p be an outer point, and count the number of inner points; (353) Repeat steps (351) and (352). When the number of points in the (k+1)th cycle is greater than the number of points in the (k)th cycle, update the optimal 3D circle model parameters to the result of the (k+1)th cycle. (354) Set the loop termination condition. When the number of loops reaches the loop termination condition, after the loop ends, take the three-dimensional circle model with the largest number of interior points as the final solution model.

9. A spatial pose recognition system for disordered stacked tire casings, characterized in that, include: The first main control module is used to obtain the ROI of the target workpiece in the depth map through a convolutional neural network; The second main control module is used to convert the depth map into the original point cloud data of the target workpiece based on the depth camera intrinsic parameter matrix and the ROI of the target workpiece in the depth map. The third main control module is used to preprocess the original point cloud data and calculate its boundary point cloud, and then use the RANSAC algorithm to fit a three-dimensional circle based on the boundary point cloud to obtain the spatial pose of the tire skin. In the third main control module, depth maps of the disordered stacked tire skin are captured from different angles using multiple depth cameras, and the depth maps of the disordered stacked tire skin captured from different angles are fused using a viewpoint fusion weight calculation formula, which includes: ， In the formula, It is the first Weight of each perspective It is the first The distance from each angle to the target tire casing. Indicates the first The distance from each angle to the target tire casing. It is the standard deviation, used to adjust the sensitivity of weight allocation. It is the total number of viewpoints; The merged depth map is as follows: ， In the formula, For the merged depth map in pixel coordinates Depth value at that location, The weight of the nth viewpoint, For the first Depth map of each viewpoint in pixel coordinates The depth value at that location; The fourth main control module is used to calculate the proportion of the number of point clouds inside the three-dimensional circle in the boundary point cloud, and to determine whether the spatial pose is accurate based on the proportion. If it is accurate, the spatial pose of the tire skin is output; otherwise, the first main control module is triggered to recalculate the ROI of the target workpiece in the depth map.

Images