Molecular sieve drying bag packaging attitude estimation method

Abstract

The invention provides a molecular sieve drying bag packaging attitude estimation method, which comprises the following steps: a packaging point cloud acquisition step: acquiring a coarse point cloud picture of a drying bag package; a packaging point cloud segmentation step: segmenting the packaging skirt through the pre-segmentation bounding box to obtain a fine point cloud picture; a target main body extraction step: screening out each target main body in the package, and extracting the target main body in the middle position; a target main body estimation step: performing feature calculation through a minimum bounding box and principal component analysis to obtain a bounding box data set of the point cloud of the main body to be analyzed, adjusting the posture of the bounding box according to the principal component direction, and cutting the point cloud, estimating the point cloud attitude and the center position of the to-be-analyzed main body by combining the bounding box data set and the mass center of the to-be-analyzed main body; the method has the advantages that the effective target point cloud main body in the package is extracted by adopting the automatic threshold iterative segmentation method, the actual grabbing position and posture of the package are estimated based on the two-time minimum bounding box and the characteristics of the principal component analysis method, and positioning grabbing of a manipulator is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of posture estimation, and more particularly to a method for estimating the packaging posture of a molecular sieve drying bag. Background Art

[0002] The design of modern automobile refrigeration systems tends to adopt lightweight materials and advanced manufacturing technology. The industry has made a new design for the traditional liquid storage tank, and a new molecular sieve drying pack has been introduced. The new molecular sieve drying pack is a filter structure that can be separated and disassembled from the liquid storage tank. At present, the packaging process of this product after production is still completed manually. In order to ensure the uniformity of product packaging quality and free people from this boring working environment, it is necessary to adopt automated production; During the automated production process, the dry packs are vacuum-sealed and packaged and then unloaded by robots. The surface structure of the packages after vacuuming is complex and lacks a fixed posture. Traditional robot automation systems find it difficult to accurately locate and grasp them. Some existing 3D recognition methods mainly match and identify 3D models, but this method is not accurate and time-consuming when facing non-rigid objects, making it difficult to meet the requirements. Although deep learning methods have high accuracy, they require a large number of training samples, and the internal working mechanism of the model is not transparent, making it difficult to debug on industrial sites once problems occur. Summary of the invention

[0003] In view of the deficiencies in the prior art, the present invention aims to provide a method for estimating the packaging posture of a molecular sieve drying bag.

[0004] To achieve the above object, the present invention provides the following technical solutions: A method for estimating the packaging posture of a molecular sieve drying package, comprising: The step of obtaining a packaging point cloud is to obtain an image of the dry package collected by the camera, and to perform point cloud processing on the image to obtain a rough point cloud image; The packaging point cloud segmentation step is to segment the packaging skirt using the pre-segmented bounding box in the coarse point cloud image to obtain a fine point cloud image; A target subject extraction step, screening out each target subject inside the package in the fine point cloud image, and extracting the target subject in the middle position as the subject to be analyzed; In the target subject estimation step, the subject to be analyzed is subjected to feature calculation through the minimum bounding box and principal component analysis to obtain the bounding box data set of the point cloud of the subject to be analyzed, and then the posture of the bounding box is adjusted according to the direction of the principal component and the point cloud is cut, and the point cloud posture and center position of the subject to be analyzed are estimated by combining the bounding box data set and the center of mass of the subject to be analyzed.

[0005] Furthermore, the target subject extraction step includes a target subject screening strategy, which includes iteratively segmenting the fine point cloud image through a directional filter to obtain a bounding box filtering direction, and combining point cloud Euclidean clustering with PCA main features to screen out each target subject.

[0006] Furthermore, the target subject screening strategy is configured as follows: , , , in, is the point threshold, is the Mth point cloud block in the fine point cloud image, is the set of all target point cloud blocks that meet the point count requirements. and Point cloud blocks and The principal eigenvector of for and The cosine similarity of two main eigenvectors, is the similarity threshold.

[0007] Furthermore, the iterative segmentation sub-strategy includes projecting the fine point cloud image to the side and front of the pre-segmented bounding box, and deduplicating the projected point cloud to obtain a projected point cloud, and calculating the projected point cloud in each direction by searching for the optimal rotation angle formula to obtain a positive angle value and a side angle value, the positive angle value reflects the angle of rotation of the projected point cloud around the rotation axis of the pre-segmented bounding box pointing to the front, and the side angle value reflects the angle of rotation of the projected point cloud around the rotation axis of the bounding box pointing to the side.

[0008] Furthermore, the formula for searching for the optimal rotation angle is configured as: , , , , , , in, is the number of iterations, For the The rotation angle at the iteration, Respectively In iteration The left rotation angle and the right rotation angle of For the The search step size for each calculation is For the The angle corresponding to the minimum area of ​​the projected point cloud in this calculation, The side projection and front projection are The angle corresponding to the minimum area of ​​the projected point cloud in this calculation, are the positive angle value and the side angle value respectively, The area of ​​the projected point cloud, is the control threshold of the iteration convergence condition.

[0009] Furthermore, the target subject extraction step includes a target subject extraction strategy, which includes respectively calculating the centroid corresponding to each target subject, and extracting the target subject in the middle position as the subject to be analyzed according to the correlation between the centroids, and the correlation is configured as: , , , , in, There are three target subjects The center of mass, The subject to be analyzed.

[0010] Furthermore, the bounding box data set includes the length, width and height of the bounding box, the center of the bounding box and the main feature direction of the point cloud in the bounding box.

[0011] Furthermore, the target body estimation step includes a posture calculation strategy, which includes re-using the minimum bounding box and principal component analysis algorithm on the cut point cloud, and using the first eigenvector, the second eigenvector and the third eigenvector calculated in the principal component analysis method as the spatial posture of the overall packaged point cloud.

[0012] Furthermore, the target subject estimation step includes a center calculation strategy, which includes constructing a plane based on the main feature direction of the point cloud in the bounding box and the center of the bounding box, taking the center of mass of the subject to be analyzed as the starting point, and obtaining the center position by intersecting the main component direction with the plane.

[0013] Furthermore, the central computing strategy configuration includes: , , , in, is the plane normal vector, are the center coordinates of the bounding box, is the centroid coordinate of the subject to be analyzed, is the principal component direction vector, The center position coordinates of the subject to be analyzed, are the first constant and the second constant respectively.

[0014] The beneficial effects of the present invention are as follows: the present invention uses the characteristics of an RGB-D depth camera and an automatic threshold iterative segmentation method to extract the effective target point cloud body in the package, estimates the actual grasping position and posture of the package based on the features of two minimum bounding boxes and principal component analysis, specifically calculates the filtering direction of the directional filter of the segmented target by a plane projection method based on the bounding box, combines Euclidean clustering and the principal component direction to perform target screening and directional filter iterative segmentation, extracts the intermediate target point cloud after all the target body point clouds are segmented, calculates the features of the extracted target body point cloud by using the minimum bounding box and principal component analysis algorithms, adjusts the posture of the bounding box according to the principal component direction and cuts the point cloud, and estimates the point cloud posture and center position of the subject to be analyzed in combination with the bounding box data set and the center of mass of the subject to be analyzed, so as to facilitate the robot to locate and grasp the entire package. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of the method for estimating the packaging posture of the molecular sieve drying package of the present invention; Figure 2 This is a rendering of the packaging point cloud preprocessing in the present invention; Figure 3 This is the effect diagram of point cloud segmentation of three target entities in the present invention; Figure 4 It is an effect diagram of the direction selection of the directional filter in the present invention; Figure 5 It is a schematic diagram of target posture estimation and position estimation in the present invention; Figure 6 It is the effect diagram of the packaging posture estimation in the present invention. DETAILED DESCRIPTION

[0016] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The same parts are represented by the same reference numerals. It should be noted that the words "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to directions in the accompanying drawings, and the words "bottom surface" and "top surface", "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.

[0017] Since the surface structure of the drying package is complex after vacuum sealing during the automated production process, and is affected by the aluminum foil material of the package itself and passive ambient light, the image quality of ordinary 2D cameras is poor due to diffuse reflection. At the same time, the surface of the molecular sieve drying package is non-rigid and relatively random, with many large skirts and presenting a flat or various upturned states. The general identification and positioning methods are difficult to meet the requirements. Therefore, this invention provides a method for estimating the posture of the molecular sieve drying package. Figure 1 As shown, the following steps are included: The step of obtaining the packaging point cloud is to obtain the image of the dry bag packaging captured by the RGB-D camera carried by the robot, and perform point cloud processing on the image to obtain a rough point cloud map; In the packaging point cloud segmentation step, the packaging skirt is segmented by the pre-segmented bounding box in the coarse point cloud to obtain a fine point cloud. The coarse point cloud is first preprocessed to extract the complete dry package point cloud, such as Figure 2 As shown in the figure, the preprocessing includes point cloud direct filtering, body centroid downsampling, and point cloud difference segmentation. The purpose is to separate the background irrelevant to the package and obtain the point cloud of the package to be estimated. Secondly, since the package has large skirt wrinkles after vacuuming, and the object in the package bag does not fill the entire package bag, by cutting the part of the package skirt with wrinkles, the influence of the package's own instability on the target subject segmentation can be reduced. Therefore, Figure 3 As shown, the minimum bounding box of the packaging point cloud is calculated for the point cloud image after separating the background, and the center, direction and size of the bounding box are obtained. A rectangular box is obtained by adjusting and shortening the size of the long side of the bounding box. The rectangular box is used for segmentation to remove part of the skirt.

[0018] The pre-segmentation configuration of the pre-segmentation bounding box is: make ,have: , , , , , , , , , in, To express the bounding box of the packed point cloud, is divided into its center point coordinates, is the Euler posture representation of the bounding box, are the length, width and height of the bounding box respectively. They are the rotation angles of the bounding box around the Z, Y, and X axes, respectively. Represents the unit vector in the X direction of the bounding box, Respectively The value in the positive and negative directions, Represents the length adjustment, Indicates the length of the adjusted bounding box, are the adjusted bounding box center coordinates, To obtain the pre-segmented bounding box.

[0019] In the target subject extraction step, each target subject inside the package is screened out in the fine point cloud image. There are three target subjects in the dry package, and the target subject in the middle position is extracted as the subject to be analyzed; specifically, Figure 3 As shown, the target subject extraction step includes a target subject screening strategy, which includes iteratively segmenting the fine point cloud through a directional filter to obtain a bounding box filtering direction (after the bounding box is rotated by an angle, the Z axis of the rotated bounding box is as shown in Figure 4 The blue axis direction is used as the filtering direction), and each target subject is selected by combining point cloud Euclidean clustering and PCA main features; the target subject selection strategy configuration includes: , , , in, is the point threshold, is the Mth point cloud block in the fine point cloud image, is the set of all target point cloud blocks that meet the point count requirements. and Point cloud blocks and The principal eigenvector of for and The cosine similarity of two main eigenvectors, is the similarity threshold.

[0020] The iterative segmentation sub-strategy includes projecting the fine point cloud image to the side and front of the pre-segmented bounding box, and deduplicating the projected point cloud to obtain the projected point cloud. The projected point cloud in each direction is calculated by searching the optimal rotation angle formula to obtain the positive angle value and the side angle value. The positive angle value reflects the angle of rotation of the projected point cloud around the pre-segmented bounding box pointing to the front rotation axis, and the side angle value reflects the angle of rotation of the projected point cloud around the bounding box pointing to the side rotation axis. The search for the optimal rotation angle formula is configured as: , , , , , , in, is the number of iterations, For the The rotation angle at the iteration, Respectively In iteration The left rotation angle and the right rotation angle of For the The search step size for each calculation is For the The angle corresponding to the minimum area of ​​the projected point cloud in this calculation, The side projection and front projection are The angle corresponding to the minimum area of ​​the projected point cloud in this calculation, are the positive angle value and the side angle value respectively, The area of ​​the projected point cloud, is the control threshold of the iteration convergence condition. According to the above formula, only the 0th iteration parameter, i.e. the initial angle, needs to be given. and the initial search step The best solution ,like Figure 3 As shown in the figure, the bounding box is rotated according to the optimal rotation angle value. The closer the bounding box is to the surface of the object, the more accurate the corresponding filtering direction is. Figure 4 In (b), the blue main axis pointing upward is the filtering direction.

[0021] The target subject extraction step includes a target subject extraction strategy, which includes calculating the centroid corresponding to each target subject respectively, and extracting the target subject in the middle position as the subject to be analyzed according to the correlation between the centroids. The correlation configuration is: , , , , in, There are three target subjects The center of mass, The subject to be analyzed.

[0022] In the target subject estimation step, the subject to be analyzed is subjected to feature calculation through the minimum bounding box and principal component analysis to obtain the bounding box data set of the point cloud of the subject to be analyzed. The bounding box data set includes the length, width and height of the bounding box, the center of the bounding box and the main feature direction of the point cloud in the bounding box. Then, the posture of the bounding box is adjusted according to the principal component direction and the point cloud is cut. The point cloud posture and center position of the subject to be analyzed are estimated by combining the bounding box data set and the centroid of the subject to be analyzed. Figure 5 is the principle of the target in the estimation stage, Figure 6 It is the effect of the final estimation. The present invention replaces the overall estimation with local estimation. Specifically, the target body estimation step includes a posture calculation strategy. The posture calculation strategy includes re-using the minimum bounding box and principal component analysis algorithm for the cut point cloud, and the first eigenvector, the second eigenvector and the third eigenvector calculated in the principal component analysis method are used as the spatial posture of the overall packaged point cloud.

[0023] The target subject estimation step includes a center calculation strategy, which includes forming a plane based on the main feature direction of the point cloud in the bounding box and the center of the bounding box, taking the center of mass of the subject to be analyzed as the starting point, and obtaining the center position by intersecting the main component direction with the plane; wherein, the center calculation strategy configuration includes: , , , in, is the plane normal vector, are the center coordinates of the bounding box, is the centroid coordinate of the subject to be analyzed, is the principal component direction vector, The center position coordinates of the subject to be analyzed, are the first constant and the second constant respectively.

[0024] Among them, for the bounding box The plane normal vector formed by them is , if any , the center of the bounding box , point cloud centroid , PCA principal axis vector ; , , , , , .

[0025] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for estimating the packaging posture of a molecular sieve drying package, characterized in that: include The step of obtaining a packaging point cloud is to obtain an image of the dry package collected by the camera, and to perform point cloud processing on the image to obtain a rough point cloud image; The packaging point cloud segmentation step is to segment the packaging skirt using the pre-segmented bounding box in the coarse point cloud image to obtain a fine point cloud image; A target subject extraction step, screening out each target subject inside the package in the fine point cloud image, and extracting the target subject in the middle position as the subject to be analyzed; In the target subject estimation step, the subject to be analyzed is subjected to feature calculation through the minimum bounding box and principal component analysis to obtain the bounding box data set of the point cloud of the subject to be analyzed, and then the posture of the bounding box is adjusted according to the direction of the principal component and the point cloud is cut, and the point cloud posture and center position of the subject to be analyzed are estimated by combining the bounding box data set and the center of mass of the subject to be analyzed.

2. According to claim 1, a method for estimating the packaging posture of a molecular sieve drying package is characterized in that: The target subject extraction step includes a target subject screening strategy, which includes iteratively segmenting the fine point cloud image through a directional filter to obtain a bounding box filtering direction, and combining point cloud Euclidean clustering with PCA main features to screen out each target subject.

3. According to claim 2, a method for estimating the packaging posture of a molecular sieve drying package is characterized in that: The target subject screening strategy configuration is: ， ， ， in, is the point threshold, is the Mth point cloud block in the fine point cloud image, is the set of all target point cloud blocks that meet the point count requirements. and Point cloud blocks and The principal eigenvector of for and The cosine similarity of two main eigenvectors, is the similarity threshold.

4. The method for estimating the packaging posture of a molecular sieve drying package according to claim 3, characterized in that: The iterative segmentation sub-strategy includes projecting the fine point cloud image to the side and front of the pre-segmented bounding box, and deduplicating the projected point cloud to obtain a projected point cloud, and calculating the projected point cloud in each direction by searching the optimal rotation angle formula to obtain a positive angle value and a side angle value. The positive angle value reflects the angle of rotation of the projected point cloud around the rotation axis of the pre-segmented bounding box pointing to the front, and the side angle value reflects the angle of rotation of the projected point cloud around the rotation axis of the bounding box pointing to the side.

5. A method for estimating the packaging posture of a molecular sieve drying package according to claim 4, characterized in that: The formula for searching for the optimal rotation angle is configured as follows: ， , , ， ， ， in, is the number of iterations, For the The rotation angle at the iteration, Respectively In iteration The left rotation angle and the right rotation angle of For the The search step size for each calculation is For the The angle corresponding to the minimum area of ​​the projected point cloud in this calculation, The side projection and front projection are The angle corresponding to the minimum area of ​​the projected point cloud in this calculation, are the positive angle value and the side angle value respectively. The area of ​​the projected point cloud, is the control threshold of the iteration convergence condition.

6. A method for estimating the packaging posture of a molecular sieve drying package according to claim 5, characterized in that: The target subject extraction step includes a target subject extraction strategy, which includes respectively calculating the centroid corresponding to each target subject, and extracting the target subject in the middle position as the subject to be analyzed according to the correlation between the centroids, and the correlation is configured as follows: ， ， ， ， in, There are three target subjects The center of mass, The subject to be analyzed.

7. A method for estimating the packaging posture of a molecular sieve drying package according to claim 1 or 6, characterized in that: The bounding box data set includes the length, width and height of the bounding box, the center of the bounding box and the main feature direction of the point cloud in the bounding box.

8. A method for estimating the packaging posture of a molecular sieve drying package according to claim 7, characterized in that: The target body estimation step includes a posture calculation strategy, which includes reusing the minimum bounding box and principal component analysis algorithm on the cut point cloud, and using the first eigenvector, the second eigenvector and the third eigenvector calculated in the principal component analysis method as the spatial posture of the overall packaged point cloud.

9. A method for estimating the packaging posture of a molecular sieve drying package according to claim 8, characterized in that: The target subject estimation step includes a center calculation strategy, which includes forming a plane based on the main feature direction of the point cloud in the bounding box and the center of the bounding box, taking the center of mass of the subject to be analyzed as the starting point, and obtaining the center position by intersecting the main component direction with the plane.

10. A method for estimating the packaging posture of a molecular sieve drying package according to claim 9, characterized in that: The central computing strategy configuration includes: , ， ， in, is the plane normal vector, are the center coordinates of the bounding box, is the centroid coordinate of the subject to be analyzed, is the principal component direction vector, The center position coordinates of the subject to be analyzed, are the first constant and the second constant respectively.

Images