A six-degree-of-freedom pose estimation dataset automatic acquisition system

Abstract

This invention discloses an automatic acquisition system for a six-DOF pose estimation dataset, comprising a data acquisition platform and a data processing device. The data acquisition platform is used to capture RGBD image sequences of a target scene using a depth camera. The data processing device is equipped with data annotation software algorithms to process the RGBD image sequences of the target scene, perform 3D reconstruction of the target objects, obtain a 3D model, and automatically annotate the segmentation mask information and six-DOF pose information of the objects in the target scene based on the 3D model. This invention can automatically annotate the pose information, segmentation mask information, and 3D model information of objects in the RGB-D image sequences acquired by the depth camera. The resulting dataset can be used for training and testing of deep learning-based robot grasping neural network models.

Description

Technical Field

[0001] This invention relates to the field of robotics, specifically to an automatic acquisition system for six-degree-of-freedom pose estimation datasets, suitable for the annotation of training or testing datasets for deep learning network models for robot grasping. Background Technology

[0002] With the rapid development of science and technology and industrial modernization, the robotics industry has ushered in new development opportunities, with its market share increasing day by day. A large number of robots are being applied to actual production tasks such as sorting, assembly, and material loading, playing an important role in many industries. Compared with traditional manual operations, robots have advantages such as high operational accuracy, strong system stability, and high return on investment.

[0003] With the rapid rise of artificial intelligence technology and the continuous iteration of smart hardware, computer vision and robotics are becoming increasingly intertwined. Robots can use cameras as "eyes" to acquire visual information about the scene and interact with the external environment.

[0004] Robotic grasping is a crucial step in robotic sorting, assembly, and loading operations, and a key technology for automating and intelligentizing robot operations. Robotic grasping can be further divided into three sub-tasks: grasping detection, grasping planning, and grasping control. Grasp detection is the foundation of grasping planning and control. It typically uses cameras, especially depth cameras, and other optical instruments to collect image and depth information of the grasping scene, constructing a 3D point cloud of the scene. Algorithms then estimate the position and pose of the target object, forming a grasping description that guides the robot to grasp. Therefore, a common grasping detection method involves registering the 3D model of the target object with its actual point cloud to estimate its pose. Classic algorithms include the LineMOD algorithm based on template matching and point-pair feature algorithms based on voting.

[0005] With the popularization and application of artificial intelligence technology, more and more deep learning-based pose estimation algorithms are being applied to robot grasping tasks. Examples include robot grasping methods based on the improved Keypoint R-CNN model and robot grasping methods based on PVNet. These methods often require a large amount of data to train the network model to achieve the desired pose estimation accuracy. The richness and annotation quality of the dataset directly affect the performance of the network model.

[0006] Currently, although datasets specifically designed for six-DOF pose estimation have emerged, such as LineMOD, HomebrewedDB, and HOPE, the annotation process is time-consuming and labor-intensive, making it difficult to obtain comprehensive and diverse dataset annotation results. For example, the LineMOD dataset only annotated approximately 1000 pose and segmentation mask labels for 15 objects, the HomebrewedDB dataset only annotated 33 objects in 13 scenes, and the HOPE dataset only annotated 28 toy objects in 50 scenes. Although there are open-source six-DOF object pose estimation dataset creation tools such as ObjectDataSetTools that can automate the process of creating pose estimation datasets (i.e., automatically annotating pose information, segmentation mask information, and object 3D model information from a given RGB-D image sequence), these tools can only annotate single objects and cannot handle complex scenes or occlusion issues between objects. Furthermore, because normal direction unification is not performed during 3D reconstruction, the 3D reconstruction results often exhibit deformation and tearing, significantly reducing the annotation quality of the pose estimation dataset. Summary of the Invention

[0007] In view of the technical defects mentioned in the background art, the purpose of this embodiment of the invention is to realize the automatic annotation of the six-degree-of-freedom pose data of objects in the shooting scene, thereby laying the foundation for the realization of unordered grasping by robots. The six-degree-of-freedom pose data mainly includes the pose information of the object, the segmentation mask information of the object in the image, and the three-dimensional model information of the object.

[0008] To achieve the above objectives, embodiments of the present invention provide an automatic acquisition system for a six-DOF pose estimation dataset, comprising a data acquisition platform and a data processing device. The data acquisition platform is used to capture RGBD image sequences of a target scene using a depth camera.

[0009] The data processing device is equipped with data annotation software algorithms, which are used to process the RGBD image sequence of the target scene, perform three-dimensional reconstruction of the target object, obtain a three-dimensional model, and automatically annotate the segmentation mask information and six-degree-of-freedom pose information of the object in the target scene based on the three-dimensional model.

[0010] In one specific implementation of the present invention, the data acquisition platform includes an electric turntable and a robotic arm; a target object is placed on the electric turntable, and the electric turntable receives control commands from a host computer and rotates according to the control commands; a depth camera is installed at the end of the robotic arm, and the depth camera can capture RGD images of the target object from various angles to obtain the RGBD image sequence.

[0011] Furthermore, the electric turntable receives control commands from the host computer via an RS-232 communication module; the control commands include the rotation angle and rotation speed of the electric turntable.

[0012] As one specific implementation, the electric turntable includes a stepper motor, a belt, an acrylic plate, a base plate, and a circular slide rail. The stepper motor is mounted on the base plate, the belt is mounted above the circular slide rail, the circular slide rail is mounted above the base plate by two studs, and the acrylic plate is mounted above the belt by two studs. The acrylic plate is marked with ArUoc.

[0013] As a specific implementation of the present invention, the data processing device uses a data annotation software algorithm to process the RGBD image sequence of the target scene, specifically including:

[0014] The first step, point cloud alignment: Find the three-dimensional coordinates of the four corner points of the m ArUco markers in the k-th frame image from the RGBD image sequence, denoted as X. k = {xki|i = 1, 2, ..., 4m}, then determine the X between frames. k The correspondence; based on the X of each frame image k Based on the correspondence between the point cloud and other frame images, the transformation matrix T between the point cloud of the k-th frame and the point cloud of the first frame is calculated using a global point cloud registration algorithm. 1k If there are n frames in the image sequence, then the point cloud transformation set T1 = {T 1k |k=1,2,…,n}, that is, the point cloud of the kth frame passes through T 1k After transformation, it can be aligned with the point cloud of the first frame;

[0015] The second step is 3D reconstruction: using the random sample consensus algorithm to reconstruct X. k Perform plane fitting and filter out points in the point cloud that are near the plane in each frame; based on T in set T1 1k The position and pose of the point cloud in frame k are transformed; the transformed point cloud is stitched together in 3D to obtain the complete point cloud of the object, and then smoothed to remove noise, resulting in the reconstructed point cloud M; the reconstructed point cloud M is segmented using the Euclidean clustering algorithm to obtain the point clouds O = {o} of c objects in the scene. i |i=1,2,…,c};

[0016] The third step is surface triangulation: The normal vectors of the point clouds for each object are calculated, and the normal vectors are standardized using a tree-based hierarchical Riemann diagram-based normal propagation method. The Poisson surface reconstruction algorithm is then used to triangulate O, resulting in the 3D model m of each object. i , i = 1, 2, ..., c;

[0017] Step 4, segmentation information annotation: Annotate the 3D model of the object m i The triangular faces in i = 1, 2, ..., c are projected onto the camera plane one by one to obtain the object segmentation mask sequence;

[0018] Step 5, Pose Information Annotation: Calculate the 3D model m of each object. i The orientation bounding box and the pose transformation matrix of the bounding box Based on the pose transformation relationship T between the first frame point cloud and the kth frame point cloud 1k Calculate the object model m in the point cloud of the kth frame. i Six-DOF pose information

[0019] In the surface triangulation step, the normal vectors of each object's point cloud are standardized using a normal propagation method based on a tree-structured hierarchical Riemann diagram. Specifically:

[0020] (1) In the principal component analysis method, a larger neighborhood radius is used to estimate the normal vector of the point cloud to obtain a rough normal vector;

[0021] (2) Select the point with the smallest curvature as the root node of the minimum spanning tree and mark it;

[0022] (3) Calculate the distance from all unmarked points to the tree, and find the point p that is closest to the tree. i and distance p i The nearest tree node;

[0023] (4) p i It is added to the tree as a new node, connected to the nearest tree node, a new edge is generated, and p is labeled. i ;

[0024] (5) Repeat steps 3 and 4 until all points in the point cloud are added to the minimum spanning tree;

[0025] (6) Traverse all edges of the minimum spanning tree and calculate the angle between the normal vectors of the nodes connected at both ends; if the angle is greater than 90°, reverse it and add the normal vector of the node to obtain a coarse normal vector with a consistent direction.

[0026] (7) In the principal component analysis method, a smaller neighborhood radius is used to estimate the normal vector of the point cloud to obtain an accurate normal vector;

[0027] (8) Calculate the angle between the precise normal vector and the coarse normal vector at each point. If the angle is greater than 90°, reverse the precise normal vector to obtain a precise normal vector with a consistent direction.

[0028] Compared with existing technologies, the advantages of this invention are as follows: It can automatically obtain the target's 3D model information, six-DOF pose information, and segmentation mask information simply by using an RGB-D camera to capture image sequences of the target placed on a data acquisition hardware platform, without requiring a 3D scanner or human intervention in the shooting process. In generating the object's 3D model, this invention uses a normal propagation method based on a tree-structured layered Riemann diagram for normal orientation, ensuring the consistency of the normal directions of the object's surface point cloud. This avoids deformation and tearing of the 3D model due to inconsistent normal directions, thus obtaining a more accurate 3D model. Based on this, perspective projection is performed using the triangulated 3D model to obtain more precise image segmentation mask information. Attached Figure Description

[0029] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below.

[0030] Figure 1 This is a schematic diagram of the data acquisition platform provided in an embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of data acquisition in this invention;

[0032] Figure 3 This is a flowchart of the data annotation software algorithm of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0035] The automatic acquisition system for a six-DOF pose estimation dataset of this invention includes a data acquisition platform (also referred to as a data acquisition hardware platform) and data processing equipment. The data acquisition hardware platform is used to capture RGBD image sequences of the target scene using a depth camera. The data acquisition hardware platform is an electrically driven turntable marked with an ArUco tag, capable of placing the object to be detected. This turntable can communicate with a host computer via an RS-232 communication module. The host computer issues rotation angle and speed information in the form of commands, and the electrically driven turntable automatically rotates according to the commands from the host computer.

[0036] The data processing device is equipped with data annotation software algorithms, which are used to process the RGBD image sequence of the target scene, perform three-dimensional reconstruction of the target object, obtain a three-dimensional model, and automatically annotate the segmentation mask information and six-degree-of-freedom pose information of the object in the target scene based on the three-dimensional model.

[0037] like Figure 1 As shown, the data acquisition hardware platform provided by this invention is an electric turntable, including a stepper motor 1, a belt 2, an acrylic plate 3, a base plate 4, a circular slide rail 5, studs 6-9, and motor wires 10. The acrylic plate is marked with ArUco and fixed to one end of the circular slide rail; the other end is connected to a pulley. The pulley is connected to the stepper motor via a belt, and the stepper motor drives the electric turntable to rotate. The hardware system communicates with a host computer via an RS-232 communication module. The host computer sends rotation angle and speed information in the form of commands, and the electric turntable rotates according to the commands from the host computer. The process of using this hardware platform to collect data (see...) Figure 2 The method involves mounting a depth camera to the end effector of a robotic arm, then placing the target object onto a motorized turntable. After the turntable rotates to a fixed angle, the depth camera captures an RGB-D image. When the turntable rotates 360°, the depth camera captures multiple RGB-D images around the object. The position of the robotic arm's end effector is then changed, altering the depth camera's shooting angle, and the process is repeated. After multiple shots from different angles, RGB-D images of the object from various perspectives are obtained. Based on these images, the data annotation software algorithm provided in this invention is used to annotate the object's 3D model information, six-degree-of-freedom pose information, and image segmentation mask information.

[0038] Please refer to this again. Figure 3 The data annotation software algorithm provided in this embodiment of the invention is mainly used for automatically annotating the 3D model information, six-degree-of-freedom pose information, and image segmentation mask information of target objects. The specific steps of this method are:

[0039] S1. Point cloud alignment: By using ArUCo markers, each frame of point cloud is aligned with the first frame of point cloud.

[0040] S2, 3D reconstruction: stitching together the aligned point clouds to obtain the complete point cloud of the scene;

[0041] S3. Surface triangulation: The scene point cloud is segmented using a clustering segmentation algorithm to obtain the point cloud of each object in the scene. On this basis, the normal vector of the object point cloud is made consistent by the Riemann diagram normal propagation algorithm based on tree layering. Then, the object point cloud is triangulated by using the Poisson surface reconstruction algorithm to obtain the three-dimensional model of the object.

[0042] S4. Label the segmentation information and project the triangular faces of the object's 3D model onto the camera's imaging plane one by one to obtain the object's segmentation mask.

[0043] S5. Obtain the 3D model of the object and label its pose information. By calculating the orientation bounding box of the object model and its pose transformation matrix, move the object model to the origin of the 3D coordinate system and align it with the three coordinate axes. Use the pose transformation relationship between image frames to obtain the pose information of the object in the point cloud of each frame.

[0044] The following is a detailed description of each part:

[0045] Step 1: Point cloud alignment.

[0046] This invention first finds the 3D coordinates of the corner points of ArUco markers in each frame of the image using the following method. ArUco markers are binary square reference markers used for camera pose estimation. They are square markers consisting of a wide black border and an internal binary matrix that defines their identifier. This invention detects the 2D coordinates of the four corner points of the ArUco markers in the image sequence, and then maps the 2D coordinates to the depth map using the pixel correspondence between the RGB image and the depth map, thereby obtaining the 3D coordinates of the corner points. If b ArUco markers are placed, the set of corner points X can be obtained from the k-th frame image. k ={x i |i=1,2,…,4b}.

[0047] Suppose the set of point cloud sequences obtained from RGB-D image sequences is P = {P i For each point cloud in P (i = 1, 2, ..., n), a multi-directional registration algorithm can be used to align the point clouds. The process is as follows: First, register each pair of point clouds in P. Let P' be a tangent to the other point cloud in P'. i and P j Then, using the random sample consensus algorithm, the corresponding corner coordinate set X obtained from the ArUco marker is processed. i and X j By performing coarse registration, we can initially obtain P. i and P j A rough rigid body transformation matrix. Based on this, the ICP algorithm is further used to transform P...i and P j All 3D points are precisely registered to obtain the transformation matrix T. ij This makes T ij P i With P j Align. Then, place P i As the i-th node of the pose graph, the transformation matrix of the adjacent point clouds, {T ij |i=1,2,…,n-1;j=i+1}, as the odometry edges of the pose graph, will transform the non-adjacent point clouds into transformation matrices {T}. ij The edges |i=1,2,…,n-1;j≠i+1} are used as loop edges in the pose graph to construct the pose graph, and a robust optimization algorithm is used to optimize the pose graph. In the optimized pose graph, the transformation matrix of the k-th frame point cloud to the 1-th frame point cloud is extracted, thus forming the transformation matrix set T1={T 1k |k=1,2,…,n}, so the point cloud in the k-th frame passes through T 1k After transformation, it can be aligned with the point cloud of the first frame.

[0048] Step 2: 3D reconstruction.

[0049] Using the RANSAC algorithm, the set of three-dimensional corner coordinates X obtained from ArUco markers in the k-th frame RGB-D image is... k Perform plane fitting and filter out point cloud P. k Points belonging to the plane are selected, thus only object point clouds are retained in the scene point cloud. The resulting object point cloud sequence is denoted as P. s ={P sk |k=1,2,...,n}, according to T1 on P s Each frame of the point cloud is transformed to align with the first frame. Then, a voting algorithm is used to fuse the transformed point clouds, resulting in the complete point cloud M of the object. A moving least squares algorithm is used to resample the fused point cloud M, eliminating unsmooth or missing regions. The resampling algorithm can also be used to reconstruct missing surface parts by performing high-order polynomial interpolation on surrounding data points, thus solving the problem of "double-wall" pseudo-data caused by multiple scan points and making the surface of the reconstructed 3D object model smoother. The pseudocode for the 3D reconstruction of the object surface is shown below.

[0050]

[0051]

[0052] Step 3: Surface triangulation

[0053] Considering that multiple objects are usually placed in a scene, it is necessary to segment the objects contained in M ​​to obtain the point cloud set O = {oi |i=1,2,…,c}. This invention uses the Euclidean clustering point cloud segmentation algorithm.

[0054] To reconstruct the surface of an object, the point cloud O is triangulated to obtain a 3D model of the object. This invention uses the Poisson surface reconstruction algorithm. Poisson surface reconstruction is an implicit function method that combines the advantages of global and local methods. It adopts an implicit fitting approach, obtaining the implicit equation representing the surface information described by the point cloud model by solving the Poisson equation. Then, isosurface extraction is performed on this equation to obtain a surface model with geometric entity information. The Poisson surface reconstruction algorithm requires point cloud and normals, and the normals need to be consistent. The normals of the point cloud can be estimated using principal component analysis, which approximates the local shape of the surface based on the position information of the sampling points and their neighbors, thereby estimating the normal direction of the point cloud. However, the normal directions obtained by this method are often inconsistent, meaning that the normal direction of a sampling point may be opposite to that of its neighbors. This can lead to deformation and tearing of the reconstructed 3D surface. To address this, this invention proposes a normal propagation method based on minimum spanning trees to unify the object's normal vectors, thus providing accurate and consistent normal vectors for the surface reconstruction algorithm.

[0055] The specific steps of the normal propagation algorithm based on minimum spanning tree are:

[0056] (1) In the principal component analysis method, a larger neighborhood radius is used to estimate the normal vector of the point cloud to obtain a rough normal vector.

[0057] (2) Select the point with the smallest curvature as the root node of the minimum spanning tree and mark it.

[0058] (3) Calculate the distance from all unmarked points to the tree, and find the point p that is closest to the tree. i and distance p i The most recent tree node.

[0059] (4) p i It is added to the tree as a new node, connected to the nearest tree node, a new edge is generated, and p is labeled. i .

[0060] (5) Repeat steps 3 and 4 until all points in the point cloud are added to the minimum spanning tree.

[0061] (6) Traverse all edges of the minimum spanning tree and calculate the angle between the normal vectors of the nodes connected at both ends. If the angle is greater than 90°, reverse the normal vector and add it to the node to obtain a coarse normal vector with a consistent direction.

[0062] (7) In the principal component analysis method, a smaller neighborhood radius is used to estimate the normal vector of the point cloud to obtain an accurate normal vector.

[0063] (8) Calculate the angle between the precise normal vector and the coarse normal vector at each point. If the angle is greater than 90°, reverse the precise normal vector to obtain a precise normal vector with a consistent direction.

[0064] Based on this, the Poisson surface reconstruction algorithm is used to reconstruct the object point cloud set O = {o i The point cloud o in |i=1,2,…,c} i By performing surface reconstruction, the three-dimensional models m of each object can be obtained. i .

[0065] Step 4: Label segmentation information

[0066] The 3D model of an object consists of multiple triangular faces. Given the camera's intrinsic parameters, these triangular faces can be projected onto the camera plane to obtain a segmentation mask for the object. Since the RGB image and depth map have been aligned, they have the same camera intrinsic parameters. According to the camera pinhole imaging model, the vertices of the triangular faces can be projected onto the 2D image using equation (1).

[0067]

[0068] In the formula f x f y c x c y Here, X, Y, and Z are the camera's intrinsic parameters, representing 3D coordinates, while u and v are the 2D coordinates projected onto the image from these 3D coordinates. By projecting the three vertices of a triangular face onto the image coordinate system and then filling the triangle formed by the three projection points, the projection of the triangular face onto the image plane can be obtained. Projecting all the triangular faces in the 3D model of the object yields the segmentation mask of the object in the image.

[0069] Step 5: Obtain the 3D model of the object and label its pose information.

[0070] Since the transformation matrix T1 = {T} between each frame point cloud and the first frame point cloud has been obtained, 1k |k=1,2,…,n}, and based on this, each frame of point cloud is aligned with the first frame of point cloud, thus reconstructing the 3D model m of the object. i It will align with the objects in the first frame of the point cloud. (M) i Calculate the orientation bounding box and pose transformation matrix of the object model. Thus through T mi -1Transform the pose of the object's 3D model, moving it to the origin of the 3D coordinate system and aligning it with the three coordinate axes. The transformed 3D model of the object is saved as the final annotation result.

[0071] Furthermore, the pose transformation matrix T between the point cloud of the k-th frame and the point cloud of the first frame is... 1k The object model m in the k-th frame can be calculated. i pose annotation information for:

[0072]

[0073] In summary, the process of using the automated dataset acquisition platform is as follows: First, the object is placed on a motorized turntable, and several ArUco markers are placed around the object. Then, the turntable begins to rotate, and an RGBD image is captured using a depth camera at each rotation angle, acquiring a sequence of RGBD images of the scene. Software algorithms are then used to perform 3D reconstruction of the objects in the scene based on the RGBD images, and their segmentation mask information and pose information are labeled.

[0074] Compared with existing technologies, the advantages of this invention are as follows: It can automatically obtain the target's 3D model information, six-DOF pose information, and segmentation mask information simply by using an RGB-D camera to capture image sequences of the target placed on a data acquisition hardware platform, without requiring a 3D scanner or human intervention in the shooting process. In generating the object's 3D model, this invention uses a normal propagation method based on a tree-structured layered Riemann diagram for normal orientation, ensuring the consistency of the normal directions of the object's surface point cloud. This avoids deformation and tearing of the 3D model due to inconsistent normal directions, thus obtaining a more accurate 3D model. Based on this, perspective projection is performed using the triangulated 3D model to obtain more precise image segmentation mask information.

[0075] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An automatic data acquisition system for a six-DOF pose estimation dataset, comprising a data acquisition platform and data processing equipment, characterized in that, The data acquisition platform is used to capture RGBD image sequences of the target scene using a depth camera; The data processing device is equipped with a data annotation software algorithm, which is used to process the RGBD image sequence of the target scene, perform three-dimensional reconstruction of the target object, obtain a three-dimensional model, and automatically annotate the segmentation mask information and six-degree-of-freedom pose information of the object in the target scene based on the three-dimensional model. The data processing device uses data annotation software algorithms to process the RGBD image sequence of the target scene, specifically including: The first step, point cloud alignment: Find the three-dimensional coordinates of the four corner points of the m ArUco markers in the k-th frame image from the RGBD image sequence, denoted as X. k = {xki|i = 1, 2, ..., 4m}, then determine the X between frames. k The correspondence; based on the X of each frame image k Based on the correspondence between the point cloud and other frame images, the transformation moment T between the point cloud of the k-th frame and the point cloud of the first frame is calculated using a global point cloud registration algorithm. 1k If there are n frames in the image sequence, then the point cloud transformation set T1 = {T 1k |k=1,2,…,n}, that is, the point cloud of the kth frame passes through T 1k The transformed point cloud is aligned with the first frame's point cloud. The second step is 3D reconstruction: using the random sample consensus algorithm to reconstruct X. k Perform plane fitting and filter out points in the point cloud that are near the plane in each frame; based on T in set T1 1k The position and pose of the point cloud in frame k are transformed; the transformed point cloud is stitched together in 3D to obtain the complete point cloud of the object, and then smoothed to remove noise, resulting in the reconstructed point cloud M; the reconstructed point cloud M is segmented using the Euclidean clustering algorithm to obtain the point clouds O = {o} of c objects in the scene. i |i=1,2,…,c}; The third step is surface triangulation: The normal vectors of the point clouds for each object are calculated, and the normal vectors are standardized using a tree-based hierarchical Riemann diagram-based normal propagation method. The Poisson surface reconstruction algorithm is then used to triangulate O, resulting in the 3D model m of each object. i , i = 1, 2, ..., c; Step 4, segmentation information annotation: Annotate the 3D model of the object m i The triangular faces in i = 1, 2, ..., c are projected onto the camera plane one by one to obtain the object segmentation mask sequence; Step 5, Pose Information Annotation: Calculate the 3D model m of each object. i The orientation bounding box and the pose transformation matrix T of the bounding box mi Based on the pose transformation relationship T between the first frame point cloud and the kth frame point cloud 1k Calculate the object model m in the point cloud of the kth frame. i The six degrees of freedom pose information.

2. The automatic data acquisition system as described in claim 1, characterized in that, The data acquisition platform includes an electric turntable and a robotic arm; the target object is placed on the electric turntable, and the electric turntable receives control commands from the host computer and rotates according to the control commands; the depth camera is installed at the end of the robotic arm, and the depth camera can capture RGD images of the target object from various angles to obtain the RGBD image sequence.

3. The automatic data acquisition system as described in claim 2, characterized in that, The electric turntable receives control commands from the host computer via an RS-232 communication module; the control commands include the rotation angle and rotation speed of the electric turntable.

4. The automatic data acquisition system as described in claim 2, characterized in that, The electric turntable includes a stepper motor, a belt, an acrylic plate, a base plate, and a circular slide rail. The stepper motor is mounted on the base plate, the belt is mounted above the circular slide rail, the circular slide rail is mounted above the base plate by two studs, and the acrylic plate is mounted above the belt by two studs. The acrylic plate is marked with ArUoc.

5. The automatic data acquisition system as described in claim 1, characterized in that, In the surface triangulation step, the normal vectors of each object's point cloud are unified using a normal propagation method based on a tree-structured hierarchical Riemann diagram. Specifically: (1) In the principal component analysis method, a larger neighborhood radius is used to estimate the normal vector of the point cloud to obtain a rough normal vector; (2) Select the point with the smallest curvature as the root node of the minimum spanning tree and mark it; (3) Calculate the distance from all unmarked points to the tree, and find the point p that is closest to the tree. i and distance p i The nearest tree node; (4) p i It is added to the tree as a new node, connected to the nearest tree node, a new edge is generated, and p is labeled. i ; (5) Repeat steps 3 and 4 until all points in the point cloud are added to the minimum spanning tree; (6) Traverse all edges of the minimum spanning tree and calculate the angle between the normal vectors of the nodes connected at both ends; If the included angle is greater than 90°, the node's normal vector is added after reversal, thus obtaining a coarse normal vector with a consistent direction; (7) In the principal component analysis method, a smaller neighborhood radius is used to estimate the normal vector of the point cloud to obtain an accurate normal vector; (8) Calculate the angle between the precise normal vector and the coarse normal vector at each point. If the angle is greater than 90°, reverse the precise normal vector to obtain a precise normal vector with a consistent direction.

Images