Key point screening method for object pose estimation
Through edge detection and projection relationship calculation, key points in areas with rich shape characteristics are selected, which solves the problem that the key point positioning accuracy affects pose estimation, and realizes high-precision pose estimation of texture-rich and weak texture objects.
Patent Information
- Application Number
- CN202411149429.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-08
AI Technical Summary
In the prior art, in object position estimation, the position error and number of key points affect the position estimation accuracy. Especially in the case of weak texture objects, it is difficult to obtain high-precision position estimation.
The edges of the object image are extracted through the edge detection algorithm, the surface coordinates of the object model are calculated, the number of projections of points on the image are counted, the distance is corrected, and the key points located in areas with rich shape characteristics are filtered out to improve the positioning accuracy of the key points.
It significantly improves the positioning accuracy of key points and the positioning estimation accuracy of object positions, and is suitable for objects with rich textures and weak textures, ensuring the accuracy of positioning estimation.
Smart Images

Figure CN118918190A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of image processing and computer vision, and in particular to a key point screening method for object pose estimation. Background Art
[0002] Estimation of the 6D pose of a rigid body is a classic problem in computer vision, which involves predicting the 3D translation and 3D rotation of an object. As a basic task, pose estimation is widely used in various application areas, such as augmented reality, autonomous driving, and robotic manipulation. Early methods mainly used depth data to achieve accurate pose estimation, while the performance of RGB image-based methods is usually lower than that of depth-based methods. However, due to the advantages of RGB image-based methods in terms of cost, running time, and complexity, it is currently an important research direction in pose estimation.
[0003] Traditional monocular methods use local image features to estimate pose. First, a 2D-3D correspondence is established between the image and the object model, such as key points and their projections. Then, pose estimation is solved by using the perspective-n-point (PnP) algorithm and the random sample consensus algorithm (RANSAC). This method can provide effective and accurate pose estimation even in the case of severe occlusion. With the development of deep learning and convolutional neural network (CNN), people use CNN to predict key point poses, which has achieved better results in pose estimation of weak texture objects compared with traditional methods. However, most of the literature at home and abroad mainly studies the network structure, and few people study the relationship between the key point position and the key point positioning accuracy, as well as the influence of the key point position on the pose estimation accuracy. In order to obtain a more accurate angle valve pose, it is necessary to shoot the angle valve from two different perspectives and use the least squares method to fit the optimal solution. Therefore, the position error and number of key points on the image will affect the accuracy of pose estimation. Summary of the invention
[0004] Purpose of the invention: The present invention provides a key point screening method for object pose estimation to improve the positioning accuracy of key points.
[0005] Technical solution: The present invention provides a key point screening method for object pose estimation, comprising the following steps:
[0006] S1: collect original images containing object images;
[0007] S2: extract pixels of the object image in the original image using the mask of the object, and detect the edge of the object image in the pixel area of the object image using the edge detection algorithm;
[0008] S3: Calculate the surface coordinates of the object model corresponding to the pixels of the object image on the original image using the projection relationship;
[0009] S4: Count the number of times that points on the surface of the object model are projected onto the object image to form edges;
[0010] S5: Use the obtained number of times to correct the distance in the algorithm used by the farthest point, and select key points on the object model.
[0011] Preferably, the pose and mask of the object in each image are annotated in S1, a pose estimation data set is constructed, and the image data set is divided into a training set, a test set and a validation set. The camera pose is then uniformly sampled, and the image with the closest pose is selected in the training data set so that the viewing angles corresponding to the screened images are evenly distributed around the object, thereby obtaining an image set T.
[0012] Preferably, the mask in step 2 is obtained by manual annotation, object posture or projection relationship.
[0013] Preferably, the step S3 further comprises the following steps:
[0014] S31: The object model has multiple vertices, which constitute multiple triangular facets. According to the object posture and camera parameters, each triangular facet of the object model is projected onto the object image extracted in S2;
[0015] S32: Calculate the coordinates of the object pixel in the corresponding triangle patch.
[0016] Preferably, the projection in step S31 is to project the three vertices of the triangular face onto the image, and the three projection points are connected to each other to form a new triangle on the image.
[0017] Preferably, in step S32, for the object pixel p, the triangular facet projections containing the pixel p are counted to obtain a triangular facet set G; the coordinates corresponding to the pixel p on each triangular facet are calculated according to the projection relationship, and the coordinates closest to the camera are taken as the surface coordinates of the object model corresponding to the pixel p.
[0018] Preferably, in step S4, the surface coordinates of the object model corresponding to the edges of the object pixel area on each picture are first counted to obtain a surface coordinate set C corresponding to the edges, and then the point cloud model is sampled by the object triangular patch model. Finally, for each point on the point cloud model, the number Num of coordinates of additional points in the set C is counted as the number of points on the surface of the object model projected onto the image as edges, and the additional points are points with a pixel distance less than 10.
[0019] Preferably, the key points in S5 include corner points, curve intersections, vertices of bounding boxes or artificially set feature points in the original image and the object image.
[0020] Preferably, the distance correction formula in step S5 is as follows:
[0021]
[0022] Where D is the distance in the original FPS algorithm, D new is the corrected distance, Num is the number of points on the object point cloud projected into edges on all images, and Num max It is the maximum value of Num corresponding to all points in the point cloud.
[0023] Beneficial effects: (1) The present invention takes into account the shape characteristics of the object model, and based on the object image and model, corrects the distance in the FPS algorithm so that the screened key points are located in the feature-rich area of the model, while the projected key points also have more obvious textures near them, making them easier to be recognized and located by the neural network, thereby greatly improving the positioning accuracy of the key points.
[0024] (2) The present invention designs a key point screening algorithm that takes into account the shape characteristics and texture characteristics of the object. It is not only suitable for objects with rich textures, but also can effectively ensure the positioning accuracy of key points and the object pose estimation accuracy for objects with weak textures. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 It is a schematic diagram of the process of the present invention;
[0026] Figure 2 It is an overall flow chart of an embodiment of the present invention. DETAILED DESCRIPTION
[0027] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
[0028] Example
[0029] This embodiment provides a key point screening method for object pose estimation, which is described by taking a monkey toy in the Linemod dataset as an example. Figure 1 As shown, the following steps are included:
[0030] S1: First, collect a large number of images containing objects, annotate the pose and mask of the objects in each image, construct a pose estimation dataset, and then divide the image dataset into training set, test set and validation set. Then, uniformly sample the camera pose and select the image with the closest pose in the training dataset so that the viewing angles corresponding to the screened images are evenly distributed around the object, and obtain the image set T.
[0031] S2: For each image in the image set T, extract the object pixels using the mask of the object, and detect the edge in the object pixel area using the edge detection algorithm. In this embodiment, the Canny detector is used to detect the edge of the object pixel area. The selected algorithm is only used to illustrate this example and is not considered to be a limitation of the present invention.
[0032] S3: Calculate the surface coordinates of the object model corresponding to the object pixels on the image using the projection relationship, specifically including:
[0033] S31: Project each triangular face of the object model onto the image according to the object pose and camera parameters. The projection of the triangular face is to project the three vertices of the triangular face onto the image, and the three projection points are connected to each other to form a new triangle on the image.
[0034] S32: Calculate the coordinates of the object pixel in the corresponding triangle patch.
[0035] Specifically, in this step, for the object pixel p, the projections of the triangles containing the pixel p are counted to obtain the triangle set G. The triangle closest to the camera will be imaged on the image, while the triangles farther from the camera cannot be imaged on the image due to occlusion. Therefore, the coordinates corresponding to the pixel p on each triangle are calculated based on the projection relationship, and the coordinates closest to the camera are used as the surface coordinates of the object model corresponding to the pixel p.
[0036] S4: Count the number of times that points on the surface of the object model are projected onto the image to become edges.
[0037] Specifically, in this step, firstly, the surface coordinates of the object model corresponding to the edge of the object pixel area on each image are counted to obtain a surface coordinate set C corresponding to the edge, and then the point cloud model is sampled by the object triangular patch model. Finally, for each point on the point cloud model, the number Num of coordinates located near the point in the set C is counted.
[0038] S5: Use the obtained number to correct the distance in the farthest point adoption algorithm and select key points on the object model.
[0039] Specifically, in this step, the distance correction formula is as follows:
[0040]
[0041] Where D is the distance in the original FPS algorithm, D new is the corrected distance, Num is the number of points on the object point cloud projected into edges on all images, and Num max It is the maximum value of Num corresponding to all points in the point cloud.
[0042] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.
Claims
1. A key point screening method for object pose estimation, characterized in that: The following steps are involved: S1: collect original images containing object images; S2: extract pixels of the object image in the original image using the mask of the object, and detect the edge of the object image in the pixel area of the object image using the edge detection algorithm; S3: Calculate the surface coordinates of the object model corresponding to the pixels of the object image on the original image using the projection relationship; S4: Count the number of times that points on the surface of the object model are projected onto the object image to form edges; S5: Use the obtained number of times to correct the distance in the farthest point adoption algorithm and select key points on the object model.
2. The key point screening method for object pose estimation according to claim 1, characterized in that: In S1, the pose and mask of the object in each image are annotated, a pose estimation dataset is constructed, and the image dataset is divided into a training set, a test set, and a validation set. The camera pose is then evenly sampled, and the image with the closest pose is selected in the training dataset so that the viewing angles corresponding to the screened images are evenly distributed around the object, thereby obtaining an image set T.
3. The key point screening method for object pose estimation according to claim 2, characterized in that: The mask in step 2 is obtained by manual annotation, object pose or projection relationship.
4. The key point screening method for object pose estimation according to claim 1, characterized in that: The step S3 further comprises the following steps: S31: The object model has multiple vertices, which constitute multiple triangular facets. According to the object posture and camera parameters, each triangular facet of the object model is projected onto the object image extracted in S2; S32: Calculate the coordinates of the object pixel in the corresponding triangle patch.
5. The key point screening method for object pose estimation according to claim 4, characterized in that: The projection in step S31 is to project the three vertices of the triangular face onto the image, and the three projection points are connected to each other to form a new triangle on the image.
6. The key point screening method for object pose estimation according to claim 4, characterized in that: In step S32, for the object pixel p, the triangular facet projections containing the pixel p are counted to obtain a triangular facet set G; the coordinates corresponding to the pixel p on each triangular facet are calculated according to the projection relationship, and the coordinates closest to the camera are used as the surface coordinates of the object model corresponding to the pixel p.
7. The key point screening method for object pose estimation according to claim 1, characterized in that: In the step S4, the surface coordinates of the object model corresponding to the edges of the object pixel areas on each object image extracted in S2 are first counted to obtain a surface coordinate set C corresponding to the edges, and then the point cloud model is sampled from the object triangular patch model. Finally, for each point on the point cloud model, the number Num of coordinates of additional points in the set C is counted as the number of points on the surface of the object model projected onto the image as edges, and the additional points are points with a pixel distance less than 10.
8. The key point screening method for object pose estimation according to claim 1, characterized in that: The key points in S5 include corner points, curve intersections, vertices of bounding boxes, or artificially set feature points in the original image and the object image.
9. The key point screening method for object pose estimation according to claim 7, characterized in that: The distance correction formula in step S5 is as follows: Where D is the distance in the original FPS algorithm, D new is the corrected distance, Num is the number of points on the object point cloud projected into edges on all images, and Num max It is the maximum value of Num corresponding to all points in the point cloud.
10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.
Citation Information
Patent Citations
Augmented reality unmarked tracking registration method and device based on edge model
CN110689573A
Shielding target 6D pose recognition system and method based on spherical amphibious robot
CN116229275A
Three-dimensional object recognition system
JP2007249592A
Machine learning for pose estimation of robotic systems
US20240221335A1
