A more accurate three-dimensional pose estimation algorithm capable of coping with complex backgrounds

Abstract

The present application belongs to the field of machine vision, and relates to an object detection and pose estimation method. The method is based on the adjustment of YOLO-6D algorithm, the YOLO V2 detection network in the original algorithm is changed into YOLO V3 network, and an attention mechanism is added to enhance the detection ability of the model to the object with complex background and occlusion. And the pose estimation method is adjusted, the cell group is selected for RANSAC-based EPnP pose estimation to improve the estimation accuracy. The algorithm well overcomes the problems of weak anti-background interference ability and poor recognition accuracy of the occluded target of the traditional pose estimation algorithm, and the algorithm runs fast and has real-time processing capability, and the comprehensive performance exceeds other CNN-based algorithms.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision and relates to an object detection and posture estimation method. Background Art

[0002] Autonomous driving and robotics require the identification and positioning of targets, as well as the determination of their rotation angles in three-dimensional space. Traditional two-dimensional image recognition cannot obtain this information, but 6D pose estimation can effectively address this problem. These scenarios place extremely high demands on the speed and accuracy of pose recognition. Furthermore, practical applications often face various interference conditions, such as low light intensity, complex backgrounds, and occlusions, which complicate pose recognition.

[0003] Point-pair feature-based pose estimation methods are highly robust and resistant to interference when dealing with slight deformations and occlusions. However, they cannot effectively extract the target pose in texture-deficient pose estimation scenes and have high parameter space dimensionality, spatial complexity, and time complexity. Point cloud-based methods can retain the original geometric information and do not require discretization, but they waste a lot of computing resources.

[0004] For 6D pose estimation, existing methods require a long detection time and more computing resources. Summary of the Invention

[0005] To address the challenges of existing technologies, this paper proposes a method for object detection and pose estimation. This method replaces the YOLOV2 detection network in the original YOLO-6D algorithm with a YOLOV3 network and adds an attention mechanism to enhance the model's ability to detect objects in complex backgrounds and with occlusions. Furthermore, the pose estimation method is adjusted, selecting a cell cluster for RANSAC-based EPnP pose estimation to improve estimation accuracy.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] A method for object detection and pose estimation that can handle complex backgrounds and is more accurate includes the following steps:

[0008] (1) Collect several images containing the detection target;

[0009] (2) Create a training dataset based on the LineMod dataset format;

[0010] (3) Improve the YOLO-6D algorithm to enhance the performance of its detection network. The improvements include the following two parts:

[0011] The YOLOV2 detection network in the original YOLO-6D algorithm is replaced with the YOLOV3 network, and the attention module is integrated as the detection network. The network input is a single RGB image, which is evenly divided into S×S regular cells. After the detection network, each cell can output a multidimensional vector, which stores the coordinate information of the 8 3D envelope corner points and the object midline point, the classification probability of the target object and the overall confidence. The network composition is as follows Figure 1 As shown, it contains three prediction feature layers, Box1, Box2, and Box3, and incorporates an attention mechanism into the feature layer Box1, allowing it to spatially fuse more features. This application uses the SENet network as the integrated attention mechanism. The SENet network obtains the importance of each feature map by learning feature weights, increases the weight of valid feature maps, and reduces the weight of invalid feature maps, making the model more optimized. After passing through the SENet network, the image is divided into two branches. One branch is convolved to the first prediction feature layer of 13×13 and predicted using a 1×1 convolution kernel predictor. The other branch will go to another convolution layer and then be upsampled. The height and width will become twice the original, that is, 26×26, and will be fused with the residual network output in Box2 in the network. At the same time, the spliced ​​matrix is ​​also processed by the feature extractor and divided into two branches. One branch goes to the second prediction feature layer, and the other branch goes to the 1×1 convolution layer for upsampling. The height and width are both reduced to 52, and then fused with the output of the residual network in Box3. The fusion result passes through the feature extractor and is then convolved to obtain the third prediction feature layer. The purpose of this operation is to extract the local features of the target object. Because it has three prediction feature layers of different scales, it can effectively enhance the prediction ability of occluded objects.

[0012] After the above detection network, all cells will have a confidence level. The cell with the highest confidence level (red area) is selected, and a 3×3 square cell group (yellow area) is selected based on this cell. Figure 2 As shown, there are three possible locations for the cell with the highest confidence, and three corresponding cell cluster selection methods. Ransac-based EpnP pose estimation is performed on the nine selected cells. RANSAC uses a voting mechanism to find the optimal fit. Compared to the original position estimation method, this step has a slight increase in computational complexity, but it can improve detection accuracy.

[0013] (4) Using the data set obtained in step (2), the improved algorithm in step (3) is used for training to obtain a target detection model;

[0014] When training the target detection model, the confidence function c(x) is used to evaluate the deviation between the predicted pose and the true pose of the target object:

[0015] In 3D space, the confidence level of the target object must also be analyzed. In 2D prediction, the object is screened based on its confidence score and Intersection over Union (IOU). However, calculating IOU in 3D space is more difficult. This application uses the confidence function c(x) based on Euclidean distance, as shown in the following formula, to evaluate the deviation between the predicted and true pose of the target object.

[0016]

[0017] Among them, D T (x) represents the Euclidean distance, d th represents the pixel threshold of the confidence function, and α represents the sharpness of the c(x) function.

[0018] (5) Use the trained target detection model to detect the target to obtain the classification information and 6D pose information of the target object. During the detection process, coordinate offset is required to ensure that the target is located within the cell grid after the image is divided.

[0019] Furthermore, the data set described in step (2) includes: a plurality of images containing the target to be detected, an added noise image, an image mask, and annotation information corresponding to each image, wherein the annotation information includes: whether the image contains the detection target, classification information, and spatial information of the target to be detected;

[0020] The beneficial effects of the present invention are as follows: the present invention performs two optimizations based on the YOLO-6D algorithm: (1) the YOLOV2 detection network in the original algorithm is changed to the YOLOV3 detection network, and an attention mechanism is added to enhance the detection capability of objects with complex backgrounds and occlusions; (2) a cell group is selected for RANSAC-based EPnP pose estimation, thereby improving the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is the network structure diagram of the improved YOLO-6D.

[0022] Figure 2 This is an improved pose estimation method; in the figure (a), the center point of the row is located at the edge of the cell group, in the figure (b), the center point of the row is located in the cell group, and in the figure (c), the center point of the row is located at the corner of the cell group.

[0023] Figure 3 It is a system flow chart of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0025] This paper proposes an improved object pose estimation model based on the YOLO-6D algorithm. The original algorithm's YOLOV2 detection network is replaced with a YOLOV3 network, and an attention mechanism is added to enhance the model's ability to detect objects with complex backgrounds and occlusions. Furthermore, the pose estimation method is adjusted, and RANSAC-based EPnP pose estimation is performed on selected cell clusters to improve estimation accuracy.

[0026] The algorithm network structure is as follows Figure 1 , mainly includes the following steps:

[0027] (1) Use the camera to take pictures of the target to be detected. The shooting environment is mainly divided into two situations: no object occlusion and object occlusion. Several images taken are used for the final algorithm detection. The targets to be detected mainly include three objects: milk carton, glasses case and ceramic cup.

[0028] (2) The LineMod dataset is selected as the dataset required for training; the dataset includes: 13 objects with less texture including the target to be detected, a total of 15,783 images, and about 1,200 images for each object, and noise images, image masks and annotation information corresponding to each image are added to it, wherein the annotation information includes: whether the image contains the detection target, classification information and spatial information of the target to be detected. (3) The YOLO V2 detection network in the original YOLO-6D algorithm is changed to the YOLO V3 network, which includes three prediction feature layers Box1, Box2 and Box3. The SENet network is integrated into the feature layer Box1 as an attention mechanism. The SENet network learns feature weights through loss, obtains the importance of each feature map, increases the weight of effective feature maps, and reduces the weight of invalid feature maps, making the model a better attention mechanism that can integrate more features in space.

[0029] Select a cell group for RANSAC-based EPnP pose estimation to improve estimation accuracy. The image is evenly divided into S×S (13×13) regular cells. When predicting the 8 3D envelope corner points and object center key points, the centroid point needs to be restricted to the cell. In this process, other corner points are not considered. The confidence of each cell is calculated, and the cell with the highest confidence (red area) is selected. A 3×3 square cell group (yellow area) is selected based on this cell, as shown in the figure. Figure 2As shown in the figure, eight key points are predicted for each cell in the cell cluster, and the RANSAC-based EPnP pose estimation is performed on the estimated results of all cells in the cell cluster. RANSAC uses a voting mechanism to find the optimal fitting result. Compared with the original pose estimation method, this step will have a certain degree of increased computational complexity, but it can improve the detection accuracy.

[0030] (4) Using the data set obtained in step (2), the improved algorithm of step (3) is trained to obtain a target detection model.

[0031] When training the target detection model, the confidence function c(x) is used to evaluate the deviation between the predicted pose and the true pose of the target object:

[0032] In three-dimensional space, the confidence of the target object must also be analyzed. In two-dimensional prediction, the target object is screened based on its confidence score and IOU (Intersection over Union). However, calculating IOU in three-dimensional space is more difficult. This paper uses the confidence function c(x) based on the Euclidean distance to evaluate the deviation between the predicted pose and the actual pose of the target object.

[0033]

[0034] (5) The target detection model trained in step (4) is used to detect the target image to be detected collected in step (1). During the detection process, the image coordinates are offset to ensure that the target is located in the cell grid after the image is divided. Finally, the classification information and 6D posture information of the target object are obtained in sequence.

[0035] The results of the example show that the detection accuracy of the algorithm for milk cartons, glasses cases and ceramic cups is 78.6%, 80.7% and 73.1% respectively when there are no objects blocking them. When there are objects blocking them, the detection accuracy is 71.3%, 72.9% and 65.4% respectively. From the analysis of the 2D projection error index, at a distance threshold of 30 pixels, the posture estimation effects of 7 target categories before and after the network structure change were compared. The 2D projection accuracy of the algorithm of this application can reach 69.87%, and the network structure has been improved by 6.77% compared with the previous one. It is effectively verified that the present invention has further improved the target detection accuracy on the basis of existing detection.

[0036] The above-described embodiments merely express the implementation methods of the present invention, but should not be understood as limiting the scope of the patent of the present invention. It should be pointed out that for those skilled in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. A more accurate 3D pose estimation algorithm that can handle complex backgrounds, characterized by: The following steps are involved: (1) Collect several images containing the detection target; (2) Create a training dataset based on the LineMod dataset format; (3) Improve the YOLO-6D algorithm. The improvements include the following two parts: The YOLOV2 detection network in the original YOLO-6D algorithm was replaced with a YOLOV3 network, and an attention module was integrated as the detection network. The network input is a single RGB image, which is evenly divided into S×S regular cells. After passing through the detection network, each cell can output a multidimensional vector, storing the coordinates of the eight 3D envelope corner points and the object midline point, the classification probability of the target object, and the overall confidence. The network contains three prediction feature layers, Box1, Box2, and Box3, and an attention mechanism is integrated into the feature layer Box1, allowing it to integrate more features spatially. After the detection network, all cells are assigned a confidence score. The cell with the highest confidence score is selected, and a square cell cluster is selected based on this cell. The cell with the highest confidence score can appear in three different locations, and there are three corresponding cell cluster selection methods. Ransac-based EpnP pose estimation is performed on the nine selected cells. RANSAC uses a voting mechanism to find the optimal fit result. (4) Using the data set obtained in step (2), the improved algorithm in step (3) is trained to obtain a target detection model. During the training of the target detection model, the confidence function c(x) is used to evaluate the deviation between the predicted pose and the true pose of the target object: (5) Use the trained target detection model to detect the target to obtain the classification information and 6D posture information of the target object; coordinate offset is required during the detection process to ensure that the target is located in the cell grid after the image is divided; In the step (3), the SENet network is selected as the integrated attention mechanism. The SENet network obtains the importance of each feature map by learning feature weights, which can increase the weight of effective feature maps and reduce the weight of invalid feature maps. The image after the SENet network will be divided into two branches, one of which is convolved to the first prediction feature layer of 13×13 and predicted using a 1×1 convolution kernel predictor. The other branch will go to another convolution layer and then be upsampled. The height and width will become twice the original, that is, 26×26, and will be fused with the residual network output in Box2 in the network. At the same time, the spliced ​​matrix is ​​also processed by the feature extractor and divided into two branches, one of which goes to the second prediction feature layer and the other goes to the 1×1 convolution layer for upsampling and then fused with the output of the residual network in Box3. The fusion result passes through the feature extractor and is then convolved to obtain the third prediction feature layer.

2. A more accurate three-dimensional pose estimation algorithm capable of coping with complex backgrounds according to claim 1, characterized in that: In the step (4), the deviation between the predicted pose and the true pose of the target object is evaluated based on the Euclidean distance using the confidence function c(x) shown in the following formula; ； Among them, D T (x) represents the Euclidean distance, d th represents the pixel threshold of the confidence function, and α represents the sharpness of the c(x) function.

3. A more accurate three-dimensional pose estimation algorithm capable of coping with complex backgrounds according to claim 1, characterized in that: The data set described in step (2) includes: a number of images containing the target to be detected, an added noise image, an image mask, and annotation information corresponding to each image, wherein the annotation information includes: whether the image contains the detection target, classification information, and spatial information of the target to be detected.

Images