Pose measurement algorithm based on statistical attention

Abstract

The invention relates to a pose measurement technology crossed in the field of computers and machinery, in particular to a pose measurement algorithm based on statistical attention. According to the method, through combination of a 3D graph convolution (3D-GC) architecture and a statistical attention mechanism, a long-distance dependency relationship between object features is effectively captured, subtle differences between different objects are identified by using high-order statistical information, and finally a more accurate object pose result is obtained.

Description

Technical Field

[0001] The present invention relates to a pose measurement technology at the intersection of computer and mechanical fields, and in particular to a pose measurement algorithm based on statistical attention. The method effectively captures the long-distance dependencies between object features by combining a 3D graph convolution (3D-GC) architecture and a statistical attention mechanism, and uses high-order statistical information to identify subtle differences between different objects, thereby obtaining more accurate object pose results. Background Art

[0002] Pose measurement is one of the most important technologies in the field of computer and mechanics. Accurately measuring the position and posture of an object is crucial for many applications, such as virtual reality, scene understanding, and robotic manipulation. Traditionally, pose measurement algorithms rely on hand-crafted features to establish the correspondence between image templates and images, which has difficulties when dealing with textureless objects and runs slowly. With the rapid development of deep learning methods, pose measurement methods based on deep learning have emerged. These methods can be roughly divided into two main types: instance-level and category-level methods.

[0003] Some instance-level 6D object pose measurement methods have high estimation accuracy for a given object instance. However, these methods need to be trained on specific instance data, require CAD models of the objects, and can only estimate a limited number of objects with known shapes and textures. In contrast, category-level 6D object pose measurement methods can be generalized to unseen objects without the need for CAD models, providing greater flexibility. Category-level methods can be divided into two types: shape prior-based methods and non-shape prior-based methods. Prior-based estimation methods first extract the shape priors of objects in an offline mode, then reconstruct their normalized object coordinate space (NOCS) shapes, and finally use the Umeyama algorithm to calculate the pose. Although this normalized object coordinate space (NOCS) shape alignment method is able to recover the pose of the object, the alignment process is non-differentiable, which can lead to significant shape errors, which in turn affects the accuracy of pose measurement.

[0004] On the other hand, driven by the development of 3D graph convolution (3D-GC), class-level object pose measurement methods based on 3D graph convolution have achieved remarkable breakthroughs. Among these methods, FS-Net designs a direction-aware 3D graph convolutional autoencoder, which learns latent features by reconstructing observations of different objects to estimate their poses. GPV-Pose uses local geometric relationship constraints to enhance the learning of intra-class object shape features by introducing three new branches on the 3D graph convolutional encoder and demonstrates excellent performance. On this basis, HS-Pose proposes a mixed-range latent feature extraction layer that can perceive both local and global geometric relationships. This layer improves the perception of translation and scale, making it more suitable for processing more complex geometric shapes. However, the inability of these methods to capture long-range dependencies limits their ability to further model geometric relationships. Long-range dependencies are crucial for modeling complex and occluded object shapes. In addition, it is essential to perceive the subtle differences between different objects, as these details are crucial for distinguishing objects with similar shapes. Some existing methods use self-attention mechanisms or Transformer encoder-decoder structures to compensate for the missing long-distance dependencies. However, these methods only focus on the first-order information of features and fail to extract more complex information. In addition, they do not pay enough attention to the detailed differences between different objects.

[0005] To overcome the limitations of previous methods, we introduce statistical attention, which exploits high-order statistics to model geometric relationships. This approach emphasizes the detailed differences between objects while capturing long-range dependencies. To verify the effectiveness of this attention mechanism in the pose measurement task, we develop a new category-level object pose measurement network, called Statistical Attention Pose Measurement Network (SAPENet). Specifically, the structure of this network is built on the basis of the 3D graph convolution (3D-GC) architecture. In the encoder part, the HS layer replaces the 3D-GC to extract features, and statistical attention is added to calculate high-order statistics. The network consists of three submodules: pose regression, point cloud reconstruction, and bounding box voting. In addition, the pose regression module integrates statistical attention to strengthen feature modeling by exploiting high-order statistics. Summary of the invention

[0006] In order to solve a series of problems existing in the visual measurement of large cabin pose, such as limited field of view, the influence of lighting conditions on measurement results, internal and external parameter calibration, and large computational load, a large cabin pose measurement algorithm based on deep learning solves the problem that the existing technology cannot effectively capture long-distance dependencies when dealing with complex and occluded objects, and ignores the difference in details when distinguishing objects with similar shapes. We propose a new pose measurement network based on statistical attention, called Statistical Attention Pose Measurement Network (SAPENet).

[0007] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0008] A large-scale cabin posture measurement algorithm based on deep learning includes the following steps:

[0009] Step 1: First, use the encoder as the backbone network to extract features from the input point cloud data.

[0010] Step 2: In the high-order statistics module, calculate the high-order statistical moments of the features and normalize them.

[0011] Step 3. Then, calculate statistics from the set Z. This includes first-order statistics such as mean μ, second-order statistics such as variance σ, and higher-order statistics such as skewness m3, kurtosis m4, etc. For convenience, let us call this set of statistics B, as shown in Formula 1.

[0012] Step 4: Use B to simulate the geometric relationship and detail changes of the features, and calculate the statistical data of the set Z in B.

[0013] Step 5: Use the attention mechanism to assign statistics to the original features, focus on related areas and semantic information at different locations according to different weights, and assign different statistics.

[0014] Step 6. After assigning statistics to the original features, use a one-dimensional convolution to increase its dimension to match the dimension of the input data.

[0015] Step 7. Then, a residual connection is added to mitigate the impact of covariance shift.

[0016] Step 8. Next, the rotation and translation of the object are predicted through the posture regression module network.

[0017] Step 9. Finally, use the Rodrigues rotation formula to calculate the corrected normal plane to obtain the final rotation matrix R and the prediction of the translation t.

[0018] The beneficial effects of the present invention are as follows:

[0019] 1) Improving the accuracy of pose estimation: By introducing the statistical attention mechanism, the present invention can effectively capture the long-distance dependencies between object features, enhance the perception of object geometry and spatial relationships, and thus improve the accuracy of pose estimation.

[0020] 2) Enhanced ability to identify detail differences: By using high-order statistical information, the present invention can identify and emphasize subtle differences between different objects, which is particularly important for distinguishing objects of similar shapes, further improving the performance of posture estimation.

[0021] 3) Balance between computational efficiency and performance: The statistical attention mechanism is implemented in the form of a residual structure, which has high computational efficiency and can improve the performance of the network without significantly increasing the computational burden.

[0022] 4) No need for additional CAD models: Since the present invention does not rely on the CAD model of a specific object, the threshold and cost of posture estimation are lowered, making posture estimation of unknown objects possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] like Figure 1 As shown in Figure 1, it is an overview of the pose measurement algorithm based on statistical attention. It adopts the HS layer as the main encoder architecture. The core unit of SAPENet is the statistical attention module, which is integrated in two appropriate stages in the architecture. This module uses high-order statistical information to model geometric relationships and capture long-distance dependencies while paying more attention to the detailed differences between objects. In addition, three sub-modules are integrated for pose regression, point cloud reconstruction, and bounding box voting to form a complete SAPENet. The pose estimation branch outputs confidence-aware rotation parameters {C1,r y ,C2,r s}, residual translation t* and size S* parameters, from which we can recover the pose {R,t,S} in closed form.

[0024] like Figure 2 Figure 1 shows the overall structure of the statistical attention module. The blue shaded area calculates the statistics. The green shaded area distributes the statistics to each position according to the weights.

[0025] like Figure 3 The following is the visualization result of the proposed method. DETAILED DESCRIPTION

[0026] The present invention is further described in detail below with reference to the accompanying drawings.

[0027] The specific implementation steps are as follows:

[0028] Step 1. First, use the encoder as the backbone network to extract features from the input point cloud data. The encoder adopts a method based on 3D graph convolution. And replace the graph convolution layer with HS layers, which can effectively handle the complex geometric shapes of objects. In order to further capture long-distance dependencies and detailed differences between objects, we introduce statistical attention at the end of the encoder. After obtaining the global features, statistical attention is used to calculate the long-distance dependencies between features, and high-order statistical information is used to obtain detailed differences between objects.

[0029] Step 2: In the high-order statistical module, calculate the high-order statistical moments of the features and normalize them. Given the input point cloud feature P, P∈R N×D , where N represents the number of center points and D represents the channel dimension. Statistical attention first uses one-dimensional convolution (1D) to reduce the dimension and decouple the correlation between channels, thereby obtaining a new feature X, X∈R N×D ', where D' is the new channel dimension. These new features are then reinterpreted as a set of Z = {x i |i=1,...,D'}.

[0030] Step 3. Then, calculate statistics from the set Z. This includes first-order statistics such as mean μ, second-order statistics such as variance σ, and higher-order statistics such as skewness m3, kurtosis m4, etc. For convenience, let us call this set of statistics B, as shown in Formula 1.

[0031] B={μ,σ,m3,m4,m5,...,} (1)

[0032] Step 4: Use B to simulate the geometric relationship and detail changes of features. We introduce the function M(Z,B) and use Formula 2 to calculate the statistical data of set Z in B.

[0033] M(Z,B)={φ b (Z)|b∈B} (2)

[0034] in Represents the statistic of the set Z.

[0035] Step 5: Use the attention mechanism to assign statistics to the original features. Through Formula 3, different statistics can be assigned according to different weights by focusing on relevant areas and semantic information at different locations.

[0036]

[0037] Among them, f(·) is the similarity function, R(χ i ) is the normalization factor. Then, we get the output Y = [y1, y2, ..., y D' ]∈R C×D'.

[0038] Step 6. After assigning the statistics to the original features, the obtained Y is further processed by one-dimensional convolution through formula 4 to increase its dimension to match the dimension of the input data. This step facilitates subsequent calculations. In addition, a batch normalization layer is used to ensure the scale consistency of input-output.

[0039]

[0040] Where Conv(·) represents a one-dimensional convolution and BN(·) represents a batch normalization layer.

[0041] Step 7: Then, add a residual connection according to Formula 5 to mitigate the impact of covariance shift

[0042]

[0043] in Denotes the output features.

[0044] Step 8. Next, the pose regression module network is used to predict the rotation and translation of the object. Specifically, the rotation is predicted in the form of the normal plane of the 3D bounding box. Two paths are used to estimate the confidence of each normal plane. We use r x ,r y Denote the predicted normal plane, with c x ,c y In order to calibrate the plane normal, the network minimizes the cost function as shown in Formula 6:

[0045]

[0046] in Indicates the angle between the predicted normal plane and the actual normal plane.

[0047] Step 9. Finally, use the Rodrigues rotation formula to calculate the corrected normal plane to obtain the final rotation matrix R and the prediction of the translation t.

Claims

1. The pose measurement algorithm based on statistical attention includes the following steps: Step 1. First, use the encoder as the backbone network to extract features from the input point cloud data. The encoder adopts a method based on 3D graph convolution. And replace the graph convolution layer with HS layers, which can effectively handle the complex geometric shapes of objects. In order to further capture long-distance dependencies and detailed differences between objects, we introduce statistical attention at the end of the encoder. After obtaining the global features, statistical attention is used to calculate the long-distance dependencies between features, and high-order statistical information is used to obtain detailed differences between objects. Step 2: In the high-order statistical module, the high-order statistical moments of the features are calculated and normalized. Given the input point cloud feature P, P∈R N×D , where N represents the number of center points and D represents the channel dimension. Statistical attention first uses one-dimensional convolution (1D) to reduce the dimension and decouple the correlation between channels, thereby obtaining a new feature X, X∈R N×D , where D′ is the new channel dimension. These new features are then reinterpreted as a set of Z = {x i |i=1,…,D′}. Step 3. Then, calculate statistics from the set Z. This includes first-order statistics such as mean μ, second-order statistics such as variance σ, and higher-order statistics such as skewness m3, kurtosis m4, etc. For convenience, let us call this set of statistics B, as shown in Formula 1. B={μ,σ,m3,m4,m5,...,} (1) Step 4: Use B to simulate the geometric relationship and detail changes of features. We introduce the function M(Z, B) and use Formula 2 to calculate the statistical data of the set Z in B. in Represents the statistic of the set Z. Step 5: Use the attention mechanism to assign statistics to the original features. Through Formula 3, different statistics can be assigned according to different weights by focusing on relevant areas and semantic information at different locations. in, f(·) is the similarity function, R(χ i ) is the normalization factor. Then, we get the output Y = [y1, y2, ..., y D′ ]∈R C×D . Step 6. After assigning the statistics to the original features, the obtained Y is further processed by one-dimensional convolution through formula 4 to increase its dimension to match the dimension of the input data. This step facilitates subsequent calculations. In addition, a batch normalization layer is used to ensure the scale consistency of input-output. Where Conv(·) represents a one-dimensional convolution and BN(·) represents a batch normalization layer. Step 7: Then, add a residual connection according to Formula 5 to mitigate the impact of covariance shift in Denotes the output features. Step 8. Next, the pose regression module network is used to predict the rotation and translation of the object. Specifically, the rotation is predicted in the form of the normal plane of the 3D bounding box. Two paths are used to estimate the confidence of each normal plane. We use r x ,r y Denote the predicted normal plane, with c x ,c y In order to calibrate the plane normal, the network minimizes the cost function as shown in Formula 6: in Indicates the angle between the predicted normal plane and the actual normal plane. Step 9. Finally, use the Rodrigues rotation formula to calculate the corrected normal plane to obtain the final rotation matrix R and the prediction of the translation t.

Images