A 6D pose estimation method and system for objects based on attention mechanism

Abstract

The present invention relates to the field of computer vision technology, and specifically to a method and system for estimating the 6D pose of an object based on an attention mechanism; the object 6D pose estimation method utilizes a 6D pose estimation network, and the 6D pose estimation network includes a target detection module, a backbone network module, and a pose prediction module. In the target detection module, the RGB image is magnified according to the location of the target. The backbone network module adopts a new ResNet variant, integrating the pixel-level wavelet multi-branch attention mechanism and the dual-channel upsampling attention mechanism to achieve efficient capture and fusion of multi-scale features of the target in the spatial domain and the wavelet domain. The pose prediction module uses a neural network to directly predict the 6D pose. The present invention not only provides an innovative idea for improving occlusion robustness, but also constructs a new paradigm for end-to-end object 6D pose estimation based on spatial-frequency domain feature fusion.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and system for estimating the 6D pose of an object based on an attention mechanism. Background Art

[0002] Six-degree-of-freedom pose estimation, a foundational technology for robotics, augmented reality, and industrial intelligence, has benefited from cutting-edge breakthroughs in the multi-layer perceptron paradigm. In particular, the collaborative optimization of network architecture innovations and multi-level feature representations has significantly enhanced the pose prediction capabilities of RGB vision. Notably, the LINEMOD dataset and its occluded version, as a universal evaluation standard, continue to drive technological advancements in this field.

[0003] Existing technology primarily builds pose estimation frameworks around convolutional neural networks. The indirect paradigm constructs a mapping between 2D projected coordinates and 3D keypoints, combined with a PnP solver to achieve pose computation. The direct paradigm, however, relies on an end-to-end architecture and uses deep networks to directly regress object pose from pixel space. The landmark PoseCNN established a performance baseline through global feature learning, while PVNet innovatively employed vector field representation to enhance occlusion robustness. Notably, attention mechanisms have demonstrated unique value in feature enhancement, with research on CBAM demonstrating the effectiveness of dual channel-spatial attention for feature extraction. PANet proposed an attention upsampling module that expands the design space for feature aggregation, allowing for better feature aggregation in the decoder. However, it does not integrate the attention mechanism into the upsampling operation, thus failing to preserve feature integrity and mitigate information loss during feature map resolution enhancement. The idea of ​​fusing spatial and wavelet domain features has been applied in computer vision, effectively improving network performance. However, the effectiveness of attention mechanisms for feature enhancement in the wavelet domain remains limited.

[0004] Despite breakthroughs, current object 6D pose estimation technology still has core bottlenecks. First, the PnP solution of the indirect method suffers from a contradiction between computational efficiency and differentiability, and the direct method is susceptible to data noise and relies on high-cost annotation. Secondly, the existing attention mechanism is mostly limited to feature enhancement in a single spatial domain, and fails to systematically deconstruct and fuse cross-domain complementary features between the spatial domain and the wavelet domain. Furthermore, the existing methods lack effective extraction of pixel-level features in the wavelet domain. In addition, the currently widely used upsampling strategy fails to fully protect the integrity of the original features during implementation, resulting in the output results losing some key information to a certain extent. More seriously, the existing scheme has the risk of pose misjudgment in heavily occluded scenes. Summary of the Invention

[0005] The present invention provides a method and system for estimating the 6D pose of an object based on an attention mechanism to solve the technical problems mentioned in the background technology.

[0006] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] The present invention provides a method for estimating the 6D pose of an object based on an attention mechanism, comprising the following steps:

[0008] S1. Build a 6D pose estimation network. The 6D pose estimation network includes a target detection module, a backbone network module, and a pose prediction module. The backbone network module embeds a pixel-level wavelet multi-branch attention mechanism and a dual-channel upsampling attention mechanism. Divide the dataset into a training set and a test set.

[0009] S2. Select an RGB image from the training set and input it into the target detection module. Identify the target object in the RGB image, magnify the RGB image including the target object to a fixed size, obtain a magnified image of the target object, and output a 2D pixel coordinate map of the target object.

[0010] S3. Input the magnified image of the target object into the backbone network module to obtain the surface area attention map, object coordinate map and mask map of the target object, and then splice the object coordinate map and the 2D pixel coordinate map of the target object to obtain a dense correspondence map;

[0011] S4. The surface area attention map and the dense correspondence map are stitched together and input into the pose prediction module to obtain the predicted 6D pose information of the target object;

[0012] S5. Construct a total loss function based on the predicted 6D pose information of the target object, the actual 6D pose information, the surface area attention map, the object coordinate map, and the mask map. Repeat S2 to S5 to minimize the total loss function until the total loss function converges to obtain the trained 6D pose estimation network.

[0013] S6. Use the test set to test the trained 6D pose estimation network to obtain the 6D pose information of the object.

[0014] On the other hand, the present invention also provides a 6D pose estimation system for an object based on an attention mechanism, including a computer device programmed or configured to execute a 6D pose estimation method for an object.

[0015] Beneficial effects of the present invention:

[0016] 1. This paper proposes a pixel-level wavelet multi-branch attention mechanism based on a multi-branch architecture with powerful position and channel feature extraction capabilities. It cleverly draws on the idea of ​​wavelet transform and, based on the extraction and fusion of spatial and wavelet domain features, gives the 6D pose estimation network excellent diversity and multi-scale feature extraction capabilities. The pixel-level wavelet multi-branch attention mechanism is integrated into the ResNet34 deep residual module to form a new ResNet34 deep residual module, which effectively improves the accuracy and robustness of the 6D pose estimation task.

[0017] 2. The present invention proposes a dual-channel upsampling attention mechanism based on a parallel architecture of sampling branches and attention branches. While improving the resolution, the feature integrity is protected through the attention mechanism. This structure breaks through the traditional single-branch upsampling paradigm and realizes the fusion output of dual-stream information through the grid sampling function. The dual-channel upsampling attention mechanism is integrated into the decoder to replace the traditional upsampling layer, effectively improving the accuracy of 6D pose estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Flowchart of the method for estimating 6D pose of an object in the present invention;

[0019] Figure 2 This is a block diagram of the 6D pose estimation network in the present invention;

[0020] Figure 3 It is a structural block diagram of the encoder in the present invention;

[0021] Figure 4 This is a flowchart of the pixel-level wavelet multi-branch attention mechanism in the present invention;

[0022] Figure 5 This is a flowchart of the dual-channel upsampling attention mechanism in the present invention. DETAILED DESCRIPTION

[0023] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The accompanying drawings illustrate preferred embodiments of the present invention. However, the present invention may be implemented in many other forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present disclosure.

[0024] Reference Figure 1 and Figure 2 , an embodiment of the present application provides a method for estimating 6D pose of an object based on an attention mechanism, comprising the following steps:

[0025] S1. Build a 6D pose estimation network. The 6D pose estimation network includes a target detection module, a backbone network module, and a pose prediction module. The backbone network module embeds a pixel-level wavelet multi-branch attention mechanism and a dual-channel upsampling attention mechanism. Divide the dataset into a training set and a test set.

[0026] Specifically, the embodiment of the present invention selected the widely used LINEMOD dataset and its extended version LINEMOD-Occluded dataset. At the same time, it can also be replaced by a self-made proprietary dataset. The LINEMOD dataset covers 15 objects of different colors, shapes and sizes, and provides 6D pose annotation information for 13 of them. The LINEMOD-Occluded dataset is based on the LINEMOD dataset extension, contains 8 objects, and additionally introduces more complex occlusion situations. During the training phase, the real annotated data is combined with the synthetic samples corresponding to each object to improve the generalization ability of the model. The dataset is divided, 85% of which is divided into a test set and 15% is divided into a training set, and additional synthetic images are generated for each object to enhance the training effect.

[0027] S2. Select an RGB image from the training set and input it into the target detection module. Identify the target object in the RGB image, zoom in on the RGB image including the target object to a fixed size, obtain the zoomed image of the target object, and output the 2D pixel coordinate map of the target object. , size is ;

[0028] S3. Input the magnified image of the target object into the backbone network module to obtain the surface area attention map, object coordinate map and mask map of the target object, and then splice the object coordinate map and the 2D pixel coordinate map of the target object to obtain a dense correspondence map;

[0029] Based on the input RGB image, the present invention adopts a backbone network module that integrates the pixel-level wavelet multi-branch attention mechanism and the dual-channel upsampling attention mechanism to extract the features of the target object. Specifically, the pixel-level wavelet multi-branch attention mechanism uses a parallel multi-branch structure to capture wavelet domain information and ultimately combines it with spatial domain features; while the dual-channel upsampling attention mechanism uses a dual-branch architecture to simultaneously perform channel-spatial attention-guided feature integrity protection during the feature map resolution enhancement process, significantly suppressing information loss.

[0030] S4. The surface area attention map and the dense correspondence map are stitched together and input into the pose prediction module to obtain the predicted 6D pose information of the target object;

[0031] S5. Construct a total loss function based on the predicted 6D pose information of the target object, the actual 6D pose information, the surface area attention map of the target object, the object coordinate map, and the mask map. Repeat S2 to S5 to minimize the total loss function until the total loss function converges to obtain the trained 6D pose estimation network.

[0032] S6. Test the trained 6D pose estimation network using the test set to obtain the 6D pose information of the object. After passing the test, the 6D pose estimation network is deployed on the device. The 6D pose estimation network on the device can be used to perform 6D pose estimation on the RGB image to obtain the 6D pose information of the target object.

[0033] In some embodiments, the target detection module uses the YOLOv5 model;

[0034] The backbone network module includes an encoder and a decoder connected in sequence. The structure of the encoder is as follows: Figure 3 As shown in the figure; the encoder includes a ResNet34 deep residual model and three pixel-level wavelet multi-branch attention mechanisms. The ResNet34 deep residual module includes the first to fourth convolutional layers connected in sequence. The three pixel-level wavelet multi-branch attention mechanisms are embedded at the ends of the first to third convolutional layers respectively; a dual-channel upsampling attention mechanism is embedded in the decoder to replace the upsampling layer in the traditional decoder.

[0035] The pixel-level wavelet multi-branch attention mechanism cleverly borrows the idea of ​​wavelet transform. Based on the extraction and fusion of spatial and wavelet domain features, it gives the 6D pose estimation network excellent diversity and multi-scale feature extraction capabilities. The pixel-level wavelet multi-branch attention mechanism is integrated into the ResNet34 deep residual module to form a new ResNet34 deep residual module, which effectively improves the accuracy and robustness of the 6D pose estimation task.

[0036] The dual-channel upsampling attention mechanism improves the resolution while protecting the feature integrity through the attention mechanism. This structure breaks through the traditional single-branch upsampling paradigm, realizes the fusion output of dual-stream information through the grid sampling function, and integrates the dual-channel upsampling attention mechanism in the decoder to replace the traditional upsampling layer, effectively improving the accuracy of 6D pose estimation.

[0037] In some embodiments, the step S2 includes the following steps:

[0038] S21. Select an RGB image from the training set and input it into the YOLOv5 model. The YOLOv5 model performs online analysis on the RGB image containing the target object and outputs the detection box coordinate parameters of the target object in the RGB image in real time.

[0039] S22, intercepting the corresponding target area from the RGB image according to the detection frame coordinate parameters, performing a size normalization operation on the target area, and uniformly adjusting the pixel resolution of the target area to a preset specification to obtain an enlarged image of the target object;

[0040] S23, after uniformly adjusting the pixel resolution of the target area to the preset specifications, generate a corresponding two-dimensional position coordinate matrix for each pixel in the RGB image based on the YOLOv5 model to obtain the 2D pixel coordinate map of the target object .

[0041] In some embodiments, reference Figure 4 The data processing steps of the pixel-level wavelet multi-branch attention mechanism are as follows:

[0042] A1. Input features The input channel is divided into two parts, namely spatial domain features and features The main purpose of dividing input features is to reduce computational complexity, preserve contextual information, and obtain diverse multi-domain fusion feature information.

[0043] A2. Features Perform discrete wavelet transform to obtain wavelet domain features ; Then the wavelet domain features Passed to parallel stages A and B;

[0044] A3. In stage A, wavelet domain features First, through a Convolutional layers and Activation function obtains feature map ;

[0045] A4. Feature map Input into the multi-branch position attention mechanism to obtain the feature map , feature map After the convolutional layer and Activation function to obtain position features , expressed by the formula, as follows:

[0046] ;

[0047] ;

[0048] ;

[0049] in, and Respectively represent Convolution operation and Convolution operation, represents a dropout operation with a retention probability of 95%, represents the multi-branch position attention mechanism, represent Activation function;

[0050] A5. In stage B, wavelet domain features First, after a convolutional layer and Activation function obtains feature map ;

[0051] A6. Feature map Input into the multi-branch channel attention mechanism set in parallel with the multi-branch position attention mechanism to obtain the feature map , feature map go through Convolutional layers and Activation function to obtain channel features , expressed by the formula, as follows:

[0052] ;

[0053] in, and Respectively represent Convolution operation and Convolution operation, represents a dropout operation with a retention probability of 95%, represents the multi-branch channel attention mechanism, represent Activation function;

[0054] A7. Position features and channel characteristics Add up the features ,feature After a The convolutional layers and Activation function gets features ;

[0055] A8. Features Use inverse wavelet transform to get the features , and then the spatial domain and features Splice and pass a The convolutional layers and Activation function to obtain fusion features ;

[0056] A9. Input features Through a After the convolution layer, the fusion feature Add together to get the output features .

[0057] The pixel-level wavelet multi-branch attention mechanism combines the robust position feature extraction capability of the multi-branch position attention module with the channel feature extraction advantages of the multi-branch channel attention module.

[0058] In some embodiments, the discrete wavelet transform in A2 is formulated as follows:

[0059] Suppose the original input image is , for each non-overlapping A pixel block is defined as:

[0060] ;

[0061] ;

[0062] ;

[0063] ;

[0064] in, , , and Representing each The four pixel values ​​in the pixel block are initially normalized (divided by 2), where and Represents the index of the pixel block on the row and column respectively.

[0065] Based on the above definition, discrete wavelet transform obtains four sub-band components:

[0066] ;

[0067] ;

[0068] ;

[0069] ;

[0070] in, represents the low-frequency subband, represents the horizontal detail subband, represents the vertical detail subband, Represents diagonal detail subband.

[0071] The inverse wavelet transform in A8 is specifically formulated as follows:

[0072] According to the four sub-band coefficients obtained , , and . Then for the same Pixel block, the formula for inverse wavelet transform to reconstruct the original pixel value is:

[0073] ;

[0074] ;

[0075] ;

[0076] ;

[0077] in , , and are the reconstructed pixel values ​​respectively, and the denominator 2 is the normalization factor generated by the normalization during the forward transform and the corresponding recovery operation in the inverse transform.

[0078] In some embodiments, reference Figure 4 , said A4 comprises the following steps:

[0079] A41, through the convolution operation, the feature map Converted into three new feature maps, namely feature maps , feature map and feature maps ;

[0080] A42, the feature map Reshape and transpose to , get the reshaped feature map , the feature map Reshape into , get the reshaped feature map , and reshape the feature map and the reshaped feature map Perform matrix multiplication, where represents the set of real numbers, and , Indicates the number of pixels, 、 Represent pixel height and pixel width respectively; the multiplied result is input into the softmax layer to calculate the spatial attention map , spatial attention map The calculation formula is as follows:

[0081] ;

[0082] in, Representing spatial attention map Middle The element at position The influence weight of the element at each position; represents an exponential function with the natural constant e as the base; Representation feature map Middle A vector of positions; Representation feature map Middle A vector of positions;

[0083] A43, feature map Reshape into , the reshaped feature map With spatial attention map Multiply and reshape the result of the multiplication into , multiply the reshaped result by the weight parameter (Weight Initialized to 0, which can be learned through training), and then compared with the feature map Perform element-by-element summation to obtain the final output feature map , the calculation formula is as follows:

[0084] ;

[0085] in, Represents the output of the multi-branch position attention mechanism The features at each position are the weighted fusion results of global context and local features. Represents the reshaped feature map Middle The vector of positions, Representation feature map No. Characteristics of a location.

[0086] A44, Feature Map After the convolutional layer and Activation function to obtain position features .

[0087] The multi-branch position attention mechanism is used to extract the spatial features of the wavelet domain. It is a weighted sum of all position features and original features, which has global context features and aggregates context based on the spatial attention map. This ensures that position features are effectively extracted while maintaining global context information.

[0088] The multi-branch positional attention mechanism captures the spatial dependencies between any two positions in the feature map and updates specific features by taking a weighted sum of all positional features. By considering the feature similarities between these positions, weights are determined and the features at specific positions are updated to extract meaningful spatial features. This helps the 6D pose estimation network better understand the spatial relationships between different regions in the image, improving the performance of 6D pose estimation of objects.

[0089] In some embodiments, reference Figure 4 , said A6 comprises the following steps:

[0090] A61, feature map Reshape into , for the reshaped feature map and the reshaped feature map The transpose of performs matrix multiplication and inputs the matrix multiplication result into the softmax layer to obtain the channel attention map , the calculation formula is as follows:

[0091] ;

[0092] in, Representative channel attention map Middle The elements of the channel are The influence weight of the elements of each channel; Represents the reshaped feature map No. The feature vector of each channel; Represents the reshaped and transposed feature map No. The feature vector of each channel;

[0093] A62, channel attention map and the reshaped feature map Multiply and reshape the result into ; Then, multiply the reshaped result by a weight parameter (Weight Initialized to 0, which can be learned through training), and the reshaped feature map Perform element-by-element summation to obtain the final output feature map , the calculation formula is as follows:

[0094] ;

[0095] in, Represents the output of the multi-branch position attention mechanism The features of each channel are the weighted fusion results of the inter-channel dependencies and the original features. Feature map representing the original input No. The feature vector of each channel.

[0096] A63, Feature Map go through Convolutional layers and Activation function to obtain channel features .

[0097] In some embodiments, reference Figure 5 , the data processing steps of the dual-channel upsampling attention mechanism are as follows:

[0098] B1. First, give the original features and upsampling factor ;

[0099] B2. The original feature map Input into the sampling point generator to generate a sampling set ;

[0100] Specifically, first use a linear layer and upsample the scale factor For the original feature map The generated size is The offset of represent The offset is multiplied by the offset range factor of 0.25 to obtain the intermediate offset, and then the intermediate offset is reshaped into a size of The final offset ; Afterwards, the final offset With the original sampling grid Add together to get the sampling set , expressed by the formula, as follows:

[0101] ;

[0102] in, represents a linear layer, Pixel reordering is an upsampling technique that converts channel dimensions to spatial dimensions. It improves resolution by rearranging the channel data of the feature map, avoiding traditional interpolation calculations.

[0103] Original sampling grid It is initialized by a standard grid using bilinear interpolation, and its implementation relies on the grid_sample function of PyTorch.

[0104] B3. The original feature map Input into the channel attention mechanism for weighting to obtain the weighted result , expressed by the formula, as follows:

[0105] ;

[0106] in, represents the sigmoid activation function, represents the adaptive average pooling layer, represents the adaptive maximum pooling layer, and Represents a convolution operation with two different convolution kernels of 1; represent Activation function, Add represents the feature addition operation;

[0107] B4. Weighted results With the original feature map Multiply to get the feature map , to achieve the integrity protection of the initial features during the upsampling process;

[0108] B5. Feature map Input into the spatial attention mechanism for weighting to obtain the weighted result , expressed by the formula, as follows:

[0109] ;

[0110] in, and Represent the maximum pooling and average pooling channel functions respectively, Represents a convolution operation with a convolution kernel of 5. Represents a splicing operation;

[0111] B6. Weighted results With feature map Multiply to get the feature map ;

[0112] B7, the sampling set and feature maps Input into the grid sampling function to generate feature maps , thereby completing the upsampling operation, which is expressed by the formula as follows:

[0113] ;

[0114] Among them, grid_sample represents the grid sampling function.

[0115] The dual-channel upsampling attention mechanism adopts a dual-branch architecture, namely the sampling branch and the attention branch; the dual-channel upsampling attention mechanism uses a dual-branch architecture to simultaneously perform channel-spatial attention-guided feature integrity protection during the process of feature map resolution improvement, significantly suppressing information loss.

[0116] In one embodiment, the settings of the sampling point generator mainly include setting the initial sampling position, setting the offset range factor and setting the grouping. Specifically:

[0117] Set the initial sampling position: The present invention uses "bilinear initialization" to set the initial sampling position. In the initial state (no offset), let The sampling points are distributed on a uniform grid (e.g. The four corners of the grid are used to simulate the grid distribution of bilinear interpolation, rather than concentrating all sampling points in the same position. When the offset is zero, this initialization is directly equivalent to standard bilinear interpolation. This initialization method retains the smoothness of the interpolation process, avoids feature confusion caused by position overlap during initialization, and reduces the difficulty of training.

[0118] Set the offset range factor: Due to the existence of the normalization layer, the value of an output feature is usually centered around 0 Therefore, local The moving ranges of the sample positions can overlap significantly, which can easily affect predictions near the boundaries. Such errors can propagate from stage to stage, causing artifacts in the output. To address this issue, we multiply the offsets by an offset range factor of 0.25, which satisfies the theoretical boundary condition between overlap and non-overlap. This locally constrains the moving range of the sample positions, alleviating this issue.

[0119] Setting grouping: The present invention divides the feature map into groups and generate The group offset,feature maps share the same sample set within each group.,The core purpose of group upsampling is to decouple feature,expression through the channel dimension, while reducing computational complexity and,the number of parameters, while enhancing the model's adaptability to different,semantic features.

[0120] In some embodiments, the prediction processing of the backbone network module in S3 obtains a series of intermediate features, which include the surface area attention map of the target object. , object coordinate diagram And the mask map Then, the object coordinate map and the 2D pixel coordinate map are spliced ​​to obtain a dense correspondence map Specifically:

[0121] Surface area attention map of the target object :The target object surface area attention map is generated based on a symmetry-aware attention mechanism. This symmetry-aware attention mechanism is used to encode the area division information of the object surface and express the confidence that pixels belong to different surface areas through probability distribution. Its core goal is to solve the pose estimation ambiguity problem caused by symmetrical objects and improve the geometric feature learning effect. The target object surface area attention map is based on the object coordinate map and generates surface area labels through farthest point sampling (FPS). The backbone network module predicts the surface area classification probability corresponding to each pixel and assigns each pixel to the corresponding area.

[0122] Object coordinate diagram :This feature map represents the normalized 3D coordinates of each pixel in the object coordinate system, which is used to establish a dense correspondence between 2D pixels and 3D object coordinates. The object coordinate map is generated by rendering the real 3D coordinate map under a given posture according to the object CAD model, and then the backbone network module predicts the normalized coordinates. Each channel By object bounding box size Zoom to range, thereby improving scale robustness.

[0123] Mask map : Identifies the visible area of ​​an object in the input image, used to filter background noise and occluded areas, and supervise the visible part. It is generated by rendering the object's CAD model to generate a true visible area mask. The backbone network module directly regresses the visibility probability (0 or 1) of each pixel.

[0124] Dense correspondence graph A feature map composed of a 2D pixel coordinate map and a normalized object coordinate map, explicitly encoding dense 2D-3D correspondences. It is generated by concatenating the normalized 3D object coordinates with the 2D coordinates of the corresponding pixel to form a 5D vector for each pixel.

[0125] In some embodiments, the step S4 includes the following steps:

[0126] S41, splicing the surface area attention map and the dense correspondence map and inputting them into the pose prediction module;

[0127] S42. Based on the surface area attention map and the dense correspondence map, the center coordinates of the target object to be solved are calculated using the following formula , And the depth value to be solved ;

[0128] ;

[0129] in, 、 、 The pose prediction module is 、 、 The relative translation prediction value in the direction, 、 Represent the center coordinates of the detection bounding box, 、 Indicates the width and height of the detection box; It is the magnification coefficient used by the target detection module;

[0130] S43, using the back projection method and the obtained target object center coordinates , and depth values Calculate the translation of the target object , the calculation formula is as follows:

[0131] ;

[0132] in, is the intrinsic parameter matrix of a given camera, Represents the transpose of a matrix;

[0133] S44. Use the following formula to solve the rotation matrix of the target object and the translation of the target object The rotation matrix of the target object constitutes the predicted 6D pose information of the target object;

[0134] ;

[0135] in, 、 、 They correspond to the three column vectors of the rotation matrix to be solved, 、 They are the first three values ​​and the last three values ​​of the rotation results predicted by the posture prediction module, It is a normalization operation.

[0136] In some embodiments, the total loss function in S5 is specifically as follows:

[0137] For asymmetric target objects, the total loss function in S5 is:

[0138] ;

[0139] in, is the rotation loss function of the asymmetric target object, is the 2D center offset loss function, is the depth loss function, is the geometric information loss function;

[0140] , , and The calculation formula is:

[0141] ;

[0142] in, 、 、 They are the pose regression modules in 、 、 The true value of the relative translation in the direction; 、 Represent the predicted value and true value of the mask map respectively; 、 The predicted value and true value of the object coordinate map respectively; 、 Represent the predicted value and true value of the surface area attention map respectively; 、 Represent objects separately The predicted value of the rotation matrix and the true value ,in , A 3D model representing an object; represents element-wise multiplication, represents the cross entropy loss; represents the 1-norm;

[0143] For symmetrical target objects, the total loss function in S5 is:

[0144] ;

[0145] in, is the rotation loss function for symmetric target objects , as follows:

[0146] ;

[0147] in, represents the true value of all valid rotations under symmetry; Represents the rotation loss function for asymmetric target objects.

[0148] The 6D pose estimation network was built in PyTorch. The training setup used the Ranger optimizer (batch size 24, learning rate 0.0001), and a cosine decay-based learning rate adjustment strategy was initiated at 72% training progress. For comparison purposes, all models were implemented in PyTorch using CUDA 11.1. Training, validation, and testing were performed on an RTX4090 GPU. A series of experiments were then conducted to demonstrate the advantages of the proposed method.

[0149] (1) Comparative experiments between the method provided by the present invention and other SOTA methods;

[0150] Table 1 presents a quantitative comparison of the proposed method with six state-of-the-art algorithms on the LINEMOD benchmark using the 0.1 ADD(-S) metric under the same experimental environment. Table 2 presents the performance of the proposed method and six state-of-the-art algorithms on the LINEMOD-Occluded dataset, using the 0.1 ADD(-S) metric. The experimental results demonstrate that the proposed method outperforms all existing state-of-the-art methods in both standard scenes and occlusion conditions.

[0151] Table 1: Performance comparison of the proposed method and other SOTA algorithms on LINEMOD.

[0152]

[0153] (A "-" indicates that the result is not available.)

[0154] Table 2: Performance comparison results of the proposed method and other SOTA algorithms on LINEMOD-Occluded;

[0155]

[0156] (A "-" indicates that the result is not available.)

[0157] As shown in Tables 1 and 2, the proposed method achieves 94.54% and 67.03% accuracy on the LINEMOD and LINEMOD-Occluded datasets, respectively. This represents a 0.84% ​​improvement on the LINEMOD dataset and a 4.83% improvement on the LINEMOD-Occluded dataset over the baseline GDR-Net method. Experimental results demonstrate that the proposed model framework demonstrates significant breakthrough performance in complex 6D object pose estimation tasks.

[0158] (2) Ablation experiment of the method provided by the present invention

[0159] To test the effectiveness of the proposed pixel-level wavelet multi-branch attention mechanism and dual-channel upsampling attention mechanism, we conducted ablation experiments on the LINEMOD dataset. Table 3 lists the ablation experiment results of the proposed method on the LINEMOD dataset under the same environment.

[0160] Table 3: Ablation test results of the method provided by the present invention on the LINEMOD dataset under the same environment;

[0161]

[0162] As shown in Table 3, compared to baseline methods, both the pixel-level wavelet multi-branch attention mechanism and the dual-channel upsampling attention mechanism, when used alone, improve the accuracy of 6D pose estimation. Combining these two mechanisms achieves the highest accuracy, reaching 38.32% at the 2% ADD(-S) level, 78.18% at the 5% ADD(-S) level, and 94.54% at the 10% ADD(-S) level. The rotation error reaches a minimum of 1.77°, while the translation error is essentially the same across all methods, remaining at an extremely low level. These experimental results demonstrate the effectiveness of the pixel-level wavelet multi-branch attention mechanism and the dual-channel upsampling attention mechanism.

[0163] Specifically, for 6D posture evaluation, the present invention adopts the ADD(-S) indicator. The indicator predicts the posture by calculating With real posture The mean of the point-by-point Euclidean distance after the transformation of the object model point set :

[0164] ;

[0165] Taking 0.1 ADD as an example, when When the distance is less than or equal to 0.1 times the diameter of the object, the prediction result is considered correct. For symmetrical objects with multiple valid true poses, the present invention calculates the ADD-S index by minimizing the distance between corresponding points:

[0166] ;

[0167] in, represents the true value, A 3D model representing an object.

[0168] Specifically, the translation error Represents the Euclidean distance between the predicted translation vector and the true translation vector (in meters). The smaller the value, the more accurate the translation estimate. The calculation method is:

[0169] ;

[0170] in, and represent the predicted value and the true value respectively.

[0171] Specifically, the rotation error Represents the angular deviation between the predicted rotation matrix and the true rotation matrix (in degrees). The smaller the value, the more accurate the rotation estimation. The calculation method is:

[0172] ;

[0173] in, and denote the predicted value and the true value respectively, represents the trace of the matrix, Represents the arccosine function, which converts the cosine value into an angle.

[0174] If the object has symmetry (such as a cylinder), then the formula will be replaced by the symmetry group The closest rotation matrix , in order to eliminate the influence of symmetry on the error.

[0175] Specifically, the symmetry group It is the set of all rotation matrices of an object in three-dimensional space that keep its appearance unchanged. For example, a cylinder: around the central axis Any angle rotation of , Cube: Contains 24 rotation operations (such as 90°, 180°, and 270° rotation around faces, edges, and vertices).

[0176] Symmetry rotation error calculation requires finding the symmetry group Zhongyu closest ,Right now:

[0177] ;

[0178] in, A symmetry transformation that acts on the object's local coordinate system.

[0179] For a continuous symmetry group (such as a cylinder), let , the error needs to be minimized:

[0180] ;

[0181] The problem is transformed into maximizing the alignment of the two matrices, that is:

[0182] ;

[0183] make , after expansion we get:

[0184] ;

[0185] The optimal solution is:

[0186] ;

[0187] By adjusting Align the two rotation matrices in directions other than the axis of symmetry.

[0188] For discrete symmetry groups (such as the cube), iterate over all possible ,calculate , select the minimum value corresponding to .

[0189] Experimental evaluation based on the LINEMOD and LINEMOD-Occluded datasets shows that the method provided by the present invention exhibits significant positioning accuracy and system robustness when dealing with cluttered and severely occluded scenes.

[0190] On the other hand, the present invention also provides a 6D pose estimation system for an object based on an attention mechanism, including a computer device programmed or configured to execute a 6D pose estimation method for an object.

[0191] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A 6D pose estimation method for an object based on an attention mechanism, characterized in that: The steps include: S1. Build a 6D pose estimation network. The 6D pose estimation network includes a target detection module, a backbone network module, and a pose prediction module. The backbone network module embeds a pixel-level wavelet multi-branch attention mechanism and a dual-channel upsampling attention mechanism. Divide the dataset into training and testing sets; S2. Select an RGB image from the training set and input it into the target detection module. Identify the target object in the RGB image, magnify the RGB image including the target object to a fixed size, obtain a magnified image of the target object, and output a 2D pixel coordinate map of the target object. S3. Input the magnified image of the target object into the backbone network module to obtain the surface area attention map, object coordinate map and mask map of the target object, and then splice the object coordinate map and the 2D pixel coordinate map of the target object to obtain a dense correspondence map; S4. The surface area attention map and the dense correspondence map are stitched together and input into the pose prediction module to obtain the predicted 6D pose information of the target object; S5. Construct a total loss function based on the predicted 6D pose information of the target object, the actual 6D pose information, the surface area attention map, the object coordinate map, and the mask map. Repeat S2 to S5 to minimize the total loss function until the total loss function converges to obtain the trained 6D pose estimation network. S6. Use the test set to test the trained 6D pose estimation network to obtain the 6D pose information of the object; The data processing steps of the pixel-level wavelet multi-branch attention mechanism are as follows: A1. Input features The input channel is divided into two parts, namely spatial domain features and features ; A2. Features Perform discrete wavelet transform to obtain wavelet domain features ; A3. Wavelet Domain Features After one Convolutional layers and Activation function obtains feature map ; A4. Feature map Input into the multi-branch position attention mechanism to obtain the feature map , feature map After the convolutional layer and Activation function to obtain position features ; A5. Wavelet Domain Features After a convolutional layer and Activation function obtains feature map ; A6. Feature map Input into the multi-branch channel attention mechanism set in parallel with the multi-branch position attention mechanism to obtain the feature map , feature map go through Convolutional layers and Activation function to obtain channel features ; A7. Position features and channel characteristics Add up the features ,feature After a The convolutional layers and Activation function gets features ; A8. Features Use inverse wavelet transform to get the features , and then the spatial domain and features Splice and pass a The convolutional layers and Activation function to obtain fusion features ; A9. Input features Through a After the convolution layer, the fusion feature Add together to get the output features .

2. The method for estimating 6D pose of an object based on an attention mechanism according to claim 1, wherein: The target detection module uses the YOLOv5 model; The backbone network module includes an encoder and a decoder connected in sequence, the encoder includes a ResNet34 deep residual model and three pixel-level wavelet multi-branch attention mechanisms, the ResNet34 deep residual module includes the first to fourth convolutional layers connected in sequence, and the three pixel-level wavelet multi-branch attention mechanisms are respectively embedded at the ends of the first to third convolutional layers; A dual-channel upsampling attention mechanism is embedded in the decoder to replace the upsampling layer in the traditional decoder.

3. The method for estimating 6D pose of an object based on an attention mechanism according to claim 2, wherein: The S2 comprises the following steps: S21. Select an RGB image from the training set and input it into the YOLOv5 model. The YOLOv5 model performs online analysis on the RGB image containing the target object and outputs the detection box coordinate parameters of the target object in the RGB image in real time. S22, intercepting the corresponding target area from the RGB image according to the detection frame coordinate parameters, performing a size normalization operation on the target area, and uniformly adjusting the pixel resolution of the target area to a preset specification to obtain an enlarged image of the target object; S23, after uniformly adjusting the pixel resolution of the target area to the preset specifications, generate a corresponding two-dimensional position coordinate matrix for each pixel in the RGB image based on the YOLOv5 model to obtain the 2D pixel coordinate map of the target object .

4. The method for estimating 6D pose of an object based on an attention mechanism according to claim 2, wherein: The A4 is expressed by the formula, as follows: ； ； ； in, and Respectively represent Convolution operation and Convolution operation, represents a dropout operation with a retention probability of 95%, represents the multi-branch position attention mechanism, represent Activation function; A6 is expressed by the formula, as follows: ； in, and Respectively represent Convolution operation and Convolution operation, represents a dropout operation with a retention probability of 95%, represents the multi-branch channel attention mechanism, represent Activation function.

5. The method for estimating 6D pose of an object based on an attention mechanism according to claim 4, wherein: The A4 comprises the following steps: A41, through the convolution operation, the feature map Converted into three new feature maps, namely feature maps , feature map and feature maps ; A42, the feature map Reshape and transpose to , get the reshaped feature map , the feature map Reshape into , get the reshaped feature map , and reshape the feature map and the reshaped feature map Perform matrix multiplication, where represents the set of real numbers, and , Indicates the number of pixels, 、 Represent pixel height and pixel width respectively; the multiplied result is input into the softmax layer to calculate the spatial attention map , spatial attention map The calculation formula is as follows: ； in, Representing spatial attention map Middle The element at position The influence weight of the element at each position; represents an exponential function with the natural constant e as the base; Representation feature map Middle A vector of positions; Representation feature map Middle A vector of positions; A43, feature map Reshape into , the reshaped feature map With spatial attention map according to Multiply and sum in the order of , and then reshape the result into Then multiply it by the weight parameter , and with the feature map Perform element-by-element summation to obtain the final output feature map , the calculation formula is as follows: ； in, Represents the output of the multi-branch position attention mechanism The feature of each position is the weighted fusion result of global context and local features; Represents the reshaped feature map Middle The vector of positions, Representation feature map No. Characteristics of each location; A44, Features After the convolutional layer and Activation function to obtain position features .

6. The method for estimating 6D pose of an object based on an attention mechanism according to claim 5, wherein: The A6 comprises the following steps: A61, feature map Reshape into , for the feature map The reshaped results and feature maps The reshaped and transposed results are subjected to matrix multiplication, and the matrix multiplication result is input into the softmax layer to obtain the channel attention map , the calculation formula is as follows: ； in, Representative channel attention map Middle The elements of the channel are The influence weight of the elements of each channel; Represents the reshaped feature map No. The feature vector of each channel; Represents the reshaped and transposed feature map No. The feature vector of each channel; A62, channel attention map and the reshaped feature map according to Multiply and sum in the order of , and reshape the result into ; Then, multiply the reshaped result by a weight parameter , and with the original feature map Perform element-by-element summation to obtain the final output feature map , the calculation formula is as follows: ； in, Represents the output of the multi-branch position attention mechanism The features of each channel are the weighted fusion results of the inter-channel dependencies and the original features; Feature map representing the original input No. The feature vector of each channel; A63, Features go through Convolutional layers and Activation function to obtain channel features .

7. The method for estimating 6D pose of an object based on an attention mechanism according to claim 6, wherein: The data processing steps of the dual-channel upsampling attention mechanism are as follows: B1. First, the original feature map is given and upsampling factor ; B2. Use a linear layer and upsample based on the scaling factor For the original feature map Generate an offset, multiply the offset by the offset range factor to get an intermediate offset, and then reshape the intermediate offset into the final offset using pixel reordering ; Afterwards, the final offset With the original sampling grid Add together to get the sampling set , expressed by the formula, as follows: ； in, represents a linear layer, represents pixel rearrangement; B3. The original feature map Input into the channel attention mechanism for weighting to obtain the weighted result , expressed by the formula, as follows: ； in, represents the sigmoid activation function, represents the adaptive average pooling layer, represents the adaptive maximum pooling layer, and Represents a convolution operation with two different convolution kernels of 1; represent Activation function, Add represents the feature addition operation; B4. Weighted results With the original feature map Multiply to get the feature map ; B5. Feature map Input into the spatial attention mechanism for weighting to obtain the weighted result , expressed by the formula, as follows: ； in, and Represent the maximum pooling and average pooling channel functions respectively, represents a convolution operation with a convolution kernel of 5. Represents a splicing operation; B6. Weighted results With feature map Multiply to get the feature map ; B7, the sampling set and feature maps Input into the grid sampling function to generate feature maps , thereby completing the upsampling operation, which is expressed by the formula as follows: ； Among them, grid_sample represents the grid sampling function.

8. The method for estimating 6D pose of an object based on an attention mechanism according to claim 7, wherein: The S4 comprises the following steps: S41, splicing the surface area attention map and the dense correspondence map and inputting them into the pose prediction module; S42. Based on the surface area attention map and the dense correspondence map, the center coordinates of the target object to be solved are calculated using the following formula , And the depth value to be solved ; ； in, 、 、 The pose prediction module is 、 、 The relative translation prediction value in the direction, 、 Represent the center coordinates of the detection bounding box, 、 Indicates the width and height of the detection box; It is the magnification coefficient used by the target detection module; S43, using the back projection method and the obtained target object center coordinates , and depth values Calculate the translation of the target object , the calculation formula is as follows: ； in, is the intrinsic parameter matrix of a given camera, Represents the transpose of a matrix; S44. Use the following formula to solve the rotation matrix of the target object and the translation of the target object The rotation matrix of the target object constitutes the predicted 6D pose information of the target object; ； in, 、 、 They correspond to the three column vectors of the rotation matrix to be solved, 、 They are the first three values ​​and the last three values ​​of the rotation results predicted by the posture prediction module, It is a normalization operation.

9. The method for estimating 6D pose of an object based on an attention mechanism according to claim 8, wherein: The total loss function in S5 is as follows: For asymmetric target objects, the total loss function in S5 is: ； in, is the rotation loss function of the asymmetric target object, is the 2D center offset loss function, is the depth loss function, is the geometric information loss function; , , and The calculation formula is: ； in, 、 、 They are the pose regression modules in 、 、 The true value of the relative translation in the direction; 、 Represent the predicted value and true value of the mask map respectively; 、 The predicted value and the true value of the object coordinate map respectively; 、 Represent the predicted value and true value of the surface area attention map respectively; 、 Represent objects separately The predicted value of the rotation matrix and the true value ,in , A 3D model representing an object; represents element-wise multiplication, represents the cross entropy loss; represents the 1-norm; For symmetrical target objects, the total loss function in S5 is: ； in, is the rotation loss function for symmetric target objects , as follows: ； in, represents the true value of all valid rotations under symmetry; Represents the rotation loss function for asymmetric target objects.

10. A 6D pose estimation system for an object based on an attention mechanism, comprising a computer device, characterized in that: The computer device is programmed or configured to execute the object 6D pose estimation method according to any one of claims 1 to 9.

Images