Monocular 6D pose estimation method and system based on enhanced feature projection decoupling

By applying enhanced feature projection decoupling technology on monocular RGB images, enhancing features of high-dimensional objects are extracted and coordinate projection decoupling is performed, and the dependence on multi-view angles or multi-intermediate representations in the prior art is solved, and efficient and robust object position estimation is achieved.

CN119904514AActive Publication Date: 2025-04-29SHANDONG UNIV
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411656311.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-19
Publication Date
2025-04-29
Estimated Expiration
2044-11-19

AI Technical Summary

Technical Problem

The existing RGB mode-based object position estimation method needs to rely on multiple perspectives or multiple intermediate representations, resulting in high labeling costs and low operating efficiency.

Method used

The monocular 6D pose estimation method based on enhanced feature projection decoupling is adopted. Complementary high-dimensional object enhancement features are extracted through double-flow branches and projected to three two-dimensional orthogonal planes. Multiple geometric constraints are introduced using coordinate consistency between planes to estimate the dense corresponding map between the three-dimensional model of the object and the two-dimensional image.

Benefits of technology

This method can accurately estimate the position of the object without relying on depth information, multi-view images and multiple intermediate representations, improve the robustness and operation efficiency in complex scenarios, and reduce network design and data labeling costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119904514A_ABST
    Figure CN119904514A_ABST
Patent Text Reader

Abstract

The invention provides a monocular 6D pose estimation method and system based on enhanced feature projection decoupling, and relates to the technical field related to robot visual perception, and the method comprises the steps: obtaining a monocular RGB image of a target object scene; inputting the monocular RGB image into the trained pose estimation network to obtain a 6D pose of the object; local features and multi-scale global features are extracted and fused through the pose estimation network, and high-dimensional object enhanced feature representation is obtained; performing coordinate projection decoupling on the enhanced feature representation of the high-dimensional object to obtain dense corresponding mapping between a three-dimensional model and a two-dimensional image of the target object and an object mask; performing pose regression on the dense corresponding mapping of the target object and the object mask to obtain a 6D pose of the object; the method does not need to depend on depth information, multi-view images and various intermediate representations, can accurately estimate the pose of the object only by using the monocular RGB image, reduces the network design and data annotation cost, and improves the robustness of a complex scene.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field related to robot visual perception, and in particular to a monocular 6D pose estimation method and system based on enhanced feature projection decoupling. Background Art

[0002] The 6D pose estimation of an object is a measurement task that calculates the rigid transformation from the coordinate system of the target object in the image to the camera coordinate system based on the input image, including 3D rotation and 3D translation. This technology is currently widely used in vision-guided robot systems such as grasping, automatic assembly, and human-computer interaction.

[0003] Most deep learning-based object pose estimation methods rely on depth information, but depth sensors are susceptible to interference from multipath and ambient light, and have inherent limitations in terms of measurement range, field of view, and frame rate, making these methods less robust to environmental noise and sparsely textured objects, and having low operating efficiency. Recent methods based on RGB modality can be divided into two technical paradigms: methods based on multi-view images and methods based on multiple intermediate representations. The essence of these two methods is to use geometric consistency to introduce multiple constraints to alleviate the feature sparseness challenges caused by complex scenes such as occlusion, truncation, clutter, and textureless objects, thereby improving the accuracy of object pose estimation.

[0004] The current technical paradigm based on RGB mode has the following main problems:

[0005] The multi-view based method involves a complex camera calibration process, which is not conducive to practical deployment. It also has high requirements on computing resources during training and inference, which limits its application in resource-constrained scenarios.

[0006] Methods based on multiple intermediate representations require the annotation of multiple types of intermediate representations such as key points, contours, dense correspondences, surface normals, etc., and corresponding estimation networks need to be designed for different types of intermediate representations, resulting in high model design and annotation costs.

[0007] Therefore, existing pose estimation methods based on RGB modality need to rely on multi-view or multi-intermediate representations, which have the problems of high annotation cost and low operating efficiency. Summary of the invention

[0008] In order to solve the above problems, the present disclosure proposes a monocular 6D pose estimation method and system based on enhanced feature projection decoupling, which does not rely on depth information, multi-view images and multiple intermediate representations. It can accurately estimate the object pose by using only one intermediate representation, namely, a monocular RGB image and dense correspondence mapping. It is also highly robust to complex scenes such as occlusion, truncation, clutter, and objects with sparse textures, thereby reducing the cost of network design and data annotation and improving the robustness to complex scenes.

[0009] According to some embodiments, the present disclosure adopts the following technical solutions:

[0010] A monocular 6D pose estimation method based on enhanced feature projection decoupling includes:

[0011] Obtain a monocular RGB image of the target object scene;

[0012] Input the monocular RGB image into the trained pose estimation network to obtain the 6D pose of the object;

[0013] Among them, the pose estimation network extracts and fuses local features and multi-scale global features to obtain a high-dimensional object enhanced feature representation; coordinate projection decoupling is performed on the high-dimensional object enhanced feature representation to obtain a dense corresponding mapping between the three-dimensional model and the two-dimensional image of the target object and an object mask; based on the dense corresponding mapping and object mask of the target object, pose regression is performed to obtain the 6D pose of the object.

[0014] According to some embodiments, the present disclosure adopts the following technical solutions:

[0015] A monocular 6D pose estimation system based on enhanced feature projection decoupling, including:

[0016] The image acquisition module is configured to: acquire a monocular RGB image of a target object scene;

[0017] The pose estimation module is configured to: input the monocular RGB image into the trained pose estimation network to obtain the 6D pose of the object;

[0018] Among them, the pose estimation network extracts and fuses local features and multi-scale global features to obtain a high-dimensional object enhanced feature representation; coordinate projection decoupling is performed on the high-dimensional object enhanced feature representation to obtain a dense corresponding mapping between the three-dimensional model and the two-dimensional image of the target object and an object mask; based on the dense corresponding mapping and object mask of the target object, pose regression is performed to obtain the 6D pose of the object.

[0019] According to some embodiments, the present disclosure adopts the following technical solutions:

[0020] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the monocular 6D pose estimation method based on enhanced feature projection decoupling is implemented.

[0021] According to some embodiments, the present disclosure adopts the following technical solutions:

[0022] A non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the monocular 6D pose estimation method based on enhanced feature projection decoupling is implemented.

[0023] According to some embodiments, the present disclosure adopts the following technical solutions:

[0024] An electronic device comprises: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the monocular 6D pose estimation method based on enhanced feature projection decoupling.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] The present invention provides a monocular 6D pose estimation method and system based on enhanced feature projection decoupling, aiming to overcome the problems of high annotation cost and low operating efficiency caused by the existing pose estimation method based on RGB modality relying on multiple viewpoints or multiple intermediate representations; the proposed method alleviates the feature sparsity problem caused by complex environments by extracting complementary high-dimensional object enhancement features using dual-stream branches, and at the same time projects the high-dimensional object enhancement features to three two-dimensional orthogonal planes, and introduces multiple geometric constraints using the coordinate consistency between the planes to estimate the dense corresponding mapping between the three-dimensional model of the object and the two-dimensional image, so that the network only needs to rely on a single-view RGB image and a single intermediate representation to effectively improve the operating efficiency and accuracy of pose estimation in various complex scenes such as occlusion, truncation, and textureless objects, and has high engineering practical value.

[0027] The present invention does not need to rely on depth information, multi-view images and multiple intermediate representations. It can accurately estimate the object pose using only a monocular RGB image and a dense correspondence mapping intermediate representation. It is also highly robust to complex scenes such as occlusion, truncation, clutter, and sparsely textured objects, greatly reducing the cost of network design and manual labeling. It can run in real time on devices with limited computing power, such as laptops, expanding its application in resource-constrained scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.

[0029] Figure 1 The figure is a flow chart of the overall method of the embodiment of the present disclosure.

[0030] Figure 2 2 is a diagram of the posture estimation network structure of an embodiment of the present disclosure.

[0031] Figure 3 This is a context clustering network structure diagram of an embodiment of the present disclosure.

[0032] Figure 4 A schematic diagram of context clustering network grouping, dynamic feature aggregation and feature updating according to an embodiment of the present disclosure.

[0033] Figure 5 A schematic diagram of coordinate projection decoupling according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0035] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0036] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0037] Example 1

[0038] In one embodiment of the present disclosure, a monocular 6D pose estimation method based on enhanced feature projection decoupling is provided, comprising:

[0039] Obtain a monocular RGB image of the target object scene;

[0040] Input the monocular RGB image into the trained pose estimation network to obtain the 6D pose of the object;

[0041] Among them, the pose estimation network extracts and fuses local features and multi-scale global features to obtain a high-dimensional object enhanced feature representation; coordinate projection decoupling is performed on the high-dimensional object enhanced feature representation to obtain a dense corresponding mapping between the three-dimensional model and the two-dimensional image of the target object and an object mask; based on the dense corresponding mapping and object mask of the target object, pose regression is performed to obtain the 6D pose of the object.

[0042] As an embodiment, the monocular 6D pose estimation method based on enhanced feature projection decoupling disclosed in the present invention uses Figure 2The pose estimation network shown does not need to rely on depth information and multi-view images. It can accurately estimate the pose of the object using only a monocular RGB image and a dense correspondence mapping as an intermediate representation. It is also highly robust to complex scenes such as occlusion, truncation, clutter, and sparsely textured objects. The specific implementation process is as follows:

[0043] Step S1: Obtain a monocular RGB image of the target object scene.

[0044] The monocular RGB image of the target object scene is obtained by cutting out a target object region from an original scene image, and using the target object region as the monocular RGB image of the target object scene.

[0045] Specifically, step S11, using a target detection algorithm, such as a YOLO series algorithm, a Mask RCNN series algorithm, etc., to estimate a two-dimensional bounding box of the area where the target object is located in the original scene image;

[0046] Step S12: crop the target object area based on the two-dimensional bounding box and scale it to a predefined size, such as 256×256. The image is a monocular RGB image of the target object scene.

[0047] Step S2: Input the RGB image into the feature enhancement network, extract the local features and multi-scale global features of the object, and fuse the global features into the local features in turn and upsample them to obtain a high-dimensional object enhanced feature representation with a size of 1 / 4 of the input image.

[0048] The feature enhancement network mainly includes local feature extraction, global feature extraction, and collaborative feature fusion upsampling, specifically:

[0049] Step S21, local feature extraction: use ConvNext-base to extract local features from the input image to obtain high-dimensional local features with a resolution of 1 / 32 of the input image size.

[0050] Step S22: using each pixel I in the input image i,j Location information To enhance the image, and convert the enhanced image into a two-dimensional point set Among them, n=H×W represents the number of points, and after enhancement, each pixel contains two kinds of information: color and position.

[0051] Step S23, multi-stage global feature extraction: using Figure 3 The context clustering network shown performs multi-stage downsampling, grouping, dynamic feature aggregation and feature updating on an image in the form of a two-dimensional point set, and obtains multi-scale features with resolutions of 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the input image size.

[0052] Specifically, the steps include:

[0053] Step S231: Use strided convolution to downsample the input features by a factor of two.

[0054] Step S232: Figure 4 As shown in the orange path in the figure, two fully connected layers are used to project the feature P in the form of a two-dimensional point set into a high-dimensional space of the same dimension to obtain P q and P v , used for subsequent similarity calculation, using average pooling from P q and P v A preset number of center points P are uniformly selected from q (c) and P v (c) The multi-head mode is used to calculate the similarity between all center points and other points. Finally, the fully connected layer FC is used to splice and fuse the multi-head calculation results to obtain the similarity matrix Sim:

[0055] Sim=FC(Cos(P q (c),P q )

[0056] Where Cos means calculating cosine similarity using multi-head mode;

[0057] Step S233: divide all feature points into several groups according to similarity. Each point is divided into the group where the center point with the highest similarity is located. All feature points in the same group are dynamically aggregated to the center point. Taking the i-th group as an example, after feature aggregation, the center point feature P v (c i )′ is calculated as:

[0058]

[0059] Where sig represents the sigmoid function, which is used to normalize the weights, and s ij It represents the feature similarity score between the center point of the i-th group and the j-th point in the group. μ and v are trainable parameters used to dynamically weight each feature point.

[0060] Step S234: Figure 4 As shown by the green path in , the aggregated center point features are assigned to other points in the group according to the similarity matrix to update their features, ensuring that the points in the same group can exchange information with each other. Taking the jth point P(I,J) in the i-th group as an example, the updated feature P(i,j)′ is:

[0061] P(i,j)′=P(i,j)+FC(sig(μs ij +v)*P v (c i )′)

[0062] Step S235: input the updated feature point set into a fully connected layer with residual connection to obtain the final output feature.

[0063] Step S24, collaborative feature fusion upsampling: fuse the local features output in step S21 with the global features output in step S23 that match the local feature resolution, and perform upsampling, and the upsampled features continue to be fused with the global features of the corresponding resolution. The final output size is a feature enhancement representation of 1 / 4 the size of the input image.

[0064] Furthermore, the method specifically includes the following steps:

[0065] Step S241: Add and fuse the local features outputted from step S21 with the global features matching the resolution and perform double upsampling. Input the fused features into 3×3 convolution and dilated convolution with residual connection in sequence for spatial interaction, and output fused features with a resolution of 1 / 16 of the input image size. Repeat this step twice to finally obtain fused features with a resolution of 1 / 4 of the input image size.

[0066] The techniques used in the three upsampling operations are bilinear sampling, bilinear upsampling, and transposed convolution.

[0067] Step S242: continue to fuse the output of step S241 with the global features corresponding to the resolution, and perform spatial interaction to obtain enhanced features with a resolution of 1 / 4 of the input image size.

[0068] Step S3, inputting the high-dimensional object enhanced feature representation into the coordinate projection decoupling module to obtain a dense correspondence mapping between the three-dimensional model and the two-dimensional image of the target object and an object mask;

[0069] Please refer to Figure 5 , specifically including the following steps:

[0070] Step S31: Input the enhanced features output from step S242 into three sub-networks and a fully connected layer, and regress the projections C of the dense corresponding maps on three two-dimensional orthogonal planes (i.e., xy, yz, and xz planes) respectively. xy ,C yz ,C xz and the mask C of the target object m :

[0071]

[0072] Among them, N xy ,N yz ,N xz ,N mThey are the subnetwork for predicting projection coordinates and the fully connected layer for predicting object masks, and their parameters are ∈,ζ,θ,v, respectively.

[0073] Step S32: Use average pooling Avg to calculate the coordinate mean between the two-dimensional projection planes to obtain a dense corresponding map, and multiply it with the target object mask to remove the background to obtain a dense corresponding map of the target object:

[0074]

[0075] Among them, p x 、p y and p z Represents the three coordinate component sets of dense projection mapping, C xy [0] represents the x-coordinate component of the projection of the dense correspondence map on the xy plane, and so on for other symbols.

[0076] The structures of the three sub-networks are consistent. Each sub-network includes three convolutional layers with a kernel size of 3×3 and a stride of 1, using group normalization and Gaussian error linear unit activation function.

[0077] Step S4: input the dense correspondence map of the target object with the background removed into the pose regression network to calculate the 6D pose of the object.

[0078] The pose regression network contains three strided downsampling convolutional layers and five fully connected layers. The first three fully connected layers are used to reduce the dimension of the flattened features, and the last two fully connected layers are used to calculate the three-dimensional rotation R and three-dimensional translation T of the object, respectively, and finally obtain the 6D pose of the object.

[0079] Step S5: Perform iterative training based on the difference between the forward output result of the network and the true value, update the parameters of all modules, and use the converged pose estimation network to process the input target object image to obtain the 6D pose of the object.

[0080] Specifically, during iterative training in step S5, a multi-task learning strategy is adopted to define multiple loss functions according to different task attributes and sum them to guide the parameter update of the model, specifically including:

[0081] S51. Define mask loss:

[0082]

[0083] in, is the true value of the i-th pixel mask, is the conditional function, when the mask prediction value of the i-th pixel Greater than or equal to Then the function is 1, otherwise it is 0. In this embodiment,

[0084] S52. Define the pose loss function:

[0085]

[0086] Among them, x i represents the i-th point sampled from the object 3D model, and are the true value and predicted value of three-dimensional rotation respectively, and They represent the true value and predicted value of the three-dimensional translation in the SITE format respectively. In this embodiment, k=1024.

[0087] S53. Define dense correspondence mapping loss:

[0088]

[0089] Among them, n and H c ×W c are the resolution and number of foreground points of the dense correspondence map, respectively.

[0090] Represents the L1 norm of the predicted value and the true value of the i-th dense corresponding mapping point.

[0091] S54. Define coordinate consistency loss:

[0092] L cc =|C xy [0]-C xz [0]|1+|C xy [1]-C yz [0]|1+|C xz [1]-C xz [1]|1)

[0093] Among them, C xy [0] represents the set of x-coordinate components of the projection of the dense correspondence map on the xy plane, and so on for other symbols.

[0094] S55. Constructing a loss function based on multi-task learning

[0095] L=L mask +L pose +L map +5L cc

[0096] Table 1 shows the comparison results of this embodiment with the most advanced methods on the LM-O and T-LESS datasets. The dataset contains 61 objects of different shapes and textures, and includes various challenges such as occlusion, clutter, and truncation. The methods listed in the table all use the BOP indicator as the performance evaluation indicator.

[0097] Table 1: Performance comparison with state-of-the-art methods on LM-O and T-LESS datasets

[0098] Method Name LM-O T-LESS DPODv2 58.6 72.0 OVE-6D 62.7 54.6 ER-Pose 61.6 65.4 MatchU 68.0 66.8 Ours 69.7 79.5

[0099] It can be seen that the method of this embodiment performs better than the most advanced methods on the data set. It does not need to rely on depth information and multi-view images, nor does it need to rely on multiple intermediate representations. It only needs to use monocular RGB images and dense correspondence mapping as an intermediate representation to accurately estimate the object pose, which reduces the cost of network design and data annotation, and is highly robust to complex scenes such as occlusion, truncation, clutter, and sparsely textured objects.

[0100] Example 2

[0101] In one embodiment of the present disclosure, a monocular 6D pose estimation system based on enhanced feature projection decoupling is provided, comprising:

[0102] The image acquisition module is configured to: acquire a monocular RGB image of a target object scene;

[0103] The pose estimation module is configured to: input the monocular RGB image into the trained pose estimation network to obtain the 6D pose of the object;

[0104] Among them, the pose estimation network extracts and fuses local features and multi-scale global features to obtain a high-dimensional object enhanced feature representation; coordinate projection decoupling is performed on the high-dimensional object enhanced feature representation to obtain a dense corresponding mapping between the three-dimensional model and the two-dimensional image of the target object and an object mask; based on the dense corresponding mapping and object mask of the target object, pose regression is performed to obtain the 6D pose of the object.

[0105] Example 3

[0106] In one embodiment of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the monocular 6D pose estimation method based on enhanced feature projection decoupling.

[0107] Example 4

[0108] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the monocular 6D pose estimation method based on enhanced feature projection decoupling is implemented.

[0109] Example 5

[0110] In one embodiment of the present disclosure, an electronic device is provided, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the monocular 6D pose estimation method based on enhanced feature projection decoupling.

[0111] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0113] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A monocular 6D pose estimation method based on enhanced feature projection decoupling, characterized in that: include: Obtain a monocular RGB image of the target object scene; Input the monocular RGB image into the trained pose estimation network to obtain the 6D pose of the object; Among them, the pose estimation network extracts and fuses local features and multi-scale global features to obtain a high-dimensional object enhanced feature representation; coordinate projection decoupling is performed on the high-dimensional object enhanced feature representation to obtain a dense corresponding mapping between the three-dimensional model and the two-dimensional image of the target object and an object mask; based on the dense corresponding mapping and object mask of the target object, pose regression is performed to obtain the 6D pose of the object.

2. The monocular 6D pose estimation method based on enhanced feature projection decoupling as claimed in claim 1, characterized in that: The monocular RGB image of the target object scene is obtained by cutting out the target object region from the original scene image, and using the target object region as the monocular RGB image of the target object scene.

3. The monocular 6D pose estimation method based on enhanced feature projection decoupling as claimed in claim 1, characterized in that: The feature enhancement network includes local feature extraction, multi-stage global feature extraction and collaborative feature fusion upsampling in sequence; The local feature extraction uses ConvNext-base to extract local features from the input image; The multi-stage global feature extraction enhances the image through the position information of each pixel in the image, and converts the enhanced image into a two-dimensional point set form, and uses a context clustering network to perform multi-stage downsampling, grouping, dynamic feature aggregation and feature updating on the image in the two-dimensional point set form to obtain a multi-scale global feature; The collaborative feature fusion upsampling fuses local features with multi-scale global features and performs upsampling, and the upsampled features are further fused with global features of corresponding resolutions to ultimately obtain a high-dimensional object enhanced feature representation.

4. The monocular 6D pose estimation method based on enhanced feature projection decoupling as claimed in claim 1, characterized in that: The coordinate projection decoupling is specifically as follows: From the high-dimensional object enhanced feature representation, three sub-networks and one fully connected layer branch are used to regress the projections of the dense correspondence map on three two-dimensional orthogonal planes and the mask of the target object respectively; Average pooling is used to calculate the coordinate mean between the two-dimensional projection planes to obtain a dense correspondence map, which is then multiplied by the target object mask to obtain a dense correspondence map of the target object.

5. The monocular 6D pose estimation method based on enhanced feature projection decoupling as claimed in claim 1, characterized in that: The pose regression is to calculate the 6D pose of the object using a pose regression network; The pose regression network includes three strided downsampling convolutional layers and five fully connected layers, wherein the first three fully connected layers are used to reduce the dimension of the flattened features, and the last two fully connected layers are used to calculate the three-dimensional rotation R and three-dimensional translation T of the object, and finally obtain the 6D pose of the object.

6. The monocular 6D pose estimation method based on enhanced feature projection decoupling as claimed in claim 1, characterized in that: The loss function of the pose estimation network includes mask loss, pose loss, dense correspondence mapping loss and coordinate consistency loss.

7. A monocular 6D pose estimation system based on enhanced feature projection decoupling, characterized in that: include: The image acquisition module is configured to: acquire a monocular RGB image of a target object scene; The pose estimation module is configured to: input the monocular RGB image into the trained pose estimation network to obtain the 6D pose of the object; Among them, the pose estimation network extracts and fuses local features and multi-scale global features to obtain a high-dimensional object enhanced feature representation; coordinate projection decoupling is performed on the high-dimensional object enhanced feature representation to obtain a dense corresponding mapping between the three-dimensional model and the two-dimensional image of the target object and an object mask; based on the dense corresponding mapping and object mask of the target object, pose regression is performed to obtain the 6D pose of the object.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the monocular 6D pose estimation method based on enhanced feature projection decoupling according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the monocular 6D pose estimation method based on enhanced feature projection decoupling as described in any one of claims 1 to 6 is implemented.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes the monocular 6D pose estimation method based on enhanced feature projection decoupling as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • 6D pose estimation method using contour information

    CN110322510A

  • Monocular 6D attitude estimation method and device based on deep convolutional neural network

    CN112767486A

  • Single-view pose estimation method and system based on multi-modal input and attention mechanism

    CN115861418A

  • 6D pose estimation method based on multi-scale feature fusion

    CN116597006A

  • Progressive feature distribution sampling 6D pose estimation method and system based on camera

    CN117078753A