General 6D attitude estimation method based on shape constraint

Abstract

The invention provides a general 6D attitude estimation method based on shape constraint, and relates to the field of image and video processing. Comprising the following steps: (1) inputting an image and rendering the image; (2) detecting a target and cutting the image; (3) predefining a rotation sample set; (4) constructing a Gaussian mixture model, and sampling by using the Gaussian mixture model; (5) generating candidate postures; (6) generating a rendered image; (7) feature extraction; (8) constructing a correlation volume; (9) performing GRU feature processing; (10) optical flow prediction and confidence calculation; (11) PnP optimization is carried out; and (12) outputting the precise 6D attitude of the target object. According to the method, through an innovative iterative optimization strategy based on the optical flow, the attitude estimation efficiency and precision are remarkably improved. The robustness and generalization ability of the algorithm are greatly improved by a cascade network architecture and a confidence decomposition mechanism.

Description

Technical Field

[0001] This invention provides a general 6D pose estimation method based on shape constraints, which relates to the field of image and video processing. Background Technology

[0002] In applications such as industrial automation, robotics, and augmented reality, 6D pose estimation technology (including 3D position and 3D rotation) plays a crucial role. This technology enables robots to accurately identify and grasp objects, achieving precise operations on automated production lines, while providing realistic visual effects in augmented reality applications. Traditional 6D pose estimation methods primarily rely on deep learning models, learning feature representations of specific objects through extensive training data. These methods perform well when handling known objects and have been widely applied in various fields such as industrial production, warehousing and logistics, and smart manufacturing. However, with the continuous expansion of application scenarios and the increasing demands of technology, traditional methods have encountered new challenges in practical applications, urgently requiring more flexible and intelligent solutions.

[0003] Existing 6D pose estimation methods suffer from severe limitations when dealing with novel objects. First, these methods require retraining the model for each new object, which is not only time-consuming and labor-intensive in practical applications but also significantly increases deployment costs. Second, existing methods often struggle to effectively utilize the object's 3D geometric information, resulting in less than ideal estimation accuracy. This is particularly problematic in industrial settings where the variety of new objects is constantly increasing, and current technologies cannot adapt quickly enough, severely hindering production efficiency. Furthermore, existing methods exhibit poor generalization ability when handling objects with complex shapes or surface features, making them prone to estimation bias. Therefore, developing a new method that balances accuracy and generalization, enabling rapid and accurate 6D pose estimation of new objects, has become a crucial research direction in this field. This will not only improve the efficiency of industrial automation but also drive the development of emerging technologies such as augmented reality. Summary of the Invention

[0004] This invention provides a general 6D pose estimation method based on shape constraints to solve the following technical problems;

[0005] (1) The primary technical problem addressed by this invention is how to effectively utilize the 3D shape information of an object while reducing computational costs. Existing technologies either require rendering a large number of multi-view images to obtain shape information, resulting in huge computational overhead; or they rely entirely on 2D feature matching while ignoring 3D geometric constraints, leading to insufficient accuracy. This trade-off between computational efficiency and estimation accuracy severely restricts the practical application of existing methods in industrial scenarios. Therefore, there is an urgent need for a new technical solution that can fully utilize 3D shape information to improve the accuracy of pose estimation while maintaining low computational costs.

[0006] (2) Another key issue that this invention also needs to address is how to improve the robustness and generalization ability of the algorithm in complex scenes. Existing technologies perform poorly when dealing with interference factors such as occlusion and changes in lighting, especially for objects with limited surface texture, where the estimation performance drops significantly due to the difficulty in extracting reliable features. Furthermore, many methods require retraining the model for new objects, which is both time-consuming and impractical in real-world applications. Therefore, a technical solution is needed that can adaptively handle various complex scenes and has good generalization ability for new objects.

[0007] The complete technical solution provided by this invention:

[0008] A general 6D pose estimation method based on shape constraints includes the following steps:

[0009] (1) Input image and rendered image:

[0010] Input the 3D model file of the target object and the RGB image of the target object whose pose needs to be estimated;

[0011] For input image calibration, the camera intrinsic parameter matrix K is in the form of:

[0012]

[0013] Where f x ,f y It is the focal length parameter, (c x ,c y () are the coordinates of the principal point;

[0014] (2) Target detection and image cropping;

[0015] (2.1) Object detection is performed using a CNOS detector:

[0016] (2.1.1) Deep features of the image are extracted using the pre-trained base model SAM. Specifically, the input image... First, the data is divided into multiple fixed-size patches (e.g., 16×16 pixels). Each patch is linearly embedded into a high-dimensional vector, forming an embedding vector. Where N is the number of patches, and D is the embedding dimension:

[0017] E = PatchEmbedding(I) (1)

[0018] The embedding vector E is passed through a multi-layer Transformer encoder, and the output of each layer can be considered as a deep feature at a different level. Assuming an L-layer Transformer encoder is used, the output of the l-th layer can be expressed as:

[0019] F (l) =TransformerEncoder (l) (F (l-1) (2)

[0020] Where F(0) = E represents the initial patch embedding;

[0021] The final deep feature map F is the output of the last Transformer encoder layer, after being processed by LayerNorm:

[0022] F = LayerNorm(F (L) (3)

[0023] (2.1.2) Detection Head Network: Contains two branches:

[0024] Bounding box regression branch: Outputs the 2D bounding box coordinates (x, y, w, h) of the object;

[0025] Category classification branch: Predicts the probability distribution of object categories;

[0026] (2.1.3) The detection results can be represented as a list: D = {d i}, where each detection di contains:

[0027] Bounding box coordinates: b i =(x i ,y i ,w i ,h i );

[0028] Category tag: c i ;

[0029] Detection confidence score: s i ;

[0030] (2.2) Image cropping transformation:

[0031] I crop =Crop(I, b) i (4)

[0032] The Crop function is based on the bounding box coordinates b. i Extract the target region from the original image I.

[0033] (2.3) Apply the resize function to the cropped image to adjust its size to a specified fixed resolution, i.e., height H and width W:

[0034] I resize =Resize(Icrop (H, W)) (5)

[0035] (2.4) Adjust camera parameters:

[0036] K new =S×K (6)

[0037] Where K represents the camera's intrinsic parameters, and S is a coefficient or adjustment matrix used to normalize the adjusted image.

[0038] I norm =(I resize -μ) / σ (7)

[0039] Where μ is the mean of the image intensity values ​​and σ is the standard deviation of the image intensity values.

[0040] (3) Predefined rotated sample set;

[0041] Using attitude representation and predefined rotation:

[0042] (3.1) The system uses Euler angles to represent 3D rotations, and each rotation is defined by three angles. Where ψ is the yaw angle and θ is the pitch angle. It is the roll angle;

[0043] (3.2) N uniformly distributed rotational sample sets are predefined: {R i}(i=1to N);

[0044] (4) Constructing a Gaussian Mixture Model (GMM) and sampling using a Gaussian Mixture Model (GMM):

[0045] (4.1) For each predefined rotation R i The system calculates a score using a classification network:

[0046] s(R i )=f(I,R i M) (8)

[0047] Where f is the scoring network, I is the input image, and M is the 3D model of the object;

[0048] (4.2) Select the k samples with the highest scores from N predefined rotations:

[0049] {R i}(i=1,2,3...k)=argtopk(s(R i (9)

[0050] (4.3) Construct the GMM density distribution using these k high-scoring samples:

[0051]

[0052] Where: X is a point in Euler angle space, μ i =R i Let be the Euler angles of the i-th high-scoring sample, Σ be the predefined covariance matrix, and N(X|μ,Σ) represent a multidimensional normal distribution.

[0053] (4.4) Sample M new rotations from the constructed GMM distribution:

[0054] {R' j}(j=1,2,3,...M)~p(X) (11)

[0055] Including the following constraints:

[0056] A. Projection constraints, using a perspective projection model:

[0057] x=K[R|t]X (12)

[0058] Where x is the 2D image coordinate, X is the 3D model point, K is the camera intrinsic parameter matrix, and [R|t] is the transformation matrix to be estimated.

[0059] B. Bounding box constraints:

[0060] x min ≤proj(RX+t)≤x max (13)

[0061] y min ≤proj(RX+t)≤y max (14)

[0062] Where proj(RX+t) is the projection operation, R is the rotation matrix, X is the original coordinates, and t is the translation vector.

[0063] C. Depth Constraints:

[0064] z min ≤t z ≤z max (15)

[0065] (5) Candidate pose generation:

[0066] Scores are calculated for all 2M pose samples (M predefined + M GMM samples):

[0067] s i =s([R i |t i (16)

[0068] Select the n poses with the highest scores as the candidate set:

[0069] {P i}(i=1,2,3...n)=argtopn(s i (17)

[0070] Where P i =[R i |t i ] is the i-th candidate pose;

[0071] (6) Rendering image generation;

[0072] Given a 3D model M of the target object and the currently estimated 6D pose P i =[R i |t i The rendered image is generated through the following steps:

[0073] (6.1) Transfer each point X in the 3D model M i Transform to camera coordinate system:

[0074] X′ i =RX i +t 18)

[0075] (6.2) Project it onto the image plane using a perspective projection model:

[0076] x i =KX′ i (19)

[0077] Where, x i These are the coordinates of the projected 2D image.

[0078] (6.3) During the projection process, the depth information of each pixel is recorded to generate a depth map.

[0079]

[0080] Where (u, v) are pixel coordinates, X′ i (z) is the Z coordinate of the transformed 3D point.

[0081] (6.4) Determine whether each pixel belongs to the target object and generate the object mask M. r ∈{0,1} H×W :

[0082]

[0083] (6.5) Based on the texture information and lighting conditions of the 3D model M, synthesize the RGB image Ir∈R. H×W×3 The final rendered image includes:

[0084] RGB image: Ir∈R H×W×3 ;

[0085] Depth map: Dr∈R H×W ;

[0086] Object mask: Mr∈{0,1} H×W ;

[0087] (7) Feature extraction;

[0088] Extracting features using a feature encoder:

[0089] Input image features: FI = Encoder(I);

[0090] Rendering image features: Fr = Encoder(Ir);

[0091] (8) Construction of correlation volume;

[0092] Constructing the correlation volume:

[0093] C i =FI i ×Fr i ^T∈RH i ×W i ×H i ×W i (twenty two)

[0094] Where i represents the i-th layer of the feature pyramid.

[0095] (9) GRU feature processing;

[0096] For the j-th internal update, the GRU processes the following input: the previous time-step hidden state: h j -1; Correlation characteristic: r j -1; Contextual feature: c;

[0097] (10) Optical flow prediction and confidence calculation;

[0098] Optical flow upsampling:

[0099] F j =Upsample(f j ,u j ) (twenty three)

[0100] Upsample uses a convex combination method with a 3×3 neighborhood.

[0101] Confidence calculation:

[0102] W j =Upsample(cj Upsample(s) j ) (twenty four)

[0103] (11) PnP optimization;

[0104] (11.1) Attitude update:

[0105] Calculate the 3D coordinates of the pixel based on the current depth Dr and the camera intrinsic parameter K:

[0106] [xyz]^T=R^T(K^(-1)Dr(u,v)[uv 1]^Tt) (25)

[0107] The Levenberg-Marquardt algorithm is used to minimize the weighted projection error. The goal is to find the optimal 6D pose (rotation matrix R and translation vector t) that minimizes the error between the projected 3D points and the actual 2D image pixels.

[0108] (11.2) Relevance search:

[0109] Calculate attitude-guided optical flow:

[0110] Flow=π(PjX)-π(Pk-1X) (26)

[0111] Where π is the projection function;

[0112] Using this flow field to sample the correlated volume:

[0113] r j =Sample(C,Flow) (27)

[0114] Cascade structure processing:

[0115] Using the correlation volume C (i) And the optical flow estimation F of the previous layer (i+1) A correlation search is performed to obtain the updated optical flow F(i):

[0116] F (i) =FlowUpdate(C (i) ,F (i+1) (28)

[0117] Based on the updated optical flow F(i), the 3D coordinates of the pixels are calculated, and the Levenberg-Marquardt algorithm is used to minimize the weighted projection error to obtain a more accurate pose estimate P. (i) :

[0118] P (i) =PoseUpdate(F (i),P (i+1) (29)

[0119] Starting from step (6) where the rendered image is generated, repeat the above steps until the lowest level (finest scale) feature map is processed to obtain the final high-precision pose estimate P. (i) .

[0120] (12) After the above series of processing steps, the system will output the precise 6D pose of the target object.

[0121] The technical solution provided by this invention has the following technical effects:

[0122] (1) This invention significantly improves the efficiency and accuracy of attitude estimation through an innovative iterative optimization strategy based on optical flow. Compared with existing technologies, this scheme only needs to generate a single-view rendering image in each iteration, greatly reducing computational overhead. By combining a differentiable PnP solver with optical flow prediction, the system can fully utilize 3D geometric constraint information, ensuring both efficient processing and accurate estimation results. This efficient shape-aware mechanism not only solves the problem of excessive computational resource consumption in traditional methods but also maintains high accuracy in attitude estimation, providing a more practical technical solution for industrial automation applications.

[0123] (2) The cascaded network architecture and confidence decomposition mechanism of this invention greatly improve the robustness and generalization ability of the algorithm. The cascaded design enables the system to effectively utilize multi-scale feature information and achieve progressive optimization from coarse to fine. The innovative confidence decomposition strategy divides the confidence into two key factors: determinism and pose sensitivity, enabling the system to better cope with various interference factors in complex scenes, such as occlusion and illumination changes. This design not only significantly improves the reliability of the estimation results, but also enables this invention to maintain stable high performance in different industrial scenarios, demonstrating excellent environmental adaptability. Attached Figure Description

[0124] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0125] The specific technical solutions of the present invention will be described in conjunction with the accompanying drawings and embodiments.

[0126] Example 1

[0127] This embodiment takes the example of an industrial robot picking up unknown parts (industrial automation scenario): On an automobile assembly line, a robot needs to pick up a batch of newly designed gearbox gears from a bin (without historical training data). The gears have a metal surface with weak texture and slight obstruction.

[0128] According to such Figure 1 The process and implementation steps are as follows:

[0129] (1) Input the CAD model (3D model file) of the gear and the RGB image of the hopper (taken by a fixed camera, with the intrinsic parameter matrix K calibrated).

[0130] (2) Use the CNOS detector (based on the SAM Transformer architecture) to locate the gear in the bin image, output the bounding box bi = (120, 80, 200, 150), crop and resize to a fixed resolution (H = 480, W = 640).

[0131] (3) Predefine 64 uniform rotation samples (e.g., ψ∈[0°, 360°], θ∈[-90°, 90°]), filter out the top 5 high-scoring rotations through a scoring network, and then supplement 5 rotations based on GMM sampling to generate a total of 10 candidate poses.

[0132] (4) For each candidate pose, render the RGB image, depth map, and mask of the gear (e.g., ...). Figure 1 Process steps 6-7).

[0133] (5) The pixel correspondence between the rendered image and the real image is calculated by optical flow prediction network (GRU processing), and the pose is updated iteratively using PnP optimization (Levenberg-Marquardt algorithm). Finally, after 3 cascades, the pose converges to a 6D pose P = [R|t] with an error of <1mm.

[0134] (6) The robot adjusts its robotic arm according to the output 6D posture and successfully completes gear grasping without retraining the model.

[0135] Example 2

[0136] This embodiment takes the real-time overlay of furniture models (augmented reality scene) on AR glasses as an example: When a user observes the living room through AR glasses, a virtual sofa (new design model) needs to be overlaid onto the real environment in real time. The sofa surface is a solid color fabric with few textures and is greatly affected by ambient light.

[0137] According to such Figure 1 The process and implementation steps are as follows:

[0138] (1) Input the 3D model of the sofa (OBJ format) and the RGB image (resolution 1920×1080, internal parameters have been calibrated) captured in real time by the AR glasses camera.

[0139] (2) When the user moves their head, the CNOS detector detects the position of the sofa in the image each frame (e.g., bounding box bi = (400, 300, 600, 400)), and after cropping, it is resized to H = 384, W = 512.

[0140] (3) Predefine 36 rotation samples (covering common sofa placement angles), combine with GMM sampling to generate 20 candidate poses, and filter low confidence areas caused by lighting changes through confidence decomposition mechanism (patent step 10).

[0141] (4) Render the RGB-D image of the sofa, use the optical flow prediction network to match the edge features of the real floor and the virtual sofa, and adjust the posture through PnP optimization to make the virtual sofa fit the real floor seamlessly (error <0.5°).

[0142] (5) The AR glasses continuously output the 6D posture of the sofa, and the virtual model is stably superimposed. Even if the user moves or the lighting changes, the system remains robust.

[0143] This invention proposes a shape-aware mechanism based on optical flow. This mechanism cleverly integrates the 3D geometric information of an object into the pose estimation process by predicting the dense correspondence between the input and rendered images. The key lies in the system's use of a differentiable PnP solver, which can directly recover the accurate 6D pose from the 2D correspondence. This design not only avoids the computational burden of extensive view rendering required in traditional methods but also achieves efficient utilization of 3D shape information, fundamentally solving the trade-off between computational efficiency and estimation accuracy.

[0144] This invention designs a confidence decomposition strategy, splitting confidence into two key components: determinism and pose sensitivity. This decomposition enables the system to simultaneously focus on and process occluded regions and key feature regions of objects in an image. In this way, the system can effectively identify and cope with various interference factors in complex scenes, such as partial occlusion or changes in illumination. This design significantly improves the adaptability and reliability of the algorithm in real-world industrial environments, enabling the system to maintain stable estimation performance under various complex conditions.

Claims

1. A general 6D pose estimation method based on shape constraints, characterized in that, Includes the following steps: (1) Input image and render image; (2) Target detection and image cropping; Object detection is performed using a CNOS detector; the image is cropped and transformed, and the cropped image is resized to a specified fixed resolution using the resize function; camera parameters are adjusted. (3) Predefined rotated sample set; Euler angles are used to represent 3D rotational poses and a predefined set of N uniformly distributed rotational samples are used. (4) Construct a Gaussian mixture model and sample using the Gaussian mixture model; For each predefined rotation, a score is calculated using a classification network. The k samples with the highest scores are selected to construct a GMM density distribution, from which M new rotations are sampled. (5) Candidate pose generation; Calculate scores for all 2M pose samples, including M predefined poses and M GMM samples; select the n poses with the highest scores as the candidate set; (6) Rendering image generation; Given a 3D model M of the target object and the currently estimated 6D pose P i =[R i |t i ], generate a rendered image; (7) Feature extraction; Extracting features using a feature encoder: Input image features: FI = Encoder(I); Rendering image features: Fr = Encoder(Ir); (8) Construction of correlation volume; (9) GRU feature processing; For the j-th internal update, the GRU processes the following input: the previous time-step hidden state: h j -1; Correlation characteristic: r j -1; Contextual feature: c; (10) Optical flow prediction and confidence calculation; (11) PnP optimization; Using the correlation volume and the optical flow estimation of the previous layer, correlation lookup is performed to obtain the updated optical flow; Based on the updated optical flow, the 3D coordinates of the pixels are calculated, and the Levenberg-Marquardt algorithm is used to minimize the weighted projection error to obtain a more accurate pose estimate. The process repeats from step (6) to the generation of the rendered image until the lowest layer of feature maps is processed to obtain the final high-precision pose estimate. (12) Output the precise 6D pose of the target object.

2. The general 6D pose estimation method based on shape constraints according to claim 1, characterized in that, Step (1) specifically includes: Input the 3D model file of the target object and the RGB image of the target object whose pose needs to be estimated; For input image calibration, the camera intrinsic parameter matrix K is in the form of: Where f x ,f y It is the focal length parameter, (c x ,c y () are the coordinates of the principal point.

3. The general 6D pose estimation method based on shape constraints according to claim 1, characterized in that, Step (2) specifically includes: (2.1) Object detection is performed using a CNOS detector: (2.1.1) Use the pre-trained base model SAM to extract deep features from the image; Input image First, it is divided into multiple fixed-size patches. Each patch is linearly embedded into a high-dimensional vector, forming an embedding vector. Where N is the number of patches, and D is the embedding dimension: E = PatchEmbedding(I) (1) The embedding vector E is passed through a multi-layer Transformer encoder, with the output of each layer considered as a deep feature at a different level. Let L layers of Transformer encoders be used, then the output of the l-th layer is represented as: F (l) =TransformerEncoder (l) (F (l-1) ) (2) Where F(0) = E represents the initial patch embedding; The final deep feature map F is the output of the last Transformer encoder layer, processed by LayerNorm normalization: F=LayerNorm(F (L) )(3) (2.1.2) Detection Head Network: Contains two branches: Bounding box regression branch: Outputs the 2D bounding box coordinates (x, y, w, h) of the object; Category classification branch: Predicts the probability distribution of object categories; (2.1.3) The detection results are represented as a list: D = {d i }, where each detection di contains: Bounding box coordinates: b i =(x i ,y i ,w i ,h i ); Category tag: c i ; Detection confidence score: s i ; (2.2) Image cropping transformation: I crop =Crop(I,b i ) (4) The Crop function is based on the bounding box coordinates b. i Extract the target region from the original image I; (2.3) Apply the resize function to the cropped image to adjust its size to a specified fixed resolution, i.e., height H and width W: I resize =Resize(I crop ,(H,W)) (5) (2.4) Adjust camera parameters: K new =S×K (6) Where K represents the camera's intrinsic parameters, and S is a coefficient or adjustment matrix used to standardize the adjusted image. I norm =(I resize -m) / s (7) Where μ is the mean of the image intensity values ​​and σ is the standard deviation of the image intensity values.

4. The general 6D pose estimation method based on shape constraints according to claim 3, characterized in that, Step (3) specifically includes: (3.1) The system uses Euler angles to represent 3D rotations, and each rotation is defined by three angles. Where ψ is the yaw angle and θ is the pitch angle. It is the roll angle; (3.2) Predefine N uniformly distributed rotated sample sets: {R i (i = 1 to N).

5. The general 6D pose estimation method based on shape constraints according to claim 4, characterized in that, Step (4) specifically includes: (4.1) For each predefined rotation R i The system calculates a score using a classification network: s(R i )=f(I,R i ,M) (8) Where f is the scoring network, I is the input image, and M is the 3D model of the object; (4.2) Select the k samples with the highest scores from N predefined rotations: {R i }(i=1,2,3...k)=argtopk(s(R i )) (9) (4.3) Construct the GMM density distribution using these k high-scoring samples: Where: X is a point in Euler angle space, μ i =R i Let X be the Euler angle of the i-th high-scoring sample, Σ be the predefined covariance matrix, and N(X|μ,Σ) represent a multidimensional normal distribution. (4.4) Sample M new rotations from the constructed GMM distribution: {R’ j {(j=1,2,3,...M)~p(X) (11) Including the following constraints: A. Projection constraints, using a perspective projection model: x=K[R|t]X(12) Where x is the 2D image coordinate, X is the 3D model point, K is the camera intrinsic parameter matrix, and [R|t] is the transformation matrix to be estimated; B. Bounding box constraints: x min ≤proj(RX+t)≤x max (13) y min ≤proj(RX+t)≤y max (14) Where proj(RX+t) is the projection operation, R is the rotation matrix, X is the original coordinates, and t is the translation vector; C. Depth Constraints: With min ≤t z ≤of max (15)。 6. The general 6D pose estimation method based on shape constraints according to claim 5, characterized in that, Step (5) specifically includes: Scores were calculated for all 2M attitude samples: s i =s([R i |t i ]) (16) Select the n poses with the highest scores as the candidate set: {P i }(i=1,2,3...n)=argtopn(s i ) (17) Where P i =[R i |t i ] is the i-th candidate pose.

7. The general 6D pose estimation method based on shape constraints according to claim 6, characterized in that, Step (6) specifically includes: (6.1) Transfer each point X in the 3D model M i Transform to camera coordinate system: X' i =RX i +t 18) (6.2) Project it onto the image plane using a perspective projection model: x i =KX' i (19) Where, x i These are the coordinates of the projected 2D image; (6.3) During the projection process, the depth information of each pixel is recorded to generate a depth map. Where (u, v) are pixel coordinates, X' i (z) is the Z-coordinate of the transformed 3D point; (6.4) Determine whether each pixel belongs to the target object and generate the object mask M. r ∈{0,1} H×W : (6.5) Based on the texture information and lighting conditions of the 3D model M, synthesize the RGB image Ir∈R. H×W×3 The final rendered image includes: RGB image: Ir∈R H×W×3 ; Depth map: Dr∈R H×W ; Object mask: Mr∈{0,1} H×W .

8. The general 6D pose estimation method based on shape constraints according to claim 7, characterized in that, Step (8) Construct the correlation volume: C i =FI i ×Fr i ^T∈RH i ×W i ×H i ×W i (22) Where i represents the i-th layer of the feature pyramid.

9. The general 6D pose estimation method based on shape constraints according to claim 8, characterized in that, Step (10) specifically includes: Optical flow upsampling: F j =Upsample(f j ,and j ) (23) Upsample uses a convex combination method with a 3×3 neighborhood; Confidence calculation: W j =Upsample(c j )*Upsample(s j ) (24)。 10. The general 6D pose estimation method based on shape constraints according to claim 9, characterized in that, Step (11) specifically includes: (11.1) Attitude update: Calculate the 3D coordinates of the pixel based on the current depth Dr and the camera intrinsic parameter K: [xyz]^T=R^T(K^(-1)Dr(u,v)[uv 1]^Tt) (25) The Levenberg-Marquardt algorithm is used to minimize the weighted projection error. The goal is to find the optimal 6D pose (rotation matrix R and translation vector t) that minimizes the error between the projected 3D points and the actual 2D image pixels. (11.2) Relevance search: Calculate attitude-guided optical flow: Flow=π(PjX)-π(Pk-1X) (26) Where π is the projection function; Using this flow field to sample the correlated volume: r j =Sample(C,Flow) (27) Cascade structure processing: Using the correlation volume C (i) And the optical flow estimation F of the previous layer (i+1) A correlation search is performed to obtain the updated optical flow F(i): F (i) =FlowUpdate(C (i) ,F (i+1) ) (28) Based on the updated optical flow F(i), the 3D coordinates of the pixels are calculated, and the Levenberg-Marquardt algorithm is used to minimize the weighted projection error to obtain a more accurate pose estimate P. (i) : P (i) =PoseUpdate(F (i) ,P (i+1) ) (29) Starting from step (6) where the image is rendered, repeat the above steps until the lowest layer feature map is processed to obtain the final high-precision pose estimate P. (i) .

Images