Workpiece pose estimation method based on comparative learning and iterative optimization

Abstract

The invention relates to the field of machine vision detection, in particular to a method for estimating the pose of a target workpiece through a CAD model. The method comprises the following steps of: rendering by using a CAD (Computer Aided Design) model of a workpiece to generate a pose labeling data set and a pose template image; performing feature extraction and similarity calculation on the template image and the target image through a trained template matching network based on comparative learning to obtain initial pose estimation of the target; and optimizing the initial pose estimation through the trained pose iterative network to obtain the final pose of the target object. According to the method, pose estimation can be efficiently carried out on industrial parts in production, compared with an existing method, the reasoning speed is greatly increased, the production efficiency of enterprises can be effectively improved, and the cost of manually detecting the pose state of the workpiece is reduced.

Description

Technical Field

[0001] The present invention relates to the field of machine vision detection, and in particular to a workpiece pose estimation method based on contrast learning and iterative optimization. Background Art

[0002] In industrial manufacturing, modern factories that rely on automated production typically utilize highly sophisticated equipment and production lines to improve production efficiency and product consistency. However, even in automated environments, production equipment can exhibit certain errors due to various factors. This can lead to missing or misaligned parts on the production line. Failure to promptly detect workpiece positioning errors during production can compromise product quality, reduce production efficiency, and even damage processing equipment. Traditional manufacturing relies on manual monitoring to detect product anomalies during production. Workers regularly inspect the production line and rely on their experience to determine whether equipment adjustments or other measures are necessary. However, this approach presents several potential problems. First, worker judgment can be influenced by subjective factors, leading to misjudgments or overlooked issues. Second, manual monitoring requires significant human resources, increasing production costs. Finally, workers in complex production lines may not be able to monitor production status in real time.

[0003] To address these issues, cameras can be placed at designated locations on the production line to capture real-time images. These images can then be analyzed using pose estimation methods to identify the pose of workpieces during processing. By processing the pose recognition results, anomalies such as missing or misplaced workpieces on the production line can be detected and addressed accordingly, minimizing losses caused by these anomalies.

[0004] With the rapid development of robotics, autonomous driving and other fields, many methods have been proposed in the field of pose estimation at home and abroad. Among them, SSD-6D uses a single convolutional neural network (CNN) to estimate the six-degree-of-freedom pose of the target object in the scene. CDPN decouples rotation and translation in pose estimation to improve accuracy, robustness, flexibility and efficiency. Although existing end-to-end pose detection methods have their own advantages, they usually learn implicit representations of the objects to be detected through a large amount of labeled data. Therefore, it is difficult to cope with the flexible changes in the recognition targets in actual production scenarios and the difficulty in obtaining pose annotation datasets. Summary of the Invention

[0005] The purpose of the present invention is to provide a workpiece pose estimation method based on contrast learning and iterative optimization, which can automatically identify the pose state of workpieces on the production line and accelerate the speed of pose tracking and the accuracy of recognition.

[0006] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0007] A workpiece pose estimation method based on contrastive learning and iterative optimization includes the following steps:

[0008] 1) Use CAD model renderings of industrial parts to generate pose annotation datasets for training and template images for similarity matching;

[0009] 2) Select positive and negative sample pairs from the template image and pose annotation dataset based on the pose information, and train a template matching network model based on similarity comparison through comparative learning;

[0010] 3) Use image acquisition equipment to collect video data from the production process, intercept key frames in the video stream as images to be detected, and use the YoloX object detector to obtain the two-dimensional bounding box of the workpiece to be detected in the image;

[0011] 4) Cropping the image to be detected using a two-dimensional bounding box to obtain a local image, using the template matching network model to calculate the similarity score map and select the template image with the highest similarity;

[0012] 5) Using the pose information of the selected template image to calculate the rough pose of the target workpiece as the initial value for pose optimization;

[0013] 6) Use iterative matching to optimize the initial pose to obtain the final pose.

[0014] The step 1) comprises the following steps:

[0015] 1.1) Obtaining feature information of the workpiece from the workpiece CAD model, wherein the feature information includes the diameter of the workpiece (i.e., the diagonal length of the three-dimensional bounding box) and rotational symmetry information;

[0016] 1.2) Set the model material parameters used to render the simulation image to simulate the material properties of the actual workpiece;

[0017] 1.3) Use different materials to set the background wall and ground environment of the scene, and simulate the actual lighting conditions to set the light source;

[0018] 1.4) Randomly place the loaded model in the set scene, randomly sample camera positions above the ground, and determine whether the field of view of each camera meets the requirements of including the set number of objects to be measured, and whether there is no occlusion greater than the threshold between the objects. If not, the camera viewpoint is discarded;

[0019] 1.5) Record the RGB images, depth images, mask images obtained by different rendering methods, as well as the target object category, detection box and pose annotation information to build a pose annotation dataset;

[0020] 1.6) Generate a background-free multi-view rendering image of a single target part and a corresponding binary mask image as a template image through spherical viewpoint rendering.

[0021] Described step 2) comprises the following steps:

[0022] 2.1) Select sample images from the pose annotation dataset and template images, and calculate the angle between the object center and the camera viewpoint of the two images based on their pose information. Sample pairs with angles less than 15° are considered positive sample pairs, and the rest are considered negative sample pairs;

[0023] 2.2) Using ResNet50 as the basic network architecture, the template matching network is trained by contrastive learning, where the loss function used is For InfoNCE:

[0024]

[0025] Among them, N is the number of sample pairs, sim(q i ,t i ) is the input image q obtained by the feature extraction network i and template image feature map t i The cosine similarity between .

[0026] The step 4) comprises the following steps:

[0027] 4.1) Use a 2D bounding box to crop the original input image to be detected, and input the cropped local image and template image into the trained template matching network model;

[0028] 4.2) Calculate the similarity score map through the template matching network model;

[0029] 4.3) Mask the similarity score map using the binary mask image in the template image pose annotation dataset. Use the area marked by the mask image as the target area for calculating the mean, and remove points in the target area with a similarity less than the threshold.

[0030] 4.4) Calculate the average cosine similarity within the remaining target area;

[0031] 4.5) Select the template with the highest average cosine similarity among all template images as the matching template.

[0032] The step 5) comprises the following steps:

[0033] 5.1) Calculate the estimated distance between the center point of the object in the original input image and the camera by matching the annotation information of the template

[0034]

[0035] Among them, t temp,z is the template image t temp The distance between the center point of the object and the camera viewpoint, ||bb temp || and ||bb input || are the diagonal lengths of the two-dimensional bounding boxes of the objects in the template image and the input image, respectively, and f input and f temp are the focal length of the camera used for the input image and the focal length of the virtual camera used for template image rendering;

[0036] 5.2) Calculate the update amount of translation from the center point of the object in the template to the center point of the actual object

[0037]

[0038] Among them, K (.) Represents the camera's intrinsic parameter matrix, bb (.),c Represents the center point of the bounding box, the subscript input represents the input image, and the subscript temp represents the template image;

[0039] 5.3) Calculate the translation vector of the initial rough pose

[0040]

[0041] 5.4) Repair the rotation component error caused by the different positions of objects in the template image and the input image to obtain the rotation matrix of the rough pose

[0042]

[0043]

[0044] Among them, R y (α y ) and R x (α x ) represent the transformation of the camera's own y-axis and x-axis rotation, and the rotation angles are α y and α x , R temp The rotation matrix of the template annotation.

[0045] The step 6) is specifically as follows:

[0046] The rough pose is used as the initial value of the pose iteration. The initial pose and the CAD model of the workpiece are superimposed and rendered on the object image. The pose iterative matching network extracts image features from the rendered image and the scene image and outputs the update amount of the translation vector (v x ,v y ,v z ), and the update amount of the rotation matrix e1, e2, after multiple iterations, the translation vector and rotation matrix are obtained as the optimized pose estimation results, where the pose is updated as follows:

[0047]

[0048]

[0049]

[0050]

[0051] in, are the translation components in the updated x, y, and z directions, respectively. is the updated rotation component, and is the focal length of the camera, and R(e1,e2) is the rotation update obtained by orthogonalizing the basis defined by the predicted rotation vectors e1 and e2.

[0052] The present invention has the following beneficial effects and advantages:

[0053] 1. The present invention can automatically identify the position and posture of workpieces on the production line, reduce manual intervention, and help improve enterprise production efficiency;

[0054] 2. This invention uses the workpiece CAD model to generate a pose dataset and template image through a physically based rendering method, and adopts a template matching method based on contrastive learning to estimate the pose. This can effectively address the problem of flexible changes in recognition targets in actual production scenarios and the difficulty in obtaining pose annotation datasets;

[0055] 3. The present invention adopts a sparse template solution, which can avoid the efficiency problem of using dense templates for matching and ensure the accuracy of the final result through iterative optimization of posture. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 This is a flow chart of the overall use of the system of the present invention;

[0058] Figure 2 This is a schematic diagram of the template viewpoint of the present invention;

[0059] Figure 3 This is a flow chart of pose estimation of the present invention;

[0060] Figure 4 This is a flow chart of the real-time posture tracking of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0062] The workpiece pose estimation method based on contrastive learning and iterative optimization includes the following steps:

[0063] 1) Generate pose annotation datasets for training and template images for similarity matching by rendering CAD models of industrial parts;

[0064] 2) Select positive and negative sample pairs from the template image and pose dataset based on the pose information, and train the template matching network model based on similarity comparison through comparative learning;

[0065] 3) Use image acquisition equipment to collect video data from the production process, intercept key frames in the video stream as images to be detected, and use the YoloX object detector to obtain the two-dimensional bounding box of the workpiece to be detected in the image;

[0066] 4) Cropping the local image using a 2D bounding box, calculating the similarity score map using a template matching network, and selecting the template image with the highest similarity;

[0067] 5) Use the pose information of the template image to calculate the rough pose of the target workpiece as the initial value for pose optimization, including the translation vector and rotation matrix;

[0068] 6) Use iterative matching to optimize the initial pose to obtain the final pose.

[0069] The step 1) comprises the following steps:

[0070] 1.1) Obtain workpiece feature information from the workpiece CAD model, including the workpiece diameter (diagonal length of the 3D bounding box) and rotational symmetry information;

[0071] 1.2) Set the model material parameters used to render the simulation image to simulate the material properties of the actual workpiece;

[0072] 1.3) Use different materials to set the background wall and ground environment of the scene, and simulate the actual lighting conditions to set the light source;

[0073] 1.4) Randomly place the loaded model in the scene and randomly sample the camera position above the ground, while ensuring that the field of view contains a certain number of objects to be measured and there is no significant occlusion between objects. Discard viewpoints that do not meet the requirements;

[0074] 1.5) Record the resulting RGB image, depth image, mask image, and the target's object category, detection bounding box, and pose annotation information;

[0075] 1.6) Generate a background-free multi-view rendering image of a single target part and a corresponding binary mask image as a template image through spherical viewpoint rendering.

[0076] Described step 2) comprises the following steps:

[0077] 2.1) Calculate the pose error using the pose information and symmetry information of the dataset and template image. Pairs with similar pose errors are considered positive, and the rest are considered negative.

[0078] 2.2) Using ResNet50 as the base network architecture, the template matching network is trained through contrastive learning, using the InfoNCE loss function:

[0079]

[0080] Among them, sim(q i ,t i ) is the cosine similarity between the input image and the template image feature map obtained by the feature extraction network.

[0081] The step 4) comprises the following steps:

[0082] 4.1) Using the two-dimensional bounding box in step 3), the original input image is cropped, and the cropped local image and the template image are input into the template matching network trained in step 2);

[0083] 4.2) Obtain a similarity score map through the pose initial estimation network calculation;

[0084] 4.3) Mask the similarity score map using the binary mask image of the template image to determine the target area to be matched, and remove points with similarity less than the threshold to reduce the impact of possible occlusion on the similarity matching results;

[0085] 4.4) Calculate the average similarity within the remaining target area;

[0086] 4.5) Select the template with the highest average similarity among all template images as the matching template.

[0087] The step 5) comprises the following steps:

[0088] 5.1) Calculate the estimated distance between the center point of the object in the original input image and the camera by matching the annotation information of the template

[0089]

[0090] where t temp,z is the template image t temp,z The distance between the center point of the object and the camera viewpoint, ||bb temp || and ||bb input || is the diagonal length of the two-dimensional bounding box of the object in the template image and the input image, f input and f temp The focal length of the camera used for the input image and the focal length of the virtual camera used for template image rendering.

[0091] 5.2) Calculate the update amount of translation from the center point of the object in the template to the center point of the actual object

[0092]

[0093] Among them, K (.) Represents the camera's intrinsic parameter matrix, bb (.),c Indicates the center point of the bounding box;

[0094] 5.3) Calculate the translation vector of the initial pose

[0095]

[0096] 5.4) Repair the rotation component error caused by the different positions of the objects in the template image and the input image to obtain the rotation matrix of the initial pose

[0097]

[0098]

[0099] Among them, R y (α y ) and R x (α x) represent the transformation of the camera's own y-axis and x-axis rotation, and the rotation angles are α y and α x , R temp The rotation matrix of the template annotation.

[0100] The step 6) is specifically as follows:

[0101] The rough pose estimate calculated in step 5) is used as the initial value of the pose iteration. The initial pose and the CAD model of the workpiece are superimposed and rendered on the object image. The pose iterative matching network extracts image features from the rendered image and the scene image and outputs the updated translation vector (v x ,v y ,v z ), and the update amount R(e1,e2) of the rotation matrix represented by two vectors e1 and e2, calculate the translation vector after the kth iteration and the rotation matrix

[0102]

[0103]

[0104]

[0105]

[0106] The translation vector obtained after multiple iterations and the rotation matrix As the final result of pose estimation.

[0107] Example

[0108] Figure 1 The overall use flow chart of the algorithm provided in the embodiment of the present invention is as follows: Figure 1 As shown in the figure, the overall usage process in the entire system is divided into the following steps:

[0109] Step 1: Use the CAD model rendering of industrial parts to generate a pose annotation dataset for training and a template image for similarity matching.

[0110] In the embodiment, the rendering model material parameters are set according to the material properties of the actual workpiece to be detected (such as metal, plastic), and the layout and lighting effects of the actual production scene are simulated. Among them, the model material parameters are specifically the surface reflectivity, metallicity and surface roughness coefficient. During the implementation of the algorithm, these parameters need to be set to random values ​​within a certain range to simulate the material properties of the actual workpiece. The pictures generated and saved by the data set are specifically RGB images, depth images, and mask images. The annotation information includes the object category, detection frame and six-degree-of-freedom pose information (translation vector and rotation matrix) of the target. In addition, it is necessary to save the feature information of the workpiece for subsequent calculations, the diameter of the workpiece (diagonal length of the three-dimensional bounding box) and rotational symmetry information. Different materials are used in the embodiment to simulate the background wall and ground environment in different scenes. A surface light source at a fixed position above the ground and a point light source at a random position within the spatial range are set, and the intensity of the light source is set to a random value.

[0111] In the embodiment, the basic strategy for selecting the viewpoint of the template image is uniform sampling on the sphere with the target object as the center, specifically the vertices of the regular icosahedron after multiple subdivisions, and each time the edges of the polyhedron are subdivided, and the viewpoint is ensured to be on the sphere, specifically as follows Figure 2 As shown in the figure, the icosahedron after two subdivisions has 162 vertices, and eight in-plane rotations of each vertex are used as templates. More subdivisions and more in-plane rotations can increase the accuracy of the initial pose estimate, but will significantly increase inference time.

[0112] Step 2: construct positive and negative sample pairs and train the template matching network model based on similarity comparison through contrastive learning.

[0113] Among them, the sample pair is defined as a template image and a local cropped image, and the local cropped image is cropped from the training image according to the real two-dimensional detection frame of the target workpiece.

[0114] In this embodiment, the template matching network uses ResNet50 as the basic network architecture, and trains the initial pose estimation network through contrastive learning. The loss function used is InfoNCE:

[0115]

[0116] Among them, sim(q i ,t i ) is the cosine similarity between the input image and the template image feature map obtained by the feature extraction network, τ is the temperature hyperparameter, which is used to adjust the model's ability to distinguish negative samples. This method sets τ = 0.1.

[0117] In the embodiment, to improve the generalization performance of the model, a series of data enhancements including Gaussian blur, contrast adjustment, brightness adjustment, and image sharpening were performed on the input image during training. The model was trained using the Adam optimizer with a learning rate set to 1e-4.

[0118] Step 3: Use the trained model to build a posture tracking framework to perform real-time analysis of the workpiece posture.

[0119] Figure 3 This is a flowchart for pose estimation of a single image, such as Figure 3 As shown in Figure 2, the pose estimation process in the entire system is divided into the following steps:

[0120] 1) Use the image captured by the camera as input to the pose estimation reasoning process;

[0121] 2) In the object detection stage, an object detector is used to obtain the two-dimensional bounding box of the workpiece to be detected in the image and crop it;

[0122] 3) In the initial pose estimation stage, the similarity score map is calculated through the template matching network and the template image with the highest similarity is selected. The initial pose value of the target relative to the camera is calculated based on the information of the template image and the original image.

[0123] 4) In the pose optimization stage, the initial pose is optimized through the pose iterative optimization network to obtain the final six-degree-of-freedom pose;

[0124] In the embodiment, YoloX is used as the target detector. During the implementation, any target detection model can be used, and the output model of the two-dimensional detection area is used as the input of the subsequent pose estimation process. In order to reduce the amount of calculation during inference, the feature extractor in the template matching network is used to pre-calculate the template image features and save them as a template database. The present invention adopts a real-time pose tracking framework to achieve target pose tracking in a production environment. The framework uses the final pose of the previous frame of the same target object as the initial pose of the current frame, which is directly used as the input of the pose optimization stage. By utilizing the continuity of the object pose in the time domain, the dynamic performance of real-time pose tracking is improved.

[0125] like Figure 4 As shown, the overall process of the target tracking framework used in the embodiment is:

[0126] 1) The camera collects image data during the processing process and intermittently intercepts detection frames from the video stream based on the machine performance and workpiece movement speed;

[0127] 2) Obtain the target category and two-dimensional detection box in the image through the target detector;

[0128] 3) Match the target category and position in the image with the detection information of the previous detection frame to determine whether it is the initial frame of the current target. If it is not the initial frame, execute step 5) and use the result of the previous detection frame as the initial pose input of the iterative optimization network of the current frame;

[0129] 4) Obtain the initial pose through the template matching network;

[0130] 5) Optimize the initial pose through the pose iterative optimization network;

[0131] 6) Save the optimized pose results, jump to step 1), and perform position and posture abnormality judgment on the stored pose results concurrently as needed.

[0132] Finally, after identifying the workpiece's pose through the pose detection process and obtaining its translation vector and rotation matrix relative to the camera, the identified results can be compared with the desired correct pose to determine if the workpiece is in an abnormal state. For workpieces with rotational symmetry, the transformation matrix between the predicted rotation matrix and the correct pose's rotation matrix can be calculated. The determination of whether the workpiece is in an abnormal pose can be made by determining whether the transformation matrix conforms to the workpiece's rotational symmetry.

[0133] The present invention also includes a system for estimating the pose of industrial parts, comprising: a processor, a memory, and an image acquisition device arranged on a production line. The memory stores a program, and the processor loads the program to execute the steps of the workpiece pose estimation method to achieve pose tracking during the production process.

[0134] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should be regarded as within the scope of protection of the present invention.

Claims

1. A workpiece pose estimation method based on contrastive learning and iterative optimization, characterized in that: The following steps are involved: 1) Use CAD model renderings of industrial parts to generate pose annotation datasets for training and template images for similarity matching; 2) Select positive and negative sample pairs from the template image and pose annotation dataset based on the pose information, and train a template matching network model based on similarity comparison through comparative learning; 3) Use image acquisition equipment to collect video data from the production process, intercept key frames in the video stream as images to be detected, and use the YoloX object detector to obtain the two-dimensional bounding box of the workpiece to be detected in the image; 4) Cropping the image to be detected using a two-dimensional bounding box to obtain a local image, using the template matching network model to calculate the similarity score map and select the template image with the highest similarity; 5) Using the pose information of the selected template image to calculate the rough pose of the target workpiece as the initial value for pose optimization; 6) Use iterative matching to optimize the initial pose to obtain the final pose.

2. A workpiece pose estimation method based on contrastive learning and iterative optimization according to claim 1, characterized in that: The step 1) comprises the following steps: 1.1) Obtaining feature information of the workpiece from the workpiece CAD model, wherein the feature information includes the diameter of the workpiece (i.e., the diagonal length of the three-dimensional bounding box) and rotational symmetry information; 1.2) Set the model material parameters used to render the simulation image to simulate the material properties of the actual workpiece; 1.3) Use different materials to set the background wall and ground environment of the scene, and simulate the actual lighting conditions to set the light source; 1.4) Randomly place the loaded model in the set scene, randomly sample camera positions above the ground, and determine whether the field of view of each camera meets the requirements of including the set number of objects to be measured, and whether there is no occlusion greater than the threshold between the objects. If not, the camera viewpoint is discarded; 1.5) Record the RGB images, depth images, mask images obtained by different rendering methods, as well as the target object category, detection box and pose annotation information to build a pose annotation dataset; 1.6) Generate a background-free multi-view rendering image of a single target part and a corresponding binary mask image as a template image through spherical viewpoint rendering.

3. The workpiece pose estimation method based on contrastive learning and iterative optimization according to claim 1, characterized in that: Described step 2) comprises the following steps: 2.1) Select sample images from the pose annotation dataset and template images, and calculate the angle between the object center and the camera viewpoint of the two images based on their pose information. Sample pairs with angles less than 15° are considered positive sample pairs, and the rest are considered negative sample pairs; 2.2) Using ResNet50 as the basic network architecture, the template matching network is trained by contrastive learning, where the loss function used is For InfoNCE: Among them, N is the number of sample pairs, sim(q i ,t i ) is the input image q obtained by the feature extraction network i and template image feature map t i The cosine similarity between .

4. The workpiece pose estimation method based on contrastive learning and iterative optimization according to claim 1, characterized in that: The step 4) comprises the following steps: 4.1) Use a 2D bounding box to crop the original input image to be detected, and input the cropped local image and template image into the trained template matching network model; 4.2) Calculate the similarity score map through the template matching network model; 4.3) Mask the similarity score map using the binary mask image in the template image pose annotation dataset. Use the area marked by the mask image as the target area for calculating the mean, and remove points in the target area with a similarity less than the threshold. 4.4) Calculate the average cosine similarity within the remaining target area; 4.5) Select the template with the highest average cosine similarity among all template images as the matching template.

5. The workpiece pose estimation method based on contrastive learning and iterative optimization according to claim 1, characterized in that: The step 5) comprises the following steps: 5.1) Calculate the estimated distance between the center point of the object in the original input image and the camera by matching the annotation information of the template Among them, t temp,z is the template image t temp The distance between the center point of the object and the camera viewpoint, ||bb temp || and ||bb input || are the diagonal lengths of the two-dimensional bounding boxes of the objects in the template image and the input image, respectively, and f input and f temp are the focal length of the camera used for the input image and the focal length of the virtual camera used for template image rendering; 5.2) Calculate the update amount of translation from the center point of the object in the template to the center point of the actual object Among them, K (.) Represents the camera's intrinsic parameter matrix, bb (.),c Represents the center point of the bounding box, the subscript input represents the input image, and the subscript temp represents the template image; 5.3) Calculate the translation vector of the initial rough pose 5.4) Repair the rotation component error caused by the different positions of objects in the template image and the input image to obtain the rotation matrix of the rough pose Among them, R y (α y ) and R x (α x ) represent the transformation of the camera's own y-axis and x-axis rotation, and the rotation angles are α y and α x , R temp The rotation matrix of the template annotation.

6. The workpiece pose estimation method based on contrastive learning and iterative optimization according to claim 1, characterized in that: The step 6) is specifically as follows: The rough pose is used as the initial value of the pose iteration. The initial pose and the CAD model of the workpiece are superimposed and rendered on the object image. The pose iterative matching network extracts image features from the rendered image and the scene image and outputs the update amount of the translation vector (v x ,v y ,v z ), and the update amount of the rotation matrix e1, e2, after multiple iterations, the translation vector and rotation matrix are obtained as the optimized pose estimation results, where the pose is updated as follows: in, are the translation components in the updated x, y, and z directions, respectively. is the updated rotation component, and is the focal length of the camera, and R(e1,e2) is the rotation update obtained by orthogonalizing the basis defined by the predicted rotation vectors e1 and e2.

Images