A Machine Vision-Based Method for Pose Estimation of Weakly Textured Objects

Abstract

This invention discloses a machine vision-based method for estimating the pose of weakly textured objects. When training the model, this method introduces a distance bias loss function to guide the training, focusing on the impact of pixel-to-keypoint distance on the generation of hypothetical keypoints. This addresses the problem of generating hypothetical keypoints far from the keypoints when different pixels have the same prediction direction vector bias, resulting in more accurate hypothetical keypoints. Simultaneously, this invention improves the model's object recognition and classification capabilities by fusing the outputs of a ResNet-18 network and a scale-aware semantic extractor, thereby enhancing the model's accuracy. Furthermore, this invention extracts features by fusing feature maps of different scales, capturing more comprehensive and richer features and improving the accuracy of pose estimation.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method for estimating the pose of weakly textured objects based on machine vision. Background Technology

[0002] Pose estimation is a fundamental problem in computer vision, aiming to obtain the pose of an object in an image relative to the camera that captured the image. Pose estimation is a key technology in fields such as augmented reality, robotic manipulation, and hand-object interaction.

[0003] Before the advent of deep learning, many methods based on hand-designed features were used for object pose estimation. However, these methods were insufficient in terms of accuracy and robustness when faced with complex scenes. These traditional methods have now been replaced by methods that utilize deep neural networks, which significantly improve accuracy and robustness in complex environments by learning high-dimensional feature representations from data.

[0004] Extensive research has been conducted on pose estimation techniques both domestically and internationally, yielding certain results. However, numerous difficulties and problems remain in practical applications. For instance, the texture of objects and the complexity of the environment significantly affect the accuracy of pose estimation results. Therefore, a more robust pose estimation method needs to be developed. Summary of the Invention

[0005] The purpose of this invention is to provide a method for estimating the pose of weakly textured objects based on machine vision.

[0006] This invention provides a machine vision-based method for estimating the pose of weakly textured objects, comprising the following steps:

[0007] Step 1: Obtain the yarn data set with vector fields and mask map labels containing key points.

[0008] Step 2: Construct a pose estimation model. The input of the pose estimation model is the RGB image of the yarn package, and the output is the vector field and mask image of the key points.

[0009] Step 3: Train the pose estimation model

[0010] The pose estimation model constructed in step two is trained using the yarn package dataset. The input to the pose estimation model is the RGB image from the yarn package dataset, and the output is a vector field of keypoints and a tensor of semantic labels. The cross-entropy loss function L... seg As the loss function for semantic segmentation tasks; using the smoothing loss function L reg and distance deviation loss function L dis The loss function is used for vector field prediction tasks. Specifically, the distance bias loss function L... dis The expression is:

[0011]

[0012] Where K is the number of keypoints; p∈O is the pixel belonging to yarn package O; s(p,k) is the distance from the pixel to the keypoint; and For vector v k (p) x and y coordinates; v k (p) is the predicted direction vector corresponding to pixel p; u k (p) is the true direction vector corresponding to pixel p; and For vector u k The x and y coordinates of (p); k = 1, 2, ..., K.

[0013] Combining the cross-entropy loss function L seg Smoothing loss function L reg and distance deviation loss function L dis Obtain the final loss function L when training the pose estimation model + .

[0014] Step 4: Input the image of the yarn bobbin to be tested into the trained pose estimation model to obtain the semantic label tensor and the vector field corresponding to different key points. Using the vector field of the key points and the semantic label tensor, obtain the set of predicted vector fields V for the yarn bobbin corresponding to different key points k. k ={v k (p i )|p i ∈O}. In the prediction vector field set V k In this process, the intersection of the straight lines containing the predicted direction vectors corresponding to different yarn pixels is taken as the hypothetical key point h corresponding to key point k. k,j Where j = 1, 2, ..., N; N is the number of hypothetical keypoints. For each keypoint k, there is a corresponding hypothetical keypoint h. k,j An evaluation is conducted, and the optimal hypothetical key point is selected as the corresponding key point.

[0015] Step 5: Establish the 2D-3D key point correspondence based on the predicted 2D key points, and obtain the final pose of the yarn package using the PnP algorithm.

[0016] Preferably, in step two, the pose estimation model includes a ResNet-18 network, a scale-aware semantic extractor, and three sequentially connected layer-by-layer feature weight fusion modules. The ResNet-18 network includes convolutional layers, max-pooling layers, eight residual blocks, and convolutional layers. The eight residual blocks are sequentially connected, and dilated convolutions replace the convolutions in the last four residual blocks; the eight residual blocks alternately output low-level feature maps. The output of the ResNet-18 network is a high-level feature map. The low-level feature maps output by the residual blocks are processed by average pooling, and after feature fusion, they are input into the scale-aware semantic extractor to obtain scale-aware global semantics.

[0017] The three-level feature weight fusion module integrates the low-level feature map, scale-aware global semantics, and the previous-level high-level feature map, and performs upsampling; the low-level feature map and the high-level feature map output by the last-level feature weight fusion module are weighted and connected, and the vector field of key points and the tensor of semantic labels are output through convolutional layers.

[0018] Preferably, the hierarchical feature weight fusion module fuses the low-level feature map, scale-aware global semantics, and the previous high-level feature map by performing the following processing on the low-level feature map, scale-aware global semantics, and the previous high-level feature map respectively:

[0019] (1) Low-level feature map: Use 1×1 convolutional layers to enhance the dimension of the low-level feature map, and then perform batch normalization to obtain a feature map of the same size as the high-level feature map.

[0020] (2) Scale-Aware Global Semantics: The scale-aware global semantics is sliced ​​according to the channel dimension of the high-level feature map, and then bilinear upsampling is used to obtain a feature map of the same size as the high-level feature map. Bilinear upsampling is a commonly used image upsampling method used to enlarge images from low resolution to high resolution. It increases the image size by interpolating each pixel in the image to generate more pixels.

[0021] (3) Previous high-level feature map: Perform 1×1 convolution, batch normalization and Sigmoid activation on the previous high-level feature map to obtain the weight matrix.

[0022] The weighted feature map is then multiplied by Hadamard on the low-level feature map and the scale-aware global semantics using a weighted matrix. The high-level feature map is then added to the weighted low-level feature map and the scale-aware global semantics. Finally, the feature map is processed by convolutional layers, batch normalization, and the Leaky ReLU activation function to obtain a weighted fused feature map.

[0023] Preferably, in step three, the final loss function L + The expression is:

[0024]

[0025] Where σ1 and σ2 are the coefficients of the loss function.

[0026] Preferably, in step three, the cross-entropy loss function L seg The expression is:

[0027]

[0028] Where I is the input image; y(p) is the true probability of the segmentation label at pixel p; The predicted probability of the segmentation label at pixel p.

[0029] The smoothing loss function L reg The expression is:

[0030]

[0031] Where K is the number of keypoints; u k (p) is the true direction vector; v k (p) Predicted direction vector.

[0032] As a preferred embodiment, the method for selecting the optimal hypothetical key point in step four is as follows:

[0033] For each hypothetical keypoint h of keypoint k k,j Calculate each pixel p i Pointing to the key point h of this hypothesis k,j Direction vector v j (p i ):

[0034]

[0035] Calculate cosine similarity cos(θ) ij The expression for ) is:

[0036]

[0037] Where, θ ij The direction vector v j (p i ) and direction vector v k (p i The angle between them.

[0038] The calculated cosine similarity cos(θ) ij Compared with the set threshold τ, the value of τ is 0.99. Each keypoint h is added... k,j Mark score j If cos(θ) ijIf )≥τ, then score j +1. Select the highest score. j The corresponding key assumption h k,j As a key point in pose estimation and prediction.

[0039] Preferably, in step two, the scale-aware semantic extractor comprises four stacked Transformer modules; each Transformer module includes a multi-head attention module and a feedforward network connected in sequence. Image features are extracted through the multi-head attention module and the feedforward network, and residual connections are used to add the input and output to form a residual block. After passing through the four Transformer modules, scale-aware global semantics are obtained.

[0040] Preferably, in step two, the third residual block of the eight consecutive residual blocks is a residual block with a step size of 2.

[0041] Preferably, in step five, the EPnP algorithm is used to solve for the final pose of the yarn package.

[0042] Preferably, in step one, the dataset includes both real-collected RGB yarn bobbin images and rendered RGB yarn bobbin images.

[0043] Preferably, in step one, the number of key points is nine.

[0044] The beneficial effects of this invention are:

[0045] 1. In training the pose estimation model, this invention introduces a distance deviation loss function to guide the training, focusing on the impact of the distance from the pixel to the key point on the generation of hypothetical key points. This solves the problem that when different pixels have the same prediction direction vector deviation, hypothetical key points that are far away from the key point are generated, making the final generated hypothetical key points more accurate.

[0046] 2. This invention improves the model's ability to identify and classify targets by fusing the outputs of the ResNet-18 network and the scale-aware semantic extractor, thereby enhancing the model's accuracy. At the same time, this invention captures more comprehensive and richer features by fusing feature maps of different scales for feature extraction, thereby improving the accuracy of prediction. Attached Figure Description

[0047] Figure 1 This is the overall flowchart of the present invention.

[0048] Figure 2 This is a schematic diagram of the network architecture of the present invention.

[0049] Figure 3 This is a structural diagram of the SASE module of the present invention.

[0050] Figure 4 This is a structural diagram of the HFWF module of the present invention.

[0051] Figure 5 A schematic diagram is generated based on the assumptions of key points. Detailed Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings.

[0053] like Figure 1 , 2 As shown, a machine vision-based method for estimating the pose of weakly textured objects includes the following steps:

[0054] Step 1: Construct a yarn yarn dataset with vector field labels and a mask image.

[0055] A yarn bobbin dataset is created by combining real-world and rendered RGB yarn bobbin images. The dataset includes four sets of yarn bobbin datasets with different colors, each containing 10,000 images. A mask image of the yarn bobbins is obtained from the dataset, and eight 2D keypoints are obtained by projecting 3D keypoints from the point cloud model of the yarn bobbins. These eight keypoints, along with the center point of the yarn bobbin, are used as the keypoints. The vector field from each pixel in the mask image to different keypoints (the direction vector u from pixel p to keypoint k) is then obtained using the mask image and the 2D keypoints. k (a set consisting of p).

[0056] Step 2: Construct a pose estimation model

[0057] 2-1. A ResNet-18 network is used to extract features from the input image. The image is downsampled sequentially using convolutional layers (Conv-BN-ReLU), max-pooling layers, and fourth residual blocks to gradually reduce the spatial dimension and extract image features. The third residual block is a residual block with a stride of 2. After downsampling to a feature map size of H / 8 × W / 8, the feature map is processed again using fourth residual blocks. Dilated convolutions replace the convolutions in the residual blocks, and zeros are padded between the original elements of the ordinary convolutions to expand the receptive field. Convolutional layers replace the original fully connected layers in processing the feature map, and the output feature map is used as the high-level feature map.

[0058] 2-2. Using the image blocks (tokens) output by the residual blocks in the ResNet-18 network as low-level feature maps, the tokens are processed by average pooling to make the size of the tokens output by the average pooling layer H / 64×W / 64. The pooled tokens are concatenated along the channel dimension and input into the scale-aware semantic extractor (SASE).

[0059] SASE comprises four stacked Transformer modules; each Transformer module includes a multi-head attention module and a feedforward network (FNN) connected in sequence. Image features are extracted through the multi-head attention module and the FFN, and residual connections are used to add the input and output to form a residual block, which enhances the representation and semantics. After passing through the four Transformer modules, scale-aware global semantics are obtained.

[0060] The multi-head attention module extends the attention mechanism to multiple heads, enhancing the model's attention to different features and enabling the model to learn more diverse and complex features. Its formula is:

[0061] MultiHead(Q,K,V)=Concat(head1,…,head h W O

[0062] Where Q, K, and V are the query vector, key vector, and value vector, respectively; head i This is the output of the i-th head; i = 1, 2, ..., h; h is the number of heads; Concat(·) is the concatenation operation; W O This is the output transformation matrix.

[0063] The output of the i-th head i The expression is:

[0064]

[0065] in, These are the query, key, and value transformation matrices for the i-th head, respectively; Attention(·) is the attention calculation function.

[0066] 2-3. For example Figure 3 As shown, a three-level High-Level Feature Weight Fusion (HFWF) module is used to fuse low-level feature maps, scale-aware global semantics, and high-level feature maps to enhance feature representation. The high-level feature map input to the first-level HFWF is the output of the ResNet-18 network, the high-level feature map input to the second-level HFWF is the output of the first-level HFWF, and the high-level feature map input to the third-level HFWF is the output of the second-level HFWF. Upsampling is performed after each level of HFWF output; after three HFWF modules and upsampling, the feature map size is restored to H×W. The low-level feature maps and the high-level feature map output from the last level of the HFWF module are weighted and concatenated, and the vector fields corresponding to different key points and the tensors of semantic labels (mask images) are output through convolutional layers.

[0067] like Figure 4As shown, during the HFWF feature fusion process, due to the semantic differences between different feature maps, the following processing is performed on the low-level feature map, scale-aware global semantic map, and high-level feature map respectively:

[0068] (1) Low-level feature map: Use 1×1 convolutional layers to enhance the dimension of the low-level feature map, and then perform batch normalization to obtain a feature map of the same size as the high-level feature map.

[0069] (2) Scale-Aware Global Semantics: The scale-aware global semantics is sliced ​​according to the channel dimension of the high-level feature map, and then bilinear upsampling is used to obtain a feature map of the same size as the high-level feature map. Bilinear upsampling is a commonly used image upsampling method used to enlarge images from low resolution to high resolution. It increases the image size by interpolating each pixel in the image to generate more pixels.

[0070] (3) High-level feature map: The high-level feature map is subjected to 1×1 convolution, batch normalization and Sigmoid activation to obtain the weight matrix.

[0071] The high-level feature map is then multiplied by a weight matrix on both the low-level feature map and the scale-aware global semantics. The weighted low-level feature map and the scale-aware global semantics are then added together. Finally, the result is processed through convolutional layers, batch normalization, and the Leaky ReLU activation function to obtain a weighted fused feature map. Batch normalization is a commonly used neural network regularization technique that helps accelerate model training and improve model performance. Leaky ReLU is a variant of ReLU that allows gradients on negative parts, helping to reduce the vanishing gradient problem and improve network stability and training speed.

[0072] Step 3: Training the model

[0073] The pose estimation model constructed in step two was trained using different groups of yarn bobbin datasets from step one. The input to the pose estimation model was the RGB image of the yarn bobbin, and the output was the vector field of key points and the tensor of semantic labels.

[0074] For semantic segmentation tasks, the softmax cross-entropy loss function L is used. seg Guided model training:

[0075]

[0076] Where I is the input image; y(p) is the true probability of the segmentation label at pixel p; The predicted probability of the segmentation label at pixel p.

[0077] For vector field prediction, a smoothing loss function L is used. regGuide model training, smooth loss function L reg The expression is:

[0078]

[0079] Where K is the number of keypoints; u k (p) is the true direction vector; v k (p) The predicted direction vector; ||·||1 is the 1-norm symbol.

[0080] Since when different pixels have the same predicted direction vector deviation, the predicted direction vector corresponding to pixels far from the keypoint may lead to the generation of keypoint assumptions that are far from the keypoint. Therefore, by adding the distance d(p) from the keypoint to the predicted direction vector in the vector field prediction task... i Distance deviation loss function L(k) dis Guided model training involves using the prior distance from pixels to keypoints to enhance the weights of corresponding pixels, making the network pay more attention to the orientation vectors v of these pixels. k Prediction of (p). Loss function L dis The expression is:

[0081]

[0082] Where p∈O are pixels belonging to yarn package O; s(p,k) is the distance from the pixel to the keypoint; v k (p) is the predicted direction vector corresponding to pixel p; u k (p) is the true direction vector corresponding to pixel p; and For vector v k (p) x and y coordinates; and For vector u k (p) represents the x and y coordinates; ||·||2 is the 2-norm symbol.

[0083] The total loss L of the vector field prediction task vote The expression is:

[0084] L vote =L reg +L dis

[0085] Homoscedastic uncertainty learning is used to optimize the losses of the two tasks as the total loss function L of the pose estimation model. + Total loss function L + The expression is:

[0086]

[0087] Where σ1 and σ2 are the coefficients of the loss function.

[0088] Step 4: Obtain key points

[0089] like Figure 5 As shown, the image of the yarn bobbin under test is input into the trained pose estimation model to obtain the tensor of semantic labels and the vector fields corresponding to different key points. The number of vector fields obtained is the same as the number of key points. Based on the vector fields of the key points and the tensor of semantic labels, the set of predicted vector fields V belonging to the yarn bobbin object is obtained. k ={v k (p i )|p i ∈0}. In the prediction vector field set V k In this context, the intersection of the straight lines containing the predicted direction vectors corresponding to different yarn pixels is assumed to be the key point h. k,j Where j = 1, 2, ..., N; N is the number of hypothetical keypoints. Each hypothetical keypoint is evaluated using the Random Sample Consensus (RANSAC) algorithm. For each hypothetical keypoint h of keypoint k... k,j Calculate each pixel p belonging to the yarn package object O. i Pointing to the key point h of this hypothesis k,j Direction vector v j (p i ):

[0090]

[0091] According to the direction vector v j (p i The prediction direction vector v of the corresponding pixel in the prediction vector field k (p i ), calculate the cosine similarity cos(θ) ij ), cosine similarity cos(θ) ij The expression for ) is:

[0092]

[0093] Where, θ ij The direction vector v j (p i ) and direction vector v k (p i The angle between them.

[0094] cosine similarity cos(θ) ij The closer the value of cosine similarity (cos(θ)) is to 1, the closer the directions of the two vectors are. ijCompared with the set threshold τ, the value of τ is 0.99. Each keypoint h is added... k,j Mark score j If cos(θ) ij If )≥τ, then score j +1. Select the highest score. j The corresponding key assumption h k,j As a key point for pose estimation model prediction.

[0095] Step 4: Establish the 2D-3D keypoint correspondence based on the predicted 2D keypoints, and obtain the final pose of the yarn package using the EPnP algorithm. The final pose of the yarn package is a 3×4 matrix, which includes a 3×3 rotation matrix and a 3×1 translation matrix.

[0096] Step 5: Inspection

[0097] The pose estimation model is evaluated using the ADD metric. The ADD metric calculates the error in 3D object space. If the average distance (ADD) between the predicted pose and the true pose in the yarn point cloud model is less than a specified multiple of the model diameter (d), the prediction is considered correct. For symmetrical objects, the nearest model point is used to calculate the average distance ADD-S.

[0098]

[0099] Where m is the number of 3D model points; M is the set of 3D model points for the yarn package; x1 and x2 are the coordinates of the model points; R is the actual rotation matrix; and t is the actual translation vector. The rotation matrix is ​​the prediction. This is the predicted translation vector.

[0100] Pose estimation was performed on different yarn bobbin datasets using both the PVNet model and our own model. During testing, a 480×640×3 RGB image was used as input to the pose estimation model. Online data augmentation was employed to prevent overfitting. The Adam optimizer was selected, with an initial learning rate of 0.001, which was halved every 20 epochs. The prediction success rates of different models on different datasets are shown in Table 1.

[0101] Table 1 compares the yarn bobbin pose estimation results based on the ADD-S index.

[0102]

Claims

1. A method for estimating the pose of weakly textured objects based on machine vision, characterized in that: Includes the following steps: Step 1: Obtain the yarn bobbin dataset with vector fields and mask map labels containing key points; Step 2: Construct a pose estimation model. The input of the pose estimation model is the RGB image of the yarn package, and the output is the vector field and mask image of the key points. Step 3: Train the pose estimation model The pose estimation model constructed in step two is trained using the yarn package dataset; the input of the pose estimation model is the RGB image from the yarn package dataset, and the output is the vector field of key points and the tensor of semantic labels; the cross-entropy loss function L is used. seg As the loss function for semantic segmentation tasks; using the smoothing loss function L reg and distance deviation loss function L dis The loss function for the vector field prediction task; where the distance bias loss function L dis The expression is: Where K is the number of key points; p∈O is the pixel belonging to yarn package O; s(p,k) is the distance from the pixel to the key point; and For vector v k (p) x and y coordinates; v k (p) is the predicted direction vector corresponding to pixel p; u k (p) is the true direction vector corresponding to pixel p; and For vector u k (p) x and y coordinates; k = 1, 2, ..., K; Combining the cross-entropy loss function L seg Smoothing loss function L reg and distance deviation loss function L dis Obtain the final loss function L when training the pose estimation model + ; Step 4: Input the image of the yarn bobbin to be tested into the trained pose estimation model to obtain the semantic label tensor and the vector field corresponding to different key points; using the vector field of the key points and the semantic label tensor, obtain the set of predicted vector fields V of the yarn bobbin corresponding to different key points k. k ={v k (p i )|p i ∈O}; in the set of prediction vector fields V k In this process, the intersection of the straight lines containing the predicted direction vectors corresponding to different yarn pixels is taken as the hypothetical key point h corresponding to key point k. k,j Where j = 1, 2, ..., N; N is the number of hypothetical keypoints; for each keypoint k, there is a corresponding hypothetical keypoint h. k,j The evaluation is conducted, and the optimal hypothetical key point is selected as the corresponding key point. Step 5: Establish the 2D-3D key point correspondence based on the predicted 2D key points, and obtain the final pose of the yarn package using the PnP algorithm.

2. The method for estimating the pose of weakly textured objects based on machine vision according to claim 1, characterized in that: In step three, the final loss function L + The expression is: Where σ1 and σ2 are the coefficients of the loss function.

3. The method for estimating the pose of weakly textured objects based on machine vision according to claim 1, characterized in that: In step two, the pose estimation model includes a ResNet-18 network, a scale-aware semantic extractor, and three sequentially connected layer-by-layer feature weight fusion modules. The ResNet-18 network includes convolutional layers, max pooling layers, eight residual blocks, and convolutional layers. The eight residual blocks are sequentially connected, and dilated convolutions replace the convolutions in the last four residual blocks. The eight residual blocks alternately output low-level feature maps. The output of the ResNet-18 network is a high-level feature map. The low-level feature maps output by the residual blocks are processed by average pooling and then input into the scale-aware semantic extractor after feature fusion to obtain scale-aware global semantics. The three-level feature weight fusion module integrates the low-level feature map, scale-aware global semantics, and the previous-level high-level feature map, and performs upsampling; the low-level feature map and the high-level feature map output by the last-level feature weight fusion module are weighted and connected, and the vector field of key points and the tensor of semantic labels are output through convolutional layers.

4. The method for estimating the pose of weakly textured objects based on machine vision according to claim 3, characterized in that: The hierarchical feature weight fusion module fuses low-level feature maps, scale-aware global semantics, and the previous high-level feature map by performing the following processing on the low-level feature maps, scale-aware global semantics, and the previous high-level feature map respectively: (1) Low-level feature map: Use 1×1 convolutional layers to enhance the dimension of the low-level feature map, and then perform batch normalization to obtain a feature map of the same size as the high-level feature map; (2) Scale-aware global semantics: The scale-aware global semantics is sliced ​​according to the channel dimension of the high-level feature map, and then bilinear upsampling is used to obtain a feature map of the same size as the high-level feature map. Bilinear upsampling is a commonly used image upsampling method used to enlarge an image from low resolution to high resolution. More pixels are generated by interpolating each pixel in the image, thereby increasing the size of the image. (3) Previous high-level feature map: Perform 1×1 convolution, batch normalization and Sigmoid activation on the previous high-level feature map to obtain the weight matrix; The weighted feature map is then multiplied by Hadamard on the low-level feature map and the scale-aware global semantics using a weighted matrix. The high-level feature map is then added to the weighted low-level feature map and the scale-aware global semantics. Finally, the feature map is processed by convolutional layers, batch normalization, and the Leaky ReLU activation function to obtain a weighted fused feature map.

5. The method for estimating the pose of weakly textured objects based on machine vision according to claim 1, characterized in that: In step three, the cross-entropy loss function L seg The expression is: Where I is the input image; y(p) is the true probability of the segmentation label at pixel p; The predicted probability of the segmentation label at pixel p; The smoothing loss function L reg The expression is: Where K is the number of key points; u k (p) is the true direction vector; v k (p) Predicted direction vector.

6. The method for estimating the pose of weakly textured objects based on machine vision according to claim 1, characterized in that: In step four, the method for selecting the optimal hypothetical key points is as follows: For each hypothetical keypoint h of keypoint k k,j Calculate each pixel p i Pointing to the key point h of this hypothesis k,j Direction vector v j (p i ): Calculate cosine similarity cos(θ) ij The expression for ) is: Where, θ ij The direction vector v j (p i ) and direction vector v k (p i The included angle; The calculated cosine similarity cos(θ) ij Compared with the set threshold τ, the value of τ is 0.99; each keypoint h is added. k,j Mark score j If cos(θ) ij If )≥τ, then score j +1; Select the highest score j The corresponding key assumption h k,j As a key point in pose estimation and prediction.

7. The method for estimating the pose of weakly textured objects based on machine vision according to claim 3, characterized in that: The scale-aware semantic extractor comprises four stacked Transformer modules; each Transformer module includes a multi-head attention module and a feedforward network connected in sequence; image features are extracted through the multi-head attention module and the feedforward network, and the input and output are added using residual connections to form a residual block; scale-aware global semantics are obtained after passing through the four Transformer modules.

8. The method for estimating the pose of weakly textured objects based on machine vision according to claim 3, characterized in that: The third residual block in the eight consecutive residual blocks is a residual block with a step size of 2.

9. The method for estimating the pose of weakly textured objects based on machine vision according to claim 1, characterized in that: In step five, the EPnP algorithm is used to solve for the final pose of the yarn package.

10. The method for estimating the pose of weakly textured objects based on machine vision according to claim 1, characterized in that: In step one, the dataset includes both real-collected RGB yarn bobbin images and rendered RGB yarn bobbin images; There are 9 key points in the yarn package image.

Images