A method and system for estimating the pose of a weakly textured object

Abstract

The present invention provides a method and system for estimating the pose of a weakly textured object. The method comprises: sending an original RGB image into a 2D detector including a multiple attention mechanism module to obtain the category, bounding box and mask information of the target object; converting the original depth image to obtain three-dimensional point cloud data of the image; calculating four pixel points of the mask boundary of the target object by a farthest point sampling algorithm, and then determining the mask diagonal information of each object; inputting the image and mask information of the target object after being cropped by multiplying the longest side of the bounding box by a coefficient into a 3D rotation estimation network, matching with a codebook, and obtaining a rotation estimate of the target object; then inputting the rotation estimate and the mask information into a 3D translation estimation network, and obtaining a translation estimate of the target object by a projection distance estimation technology; finally, inputting the obtained three-dimensional point cloud data of the image and the rotation estimate and translation estimate of the target object into an ICP pose refinement network to obtain the final 6D pose of the target object.

Description

Technical Field

[0001] The present invention relates to the technical field of pose estimation, and in particular to a pose estimation method and system for a weak-texture object. Background Art

[0002] Machine vision is a technology based on computer vision and combined with a variety of modern scientific technologies to achieve the measurement and positioning of spatial objects in industrial inspection. The three-dimensional coordinates of spatial objects are detected through image sensors to calculate the size, movement status, shape and other characteristics of the object.

[0003] Texture information is a global information describing the surface of an object, describing the spatial distribution of intensity levels in the corresponding neighborhood space. Rigid objects without texture are common in human environments, and they need to be learned, detected, and accurately located from images in various applications. The pose of a rigid object has 6 degrees of freedom, namely 3 degrees of freedom in translation and 3 degrees of freedom in rotation. Object pose estimation aims to detect objects and estimate their orientation and translation relative to the camera coordinate system, that is, to estimate the 3D rotation and 3D translation of the object relative to the camera. Accurate pose estimation is essential for various applications, such as augmented reality, autonomous driving, and robotic manipulation. In robotics, 6D object pose helps spatial reasoning and allows the end effector to manipulate objects. In augmented reality scenarios, object pose can enhance people's perception of reality with additional information such as assembly guidance prompts.

[0004] With the development of deep learning algorithms, more and more pose estimation methods use the powerful learning ability of neural networks to improve the accuracy of pose estimation. However, since weak texture means the lack of texture, the texture pixels are large, and the tonal difference between texture pixels is small, it is difficult to perform feature extraction and recognition, resulting in great challenges in the pose estimation task for weak texture objects. Summary of the invention

[0005] In order to solve the above problems in the prior art, the present invention provides a method for estimating the pose of a weak-textured object, which improves the accuracy of the pose estimation of the weak-textured object.

[0006] A first aspect of the present invention provides a method for estimating a pose of a weakly textured object, comprising:

[0007] Obtain a first image, and process the first image through a 2D detector network including a multiple attention mechanism module to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object;

[0008] Converting the depth information of the first image into point cloud data through an intrinsic parameter matrix of the camera;

[0009] The mask boundary of each object is processed by the farthest point sampling algorithm to obtain the four pixel points corresponding to the mask boundary of each object. The mask diagonal information of each object is determined by calculating the diagonal length of the four pixel points.

[0010] Processing the image obtained after cropping the first image, the 2D bounding box of the target object, and the mask diagonal information of the target object according to the 3D rotation estimation network to obtain 3D rotation estimation information of the target object in the first image;

[0011] Processing the mask diagonal information and the 3D rotation estimation information of the target object according to the 3D translation estimation network to obtain 3D translation estimation information of the target object;

[0012] The point cloud data, the 3D rotation estimation information of the target object and the 3D translation estimation information of the target object are processed through the ICP pose refinement network to obtain a final pose of the target object.

[0013] Furthermore, the first image is processed by a 2D detector network including a multiple attention mechanism module to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object, specifically:

[0014] Perform object detection and instance segmentation on the first image to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object.

[0015] Furthermore, performing object detection and instance segmentation on the first image to obtain the category of each object in the first image, the 2D bounding box of each object, and the mask of each object is specifically:

[0016] A 2D detector network including a multiple attention mechanism module is used to perform object detection and instance segmentation on the first image to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object.

[0017] Furthermore, the processing of the mask boundary of each object by the farthest point sampling algorithm includes:

[0018] Randomly select a point from the N pixel points on the mask boundary of each object as the starting point and write it into the starting point set;

[0019] Calculate the distances between the remaining (N-1) pixel points and the starting point, and move the point with the farthest distance into the starting point set to update the starting point set;

[0020] Calculate the distance between each of the (N-2) pixel points and each point in the updated starting point set, and take the shortest distance as the distance from the point to the starting point set. This will give (N-2) distances to the starting point set, and then select the point with the farthest distance from these (N-2) distances and write it into the starting point set.

[0021] Repeat the previous step until the starting point set meets the preset number of points.

[0022] Furthermore, the image obtained after cropping the first image, the 2D bounding box of the target object, and the mask diagonal information of the target object are processed according to the 3D rotation estimation network to obtain 3D rotation estimation information of the target object in the first image, specifically:

[0023] The first image is cropped by multiplying the longest side of the target bounding box by a preset coefficient, and then the mask diagonal information of the target object is added and input into the 3D rotation estimation network;

[0024] The cropped image and the mask diagonal information of the target object are encoded through an encoder, and the cosine similarity between the encoding result and all the encodings of the codebook in the 3D rotation estimation network is calculated to obtain the 3D rotation estimation of the target object in the first image.

[0025] Further, the mask diagonal information and the 3D rotation estimation information of the target object are processed according to the 3D translation estimation network to obtain the 3D translation estimation information of the target object, specifically:

[0026] The 3D rotation estimation and mask diagonal information of the target object are input into the 3D translation estimation network, and the ratio between the detected mask diagonal and the corresponding mask diagonal in the code book is calculated to obtain the 3D translation estimation of the target object.

[0027] In a second aspect, the present invention further provides a pose estimation system for a weak-texture object, comprising:

[0028] An image processing unit, configured to obtain a first image, and process the first image through a 2D detector network including a multiple attention mechanism module to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object;

[0029] A point cloud data determination unit, configured to convert the depth information of the first image into point cloud data by using an intrinsic parameter matrix of a camera;

[0030] A mask processing unit is used to process the mask boundary of each object by using the farthest point sampling algorithm to obtain four pixel points corresponding to the mask boundary of each object, and determine the mask diagonal information of each object by calculating the diagonal length of the four pixel points;

[0031] a 3D rotation estimation information determining unit, configured to process the image obtained after cropping the first image, the 2D bounding box of the target object, and the mask diagonal information of the target object according to the 3D rotation estimation network, so as to obtain 3D rotation estimation information of the target object in the first image;

[0032] a 3D translation estimation information determining unit, configured to process the mask diagonal information and the 3D rotation estimation information of the target object according to the 3D translation estimation network to obtain 3D translation estimation information of the target object;

[0033] The pose estimation unit is used to process the point cloud data, the 3D rotation estimation information of the target object and the 3D translation estimation information of the target object through an ICP pose refinement network to obtain a final pose of the target object.

[0034] Furthermore, the image processing unit is specifically used for:

[0035] A 2D detector network including a multiple attention mechanism module is used to perform object detection and instance segmentation on the first image to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object.

[0036] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0037] The present invention provides a method for estimating the pose of a weakly textured object, comprising: acquiring a first image, processing the first image through a 2D detector network including a multiple attention mechanism module, obtaining the category of each object in the first image, the 2D boundary box of each object and the mask of each object; converting the depth information of the first image into point cloud data through the intrinsic parameter matrix of the camera; processing the mask boundary of each object through a farthest point sampling algorithm, obtaining 4 pixel points corresponding to the mask boundary of each object, and determining the mask diagonal length of each object by calculating the diagonal length of the 4 pixel points. information; according to the 3D rotation estimation network, the image obtained after cropping the first image, the 2D bounding box of the target object and the mask diagonal information of the target object are processed to obtain the 3D rotation estimation information of the target object in the first image; according to the 3D translation estimation network, the mask diagonal information and the 3D rotation estimation information of the target object are processed to obtain the 3D translation estimation information of the target object; the point cloud data, the 3D rotation estimation information of the target object and the 3D translation estimation information of the target object are processed by the ICP pose refinement network to obtain the final pose of the target object. The method provided by the present invention can effectively improve the accuracy of pose estimation of weak texture objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the implementation mode will be briefly introduced below. Obviously, the drawings described below are only some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 is a flow chart of a method for estimating the position and posture of a weakly textured object provided by a certain embodiment of the present invention;

[0040] Figure 2 is a structural diagram of a 2D detector provided by an embodiment of the present invention;

[0041] Figure 3 This is a structural diagram of a channel attention mechanism module provided by an embodiment of the present invention.

[0042] Figure 4 is a structural diagram of a position attention mechanism module provided by an embodiment of the present invention;

[0043] Figure 5 is a translation estimation map based on an object mask provided by an embodiment of the present invention;

[0044] Figure 6 This is a flow chart of domain randomization automatic encoding provided by a certain embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0047] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0048] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0049] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0050] The first aspect.

[0051] See also Figure 1 An embodiment of the present invention provides a method for estimating the pose of a weakly textured object, comprising:

[0052] S1. Obtain a first image, and process the first image through a 2D detector network including a multiple attention mechanism module to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object;

[0053] S2. Converting the depth information of the first image into point cloud data through the intrinsic parameter matrix of the camera;

[0054] S3, processing the mask boundary of each object by the farthest point sampling algorithm, obtaining the four pixel points corresponding to the mask boundary of each object, and determining the mask diagonal information of each object by calculating the diagonal length of the four pixel points;

[0055] S4. Processing the image obtained after cropping the first image, the 2D bounding box of the target object, and the mask diagonal information of the target object according to the 3D rotation estimation network to obtain 3D rotation estimation information of the target object in the first image;

[0056] S5. Processing the mask diagonal information and the 3D rotation estimation information of the target object according to the 3D translation estimation network to obtain 3D translation estimation information of the target object;

[0057] S6. Processing the point cloud data, the 3D rotation estimation information of the target object and the 3D translation estimation information of the target object through an ICP pose refinement network to obtain a final pose of the target object.

[0058] Preferably, the first image is processed by a 2D detector network including a multiple attention mechanism module to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object, specifically:

[0059] A 2D detector network including a multiple attention mechanism module is used to perform object detection and instance segmentation on the first image to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object.

[0060] The mask boundary of each object is processed by the farthest point sampling algorithm, including:

[0061] Randomly select a point from the N pixel points on the mask boundary of each object as the starting point and write it into the starting point set;

[0062] Calculate the distances between the remaining (N-1) pixel points and the starting point, and move the point with the farthest distance into the starting point set to update the starting point set;

[0063] Calculate the distance between each of the (N-2) pixel points and each point in the updated starting point set, and take the shortest distance as the distance from the point to the starting point set. This will give (N-2) distances to the starting point set, and then select the point with the farthest distance from these (N-2) distances and write it into the starting point set.

[0064] Repeat the previous step until the starting point set meets the preset number of points.

[0065] Preferably, the image obtained after cropping the first image, the 2D bounding box of the target object and the mask diagonal information of the target object are processed according to the 3D rotation estimation network to obtain the 3D rotation estimation information of the target object in the first image, specifically:

[0066] The first image is cropped by multiplying the longest side of the target bounding box by a preset coefficient, and then the mask diagonal information of the target object is added and input into the 3D rotation estimation network;

[0067] The cropped image and the mask diagonal information of the target object are encoded through an encoder, and the cosine similarity between the encoding result and all the encodings of the codebook in the 3D rotation estimation network is calculated to obtain the 3D rotation estimation of the target object in the first image.

[0068] Preferably, the mask diagonal information and the 3D rotation estimation information of the target object are processed according to the 3D translation estimation network to obtain the 3D translation estimation information of the target object, specifically:

[0069] The 3D rotation estimation and mask diagonal information of the target object are input into the 3D translation estimation network, and the ratio between the detected mask diagonal and the corresponding mask diagonal in the code book is calculated to obtain the 3D translation estimation of the target object.

[0070] In another embodiment, the present invention discloses a method for estimating the pose of a weak-texture object, which mainly solves the problem in life or industrial fields that the weak-texture object lacks texture, has large texture pixels, and has small tonal differences between texture pixels, making it difficult to perform feature extraction and recognition, so as to better perform 6D pose estimation on the weak-texture object.

[0071] In this embodiment, the method includes: sending the original RGB image to a 2D detector including a multiple attention mechanism module to obtain the category, bounding box and mask information of the target object; converting the original depth image to obtain three-dimensional point cloud data of the image; calculating the four pixel points of the mask boundary of the target object by the farthest point sampling algorithm, and calculating its diagonal length; inputting the image and mask information after the target object is cropped by multiplying the longest side of the bounding box by a coefficient into a 3D rotation estimation network, matching with the codebook, and obtaining the rotation estimate of the target object; then inputting the rotation estimate and the relevant information of the mask diagonal into the 3D translation estimation network, and obtaining the translation estimate of the target object through the projection distance estimation technology; finally, inputting the obtained three-dimensional point cloud data of the image and the rotation estimate and translation estimate of the target object into the ICP pose refinement network to obtain the final 6D pose of the target object.

[0072] See also Figure 2-6 This embodiment provides a method for estimating the pose of a weak-texture object, which specifically includes:

[0073] ① Feed the RGB image into a 2D detector network containing multiple attention mechanism modules to obtain more accurate categories, 2D bounding boxes and masks for each object in the RGB image;

[0074] ② Pass the depth image through the camera's intrinsic parameter matrix to convert the image's depth information into point cloud data;

[0075] ③ The mask boundary of the RGB image obtained in ① is passed through the farthest point sampling (FPS) algorithm to obtain the 4 pixel points of the mask boundary of the target object in the RGB image, and the diagonal length of the 4 pixel points is calculated to obtain the mask diagonal related information

[0076] ④ The RGB image in ① is cropped twice and the relevant information of the target object bounding box and mask obtained in ③ is sent to the 3D rotation estimation network to obtain the 3D rotation estimation of the target object in the RGB image;

[0077] ⑤ Send the mask diagonal related information obtained in ③ and the 3D rotation estimation information obtained in ④ into the 3D translation estimation network to obtain the 3D translation estimation of the target object in the RGB image;

[0078] ⑥ The point cloud data of the scene obtained in ② is combined with the 3D rotation estimation obtained in ④ and the 3D translation estimation obtained in ⑤ and sent to the ICP pose refinement network to obtain the final pose of the target object in the RGB image.

[0079] Preferably, object detection and instance segmentation are performed on the RGB image to obtain a 2D bounding box and mask for each object in the RGB image, including:

[0080] A 2D detector network combined with a multi-attention mechanism is used to perform object detection and instance segmentation on RGB images to obtain the 2D bounding box and mask of each object in the RGB image.

[0081] Preferably, the depth information of the image is converted into point cloud data using the intrinsic parameter matrix of the camera, so as to be subsequently used for ICP pose refinement.

[0082] Preferably, the farthest point sampling algorithm is used to obtain the 2D pixel points of the target object mask boundary, and the diagonal length of the pixel points is calculated to estimate the 3D translation of the target object in the RGB image. Including:

[0083] The steps of the farthest point sampling algorithm are:

[0084] 1. Randomly select a point in the mask boundary as the starting point and write the starting point set P = {p i};

[0085] 2. Select the remaining N-1 points to calculate and p i The distance between the points, select the farthest point p j Write starting point set P = {p i ,p j};

[0086] 3. Select the remaining N-2 points to calculate the distance to each point in the point set P, and take the shortest distance as the distance from the point to the point set. In this way, N-2 distances to the point set are obtained, and the farthest point is selected and written into the starting point set P = {p i ,p j ,p k}, and N-3 points remain;

[0087] 4. Repeat step 3 until the number of selected points meets the requirements.

[0088] Preferably, the RGB image is cropped by multiplying the longest side of the target bounding box by a coefficient and then added with the mask information of the target object and input into the 3D rotation estimation network. The cropped image and the mask diagonal information are encoded by an encoder, and then the cosine similarity between the image and all the encodings of the codebook in the 3D rotation estimation network is calculated to obtain the 3D rotation estimation of the target object in the RGB image.

[0089] Preferably, the 3D rotation estimate and mask diagonal information of the target object are input into a 3D translation estimation network, and the ratio between the detected mask diagonal and the corresponding mask diagonal in the code book is calculated to obtain the 3D translation estimate of the target object.

[0090] Preferably, since the estimated 3D rotation and 3D translation may have certain errors, the depth information of the image is used for ICP refinement to obtain a more accurate 6D pose.

[0091] The method for estimating the pose of a weakly textured object provided by the above embodiment of the present invention includes the following technologies and corresponding beneficial effects:

[0092] 1. Instance segmentation technology combined with multiple attention mechanisms.

[0093] The attention mechanism focuses on local information of the image, locates information of interest, suppresses useless information, and improves the neural network's ability to understand the image, which is conducive to extracting deep features in the image and improving the image segmentation ability.

[0094] Specifically, the original image is input into the 2D detection network, and the feature is extracted through the ResNet backbone network. The extracted feature map is input into the channel attention mechanism module and the position attention mechanism module (where the channel attention mechanism module improves the feature reconstruction process through channel-like weighting to obtain better contextual expression; the position attention mechanism module aims to use the association between any two point features to enhance the expression of each feature), and the feature maps containing channel and position context information are obtained respectively, and they are added bit by bit. At the same time, the feature map is input into the FPN module to form a complete multi-attention module, and the outputs of these two parts are added bit by bit to obtain a feature map that fully learns the contextual information, and then the feature map is sent to the RPN module to obtain the category, bounding box and mask of the target object.

[0095] 2. Rotation estimation technology based on automatic encoding.

[0096] Considering the use of purely synthetic images for network training, the effect of using the trained network for pose estimation of real objects is not good. This is because there is a domain gap between synthetic images and real images. Domain randomization (DR) can generalize the model trained with synthetic images to real images by training the model on rendered views in various semi-realistic settings (plus random lighting conditions, backgrounds, saturation, etc.), thus reducing the gap between synthetic images and real images.

[0097] During training, the object model (CAD model or 3D reconstruction model) is rendered with almost equidistant viewpoints to obtain a synthetic object view, and then each sampled view is rotated in the plane at a fixed interval to ensure that the entire SO(3) space is covered. The obtained object view is then domain randomized, that is, the Pascal VOC dataset is used as a random background image with random contrast, brightness, Gaussian blur and color distortion settings, and the input view is randomly enhanced using random light source positions, random mirror reflections and diffuse reflections, and black squares to block the image. The enhanced object view is resized to 128x128 and input into the encoder to obtain the codebook corresponding to the object view. Finally, the codebook is decoded to output the reconstructed view of the object. The training objectives and loss functions are:

[0098]

[0099]

[0100] When testing, we first detect the target object in the RGB scene, then crop the image twice using the long side of the bounding box multiplied by a factor and resize the image to 128x128 to match the encoder input size. Then we calculate the predicted test code z test ∈R 128 and all the codes in the code book i ∈R 128 The cosine similarity between:

[0101]

[0102] Finally, the rotation matrix corresponding to the code with the highest cosine similarity is returned from the codebook.

[0103] 3. Projection distance estimation technology based on object mask.

[0104] Since the aspect ratio of the 2D bounding box of the synthetic object is fixed and cannot accurately describe the size of the synthetic object, it is easy to produce errors when using the bounding box for distance estimation. In order to accurately estimate the complete 3D translation process from the camera to the center of the object, the farthest point sampling (FPS) algorithm is used to calculate the 4 pixel points of the mask image boundary of each synthetic object view, which are stored in the code book, and the diagonal length of the 4 points is calculated. syn ||.

[0105] When performing 3D translation estimation, the detected mask diagonal ‖pp can be calculated real ‖ and the corresponding mask diagonal in the codebook The 3D translation is calculated by the ratio between

[0106]

[0107] in, is the distance estimate generated by the pinhole camera model, t real,z is the synthetic rendering distance, f real and f syn is the focal length. Therefore,

[0108]

[0109]

[0110] in, is the estimated vector from the center of the synthetic object to the center of the real object, K real and K syn is the camera matrix, pp real,c and pp syn,c is the object mask center in homogeneous coordinates, is the translation vector from the camera to the center of the object.

[0111] In the second aspect, another embodiment of the present invention also provides a pose estimation system for a weakly textured object, comprising an image processing unit, a point cloud data determination unit, a mask processing unit, a 3D rotation estimation information determination unit, a 3D translation estimation information determination unit and a pose estimation unit.

[0112] Among them, the image processing unit is used to obtain a first image, and process the first image through a 2D detector network including a multiple attention mechanism module to obtain the category of each object in the first image, the 2D bounding box of each object, and the mask of each object.

[0113] The point cloud data determination unit is used to convert the depth information of the first image into point cloud data through the intrinsic parameter matrix of the camera.

[0114] The mask processing unit is used to process the mask boundary of each object through the farthest point sampling algorithm to obtain the four pixel points corresponding to the mask boundary of each object, and determine the mask diagonal information of each object by calculating the diagonal length of the four pixel points.

[0115] The 3D rotation estimation information determination unit is used to process the image obtained after cropping the first image, the 2D bounding box of the target object and the mask diagonal information of the target object according to the 3D rotation estimation network to obtain 3D rotation estimation information of the target object in the first image.

[0116] The 3D translation estimation information determination unit is used to process the mask diagonal information and the 3D rotation estimation information of the target object according to the 3D translation estimation network to obtain the 3D translation estimation information of the target object.

[0117] The pose estimation unit is used to process the point cloud data, the 3D rotation estimation information of the target object and the 3D translation estimation information of the target object through the ICP pose refinement network to obtain the final pose of the target object.

[0118] As the information interaction, execution process and other contents between the units in the above-mentioned system are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.

Claims

1. A method for estimating the pose of a weakly textured object, characterized in that: include: Obtain a first image, and process the first image through a 2D detector network including a multiple attention mechanism module to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object; Converting the depth information of the first image into point cloud data through an intrinsic parameter matrix of the camera; The mask boundary of each object is processed by the farthest point sampling algorithm to obtain the four pixel points corresponding to the mask boundary of each object. The mask diagonal information of each object is determined by calculating the diagonal length of the four pixel points. Processing the image obtained after cropping the first image, the 2D bounding box of the target object, and the mask diagonal information of the target object according to the 3D rotation estimation network to obtain 3D rotation estimation information of the target object in the first image; Processing the mask diagonal information and the 3D rotation estimation information of the target object according to the 3D translation estimation network to obtain 3D translation estimation information of the target object; The point cloud data, the 3D rotation estimation information of the target object and the 3D translation estimation information of the target object are processed through the ICP pose refinement network to obtain a final pose of the target object.

2. A method for estimating the pose of a weakly textured object as claimed in claim 1, characterized in that: The first image is processed by a 2D detector network including a multiple attention mechanism module to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object, specifically: Perform object detection and instance segmentation on the first image to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object.

3. A method for estimating the pose of a weakly textured object as claimed in claim 2, characterized in that: The performing of object detection and instance segmentation on the first image to obtain the category of each object in the first image, the 2D bounding box of each object, and the mask of each object is specifically: A 2D detector network including a multiple attention mechanism module is used to perform object detection and instance segmentation on the first image to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object.

4. The method for estimating the pose of a weakly textured object according to claim 1, wherein: The mask boundary of each object is processed by the farthest point sampling algorithm, including: Randomly select a point from the N pixel points on the mask boundary of each object as the starting point and write it into the starting point set; Calculate the distances between the remaining (N-1) pixel points and the starting point, and move the point with the farthest distance into the starting point set to update the starting point set; Calculate the distance between each point in the (N-2) pixel points and each point in the updated starting point set, and take the shortest distance as the distance from the point to the starting point set. In this way, (N-2) distances to the starting point set are obtained, and then the point with the farthest distance is selected from these (N-2) distances and written into the starting point set; Repeat the previous step until the starting point set meets the preset number of points.

5. The method for estimating the pose of a weakly textured object according to claim 1, wherein: The 3D rotation estimation network processes the image obtained after cropping the first image, the 2D bounding box of the target object, and the mask diagonal information of the target object to obtain 3D rotation estimation information of the target object in the first image, specifically: The first image is cropped by multiplying the longest side of the target bounding box by a preset coefficient, and then the mask diagonal information of the target object is added and input into the 3D rotation estimation network; The cropped image and the mask diagonal information of the target object are encoded through an encoder, and the cosine similarity between the encoding result and all the encodings of the codebook in the 3D rotation estimation network is calculated to obtain the 3D rotation estimation of the target object in the first image.

6. The method for estimating the pose of a weakly textured object according to claim 1, wherein: The mask diagonal information and the 3D rotation estimation information of the target object are processed according to the 3D translation estimation network to obtain the 3D translation estimation information of the target object, specifically: The 3D rotation estimation and mask diagonal information of the target object are input into the 3D translation estimation network, and the ratio between the detected mask diagonal and the corresponding mask diagonal in the code book is calculated to obtain the 3D translation estimation of the target object.

7. A pose estimation system for a weakly textured object, characterized in that: include: An image processing unit, configured to obtain a first image, and process the first image through a 2D detector network including a multiple attention mechanism module to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object; A point cloud data determination unit, configured to convert the depth information of the first image into point cloud data by using an intrinsic parameter matrix of a camera; A mask processing unit is used to process the mask boundary of each object by using the farthest point sampling algorithm to obtain four pixel points corresponding to the mask boundary of each object, and determine the mask diagonal information of each object by calculating the diagonal length of the four pixel points; a 3D rotation estimation information determining unit, configured to process the image obtained after cropping the first image, the 2D bounding box of the target object, and the mask diagonal information of the target object according to the 3D rotation estimation network, so as to obtain 3D rotation estimation information of the target object in the first image; a 3D translation estimation information determining unit, configured to process the mask diagonal information and the 3D rotation estimation information of the target object according to the 3D translation estimation network to obtain 3D translation estimation information of the target object; The pose estimation unit is used to process the point cloud data, the 3D rotation estimation information of the target object and the 3D translation estimation information of the target object through an ICP pose refinement network to obtain a final pose of the target object.

8. A pose estimation system for a weakly textured object as claimed in claim 7, characterized in that: The image processing unit is specifically used for: A 2D detector network including a multiple attention mechanism module is used to perform object detection and instance segmentation on the first image to obtain a category of each object in the first image, a 2D bounding box of each object, and a mask of each object.

Images