A model construction method, a posture estimation method and an object picking device

By constructing a pose estimation model based on dense fusion data, the problems of poor recognition effect and low training efficiency of low textured parts are solved in the prior art, and efficient pose estimation and picking capabilities of industrial parts are achieved.

CN113034575BActive Publication Date: 2024-11-29SHENZHEN HUAHAN WEIYE TECH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202110111623.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-01-27
Publication Date
2024-11-29
Estimated Expiration
2041-01-27

AI Technical Summary

Technical Problem

The existing deep learning-based pose estimation technology has poor recognition of parts with low texture and reflective surfaces in the industry. The traditional reinforcement learning method has limitations in high-dimensional state and action space problems, with long training cycles and low efficiency. , cannot meet the needs of industrial production applications.

Method used

A model construction method is adopted to train the network model by obtaining the dense fusion data of the target object (two-dimensional image data and three-dimensional point cloud data), and use the backbone node layer and the head node layer to build high-level semantic information, combining the classification layer and the regression layer. Predict the category and pose of the target object, establish the total loss function for iterative optimization, and form a pose estimation model.

Benefits of technology

The pose estimation model's detection ability of complex background and strong interference scenes is improved, and the accurate classification and sorting of scattered parts scenes is achieved, the training cycle is reduced, and the efficiency and accuracy of robot grabbing operations are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113034575B_ABST
    Figure CN113034575B_ABST
Patent Text Reader

Abstract

The present application relates to a model construction method, a pose estimation method and an object picking device, wherein the model construction method includes: obtaining dense fusion data of the target object; training a preset network model according to the dense fusion data to learn and obtain network weight parameters; configuring a pose estimation model of the target object according to the network weight parameters; wherein the network model includes a backbone node layer and a head node layer, and the head node layer includes a classification layer and a regression layer, the backbone node layer is used to construct high-level semantic information of the target object according to the dense fusion data, the classification layer in the head node layer is used to process high-level semantic information to determine the category and score of the target object, and the regression layer in the head node layer is used to process high-level semantic information to predict the pose and confidence of the target object. The technical solution can improve the ability of the pose estimation model to reconstruct panoramic information of detection scenes with complex backgrounds and strong interference, thereby improving the ability to detect the pose of the target object.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a model building method, a posture estimation method and an object picking device. Background Art

[0002] In today's manufacturing industry, the assembly process takes a lot of time and money. In order to improve production efficiency and reduce labor costs, people have begun to explore the use of robots to achieve automated assembly. Parts recognition and grasping position planning are an indispensable and important part of the automated assembly process, and have a vital impact on the quality of assembly. Vision-based part posture determination and grasping position planning can significantly improve the automation and flexibility of product assembly, reduce time and cost, and thus improve production efficiency. Robot automated assembly involves two key technologies: parts recognition and automatic grasping.

[0003] Computer vision technology plays an important role in the perception of unstructured scenes by robots. Visual images are an effective means of obtaining real-world information. Visual perception algorithms can be used to extract features of corresponding tasks, such as object position, angle, posture, and other information, so that robots can perform corresponding operations and complete designated tasks. For industrial robot sorting, although visual sensors can be used to obtain scene data, how to identify target objects from the scene and estimate their position and posture, thereby calculating the grasping position and grasping path of industrial robots, has become a core issue. In recent years, with the rapid development of deep learning technology, deep learning-based pose estimation technology has become the mainstream algorithm in the field of pose estimation. However, most of the existing mainstream pose estimation algorithms based on deep learning rely on information such as the color and texture of the object surface. They have poor recognition effects on industrial parts with low texture and reflective surfaces, which has created certain obstacles to the realization of efficient automatic part sorting.

[0004] At present, the more mature machine vision grasping method based on artificial intelligence is to predict the workpiece posture based on the two-dimensional image captured by the camera, but this method often lacks the three-dimensional information of the workpiece and can only achieve two-dimensional posture estimation. Traditional reinforcement learning methods have great limitations when solving high-dimensional state and action space problems. They have limited ability to represent complex functions under the conditions of limited samples and computing units, and their performance in practical applications is often not very ideal. At the same time, traditional deep reinforcement learning algorithms require a large amount of data for training. During the training process, the robot needs to continuously grasp and try and error in order to obtain stable picking capabilities. This training method has a long cycle and low efficiency. There are safety hazards in the actual training process and it often cannot meet the needs of industrial production applications. Summary of the invention

[0005] The main technical problem solved by the present invention is: how to improve the accuracy of machine vision picking. In order to solve the above technical problems, the present application provides a model construction method, a pose estimation method and an object picking device. According to the first aspect, an embodiment provides a method for constructing a pose estimation model, which includes: obtaining dense fusion data of the target object; the dense fusion data is obtained by heterogeneous fusion of the two-dimensional image data and three-dimensional point cloud data of the target object; training a preset network model according to the dense fusion data to learn and obtain network weight parameters; and configuring the pose estimation model of the target object according to the network weight parameters.

[0006] The network model includes a backbone node layer and a head node layer, and the head node layer includes a classification layer and a regression layer; the backbone node layer is used to construct high-level semantic information of the target object based on the dense fusion data; the high-level semantic information includes the coordinates and feature vectors of each feature point on the surface of the target object; the classification layer in the head node layer is used to process the high-level semantic information to determine the category and score of the target object, and the regression layer in the head node layer is used to process the high-level semantic information to predict the position and confidence of the target object.

[0007] For the regression layer in the network model, one or more postures of the target object are predicted according to the coordinates of each feature point in the high-level semantic information, the confidence of each predicted posture is calculated according to the feature vector of each feature point in the high-level semantic information, and the confidence of each posture is compared to determine the posture corresponding to the highest confidence as the optimal posture; the total loss function for the network model is established and expressed as

[0008] L=L conf +L loc ;

[0009]

[0010]

[0011] Among them, L conf , L loc are the loss function of the classification layer and the loss function of the regression layer respectively, the superscript p is the index of the category, the superscript 0 is the background information, is the weight coefficient, represents the score of the i-th category, Pos represents the set of feature points outside the background, Neg represents the set of feature points of the background; N is the number of feature points, L i is the posture loss function required to predict the i-th pose, s i is the confidence of the i-th pose, w is the weight coefficient, and log() is the logarithmic operation function.

[0012] The step of predicting one or more positions of the target object according to the coordinates of each feature point in the high-level semantic information comprises: obtaining the coordinates of each feature point in the high-level semantic information and expressing them as x j ; When the shape of the target object is judged to be an asymmetric structure, the first loss function for the i-th posture is established and expressed as

[0013]

[0014] Among them, M is the number of selected feature points, j is the traversal number within the range of M, are the labeled rotation matrix and translation vector, R and t are the calculated rotation matrix and translation vector, respectively; R and t are calculated by iterative calculation when the total loss function converges, and are used as one or more poses of the target object.

[0015] The step of predicting one or more positions of the target object according to the coordinates of each feature point in the high-level semantic information comprises: obtaining the coordinates of each feature point in the high-level semantic information and expressing them as x j ; When the shape of the target object is judged to be a symmetrical structure, a second loss function for the i-th posture is established and expressed as

[0016]

[0017] Among them, M is the number of selected feature points, j and k are the traversal numbers within the range of M, are the labeled rotation matrix and translation vector, R and t are the calculated rotation matrix and translation vector, respectively; R and t are obtained by iterative calculation when the total loss function converges, and are used as one or more poses of the target object.

[0018] The confidence of each predicted posture is calculated according to the feature vector of each feature point in the high-level semantic information, including: obtaining the feature vector of each feature point in the high-level semantic information and expressing it as p v ; The confidence of the i-th pose is calculated using the normalization function; the normalization function is expressed as

[0019]

[0020] Among them, p i is the i-th feature point in the high-level semantic information, and the subscript v is the traversal number within the range of N.

[0021] The method of training a preset network model according to the dense fusion data to learn and obtain network weight parameters includes: inputting the dense fusion data into the network model, optimizing the total loss function through back propagation, and obtaining the network weight parameters when the total loss function converges.

[0022] According to a second aspect, an embodiment provides a method for estimating a pose of a target object, comprising: acquiring a scene image of the target object; processing the scene image according to the pose estimation model of the target object configured in the first aspect, and obtaining category information and pose information of the target object through pose estimation; and outputting the category information and pose information of the target object;

[0023] According to the third aspect, an embodiment provides a device for acquiring a target object, comprising: a sensor for collecting a scene image of the target object; a processor connected to the sensor, for processing the scene image by the posture estimation method described in the second aspect above to output category information and posture information of the target object; a controller connected to the sensor and the processor, for controlling the sensor to capture the target object, and controlling a motion mechanism to capture the target object according to the category information and posture information of the target object.

[0024] According to the fourth aspect, a computer-readable storage medium is provided in an embodiment, characterized in that a program is stored on the medium, and the program can be executed by a processor to implement the construction method described in the first aspect above, and to implement the pose estimation method described in the second aspect above.

[0025] The beneficial effects of this application are:

[0026] A model building method, a pose estimation method and an object picking device according to the above-mentioned embodiments, wherein the model building method includes: obtaining dense fusion data of the target object; training a preset network model according to the dense fusion data to learn and obtain network weight parameters; forming a pose estimation model of the target object according to the network weight parameter configuration; wherein the network model includes a backbone node layer and a head node layer, and the head node layer includes a classification layer and a regression layer, the backbone node layer is used to build high-level semantic information of the target object according to the dense fusion data, the classification layer in the head node layer is used to process high-level semantic information to determine the category and score of the target object, and the regression layer in the head node layer is used to process high-level semantic information to predict the pose and confidence of the target object. On the one hand, since the pose estimation model uses the backbone node layer and the head node layer to perform deep learning processing on the dense fusion data of the target object, it can realize deep analysis of the dense fusion data and improve the pose estimation model's ability to reconstruct panoramic information of detection scenes with complex backgrounds and strong interference, thereby improving the performance of target object pose detection; on the other hand, the constructed pose estimation model makes it possible to estimate the pose of the target object. It only needs to input the scene image of the target object into the model to accurately output the category information and pose information of the target object by extracting the object features. Based on this, the motion path of the motion mechanism can be controlled, thereby realizing adaptive grasping operations on the target object, especially improving the robot's ability to classify and pick up scattered parts scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flowchart of a method for estimating the position and posture of a target object in this application;

[0028] Figure 2 It is a structural diagram of the network model;

[0029] Figure 3 Schematic diagram of the principles of the classification layer and regression layer;

[0030] Figure 4 Schematic diagram of the pose estimation principle;

[0031] Figure 5 Flow chart of the method for constructing the pose estimation model in this application;

[0032] Figure 6 Flowchart for training network model;

[0033] Figure 7 Flowchart for establishing the total loss function;

[0034] Figure 8 Flowchart for predicting the pose of the target object;

[0035] Fig. 9It is a schematic diagram of the structure of the acquisition device used for the target object in this application;

[0036] Fig.10 It is a schematic diagram of the structure of the processor and the controller;

[0037] Fig.11 Schematic diagram of the structure of a posture recognition device in an embodiment. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0039] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0040] The serial numbers of the components in this document, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0041] As one of the world's top five industrial robot consumers, China saw a 59% year-on-year increase in the number of robots installed in 2019, with consumption exceeding that of Europe and the United States combined. The demand for intelligent industrial robots is huge. Robot applications for handling and loading and unloading account for more than two-thirds of the total, and the added value of upgrading them to intelligent ones is obvious. With the development of artificial intelligence, technology will gradually focus on the estimation of workpiece grasping posture based on artificial intelligence technology, using pre-trained deep reinforcement learning networks to reduce the dimension of images and extract features, and derive the robot's control strategy based on the feature extraction results. The robot uses the control strategy to control the motion path and the posture of the robotic arm, thereby achieving adaptive picking operations of the target.

[0042] The machine vision grasping method based on artificial intelligence predicts the workpiece pose based on the two-dimensional image collected by the camera, but this method often lacks the three-dimensional information of the workpiece and can only realize the two-dimensional pose estimation. There are relatively few artificial intelligence methods for estimating the workpiece pose based on the three-dimensional point cloud information of the workpiece. At present, the deep reinforcement learning method is commonly used to achieve it. However, the traditional reinforcement learning method has great limitations when solving high-dimensional state and action space problems. The representation ability of complex functions is limited under the conditions of limited samples and computing units, and the performance in practical applications is often not very ideal. At the same time, the traditional deep reinforcement learning algorithm needs to provide a large amount of data for training. During the training process, the robot needs to continuously grasp trial and error to obtain stable picking ability; this training method has a long cycle and low efficiency. There are safety hazards in the actual training process, and it often cannot meet the needs of industrial production applications. Therefore, the present invention studies the simulation of real scenes by combining two-dimensional images with depth information, establishes a pose estimation network based on deep learning and a multimodal data fusion network, and constructs a corresponding data set to obtain an object pose estimation algorithm through continuous iteration to improve the robot's sorting ability for scattered parts scenes.

[0043] The technical solution of the present application is described in detail below with reference to the embodiments.

[0044] Embodiment 1

[0045] Please refer to Figure 1 , the present application discloses a method for estimating the pose of a target object, which includes steps S110-S130, which are described below respectively.

[0046] Step S110, acquiring a scene image of the target object.

[0047] It should be noted that the target object in this embodiment can be a product on an industrial assembly line, a mechanical part in an object box, a tool on an operating table, etc. Fig. 9Then, the scene image of the target object can be obtained through imaging devices such as cameras and visual sensors.

[0048] It can be understood that the scene image of the target object refers to the two-dimensional imaging result and the three-dimensional imaging result of the target object in a simple or complex scene, such as RGB-D data (i.e., the registration data of the color image and the depth image). The scene image includes not only the target object, but also some other objects and background. The visual image including the scene image is an effective means to obtain the scene where the target object is located. The features of the corresponding task, such as the object position, angle, posture and other information, are extracted through the visual perception algorithm, so that the robot can perform the corresponding operation and complete the specified task. Of course, the scene image of the target object can also refer to the two-dimensional image or three-dimensional image of the target object in the scene, but the use of a single image will reduce the ability to express certain features of the object and affect the accuracy of the subsequent object posture estimation.

[0049] Step S120, processing the scene image according to a preset target object pose estimation model, and obtaining category information and pose information of the target object through pose estimation.

[0050] For industrial robot sorting, visual sensors are used to obtain scene data, but how to identify the target object from the scene and estimate its position and posture, so as to calculate the grasping position and grasping path of the industrial robot becomes the core problem. The pose estimation technology based on deep learning has become an algorithm implementation method for pose estimation. In the past, the mainstream pose estimation algorithms based on deep learning mostly rely on the color, texture and other information of the object surface, and the recognition effect of industrial parts with low texture and reflective surface is poor, which has a certain obstacle to the realization of efficient automatic sorting of parts. In this embodiment, the real scene is simulated by combining two-dimensional images with depth information, and a pose estimation network and a multimodal data fusion network based on deep learning are established. The corresponding data set is constructed to obtain the object pose estimation algorithm (i.e., pose estimation model) through continuous iteration, thereby improving the robot's sorting ability for scattered parts scenes. Then, when the pose estimation model that has been constructed is used to perform pose estimation processing on the scene image, the target object can be identified from the scene image and its pose can be estimated separately, and finally the category information and pose information of the target object are given.

[0051] Step S130, output the category information and posture information of the target object. The category information of the target object reflects the attributes of the target object, such as bottles, fruits, tables, toolboxes, etc. This is related to the training samples and label attributes of the posture estimation model. As long as the model is trained, it will be able to output the object category. In addition, the posture information of the target object reflects the spatial posture of the target object, such as placement, appearance, posture orientation, etc. This is related to the estimation algorithm of the posture estimation model. The posture estimation model in this application will accurately estimate the spatial posture of the target object.

[0052] The pose estimation model involved is obtained by training a network model through sample learning, so the pose estimation model has the same network structure as the network model. Figure 2 The network structure specifically includes a backbone node layer 100 and a head node layer 200 , wherein the head node layer 200 may include a classification layer 201 and a regression layer 202 .

[0053] The backbone node layer 100 is used to construct high-level semantic information of the target object according to the scene image of the target object; the high-level semantic information here includes the coordinates and feature vectors of each feature point on the surface of the target object. The head node layer 200 is used to process the high-level semantic information output by the backbone node layer and analyze and obtain the category and posture of the target object.

[0054] The classification layer 201 in the head node layer 200 is used to process high-level semantic information to determine the category and score of the target object, and the regression layer 202 in the head node layer 200 is used to process high-level semantic information to predict the position and confidence of the target object. Figure 3 The network structure of full connection + ReLU in the network, for example, the classification layer 201 uses one layer of full connection + ReLU, and the regression layer 202 uses two layers of full connection + ReLU. After one layer of full connection + ReLU, the category and score of the target object can be calculated, and the other layer of high-level semantic information can be calculated after two layers of full connection + ReLU to obtain the position and confidence of the target object.

[0055] It should be noted that when the classification layer 201 is used to process high-level semantic information, the possibility of the target object belonging to a certain category will be determined and numerically represented by the score corresponding to the category. The higher the score, the greater the possibility of belonging to a certain category. Preferably, the category corresponding to the maximum score is selected as the final category determination result. Similarly, when the regression layer 202 is used to process high-level semantic information, the possibility of the target object being in a certain posture will be determined and numerically represented by the confidence of the predicted posture. The higher the confidence, the greater the possibility of being in a certain posture. Preferably, the predicted posture corresponding to the maximum confidence is selected as the final posture determination result.

[0056] In this embodiment, the main function of the regression layer 202 is to predict the position and confidence of the target object. The working principle of the regression layer 202 can be Figure 4 The high-level semantic information (including the feature vector and feature point coordinates of the target object) is input to the regression layer, which outputs the rotation matrix R i , translation vector t i and confidence s i , where the rotation matrix R i and the translation vector t i It is used to characterize the rotation and translation relationship of the current spatial pose of the target object in the camera coordinate system, which can be used as the i-th predicted pose of the target object; confidence s i Used to characterize the possibility of the i-th predicted pose.

[0057] See also Figure 3 For the classification layer, the feature vector and feature point coordinates of high-level semantic information can be processed through a "full connection + ReLU" operation to obtain the number of target object categories and the scores corresponding to each category. If the number of categories that the target object can be divided into is n+1 (n is the object category and 1 is the background), then after the full connection processing of the classification layer, n+1 scores will be obtained.

[0058] See also Figure 3 and Figure 4 For the regression layer, the feature vector of high-level semantic information and the feature point coordinates are subjected to two "full connection + ReLU" operations to obtain the corresponding position estimate The transformation relationship is p is the input feature vector, and the subscript * represents x, y, and z. In other words, the feature vector is processed by a series of fully connected functions and the activation function ReLU. The estimated coordinates are obtained by Finally, the pose is estimated through the transformation relationship R and t of the loss function. There are two transformation relationships here. One is to predict the estimated value of the corresponding position (i.e., confidence) based on the feature vector, and the other is to solve the corresponding transformation relationship. The predicted estimated value is carried by the weight of the neural network, and the transformation relationship is carried by the variable regression of the loss function. In addition, the regression layer mainly analyzes the input feature point coordinates and feature vectors, and outputs the rotation matrix R and the translation vector t, where the rotation matrix is ​​represented by the equivalent axis angle, such as the equivalent axis angle Indicates the direction of the rotation axis, and the modulus indicates the angle of rotation.

[0059] It can be understood that the pose estimation model involved in this embodiment refers to a neural network, whose functional components are related to the structure of the model itself and to the training and learning process of the model. Therefore, in order to clearly understand the construction process of the pose estimation model, a construction method will be used for specific explanation, and the construction method of the pose estimation model will be described in detail in the following embodiment 2.

[0060] Embodiment 2

[0061] Please refer to Figure 5 In this embodiment, a method for constructing a pose estimation model is disclosed, which includes steps S210-S230, which are described below respectively.

[0062] Step S210, dense fusion data of the target object is obtained, where the dense fusion data is obtained by heterogeneous fusion of the two-dimensional image data and the three-dimensional point cloud data of the target object. The two-dimensional image data and the three-dimensional point cloud data are heterogeneous data located in different feature spaces, so the heterogeneous network can be used to process the two types of data separately, so as to retain the structure of the two types of data at the same time, make full use of the respective advantages of the object depth information and the object image information, and accurately represent the surface feature points of the target object with the help of dense fusion data.

[0063] It should be noted that the target object here can be a product on an industrial assembly line, a mechanical part in an object box, a tool on an operating table, etc. Then, a two-dimensional image of the target object can be collected and two-dimensional image data can be generated by an optical imaging component such as a camera or a video camera; and the three-dimensional point cloud data of the target object can be obtained by a contact or non-contact scanning device (such as a laser scanning device), and even the three-dimensional point cloud data of the target object can be exported by a three-dimensional mapping software. The three-dimensional point cloud data here can be a part of the appearance shape data of the surface of the target object. In a specific embodiment, in order to obtain the two-dimensional image data and three-dimensional point cloud data of the target object, a 2D camera and a 3D camera can be used to capture images. The two-dimensional image of the target object can be captured by a 2D camera, and the three-dimensional point cloud of the target object can be captured by a 3D camera; of course, a 3D camera with RGB-D data can also be used to capture the three-dimensional image and two-dimensional image of the target object, so that the three-dimensional point cloud data and two-dimensional image data are composed of depth and color information.

[0064] Step S220, training the preset network model according to the dense fusion data to learn and obtain network weight parameters.

[0065] In a specific embodiment, see Figure 2 and Figure 3 , the network model may include a backbone node layer 100 and a head node layer 200, wherein the head node layer 200 includes a classification layer 201 and a regression layer 202. In addition, the backbone node layer 100 is used to construct high-level semantic information of the target object based on dense fusion data, and the high-level semantic information includes the coordinates and feature vectors of each feature point on the surface of the target object. The classification layer 201 in the head node layer 200 is used to process high-level semantic information to determine the category and score of the target object, and the regression layer 202 in the head node layer 200 is used to process high-level semantic information to predict the position and confidence of the target object. For the specific functions of the classification layer 201 and the regression layer 202 in the network model, please refer to the relevant description in Example 1.

[0066] It should be noted that the dense fusion data here is input into the network model as a training sample, and the network is optimized through iterative calculation and network back propagation algorithm to learn the network weight parameters.

[0067] Step S230, forming a pose estimation model of the target object according to the network weight parameter configuration. The pose estimation model can be constructed by configuring the parameters of each layer of the network model to the network weight parameters finally learned.

[0068] In this embodiment, the above-mentioned step S210 mainly involves obtaining dense fusion data of the target object. The following will specifically describe how to obtain dense fusion data through heterogeneous fusion based on the two-dimensional image data and three-dimensional point cloud data of the target object.

[0069] (1) Obtain the two-dimensional image data and three-dimensional point cloud data of the target object. For example, a 3D camera with RGB-D data is used to collect relevant data and output a color image and a depth image at the same time. At this time, the RGB image and the depth image are registered, so there is a one-to-one correspondence between the pixels. Of course, in some cases, the data corresponding to the 2D camera and the 3D camera can also be used. The 2D camera can be used to collect the two-dimensional image of the target object, and the 3D camera can be used to collect the three-dimensional point cloud of the target object.

[0070] (2) Extracting semantic information from two-dimensional image data to obtain color space features, and extracting point cloud features from three-dimensional point cloud data to obtain geometric space features.

[0071] The image corresponding to the two-dimensional image data often contains rich semantics, such as the three levels of image semantics: low-level semantics, intermediate semantics and high-level semantics; among them, low-level semantics are the color and texture of pixels, intermediate semantics are the roughness, contrast, compactness, etc. of image blocks, and high-level semantics are the categories of objects contained in the image or image area. Then, extracting the semantic information of two-dimensional image data can be regarded as semantic segmentation, which mainly uses semantic information to segment the image. During the segmentation process, which semantics can be selected as the segmentation object according to actual needs, considering both expanding the receptive field and improving feature abstraction. For example, when extracting the semantic information of two-dimensional image data, it is hoped that the semantic information contains both high-level semantics and low-level semantics. Since high-level semantics have good abstraction ability and low-level semantics have good positionality, the effective combination of the two can well express objects in color space, thereby obtaining color space features. In a specific embodiment, in order to obtain color space features, the region of interest of the image corresponding to the two-dimensional image data can be first segmented to obtain mask data; two-dimensional image segmentation is the process of dividing the image into several specific regions with unique categories and proposing targets of interest. It is a key step from image processing to image analysis, such as using a binarization method for image processing, instance segmentation based on deep learning, and semantic segmentation. The present invention preferably uses a semantic segmentation method based on deep learning. When the present invention uses a convolutional neural network for semantic segmentation, a segmentation network such as SegNet and FCN can be specifically used. Then, according to the mask data, the image corresponding to the two-dimensional image data is cropped to obtain a regional image, and each pixel in the regional image is mapped to the color space to obtain low-level semantic information and / or high-level semantic information; since the mask data has a shielding effect on the region of interest of the image corresponding to the two-dimensional image data, only the region of interest can be processed, and a bounding box is formed in the region of interest, thereby cropping the image in the bounding box, thereby forming a regional image. Next, color space features are obtained based on low-level semantic information and / or high-level semantic information; since high-level semantics have good abstract ability and low-level semantics have good positionality, effectively combining the two can express objects well in color space, so it is preferred to combine low-level semantic information and high-level semantic information to obtain color space features.

[0072] Three-dimensional point cloud data refers to a set of vectors of a target object in a three-dimensional coordinate system, recorded in the form of data points, and each point contains three-dimensional coordinates. With the increasing application of three-dimensional point cloud data in many fields, the feature extraction of three-dimensional point cloud data, as a key technology in point cloud data processing, is the basis for subsequent work such as regional segmentation and surface reconstruction, and affects the application effect of three-dimensional point cloud data. The point cloud features of three-dimensional point cloud data mainly refer to the three-dimensional coordinate vectors of data points, which can be specifically 3D geometric features (such as radius, elevation difference, elevation standard deviation, point density) and 3D local shape features (such as linear features, plane features, scattered features, total variance, anisotropy, eigenvalues, normal vectors, curvature), and of course, color information (such as R, G, B) and reflection intensity information (Intensity). In this embodiment, it is hoped to establish high-dimensional features through three-dimensional point cloud data, that is, to express the point cloud features of three-dimensional point cloud data through fitting of continuous functions. Since point clouds also have rotation characteristics, the network can correctly identify the point clouds no matter what coordinate system they are presented in. In order to extract the point cloud features of three-dimensional point cloud data, the extraction process may include: first, mapping the input three-dimensional point cloud data to a high-dimensional space, which can be called a feature extraction layer, and then performing symmetrical operations on the high-dimensional features to obtain local invariant feature information output by the feature extraction layer, such as using maximum pooling or average pooling operations, and finally combining the acquired features to obtain the final point cloud features; then, after the point cloud features are expressed in the geometric space, the geometric space features can be obtained. In a specific embodiment, when mask data corresponding to two-dimensional image data is obtained, the pixel points corresponding to the mask data can be mapped to the geometric space, and data matching is performed with the three-dimensional point cloud data in the geometric space to obtain point cloud mask data; for example, when the three-dimensional point cloud data is determined to be an ordered point cloud, the pixel points corresponding to the mask data are mapped to the pixel index of the geometric space, the data index of the three-dimensional point cloud data mapped to the geometric space is determined according to the pixel index, and the point cloud mask data is obtained from the three-dimensional point cloud data using the data index; or, for example, when the three-dimensional point cloud data is determined to be an unordered point cloud, the pixel points corresponding to the mask data are mapped to the pixel coordinates of the geometric space, the pixel coordinates are numerically converted according to the preset camera intrinsic parameter matrix, the point cloud coordinates are obtained, and the point cloud mask data is obtained from the three-dimensional point cloud data using the point cloud coordinates. Then, feature transformation is performed on the point cloud mask data to obtain local point cloud features and global point cloud features. The feature transformation here is to generate a set of new high-dimensional feature vectors through a certain nonlinear transformation of the input data, so as to obtain point cloud features with different dimensional characteristics. Finally, the geometric space features are obtained by combining the local point cloud features and the global point cloud features.

[0073] (3) The color space features and the geometric space features are fused to obtain pixel-level fusion features. For example, the color features of each pixel in the color space features and the point cloud features corresponding to the pixel in the geometric space features are spliced ​​on the channel to obtain pixel-level fusion features.

[0074] (4) Obtain global features by pooling the pixel-level fusion features. For example, a convolutional neural network (CNN) can be used to perform global average pooling or maximum pooling on the pixel-level fusion features, and the global features corresponding to the pixel-level fusion features are obtained after processing. For example, a copy of the pixel-level fusion features is sent to the CNN for information integration, and a global average pooling or maximum pooling is used to obtain the global features; average pooling or maximum pooling is to solve the problem of disorder of point clouds. It is a symmetric function and the output value does not depend on the order of the input variables.

[0075] Global features refer to the overall attributes of an image, usually including color features, texture features, and shape features. Since global features are low-level visual features at the pixel level, they have the characteristics of good invariance, simple calculation, and intuitive representation, but they also have the characteristics of high feature dimension and large amount of calculation. Compared with global features, pixel-level fusion features have the characteristics of local image features, which are abundant in the image and have low correlation between features. Even in the case of occlusion, the disappearance of some features will not affect the detection of other features. In addition, here, a convolutional neural network is used for pooling to obtain global features, which can increase the adaptability of the algorithm to changing features such as position and scale. The global features are combined with pixel-level fusion features to obtain dense fusion data, which can make the dense fusion data not only have the local feature information of the image and the local feature information of the point cloud, but also have the global feature information of the pixel-level fusion features, thereby mapping the low-dimensional features to high-dimensional global feature information that can reflect the object information through nonlinear transformation.

[0076] (5) The pixel-level fusion features and the global features are spliced ​​to obtain dense fusion data. The dense fusion data here is used for surface feature point detection of the target object. For example, the global features can be spliced ​​onto the back of the pixel-level fusion features on the channel to obtain a set of fusion features with context information. Since it is at the pixel level, it is also called dense fusion data.

[0077] The above is a process of obtaining dense fusion data of the target object by heterogeneous fusion of the two-dimensional image data and the three-dimensional point cloud data of the target object. When the dense fusion data is obtained, it can be used as a training sample to train the network model.

[0078] It should be noted that a dense fusion data can correspond to a predicted posture, that is, the i-th rotation matrix R and translation vector t, so the final output is a set of predicted postures. In this embodiment, the optimal predicted posture can be selected by self-supervision, and a confidence level is output for each predicted posture as a basis for judgment. It can be understood that the posture prediction and confidence level calculation of the network model are realized in the training optimization through the design of the loss function, so the following will explain in detail how to establish the loss function and how to train the model.

[0079] In this embodiment, see Figure 6 The above-mentioned step S220 mainly involves the training and learning process of the network model, which may specifically include steps S221-S223, which are described as follows.

[0080] Step S221, constructing high-level semantic information of the target object based on the dense fusion data, where the high-level semantic information includes the coordinates and feature vectors of each feature point on the surface of the target object. Figure 2 , the dense fusion data is processed using the backbone node layer 100 in the network model, thereby constructing high-level semantic information of the target object. High-level semantic information refers to feature information that has good abstraction ability and can reflect the surface texture, shape, and contour features of the object.

[0081] Step S222: Determine the category and score of the target object based on the high-level semantic information. Figure 2 , the classification layer 201 in the head node layer 200 is used to process high-level semantic information to determine the category and score of the target object; for example, the possibility of which category the target object belongs to is determined and numerically represented by the score corresponding to the category, and the higher the score, the greater the possibility of belonging to a certain category.

[0082] Step S223, predicting the position and confidence of the target object based on the high-level semantic information. Figure 2 , the regression layer 202 in the head node layer 200 is used to process high-level semantic information to predict the position and confidence of the target object; for example, the possibility of the target object being in a certain position is determined and numerically represented by the confidence of the predicted position, and the higher the confidence, the greater the possibility of being in a certain position.

[0083] For example Figure 7 The above step S223 mainly involves the process of predicting the position and confidence of the target object, which may specifically include steps S310-S340, which are described as follows.

[0084] Step S310, predicting one or more positions and poses of the target object according to the coordinates of each feature point in the high-level semantic information. Since the shape of the target object is divided into symmetrical structure and asymmetrical structure, it is necessary to establish different loss functions according to the situation to predict the position and pose of the target object, so step S310 can specifically include steps S311-S314, and the specific content can be referred to Figure 8 .

[0085] Step S311, obtaining the coordinates and feature vectors of each feature point in the high-level semantic information. For example, obtaining the coordinates of each feature point in the high-level semantic information and expressing it as x j , and obtain the feature vector of each feature point in the high-level semantic information and represent it as p v .

[0086] Step S312, determining whether the shape of the target object is an asymmetric structure, if it is an asymmetric structure, proceeding to step S313, if it is not an asymmetric structure (ie, a symmetric structure), proceeding to step S314.

[0087] Step S313, when the shape of the target object is an asymmetric structure, a first loss function for the i-th pose is established and expressed as

[0088]

[0089] Among them, M is the number of selected feature points, j is the traversal number within the range of M, are the labeled rotation matrix and translation vector, R and t are the calculated rotation matrix and translation vector respectively.

[0090] Then, R and t can be calculated by iterative calculation when the total loss function converges, wherein the construction of the total loss function will be described in the following step S340. Figure 2 and Figure 3 , the high-level semantic information (including the feature vector and feature point coordinates of the target object) is input into the regression layer, and the regression layer outputs the rotation matrix R in the iterative calculation process i , translation vector t i , where the rotation matrix R i and the translation vector t i It is used to characterize the rotation and translation relationship of the current spatial pose of the target object in the camera coordinate system, so it can be used as the i-th predicted pose of the target object. Of course, when the total loss function converges, one or more predicted poses will be obtained, and at this time, only one of them needs to be selected as the optimal pose.

[0091] It should be noted that the purpose of using the first loss function is to minimize the mean of the Euclidean distance between the coordinates of the sampling points on the network model under the marked pose and the coordinates under the predicted pose. Of course, for different poses predicted by different dense fusion data, there is a loss function as above.

[0092] It should be noted that the first loss function is used for a target object with an asymmetric shape. If the object is a symmetric object, there are multiple different poses that can be used as solutions to the optimization problem. For a symmetric object such as a sphere, there are even infinite poses that can be used as solutions to the optimization problem, so the first loss function above will become ambiguous, which is not conducive to the training of the network model.

[0093] Step S314: when the shape of the target object is a symmetrical structure, a second loss function for the i-th pose is established and expressed as

[0094]

[0095] Among them, M is the number of selected feature points, j and k are the traversal numbers within the range of M, are the labeled rotation matrix and translation vector, R and t are the calculated rotation matrix and translation vector respectively.

[0096] Then, R and t are obtained through iterative calculation when the total loss function converges, wherein the construction of the total loss function will be described in the following step S340. Figure 2 and Figure 3 , the high-level semantic information (including the feature vector and feature point coordinates of the target object) is input into the regression layer, and the regression layer outputs the rotation matrix R in the iterative calculation process i , translation vector t i and confidence s i , where the rotation matrix R i and the translation vector t i It is used to characterize the rotation and translation relationship of the current spatial pose of the target object in the camera coordinate system, so it can be used as the i-th predicted pose of the target object. Of course, when the total loss function converges, one or more predicted poses will be obtained, and at this time, only one of them needs to be selected as the optimal pose.

[0097] It should be noted that the purpose of using the second loss function is to find the nearest point of each network model sampling point under the predicted posture in the network model sampling point under the labeled posture, calculate the distance between the pair of sampling points, and then find the minimum value of the average of these distances. In the process of optimization calculation, this second loss function will gradually make the corresponding points of the model of the labeled posture information and the predicted posture information fit together, and finally converge to an optimal value. It avoids the situation where a symmetrical distribution can fit the shape of the object to become a feasible solution to the optimization target, but requires point-by-point fitting to become a feasible solution to the optimization target.

[0098] Step S320, calculating the confidence of each predicted posture according to the feature vector of each feature point in the high-level semantic information. In a specific embodiment, if the feature vector of each feature point in the high-level semantic information is represented by p v , then the confidence of the i-th pose can be calculated using the normalization function (i.e., Softmax function). The normalization function used here is expressed as

[0099]

[0100] Among them, p i is the i-th feature point in the high-level semantic information, and the subscript v is the traversal number within the range of N. For the i-th feature point, the confidence s i It is used to characterize the possibility of the i-th predicted pose, and the rotation matrix R i and the translation vector t i The i-th predicted pose used to characterize the target object.

[0101] Step S330, compare the confidences of the various postures, and determine the posture corresponding to the highest confidence as the optimal posture. It can be understood that the higher the confidence, the greater the possibility of being in a certain posture, so selecting the predicted posture corresponding to the maximum confidence as the final posture determination result can ensure the accuracy of the prediction result.

[0102] Step S340, after obtaining the first loss function L i (or the second loss function) and the confidence s i In the case of , the total loss function for the network model can be established and expressed as

[0103] L=L conf +L loc ;

[0104]

[0105]

[0106] Among them, L conf , Lloc are the loss functions of the classification layer and the regression layer respectively; the superscript p is the index of the category, the superscript 0 is the background information, is the weight coefficient (the weight coefficient corresponding to the background information is assigned to 0, and the weight coefficient corresponding to the foreground information is assigned to 1), represents the score of the i-th category, Pos represents the set of feature points outside the background, Neg represents the set of feature points of the background; N is the number of feature points, L i is the posture loss function required to predict the i-th pose, s i is the confidence of the i-th pose, w is the weight coefficient, and log() is the logarithmic operation function.

[0107] It should be noted that the total loss function L obtained here is used for iterative calculation when the network model is trained. Then, the dense fusion data can be input into the network model, the total loss function L can be optimized by back propagation, and the network weight parameters can be obtained when the total loss function L converges. After obtaining the network weight parameters of the network model, a posture estimation model for the target object can be configured.

[0108] Those skilled in the art will understand that the second embodiment of the present application introduces a method for constructing a pose estimation model, which can be used for pose estimation operations on target objects after the pose estimation model is constructed. The first embodiment introduces a method for pose estimation of a target object, which only requires a scene image as input, and through the pose estimation processing of the pose estimation model, not only the category information of the target object (i.e., which category of object the target object is most likely to belong to) can be output, but also the pose information of the target object (i.e., which predicted pose the target object is most likely to be in) can be output.

[0109] Embodiment 3

[0110] Please refer to Fig. 9 In this embodiment, a device for acquiring a target object is disclosed, which mainly includes a sensor 41, a processor 42, a controller 43 and a motion mechanism 44, which are described below respectively.

[0111] The sensor 41 is used to collect scene images of the target object. For the description of the scene image, please refer to the relevant content in the above embodiment 1. The sensor 41 here can be some visual sensors with image acquisition function, such as camera equipment, laser scanning equipment. The target object here can be a product on an industrial assembly line, a mechanical part in an object box, a tool on an operating table, etc., without specific limitation.

[0112] The processor 42 is connected to the sensor. The processor 42 is used to output the category information and the pose information of the target object through the pose estimation method disclosed in the first embodiment. For the extraction method used by the processor 42, refer to steps S110-S130 in the first embodiment.

[0113] The controller 43 is connected to the sensor 41 and the processor 42. The controller 43 is used to control the sensor 41 to collect scene images of the target object, for example, the image collection time, collection interval, collection position and other parameters of the sensor 41 can be set. In addition, the controller 43 is also used to control the motion mechanism 44 to capture the target object according to the category information and posture information of the target object output by the processor 42. For example, Fig. 9 In the embodiment, the controller 43 can output a motion instruction to the motion mechanism 44 according to the category information and posture information output by the processor 42, so that the motion mechanism grabs the target object 45 in the tool box.

[0114] In this embodiment, the motion mechanism 44 is a gripper or suction cup with a mechanical arm; the motion mechanism 44 is used to receive control instructions from the controller 43 and perform a grabbing or sucking operation on the target object within the range of motion of the mechanical arm. If the target object is determined to be a cylinder through category information, the motion mechanism 44 performs a grabbing operation on the cylinder; if the target object is determined to be a plane or a sphere through category information, the motion mechanism 44 performs a sucking operation on the plane or the sphere; if the plane is determined to be in an inclined state through posture information, the motion mechanism 44 rotates the suction cup to an inclined suction angle. It can be understood that the motion mechanism 44 adopts an adaptive operation mode to enhance the acquisition effect of the target object.

[0115] In one specific implementation, see Fig.10 The processor 42 may include an image acquisition module 421 and a posture estimation module 422. The image acquisition module 421 is used to acquire a scene image of a target object, such as directly acquiring RGB-D data from the sensor 41, thereby obtaining a scene image having both color and depth information; the posture estimation module 422 is connected to the image acquisition module 421, and is used to perform category determination and posture estimation on the target object, thereby obtaining category information and posture information of the target object.

[0116] In one specific embodiment, see Fig.10, the controller 43 includes an optimization module 431 and a control module 432. The optimization module 431 is connected to the posture estimation module 422 in the processor 42, and is used to plan the motion route and the grasping / sucking position of the motion mechanism 44 according to the posture information of the target object relative to the sensor 41. The control module 432 is connected to the optimization module 431, and is used to output control instructions, on the one hand, to control the motion mechanism 44 to grasp / suck the target object according to the planned motion route and grasping position, and on the other hand, the control module 432 also outputs control instructions to control the sensor 41 to collect the image of the target object.

[0117] Those skilled in the art will understand that the target object picking device disclosed in this embodiment enables the controller to control the motion mechanism to accurately grasp / suck the target object according to the posture information output by the processor, while ensuring the execution efficiency and effectively improving the accuracy of grasping / sucking, thereby enhancing the practical performance of the device during the application process.

[0118] Embodiment 4:

[0119] Please refer to Fig.11 , this embodiment discloses a device for estimating the pose of a target object, which may include a memory 51 and a processor 52, which are described below respectively.

[0120] The memory 51 is a computer-readable storage medium for storing a program, which may be a program code corresponding to the pose estimation method S110-S130 in the first embodiment.

[0121] Of course, the memory 52 can also store some network weight parameters, network training process data, and can also store two-dimensional image data, three-dimensional point cloud data, training sample data, category information, posture information and other data.

[0122] The processor 52 is connected to the memory 51, and is used to execute the program stored in the memory 51 to implement the corresponding posture estimation method. In a specific embodiment, the functions implemented by the processor 52 can be summarized as: obtaining a scene image of the target object from the sensor; processing the scene image according to the configured posture estimation model of the target object, and obtaining the category information and posture information of the target object through posture estimation; and outputting the category information and posture information of the target object to the controller.

[0123] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0124] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A method for constructing a pose estimation model, It is characterized in that include: Obtain dense fusion data of the target object; The dense fusion data is obtained by heterogeneous fusion of the two-dimensional image data and the three-dimensional point cloud data of the target object; The preset network model is trained according to the dense fusion data to learn and obtain network weight parameters; wherein the network model includes a backbone node layer and a head node layer, the head node layer includes a classification layer and a regression layer, the backbone node layer is used to construct high-level semantic information of the target object according to the dense fusion data, the high-level semantic information includes the coordinates and feature vectors of each feature point on the surface of the target object, the classification layer in the head node layer is used to process the high-level semantic information to determine the category and score of the target object, and the regression layer in the head node layer is used to process the high-level semantic information to predict the position and confidence of the target object; A pose estimation model of the target object is formed according to the network weight parameter configuration.

2. The construction method according to claim 1, It is characterized in that For the regression layer in the network model, predict one or more postures of the target object according to the coordinates of each feature point in the high-level semantic information, calculate the confidence of each predicted posture according to the feature vector of each feature point in the high-level semantic information, and compare the confidence of each posture to determine the posture corresponding to the highest confidence as the optimal posture; The total loss function for the network model is established and expressed as L=L conf +L loc ; Among them, L conf , L loc are the loss function of the classification layer and the loss function of the regression layer respectively, the superscript p is the index of the category, the superscript 0 is the background information, is the weight coefficient, represents the score of the i-th category, Pos represents the set of feature points outside the background, Neg represents the set of feature points of the background; N is the number of feature points, L i is the posture loss function required to predict the i-th pose, s i is the confidence of the i-th pose, w is the weight coefficient, and log() is the logarithmic operation function.

3. The construction method according to claim 2, It is characterized in that The predicting one or more positions and postures of the target object according to the coordinates of each feature point in the high-level semantic information includes: Get the coordinates of each feature point in the high-level semantic information and represent it as x j ; When the shape of the target object is judged to be an asymmetric structure, a first loss function for the i-th posture is established and expressed as Among them, M is the number of selected feature points, j is the traversal number within the range of M, are the labeled rotation matrix and translation vector, R and t are the calculated rotation matrix and translation vector respectively; R and t are calculated by iterative calculation when the total loss function converges, and are used as one or more poses of the target object.

4. The construction method according to claim 2, It is characterized in that The predicting one or more positions and postures of the target object according to the coordinates of each feature point in the high-level semantic information includes: Get the coordinates of each feature point in the high-level semantic information and represent it as x j ; When the shape of the target object is judged to be a symmetrical structure, a second loss function for the i-th posture is established and expressed as Among them, M is the number of selected feature points, j and k are the traversal numbers within the range of M, are the labeled rotation matrix and translation vector, R and t are the calculated rotation matrix and translation vector respectively; R and t are obtained through iterative calculation when the total loss function converges, and are used as one or more poses of the target object.

5. The construction method according to claim 2, It is characterized in that The calculating the confidence of each predicted posture according to the feature vector of each feature point in the high-level semantic information includes: Obtain the feature vector of each feature point in the high-level semantic information and represent it as p v ; The confidence of the i-th pose is calculated using a normalization function; the normalization function is expressed as Among them, p i is the i-th feature point in the high-level semantic information, and the subscript v is the traversal number within the range of N.

6. The construction method according to any one of claims 2 to 5, It is characterized in that The method of training a preset network model according to the dense fusion data to learn and obtain network weight parameters includes: The dense fusion data is input into the network model, the total loss function is optimized by back propagation, and the network weight parameters are obtained when the total loss function converges.

7. A method for estimating the pose of a target object, It is characterized in that include: Acquire a scene image of a target object; Processing the scene image using the pose estimation model of the target object configured according to any one of claims 1 to 6, and obtaining category information and pose information of the target object through pose estimation; Output the category information and posture information of the target object.

8. A device for acquiring a target object, It is characterized in that include: A sensor for collecting scene images of target objects; a processor connected to the sensor, and configured to process the scene image by the pose estimation method described in claim 7 to output category information and pose information of the target object; The controller is connected to the sensor and the processor, and is used to control the sensor to capture the target object, and to control a motion mechanism to capture the target object according to the category information and posture information of the target object.

9. A computer-readable storage medium, It is characterized in that A program is stored on the medium, and the program can be executed by a processor to implement the construction method as described in any one of claims 1 to 6, or to implement the pose estimation method as described in claim 7.

Citation Information

Patent Citations

  • Workpiece 6D pose estimation method based on deep learning

    CN111899301A