Object pose estimation method and device, electronic equipment and readable storage medium

Abstract

The application is suitable for the technical field of pose estimation, and provides a pose estimation method and device of an object, an electronic device and a readable storage medium. The pose estimation method of the object comprises: acquiring point cloud data of the object; performing point cloud clustering according to the point cloud data to obtain a point cloud cluster; calculating surface curvature of the point cloud cluster, and determining a target normal vector according to the surface curvature; and determining pose information of the object according to the target normal vector. Embodiments of the application can perform pose estimation by referring to the shape of the surface of the object, and provide more effective pose information for a robot.

Description

Technical Field

[0001] This application belongs to the field of pose estimation technology, and particularly relates to a pose estimation method, apparatus, electronic device and readable storage medium for an object. Background Technology

[0002] Current object pose estimation methods mainly fall into two categories: one is based on deep learning, and the other is based on traditional 3D model matching. Deep learning-based methods require high computing power and a large training dataset; 3D model matching methods require a pre-built 3D model, and the object's pose is calculated by matching the shape of feature points on the object's surface with the model point-to-point.

[0003] The limitations of deep learning-based methods are that they require extensive pre-training data and consume significant computing power. Traditional 3D model matching methods, on the other hand, require a pre-built 3D model. Both methods are prone to failure when dealing with objects of different shapes or when no similar shapes have been previously trained. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and readable storage medium for estimating the pose of an object, which can solve the problem of poor effectiveness of pose estimation in related technologies.

[0005] The first aspect of this application provides a method for estimating the pose of an object, comprising: acquiring point cloud data of the object; performing point cloud clustering based on the point cloud data to obtain point cloud clusters; calculating the surface curvature of the point cloud clusters and determining a target normal vector based on the surface curvature; and determining the pose information of the object based on the target normal vector.

[0006] In some embodiments of the first aspect, determining the target normal vector based on the surface curvature includes: determining candidate normal vectors for each position on the target surface corresponding to the point cloud cluster; and determining the target normal vector from the candidate normal vectors based on the surface curvature.

[0007] In some embodiments of the first aspect, determining the target normal vector from the candidate normal vectors based on the surface curvature includes: calculating the magnitude of each candidate normal vector, the magnitude being positively correlated with the surface curvature at the corresponding position; voting on each candidate normal vector based on the magnitude; and determining the target normal vector from the candidate normal vectors based on the number of votes received.

[0008] In some embodiments of the first aspect, determining the target normal vector from the candidate normal vectors based on the number of votes includes: acquiring shape information of the robot; and determining N target normal vectors corresponding to the robot from the candidate normal vectors based on the shape information and the number of votes, wherein N is a positive integer greater than 0.

[0009] In some embodiments of the first aspect, before determining the target normal vector from the candidate normal vectors based on the surface curvature, the pose estimation method for the object includes: determining the shape of the target surface based on the surface curvature; determining the target normal vector from the candidate normal vectors based on the surface curvature includes: if the shape of the target surface is planar, then taking the candidate normal vector at any position of the target surface as the target normal vector based on the surface curvature.

[0010] In some embodiments of the first aspect, determining the pose information of the object based on the target normal vector includes: calculating the centroid coordinates based on the point cloud clusters; and determining the pose information of the object based on the centroid coordinates and the target normal vector.

[0011] In some embodiments of the first aspect, acquiring point cloud data of an object includes: acquiring a color image and a depth image of the object; performing target detection on the color image to obtain a detection box of the object; acquiring depth information within the detection box based on the depth image; and determining the point cloud data based on the depth information.

[0012] A second aspect of this application provides an object pose estimation device, comprising: a data acquisition unit for acquiring point cloud data of an object; a point cloud clustering unit for performing point cloud clustering based on the point cloud data to obtain point cloud clusters; a normal vector determination unit for calculating the surface curvature of the point cloud clusters and determining a target normal vector based on the surface curvature; and a pose estimation unit for determining the pose information of the object based on the target normal vector.

[0013] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described object pose estimation method.

[0014] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described object pose estimation method.

[0015] The fifth aspect of this application provides a computer program product that, when run on an electronic device, causes the electronic device to perform the steps of the above-described object pose estimation method.

[0016] In the embodiments of this application, point cloud data of an object is acquired, and point cloud clusters are obtained based on the point cloud data. Then, the surface curvature of the point cloud clusters is calculated to determine the target normal vector based on the surface curvature. In turn, the pose information of the object is determined based on the target normal vector. The pose estimation result of the object can be given by referring to the surface shape of the object. It does not require the establishment of a three-dimensional model or model training on a rich training set, thus improving the effectiveness of pose estimation. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram illustrating the implementation process of an object pose estimation method provided in an embodiment of this application;

[0019] Figure 2 This is a schematic diagram illustrating the specific implementation process of step S101 provided in the embodiments of this application;

[0020] Figure 3 This is a schematic diagram illustrating the specific implementation process of step S103 provided in the embodiments of this application;

[0021] Figure 4 This is a schematic diagram illustrating the specific implementation process of an object pose estimation method provided in an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the structure of an object pose estimation device provided in an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are protected by this application.

[0025] Current object pose estimation methods require training on a large amount of data in advance and consume a lot of computing power, or require the establishment of a 3D model in advance. Existing methods are prone to failure when the object shape is different or when a model with a similar shape has not been trained before.

[0026] In view of this, this application proposes a pose estimation method for objects, which can give the pose estimation result of the object based on the shape of the object's surface, without relying on the establishment of a 3D model or model training on a rich training set, thus improving the effectiveness of pose estimation.

[0027] To illustrate the technical solution of this application, specific embodiments are described below.

[0028] Figure 1 This illustration shows a schematic flowchart of an object pose estimation method provided in an embodiment of this application. This method can be applied to electronic devices and is suitable for situations where the effectiveness of object pose estimation needs to be improved. The electronic device can be a robot, an unmanned outdoor vehicle, a computer, a smartphone, or other intelligent device; this application does not limit its application to such devices.

[0029] Specifically, the pose estimation method for the above-mentioned object may include the following steps S101 to S104.

[0030] Step S101: Obtain the point cloud data of the object.

[0031] The aforementioned objects are those for which pose estimation is to be performed. In some specific scenarios, these objects may refer to those that the robot needs to grasp or avoid.

[0032] In the embodiments of this application, electronic devices can collect information from the surface of an object using lidar and obtain point cloud data of the object's surface based on reflected light. Alternatively, they can use visual processing to analyze the point cloud data of the object's surface based on the object's image. This application does not limit the scope of the invention.

[0033] Step S102: Perform point cloud clustering based on the point cloud data to obtain point cloud clusters.

[0034] In this context, a point cloud cluster refers to a set of points. Specifically, point cloud clustering can identify points of the same type in point cloud data, and clusters of points of the same type can form a point cloud cluster (or simply a point cluster or cluster).

[0035] In the embodiments of this application, point cloud clustering can be implemented using Euclidean clustering algorithm or other existing clustering algorithms, and this application does not limit this.

[0036] Step S103: Calculate the surface curvature of the point cloud clusters and determine the target normal vector based on the surface curvature.

[0037] In embodiments of this application, a target surface can be fitted to a point cloud cluster. This target surface can be a plane or a curved surface, and can characterize the outer surface of an object. Based on the fitted target surface, the surface curvature at each location on the target surface can be calculated.

[0038] Surface curvature reflects the flatness at a corresponding location. The greater the surface curvature, the more pronounced the undulations in the neighborhood at that location; conversely, the smaller the surface curvature, the flatter the neighborhood at that location.

[0039] For each location on the target surface, a corresponding normal vector can also be calculated. The normal vector can represent the orientation information of each point in the 3D point cloud data, and can be determined based on the point cloud in the neighborhood of each location.

[0040] It should be understood that the surface curvature and normal vector at different locations can be the same or different. When multiple distinct normal vectors exist, the target normal vector can be determined from among them based on the surface curvature. The target normal vector is the normal vector used for pose estimation.

[0041] Step S104: Determine the pose information of the object based on the target normal vector.

[0042] Specifically, after obtaining the target normal vector, the roll angle, pitch angle, and yaw angle of the object can be calculated using the Euler angle rotation matrix, thereby obtaining the pose information of the object.

[0043] In the embodiments of this application, point cloud data of an object is acquired, and point cloud clusters are obtained based on the point cloud data. Then, the surface curvature of the point cloud clusters is calculated to determine the target normal vector based on the surface curvature. Furthermore, the pose information of the object is determined based on the target normal vector. The pose estimation result of the object can be given by referring to the surface shape of the object. It does not require the establishment of a 3D model or model training on a rich training set, which improves the effectiveness of pose estimation. Moreover, this method has good generalization ability.

[0044] Specifically, in step S101, considering that some platforms do not deploy high-precision LiDAR, it is difficult to obtain high-precision point cloud data through LiDAR, in some embodiments of this application, visual processing can be used to obtain the point cloud data of the object.

[0045] Specifically, such as Figure 2 As shown, step S101 may include steps S201 to S204.

[0046] Step S201: Obtain the color image and depth image of the object.

[0047] Specifically, electronic devices can acquire RGBD data of objects collected by the RGB-D platform, where RGB represents color images and D represents depth images.

[0048] Step S202: Perform target detection on the color image to obtain the object detection box.

[0049] Object detection is the detection of objects in a color image. In the embodiments of this application, a deep learning model can be used to perform object detection on the color image. For example, the object detection of the color image can be performed based on the YOLO-V5 (You Only Look Once Version 5) model, or object detection can be achieved by contour recognition or other methods. This application does not limit the scope of the application.

[0050] After object detection is completed, the bounding boxes of objects within the color image can be obtained. The bounding box is the image region where the object is located, and it can usually be the smallest bounding box of the object in the color image.

[0051] It should be understood that when there are multiple objects in a color image, the color image can contain a detection box corresponding to each object.

[0052] It should also be understood that when performing object detection on a color image, several detection boxes and the confidence level of each detection box can be obtained. The confidence level can represent the probability that a real object is detected within the detection box. Therefore, detection boxes with a confidence level higher than the confidence level threshold (e.g., 0.8, 0.9, etc.) can be retained, while detection boxes with a confidence level lower than the confidence level threshold can be filtered out.

[0053] Step S203: Obtain depth information within the detection box based on the depth image.

[0054] Specifically, the depth image can be aligned with the color image. After alignment, the image region in the depth image corresponding to the detection box in the color image is found, and the depth information within the detection box is obtained based on the image region found in the depth image. The depth information represents the distance between each pixel in the depth image and the camera, and can characterize the distance information of objects.

[0055] Step S204: Determine the point cloud data based on the depth information.

[0056] In the embodiments of this application, for each pixel within the detection box, based on the corresponding depth information in the depth image, it can be converted into a 3D point cloud using extrinsic parameters. By traversing all pixels within the detection box, the point cloud data of the object can be obtained.

[0057] It should be noted that since the pixels within the detection box do not necessarily have valid depth information, the pixels with valid depth information can be retained, and the point cloud data can be determined based on these pixels with valid depth information.

[0058] This application uses visual processing to acquire point cloud data using RGB-D data, which does not require a platform equipped with high-precision LiDAR. Furthermore, it can perform point cloud conversion on pixels within the detection box without processing the entire image, thus reducing computational complexity to some extent and making it suitable for deployment on low-computing-power platforms.

[0059] After obtaining point cloud data, the point cloud data can be clustered and the target normal vector can be determined.

[0060] Specifically, such as Figure 3 As shown, step S103 may include steps S301 to S302.

[0061] Step S301: Determine the candidate normal vector for each position on the target surface corresponding to the point cloud cluster.

[0062] The target surface, formed by fitting point cloud clusters, represents the outer surface of an object. For each position on the target surface, a candidate normal vector can be calculated, representing the directional information at that position. In the embodiments of this application, the target surface can be reconstructed using surface reconstruction techniques, and then the candidate normal vector for each position can be calculated from the obtained surface model. Alternatively, the candidate normal vector for each position can be calculated using triangulation, robust statistical methods, or other methods; this application does not impose any limitations on this approach.

[0063] Step S302: Determine the target normal vector from the candidate normal vectors based on the surface curvature.

[0064] In some embodiments of this application, before step S302, the shape of the target surface can be determined based on the surface curvature. Specifically, if the surface curvature at different locations is all 0, the target surface is a plane; otherwise, it is a curved surface.

[0065] Correspondingly, in step S302, if the shape of the target surface is planar, it means that the normal vectors at all positions of the target surface are the same. In this case, the candidate normal vectors at any position of the target surface can be used as the target normal vectors based on the surface curvature.

[0066] In step S302, if the target surface is curved, it means that the normal vectors at at least some positions on the target surface are not the same. In this case, the target normal vector can be selected from the candidate normal vectors at various positions on the target surface according to the surface curvature.

[0067] In some embodiments of this application, the electronic device can calculate the magnitude of each candidate normal vector. The magnitude is the vector length of the candidate normal vector, and the magnitude of the candidate normal vector at each position is positively correlated with the surface curvature at the corresponding position. Based on the magnitude, each candidate normal vector can be voted on, and the target normal vector can be determined from the candidate normal vectors according to the number of votes received.

[0068] Specifically, voting can be implemented using Hough voting. For each candidate normal vector, the normal vector with a longer magnitude can get a higher number of votes in Hough voting.

[0069] Accordingly, based on the number of votes, the candidate normal vector with the highest number of votes can be used as the target normal vector. In this way, the normal vector at the position with the largest surface curvature can be selected for pose prediction. This position is usually a fulcrum that is easy for the robot to grasp, so the predicted pose is helpful for the use of the robot.

[0070] Taking a water bottle as an example, the calculated candidate normal vectors are in multiple different directions. If the outer surface of the water bottle is relatively uniform, the magnitudes of the candidate normal vectors are basically the same. Therefore, any candidate normal vector can be used as the target normal vector during the voting process, and the grasp can be made from the circumference of the water bottle. If the outer surface of the water bottle is non-uniform, the candidate normal vector corresponding to the position with the greatest surface curvature can be selected as the target normal vector. That is, the candidate normal vector with the longer magnitude is selected, and the posture of this grasping point is finally output.

[0071] Furthermore, in some embodiments of this application, the shape information of the robotic arm can be obtained, and based on the shape information and the number of votes, N target normal vectors corresponding to the robotic arm can be determined from the candidate normal vectors.

[0072] Where N is a positive integer greater than 0.

[0073] Specifically, the shape information of the robotic arm can be obtained through image recognition or user input. Different robotic arm shapes have different numbers of fulcrums, thus affecting the number of fulcrums used during grasping. For example, when the robotic arm is a symmetrically distributed dual-gripper arm design, N can be 2; when the robotic arm is hand-shaped, N can be 5. Based on the shape information, the corresponding value of N can be determined, and the N candidate normal vectors with the most votes are used as the target normal vector.

[0074] In this way, the object's pose can be analyzed by comprehensively considering the surface shape of the object and the shape of the robot arm, thereby providing the robot with more effective pose information.

[0075] Once the target normal vector is obtained, the pose of the object can be determined based on the target normal vector.

[0076] In some embodiments of this application, after obtaining the point cloud clusters, the centroid coordinates can be calculated based on the point cloud clusters, and then the pose information of the object can be determined based on the centroid coordinates and the target normal vector.

[0077] The centroid coordinates represent the position of an object. To make the centroid coordinates more accurate, outliers can be filtered out after obtaining the point cloud clusters.

[0078] Please refer to Figure 4 , Figure 4 The specific implementation flow of pose prediction in this application is illustrated. After acquiring color and depth images, the electronic device can perform object detection on the color image to obtain detection boxes with a confidence level greater than a confidence threshold, and then use the depth image to obtain point cloud data within the detection boxes. Next, point cloud clustering is performed on the point cloud data within the detection boxes, and surface curvature and candidate normal vectors are calculated. The target normal vector is determined through Hough voting to predict the pose information of the object.

[0079] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders.

[0080] like Figure 5 The diagram shown is a structural schematic of an object pose estimation device 500 provided in an embodiment of this application. The object pose estimation device 500 is disposed on an electronic device.

[0081] The data acquisition unit 501 is used to acquire point cloud data of the object;

[0082] Point cloud clustering unit 502 is used to perform point cloud clustering based on the point cloud data to obtain point cloud clusters;

[0083] The normal vector determination unit 503 is used to calculate the surface curvature of the point cloud cluster and determine the target normal vector based on the surface curvature.

[0084] The pose estimation unit 504 is used to determine the pose information of the object based on the target normal vector.

[0085] In some embodiments of this application, the normal vector determination unit 503 described above can be specifically used to: determine the candidate normal vectors at each position on the target surface corresponding to the point cloud cluster; and determine the target normal vector from the candidate normal vectors based on the surface curvature.

[0086] In some embodiments of this application, the normal vector determination unit 503 described above may be specifically used to: calculate the magnitude of each candidate normal vector, wherein the magnitude is positively correlated with the surface curvature at the corresponding position; vote on each candidate normal vector according to the magnitude, and determine the target normal vector from the candidate normal vectors according to the number of votes received.

[0087] In some embodiments of this application, the normal vector determination unit 503 described above can be specifically used to: obtain the shape information of the robot; and determine the N target normal vectors corresponding to the robot from the candidate normal vectors based on the shape information and the number of votes, where N is a positive integer greater than 0.

[0088] In some embodiments of this application, the normal vector determination unit 503 described above can be specifically used to: determine the shape of the target surface based on the surface curvature; if the shape of the target surface is a plane, then based on the surface curvature, take the candidate normal vector at any position of the target surface as the target normal vector.

[0089] In some embodiments of this application, the pose estimation unit 504 described above can be specifically used to: calculate the centroid coordinates based on the point cloud clusters; and determine the pose information of the object based on the centroid coordinates and the target normal vector.

[0090] In some embodiments of this application, the data acquisition unit 501 described above may be specifically used to: acquire a color image and a depth image of an object; perform target detection on the color image to obtain a detection box of the object; acquire depth information within the detection box based on the depth image; and determine the point cloud data based on the depth information.

[0091] It should be noted that, for the sake of convenience and brevity, the specific working process of the above-mentioned object pose estimation device 500 can be found in the following reference: Figures 1 to 4 The corresponding process of the method will not be described in detail here.

[0092] like Figure 6 The diagram shown is a schematic of an electronic device provided in an embodiment of this application. Specifically, the electronic device 6 may include: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60, such as an object pose estimation program. When the processor 60 executes the computer program 62, it implements the steps in the embodiments of the various object pose estimation methods described above, for example... Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 5The functions of the data acquisition unit 501, point cloud clustering unit 502, normal vector determination unit 503, and pose estimation unit 504 are shown.

[0093] The computer program can be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0094] For example, the computer program can be divided into: a data acquisition unit, a point cloud clustering unit, a normal vector determination unit, and a pose estimation unit. The specific functions of each unit are as follows: the data acquisition unit acquires point cloud data of the object; the point cloud clustering unit performs point cloud clustering based on the point cloud data to obtain point cloud clusters; the normal vector determination unit calculates the surface curvature of the point cloud clusters and determines the target normal vector based on the surface curvature; the pose estimation unit determines the pose information of the object based on the target normal vector.

[0095] The electronic device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 6 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0096] The processor 60 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0097] The memory 61 can be an internal storage unit of the electronic device, such as a hard drive or memory. The memory 61 can also be an external storage device of the electronic device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 61 can include both internal and external storage units. The memory 61 is used to store the computer program and other programs and data required by the electronic device. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0098] It should be noted that, for the sake of convenience and brevity, the structure of the above-mentioned electronic device can also be referred to the specific description of the structure in the method embodiment, which will not be repeated here.

[0099] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0100] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0101] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for various specific applications, but such implementations should not be considered beyond the scope of this application.

[0102] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0103] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0104] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0105] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0106] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for estimating the pose of an object, characterized in that, include: Acquire point cloud data of the object; Point cloud clusters are obtained by performing point cloud clustering based on the point cloud data; A target surface is formed by fitting the point cloud clusters, and the surface curvature at each position on the target surface is determined; wherein, the target surface is the outer surface of the object; Determine the candidate normal vector for each position on the target surface corresponding to the point cloud cluster; Calculate the magnitude of each candidate normal vector, where the magnitude is positively correlated with the surface curvature at the corresponding location; Voting is performed on each of the candidate normal vectors based on the magnitude; Obtain the shape information of the robotic arm; Based on the shape information and the number of votes, N target normal vectors corresponding to the robotic arm are determined from the candidate normal vectors, where N is a positive integer greater than 0, and N corresponds to the number of fulcrums used by the robotic arm when grasping. The target normal vector is the normal vector at the position with the largest surface curvature. The pose information of the object is determined based on the target normal vector, and the pose information of the object is used by the robot arm to grip the object.

2. The object pose estimation method as described in claim 1, characterized in that, Before determining the target normal vector, the pose estimation method for the object includes: The shape of the target surface is determined based on the surface curvature; The step of determining the target normal vector from the candidate normal vectors based on the surface curvature includes: If the target surface is planar, then based on the surface curvature, the candidate normal vector at any position on the target surface is taken as the target normal vector.

3. The pose estimation method for an object as described in any one of claims 1 to 2, characterized in that, Determining the pose information of the object based on the target normal vector includes: Calculate the centroid coordinates based on the point cloud clusters; The pose information of the object is determined based on the centroid coordinates and the target normal vector.

4. The pose estimation method for an object as described in any one of claims 1 to 2, characterized in that, The acquisition of point cloud data of the object includes: Acquire color and depth images of the object; Target detection is performed on the color image to obtain the detection bounding box of the object; Based on the depth image, obtain the depth information within the detection box; The point cloud data is determined based on the depth information.

5. A pose estimation device for an object, characterized in that, include: The data acquisition unit is used to acquire point cloud data of the object; A point cloud clustering unit is used to perform point cloud clustering based on the point cloud data to obtain point cloud clusters; A normal vector determination unit is used to fit the point cloud clusters to form a target surface and determine the surface curvature at each position on the target surface; wherein, the target surface is the outer surface of the object; determine candidate normal vectors at each position on the target surface corresponding to the point cloud clusters; calculate the magnitude of each candidate normal vector, wherein the magnitude is positively correlated with the surface curvature at the corresponding position; vote on each candidate normal vector according to the magnitude; obtain the shape information of the robotic arm; and determine N target normal vectors corresponding to the robotic arm from the candidate normal vectors according to the shape information and the number of votes, wherein N is a positive integer greater than 0, and N corresponds to the number of fulcrums used by the robotic arm when grasping, and the target normal vector is the normal vector at the position with the largest surface curvature; The pose estimation unit is used to determine the pose information of the object based on the target normal vector, and the pose information of the object is used by the robot arm to grip the object.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the pose estimation method for the object as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the pose estimation method for the object as described in any one of claims 1 to 4.

Images