Object pose estimation method and device based on three-dimensional features, medium and terminal

Abstract

The invention provides an object pose estimation method and device based on three-dimensional features, a terminal and a medium, target detection is performed on an RGB image, and three-dimensional point cloud reconstruction is performed by using a target detection result and a depth image. And performing pose coarse estimation by using the three-dimensional digital model, and optimizing the pose in a manner of matching the three-dimensional digital model with the point cloud data through a matching algorithm. According to the method, the three-dimensional digital model is used as priori knowledge of object pose estimation, and the global information of the three-dimensional digital model is used for complete matching, so that the precision and robustness of pose estimation are improved, and the accurate pose of the target object can be obtained even if the target features are shielded. Compared with an existing object pose estimation technology, the problems of pose ambiguity, poor stability and poor reliability are solved, and the overall performance of pose estimation is improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a method, device, medium, and terminal for estimating the pose of an object based on three-dimensional features. Background Art

[0002] Object pose estimation is a key research area in computer vision. It aims to determine the spatial position and pose of three-dimensional objects, known as the six-degrees-of-freedom (6DoF) pose, from visual data. This technology has widespread applications in robotics, augmented reality, industrial automation, and other fields. In practical applications, multiple objects often need to be assembled or docked in specific relative poses to achieve more complex functions. Object pose estimation provides key technical support for this purpose. By analyzing the relative poses of objects, assembly or docking strategies can be formulated.

[0003] In recent years, deep learning methods have made progress in the field of object pose estimation. End-to-end methods based on deep neural networks, such as DenseFusion, PVN3D, and G2L-Net, directly predict object pose by learning high-level features from image or point cloud data. These methods leverage the representation and generalization capabilities of deep learning models to improve estimation accuracy and robustness. Furthermore, new techniques such as point-wise embedding features, 3D keypoint voting, and differentiable proxy voting losses have been introduced to enhance algorithm performance and adaptability.

[0004] However, existing methods still face challenges in complex scenarios, such as pose ambiguity caused by object occlusion, overlap, and symmetry, as well as factors such as lighting, noise, and material that affect algorithm stability, limiting their performance and reliability in practical applications. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a method, device, medium and terminal for object pose estimation based on three-dimensional features, which are used to solve the problems of pose ambiguity, poor stability and poor reliability in existing object pose estimation technologies.

[0006] To achieve the above objectives and other related objectives, the first aspect of the present application provides a method for estimating the position and posture of an object based on three-dimensional features.

[0007] In some embodiments of the first aspect of the present application, the method includes: acquiring an RGB image set and a corresponding depth atlas containing a target object, and preprocessing the RGB image set and the corresponding depth atlas; performing a target detection operation on the RGB images in the RGB image set to generate a corresponding bounding box set, and reconstructing a three-dimensional point cloud based on the bounding box set to generate a three-dimensional point cloud containing features of the target object; obtaining a three-dimensional digital model corresponding to the target object, and generating a first pose based on the three-dimensional digital model and the bounding box set; performing a pose optimization operation based on the first pose and the three-dimensional point cloud to generate a second pose; and calculating a pose evaluation result of the target object based on the first pose and the second pose.

[0008] In some embodiments of the first aspect of the present application, the process of generating the first pose includes: performing a cropping operation on the RGB image set and the corresponding depth atlas according to the bounding box set; generating a multi-angle model rendering image set of the target object through the three-dimensional digital model; and inputting the cropped RGB image set and the corresponding depth atlas, as well as the multi-angle model rendering image set into a deep learning network to generate the first pose.

[0009] In some embodiments of the first aspect of the present application, the process of generating the second pose includes: performing a pose conversion operation on the three-dimensional digital model based on the first pose; and performing a data matching operation on the three-dimensional digital model after the pose conversion operation and the three-dimensional point cloud data based on a matching algorithm to generate the second pose.

[0010] In some embodiments of the first aspect of the present application, the process of calculating the pose evaluation result of the target object includes: performing matrix multiplication on the first pose and the second pose to obtain the pose evaluation result.

[0011] In some embodiments of the first aspect of the present application, the process of generating a three-dimensional point cloud containing target object features includes: based on the bounding box set, cropping the depth map in the depth map set to generate local depth information of the object; and generating a three-dimensional point cloud containing target object features based on the local depth information of the object.

[0012] In some embodiments of the first aspect of the present application, the method also includes: using the above-mentioned three-dimensional feature-based object pose estimation method to perform pose estimation operations on two target objects to generate corresponding first pose evaluation results and second pose evaluation results; performing matrix multiplication operations on the first pose evaluation results and the second pose evaluation results to obtain the relative pose relationship between the first target object and the second target object.

[0013] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides an object pose estimation device based on three-dimensional features, including: a data acquisition module: acquiring an RGB image set and a corresponding depth atlas containing a target object, and preprocessing the RGB image set and the corresponding depth atlas; a three-dimensional point cloud reconstruction module: performing a target detection operation on the RGB images in the RGB image set to generate a corresponding bounding box set, and reconstructing a three-dimensional point cloud based on the bounding box set to generate a three-dimensional point cloud containing the features of the target object; a pose coarse estimation module: obtaining a three-dimensional digital model corresponding to the target object, and generating a first pose based on the three-dimensional digital model and the bounding box set; a pose optimization module: performing a pose optimization operation based on the first pose and the three-dimensional point cloud to generate a second pose; and calculating the pose evaluation result of the target object based on the first pose and the second pose.

[0014] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the object pose estimation method based on three-dimensional features when the computer program is executed by a processor.

[0015] To achieve the above-mentioned objectives and other related objectives, the fourth aspect of the present application provides an electronic terminal, comprising: a processor and a memory; the memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory, so that the terminal executes the object pose estimation method based on three-dimensional features.

[0016] As described above, the present application relates to a method, device, medium, and terminal for object pose estimation based on three-dimensional features in the field of computer vision, which has the following beneficial effects: improving the accuracy and overall robustness of pose estimation. Accurate pose estimation results can be obtained even when target features are obscured, while also being able to cope with various interference factors in complex scenes, such as lighting changes, background clutter, and target deformation. This solves the problems of pose ambiguity, poor stability, and poor reliability that cannot be solved in existing object pose estimation technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart of an embodiment of a method for estimating an object pose based on three-dimensional features of the present application is shown.

[0018] Figure 2 A schematic diagram of the structure of a target object in an embodiment of a method for estimating object pose based on three-dimensional features of the present application is shown.

[0019] Figure 3 A structural diagram of an embodiment of an object pose estimation device based on three-dimensional features of the present application is shown.

[0020] Figure 4 A schematic diagram of the structure of an electronic terminal for object pose estimation based on three-dimensional features of the present application is shown. DETAILED DESCRIPTION

[0021] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0022] It should be noted that in the following description, reference is made to the accompanying drawings, which describe several embodiments of the present application. It should be understood that other embodiments may also be used, and that mechanical, structural, electrical, and operational changes may be made without departing from the spirit and scope of the present application. The following detailed description should not be considered restrictive, and the scope of the embodiments of the present application is limited only by the claims of the published patents. The terms used herein are only for describing specific embodiments and are not intended to limit the present application. Spatially related terms, such as "upper", "lower", "left", "right", "below", "below", "lower", "above", "upper", etc., may be used in the text to facilitate the description of the relationship between one element or feature shown in the figure and another element or feature.

[0023] In this application, unless otherwise specified or limited, the terms "mounted," "connected," "connect," "fixed," "holding," and the like should be understood broadly. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediary; or internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in this application based on specific circumstances.

[0024] Furthermore, as used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprise", "include" indicate the presence of the described features, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, "A, B, or C" or "A, B, and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition occur only when the combination of elements, functions, or operations is inherently mutually exclusive in some way.

[0025] In order to solve the problems in the above-mentioned background technology, the present invention provides a method, device, medium and terminal for object pose estimation based on three-dimensional features, which aims to solve the problems of pose ambiguity, poor stability and poor reliability in existing object pose estimation technologies. At the same time, in order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention are further described in detail through the following embodiments and in combination with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention and are not used to limit the invention.

[0026] Before further explaining the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations:

[0027] <1> Pose estimation: The process of determining the position and orientation of an object in three-dimensional space, typically using sensors to measure the object's motion and position relative to a reference frame.

[0028] <2> 3D camera: A camera capable of capturing three-dimensional information, typically by using multiple lenses or sensors to measure depth.

[0029] <3> RGB image: A two-dimensional image that contains only color information, consisting of red, green, and blue channels.

[0030] <4> Depth map: A two-dimensional image containing information about the distance of each pixel from the camera, often used along with an RGB image to create a representation of a three-dimensional scene.

[0031] <5> 3D point cloud: An object represented by a set of discrete points in three-dimensional space, typically acquired from a 3D camera or laser scanner.

[0032] <6> CAD model: A digital representation of an object created using computer-aided design software that typically contains the object's geometry, dimensions, and other properties.

[0033] Embodiments of the present invention provide a method for estimating an object pose based on three-dimensional features, a system for performing such a method, and a storage medium storing an executable program for implementing such a method. Regarding the implementation of such a method, the present invention will describe an exemplary implementation scenario for such a method.

[0034] like Figure 1 FIG. 1 shows a flow chart of a method for estimating an object's pose based on three-dimensional features according to an embodiment of the present invention. The method for estimating an object's pose based on three-dimensional features according to this embodiment mainly includes the following steps:

[0035] Step S11: collecting an RGB image set and a corresponding depth atlas containing a target object, and preprocessing the RGB image set and the corresponding depth atlas.

[0036] In one embodiment of the present invention, the process of preprocessing the RGB image set and the corresponding depth atlas includes: aligning the depth atlas with the RGB image set according to preset acquisition parameters; and performing a resolution unification operation on the RGB image set and the depth atlas.

[0037] In one embodiment of the present invention, a 3D camera is used to obtain a real-time RGB image and depth map of a target object, and the acquired depth map is aligned with the RGB image using camera extrinsics. The camera extrinsics refers to the transformation matrix between the camera coordinate system and the world coordinate system, which is used to describe the position and posture of the camera in the world coordinate system. The alignment process includes: using calibrated camera extrinsics to convert each pixel in the depth map from the camera coordinate system to the world coordinate system, and projecting the converted depth map into the RGB image to obtain the depth value of each pixel, and then using the projected depth value to align the depth map and the RGB image. It should be noted that this application does not limit the specific brand and specifications of the 3D camera, and only requires a 3D camera that can capture the RGB and depth maps of the scene at the same time.

[0038] In one embodiment of the present invention, the resolution unification operation further includes scaling the RGB image and the depth map to a suitable size. For example, the resolutions of the RGB image and the depth map are unified to 640*480.

[0039] Step S12: performing a target detection operation on the RGB images in the RGB image set to generate a corresponding bounding box set, and reconstructing a three-dimensional point cloud based on the bounding box set to generate a three-dimensional point cloud containing target object features.

[0040] In one embodiment of the present invention, the process of performing target detection based on real-time RGB images includes first determining the target object to be detected based on the pose estimation task. For example, Figure 2 illustrates an assembly task involving objects A and B in this embodiment, where the lugs of object A need to be assembled onto the hooks of object B. Accordingly, object A has a pair of lugs as matching components, and object B has a pair of hooks. In this embodiment, the target objects to be detected are the hooks and lugs.

[0041] Furthermore, regarding the aforementioned hooks and ears, a hook dataset and an ear dataset are constructed, respectively. Object detection networks are trained based on the hook dataset and ear dataset, respectively, and the object detection networks output the bounding boxes or masks of the hooks and ears in the RGB image in real time. The object detection networks used in this application include, but are not limited to, YOLO networks, Mask RCNN networks, Faster R-CNN networks, and other object detection networks.

[0042] In one embodiment of the present invention, the process of generating a three-dimensional point cloud containing target object features includes: cropping a depth map in a depth map set based on the bounding box set to generate local depth information of the object, and generating a three-dimensional point cloud containing target object features based on the local depth information of the object.

[0043] In one embodiment of the present invention, the process of generating a three-dimensional point cloud containing features of the target object based on the local depth information of the object includes obtaining camera intrinsic parameters, where the intrinsic parameters include the camera focal length and the camera principal point. Using the camera intrinsic parameters, a depth map is back-projected from the local depth information of the object to convert the local depth information into a plurality of discrete points in three-dimensional space. These discrete points constitute the three-dimensional point cloud containing features of the target object.

[0044] Step S13: obtaining a three-dimensional digital model corresponding to the target object, and generating a first pose based on the three-dimensional digital model and the bounding box set.

[0045] In one embodiment of the present invention, the process of generating the first pose includes: performing a cropping operation on the RGB image set and the corresponding depth atlas according to the bounding box set; generating a multi-angle model rendering image set of the target object using the three-dimensional digital model; and inputting the cropped RGB image set and the corresponding depth atlas, as well as the multi-angle model rendering image set, into a deep learning network to generate the first pose.

[0046] In one embodiment of the present invention, a rendering tool is used to render a digital model of a target object to generate projection images from multiple perspectives. The digital model includes, but is not limited to, CAD models, OBJ models, FBX models, and STL models. The rendering tool includes, but is not limited to, Blender, Maya, or Cinema4D.

[0047] In one embodiment of the present invention, a rough preliminary pose estimation result can be obtained by inputting the cropped RGB image, depth image, and model rendering image into the above-mentioned network model. The deep learning network model includes but is not limited to object pose estimation models such as DCSPose network model, DenseFusion network model, and Gen6D network model.

[0048] Step S14: performing a posture optimization operation based on the first posture and the three-dimensional point cloud to generate a second posture.

[0049] In one embodiment of the present invention, the process of generating the second pose includes: performing a pose conversion operation on the three-dimensional digital model based on the first pose. Then, based on a matching algorithm, performing a data matching operation on the three-dimensional digital model after the pose conversion operation and the three-dimensional point cloud data to generate the second pose. The matching algorithm includes, but is not limited to, an ICP matching algorithm, a NICP matching algorithm, and an NDT matching algorithm.

[0050] It should be noted that, considering that the first pose is directly generated from the three-dimensional digital model as prior knowledge, there may be some differences in the simulation process. Therefore, the present invention designs a pose optimization operation to further optimize the first pose. In this embodiment, the digital model of the target object obtained in step S13 is first converted to be unified with the first pose. Considering that the pose of the digital model is generally a default perspective, in order to implement the subsequent matching algorithm, the present invention unifies the pose of the digital model to the same perspective as the first pose.

[0051] In one embodiment of the present invention, an ICP algorithm is used to perform a data matching operation on the three-dimensional point cloud generated in step S12 and the three-dimensional digital model to generate a second pose for optimizing the first pose.

[0052] Step S15: Calculate a pose evaluation result of the target object based on the first pose and the second pose.

[0053] In one embodiment of the present invention, the process of calculating the pose evaluation result of the target object includes: performing matrix multiplication on the first pose and the second pose to obtain the pose evaluation result.

[0054] For example, if the first pose is Pose1 and the second pose is Pose2, the obtained optimized pose evaluation result PoseA is as shown in Formula 1, where PoseA is the matrix product of matrix Pose1 and matrix Pose2.

[0055] PoseA=Pose1*Pose2 (Formula 1)

[0056] In one embodiment of the present invention, the above-described three-dimensional feature-based object pose estimation method is used to perform pose estimation on two target objects to generate corresponding first pose evaluation results and second pose evaluation results. A matrix multiplication operation is performed on the first pose evaluation results and the second pose evaluation results to obtain a relative pose relationship between the first target object and the second target object.

[0057] In this embodiment, if the pose matching operation needs to be performed as follows Figure 2 For the objects A and B shown in FIG1 , the first pose evaluation result of object A and the second pose evaluation result of object B are respectively obtained through the method of steps S11 to S14, and the pose of object A relative to object B is calculated by matrix multiplication. For example, the pose Post of object A relative to object B is shown in Formula 2.

[0058] Pose = PoseA*PoseB -1 (Formula 2)

[0059] like Figure 3 FIG. 3 is a block diagram of a device for estimating an object's pose based on three-dimensional features according to an embodiment of the present invention. In this embodiment, the device 300 for estimating an object's pose based on three-dimensional features includes:

[0060] Data acquisition module 301: acquires an RGB image set and a corresponding depth atlas containing a target object, and preprocesses the RGB image set and the corresponding depth atlas.

[0061] The 3D point cloud reconstruction module 302 performs target detection on the RGB images in the RGB image set to generate a corresponding bounding box set, and reconstructs a 3D point cloud based on the bounding box set to generate a 3D point cloud containing target object features.

[0062] The pose rough estimation module 303 obtains a three-dimensional digital model corresponding to the target object, and generates a first pose based on the three-dimensional digital model and the bounding box set.

[0063] The pose optimization module 304 performs a pose optimization operation based on the first pose and the three-dimensional point cloud to generate a second pose, and calculates a pose evaluation result of the target object based on the first pose and the second pose.

[0064] It should be noted that the above-mentioned embodiment provides an apparatus for estimating the object pose based on three-dimensional features, and only uses the division of the above-mentioned program modules as an example to illustrate when performing object pose estimation based on three-dimensional features. In actual applications, the above-mentioned processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the apparatus for estimating the object pose based on three-dimensional features provided in the above-mentioned embodiment and the embodiment of the method for estimating the object pose based on three-dimensional features belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0065] The object pose estimation method based on three-dimensional features provided in the embodiment of the present invention can be implemented on the terminal side or the server side. As for the hardware structure of the object pose estimation terminal based on three-dimensional features, please refer to Figure 4 , which is an optional hardware structure diagram of the terminal 400 for estimating the object pose based on three-dimensional features provided in an embodiment of the present invention. The terminal 400 may be a mobile phone, a computer device, a tablet device, a personal digital processing device, a factory background processing device, etc. The terminal 400 for estimating the object pose based on three-dimensional features includes: at least one processor 401, a memory 402, at least one network interface 404 and a user interface 406. The various components in the device are coupled together through a bus system 405. It can be understood that the bus system 405 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 405 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 4 Various buses are labeled as bus systems.

[0066] The user interface 406 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0067] It will be appreciated that the memory 402 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM) or a programmable read-only memory (PROM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memories.

[0068] The memory 402 in the embodiment of the present invention is used to store various categories of data to support the operation of the object pose estimation terminal 400 based on three-dimensional features. Examples of these data include: any executable program for operating on the object pose estimation terminal 400 based on three-dimensional features, such as an operating system 4021 and an application 4022; the operating system 4021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 4022 can include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The object pose estimation method based on three-dimensional features provided in the embodiment of the present invention can be included in the application 4022.

[0069] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 401. Processor 401 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 401 or by software instructions. The above processor 401 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 401 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor 401 may be a microprocessor or any conventional processor. The steps of the accessory optimization method provided in conjunction with the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium located in a memory. The processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0070] In an exemplary embodiment, the three-dimensional feature-based object pose estimation terminal 400 can be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), and complex programmable logic devices (CPLDs) to execute the aforementioned three-dimensional feature-based object pose estimation method.

[0071] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with a computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0072] In the embodiments provided herein, the computer readable and writable storage medium may include a read-only memory, a random access memory, an EEPROM, a CD-ROM or other optical disk storage device, a magnetic disk storage device or other magnetic storage device, a flash memory, a USB flash drive, a mobile hard disk, or any other medium that can be used to store desired program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer readable medium. For example, if the instruction is sent from a website, a server or other remote source using a coaxial cable, a fiber optic cable, a twisted pair, a digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwaves, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwaves are included in the definition of the medium. However, it should be understood that computer readable and writable storage media and data storage media do not include connections, carriers, signals or other temporary media, but are intended to be non-temporary, tangible storage media. Disk and disc, as used in this application, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

[0073] In summary, this application provides a method, device, terminal, and medium for object pose estimation based on three-dimensional features. The present invention provides a method for improving the efficiency and accuracy of object pose estimation by acquiring RGB images and depth images, performing target detection on the RGB images to generate a bounding box, and then reconstructing a three-dimensional point cloud within the bounding box in combination with the depth map. Subsequently, a three-dimensional digital model of the target object is introduced as prior knowledge for a rough pose estimation, and a matching algorithm is then used to match the three-dimensional digital model with the point cloud data for pose optimization.

[0074] This application utilizes a three-dimensional digital model as prior knowledge and uses the global nature of the three-dimensional digital model to fully match the object's pose, significantly improving the accuracy of the resulting pose estimation results. This approach remains unaffected even if the target features in the captured image are obscured, resolving the issues of pose ambiguity, poor stability, and poor reliability that existing object pose estimation technologies cannot address. Therefore, this application effectively overcomes the various shortcomings of the existing technology and possesses high industrial value.

[0075] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A method for estimating object pose based on three-dimensional features, characterized in that: include: Collecting an RGB image set and a corresponding depth atlas containing a target object, and preprocessing the RGB image set and the corresponding depth atlas; performing a target detection operation on the RGB images in the RGB image set to generate a corresponding bounding box set, and reconstructing a three-dimensional point cloud based on the bounding box set to generate a three-dimensional point cloud containing features of the target object; Acquire a three-dimensional digital model corresponding to the target object, and generate a first pose based on the three-dimensional digital model and the bounding box set; Performing a pose optimization operation based on the first pose and the three-dimensional point cloud to generate a second pose; A pose evaluation result of the target object is calculated based on the first pose and the second pose.

2. The object pose estimation method based on three-dimensional features according to claim 1, characterized in that: The process of generating the first pose includes: Performing a cropping operation on the RGB image set and the corresponding depth atlas according to the bounding box set; Generate a multi-angle model rendering image set of the target object using the three-dimensional digital model; The cropped RGB image set and the corresponding depth atlas, as well as the multi-angle model rendering image set, are input into a deep learning network to generate the first pose.

3. The object pose estimation method based on three-dimensional features according to claim 2, characterized in that: The process of generating the second pose includes: Based on the first posture, performing a posture conversion operation on the three-dimensional digital model; Based on a matching algorithm, a data matching operation is performed on the three-dimensional digital model after the posture conversion operation and the three-dimensional point cloud data to generate a second posture.

4. The object pose estimation method based on three-dimensional features according to claim 1, characterized in that: The process of calculating the pose evaluation result of the target object includes: performing matrix multiplication on the first pose and the second pose to obtain the pose evaluation result.

5. The object pose estimation method based on three-dimensional features according to claim 1, characterized in that: The process of generating a three-dimensional point cloud containing target object features includes: based on the bounding box set, cropping the depth map in the depth map set to generate local depth information of the object; and generating a three-dimensional point cloud containing target object features based on the local depth information of the object.

6. The object pose estimation method based on three-dimensional features according to claim 1, characterized in that: The method further comprises: Performing a pose estimation operation on two target objects using the object pose estimation method based on three-dimensional features as claimed in claim 1 to generate corresponding first pose evaluation results and second pose evaluation results; A matrix multiplication operation is performed on the first pose evaluation result and the second pose evaluation result to obtain a relative pose relationship between the first target object and the second target object.

7. The object pose estimation method based on three-dimensional features according to claim 1, characterized in that: The process of preprocessing the RGB image set and the corresponding depth atlas includes: Aligning the depth atlas with the RGB image set according to preset acquisition parameters; A resolution unification operation is performed on the RGB image set and the depth atlas.

8. A device for estimating object pose based on three-dimensional features, characterized in that: include: Data acquisition module: collects RGB image sets and corresponding depth atlases containing target objects, and preprocesses the RGB image sets and corresponding depth atlases; A 3D point cloud reconstruction module performs target detection on the RGB images in the RGB image set to generate a corresponding bounding box set, and reconstructs a 3D point cloud based on the bounding box set to generate a 3D point cloud containing target object features; A pose rough estimation module is configured to obtain a three-dimensional digital model corresponding to the target object and generate a first pose based on the three-dimensional digital model and the bounding box set; A pose optimization module: performing a pose optimization operation based on the first pose and the three-dimensional point cloud to generate a second pose; A pose evaluation result of the target object is calculated based on the first pose and the second pose.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the object pose estimation method based on three-dimensional features described in any one of claims 1 to 7 is implemented.

10. An electronic terminal, characterized in that: include: processor and memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory, so that the terminal performs the object pose estimation method based on three-dimensional features as described in any one of claims 1 to 7.

Images