Object pose estimation method and device based on anchor frame characteristics, medium and terminal

Abstract

The invention provides an anchor frame feature-based object pose estimation method and device, a terminal and a medium, and the method comprises the steps: collecting an RGB image and a depth map, and carrying out the target detection and anchor frame positioning of the RGB image. And performing three-dimensional point cloud reconstruction by using the anchor frame and the corresponding depth map to generate three-dimensional point cloud data of the target object. And finally, based on the three-dimensional point cloud, calculating an inter-object deflection angle and a three-axis translation amount of the target object so as to realize object pose estimation. According to the method, the pose estimation efficiency is remarkably improved. The method has the advantages of being small in calculation amount, high in calculation speed and suitable for real-time processing, and different anchor frames can be flexibly arranged according to changes of a target object so as to meet various task requirements.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a method, device, medium, and terminal for estimating object pose based on anchor frame features. Background Art

[0002] Object pose estimation is a crucial problem in computer vision, aiming to detect and localize three-dimensional objects from images or point cloud data and estimate their six-degree-of-freedom (6DoF) pose relative to a reference coordinate system. In recent years, the application of deep learning and end-to-end neural networks, such as DenseFusion, PVN3D, and G2L-Net, has improved the accuracy and efficiency of object pose estimation. These methods leverage the feature extraction and representation capabilities of deep neural networks, combined with the geometric information of point cloud data, and introduce new techniques such as point-wise embedding features, 3D keypoint voting, and differentiable proxy voting losses, further enhancing the performance and generalization of object pose estimation.

[0003] Object pose estimation technology provides a solution for the assembly or docking of multiple objects. Using cameras or sensors to acquire the pose information of target objects, the relative poses and pose differences between the objects are calculated, and robotic arms and other devices are controlled for docking or assembly. This technology enables high-precision pose measurement and calculation, real-time pose feedback and adjustment, and automated and intelligent docking or assembly, improving quality and efficiency while reducing manual intervention and errors.

[0004] However, existing object pose estimation technologies still face problems such as insufficient accuracy, weak robustness, and low computational efficiency in practical applications, making it difficult to meet the requirements of high-precision, high-efficiency, and high-reliability multi-object assembly or docking. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a method, device, medium and terminal for object pose estimation based on anchor frame features, which is used to solve the problems of insufficient accuracy, weak robustness and low computational efficiency faced by existing object pose estimation technologies in practical applications.

[0006] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides an object pose estimation method based on anchor frame features, including: collecting an RGB image set and a corresponding depth atlas containing an assembly and a target object, and preprocessing the RGB image set and the corresponding depth atlas; performing target detection on each RGB image in the RGB image set to generate multiple anchor frames containing the assembly and the target object; extracting the coordinate information of each anchor frame, and performing three-dimensional point cloud reconstruction based on the coordinate information of the anchor frame and the corresponding depth map in the depth atlas to generate a three-dimensional point cloud; based on the three-dimensional point cloud, calculating the inter-object deflection angle and three-axis translation between the assembly and the target object.

[0007] In some embodiments of the first aspect of the present application, the process of generating multiple anchor frames containing the assembly and the target object includes: constructing a target dataset based on a preset target detection area image, and training a target detection network based on the target dataset; inputting the RGB image into the target detection network to generate anchor frames containing the assembly and the target object.

[0008] In some embodiments of the first aspect of the present application, the target detection network includes: a YOLO series network, a Faster R-CNN network, and an SSD network.

[0009] In some embodiments of the first aspect of the present application, the process of calculating the inter-object deflection angle and three-axis translation between the assembly and the target object includes: fitting a three-dimensional point cloud to obtain multiple three-dimensional point cloud center points; calculating the assembly direction vector and the assembly pairing vector, as well as the target object direction vector and the target object pairing vector according to the three-dimensional center points; calculating the three-axis translation between the assembly and the target object based on the target object direction vector and the assembly pairing vector; and calculating the inter-object deflection angle between the assembly and the target object based on the assembly direction vector and the target object direction vector.

[0010] In some embodiments of the first aspect of the present application, the process of calculating the three-axis translation between the assembly and the target object includes: performing a coordinate system transformation operation on the assembly pairing vector based on the target object direction vector, and transforming the assembly pairing vector from the camera coordinate system to a coordinate system constructed with the target object as the center; and extracting the three-axis translation from the assembly pairing vector after performing the coordinate system transformation operation.

[0011] In some embodiments of the first aspect of the present application, the inter-object deflection angle between the assembly and the target object is calculated by the law of cosines.

[0012] In some embodiments of the first aspect of the present application, the process of reconstructing the three-dimensional point cloud includes: based on the coordinate information of the anchor frame, cropping the corresponding depth map in the depth map set to obtain local depth information, and reconstructing the three-dimensional point cloud based on the local depth information.

[0013] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides an object pose estimation device based on anchor frame features, including: a data acquisition module: acquiring an RGB image set and a corresponding depth atlas containing an assembly and a target object, and preprocessing the RGB image set and the corresponding depth atlas; an anchor frame generation module: performing target detection on each RGB image in the RGB image set to generate multiple anchor frames containing the assembly and the target object; a point cloud reconstruction module: extracting the coordinate information of each anchor frame, and performing three-dimensional point cloud reconstruction based on the coordinate information of the anchor frame and the corresponding depth map in the depth atlas to generate a three-dimensional point cloud; a pose estimation module: calculating the inter-object deflection angle and three-axis translation between the assembly and the target object based on the three-dimensional point cloud.

[0014] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the object pose estimation method based on anchor frame features.

[0015] To achieve the above-mentioned purpose and other related purposes, the fourth aspect of the present application provides an electronic terminal, including: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the object pose estimation method based on anchor frame features.

[0016] As described above, this application relates to a method, device, medium, and terminal for object pose estimation based on anchor frame features in the field of computer vision, which has the following beneficial effects: This application significantly improves the efficiency of pose estimation. It has the characteristics of low computational complexity, high computational speed, the ability to set different anchor frames based on object changes to meet various task requirements, and suitability for real-time processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of an embodiment of an object pose estimation method based on anchor frame features of the present application is shown.

[0018] Figure 2 A flowchart of another embodiment of the object pose estimation method based on anchor frame features of the present application is shown.

[0019] Figure 3A schematic diagram of the structure of a target object in an embodiment of an object pose estimation method based on anchor frame features of the present application is shown.

[0020] Figure 4 A structural diagram of an embodiment of an object pose estimation device based on anchor frame features of the present application is shown.

[0021] Figure 5 A schematic diagram of the structure of an electronic terminal for object pose estimation based on anchor frame features of the present application is shown. DETAILED DESCRIPTION

[0022] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0023] It should be noted that in the following description, reference is made to the accompanying drawings, which describe several embodiments of the present application. It should be understood that other embodiments may also be used, and that mechanical, structural, electrical, and operational changes may be made without departing from the spirit and scope of the present application. The following detailed description should not be considered restrictive, and the scope of the embodiments of the present application is limited only by the claims of the published patents. The terms used herein are only for describing specific embodiments and are not intended to limit the present application. Spatially related terms, such as "upper", "lower", "left", "right", "below", "below", "lower", "above", "upper", etc., may be used in the text to facilitate the description of the relationship between one element or feature shown in the figure and another element or feature.

[0024] In this application, unless otherwise specified or limited, the terms "mounted," "connected," "connect," "fixed," "holding," and the like should be understood broadly. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediary; or internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in this application based on specific circumstances.

[0025] Furthermore, as used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprise", "include" indicate the presence of the described features, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, "A, B, or C" or "A, B, and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition occur only when the combination of elements, functions, or operations is inherently mutually exclusive in some way.

[0026] In order to solve the problems in the above-mentioned background technology, the present invention provides a method, device, medium and terminal for object pose estimation based on anchor frame features, aiming to solve the problems of insufficient accuracy, weak robustness and low computational efficiency faced by existing object pose estimation technologies in practical applications. At the same time, in order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention are further described in detail through the following embodiments and in combination with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention and are not used to limit the invention.

[0027] Before further explaining the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations:

[0028] <1> Pose estimation: A technique used in computer vision to determine the position and orientation of an object in three-dimensional space.

[0029] <2> RGB image: A color image composed of three color channels: red, green, and blue, used to capture the appearance and color information of an object.

[0030] <3> Depth map: An image that shows how far objects in the scene are from the camera, usually using grayscale values ​​to represent distance information.

[0031] <4> 3D point cloud: A data set consisting of a large number of 3D points used to represent the 3D shape information of an object or scene.

[0032] <5> Anchor box: A rectangular bounding box used to locate and mark the position of the target object, which plays a guiding role in the object detection task.

[0033] <6> Deflection angle between objects: describes the rotation angle of an object relative to the reference coordinate system and is used to express the orientation of the object.

[0034] <7> Three-axis translation: represents the translation distance of an object on the three coordinate axes, and is used to describe the position of an object in space.

[0035] Embodiments of the present invention provide a method for estimating an object pose based on anchor frame features, an apparatus for performing such a method, a storage medium storing an executable program for implementing such a method, and a terminal. Regarding the implementation of such a method, embodiments of the present invention will describe exemplary implementation scenarios for such method.

[0036] like Figure 1 and Figure 2 FIG. 1 shows a flow chart of an object pose estimation method based on anchor frame features in an embodiment of the present invention. The object pose estimation method based on anchor frame features in this embodiment mainly includes the following steps:

[0037] Step S11: collecting an RGB image set and a corresponding depth atlas containing an assembly and a target object, and preprocessing the RGB image set and the corresponding depth atlas.

[0038] In one embodiment of the present invention, the RGB image and depth map of the 3D camera are acquired in real time, and the depth map is aligned to the RGB image using camera extrinsics. This application does not limit the specific brand and specifications of the 3D camera. It only requires a 3D camera that can capture the RGB and depth maps of the scene at the same time. The alignment process includes aligning each pixel in the depth map with the RGB image plane using the relative position and direction of the camera extrinsic matrix of the RGB camera and the depth camera described in . The RGB image and depth map are then scaled to a suitable size at the same time to unify the resolution. Exemplarily, the RGB image and depth map are scaled to a resolution of 640*480.

[0039] Step S12: performing target detection on each RGB image in the RGB image set to generate a plurality of anchor boxes containing the assembly and the target object.

[0040] In one embodiment of the present invention, the process of generating multiple anchor frames containing the assembly and the target object includes: constructing a target dataset based on a preset target detection area image, and training a target detection network based on the target dataset; inputting the RGB image into the target detection network to generate anchor frames containing the assembly and the target object.

[0041] In this embodiment, the process of using RGB images for real-time detection and outputting the anchor frame coordinates of the target area includes: analyzing the target area to be detected, and training and expressing relevant data sets based on the target detection area to generate a target detection network. For example, Figure 3The pose relationship between target object A and assembly object B in this embodiment is shown. In this embodiment, the target areas are the lugs of target object A and the hooks of assembly object B. A dataset is constructed and annotated based on the lugs of target object A and the hooks of assembly object B, and a corresponding object detection network is trained based on the dataset. Object detection networks used in this invention include, but are not limited to, YOLO networks, Faster R-CNN networks, and SSD networks.

[0042] Furthermore, the target detection network is used to output the anchor box of the target area in the RGB image in real time, that is, the anchor box coordinates. Figure 3 As shown, object A includes two hanging ears, object B includes two hooks, and the anchor frame includes the two hanging ears of object A and the two hooks of object B, so there are 4 anchor frames. Figure 3 The embodiments and descriptions of the target areas are only used to illustrate the present invention and are not intended to limit the present invention.

[0043] Step S13: extracting the coordinate information of each anchor frame, and performing 3D point cloud reconstruction based on the coordinate information of the anchor frame and the corresponding depth map in the depth map set to generate a 3D point cloud.

[0044] In one embodiment of the present invention, the process of reconstructing the three-dimensional point cloud includes: based on the coordinate information of the anchor frame, cropping the corresponding depth map in the depth map set to obtain local depth information, and reconstructing the three-dimensional point cloud according to the local depth information.

[0045] In one embodiment of the present invention, the process of reconstructing a 3D point cloud by combining pixels within the anchor frame region with depth information includes cropping the depth map obtained in step S11 using the anchor frame coordinate information obtained in step S12 to generate local depth information. The advantage of performing local feature cropping is that cropping the depth map can remove invalid areas, reduce computational complexity, improve accuracy, and adapt to specific tasks. Furthermore, the corresponding 3D point cloud is restored based on the camera intrinsic parameters and the local depth information.

[0046] Step S14: Based on the three-dimensional point cloud, the inter-object deflection angle and the three-axis translation between the assembly and the target object are calculated.

[0047] In one embodiment of the present invention, the process of calculating the inter-object deflection angle and three-axis translation between the assembly and the target object includes: fitting a three-dimensional point cloud to obtain a plurality of three-dimensional point cloud center points; calculating the assembly direction vector and the assembly pairing vector, as well as the target object direction vector and the target object pairing vector according to the three-dimensional center points; calculating the three-axis translation between the assembly and the target object based on the target object direction vector and the assembly pairing vector; and calculating the inter-object deflection angle between the assembly and the target object based on the assembly direction vector and the target object direction vector.

[0048] In one embodiment of the present invention, by collecting multiple RGB images and corresponding depth maps under multiple frames and multiple camera perspectives, a point cloud set of multiple target areas is obtained according to steps S11 to S13, and the three-axis mean is calculated based on the point cloud set to fit the center point of the point cloud, as shown in Formula 1.

[0049]

[0050]

[0051]

[0052] Among them, x ij 、y ij 、z ij They represent the x-axis, y-axis, and z-axis coordinate values ​​of the j-th point cloud in the ith viewpoint or the ith frame, respectively. There are a total of m point clouds and n viewpoints or frames.

[0053] Furthermore, the process of calculating the three-axis translation between the assembly and the target object includes: performing a coordinate system transformation operation on the assembly pairing vector based on the target object direction vector, transforming the assembly pairing vector from the camera coordinate system to a coordinate system constructed with the target object as the center; and extracting the three-axis translation from the assembly pairing vector after performing the coordinate system transformation operation.

[0054] In one embodiment of the present invention, the direction vector points from a paired component of a single object to its own paired component, and the pairing vector points from a paired component of a single object to the corresponding paired component of another object. By transforming the assembly pairing vector in the camera coordinate system using the coordinate system, the three-dimensional translation of assembly A in the coordinate system of target object B is calculated.

[0055] For example, by using the pairing vector calculated in the camera coordinate system and performing the coordinate system transformation operation, the three-dimensional translation of assembly A in the coordinate system of target object B can be obtained. In this embodiment, the transformed pairing vector v is shown in Formula 2.

[0056] v = v pair * R (Formula 2)

[0057]

[0058] Among them, v pair Represents the pairing vector obtained in the camera coordinate system, R represents the transformation matrix from the camera coordinate system to the target object B coordinate system. The direction vector v of the target object B is dir As shown in Formula 4, the positive direction of the camera x-axis v x is (1,0,0).

[0059] v dir = (x,y,z) (Formula 4)

[0060] The deflection angle β between the x-axis of the camera coordinate system and the x-axis of the target object B coordinate system is shown in Formula 5.

[0061]

[0062] Furthermore, the inter-object deflection angle between the assembly and the target object is calculated by the law of cosines.

[0063] As shown in Formula 6.

[0064]

[0065] Where α represents the deflection angle between the target object and the assembly, v1 and v2 represent the target object direction vector and the assembly direction vector, respectively.

[0066] like Figure 4 FIG. 4 is a block diagram showing a device for estimating an object pose based on anchor frame features according to an embodiment of the present invention. In this embodiment, the device 400 for estimating an object pose based on anchor frame features includes:

[0067] Data acquisition module 401: acquires an RGB image set and a corresponding depth atlas containing an assembly and a target object, and pre-processes the RGB image set and the corresponding depth atlas.

[0068] Anchor frame generation module 402: performs target detection on each RGB image in the RGB image set to generate multiple anchor frames containing the assembly and the target object.

[0069] Point cloud reconstruction module 403: extracts the coordinate information of each anchor frame, and performs 3D point cloud reconstruction based on the coordinate information of the anchor frame and the corresponding depth map in the depth map set to generate a 3D point cloud.

[0070] The pose estimation module 404 calculates the inter-object deflection angle and the three-axis translation between the assembly and the target object based on the three-dimensional point cloud.

[0071] It should be noted that the above-mentioned embodiment provides an apparatus for estimating the object pose based on anchor frame features, and only uses the division of the above-mentioned program modules as an example when performing object pose estimation based on anchor frame features. In actual applications, the above-mentioned processing can be assigned to different program modules as needed, that is, the internal structure of the apparatus can be divided into different program modules to complete all or part of the processing described above. In addition, the apparatus for estimating the object pose based on anchor frame features provided in the above-mentioned embodiment and the embodiment of the method for estimating the object pose based on anchor frame features belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0072] The object pose estimation method based on anchor frame features provided in the embodiment of the present invention can be implemented on the terminal side or the server side. As for the hardware structure of the object pose estimation terminal based on anchor frame features, please refer to Figure 5 , which is an optional hardware structure diagram of the object pose estimation terminal 500 based on anchor frame features provided in an embodiment of the present invention. The terminal 500 can be a mobile phone, a computer device, a tablet device, a personal digital processing device, a factory background processing device, etc. The object pose estimation terminal 500 based on anchor frame features includes: at least one processor 501, a memory 502, at least one network interface 504 and a user interface 506. The various components in the device are coupled together through a bus system 505. It can be understood that the bus system 505 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 505 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 5 Various buses are labeled as bus systems.

[0073] The user interface 506 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0074] It will be appreciated that the memory 502 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM) or a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0075] The memory 502 in the embodiment of the present invention is used to store various categories of data to support the operation of the object pose estimation terminal 500 based on anchor frame features. Examples of these data include: any executable program for operating on the object pose estimation terminal 500 based on anchor frame features, such as an operating system 5021 and an application 5022; the operating system 5021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 5022 can include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The object pose estimation method based on anchor frame features provided in the embodiment of the present invention can be included in the application 5022.

[0076] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 501. Processor 501 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 501 or by software instructions. The above processor 501 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 501 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor 501 may be a microprocessor or any conventional processor. The steps of the accessory optimization method provided in conjunction with the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium located in a memory. The processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0077] In an exemplary embodiment, the object pose estimation terminal 500 based on anchor frame features can be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), and complex programmable logic devices (CPLDs) to execute the aforementioned object pose estimation method based on anchor frame features.

[0078] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with a computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0079] In the embodiments provided herein, the computer readable and writable storage medium may include a read-only memory, a random access memory, an EEPROM, a CD-ROM or other optical disk storage device, a magnetic disk storage device or other magnetic storage device, a flash memory, a USB flash drive, a mobile hard disk, or any other medium that can be used to store desired program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer readable medium. For example, if the instruction is sent from a website, a server or other remote source using a coaxial cable, a fiber optic cable, a twisted pair, a digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwaves, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwaves are included in the definition of the medium. However, it should be understood that computer readable and writable storage media and data storage media do not include connections, carriers, signals or other temporary media, but are intended to be non-temporary, tangible storage media. Disk and disc, as used in this application, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

[0080] In summary, the present application provides an object pose estimation method, device, terminal and medium based on anchor frame features. The present invention provides a method for improving the efficiency of object pose estimation by collecting RGB images and depth maps, and performing target detection and anchor frame positioning on the RGB images. Subsequently, the anchor frame and depth map are used to reconstruct a three-dimensional point cloud to generate three-dimensional point cloud data of the target object. Based on the three-dimensional point cloud, the inter-object deflection angle and three-axis translation of the target object are calculated, and finally the object pose estimation is achieved. The efficiency of pose estimation is significantly improved. It has the characteristics of small computational complexity and fast operation speed, different anchor frames can be set according to object changes, adapting to various task requirements, and suitable for real-time processing. Therefore, the present application effectively overcomes the various shortcomings in the prior art and has high industrial utilization value.

[0081] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A method for object pose estimation based on anchor frame features, characterized in that: include: Acquire an RGB image set and a corresponding depth atlas containing an assembly and a target object, and preprocess the RGB image set and the corresponding depth atlas; Performing target detection on each RGB image in the RGB image set to generate a plurality of anchor boxes containing the assembly and the target object; Extracting coordinate information of each anchor frame, and performing three-dimensional point cloud reconstruction based on the coordinate information of the anchor frame and the corresponding depth map in the depth map set to generate a three-dimensional point cloud; Based on the three-dimensional point cloud, an inter-object deflection angle and a three-axis translation between the assembly and the target object are calculated.

2. The object pose estimation method based on anchor frame features according to claim 1, characterized in that The process of generating multiple anchor boxes containing the assembly and the target object includes: Constructing a target data set based on a preset target detection area image, and training a target detection network based on the target data set; The RGB image is input into the object detection network to generate an anchor box containing the assembly and the target object.

3. The object pose estimation method based on anchor frame features according to claim 2, characterized in that: The target detection network includes: YOLO series network, Faster R-CNN network and SSD network.

4. The object pose estimation method based on anchor frame features according to claim 1, characterized in that The process of calculating the inter-object deflection angle and the three-axis translation between the assembly and the target object includes: Fitting the three-dimensional point cloud to obtain multiple three-dimensional point cloud center points; Calculating the assembly direction vector and the assembly pairing vector, as well as the target object direction vector and the target object pairing vector respectively according to the three-dimensional center point; Calculating the three-axis translation between the assembly and the target object based on the target object direction vector and the assembly pairing vector; Based on the assembly direction vector and the target object direction vector, an inter-object deflection angle between the assembly and the target object is calculated.

5. The method for object pose estimation based on anchor frame features according to claim 4, characterized in that: The process of calculating the three-axis translation between the assembly and the target object includes: Based on the target object direction vector, performing a coordinate system conversion operation on the assembly pairing vector to convert the assembly pairing vector from a camera coordinate system to a coordinate system constructed with the target object as the center; The three-axis translation is extracted from the assembly pair vector after the coordinate system transformation operation is performed.

6. The method for object pose estimation based on anchor frame features according to claim 4, characterized in that: The inter-object deflection angle between the assembly and the target object is calculated by the law of cosines.

7. The object pose estimation method based on anchor frame features according to claim 1, characterized in that: The process of reconstructing the three-dimensional point cloud includes: based on the coordinate information of the anchor frame, cropping the corresponding depth map in the depth map set to obtain local depth information, and reconstructing the three-dimensional point cloud according to the local depth information.

8. An object pose estimation device based on anchor frame features, characterized in that: include: Data acquisition module: collects RGB image sets and corresponding depth atlases containing the assembly and the target object, and preprocesses the RGB image sets and corresponding depth atlases; Anchor frame generation module: performing target detection on each RGB image in the RGB image set to generate multiple anchor frames containing the assembly and the target object; Point cloud reconstruction module: extracts the coordinate information of each anchor frame and performs 3D point cloud reconstruction based on the coordinate information of the anchor frame and the corresponding depth map in the depth map set to generate a 3D point cloud; A pose estimation module calculates the inter-object deflection angle and three-axis translation between the assembly and the target object based on the three-dimensional point cloud.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for estimating an object pose based on anchor frame features according to any one of claims 1 to 7 is implemented.

10. An electronic terminal, characterized in that: include: processor and memory; The memory is used to store computer programs; The processor is configured to execute the computer program stored in the memory, so that the terminal performs the object pose estimation method based on anchor frame features as described in any one of claims 1 to 7.

Images