Method and device for estimating 6D pose of target in monocular RGB image

Abstract

The invention discloses a 6D pose estimation method and device for a target in a monocular RGB image, and relates to the technical field of pose estimation. According to the invention, based on the pre-constructed 3D Gaussian splash model of the target, multiple groups of virtual cameras surrounding the 3D Gaussian splash model of the target at different angles are formed, and multiple groups of virtual RGB images and corresponding depth images are generated through 3D Gaussian rendering based on the multiple groups of virtual cameras, so that 2D-2D feature matching is carried out on the multiple groups of virtual RGB images and the target RGB image. The pixel-level matching relation between the target RGB image and the depth image is established based on the corresponding depth image, so that matching from 2D to 3D is completed, the matching relation between the matching pixel of the target RGB image and the 3D point generated by the corresponding depth image is determined, the matching precision from 2D to 3D is improved with low cost, and the initial pose of the target can be solved and optimized based on the matching relation subsequently. And low cost is taken into account while the pose estimation precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of pose estimation, and in particular to a method and device for estimating the 6D pose of a target in a monocular RGB image. Background Art

[0002] Currently, the pose of a specific target can be estimated based on the target image. Among various sensing modalities, the RGB image-based method has attracted much attention due to its simplicity, low cost and wide applicability.

[0003] In the existing technology, some research proposals have proposed to estimate the 6D pose of the target based on a neural network using the target RGB image with depth information. However, the depth information needs to be collected using a specific depth camera, which increases the sensor cost. In addition, the 6D pose estimation is based on a machine learning algorithm, and a data set is required to train the proposed neural network.

[0004] Another research proposal proposes to establish the target's point cloud data through a series of target RGB images containing the target, and then determine the matching relationship between certain points in the point cloud data and the target RGB image through a 2D-3D matching network based on a neural network. Finally, the 6D pose of the target is obtained through the PNP solution algorithm. The 2D-3D matching network in this algorithm also requires a large data set for training.

[0005] In addition, there are research proposals that match the target RGB image with images in a pre-built reference dataset, and use the pose of the target in the image in the reference dataset that is most similar to the target RGB image as the pose of the target in the target RGB image, but the matching accuracy is low.

[0006] In summary, current methods for estimating the 6D pose of targets in monocular RGB images require additional depth information or need to be assisted by neural networks, which are costly and difficult to promote and apply. Solutions that do not rely on depth information and neural networks have low matching accuracy, making it difficult to achieve low-cost and high-precision 6D pose estimation of targets. Summary of the Invention

[0007] Based on this, it is necessary to provide a 6D pose estimation method and device for a target in a monocular RGB image to address the above technical problems.

[0008] The present invention adopts the following technical solutions: The present invention provides a 6D pose estimation method for a target in a monocular RGB image. The method comprises the following steps: firstly obtaining a target RGB image and a pre-built 3D Gaussian splash model of the target; the target RGB image is a monocular RGB image; then obtaining multiple sets of camera extrinsics at multiple angles of the 3D Gaussian splash model around the target, and generating multiple sets of virtual cameras by combining the camera intrinsic parameters; obtaining multiple sets of virtual RGB images and corresponding depth images by 3D Gaussian rendering based on the 3D Gaussian splash model of the target and the multiple sets of virtual cameras; and then performing 2D-2D feature extraction on the multiple sets of virtual RGB images and the target RGB image. The matching is performed and corresponded to the corresponding depth image to determine the matching relationship between the matching pixels of the target RGB image and the 3D points generated by the corresponding depth image based on the corresponding depth image and the internal and external parameters of each group of virtual cameras; the initial 6D pose of the target is determined based on the matching relationship, and an updated virtual RGB image is obtained through 3D Gaussian rendering based on the target's 3D Gaussian splash model, the target's initial 6D pose and the camera internal parameters; finally, the initial 6D pose is optimized to obtain the final 6D pose of the target with the goal of minimizing the deviation between the updated virtual RGB image and the target RGB image.

[0009] The present invention provides a 6D pose estimation device for a target in a monocular RGB image, comprising: An acquisition module is used to acquire a target RGB image and a pre-built 3D Gaussian splash model of the target; the target RGB image is a monocular RGB image; The virtual module is used to obtain multiple sets of camera extrinsic parameters at multiple angles of the 3D Gaussian splash model around the target and generate multiple sets of virtual cameras based on the camera intrinsic parameters. Based on the 3D Gaussian splash model of the target and the multiple sets of virtual cameras, multiple sets of virtual RGB images and corresponding depth images are obtained through 3D Gaussian rendering. A matching module is used to perform 2D-2D feature matching on multiple sets of virtual RGB images and target RGB images and map them to corresponding depth images, so as to determine the matching relationship between the matching pixels of the target RGB image and the 3D points generated by the corresponding depth image based on the corresponding depth image and the internal and external parameters of each set of virtual cameras; An initial solution module is used to determine the initial 6D pose of the target based on the matching relationship, and to obtain an updated virtual RGB image through 3D Gaussian rendering based on the target's 3D Gaussian splash model, the target's initial 6D pose, and the camera's intrinsic parameters; The optimization module is used to optimize the initial 6D pose to obtain the final 6D pose of the target by minimizing the deviation between the updated virtual RGB image and the target RGB image.

[0010] The present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method for estimating the 6D pose of a target in a monocular RGB image is implemented.

[0011] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for estimating the 6D pose of a target in a monocular RGB image is implemented.

[0012] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects: The present invention is based on a pre-built 3D Gaussian splash model of the target, forms multiple groups of virtual cameras at different angles around the 3D Gaussian splash model of the target, generates multiple groups of virtual RGB images and corresponding depth images through 3D Gaussian rendering based on the multiple groups of virtual cameras, thereby performing 2D-2D feature matching on the multiple groups of virtual RGB images and the target RGB image, and establishing a pixel-level matching relationship between the target RGB image and the depth image based on the corresponding depth image, thereby completing 2D to 3D matching, and determining the matching relationship between the matching pixels of the target RGB image and the 3D points generated by the corresponding depth image. The target RGB image does not need to carry depth information, that is, the target RGB image can be a monocular RGB image, and high-precision 2D to 3D matching can still be achieved. At the same time, without the assistance of a neural network algorithm, the initial pose of the target can be solved and optimized based on the 2D to 3D matching relationship, thereby improving the 6D pose estimation accuracy of the target in the monocular RGB image, while being low in cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 A schematic flow chart of a method for estimating the 6D pose of a target in a monocular RGB image provided by the present invention; Figure 2 A schematic diagram of a target pose estimation framework provided by the present invention; Figure 3 A schematic diagram of 2D-2D matching provided by the present invention; Figure 4 Schematic diagram of a 6D pose estimation device for a target in a monocular RGB image provided by the present invention. DETAILED DESCRIPTION

[0014] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0015] Currently, estimating 6D pose from a single RGB image remains a significant challenge. This is primarily due to the fact that algorithms must resolve ambiguities in depth and scale when relying solely on color information. Among existing methods for estimating the 6D pose of objects in monocular RGB images, the Zero-Shot 3D Pose Estimation of Unseen Objects by Two-step RGB-D Fusion requires depth information, which requires a specialized depth camera, increasing sensor costs. Furthermore, the proposed method relies on machine learning algorithms, requiring a dataset to train the proposed neural network.

[0016] OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CADMoDels estimates 6D pose using only monocular RGB information. This method first obtains a series of images of the target to be detected. It then uses the loftr 2D-2D matching algorithm to perform SFM to construct a point cloud. The feature vectors of the images extracted by loftr are then stored in the point cloud. This results in a point cloud with a set of feature vectors stored for each 3D point. The method then constructs a 2D-3D matching network based on a transformer architecture. The input is the point cloud and an unseen RGB image of the target. The output is the matching relationship between certain points in the point cloud and the target RGB image. The 6D pose is then determined using a PNP solver. The proposed 2D-3D matching network requires a large dataset for training.

[0017] GS-Pose: CascaDeD Framework for Generalizable Segmentation-baseD 6DObject Pose Estimation first obtains the initial position of the target and then obtains the final pose through 3D Gaussian iterative optimization. However, this method obtains the initial pose by matching the target RGB image with the images in the reference dataset, and uses the pose of the most similar photo in the reference dataset as the pose of the target RGB image. This does not achieve pixel-level matching, and in this process does not fully utilize the advantage of 3D Gaussian in rendering high-quality images. Therefore, this method has low accuracy in obtaining the initial pose.

[0018] The technical solutions provided by various embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0019] Figure 1 The figure is a flow chart of a method for estimating the 6D pose of a target in a monocular RGB image in the present invention, which specifically includes the following steps: S101: Acquire a target RGB image and a pre-built 3D Gaussian splash model of the target; the target RGB image is a monocular RGB image.

[0020] S102: Obtain multiple sets of camera extrinsic parameters at multiple angles of a 3D Gaussian splash model surrounding the target, and generate multiple sets of virtual cameras in combination with the camera intrinsic parameters; obtain multiple sets of virtual RGB images and corresponding depth images through 3D Gaussian rendering based on the 3D Gaussian splash model of the target and the multiple sets of virtual cameras.

[0021] S103: Perform 2D-2D feature matching on multiple sets of virtual RGB images and target RGB images and correspond them to corresponding depth images, so as to determine the matching relationship between the matching pixels of the target RGB image and the 3D points generated by the corresponding depth image based on the corresponding depth image and the internal and external parameters of each set of virtual cameras.

[0022] S104: Determine the initial 6D pose of the target based on the matching relationship, and obtain an updated virtual RGB image through 3D Gaussian rendering based on the target's 3D Gaussian splash model, the target's initial 6D pose, and the camera's intrinsic parameters.

[0023] S105: Taking minimizing the deviation between the updated virtual RGB image and the target RGB image as the optimization goal, the initial 6D pose is optimized to obtain the final 6D pose of the target.

[0024] For the sake of convenience, the following description will only be based on the server as the execution subject. The server mentioned in the present invention can be a server set up on a business platform, or a device such as a desktop computer or a laptop computer that can execute the solution of the present invention.

[0025] The proposed method for 6D pose estimation of a target in a monocular RGB image consists of three key stages: (1) 2D-2D image matching based on 3D Gaussian projection (3DGS); (2) 2D-3D correspondence computation for 6D pose estimation; and (3) optimization of the final pose using a 3D Gaussian splatter model. Figure 2 Schematic diagram of a target pose estimation framework in the present invention.

[0026] First, the present invention can obtain a target RGB image and a pre-built 3D Gaussian Splatting model (3DGS) of the target. The target RGB image is a monocular RGB image captured by a monocular camera without depth information.

[0027] A 3D Gaussian splatter model of a target may be pre-constructed. Specifically, in one or more embodiments of the present invention, the 3D Gaussian splatter model of the target is generated as follows: Goal: Generate a 3D Gaussian splatter model of a target from a set of images (reference images) with known camera intrinsic and extrinsic parameters.

[0028] Steps: Obtain an RGB image sequence of the target and generate point cloud data of the target using a 3D reconstruction algorithm (e.g., the Structure from Motion (SFM) algorithm). Based on the RGB image sequence and point cloud data of the target, generate a 3D Gaussian splatting model of the target using the Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction (PGSR) algorithm.

[0029] After that, the 6D pose estimation process of the target can be carried out: Algorithm inputs may include: 1. 3D Gaussian splash model of the target (generated by the previous preprocessing).

[0030] 2. The target RGB image of the pose to be detected, as well as the camera intrinsic parameters of the camera that captured the target RGB image (the internal parameters are fx, fy, cx, and cy; fx and fy represent the focal length, and cx and cy represent the position of the image optical center).

[0031] 3. A limited set of reference camera extrinsics.

[0032] Extrinsic parameters are the relative poses of the camera and the target. In one or more embodiments of the present invention, a library of viewpoints for a pre-set 3D Gaussian splatter model surrounding the target can be used to select viewpoints at multiple angles using the farthest point sampling method to form multiple sets of camera extrinsic parameters for the 3D Gaussian splatter model surrounding the target. The primary purpose here is to select multiple representative viewpoints.

[0033] Based on the selected multiple sets of camera extrinsic parameters and the determined camera intrinsic parameters, the server can first generate a finite number of virtual cameras around the 3D Gaussian splash model of the target, so that it covers various angles and constitutes a virtual camera set.

[0034] Then, based on the target's 3D Gaussian splatter model and multiple sets of virtual cameras, 3D Gaussian rendering can be used to generate multiple sets of virtual RGB images and corresponding depth images. Each set of depth images and virtual RGB images has a one-to-one correspondence in terms of pixels.

[0035] Based on this, a 2D-2D match can be performed between the target RGB image and the finite number of generated virtual RGB images, and the matching results can be mapped to the virtual depth map. Using the corresponding depth map and the internal and external parameters of the virtual camera, the matching relationship between the matching pixels of the target RGB image and the 3D points generated by the corresponding depth image is calculated.

[0036] Among them, after 2D-2D matching between the target RGB image and the generated finite virtual RGB images, a series of matching 2D pixel points will be obtained. For the virtual RGB image, it has a one-to-one correspondence with the corresponding depth image in terms of pixels. For each pixel in the depth image, assuming its pixel coordinates are ( u , v ), the depth value is d, then the matching pixels of the target RGB image can be mapped to the pixels of the depth image of the virtual RGB image based on the matching relationship, and the matching relationship between the matching pixels of the target RGB image and the pixels of the corresponding depth image is obtained.

[0037] Each pixel in the depth image can be converted to a 3D coordinate in the corresponding virtual camera's coordinate system based on the intrinsic parameters of the corresponding virtual camera. This 3D coordinate can then be converted to a 3D point in the world coordinate system based on the extrinsic parameters of the corresponding virtual camera. This is the 3D point generated by the corresponding depth image. Combining the above, we can derive a matching relationship between the matching pixels in the target RGB image and the 3D points generated by the corresponding depth image (in the world coordinate system).

[0038] Based on the matching relationship between the matching pixels of the target RGB image and the 3D points generated by the corresponding depth image (in the world coordinate system), the initial pose of the target can be obtained by solving it through the PNP (Perspective-n-Point) algorithm (for example, solvePNP, the algorithm in the OpenCV library). Figure 3This is a schematic diagram of 2D-2D matching in the present invention.

[0039] Finally, an updated virtual RGB image is rendered based on the target's initial pose, the target's 3D Gaussian splatter model, and the camera's intrinsic parameters. This updated virtual RGB image is compared with the target RGB image to calculate the loss. The deviation pose of the point cloud of the 3D Gaussian splatter model is then iteratively optimized. This process is repeated until the loss falls within an appropriate range or exceeds the number of iterations. The deviation pose is then multiplied by the initial pose to obtain the optimized pose.

[0040] Specifically, in one or more embodiments of the present invention, the server may determine the posture deviation of the target's 3D Gaussian splash model based on the deviation between the updated virtual RGB image and the target RGB image; optimize the posture deviation of the target's 3D Gaussian splash model with the goal of minimizing the deviation between the updated virtual RGB image and the target RGB image; and thereby optimize the target's initial 6D pose based on the posture deviation of the target's 3D Gaussian splash model to obtain the target's final 6D pose.

[0041] based on Figure 1 The 6D pose estimation method of the target in the monocular RGB image shown in the figure is based on a pre-built 3D Gaussian splash model of the target, forming multiple groups of virtual cameras at different angles around the 3D Gaussian splash model of the target, and generating multiple groups of virtual RGB images and corresponding depth images through 3D Gaussian rendering based on the multiple groups of virtual cameras, thereby performing 2D-2D feature matching on the multiple groups of virtual RGB images and the target RGB image, and establishing a pixel-level matching relationship between the target RGB image and the depth image based on the corresponding depth image, thereby completing 2D to 3D matching, determining the matching relationship between the matching pixels of the target RGB image and the 3D points generated by the corresponding depth image, and improving the 2D to 3D matching accuracy at a low cost. Subsequently, the initial pose of the target can be solved and optimized based on the 2D to 3D matching relationship, thereby improving the 6D pose estimation accuracy of the target in the monocular RGB image while taking into account low cost.

[0042] The present invention does not require training of a neural network model. In step S103, any general 2D-2D matching algorithm can be used. With the development of 2D-2D matching algorithms, the accuracy of this algorithm can be further improved. Good results have been achieved on the onepose dataset and linemoD dataset.

[0043] When applying the 6D pose estimation method of the target in the monocular RGB image provided by the present invention, it is not necessary to Figure 1 The steps are executed in the order shown. The specific execution order of the steps can be determined according to needs, and the present invention does not limit this.

[0044] The above is a method for estimating the 6D pose of a target in a monocular RGB image provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding device for estimating the 6D pose of a target in a monocular RGB image, such as Figure 4 shown.

[0045] Figure 4 A schematic diagram of a 6D pose estimation device for a target in a monocular RGB image provided by the present invention, comprising: An acquisition module 201 is configured to acquire a target RGB image and a pre-built 3D Gaussian splash model of the target; the target RGB image is a monocular RGB image; The virtual module 202 is configured to obtain multiple sets of camera extrinsic parameters at multiple angles of a 3D Gaussian splash model around the target and generate multiple sets of virtual cameras based on the camera intrinsic parameters; and to obtain multiple sets of virtual RGB images and corresponding depth images through 3D Gaussian rendering based on the 3D Gaussian splash model of the target and the multiple sets of virtual cameras; Matching module 203, configured to perform 2D-2D feature matching on the multiple sets of virtual RGB images and the target RGB images and map them to the corresponding depth images, thereby determining the matching relationship between the matching pixels of the target RGB images and the 3D points generated by the corresponding depth images based on the corresponding depth images and the intrinsic and extrinsic parameters of each set of virtual cameras; An initial solution module 204 is configured to determine the initial 6D pose of the target based on the matching relationship, and obtain an updated virtual RGB image through 3D Gaussian rendering based on the target's 3D Gaussian splash model, the target's initial 6D pose, and camera intrinsic parameters; The optimization module 205 is configured to optimize the initial 6D pose to obtain a final 6D pose of the target by minimizing the deviation between the updated virtual RGB image and the target RGB image.

[0046] Regarding the specific definition of the 6D pose estimation device for a target in a monocular RGB image, please refer to the definition of the 6D pose estimation method for a target in a monocular RGB image above, which will not be repeated here. Each module in the above-mentioned 6D pose estimation device for a target in a monocular RGB image can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0047] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 The 6D pose estimation method of the target in the monocular RGB image is provided.

[0048] The present invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The 6D pose estimation method of the target in the monocular RGB image is provided.

[0049] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes in the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0050] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A method for estimating the 6D pose of an object in a monocular RGB image, characterized in that: include: Obtaining a target RGB image and a pre-built 3D Gaussian splash model of the target; the target RGB image is a monocular RGB image; Obtain multiple sets of camera extrinsic parameters at multiple angles of a 3D Gaussian splash model around the target, and generate multiple sets of virtual cameras based on the camera intrinsic parameters. Based on the target's 3D Gaussian splash model and multiple sets of virtual cameras, perform 3D Gaussian rendering to obtain multiple sets of virtual RGB images and corresponding depth images. Perform 2D-2D feature matching on multiple sets of virtual RGB images and target RGB images and map them to the corresponding depth images. Then, based on the corresponding depth images and the intrinsic and extrinsic parameters of each set of virtual cameras, determine the matching relationship between the matching pixels of the target RGB image and the 3D points generated by the corresponding depth images. Determine the initial 6D pose of the target based on the matching relationship, and obtain an updated virtual RGB image through 3D Gaussian rendering based on the target's 3D Gaussian splash model, the target's initial 6D pose, and camera intrinsic parameters; Taking minimizing the deviation between the updated virtual RGB image and the target RGB image as the optimization goal, the initial 6D pose is optimized to obtain the final 6D pose of the target.

2. The method for estimating the 6D pose of a target in a monocular RGB image according to claim 1, wherein: Obtaining a 3D Gaussian splash model of the target, specifically including: Obtain the target's RGB image sequence and generate the target's point cloud data through a 3D reconstruction algorithm; According to the RGB image sequence and point cloud data of the target, a 3D Gaussian splash model of the target is generated by a plane-based Gaussian splash model generation technology.

3. The method for estimating the 6D pose of a target in a monocular RGB image according to claim 1, wherein: The step of obtaining multiple sets of camera extrinsic parameters at multiple angles of a 3D Gaussian splash model around a target specifically includes: According to the preset viewpoint library of the 3D Gaussian splash model around the target, viewpoints at multiple angles are selected by the farthest point sampling method to form multiple sets of camera extrinsics at multiple angles of the 3D Gaussian splash model around the target.

4. The method for estimating the 6D pose of a target in a monocular RGB image according to claim 1, wherein: The optimization objective is to minimize the deviation between the updated virtual RGB image and the target RGB image, and to optimize the initial 6D pose to obtain the final 6D pose of the target, specifically including: Determine the pose deviation of the target's 3D Gaussian splash model based on the deviation between the updated virtual RGB image and the target RGB image; The optimization goal is to minimize the deviation between the updated virtual RGB image and the target RGB image, and optimize the pose deviation of the target's 3D Gaussian splash model; The initial 6D pose of the target is optimized according to the pose deviation of the target's 3D Gaussian splash model to obtain the final 6D pose of the target.

5. A 6D pose estimation device for a target in a monocular RGB image, characterized in that: include: An acquisition module is used to acquire a target RGB image and a pre-built 3D Gaussian splash model of the target; the target RGB image is a monocular RGB image; The virtual module is used to obtain multiple sets of camera extrinsic parameters at multiple angles of the 3D Gaussian splash model around the target and generate multiple sets of virtual cameras based on the camera intrinsic parameters. Based on the 3D Gaussian splash model of the target and the multiple sets of virtual cameras, multiple sets of virtual RGB images and corresponding depth images are obtained through 3D Gaussian rendering. A matching module is used to perform 2D-2D feature matching on multiple sets of virtual RGB images and target RGB images and map them to corresponding depth images, so as to determine the matching relationship between the matching pixels of the target RGB image and the 3D points generated by the corresponding depth image based on the corresponding depth image and the internal and external parameters of each set of virtual cameras; An initial solution module is used to determine the initial 6D pose of the target based on the matching relationship, and to obtain an updated virtual RGB image through 3D Gaussian rendering based on the target's 3D Gaussian splash model, the target's initial 6D pose, and the camera's intrinsic parameters; The optimization module is used to optimize the initial 6D pose to obtain the final 6D pose of the target by minimizing the deviation between the updated virtual RGB image and the target RGB image.

6. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

7. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 4 when executing the program.

Images