Handheld object three-dimensional reconstruction method based on three-dimensional generation prior and semantic consistency

By generating 3D priors and semantic consistency methods, a 3D candidate set is generated using RGB image sequences. Combined with semantic segmentation and geometric priors, the object pose is optimized, solving the problem of inaccurate pose estimation in 3D reconstruction of handheld objects. This achieves high-quality 3D reconstruction and improves the practicality and generalization ability of the method.

CN120876780APending Publication Date: 2025-10-31ZHEJIANG UNIV +1
0 Cites 3 Cited by

Patent Information

Application Number
CN202510985090.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-17
Publication Date
2025-10-31

AI Technical Summary

Technical Problem

In existing 3D reconstruction technologies for handheld objects, inaccurate object pose estimation leads to low reconstruction quality, and the technology relies on high-precision CAD models and datasets, making it difficult to generalize to diverse handheld objects.

Method used

By generating 3D priors and semantic consistency methods, a 3D candidate set is generated using RGB image sequences. Combined with semantic segmentation and geometric priors, the object pose is optimized, and a neural radiation field network for object reconstruction is constructed. The pose is iteratively optimized to improve accuracy.

Benefits of technology

This method achieves high-quality 3D reconstruction without the need for CAD models and dataset supervision, improves the practicality and generalization ability of the method, mitigates the impact of pose estimation errors, and significantly improves the accuracy and stability of the reconstruction results.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses a handheld object three-dimensional reconstruction method based on three-dimensional generation prior and semantic consistency, and the method comprises the steps: obtaining the three-dimensional prior from an input RGB image sequence in combination with a pre-trained multi-modal model and a three-dimensional generation model; performing semantic alignment on the three-dimensional priori and an input image based on feature similarity measurement of a visual basic model, and preliminarily estimating the pose of an object; completing initial three-dimensional reconstruction of the object according to the rough pose by using a neural radiation field; fine adjustment is carried out on the object pose based on the initial reconstruction result; and generating a high-precision three-dimensional reconstruction result through a nerve radiation field by using the optimized pose. The method has the advantages that the inherent problem of pose estimation in a hand-held object scene is solved through semantic consistency constraint; high-precision object three-dimensional reconstruction can be realized only by depending on RGB video data easy to obtain; dependence of a traditional method on complex sensor data or manual annotation is remarkably reduced, and reconstruction efficiency and practicability are effectively improved.
Need to check novelty before this filing date? Find Prior Art