Uncertainty-Aware Fusion for Large-Scale NeRF

JP2024541049AActive Publication Date: 2024-11-06NEC CORP

Patent Information

Application Number
JP2024525800
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2022-10-11
Filing Date
2022-10-20
Publication Date
2024-11-06
Estimated Expiration
2042-10-20

AI Technical Summary

Technical Problem

Neural Radiance Fields (NeRF) are limited to small-scale scenes due to the limited capacity of multilayer perceptrons (MLPs), leading to a trade-off between scale and resolution, and lack flexibility in representing different scene complexities.

Method used

Implementing a method that clusters images into separate groups, trains individual NeRFs and uncertainty MLPs for each cluster, applies rendering and entropy losses, and performs uncertainty-based fusion to create a fused NeRF for large-scale scenes.

Benefits of technology

Enables high-fidelity novel perspective synthesis and 3D reconstruction for large-scale scenes by allowing flexible representation capacity and granularity, leveraging distributed computing and improving geometric fidelity through uncertainty quantification and entropy regularization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024541049000001_ABST
    Figure 2024541049000001_ABST
Patent Text Reader

Abstract

A method for achieving high-fidelity novel viewpoint synthesis and 3D reconstruction of large-scale scenes is disclosed. The method includes acquiring images from video streams received from multiple video image capture devices, grouping the images into separate image clusters representing the large-scale 3D scene, and training a neural radiance field (NeRF) and an uncertainty multilayer perceptron (MLP). For each of the image clusters, generate multiple NeRFs and multiple uncertainty MLPs for the large-scale 3D scene, apply rendering loss and entropy loss to the multiple NeRFs, and perform uncertainty-based fusion on the multiple NeRFs. Define a fused NeRF, jointly fine-tune the multiple NeRFs and the multiple uncertainty MLPs, and apply the fused NeRF to novel viewpoint synthesis of the large-scale 3D scene during inference.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • Implementation method of NERF new view angle synthesis model based on self-integration module

    CN115457192A

  • Robust View Synthesis for Unconstrained Image Data

    JP2023543538A

Cited By

  • Dividing device

    WO2026120777A1