Neural Radiance Field Rig for Controllable 3D Shape and Appearance

The patent segments the 3D representation into two distinct components: a volumetric appearance model (NeRF) for high-quality rendering and a parametric shape model (SMPL) for controllable pose and shape manipulation. This segmentation allows each component to specialize in its strength while working together through a unified framework that maintains both photorealistic quality and geometric controllability.

The patent introduces an intermediary deformation field that acts as a bridge between the controllable parametric shape model and the appearance representation. This deformation field transforms the canonical 3D points from the parametric model into deformed space, allowing the appearance model to render high-quality images of posed shapes without being constrained by the limitations of either model alone.

The patent merges two established models (NeRF for appearance and SMPL for shape) into a unified framework that leverages the strengths of both. By combining these models rather than developing a completely new complex system, the patent achieves high rendering accuracy while managing complexity through the use of proven, interoperable components.

The patent performs preliminary action by pre-processing the input images to extract initial blendshape and camera parameters before the main rendering process. This preliminary extraction of structural information reduces the computational burden during novel view synthesis, as the deformation field and appearance model can work with pre-extracted parameters rather than processing raw images from scratch.