Fusing, texturing, and rendering views of dynamic three-dimensional models

Abstract

Various approaches described herein improve the quality of results when fusing depth maps to generate dynamic three-dimensional ('3D') models, applying texture details to dynamic 3D models, or rendering views of textured, dynamic 3D models. For example, when fusing depth maps to generate a dynamic 3D model, a fusion component can also incorporate intrinsic color values for points of the dynamic 3Dmodel, potentially making the dynamic 3D model more accurate, especially for areas in which depth values are not reliable or not available. As another example, when applying texture details, a rendering component can apply smoothed, viewpoint-dependent texture weights to texture values from different texture maps, which can reduce blurring and avoid the introduction of noticeable seams. As another example, a rendering component can apply special effects indicated by metadata to rendered views, thereby allowing a content provider to assert artistic control over presentation.

Description

Background technique [0001] Virtual reality ("VR") technology simulates a user's physical presence in a synthetic computer-generated environment. Typically, a VR device includes a VR headset with a display screen and headphones to present realistic images and sounds for a computer-generated environment. With VR technology, users can look around a computer-generated environment and, in many cases, browse and interact with features of the environment. In some VR systems, a user can communicate with one or more other remote users who are virtually present in the environment. Similarly, augmented reality (“AR”) technology superimposes artificial, computationally-generated content on top of camera input and / or audio mixed. [0002] While some VR systems generate a synthetic environment from scratch, many VR and AR systems use devices that capture details of the physical environment to create a three-dimensional ("3D") model of the actual physical environment. For example, the p...

AI Technical Summary

Problems solved by technology

Although such systems can produce high-fidelity dynamic 3D models in many scenes, the resulting dynamic 3D models may be inaccurate or incomplete when depth values ​​are unreliable or unavailable

Also, in some cases, previous methods of stitching texture details from different viewpoints onto dynamic 3D models can produce blurred details or obvious seams, especially for regions of interest such as faces, which can degrade the overall quality

Finally, previous methods of rendering views of dynamic 3D models are limited in terms of how artistic effects can be applied to the rendered views, especially for real-time applications

Images