Local irradiance estimation for use in rendering an image

Abstract

Relightable free-viewpoint rendering allows a novel view of a scene to be rendered and relit based on multiple views of the scene from multiple camera viewpoints. Image values from the multiple camera viewpoints can be separated into diffuse image components and specular image components, such that an intrinsic colour component of a relightable texture can be determined for a specular scene, by using the separated diffuse image components. Furthermore, surface normals of geometry in the scene can be refined by constructing a height map based on a conservative component of an initial surface normal field and then determining the refined surface normals based on the constructed height map.

Description

technical field [0001] This disclosure relates to relightable textures for use in rendering images. Background technique [0002] An image of the scene may be captured from the viewpoint of the camera. In some cases, there may be more than one camera capturing different images of the scene. Each image of the scene represents a view of the scene from the viewpoint of the corresponding camera. However, there will be some viewpoints of the scene that will not correspond to any of the camera viewpoints. An image can be a video sequence of frames. Techniques such as Free Viewpoint Video Rendering (FVVR) allow generation of images representing new views of a scene based on a set of multiple views of the scene from multiple camera viewpoints. The cameras are preferably calibrated and synchronized with each other to facilitate inferring intermediate images of the scene. [0003] Based on different images of the scene, for example using multi-view stereo (MVS), a model of the sc...

AI Technical Summary

Problems solved by technology

However, this method requires expensive equipment and is usually limited to static scenes

Relighting a scene with an arbitrary lighting arrangement is considerably more challenging

Images