Disentangling content and camera control for image view synthesis

Abstract

Example embodiments of aspects of the present disclosure provide techniques for training and implementing machine learning models for image view synthesis that feature separately controllable camera pose and scene content dynamics. The proposed techniques include a latent swapping training approach that enables the learning of a latent camera space and a latent content space that are disentangled from each other, enabling separate control over the camera pose and the scene content dynamics for synthetic imagery generated by an image synthesis model. In particular, the proposed models and training techniques can be applied to dynamic scenes by extending the latent pose estimation through time.

Images