Processing videos using machine learning models

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing a video. In one aspect, a method comprises: obtaining a video comprising multiple video frames; obtaining a set of keyframes from the multiple video frames; for each keyframe, initializing a corresponding disparity map; for each keyframe, initializing a corresponding camera pose; at each of one or more iterations: updating the set of keyframes to include one or more new keyframes; for each new keyframe, initializing a corresponding disparity map; generating at least a set of camera pose updates and a set of disparity map updates; for one or more of the keyframes, applying the respective camera pose update to the corresponding camera pose for the keyframe; and for one or more of the keyframes, applying the respective disparity map update to the corresponding disparity map for the keyframe.

Images