DEPTH ESTIMATION BASED ON FEATURE RECONSTRUCTION WITH ADAPTIVE CLOSING AND MOTION PREDICTION

Abstract

Systems and techniques are provided for generating depth information. For example, the process may include obtaining a first feature volume that includes visual features corresponding to each respective frame included in a first set of frames. A first query generator network may generate reconstruction features corresponding to the reconstructed feature volume corresponding to the first feature volume. Based on the first feature volume, a second query generator network may generate motion features corresponding to predicted future motion corresponding to the first feature volume. An initial depth prediction may be generated for each respective frame based on cross-attention among features from the depth prediction decoder, reconstruction features, and motion features.Refined depth predictions can be generated for each frame respectively based on cross-attention among the initial depth predictions, reconstruction features, and motion features.