Video Depth Estimation Using Temporal Attention Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current depth estimation methods in video processing do not effectively utilize temporal consistency between frames, limiting the accuracy of depth estimation in video sequences.
Innovation Solution
A video depth estimation system that includes a motion compensator, a temporal attention subsystem, and a depth estimator, which utilizes temporal attention maps to align and emphasize consistent information across frames, improving depth estimation accuracy by considering the similarity levels between feature maps from multiple frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If temporal consistency between frames is utilized, then depth estimation accuracy is improved, but device complexity increases
Solution Approach 1:
The system segments the temporal information processing by dividing video frames into multiple feature maps at different temporal stages. Each feature map processes specific temporal relationships, allowing the complex task of temporal consistency utilization to be broken down into manageable segments that improve depth estimation without overwhelming system complexity
Solution Approach 2:
The patent introduces temporal attention maps as intermediary elements that mediate between multiple video frames and the final depth estimation. These attention maps serve as intermediate representations that capture temporal consistency information, enabling accurate depth estimation while maintaining a structured processing pipeline that manages complexity
2Measurement precision
If multiple video frames are processed, then depth estimation accuracy is improved, but processing time increases
Solution Approach 1:
The system performs preliminary processing by extracting feature maps from multiple video frames before conducting the actual depth estimation. This preliminary extraction of temporal features allows the main depth estimation process to work with pre-processed information, reducing the computational burden during final processing while maintaining high accuracy
Solution Approach 2:
The patent applies partial processing by selectively processing only the necessary temporal relationships through attention mechanisms. Rather than exhaustively processing all possible frame combinations, the system focuses on key temporal relationships that most contribute to depth accuracy, achieving good results with reduced processing overhead
Data Source
AI summary
A method of depth detection based on a plurality of video frames includes receiving a plurality of input frames including a first input frame, a second input frame, and a third input frame respectively corresponding to different capture times, convolving the first to third input frames to generate a first feature map, a second feature map, and a third feature map corresponding to the different capture times, calculating a temporal attention map based on the first to third feature maps, the temporal attention map including a plurality of weights corresponding to different pairs of feature maps from among the first to third feature maps, each weight of the plurality of weights indicating a similarity level of a corresponding pair of feature maps, and applying the temporal attention map to the first to third feature maps to generate a feature map with temporal attention.