Block-Based Motion Estimation in Deep Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video compression methods struggle to efficiently determine optimal motion vectors for motion-compensated prediction, leading to high bitrate contributions due to inefficient motion information transmission.
Innovation Solution
Employing classical block-based motion estimation techniques within a neural network framework to enhance the end-to-end video codec, incorporating block-based motion estimation strategies such as diamond search and sub-pel search, and modifying the cost criterion to include a rate term, along with improved distortion measures to estimate residual coding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If neural network-based motion search is used, then the end-to-end video compression is simplified, but the bitrate contribution from motion information transmission increases
Solution Approach 1:
The patent replaces the traditional mechanical block-matching motion estimation with a neural network-based approach. The neural network directly processes reference pictures to generate motion vectors, eliminating the need for complex search algorithms and reducing computational complexity while maintaining compression efficiency.
Solution Approach 2:
The patent changes the parameters of motion vector representation by using neural networks to directly output optimized motion vectors rather than traditional block-matching results. This parameter transformation enables more efficient bitrate allocation and reduces the bitrate contribution from motion information transmission.
2Measurement precision
If block-based motion estimation is used, then motion vector determination is optimized, but the computational cost increases
Solution Approach 1:
The patent segments the motion estimation process into block-based units, where each block independently determines its motion vector. This segmentation enables parallel processing and reduces the overall computational cost while maintaining high accuracy through localized optimization of motion vectors in each block.
Solution Approach 2:
The patent uses copying strategies where motion vectors are predicted based on neighboring blocks' motion vectors. This copying approach reduces computational cost by avoiding exhaustive search while maintaining accuracy through the exploitation of spatial correlation in motion fields.
3Measurement precision
If full search of motion vector candidates is performed, then optimal motion vectors are found, but the computational expense becomes too high
Solution Approach 1:
The patent applies partial search strategies where instead of exhaustively searching all possible motion vector candidates, the algorithm performs a limited but sufficient search. This partial action approach finds optimal or near-optimal motion vectors with significantly reduced computational expense, balancing accuracy and efficiency.
Solution Approach 2:
The patent incorporates feedback mechanisms where the neural network continuously refines motion vector estimates based on prediction error feedback. This feedback loop enables the system to converge on optimal motion vectors without requiring exhaustive search, reducing computational expense while maintaining optimality.
Data Source
AI summary
An apparatus for determining an encoding of a motion field for a picture of a video sequence comprising a sequence of pictures, such that said picture is decodable using a reference picture, the motion field and the residual, according to an embodiment is provided. The apparatus comprises a trained neural network configured to determine the encoding of the motion field, being associated with said picture, depending on said picture and depending on the reference picture.