Affine Motion Model for Frame Rate Up-Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current frame rate up-conversion techniques, such as bilateral matching, increase computation complexity without reducing bitstream size, and fail to effectively handle non-linear object motions like rotation and zooming, which are common in video content.
Innovation Solution
The implementation of an affine motion model in frame rate up-conversion, which uses a simplified motion vector field determined by control point vectors to capture non-linear motions, reducing computational complexity and bitstream size while improving coding efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If bilateral matching is used for frame rate up-conversion, then video quality is improved, but computation complexity increases without bitstream size reduction
Solution Approach 1:
The patent segments the motion representation into control point motion vectors (CPMVs) that define an affine transformation field. Instead of computing bilateral matching for all pixels, the method divides the block into regions governed by control points, reducing computation while maintaining quality through affine motion modeling.
Solution Approach 2:
The patent changes the motion model parameters from simple translational vectors to affine transformation parameters (CPMVs). This parameter change enables the model to represent complex non-linear motions (rotation, zooming) with fewer parameters, reducing computation complexity while improving video quality through more accurate motion compensation.
2Device complexity
If translational motion model is used, then computation is simpler, but non-linear motions like rotation and zooming are not effectively handled
Solution Approach 1:
The patent introduces dynamic affine transformation parameters (CPMVs) that can adapt to different motion patterns within a block. The control point motion vectors enable the motion model to dynamically adjust to non-linear motions such as rotation and zooming, making the system versatile while maintaining computational efficiency through parameterized transformation.
3Productivity
If affine motion model is applied to capture non-linear motions, then coding efficiency is improved, but bitstream size may increase
Solution Approach 1:
The patent applies partial affine transformation by using a limited number of control points (typically 2-4 per block) rather than full affine modeling for the entire block. This partial application captures the essential non-linear motion characteristics while keeping the bitstream overhead minimal, as only the CPMVs for control points need to be encoded rather than parameters for all pixels.
Data Source
Figure 1
Figure 2A~2B
Figure 3A~3B
AI summary
Methods, apparatuses, and computer-readable medium are provided for a frame rate up-conversion coding mode, in which an affine motion model is applied when conducting bilateral matching. The frame rate up-conversion coding mode can include generated additional frames from frames provided in a bitstream. In various implementations, bilateral matching includes, for a current block in a frame that is being generated, identifying a first block in a first reference picture a second block in a second reference picture. Affine (e.g., non-linear) motion information can be determined as between the first block and the second block. The current block can be predicted using the affine motion information.