System and method for frame interpolation for a compressed video bitstream

Abstract

A system and a method perform frame interpolation for a compressed video bitstream. The system and the method may combine candidate pictures to generate an interpolated video picture inserted between two original video pictures. The system and the method may generate the candidate pictures from different motion fields. The candidate pictures may be generated partially or wholly from motion vectors extracted from the compressed video bitstream. The system and the method may reduce computation required for interpolation of video frames without a negative impact on visual quality of a video sequence.

Description

[0001]This application claims the benefit of U.S. Provisional Application Ser. No. 61 / 207,381, filed Feb. 11, 2009.BACKGROUND OF THE INVENTION[0002]The present invention generally relates to a system and a method for frame interpolation for a compressed video bitstream. More specifically, the present invention relates to a system and a method that combine candidate pictures to generate an interpolated video picture inserted between two original video pictures. The system and the method may generate the candidate pictures from different motion fields. The candidate pictures may be generated partially or wholly from motion vectors extracted from the compressed video bitstream. The system and the method may reduce computation required for interpolation of video frames without a negative impact on visual quality of a video sequence.[0003]It is well known to utilize video compression to reduce a size of video data transmitted from a first location to a second location. A video encoder at...

AI Technical Summary

Benefits of technology

[0037]It is, therefore, an advantage of the present invention to provide a system and a method for frame interpolation for a compressed video bitstream.

[0038]Another advantage of the present invention is to provide a system and a method that combine motion compensated interpolations from a forward interpolation path, a backward interpolation path and / or a bi-directional interpolation path using a median filter.

[0039]And, another advantage of the present invention is to provide a system and a method that test reliability of motion vectors obtained from the bitstream without using block matching operations.

[0040]Yet another advantage of the present invention is to provide a system and a method that split a subset of blocks to improve interpolation quality in areas of complex local motion while maintaining a size of blocks where local motion is not complex.

[0041]Still further, an advantage of the present invention is to provide a system and a method that perform a blockwise artifact count estimation using SAD operations applied to three candidate interpolation pictures.

[0042]And, another advantage of the present invention is to provide a system and a method that reduce computation required for interpolation of video frames without a negative impact on visual quality of a video sequence.

Problems solved by technology

However, the “lossy encoding” technique causes loss of some of the video data.

Thus, use of the “lossy encoding” technique may result in visible degradation of visual quality, loss of spatial resolution of video frames and / or a reduced number of video frames displayed per second.

Motion estimates may be unreliable for upconversion techniques that utilize motion compensation to predict contents of the lost frames.

For example, the motion estimates may be unreliable due to fast or complex motion, uncovered or occluded areas and / or the like.

The unreliable motion estimates may introduce visible artifacts which may degrade visual quality of the upconverted video sequence.

In addition, the motion estimation may be challenging for mobile devices.

Since computational resources on a mobile device are scarce, the motion estimation and the motion compensation must be limited in computational complexity.

Instead, computationally limited mobile devices typically utilize a block-based motion estimation method that requires a small number of block matching operations.

Therefore, the motion estimation has a relatively low computational complexity.

A disadvantage of the block-based motion estimation method is that the method has limited capabilities and may provide erroneous motion estimates that may introduce visible artifacts into the temporally interpolated frames.

As discussed previously, visible artifacts located in the temporally interpolated frames degrade the visual quality of the upconverted video sequence.

Thus, the computational limitations inherent to mobile devices reduce effectiveness of the upconversion performed by mobile devices.

However, accurately estimating the visual quality of the temporally interpolated frames may be difficult since the original frames replaced by the temporally interpolated frames are not available.

Thus, the original frames are not available to the decoder that performs the upconversion.

However, problems exist with estimating the visual quality of the temporally interpolated frames based on the smoothness of the motion field.

A visual quality estimation technique based on the smoothness of the motion field may suggest an unsatisfactory visual quality of the temporally interpolated frame in each of these examples; however, the non-uniformities in these examples may be harmless in that they may not correspond to poor visual quality in the temporally interpolated frame.

The unreliable estimates of the visual quality of temporally interpolated frames may cause the system to suspend the interpolation even if the temporally interpolated frames actually have sufficient visual quality.

Suspension of the interpolation if the temporally interpolated frames have sufficient visual quality reduces effectiveness of the upconversion and degrades the visual quality of the upconverted video sequence.

Images