Video frame rate up-conversion method based on pixel semantic matching

Abstract

The invention relates to a video frame rate up-conversion method based on pixel semantic matching, and effectively solves the problem that the prediction quality of an interpolated frame is seriously influenced due to the fact that the BME block mismatching suppression capability is limited by adopting inter-block semantics in the prior art. The method comprises the following steps: taking a tth frame Ft in a video sequence as a to-be-interpolated frame, a (t-1) th frame Ft-1 in the video sequence as a forward reference frame of the to-be-interpolated frame Ft, and a (t + 1) th frame Ft + 1 in the video sequence as a backward reference frame of the to-be-interpolated frame Ft, extracting semantic features pixel by pixel for the forward reference frame Ft-1 and the backward reference frame Ft + 1; generating a front semantic layer Lt-1 of the forward reference frame Ft-1 and a rear semantic layer Lt + 1 of the backward reference frame Ft + 1; taking Lt-1 and Lt + 1 as new channels of Ft-1 and Ft + 1 respectively, merging and calculating as Xt-1 and Xt + 1, taking Ft as a center, implementing bidirectional block matching according to the new channel Xt-1 of Ft-1 and the new channel Xt + 1 of Ft + 1, and generating a motion vector field Vt of the frame Ft to be interpolated; and performing overlapped block motion compensation interpolation according to the motion vector fields Vt of adjacent Ft-1, Ft + 1 and Ft to obtain a frame Ft to be interpolated.

Description

technical field [0001] The invention relates to the technical field of video coding, in particular to a video frame rate up-conversion method based on pixel semantic matching. Background technique [0002] Frame Rate Up-Conversion (FRUC) is a video post-processing technology that increases the video frame rate by inserting intermediate frames into adjacent frames. Frame rate is an important indicator to determine the visual quality of video. However, backward imaging technology and industry standards cause low frame rate to be popular in various visual media. At present, 5G technology is developing rapidly, and its high transmission bandwidth has greatly stimulated the public's demand for high frame rate video, which poses new challenges to FRUC. In the popular low-speed imaging equipment, how to explore the potential spatio-temporal correlation contained in the video sequence, and use FRUC technology to output the high frame rate video that is close to the real one as much...

AI Technical Summary

Problems solved by technology

[0005] In view of the above situation, in order to overcome the defects of the prior art, the purpose of the present invention is to provide a video frame rate up-conversion method based on pixel semantic matching, which can effectively solve the problem of suppressing BME block mismatch caused by the use of inter-block semantics in the prior art. Capability is limited, which seriously affects the quality of interpolated frame prediction

Images