Video scaling method based on block segmentation and frame-by-frame optimization

Abstract

The invention provides a video scaling method based on block segmentation and frame-by-frame optimization and relates to common image data processing. The method is characterized by segmenting a videosequence into N video subsequences according to shot cut, and carrying out scaling and optimization on the video subsequences in the horizontal direction and vertical direction respectively. The method comprises the following steps: inputting a video, and carrying out shot segmentation according to shot switching frames; tracking a video main body target in each video subsequence to obtain a video trajectory tracking box of the corresponding video subsequence; uniformly dividing the whole frame of video into x column-wise sub-blocks; merging labels and finally obtaining a frame of video imageframe fj_pt and a frame of shadow image frame gj_pt; carrying out frame-by-frame optimization based on scaling proportion; scaling a protection area and a non-protection area in the video image framefj_pt and the shadow image frame gj_pt according to the proportion respectively; and carrying out frame-by-frame optimization based on the main body target position. The method overcomes the defectsof video jitter and serious inconsistency between the target image and an original image in video scaling in the prior art.

Description

technical field [0001] The technical solution of the present invention relates to general image data processing, specifically to a video scaling method based on segmentation and frame-by-frame optimization. Background technique [0002] With the diversification of video playback devices, videos often need to be changed in size to adapt to display devices of different specifications. Traditional video scaling methods, such as nearest neighbor interpolation methods, bilinear interpolation methods or bicubic interpolation methods, are used to interpolate and zoom. As a result, the display area of ​​the video display device cannot be used reasonably, and large areas of black borders are filled on both sides of the video. And when the user chooses to play the video in full screen, the information at the edge of the video will be lost. The content-aware video scaling technology can effectively solve the size matching problem between the video playback device and the video playbac...

AI Technical Summary

Problems solved by technology

This method replaces the clipping line in the 2D plane with the clipping surface in 3D. The problem is that it is not easy to avoid the video subject when the movement range of the moving subject between different video image frames is large. , leading to loss of information and video jitter

The document "Coarse-to-finetemporal optimization for video retargeting based on seam carving" uses the information of the previous frame to optimize the energy map of the current frame, which can constrain the generation position of the cropping line of the current frame, but it does not fundamentally solve the problem in the process of cropping line generation. Randomness problem, due to the existence of visual persistence, small differences in cropping lines between frames before and after a complex textured background may cause flickering in the background area and video jitter

The literature "Low complexity content-aware video retargeting" considers the intra-frame information and inter-frame motion information, and uses a unified scaling algorithm for scaling, but it still cannot solve the problem of preventing and eliminating the differences caused by inter-frame information. The video jitter defect

CN104517262A discloses an adaptive image scaling method based on visual saliency detection in the DCT domain. This method has the disadvantages that the longer inclined straight line features in the image cannot be well protected, and too many features will increase the time complexity of the algorithm. defect

CN104166992A discloses a content-aware binocular image scaling method based on grid deformation. This method does not consider the constraints of flipping and rotation. During the scaling process, it may cause the defect that the main target of the video image is seriously inconsistent with the original image

CN104822088A discloses a video image zooming method and device. The method is essentially a traditional zooming technology, and there is a defect that a specified video area can only be played by using an interactive function. After zooming, part of the video content will be lost like a traditional clipping algorithm.

CN102831576B discloses a video image zooming method and system, the method has the defects that the main part of the video cannot be well protected during the zooming process and the zooming process is prone to jittering and the like

CN101976558B discloses a video image zooming method and device, the method does not take into account the time correlation between frames after zooming, resulting in stroboscopic and jittering phenomenon prone to appear after video image zooming

Images