Video Scaling Method Based on Blocking and Frame-by-Frame Optimization

Abstract

The present invention is based on block and frame-by-frame optimized video zooming method, relates to general image data processing, divides the video sequence into N video sub-sequences according to the camera switching, and carries out horizontal and vertical directions on the video sub-sequences respectively. Scaling and optimization, the steps are: input a video, and divide the shot according to the switching frame of the shot; track the video subject target in each video sub-sequence, and obtain the video track tracking frame of the corresponding video sub-sequence; divide the whole frame of video evenly into x columns To the sub-block; merge the annotations, and finally obtain a frame of video image frame f j_pt and a shadow image frame g j_pt ; Frame-by-frame optimization based on the scaling factor; scale the video image frame f separately in proportion j_pt and the shadow image frame g j_pt Protected and non-protected regions in ; frame-by-frame optimization based on subject object position. The method of the invention overcomes the defects of video jitter and serious discrepancy between the target image and the original image in the prior art video scaling.

Description

technical field [0001] The technical solution of the present invention relates to general image data processing, specifically to a video scaling method based on segmentation and frame-by-frame optimization. Background technique [0002] With the diversification of video playback devices, videos often need to be changed in size to adapt to display devices of different specifications. Traditional video scaling methods, such as nearest neighbor interpolation methods, bilinear interpolation methods or bicubic interpolation methods, are used to interpolate and zoom. As a result, the display area of ​​the video display device cannot be used reasonably, and large areas of black borders are filled on both sides of the video. And when the user chooses to play the video in full screen, the information at the edge of the video will be lost. The content-aware video scaling technology can effectively solve the size matching problem between the video playback device and the video playbac...

AI Technical Summary

Problems solved by technology

This method replaces the clipping line in the 2D plane with the clipping surface in 3D. The problem is that it is not easy to avoid the video subject when the movement range of the moving subject between different video image frames is large. , leading to loss of information and video jitter

The document "Coarse-to-finetemporal optimization for video retargeting based on seam carving" uses the information of the previous frame to optimize the energy map of the current frame, which can constrain the generation position of the cropping line of the current frame, but it does not fundamentally solve the problem in the process of cropping line generation. Randomness problem, due to the existence of visual persistence, small differences in cropping lines between frames before and after a complex textured background may cause flickering in the background area and video jitter

The literature "Low complexity content-aware video retargeting" considers the intra-frame information and inter-frame motion information, and uses a unified scaling algorithm for scaling, but it still cannot solve the problem of preventing and eliminating the differences caused by inter-frame information. The video jitter defect

CN104517262A discloses an adaptive image scaling method based on visual saliency detection in the DCT domain. This method has the disadvantages that the longer inclined straight line features in the image cannot be well protected, and too many features will increase the time complexity of the algorithm. defect

CN104166992A discloses a content-aware binocular image scaling method based on grid deformation. This method does not consider the constraints of flipping and rotation. During the scaling process, it may cause the defect that the main target of the video image is seriously inconsistent with the original image

CN104822088A discloses a video image zooming method and device. The method is essentially a traditional zooming technology, and there is a defect that a specified video area can only be played by using an interactive function. After zooming, part of the video content will be lost like a traditional clipping algorithm.

CN102831576B discloses a video image zooming method and system, the method has the defects that the main part of the video cannot be well protected during the zooming process and the zooming process is prone to jittering and the like

CN101976558B discloses a video image zooming method and device, the method does not take into account the time correlation between frames after zooming, resulting in stroboscopic and jittering phenomenon prone to appear after video image zooming

Images