HEVC intra-frame coding fast mode decision algorithm based on ResNet

Abstract

The invention discloses an HEVC intra-frame coding fast mode decision algorithm based on ResNet. According to the method, the CU depth prediction process and the PU mode number screening process are optimized, so that the computational complexity of CTU division and optimal mode decision in the intra-frame coding process is reduced; The proposed HEVC intra-frame coding optimization algorithm basedon ResNet mainly comprises: an HEVC intra-frame coding optimization algorithm, The method comprises the steps that (1) adopting a residual error network in the technology, using a CU original depthprediction result and a corresponding video sequence data set for training to obtain a ResNet model, then acquiring a CU depth prediction value through the model, and replacing an HEVC original CU division process; and (2) on the basis of the prediction depth, proposing a PU texture direction analysis and mode adaptive decision-making combined algorithm, grouping 35 intra-frame prediction modes, and performing adaptive mode selection, thereby simplifying the existing mode decision-making process of HEVC.

Description

technical field [0001] The present invention relates to the field of video coding technology, especially the application of convolutional neural network (CNN) model and fast mode decision in the H.265 / HEVC video coding standard, and optimizes the depth prediction and mode decision in the intra-frame coding process Research, specifically a ResNet-based fast mode decision algorithm for HEVC intra coding. Background technique [0002] The high efficiency video coding (HEVC) standard is formulated by JCT-VC (Joint collaborative team on video coding). Compared with H.264 / AVC, HEVC can better realize the encoding operation of high-resolution video, and the encoding bit rate is reduced by nearly 50%, but it also greatly increases the encoding complexity. [0003] The quadtree recursive division process in HEVC intra-frame coding needs to traverse 4 depths and transform multiple CU sizes, and select the division method with a smaller rate-distortion cost (RDcost) as the intra-frame...

AI Technical Summary

Problems solved by technology

[0015] Aiming at the deficiencies in the prior art, the purpose of the present invention is to provide a fast HEVC intra-frame encoding algorithm based on ResNet, which can solve the following problems: (1) In the HEVC intra-frame encoding process, the Coding tree unit (Coding tree unit, CTU) Quadtree recursive division and RDO process complexity is high; (2) When the ordinary CNN model is used for depth or mode prediction, the phenomenon of gradient disappearance occurs when the data set is large or the number of convolutional layers is deep, which hinders Its performance is improved; (3) The computational complexity brought by the PU texture direction analysis process, and the error caused by the reprocessing process of the obtained candidate pattern set

Images