VVC intra-frame fast coding optimization method based on deep learning
Through the two-stage VVC intra-fast coding prediction framework, the encoding depth and division mode are predicted by D-CNN and P-CNN models, the problem of high encoding complexity in the existing technology is solved, and efficient VVC intra-factor encoding is achieved. Reduces coding complexity and inference time overhead.
Patent Information
- Application Number
- CN202411042801.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing VVC intra-fast coding method based on deep learning fails to effectively consider the impact of encoding depth on the intra-coding complexity of VVC, resulting in high encoding complexity and low efficiency.
Using a two-stage VVC intra-fast coding prediction framework, the maximum coding depth of the encoding unit is predicted through the D-CNN model and the rate distortion optimization process is terminated in advance. Combined with the optimal division mode of the P-CNN model to predict the encoding unit, skip unnecessary to reduce coding complexity.
It effectively reduces the intra-coding complexity of VVC frames, improves coding efficiency, and predicts the division mode of multiple encoding units through one-time inference, reduces the model inference time overhead and avoids the degradation of coding performance caused by incorrect prediction.
Smart Images

Figure CN119052470A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of Internet big data and fast video coding, and in particular to a VVC intra-frame fast coding optimization method based on deep learning. Background Art
[0002] With the rapid development of video acquisition, communication and display technologies, users are increasingly relying on ultra-high-definition and 360-degree video applications. These applications cover a variety of fields such as education, entertainment, medical services and cultural heritage protection, and the transmission and storage of the large amount of data generated has become a key issue.
[0003] The compression performance of the previous generation video coding standard High Efficiency Video Coding (HEVC) has gradually failed to meet the requirements of video application development. To address this problem, the Joint Video Experts Team (JVET) has developed a video compression standard called Versatile Video Coding (VVC). In the VVC coding framework, new technologies such as the Quad Tree withnested Multi-type Tree (QTMT) partition structure, a larger Coding Tree Unit (CTU), adaptive filtering, and affine motion compensation prediction are introduced. These technologies make VVC coding significantly more efficient than HEVC, and can save 50% of the bit rate under the same peak signal-to-noise ratio (PSNR) conditions. However, these new technologies also lead to a significant increase in coding complexity, especially in AI full-frame intra mode, the average complexity of VVC coding is 18 times that of HEVC, making the VVC coding standard difficult to promote in practical applications. Therefore, it is very necessary to effectively reduce the complexity of VVC coding while maintaining a high coding efficiency.
[0004] VVC coding adopts a more complex and diverse partitioning structure, introducing the QTMT partitioning structure of horizontal and vertical binary tree partitioning (Binary-Tree, BT) and horizontal and vertical ternary tree partitioning (Ternary-Tree, TT), such as Figure 7The QTMT shown in the figure includes five partitioning modes: quadtree (QT), vertical binary tree (BTV), horizontal binary tree (BTH), vertical ternary tree (TTV) and horizontal ternary tree (TTH), which makes VVC have a more flexible partitioning method and can better adapt to the local characteristics of different image regions, thereby improving the coding performance. However, the application of this partitioning structure also leads to a sharp increase in the complexity of VVC intra-frame coding, which is mainly attributed to the following two aspects: 1) QTMT technology needs to consider more partitioning combinations to find the best partitioning mode to improve coding performance; 2) VVC allows finer-grained partitioning of images to better capture complex texture information, but deeper partitioning increases the computational workload and memory overhead in the encoding process, thereby further increasing the coding complexity.
[0005] At present, scholars have proposed a variety of VVC intra-frame fast coding methods based on statistical analysis, traditional machine learning and convolutional neural network (CNN). The statistical analysis-based method mainly uses intermediate coding features such as variance, gradient and texture to establish a prediction model. This type of method relies on manual feature selection and threshold setting, which limits the accuracy of intra-frame prediction and leads to a significant decrease in coding efficiency. The traditional machine learning-based method predicts the division pattern of coding units (CU) by introducing technologies such as Bayesian decision rules, decision trees, random forests and support vector machines to improve the model prediction accuracy. With the development of deep learning, CNN-based video coding methods have attracted great attention, such as applying classic CNN structures (ResNet, DenseNet and U-Net) to predict the division pattern of VVC coding units.
[0006] However, VVC fast encoding methods based on deep learning usually convert the prediction of division mode into a multi-classification problem. There are two main problems with this type of method: 1) It fails to take into account the impact of coding depth on the complexity of VVC intra-frame coding. Under simple textures, it is still necessary to search for division modes for deep coding units, resulting in high complexity and low efficiency of VVC coding. 2) It is necessary to predict the five division modes of each coding unit, which consumes a lot of computing resources and also leads to low efficiency of VVC coding. Therefore, how to effectively reduce the coding complexity of VVC at a higher coding efficiency has become a problem that technicians in this field need to solve urgently. Summary of the invention
[0007] In view of the shortcomings of the above-mentioned prior art, the technical problem to be solved by the present invention is: how to provide a VVC intra-frame fast coding optimization method based on deep learning, predict the maximum coding depth of the coding unit through the coding depth prediction (D-CNN) model, effectively consider the influence of the coding depth on the complexity of VVC intra-frame coding, and terminate the rate-distortion optimization process in advance to reduce the complexity of VVC intra-frame coding; predict the optimal division mode of the coding unit through the division mode prediction (P-CNN) model, skip unnecessary division modes, and further reduce the coding complexity of VVC.
[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:
[0009] A VVC intra-frame fast encoding optimization method based on deep learning, comprising:
[0010] S1: Get the video frame to be encoded;
[0011] S2: Divide the video frame to be encoded into a number of N1×N1 coding units; input the N1×N1 coding units into the trained D-CNN model to obtain the corresponding predicted coding depth;
[0012] S3: Divide the N1×N1 coding unit into several N2×N2 coding units; input several N2×N2 coding units into the trained P-CNN model together to obtain the division prediction probability vector of each N2×N2 coding unit;
[0013] S4: Determine a partitioning mode candidate list of each N2×N2 coding unit and its sub-coding units by combining the partitioning prediction probability vector of the N2×N2 coding unit with the predicted coding depth of the corresponding N1×N1 coding unit;
[0014] S5: Execute steps S3 to S4 for each N1×N1 coding unit in the to-be-encoded video frame, and obtain a candidate list of partitioning modes for each N2×N2 coding unit and its sub-coding units in each N1×N1 coding unit; for each coding unit, select a partitioning mode with the smallest rate-distortion cost from its candidate list of partitioning modes as its encoding result.
[0015] Preferably, in step S2, the D-CNN model includes an input layer, a convolution layer, a batch normalization layer, a ReLU function layer, a first fusion module, a second fusion module, a first maximum pooling layer, a third fusion module, a fourth fusion module, a second maximum pooling layer, a Flatten layer, a fully connected layer and a Softmax activation function layer connected end to end in sequence;
[0016] The first to fourth fusion modules all include a convolution layer, a batch normalization layer, a ReLU function layer, a convolution layer, a batch normalization layer, a Sum layer and a ReLU function layer connected end to end in sequence; the input of each fusion module is processed by the convolution layer and the batch normalization layer and then residually connected with the input of the Sum layer.
[0017] Preferably, in step S2, the loss function when training the D-CNN model is as follows:
[0018]
[0019] Where: N d is the number of training samples for each N1×N1 coding unit in each training batch; For true encoding depth; Depth of the predicted encoding output of D-CNN.
[0020] Preferably, in step S2, the output of the D-CNN model includes six coding depth probabilities and their index values, and the index value corresponding to the optimal coding depth probability is selected as the predicted coding depth through the following steps:
[0021] S201: Select the maximum coding depth probability P from all coding depth probabilities d [u1];
[0022] S202: Determine the maximum coding depth probability P d [i1] Whether it exceeds the preset value: If so, the maximum coding depth probability P d [i1] The index value i1 corresponding to is used as the predicted coding depth; otherwise, execute step S203;
[0023] S203: The maximum coding depth probability P d [i1] is set to 0 and deleted, and the largest coding depth probability P is selected from all remaining coding depth probabilities d [i2];
[0024] S204: Select the larger index value max[u1,i2] of the index values i1 and u2 as the predicted coding depth.
[0025] Preferably, in step S3, the P-CNN model includes an input layer, a convolution layer, a batch normalization layer, a ReLU function layer, a first fusion module, a second fusion module, a first maximum pooling layer, a third fusion module, a fourth fusion module, a second maximum pooling layer, a fifth fusion module, a sixth fusion module, a third maximum pooling layer, a Flatten layer, a fully connected layer and a Sigmoid activation function layer connected in sequence;
[0026] The first to sixth fusion modules all include a convolution layer, a batch normalization layer, a ReLU function layer, a convolution layer, a batch normalization layer, a Sum layer, and a ReLU function layer connected end to end in sequence; the input of each fusion module is processed by the convolution layer and the batch normalization layer and then residually connected with the input of the Sum layer.
[0027] Preferably, in step S3, an N2×N2 coding unit is divided into a plurality of 4×4 coding units;
[0028] The output of the P-CNN model for each N2×N2 coding unit includes: the partition prediction probability vector of all boundaries of all 4×4 coding units in the N2×N2 coding unit except the boundary of the N2×N2 coding unit itself; if the partition prediction probability of a certain boundary is close to 1, it means that the boundary exists, that is, it will be further divided; if the partition prediction probability of a certain boundary is close to 0, it means that the boundary does not exist, that is, it will not be further divided;
[0029] The dimension N of the partition prediction probability vector is calculated by the following formula p :
[0030]
[0031] Where: N w is the width of the N2×N2 coding unit; N h is the height of the N2×N2 coding unit.
[0032] Preferably, in step S3, the loss function when training the P-CNN model is as follows:
[0033]
[0034] Where: N is the number of training samples of N2×N2 coding units in each training batch; y pi is the true boundary probability vector of the sample; Represents the boundary partition prediction probability vector output by P-CNN.
[0035] Preferably, in step S4, the candidate list of partitioning modes of the N2×N2 coding unit and its sub-coding units is determined by the following steps:
[0036] S401: Generate a partition mode candidate list for the current coding unit;
[0037] S402: traverse each partition mode in the partition mode candidate list of the current coding unit, and select the partition mode with the smallest rate-distortion cost as the optimal partition mode of the current coding unit;
[0038] S403: Divide the current coding unit by an optimal division mode to obtain a plurality of sub-coding units of the next size;
[0039] S404: Return to step S401 and use the sub-coding unit as the current coding unit;
[0040] S405: Repeat steps S401 to S404 until the coding depth of the current coding unit is greater than or equal to the predicted coding depth or the current coding unit is the minimum coding unit;
[0041] S406: Obtain a partitioning mode candidate list of the N2×N2 coding unit and its sub-coding units.
[0042] Preferably, in step S401, the candidate list of division modes of the coding unit is determined by the following steps:
[0043] S4011: Create a partition mode candidate list for the current coding unit;
[0044] S4012: Determine whether the coding depth of the current coding unit is greater than or equal to the predicted coding depth: if so, terminate in advance and execute step S4020; otherwise, execute step S4013;
[0045] S4013: Calculate the probabilities of five partition modes of the current coding unit based on the partition prediction probability vector of the N2×N2 coding unit; the five partition modes include QT mode, BTV mode, BTH mode, TTV mode and TTH mode;
[0046] S4014: adding the partitioning mode with the highest probability to the partitioning mode candidate list;
[0047] S4015: Determine whether the division mode with the highest probability is the QT mode: if so, execute step S4016; otherwise, execute S4017;
[0048] S4016: Determine whether abs(P BTV -P BTH )>β: If yes, then add the BTV mode and the BTH mode with a higher probability into the candidate list of division modes, and execute step S4020; otherwise, directly execute step S4020;
[0049] S4017: Determine whether the maximum probability type is BTH mode or TTH mode: if yes, execute step S4018; otherwise, execute step S4019;
[0050] S4018: Determine whether abs(P BTH -P TTH)<β: If yes, then add the TTH mode or BTH model to the candidate list of partition modes and execute step S4020; otherwise, directly execute step S4020;
[0051] S4019: Determine whether abs(P BTV -P TTV )<β: If yes, then add the TTV mode or the BTV mode to the candidate list of division modes and execute step S4020; otherwise, directly execute step S4020;
[0052] S4020: Output a partition mode candidate list of the current coding unit and perform the next coding depth.
[0053] Preferably, in step S4013, the probabilities of the five division modes are calculated by the following formula:
[0054] 1) QT mode
[0055]
[0056] 2) BTV Model
[0057]
[0058] 3) BTH mode
[0059]
[0060] 4) TTV Model
[0061]
[0062] 5) TTH mode
[0063]
[0064] Where: P QT , P BTV , P BTH , P TTV , P TTH They represent the probabilities of QT mode, BTV mode, BTH mode, TTV mode and TTH mode respectively; w and h represent the width and height of the current coding unit respectively; x c and c Indicates the coordinates of the current coding unit in the video frame to be encoded; x p and p represents the coordinates of the 32×32 coding unit in the video frame to be encoded; p represents the partition prediction probability vector.
[0065] Compared with the prior art, the VVC intra-frame fast coding optimization method based on deep learning in the present invention has the following beneficial effects:
[0066] The present invention makes full use of the coding depth information and division mode information of the coding unit in the coding process, and designs a two-stage VVC intra-frame fast coding prediction framework. First, in the first stage, D-CNN and depth decision post-processing method are used to predict the maximum coding depth of N1×N1 (such as 64×64) coding units, taking into account the influence of coding depth on the complexity of VVC intra-frame coding, which helps to terminate the rate-distortion optimization process in advance, and can avoid searching for division modes of deep-level coding units under simple textures, thereby reducing the complexity of VVC intra-frame coding. Secondly, P-CNN is used in the second stage to generate the division prediction probability vector of each N2×N2 (such as 32×32) coding unit, and then the division mode candidate list of each N2×N2 coding unit and its sub-coding unit is determined in combination with the division candidate list decision method, which is conducive to determining the optimal division mode of the coding unit and skipping unnecessary division modes, thereby further reducing the coding complexity of VVC. Finally, in order to avoid a large amount of time overhead other than encoding caused by neural network reasoning, the present invention uses P-CNN to predict the division patterns of multiple N2×N2 coding units and their sub-coding units at one time, which greatly reduces the time overhead of model reasoning; at the same time, in order to deal with the degradation of coding performance caused by incorrect prediction of P-CNN, the present invention also designs a P-CNN post-processing method to avoid the degradation of coding performance caused by incorrect prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:
[0068] Figure 1 This is the overall framework diagram of the VVC intra-frame fast coding optimization method based on deep learning;
[0069] Figure 2 Optimizing methods for model inference;
[0070] Figure 3 This is the network structure diagram of D-CNN;
[0071] Figure 4 is a flowchart of coding depth post-processing;
[0072] Figure 5 This is the network structure diagram of P-CNN;
[0073] Figure 6 A flowchart for generating a candidate list of partitioning patterns;
[0074] Figure 7 QTMT partition structure for VVC;
[0075] Figure 8This is the correspondence between the actual division of the 32×32 coding unit and the division prediction probability vector. DETAILED DESCRIPTION
[0076] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.
[0077] The following is a further detailed description through specific implementation methods:
[0078] Example:
[0079] This embodiment discloses a VVC intra-frame fast encoding optimization method based on deep learning.
[0080] like Figure 1 As shown, the VVC intra-frame fast encoding optimization method based on deep learning includes:
[0081] S1: Get the video frame to be encoded;
[0082] In this embodiment, the luminance coding block and quantization parameter (Quantization Parameter, QP) of the video frame to be encoded are obtained, wherein the video to be encoded is stored in YUV format, including luminance component (Y) and chrominance component (UV). The luminance coding block is obtained by cropping the encoding frame.
[0083] S2: Divide the video frame to be encoded into a number of N1×N1 coding units; input the N1×N1 coding units (and their quantization parameters) into the trained D-CNN (Depth Convolutional Neural Network, coding depth prediction) model to obtain the corresponding predicted coding depth;
[0084] In this embodiment, the video frame to be encoded is divided into a plurality of 64×64 encoding units.
[0085] S3: Divide the N1×N1 coding unit into several N2×N2 coding units; input the several N2×N2 coding units into the trained P-CNN (Partition Convolutional Neural Network, partition pattern prediction) model to obtain the partition prediction probability vector of each N2×N2 coding unit;
[0086] In this embodiment, for the reasoning acceleration of the P-CNN model, the strategy adopted by the present invention is to use four 32×32 coding units to be simultaneously input into the P-CNN. The specific steps are as follows: the video frame to be encoded is divided into multiple 64×64 coding units, each 64×64 coding unit contains 4 32×32 coding units. In the central area of the image, the 64×64 coding unit is subdivided into four 32×32 coding units as input. For the lower and right boundaries of the video frame that cannot accommodate the 64×64 coding unit, this embodiment selects a method of horizontally or vertically aligning four 32×32 coding units to meet the input conditions of the four coding units. If four 32×32 coding units are missing in the lower right corner of the video frame, 0 padding is applied to ensure input consistency. This method significantly reduces the time overhead during P-CNN reasoning. Figure 2 A schematic diagram of this optimization technique is described, which significantly improves inference efficiency.
[0087] S4: Determine a partitioning mode candidate list of each N2×N2 coding unit and its sub-coding units by combining the partitioning prediction probability vector of the N2×N2 coding unit with the predicted coding depth of the corresponding N1×N1 coding unit;
[0088] S5: For each N1×N1 coding unit in the video frame to be encoded, steps S3 to S4 are executed to obtain a list of candidate partition modes for each N2×N2 coding unit and its sub-coding units in each N1×N1 coding unit; for each coding unit, a partition mode with the lowest rate-distortion (RD) cost is selected from its partition mode candidate list as its encoding result. For each coding unit, the determined partition mode candidate list and intra-frame prediction mode (such as angle prediction) are traversed to perform intra-frame prediction. The process calculates the rate-distortion cost of each candidate mode, and selects the partition mode with the lowest cost as the final encoding result of the coding unit.
[0089] In this embodiment, during the VVC intra-frame coding process, the VVC encoder uses a mode candidate list to store the test mode of the current coding unit, including the five division modes in QTMT and the intra-frame prediction mode. If the rate-distortion optimization process is terminated early in the prediction results of the first stage, only the intra-frame mode is retained in the mode candidate list, and the division mode is no longer tested further. If the optimal division mode and the division type strongly related to it are predicted in the second stage, other modes are removed from the candidate list. Finally, through rate-distortion optimization, the final division mode is obtained from the predicted candidate list. After completing the above process, the encoder will perform entropy coding, frequency domain conversion, and quantization to finally obtain the encoded bitstream.
[0090] The present invention makes full use of the coding depth information and division mode information of the coding unit in the coding process, and designs a two-stage VVC intra-frame fast coding prediction framework. First, in the first stage, D-CNN and depth decision post-processing method are used to predict the maximum coding depth of N1×N1 (such as 64×64) coding units, taking into account the influence of coding depth on the complexity of VVC intra-frame coding, which helps to terminate the rate-distortion optimization process in advance, and can avoid searching for division modes of deep-level coding units under simple textures, thereby reducing the complexity of VVC intra-frame coding. Secondly, P-CNN is used in the second stage to generate the division prediction probability vector of each N2×N2 (such as 32×32) coding unit, and then the division mode candidate list of each N2×N2 coding unit and its sub-coding unit is determined in combination with the division candidate list decision method, which is conducive to determining the optimal division mode of the coding unit and skipping unnecessary division modes, thereby further reducing the coding complexity of VVC. Finally, in order to avoid a large amount of time overhead other than encoding caused by neural network reasoning, the present invention uses P-CNN to predict the division patterns of multiple N2×N2 coding units and their sub-coding units at one time, which greatly reduces the time overhead of model reasoning; at the same time, in order to deal with the degradation of coding performance caused by incorrect prediction of P-CNN, the present invention also designs a P-CNN post-processing method to avoid the degradation of coding performance caused by incorrect prediction.
[0091] In order to better introduce the technical solution of the present invention, this embodiment is described through the following parts.
[0092] 1. D-CNN Model
[0093] The D-CNN structure introduces residual connections, which effectively solves the problems of gradient disappearance and feature transfer, while reducing the number of parameters. In order to better deploy the model to the VVC video coding reference software, this embodiment optimizes the ResNet structure as follows:
[0094] 1) In order to avoid a lot of time consumption during reasoning, the residual block is reduced to 4 layers.
[0095] 2) The feature extraction layers in the network all use a 3×3 convolution kernel size. In the downsampling convolution layer, a 1×1 convolution kernel is used to ensure the consistency of the feature map size in the cross-layer connection and to be fused with the backbone network features.
[0096] 3) The network uses a 2×2 convolution kernel and a maximum pooling layer with a stride of 2. This structure effectively reduces the size of the feature map while retaining important features.
[0097] 4) In order to reduce the dimension and further simplify the network structure, a global average pooling (GAP) layer is used.
[0098] 5) A fully connected layer is introduced at the end of the network to ensure that the model outputs six features.
[0099] 6) In order to convert each value into a probability value between 0 and 1 and ensure that the sum of all output probabilities is 1, this embodiment uses a Softmax activation function to obtain the probability of each coding depth.
[0100] like Figure 3 As shown, the D-CNN model in this embodiment includes an input layer, a convolution layer, a batch normalization layer, a ReLU function layer, a first fusion module, a second fusion module, a first maximum pooling layer, a third fusion module, a fourth fusion module, a second maximum pooling layer, a Flatten layer, a fully connected layer and a Softmax activation function layer, which are connected end to end in sequence; wherein the function of the Flatten layer is to flatten the features to facilitate subsequent full connection.
[0101] The first to fourth fusion modules all include a convolution layer, a batch normalization layer, a ReLU function layer, a convolution layer, a batch normalization layer, a Sum layer, and a ReLU function layer connected end to end in sequence; the input of each fusion module is processed by the convolution layer and the batch normalization layer and then residually connected with the input of the Sum layer. The function of the Sum layer is to add the features of the two branches.
[0102] Specifically, in the D-CNN method, the cross entropy loss function is selected to supervise the D-CNN model training, and the loss function L d The calculation formula is as follows:
[0103]
[0104] Where: N d is the number of training samples for each N1×N1 coding unit in each training batch; For true encoding depth; Depth of the predicted encoding output of D-CNN.
[0105] During training, the DIV2K, HDR Google, and RAISE super-resolution datasets are normally encoded by the VVC reference software VTM to save the brightness coding blocks and quantization parameters to build a dataset, and the created dataset is used to train the D-CNN model. In this embodiment, the Adam optimizer is used to optimize the weight parameters of the D-CNN. The initial learning rate is set to 1×10 -4 , 200 training epochs. In the first 5 epochs, the learning rate is increased from 2×10 -5 Increase to 1×10 -4 This gradual increase in learning rate will accelerate convergence at the beginning of the training process, thereby improving the speed and efficiency of model optimization. In subsequent training, the exponential decay of the learning rate reduces the risk of leading to local optimality. In order to maintain effective weight updates in the later training stages, the minimum learning rate is limited to 1×10 -6 .
[0106] 2. Encoding Depth Post-Processing
[0107] At the beginning of model design, this embodiment makes a compromise between prediction accuracy and model complexity, which results in that D-CNN cannot completely and accurately predict the optimal coding depth. In order to improve the prediction error tolerance and coding efficiency, this embodiment designs a coding depth prediction post-processing method.
[0108] In this embodiment, the output of the D-CNN model includes six coding depth probabilities and their index values, where the coding unit size corresponding to the first coding depth is 64×64; the coding unit size corresponding to the second coding depth is 32×32; the coding unit size corresponding to the third coding depth is 32×16, 16×32, 32×8, 8×32 and 16×16; the coding unit size corresponding to the fourth coding depth is 32×8, 8×32, 32×4, 4×32, 16×8, 8×16, 16×4, 4×16 and 8×8; the coding unit size corresponding to the fifth coding depth is 32×4, 4×32, 16×8, 8×16, 16×4, 4×16, 8×4, 4×8 and 4×4; the coding unit size corresponding to the sixth coding depth is 8×4, 4×8 and 4×4.
[0109] like Figure 4 As shown, the index value corresponding to the optimal coding depth probability is selected as the predicted coding depth through the following steps:
[0110] S201: Select the maximum coding depth probability P from all coding depth probabilities d [i1];
[0111] S202: Determine the maximum coding depth probability P d[i1] Whether it exceeds the preset value (0.9): If so, the maximum coding depth probability P d [i1] The index value i1 corresponding to is used as the predicted coding depth; otherwise, execute step S203;
[0112] S203: The maximum coding depth probability P d [i1] is set to 0 and deleted, and the largest coding depth probability P is selected from all remaining coding depth probabilities d [i2];
[0113] S204: Select the larger index value max[i1, i2] of the index values i1 and i2 as the predicted coding depth.
[0114] Through the above steps, the post-processing decision method can effectively improve the accuracy of coding depth prediction, thereby improving the overall coding performance.
[0115] 3. Partitioning Mode Prediction Method
[0116] During the VVC encoding process, each coding unit will iteratively perform the division process. At each coding depth, the QTMT structure is used to further divide and search until the minimum RD Cost is reached. Table 1 shows the possible division modes that different-sized coding units may traverse when searching for the next coding depth. This process requires calculating the RD Cost of each coding unit of different sizes and selecting the optimal division mode, which takes up a lot of encoding time.
[0117] Table 1 Possible division modes for the next coding depth of different coding units
[0118]
[0119] In order to reduce the coding complexity, the P-CNN method is designed in the second stage of this embodiment. The motivation of this method comes from the following two points: 1) In the VVC standard, the smallest coding unit is a 4×4 block, and the division boundary of each 4×4 block can be predicted to predict the division mode of the current coding unit. 2) The 64×64 coding unit only supports QT division. In the coding depth prediction method, it can be predicted whether it is a division mode (QT mode) or a non-division mode; therefore, this embodiment only predicts the division method of 32×32 blocks and sub-blocks. This embodiment can obtain the division method of 32×32 blocks and sub-blocks with only one simple reasoning.
[0120] The core of this method is to divide a 32×32 coding unit into several 4×4 coding units, and then convert the boundary features of each 4×4 coding unit into an n pdimensional feature vector. This vector contains the boundary information of the partition and can be used to independently predict the boundary partition probability of each 4×4 block.
[0121] The output of the P-CNN model for each 32×32 coding unit includes: the partition prediction probability vectors of all 112 boundaries of all 4×4 coding units in the 32×32 coding unit excluding the boundary of the 32×32 coding unit itself, that is, the partition prediction probability vector is a 112-dimensional vector.
[0122] Among them, if the predicted probability of a boundary is close to 1, it means that the boundary exists and will be further divided; if the predicted probability of a boundary is close to 0, it means that the boundary does not exist and will not be further divided.
[0123] The dimension n of the partition prediction probability vector is calculated by the following formula p :
[0124]
[0125] Where: n w is the width of N2×N2 coding units (32×32 coding units in this embodiment); N h is the height of the N2×N2 coding unit. Figure 8 As shown in the figure, the actual division of the 32×32 coding unit and the boundary relationship between the prediction vector and the 64 4×4 coding units are shown. Among them, P1 and P3 are the predicted values of the 16×8 pixel size coding unit. The predicted probability values of these boundary values are close to 1, indicating that further division will be carried out. On the contrary, P 111 and P 112 The left boundaries of the last two 4×4 coding units belonging to the lower right corner, the predicted probability values of these boundaries are close to 0, and they will not be further divided.
[0126] 4. P-CNN Model
[0127] The partition mode prediction and partition depth prediction structures are similar and are both based on the residual structure. The difference is that the partition mode prediction requires feature extraction of 112 partition boundaries, while the partition depth prediction only needs to extract six features. In order to learn deeper feature representations, the P-CNN structure adds two residual layers based on the D-CNN. The main difference between P-CNN and D-CNN lies in the last layer. In the D-CNN prediction results, the Softmax activation function is used to obtain the probabilities of 6 coding depths. In P-CNN, the boundaries of each 4×4 coding unit need to be classified into two categories, so the Sigmoid activation function is used to limit the prediction results of each boundary to between [0,1]. If the value is close to 1, it means that the boundary needs to be divided; if it is close to 0, it means that no division is required.
[0128] like Figure 5 As shown, the P-CNN model in this embodiment includes an input layer, a convolution layer, a batch normalization layer, a ReLU function layer, a first fusion module, a second fusion module, a first maximum pooling layer, a third fusion module, a fourth fusion module, a second maximum pooling layer, a fifth fusion module, a sixth fusion module, a third maximum pooling layer, a Flatten layer, a fully connected layer, and a Sigmoid activation function layer, which are sequentially connected end to end;
[0129] The first to sixth fusion modules all include a convolution layer, a batch normalization layer, a ReLU function layer, a convolution layer, a batch normalization layer, a Sum layer, and a ReLU function layer connected end to end in sequence; the input of each fusion module is processed by the convolution layer and the batch normalization layer and then residually connected with the input of the Sum layer.
[0130] Specifically, in the P-CNN structure, the binary cross entropy loss is selected to supervise the training process, and the loss function calculation formula is as follows:
[0131]
[0132] Where: N is the number of training samples of N2×N2 coding units in each training batch; y pi is the true boundary probability vector of the sample; Represents the boundary partition prediction probability vector output by P-CNN.
[0133] The training method of P-CNN is similar to that of D-CNN, using the Adam optimizer, the same learning rate, and the same number of epochs.
[0134] 5. Partitioning Mode Candidate List
[0135] After completing the reasoning of P-CNN, the boundary prediction probability of each 4×4 coding unit is obtained. This probability value cannot directly represent the division mode of the coding unit. Therefore, this embodiment constructs a decision algorithm to determine the final candidate list of division modes. At the same time, the P-CNN model will have a certain error rate in prediction. In order to avoid the degradation of coding performance caused by wrong prediction, this embodiment strategically expands the list of candidate division modes based on the optimal mode predicted by P-CNN to enhance the robustness of the decision.
[0136] In this embodiment, the candidate list of partition modes of the N2×N2 coding unit and its sub-coding units is determined by the following steps:
[0137] S401: Generate a partition mode candidate list for a coding unit (32×32 coding unit) of a current coding depth;
[0138] S402: traverse each partition mode in the partition mode candidate list of the current coding unit, and select the partition mode with the smallest rate-distortion cost as the optimal partition mode of the current coding unit;
[0139] In this embodiment, the candidate list of partition modes and intra-frame prediction modes (such as angle prediction) of the coding unit are traversed to perform intra-frame prediction. The process calculates the rate-distortion cost of each candidate mode and selects the partition mode with the lowest cost as the optimal partition mode of the coding unit.
[0140] S403: Divide the current coding unit by an optimal division mode to obtain a plurality of sub-coding units of the next size;
[0141] S404: Return to step S401, and use the sub-coding unit as the current coding unit (i.e., execute the next coding depth);
[0142] S405: Repeat steps S401 to S404 until the coding depth of the current coding unit is greater than or equal to the predicted coding depth or the current coding unit is a minimum coding unit (ie, a 4×4 coding unit);
[0143] S406: Obtain a partitioning mode candidate list of the N2×N2 coding unit and its sub-coding units.
[0144] Specific decision-making and optimization algorithm processes such as Figure 6 As shown. The first step is to determine the predicted optimal partitioning mode; in this step, an empty partitioning mode candidate list is first established, and then the coding unit is evaluated whether the rate-distortion optimization process should be terminated in advance based on the coding depth prediction results of the D-CNN stage. If the rate-distortion optimization process is not terminated in advance, the probabilities of the five partitioning modes of the current coding unit are obtained by the average value, and the maximum probability mode is selected to be added to the candidate list. The second step is to selectively expand the candidate list; first, based on the prediction results of the previous step, determine whether the optimal mode is QT or MTT mode. If it is QT division, considering that the QT mode is composed of BTH and BTV modes (such as Figure 7 b), there is a significant correlation between the three modes; therefore, if QT is predicted as the optimal mode, the probability difference between BTH and BTV will be compared next: if the difference is small, the current coding unit is determined to be QT mode; if the difference is large, the mode with a higher probability (BTH or BTV) is selected to be added to the candidate list. If the prediction result is not QT mode, due to the strong correlation between BT and TT modes in the vertical and horizontal directions, the corresponding vertical or horizontal division mode will also be considered to be added to the candidate list. In the entire decision-making process, the balance between coding performance and coding complexity can be controlled by adjusting the difference threshold β.
[0145] Combination Figure 6 As shown, the candidate list of division modes of the coding unit is determined by the following steps:
[0146] S4011: Create a partition mode candidate list for the current coding unit;
[0147] S4012: Determine whether the coding depth of the current coding unit is greater than or equal to the predicted coding depth: if so, terminate in advance and execute step S4020; otherwise, execute step S4013;
[0148] S4013: Calculate the probabilities of five partition modes of the current coding unit based on the partition prediction probability vector of the N2×N2 coding unit; the five partition modes include QT mode, BTV mode, BTH mode, TTV mode and TTH mode;
[0149] In this embodiment, VVC adopts a more complex and diversified partitioning structure, introducing a QTMT partitioning structure of horizontal and vertical binary tree partitioning (Binary-Tree, BT) and horizontal and vertical ternary tree partitioning (Ternary-Tree, TT), such as Figure 7 The QTMT shown includes five partitioning modes: quadtree (QT), vertical binary tree (BTV), horizontal binary tree (BTH), vertical ternary tree (TTV) and horizontal ternary tree (TTH), which makes VVC have a more flexible partitioning method and can better adapt to the local characteristics of different image areas, thereby improving the coding performance.
[0150] S4014: adding the partitioning mode with the highest probability to the partitioning mode candidate list;
[0151] S4015: Determine whether the division mode with the highest probability is the QT mode: if so, execute step S4016; otherwise, execute S4017;
[0152] S4016: Determine whether abs(P BTV -P BTH )>β: If yes, then add the BTV mode and the BTH mode with a higher probability into the candidate list of division modes, and execute step S4020; otherwise, directly execute step S4020;
[0153] S4017: Determine whether the maximum probability type is BTH mode or TTH mode: if yes, execute step S4018; otherwise, execute step S4019;
[0154] S4018: Determine whether abs(P BTH -P TTH)<β: If yes, then add the TTH mode or BTH model (not the optimal mode) to the candidate list of partition modes and execute step S4020; otherwise, directly execute step S4020;
[0155] S4019: Determine whether abs(P BTV -P TTV )<β: If yes, then add the TTV mode or the BTV mode (not the optimal mode) to the candidate list of partition modes, and execute step S4020; otherwise, directly execute step S4020;
[0156] S4020: Output a partition mode candidate list of the current coding unit and perform the next coding depth.
[0157] Specifically, the probabilities of the five division modes are calculated by the following formula:
[0158] 1) QT mode
[0159]
[0160] 2) BTV Model
[0161]
[0162] 3) BTH mode
[0163]
[0164] 4) TTV Model
[0165]
[0166] 5) TTH mode
[0167]
[0168] Where: P QT , P BTV , P BTH , P TTV , P TTH They represent the probabilities of QT mode, BTV mode, BTH mode, TTV mode and TTH mode respectively; w and h represent the width and height of the current coding unit respectively; x c and c Indicates the coordinates of the current coding unit in the video frame to be encoded; x p and p represents the coordinates of the 32×32 coding unit in the video frame to be encoded; p represents the 112-dimensional partition prediction probability vector.
[0169] 6. Experimental Description
[0170] In order to better illustrate the advantages of the technical solution of the present invention, this embodiment discloses the following experiment.
[0171] In the experimental configuration, this experiment uses C++ and VVC reference software VTM17.0 to process and test the data. The encoder default configuration file encoder_intra_vtm.cfg is used, with four QPs (22, 27, 32, and 37). The test dataset comes from 22 video sequences of the standard test provided by JVET, covering six categories: three each of A1 and A2 (3840×2160), five of B (1920×1080), four of C (832×480), four of D (416×240), and three of E (1280×720). These sequences can effectively evaluate the performance of this method at different resolutions and scenes. In order to balance the coding complexity and rate-distortion performance, the control parameters β of ours-i and ours-ii are set to 0.4 and 0.2 in this experiment. The D-CNN and P-CNN models are designed and trained using the PyTorch framework and Python. The trained models are converted to TensorRT to achieve accelerated reasoning in the C++ environment. The evaluation process was performed on a 32GB RAM and The experiment was conducted on a Windows 10 test platform with a CoreTM i5-12490F CPU. To speed up the inference process, an NVIDIA GeForce GTX 1060 (3GB) GPU was used. This experiment selected the existing method and this method to compare the encoding performance. BD-BR was used as the main indicator for evaluating the encoding performance, and the encoding time saving rate ΔT was calculated as follows:
[0172]
[0173] Where T test QP i represents the encoding time required by the proposed method, T ori {QP i} indicates the encoding time on the VTM17.0 anchor.
[0174] Table 2 CTC encoding test results under VTM17.0 reference software
[0175]
[0176]
[0177] The results of this experiment are shown in Table 2. All methods in this table have coding performance degradation (a small increase in BD-BR) in different sequences, but the coding time has been significantly reduced. The method of Wu et al. (from S. Wu, J. Shi, and Z. Chen, "HG-FCN: Hierarchical grid fully convolutional network for fast VVC intra coding") and the method of Saldanha et al. (from M. Saldanha, G. Sanchez, C. Marcon, and L. Agos-tini, "Configurable fast block partitioning for VVC intra coding using Light Gradient Boosting Machine") save 48.76% and 53.64% of coding time respectively. The BD-BR increase of this method ranges from 0.95% to 1.45%, saving 49.57% to 58.52% of coding time. The BD-BR increase of most sequences is about 1%; however, the Basketball Drill sequence has a higher coding performance degradation, with a BD-BR increase of 2.14%. This is because the strong motion in the test sequence causes ghosting, while the training dataset mainly includes static and clear images, which cannot effectively fit these scenes, resulting in a decrease in encoding performance.
[0178] This method (ours-i) achieved an average 0.95% BDBR increase and saved 49.57% encoding time, which is a performance improvement over Wu et al.'s method in both BDBR and time savings. This method improved encoding performance by 0.19% and saved an additional 0.81% encoding time. This method (ours-ii) achieved an average 57.87% encoding time savings and a 1.47% BD-BR increase, while Saldanha et al.'s method achieved a 1.44% BD-BR increase and a 53.64% encoding time savings; with comparable BDBR performance, an additional 4.86% time savings.
[0179] Table 3 The proportion of D-CNN and P-CNN reasoning time in the encoding process (* is the proportion of reasoning time after P-CNN deployment optimization)
[0180]
[0181]
[0182] Table 3 shows the encoding performance improvement of model deployment. This method significantly reduces the time overhead of P-CNN inference. * indicates the acceleration result of model deployment with BatchSize of 4. Compared with the 1 BatchSize method, the average time saving increased by 0.49%. In particular, 1.45% time saving was achieved for the A1 test sequence with QP=37.
[0183] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.
Claims
1. A VVC intra-frame fast coding optimization method based on deep learning, It is characterized in that include: S1: Get the video frame to be encoded; S2: Divide the video frame to be encoded into a number of N1×N1 coding units; input the N1×N1 coding units into the trained D-CNN model to obtain the corresponding predicted coding depth; S3: Divide the N1×N1 coding unit into several N2×N2 coding units; input several N2×N2 coding units into the trained P-CNN model together to obtain the division prediction probability vector of each N2×N2 coding unit; S4: Determine a partitioning mode candidate list of each N2×N2 coding unit and its sub-coding units by combining the partitioning prediction probability vector of the N2×N2 coding unit with the predicted coding depth of the corresponding N1×N1 coding unit; S5: Execute steps S3 to S4 for each N1×N1 coding unit in the to-be-encoded video frame, and obtain a candidate list of partitioning modes for each N2×N2 coding unit and its sub-coding units in each N1×N1 coding unit; for each coding unit, select a partitioning mode with the smallest rate-distortion cost from its candidate list of partitioning modes as its encoding result.
2. The VVC intra-frame fast coding optimization method based on deep learning as claimed in claim 1, Features: In step S2, the D-CNN model includes an input layer, a convolution layer, a batch normalization layer, a ReLU function layer, a first fusion module, a second fusion module, a first maximum pooling layer, a third fusion module, a fourth fusion module, a second maximum pooling layer, a Flatten layer, a fully connected layer, and a Softmax activation function layer, which are connected end to end in sequence; The first to fourth fusion modules all include a convolution layer, a batch normalization layer, a ReLU function layer, a convolution layer, a batch normalization layer, a Sum layer and a ReLU function layer connected end to end in sequence; the input of each fusion module is processed by the convolution layer and the batch normalization layer and then residually connected with the input of the Sum layer.
3. The VVC intra-frame fast coding optimization method based on deep learning as claimed in claim 2, Features: In step S2, the loss function when training the D-CNN model is as follows: Where: N d is the number of training samples for each N1×N1 coding unit in each training batch; For true encoding depth; Depth of the predicted encoding output of D-CNN.
4. The VVC intra-frame fast coding optimization method based on deep learning as claimed in claim 2, Features: In step S2, the output of the D-CNN model includes six coding depth probabilities and their index values. The index value corresponding to the optimal coding depth probability is selected as the predicted coding depth through the following steps: S201: Select the maximum coding depth probability P from all coding depth probabilities d [i 1 ]; S202: Determine the maximum coding depth probability P d [i 1 ] exceeds the preset value: If so, the maximum coding depth probability P d [i 1 ] corresponds to the index value i 1 as the predicted coding depth; otherwise, executing step S203; S203: The maximum coding depth probability P d [i 1 ] is set to 0 and deleted, and the largest coding depth probability P is selected from all remaining coding depth probabilities d [i 2 ]; S204: Select index value i 1 and i 2 The larger index value max[i 1 ,i 2 ] as the prediction coding depth.
5. The VVC intra-frame fast coding optimization method based on deep learning as claimed in claim 1, Features: In step S3, the P-CNN model includes an input layer, a convolution layer, a batch normalization layer, a ReLU function layer, a first fusion module, a second fusion module, a first maximum pooling layer, a third fusion module, a fourth fusion module, a second maximum pooling layer, a fifth fusion module, a sixth fusion module, a third maximum pooling layer, a Flatten layer, a fully connected layer, and a Sigmoid activation function layer, which are connected end to end in sequence; The first to sixth fusion modules all include a convolution layer, a batch normalization layer, a ReLU function layer, a convolution layer, a batch normalization layer, a Sum layer, and a ReLU function layer connected end to end in sequence; the input of each fusion module is processed by the convolution layer and the batch normalization layer and then residually connected with the input of the Sum layer.
6. The VVC intra-frame fast coding optimization method based on deep learning as claimed in claim 5, Features: In step S3, an N2×N2 coding unit is divided into a plurality of 4×4 coding units; The output of the P-CNN model for each N2×N2 coding unit includes: the partition prediction probability vector of all boundaries of all 4×4 coding units in the N2×N2 coding unit except the boundary of the N2×N2 coding unit itself; if the partition prediction probability of a certain boundary is close to 1, it means that the boundary exists, that is, it will be further divided; if the partition prediction probability of a certain boundary is close to 0, it means that the boundary does not exist, that is, it will not be further divided; The dimension N of the partition prediction probability vector is calculated by the following formula p : Where: N w is the width of the N2×N2 coding unit; N h is the height of the N2×N2 coding unit.
7. The VVC intra-frame fast coding optimization method based on deep learning as claimed in claim 5, Features: In step S3, the loss function when training the P-CNN model is as follows: Where: N is the number of training samples of N2×N2 coding units in each training batch; y pi is the true boundary probability vector of the sample; Represents the boundary partition prediction probability vector output by P-CNN.
8. The VVC intra-frame fast coding optimization method based on deep learning as claimed in claim 1, Features: In step S4, a candidate list of partition modes for the N2×N2 coding unit and its sub-coding units is determined by the following steps: S401: Generate a partition mode candidate list for the current coding unit; S402: traverse each partition mode in the partition mode candidate list of the current coding unit, and select the partition mode with the smallest rate-distortion cost as the optimal partition mode of the current coding unit; S403: Divide the current coding unit by an optimal division mode to obtain a plurality of sub-coding units of the next size; S404: Return to step S401 and use the sub-coding unit as the current coding unit; S405: Repeat steps S401 to S404 until the coding depth of the current coding unit is greater than or equal to the predicted coding depth or the current coding unit is the minimum coding unit; S406: Obtain a partitioning mode candidate list of the N2×N2 coding unit and its sub-coding units.
9. The VVC intra-frame fast coding optimization method based on deep learning as claimed in claim 8, Features: In step S401, a candidate list of division modes of a coding unit is determined by the following steps: S4011: Create a partition mode candidate list for the current coding unit; S4012: Determine whether the coding depth of the current coding unit is greater than or equal to the predicted coding depth: if so, terminate in advance and execute step S4020; Otherwise, execute step S4013; S4013: Calculate the probabilities of five partition modes of the current coding unit based on the partition prediction probability vector of the N2×N2 coding unit; the five partition modes include QT mode, BTV mode, BTH mode, TTV mode and TTH mode; S4014: adding the partitioning mode with the highest probability to the partitioning mode candidate list; S4015: Determine whether the division mode with the highest probability is the QT mode: if so, execute step S4016; Otherwise, execute S4017; S4016: Determine whether abs(P BTV -P BTH )>β: If yes, then add the BTV mode and the BTH mode with a higher probability into the candidate list of division modes, and execute step S4020; otherwise, directly execute step S4020; S4017: Determine whether the maximum probability type is BTH mode or TTH mode: if yes, execute step S4018; otherwise, execute step S4019; S4018: Determine whether abs(P BTH -P TTH )<β: If yes, then add the TTH mode or BTH model to the candidate list of partition modes and execute step S4020; otherwise, directly execute step S4020; S4019: Determine whether abs(P BTV -P TTV )<β: If yes, then add the TTV mode or the BTV mode to the candidate list of division modes and execute step S4020; otherwise, directly execute step S4020; S4020: Output a partition mode candidate list of the current coding unit and perform the next coding depth.
10. The VVC intra-frame fast coding optimization method based on deep learning as claimed in claim 9, Features: In step S4013, the probabilities of the five division modes are calculated by the following formula: 1) QT mode 2) BTV Model 3) BTH mode 4) TTV Model 5) TTH mode Where: P QT , P BTV , P BTH , P TTV , P TTH They represent the probabilities of QT mode, BTV mode, BTH mode, TTV mode and TTH mode respectively; w and h represent the width and height of the current coding unit respectively; x c and c Indicates the coordinates of the current coding unit in the video frame to be encoded; x p and p represents the coordinates of the 32×32 coding unit in the video frame to be encoded; p represents the partition prediction probability vector.
Citation Information
Patent Citations
Multifunctional video fast coding method based on deep neural network
CN111263145A
Fast VVC intra-frame prediction method based on ensemble learning and probability model
CN111711815A
VVC intra-frame fast coding method based on deep learning
CN112702599A
Cited By
VVC rapid CU division method based on random forest and residual network
CN119676437A
