A VVC Intra-Prediction Optimization Method Based on Inter-Block Correlation

Abstract

This invention proposes a VVC intra-prediction optimization method based on inter-block correlation. Since a single prediction mode is sometimes unsatisfactory and the correlation between adjacent blocks is not fully utilized, we fuse the modes of adjacent blocks in the current block and then apply different weighting methods to different mode selections to obtain the final prediction value. To better predict complex regions, we propose an adaptive template matching algorithm, using adaptive weighting for small blocks and linear weighting for large blocks. To compensate for the shortcomings of single linear prediction, we combine CCLM and DM, and use adaptive weighting to obtain the final prediction value. Experimental results show that compared with the VVC standard, the average bitrate savings for Y, Cb, and Cr are 0.63%, 0.82%, and 0.89%, respectively, and the utilization rate of our proposed mode is higher than many traditional VVC intra-prediction modes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video coding, and particularly to a method for optimizing intra prediction of a Versatile Video Coding (VVC) based on inter-block correlation. BACKGROUND

[0002] The proposal of Versatile Video Coding (VVC) meets the demand of various videos. VVC still follows the block-based hybrid coding framework of High Efficiency Video Coding (HEVC) and introduces many new technologies to improve the coding performance, so its compression performance is higher. On the basis of HEVC, VVC reduces the coding bit rate by about 50%, but its coding complexity is also higher. Taking intra prediction as an example, VVC extends the original 35 intra prediction modes of HEVC to 67, and increases the length of the original most probable mode list from 3 to 6. At the same time, in the CU division, in addition to the quad-tree in HEVC, binary tree and ternary tree are also used, so that the division result is more suitable for video content. In addition, other new technologies are also added in VVC to improve its prediction accuracy, such as matrix weighted prediction, multiple reference lines, wide-angle prediction, intra sub-block division, cross-component linear model, and position-based intra prediction combination.

[0003] Although the performance of VVC has been greatly improved, there is still room for further optimization, and there are many studies in the academic field on this aspect. In view of the fact that the single angle prediction mode can only predict a single direction well, resulting in a small coding block after using side information for rate distortion optimization, Gagan et al. proposed an optimization algorithm that can make the prediction direction change slowly within a prediction block, so that for those prediction blocks with slowly changing texture direction, better prediction can be achieved without splitting them. Since the existing prediction modes mostly cannot predict the texture complex region well, and the texture complex region is very common in video sequences, Lei et al. proposed a two-step progressive intra prediction algorithm that simultaneously uses local and non-local information to improve coding performance. The CCLM mode in VVC uses a linear model to obtain the chroma component from the reconstructed luminance component, but this single linear model is relatively simple, and the samples used to obtain the model parameters are also limited, so sometimes the prediction is not accurate enough. In order to solve this problem, Ramin et al. average the prediction values obtained by the CCLM mode and the DM mode, and use the DM mode to compensate for the shortcomings of the CCLM mode, because DM also assumes a linear relationship between the luminance component and the chroma component, so DM is selected. In order to better predict complex regions in video sequences, Lei et al. divided the search region on the basis of the traditional template matching algorithm, and changed the shape of the matching template to make the result more accurate. SUMMARY

[0004] In view of the fact that the multi-functional video coding standard VVC still has certain room for improvement in the aspect of intra prediction, and can further reduce the code rate, the present application aims to propose a VVC intra prediction optimization method based on inter-block correlation.

[0005] The basic idea of the present application is to add two additional modes to the traditional 67 intra prediction modes of VVC to solve the problem that the traditional prediction mode cannot predict complex blocks well, and these two modes are applied to both the luminance component and the chroma component. In addition, another mode is proposed for the chroma mode to solve the problem of insufficient single linear prediction. The method mainly includes three steps, which are:

[0006] Step 1: A multi-prediction mode fusion algorithm is proposed, which fully utilizes the correlation between the current prediction block and its adjacent blocks and makes up for the shortcomings of single prediction mode, and is applied to both the luminance component and the chroma component;

[0007] Step 2: An adaptive template matching algorithm is proposed, which uses different weighting methods for different sizes of prediction blocks due to different texture information of different sizes of blocks, and is applied to both the luminance component and the chroma component;

[0008] Step three: a colorimetric algorithm is proposed to fuse the CCLM mode and the DM mode, which makes up for the shortcomings of single linear prediction and only targets the chroma component;

[0009] Step four: compare the prediction values obtained by the proposed mode with those obtained by the traditional VVC intra prediction mode, and select the best prediction mode for the current prediction unit (PU) through the rate-distortion criterion.

[0010] The specific process is as follows:

[0011] (1) Obtain the prediction modes of the prediction blocks above and to the left of the current prediction block, denoted as a and b respectively;

[0012] (2) If a or b does not exist, record the first mode as Planar mode and the second mode as DC mode; if a and b both exist, if a and b are equal and are Planar mode or DC mode or multi-prediction mode fusion, record the first mode as Planar mode and the second mode as DC mode;

[0013] (3) If a and b both exist, if a and b are equal and are not Planar mode or DC mode or multi-prediction mode fusion, record the first mode as a-1, the second mode as a, and the third mode as a+1, and when a+1 does not exist, replace it with a-2;

[0014] (4) If a and b are not equal, record the first mode as a and the second mode as b;

[0015] (5) After obtaining the corresponding mode, use different weighting algorithms to weight the prediction values of the selected modes to obtain the final prediction value.

[0016] (6) In the reconstructed 64x64 region, perform template matching on blocks with a size less than or equal to 32x32, and obtain the four best candidate templates with the smallest mean square error according to the mean square error criterion;

[0017] (7) Determine the size of the current PU block, if the PU size is less than or equal to 8x8, the texture information is complex, use adaptive weighting method to adaptively weight the prediction values corresponding to the four candidate templates to obtain the final prediction value;

[0018] (8) If the PU size is greater than 8x8 and less than or equal to 32x32, the texture information is relatively simple, then linearly weight the prediction values corresponding to the three best candidate templates to obtain the final prediction value;

[0019] (9) Other blocks are filled with DC values.

[0020] (10) For the chroma component, first, the prediction value of the CCLM mode is obtained, then the prediction value of the DM mode is obtained, and finally the two prediction values are adaptively weighted to obtain the final chroma prediction value.

[0021] In process (2), the final prediction modes are Planar mode and DC mode, and the prediction values obtained by the two prediction modes are weighted by formula (1) to obtain the final prediction value,

[0022]

[0023] wherein p(x, y) represents the final prediction value, (x, y) represents the position of the pixel point, p planar (x, y) and p DC (x, y) represent the prediction values corresponding to the Planar mode and the DC mode respectively.

[0024] In process (3), the final modes are three modes of a-1, a and a+1, and the prediction values obtained by the three prediction modes are weighted by formula (2),

[0025]

[0026] wherein p a-1 (x, y), p a (x, y) and p a+1 (x, y) represent the prediction values corresponding to the mode a-1, the mode a and the mode a+1 respectively.

[0027] In process (4), the final modes are two modes of a and b, and the prediction values obtained by the two prediction modes are weighted by formula (3),

[0028]

[0029] wherein p a (x, y) and p b (x, y) represent the prediction values of the mode a and the mode b respectively, ω = ω1 + ω2 is a normalization factor, and ω1 and ω2 are weight factors of the mode a and the mode b, which are obtained by formula (4) and formula (5) respectively,

[0030]

[0031]

[0032] wherein M a , N a and R a represent the width, height and residual value respectively when the prediction mode is the mode of the left block, M b , Nb and R b respectively represent the width, height and residual value when using the upper block mode, m a and m b respectively represent the average value of total pixels when using the left prediction mode and the upper prediction mode.

[0033] The adaptive weighting used in process (7) is shown in formula (6),

[0034]

[0035] where ω i represents the weight of the candidate template B i , which is obtained by formula (7),

[0036]

[0037] where, represents the mean square error between the candidate template T i and the reference template T r , which is obtained by formula (8),

[0038]

[0039] where M is the total pixel of the current prediction block.

[0040] The linear weighting used in process (8) is shown in formula (9),

[0041]

[0042] where, is sorted from small to large, that is, They respectively correspond to the candidate templates T1, T2, T3.

[0043] In process (10), the final prediction value is obtained by using formula (10),

[0044]

[0045] where p DM (x, y) and p CCLM (x, y) are the prediction values corresponding to the DM mode and the CCLM mode respectively, ω1 and ω2 are weighting factors, which are obtained by formula (4) and (5).

[0046] Compared with the prior art, the present application has the advantages and beneficial technical effects:

[0047] (1) The VVC intra-frame prediction optimization method based on inter-block correlation proposed in this invention integrates the prediction modes of the adjacent blocks of the current block, which makes up for the problem that the prediction of a single prediction mode is sometimes not ideal and makes full use of the correlation between adjacent blocks.

[0048] (2) The VVC intra-frame prediction optimization method based on inter-block correlation proposed in this invention fully considers the different texture information of blocks of different sizes when performing template matching prediction. It adopts adaptive weighting for small blocks and linear weighting for large blocks.

[0049] (3) The VVC intra-frame prediction optimization method based on inter-block correlation proposed in this invention combines CCLM mode and DM mode, which makes up for the problem of insufficient single linear prediction of CCLM, and adopts adaptive weighting of the prediction values ​​of the two modes to obtain the final prediction value. Attached Figure Description

[0050] Figure 1 This is the overall flowchart of the VVC intra-prediction optimization method based on inter-block correlation.

[0051] Figure 2 This is a flowchart of the adaptive template matching algorithm.

[0052] Figure 3 Rate-distortion curves of the sequence Johnny based on the method of this invention and VVC under AI configuration.

[0053] Figure 4 For the sequence Johnny, the rate-distortion curve local amplification part based on the method of this invention and VVC under AI configuration ( Figure 3 (The rectangular frame part in the text).

[0054] Figure 5 Rate-distortion curves of sequence BQTerrace based on the method of this invention and VVC under AI configuration.

[0055] Figure 6 A partial magnification of the rate-distortion curve of sequence BQTerrace under AI configuration based on the method of this invention and VVC ( Figure 5 (The rectangular frame part in the text).

[0056] Figure 7 The graph shows the number of brightness modes used in the sequence KristenAndSara at QP27, under AI configuration, based on the method of this invention. Detailed Implementation

[0057] The application will be further described in detail in connection with the following examples. It is necessary to point out that the following examples are only used to further illustrate the application and cannot be understood as limiting the protection scope of the application. Those skilled in the art can make some non-essential improvements and adjustments to the specific implementation of the application according to the above description, which should still belong to the protection scope of the application.

[0058] (1) The method of the application is carried out on the VVC standard test code VTM-9.1 platform. Among them, in the AI configuration, the configuration file is selected as encoder_intra_vtm.cfg, the number of encoded frames is 30, and the standard video test sequence is coded and decoded under the condition that the quantization parameter QP is 22, 27, 32 and 37, and the code rate and peak signal-to-noise ratio PSNR during standard VVC video coding are recorded;

[0059] (2) The coded object is a standard VVC test video, and its name and resolution are respectively: Cactus (1920x1080), BasketballDrive (1920x1080), BQTerrace (1920x1080), FourPeople (1280x720), Johnny (1280x720), KristenAndSara (1280x720), BasketballDrill (832x480), BQMall (832x480), PartyScene (832x480), RaceHorseC (832x480), BasketballPass (416x240), BQSquare (416x240), BlowingBubbles (416x240), RaceHorse (416x240);

[0060] (3) The method of the application is used to code the video sequence under the above conditions, and the code rate and peak signal-to-noise ratio PSNR during video coding by the method of the application are recorded;

[0061] (4) We use BD-rate to calculate the compression performance of the application method relative to VVC. If the index is negative, it indicates that the performance is improved;

[0062] (5) Table 1 shows the comparison of compression performance between VVC and the application under AI configuration. On average, compared with the VVC standard, the bit rate of Y, Cb and Cr components is saved by 0.63%, 0.82% and 0.89% respectively, and the highest bit rate saving is 2.59%, 3.00% and 2.72% respectively, which shows that the method has good effect;

[0063] (6) Table 2 is the proportion of the total prediction mode occupied by the proposed multi-prediction mode fusion algorithm in the best prediction mode of the luminance component. The selected sequences are BasketballPass, Johnny, KristenAndSara, BasketballDrive and Cactus. The mode is selected to different degrees in different sequences, indicating that our mode has a certain effect;

[0064] (7) Table 3 is the proportion of the total prediction mode occupied by the proposed adaptive template matching algorithm in the best prediction mode of the luminance component. The selected sequences are BasketballPass, Johnny, KristenAndSara, BasketballDrive and Cactus. The mode is selected at a higher rate in different sequences, indicating that the mode has better effect;

[0065] (8) Table 4 is the proportion of the total prediction mode occupied by the proposed algorithm in the best prediction mode of the chrominance component. The first row is the total proportion of the three algorithms, and the second row is the proportion of the third algorithm in the three algorithms. The selected sequences are BasketballPass, Johnny, KristenAndSara, BasketballDrive and Cactus. In the chrominance mode, our method is also selected to different degrees, proving the effectiveness of our proposed method;

[0066] (9) From Figures 3 to 6 It can be seen that the rate-distortion curve of our proposed method is above the VVC standard, which shows that under the same PSNR condition, the bit rate required by our proposed method is lower, that is, our proposed method has better effect in saving bit rate;

[0067] (10) From Figure 7 It can be seen that the use rate of our proposed method exceeds most of the VVC traditional prediction modes, indicating that under the rate-distortion optimization criterion, our algorithm is more competitive many times and can save more bit rate.

[0068] Table 1 bit rate comparison between the present application and VVC under AI configuration

[0069]

[0070]

[0071] Table 2 proportion of multi-prediction mode fusion algorithm in luminance component

[0072]

[0073] Table 3 Proportion of adaptive template matching algorithm in luminance component

[0074]

[0075] Table 4 Proportion of proposed algorithm in chrominance component

[0076]

[0077]

Claims

1. A VVC intra prediction optimization method based on inter-block correlation, characterized in that: (1) a multi-prediction mode fusion algorithm is proposed, and the specific process is as follows: the prediction modes of the top and left prediction blocks of the current prediction block are obtained, and are denoted as a and b respectively; if a or b does not exist, mode 1 is Planar mode and mode 2 is DC mode; if a and b both exist, if a and b are equal and are Planar mode or DC mode or multi-prediction mode fusion, mode 1 is Planar mode and mode 2 is DC mode; if a and b are equal and are not Planar mode or DC mode or multi-prediction mode fusion, mode 1 is a-1, mode 2 is a, and mode 3 is a+1, and when a+1 does not exist, a-2 is used instead; if a and b are not equal, mode 1 is a and mode 2 is b; after obtaining the corresponding modes, different weighting algorithms are used to weight the prediction values of the recorded modes to obtain the final prediction value; (2) an adaptive template matching algorithm is proposed, and the specific process is as follows: first, template matching is performed in a specific area, and according to the mean square error criterion, the four best candidate templates with the smallest mean square error are obtained; then, the size of the current PU is judged, if the PU size is less than or equal to 8x8, an adaptive weighting method is used to adaptively weight the prediction values corresponding to the four best candidate templates to obtain the final prediction value; if the PU size is greater than 8x8 and less than or equal to 32x32, the prediction values corresponding to the three best candidate templates are linearly weighted to obtain the final prediction value; other blocks do not perform template matching and are directly filled with DC values; (3) a color algorithm that fuses CCLM mode and DM mode is proposed, and the specific process is as follows: for the chroma component, first, the prediction value of the CCLM mode is obtained, then the prediction value of the DM mode is obtained, and finally the two prediction values are adaptively weighted to obtain the final chroma prediction value; (4) the prediction value obtained by the algorithm is compared with the prediction value obtained by the traditional VVC intra prediction mode, and the best prediction mode of the current prediction block (PU) is selected through the rate-distortion criterion.

2. The VVC intra prediction optimization method based on inter-block correlation of claim 1, wherein The multi-prediction mode fusion algorithm described in (1) is applied to both the luminance component and the chroma component.

3. The VVC intra prediction optimization method based on inter-block correlation of claim 1, wherein The adaptive template matching algorithm described in (2) is applied to both the luminance component and the chroma component.

4. The VVC intra prediction optimization method based on inter-block correlation of claim 1, wherein The color algorithm that fuses CCLM mode and DM mode described in (3) is only for the chroma component.

Images