Vvc intra fast cu partitioning method based on texture and neighborhood partition information

Abstract

The present application discloses a VVC intra fast CU partitioning method based on texture and neighbor partitioning information from the correlation between the VVC intra CU partitioning mode and the texture information and the partitioning mode of the adjacent CU. Firstly, the texture information of the current CU and the partitioning mode of the adjacent CU are extracted, and then the CU is divided into three categories according to the size, and the texture and neighbor information are used to skip some unnecessary partitioning modes in advance, thereby reducing the encoding time of intra prediction. The VVC intra fast CU partitioning method disclosed by the present application significantly reduces the time complexity of VVC intra coding without significantly reducing the compression coding efficiency compared with the VVC coding standard.

Description

Technical Field

[0001] This invention relates to the problem of reducing intra-frame coding complexity in the field of video coding, and in particular to a fast intra-frame CU partitioning method for VVC based on texture and neighborhood partitioning information. Background Technology

[0002] With the development and popularization of 5G technology and the rise of various video platforms, people's demand for high-definition video is constantly growing. The compression capabilities of existing video coding technologies H.264 / AVC and H.265 / HEVC can no longer meet market demands. In order to research more efficient video compression coding technologies to support the next generation of high-resolution video coding standards, JVET (Joint Video Exploration Team), jointly established by MPEG and ITU, proposed a new generation of video compression coding standard H.266 / VVC. The overall coding framework of VVC is not much different from HEVC, but new technologies are introduced in each coding module. Without affecting the visual quality of the video, VVC coding efficiency is more than 50% higher than HEVC. However, this also brings a significant increase in coding time, which is not conducive to real-time communication. In the CU partitioning module, unlike the quadtree partitioning of HEVC, VVC introduces a multi-type tree block mode, adding binary tree and ternary tree partitioning (QTMT) on the basis of quadtree partitioning. This means that the CU in VVC is no longer just a square, but can be a square or a rectangle. The more flexible block segmentation method allows VVC to predict coding blocks more accurately, reducing prediction errors and thus lowering the bitrate. However, this also introduces significant time complexity. Furthermore, the number of intra-frame prediction modes in VVC has increased from 35 in HEVC to 67, further increasing time complexity. Therefore, finding a fast video compression coding algorithm that does not significantly reduce video compression efficiency is essential.

[0003] In Video Visual C++ (VVC), the partitioning method of intra-frame CUs is related to texture information. Larger CUs are typically used in areas with relatively uniform texture, while smaller CUs are used in areas with richer texture. VVC adds binary and ternary tree partitioning, both including vertical and horizontal partitioning methods. The horizontal and vertical partitioning is related to the texture direction; horizontal partitioning is typically used in areas with predominantly horizontal texture, while vertical partitioning is typically used in areas with predominantly vertical texture. Furthermore, since most of the image frames in a video sequence are continuous, there is also a correlation between the partitioning method of intra-frame CUs and the partitioning methods of their adjacent CUs. When adjacent CUs are partitioned into large blocks, the current CU usually also uses a large size as the optimal size; when adjacent CUs use quadtree, horizontal, or vertical partitioning, the current CU usually uses the same partitioning method.

[0004] Therefore, we can use these correlations to skip certain unnecessary CU partitioning methods in advance, reducing the time complexity of intra-frame prediction without significantly affecting the bitrate and quality of intra-frame coding. Summary of the Invention

[0005] In the VVC intra-frame CU partitioning process, all partitioning methods are traversed, and the one with the lowest cost is selected as the optimal partitioning method. VVC, based on the quadtree partitioning method of HEVC, adds multiple tree partitioning methods, namely binary and ternary tree partitioning. These multiple tree types also include horizontal and vertical partitioning. This means that the CU partitioning methods in VVC are more complex, requiring the traversal of more partitioning methods, and significantly increasing the time complexity. In actual partitioning, the optimal CU partitioning method is strongly correlated with texture information and the partitioning methods of its adjacent CUs, but VVC does not utilize these correlations; instead, it directly traverses all partitioning methods, thus adding a lot of unnecessary computation time.

[0006] To address the correlation between CU partitioning patterns and their texture and neighborhood partitioning information during VVC intra-frame CU partitioning, this invention aims to propose a method to effectively reduce intra-frame prediction complexity without significantly decreasing VVC compression coding efficiency. This method involves extracting the texture information of the current CU and the partitioning information of adjacent CUs during VVC intra-frame CU partitioning to preemptively skip certain unnecessary partitioning methods. This reduces the complexity of intra-frame CU partitioning, thereby achieving the goal of reducing intra-frame prediction coding time without significantly impacting video bitrate and quality.

[0007] This invention patent proposes a method for reducing the time complexity of intra-CU partitioning for VVC. When performing intra-CU partitioning, for the current CU, first extract the texture richness and texture directionality of the current CU, as well as the partitioning methods of the adjacent CUs on the left and above, to determine which partitioning method the current CU is more likely to adopt, so as to skip other less likely partitions in advance. The specific steps are as follows:

[0008] (1) Use the Sobel operator to extract the gradient value G and the horizontal-vertical gradient ratio B of the current CU, and obtain the width LW and height LH of the adjacent CU on the left of the current CU and the width AW and height AH of the adjacent CU above. The calculation formulas of G and B are shown in Formulas (1) and (2) respectively.

[0009]

[0010]

[0011] where N is the total number of pixel points in the current CU, G , ,

[0013] , , , , ,

[0015] , x,y ,

[0014] , ,

[0017] , ,

[0016] ,

[0012] and G y are the Sobel gradient values in the horizontal and vertical directions respectively, and their calculation formulas are shown in Formulas (3) and (4) respectively.

[0012]

[0013]

[0014] where (x, y) is the coordinate of the current pixel point, and M x,y is a 3×3 pixel matrix centered on the current pixel point;

[0015] (2) According to the size of the CU, divide the CU into three categories for processing: CUs with a size of 64×64, CUs with a size of 32×32, and CUs of other sizes. If the current CU size is 64×64, execute step (3). If the current CU size is 32×32, then go to step (4). If it is a CU of other sizes, then go to step (5);

[0016] (3) When G < TH1, if at least one of LW, LH, AW, and AH is equal to 64, terminate the subsequent partitioning. If the adjacent CUs do not have a side of 64, but there is a square (LW == LH or AW == AH) among the adjacent CUs, then directly perform quadtree partitioning and skip other partitions. If T > TH2, then go to step (6). If it is other situations, then perform partitioning according to the original process;

[0017] (4) When G < TH3, if at least one of LW, LH, AW, and AH is equal to 64 or 32, terminate the subsequent partitioning. If the adjacent CUs do not have sides of 64 and 32, but there are squares (LW == LH or AW == AH) among the adjacent CUs, directly perform quadtree partitioning and skip other partitioning. If T > TH4, go to step (6). In other cases, perform partitioning according to the original process;

[0018] (5) If G < TH5, only perform non - partitioning and quadtree partitioning (only non - partitioning when quadtree is not available). If G > TH6, go to step (6). In other cases, perform partitioning according to the original process;

[0019] (6) If B > TH7 and there is horizontal partitioning (LW > LH or AW > AH) in the adjacent CUs on the left and above, only perform horizontal partitioning (horizontal binary tree and horizontal ternary tree). If B < TH8 and there is vertical partitioning (LW < LH or AW < AH) in the adjacent CUs on the left and above, only perform vertical partitioning (vertical binary tree and vertical ternary tree). If TH8 < B < TH7, perform partitioning according to the original process.

[0020] Advantages and beneficial technical effects of the present invention compared with the prior art:

[0021] (1) The fast intra - CU partitioning method for VVC based on texture and neighborhood partitioning information proposed by the present invention, compared with the traditional VVC standard encoding, without significantly reducing the intra - encoding performance, the time complexity of the intra - prediction encoding of the fast intra - CU partitioning method proposed by the present invention is significantly lower than that of the VVC standard algorithm;

[0022] (2) The fast intra - CU partitioning method for VVC based on texture and neighborhood partitioning information proposed by the present invention analyzes the correlation between the best partitioning method of VVC intra - CU and its texture information. Using this correlation, for CUs with uniform texture, skip the subsequent partitioning in advance, for CUs with rich texture, directly perform the next partitioning, and for CUs with horizontal - biased texture, directly adopt horizontal partitioning, and for CUs with vertical - biased texture, directly adopt vertical partitioning, thereby reducing the complexity of CU partitioning and reducing the encoding time of intra - prediction;

[0023] (3) The fast intra - CU partitioning method for VVC based on texture and neighborhood partitioning information proposed by the present invention analyzes the correlation between the best partitioning method of VVC intra - CU and the partitioning methods of its adjacent CUs. Using this correlation, by skipping unnecessary direction partitioning in advance, further saving the encoding time of intra - prediction;

[0024] (4) The VVC intra-frame fast CU partitioning method based on texture and neighborhood partitioning information proposed in this invention analyzes the gradient value distribution of CUs of different sizes and divides the CUs into three categories of CUs of 64×64, 32×32 and other sizes for processing respectively. While reducing time complexity, it also ensures that the coding efficiency is basically unaffected. Attached Figure Description

[0025] Figure 1 These are the 67 intra-frame prediction modes in VVC.

[0026] Figure 2 Five CU partitioning methods are given in the QTMT structure of VVC.

[0027] Figure 3 The intra-frame partitioning results for the test sequence BasketballPass_416x240_50.

[0028] Figure 4 The CU gradient distribution of the test sequence BasketballDrill_832x480_50 when QP is 22.

[0029] Figure 5 This is a flowchart of a fast intra-CU partitioning method for VVC based on texture and neighborhood partitioning information.

[0030] Figure 6 Rate distortion curves for ParkScene and RaceHorsesC test sequences. Detailed Implementation

[0031] The present invention will be further described in detail below with reference to the embodiments. It should be noted that the following embodiments are only used to further illustrate the present invention and should not be construed as limiting the scope of protection of the present invention. Those skilled in the art can make some non-essential improvements and adjustments to the present invention based on the above-described invention, and these improvements and adjustments should still fall within the scope of protection of the present invention.

[0032] (1) The method proposed in this invention is implemented on the VVC standard test code VTM9.1 platform. The configuration file is encoder_intra_main.cfg, the encoding structure is full I-frame encoding, and the quantization parameter QP is set to 22, 27, 32, 37;

[0033] (2) The test sequences used to verify the coding performance of the proposed method were official standard test sequences with resolutions of 416×240, 832×480, 1280×720, 1920×1080, and 2560×1600. Specifically, the standard test sequences used were BasketballPass, BlowingBubbles, BQSquare, Flowervase, Keiba, Mobisode2, RaceHorses, BasketballDrill, BQMall, Keiba, PartyScene, RaceHorsesC, FourPeople, Johnny, KristenAndSara, SlideEditing, SlideShow, BasketballDrive, BQTerrace, Cactus, Kimono, ParkScene, Traffic, PeopleOnStreet, and SteamLocomotiveTrain.

[0034] (3) In the program of the algorithm proposed in this invention, the parameters in (1) are set in the configuration file encoder_intra_main.cfg, and then all the video standard sequences to be tested are input to obtain the results of the method proposed in this invention under the case of full I frame, and the encoding time, bit rate and video quality are recorded and statistically analyzed.

[0035] (4) In the VVC standard algorithm program, set the parameters in (1) in the configuration file encoder_intra_main.cfg, then input all the video standard sequences that need to be tested, obtain the results of the VVC standard algorithm in the case of full I frames, and record and count the encoding time, bit rate and video quality.

[0036] (5) The results of the above two types are processed respectively. The coding time complexity is measured by ΔT, which represents the coding time reduction of the proposed method relative to VVC. The quality of coding performance is measured by the objective evaluation standards BDBR and BDPSNR.

[0037] (6) As can be seen from Table 1, compared with the VVC standard algorithm, when the coding structure is all I-frames, the method proposed in this invention reduces the coding time by an average of 44.22%, while BDBR only increases by 0.93% and BDPSNR only decreases by 0.04dB. In summary, while ensuring the coding efficiency of intra-frame prediction, the intra-frame fast CU partitioning method proposed in this invention can effectively reduce the coding time compared with the VVC standard algorithm;

[0038] (7) Figure 6Rate-distortion curves for test sequences at different resolutions are shown. The horizontal axis represents the coding bit rate in Kbps, and the vertical axis represents the PSNR in dB. Figure 6 As can be seen, under the premise of effectively reducing the coding time of intra-frame prediction, the rate-distortion curve of the proposed method basically coincides with the rate-distortion curve of the VVC standard algorithm, indicating that the compression coding efficiency of the proposed method is basically not reduced.

[0039] Table 1 shows the experimental results of this method when the coding structure is full I-frame.

[0040]

Claims

1. A fast intra CU partitioning method for VVC based on texture and neighborhood partitioning information, characterized in that: (1) Use the Sobel operator to extract the gradient value G and the horizontal-vertical gradient ratio B of the current CU, and obtain the width LW and height LH of the left adjacent CU of the current CU and the width AW and height AH of the upper adjacent CU. The calculation formulas of G and B are shown in formulas (1) and (2) respectively. Where N is the total number of pixels in the current CU, and G x and G y These are the Sobel gradient values ​​in the horizontal and vertical directions, respectively, and their calculation formulas are shown in equations (3) and (4), respectively. Where (x,y) are the coordinates of the current pixel, M x,y It is a 3×3 pixel matrix centered on the current pixel; (2) Divide the CUs into three categories according to the size of the CU, namely CUs with a size of 64×64, CUs with a size of 32×32, and CUs of other sizes. If the current CU size is 64×64, execute step (3). If the current CU size is 32×32, then go to step (4). If it is a CU of other sizes, then go to step (5). (3) When G < TH1, if at least one of LW, LH, AW, and AH is equal to 64, terminate the subsequent partitioning. If the adjacent CUs do not have a side of 64, but there is a square in the adjacent CUs, that is, LW == LH or AW == AH, then directly perform quadtree partitioning and skip other partitioning. If G > TH2, then go to step (6). If it is other cases, then perform partitioning according to the original process. (4) When G < TH3, if at least one of LW, LH, AW, and AH is equal to 64 or 32, terminate the subsequent partitioning. If the adjacent CUs do not have sides of 64 and 32, but there is a square in the adjacent CUs, then directly perform quadtree partitioning and skip other partitioning. If G > TH4, then go to step (6). In other cases, perform partitioning according to the original process. (5) If G < TH5, only perform no partitioning and quadtree partitioning, and only perform no partitioning when the quadtree is not available. If G > TH6, then go to step (6). In other cases, perform partitioning according to the original process. (6) If B > TH7 and there is a horizontal partition in the left and upper adjacent CUs, that is, LW > LH or AW > AH, then only perform horizontal partitioning, including horizontal binary tree and horizontal ternary tree. If B < TH8 and there is a vertical partition in the left and upper adjacent CUs, that is, LW < LH or AW < AH, then only perform vertical partitioning, including vertical binary tree and vertical ternary tree. If TH8 < B < TH7, then perform partitioning according to the original process.

Images