Intra fast mts method based on multi-core transform and neighbor coding information

Abstract

The application discloses an intra-frame fast MTS method based on multi-core transformation and neighborhood coding information from the MTS process of VVC intra-frame coding. First, the correlation between sub-CUs and the relationship between the RD costs of various transformations are utilized to predict the RD cost of the last sub-CU, then whether the current CU needs to be divided is judged according to the relationship between the sum of the RD costs of the sub-CUs and the RD cost of the parent CU, and the MTS candidate list of the current CU is rearranged according to the coding information of the adjacent CUs to further reduce the coding time. The VVC intra-frame fast MTS method disclosed by the application obviously reduces the time complexity of VVC intra-frame coding under the premise that the compression coding efficiency is not obviously reduced compared with the VVC coding standard.

Description

Technical Field

[0001] This invention relates to the problem of reducing intra-frame coding complexity in the field of video coding, and in particular to a fast intra-frame MTS method for Versatile Video Coding (VVC) based on multi-core transform and neighborhood coding information. Background Technology

[0002] Today, video applications are evolving towards higher resolution, high dynamic range (HDR), and high frame rate (HFR). This has rendered the High Efficiency Video Coding (HEVC) standard inadequate for video coding tasks. Consequently, the demand for next-generation video coding technologies in the video-related industries has become increasingly urgent. Against this backdrop, the Joint Video Experts Team (JVET) developed a new technology called Versatile Video Coding (VVC) to better address these video development trends. While VVC achieves better coding performance than HEVC through a suite of coding tools, its high computational complexity limits its use in real-time multimedia applications.

[0003] Transformation is one of the most important modules in video codecs because it is applied not only to a specific type of video frame but also to the prediction residual blocks in all video frames for subsequent quantization and entropy coding. Mathematically, there are eight types of DCT transforms. In previous video coding standards, DCT-2 became the mainstream transform tool due to its good balance between compression efficiency and time complexity. Currently, it has been theoretically proven that under the first-order stationary Markov condition, DCT-2 can effectively approximate the optimal signal-correlated Karhunen-Loève Transform (KLT). However, in natural video images containing rapidly moving regions, the first-order Markov condition is not always satisfied. To better capture dynamic content in video images, VVC introduced the Adaptive Multiple Transform (AMT) technique at a previous JVET conference. AMT, while retaining DCT-2 in HEVC, adds several DCT / DST transforms, including DCT-8, DCT-5, DST-1, and DST-7, achieving effective gains in compression efficiency, especially in high-resolution video sequences. Because it requires calculating five transform types to select the optimal transform mode, the computational cost is considerable, resulting in very high encoder time complexity. In the VVC working draft, AMT was simplified, retaining only DCT-2, DCT-8, and DST-7, and renamed Multiple Transform Selection (MTS). Although the simplified MTS eliminates two transform kernels, saving encoding time to some extent compared to AMT, its computational complexity remains high. Furthermore, VVC employs many advanced encoding tools, such as a quad-tree plus multi-type tree (QTMT) partitioning structure, affine motion compensation prediction, and 67 intra-frame direction prediction modes. These advanced tools make VVC's encoding process quite flexible, but they also increase computational complexity. Therefore, it is necessary to simplify the VVC encoding process to make it more suitable for real-time applications.

[0004] Currently, most work focuses on terminating CU partitioning early to accelerate the encoding process. For example, Tang et al. proposed a fast CU partitioning method for intra-frame and inter-frame modes based on edge information extracted by the Canny operator. Lin et al. introduced a spatial feature-based method to accelerate binary tree partitioning of CUs. Zhang et al. proposed a method based on information entropy to accelerate CU partitioning. To more effectively extract and utilize features in videos, some methods based on convolutional neural networks (CNNs) have been proposed to further accelerate CU partitioning. For intra-frame coding of VVC, Jin et al. proposed a fast CU partitioning method based on CNNs. Pan et al. proposed a fast inter-frame coding method based on joint multi-domain information to terminate the CU partitioning process early. However, there is still significant room for improvement in accelerating VVC coding. Therefore, we propose a fast intra-frame MTS method based on multi-kernel transform and neighborhood coding information. The proposed algorithm accelerates the VVC coding process and can also be combined with other fast algorithms to achieve even faster VVC video coding. Summary of the Invention

[0005] To address the issue that VVC still has room for improvement in transform kernel MTS and can further reduce the bit rate, this invention aims to propose an intra-frame fast MTS method based on multi-kernel transform and neighborhood coding information.

[0006] In the intra-frame coding process, we first predict the RD cost of the last sub-CU based on the correlation between the last and previous sub-CUs in the binary CU partition, using the RD cost of the previous sub-CU and the information entropy ratio of the two CUs. Then, we compare the sum of the RD costs of all sub-CUs with the RD cost of the parent CU; if the former is greater than the latter, we skip the MTS process and do not partition that CU. For cases other than quadtree partitioning, extensive statistical analysis shows that in most cases, the RD cost of a CU using DCT-2 transform is not significantly different from that using MTS. Therefore, in these cases, we only perform DCT-2 transform on the last sub-CU and calculate its corresponding RD cost. In addition, we reorder the MTS candidate sequence based on the coding information of its neighboring CUs. This directly affects the subsequent RD checking process, further accelerating the intra-frame coding process. Specifically, this includes the following steps:

[0007] (1) Calculate the ratio of the information entropy contained in the last sub-CU and the previous sub-CU under the binary tree segmentation, and determine the similarity between the two sub-CUs by the ratio;

[0008] (2) For two similar sub-CUs under binary tree segmentation (i.e., the information entropy ratio is in the range of 0.9 to 1.1), the product of the former's RD cost and the information entropy ratio is used as the estimated value of the latter's RD cost. Otherwise, only DCT-2 transformation is performed on the last sub-CU, and the value obtained after subsequent calculation is used as its RD cost.

[0009] (3) Calculate the sum of the RD costs of all child CUs. If the sum is greater than the RD cost of the parent CU, then the CU is not split.

[0010] (4) Based on the encoding information in the neighboring CUs of the current CU, reorder its MTS candidate sequences to terminate its MTS process in advance.

[0011] In process (2), the entropy information contained in the sub-CU is calculated using formula (1).

[0012]

[0013] Where P(m) represents the probability of element m appearing in sub-CU, and n represents the total number of elements contained in sub-CU.

[0014] In process (2), formula (2) is used to calculate the similarity between two sub-CUs under the binary tree segmentation.

[0015]

[0016] Where H1 and H2 represent the information entropy contained in the last sub-CU and the previous sub-CU respectively, and S represents the similarity between the two sub-CUs. The closer the value is to 1, the higher the similarity. According to a large number of experimental statistical analyses, when 0.9≤S≤1.1, the two sub-CUs have strong similarity.

[0017] In process (3), when the current CU is a binary tree segment and the two sub-CUs are highly similar, the Split_flag flag that determines whether to segment is obtained using formula (3).

[0018]

[0019] Among them, RD p ,RD i represents the RD cost of the parent CU and the i-th child CU, respectively, and N is the total number of child CUs contained in the parent CU under the current partitioning mode.

[0020] In other cases in process (3), the Split_flag that determines whether to split is obtained using formula (4).

[0021]

[0022] Among them, RD dct The RD cost is the result of the DCT-2 transformation of the last sub-CU.

[0023] The advantages and beneficial technical effects of this invention compared with the prior art are as follows:

[0024] (1) The intra-frame fast MTS method based on multi-core transformation and neighborhood coding information proposed in this invention, compared with the traditional VVC standard coding, significantly reduces the intra-frame prediction coding time complexity of the intra-frame prediction coding algorithm without significantly reducing the intra-frame coding performance.

[0025] (2) The intra-frame fast MTS method based on multi-core transform and neighborhood coding information proposed in this invention fully considers different CU partitioning situations and the correlation between sub-CUs when estimating the RD cost of the last sub-CU. For the case where the two sub-CUs are highly similar under binary tree partitioning, the product of the former's RD cost and information entropy is directly used as the estimated value of the latter's RD cost. For other cases besides quadtree partitioning, the relationship between the RD cost of DCT transform and MST is used to estimate the RD cost value of the last sub-CU, and the relationship between the RD cost of the sub-CU and the RD cost of the parent CU is used to determine whether the current CU needs to be partitioned. For cases where partitioning is not required, the proposed method directly skips the subsequent MTS process, reducing the coding computation complexity.

[0026] (3) The MTS candidate list of the current CU is reordered by using the coding information of the neighboring CUs of the current CU, so as to further reduce the coding time of VVC intra-frame prediction. Attached Figure Description

[0027] Figure 1 This is the overall flowchart of the VVC intra-prediction optimization method based on inter-block correlation.

[0028] Figure 2 This is a diagram showing the positional relationship between the current CU and its neighboring CUs.

[0029] Figure 3 Rate-distortion curves of the PeopleOnStreet sequence based on the method of this invention and VVC under AI configuration.

[0030] Figure 4 Rate-distortion curves of the sequence RaceHorsesC based on the method of this invention and VVC under AI configuration. Detailed Implementation

[0031] The present invention will be further described in detail below with reference to the embodiments. It should be noted that the following embodiments are only used to further illustrate the present invention and should not be construed as limiting the scope of protection of the present invention. Those skilled in the art can make some non-essential improvements and adjustments to the present invention based on the above-described invention, and these improvements and adjustments should still fall within the scope of protection of the present invention.

[0032] (1) The method proposed in this invention is implemented on the VVC standard test code VTM-9.1 platform. The configuration file is encoder_intra_main.cfg, the encoding structure is full I-frame encoding, and the quantization parameter QP is set to 22, 27, 32, and 37.

[0033] (2) The test sequences used to verify the coding performance of the proposed method were five types of official standard test sequences with resolutions of 416×240, 832×480, 1280×720, 1920×1080, and 2560×1600. Specifically, the standard test sequences used were BasketballPass, BlowingBubbles, BQSquare, RaceHorses, BasketballDrill, BQMall, PartyScene, RaceHorsesC, BasketballDrillText, FourPeople, Johnny, KristenAndSara, BasketballDrive, BQTerrace, Cactus, Traffic, and PeopleOnStreet;

[0034] (3) In the program of the algorithm proposed in this invention, the parameters in (1) are set in the configuration file encoder_intra_main.cfg, and then all the video standard sequences to be tested are input to obtain the results of the method proposed in this invention under the case of full I frame, and the encoding time, bit rate and video quality are recorded and statistically analyzed.

[0035] (4) In the VVC standard algorithm program, set the parameters in (1) in the configuration file encoder_intra_main.cfg, then input all the video standard sequences that need to be tested, obtain the results of the VVC standard algorithm in the case of full I frames, and record and count the encoding time, bit rate and video quality.

[0036] (5) The results of the above two types are processed respectively. The coding time complexity is measured by ΔT, which represents the coding time reduction of the proposed method relative to VVC. The quality of coding performance is measured by the objective evaluation standards BDBR and BDPSNR.

[0037] (6) As can be seen from Table 1, compared with the VVC standard algorithm, when the coding structure is all I-frames, the method proposed in this invention reduces the coding time by an average of 26.4%, while BDBR only increases by 0.13% and BDPSNR only decreases by 0.007dB. In summary, while ensuring the coding efficiency of intra-frame prediction, the intra-frame fast CU partitioning method proposed in this invention can effectively reduce the coding time compared with the VVC standard algorithm;

[0038] (7) Figure 3 , Figure 4 Rate-distortion curves for test sequences at different resolutions are shown. The horizontal axis represents the coding bit rate in Kbps, and the vertical axis represents the PSNR in dB. Figure 3 and Figure 4 As can be seen from the above, the rate-distortion curve of the method proposed in this invention basically coincides with the rate-distortion curve of the VVC standard algorithm while effectively reducing the coding time of intra-frame prediction. This indicates that the compression coding efficiency of this method is not reduced at all.

[0039] Table 1 shows the experimental results of this method when the coding structure is full I-frame.

[0040]

Claims

1. A fast intra-frame MTS method based on multi-kernel transform and neighborhood coding information, characterized in that: (1) The similarity between the two is measured by the ratio of the information entropy contained in the last sub-CU and the previous sub-CU under the binary tree partition. (2) For two sub-CUs with strong similarity under binary tree partitioning, the RD cost of the former and the ratio of their entropies are combined to predict the RD cost of the latter. For other cases besides quadtree partitioning, the relationship between RD cost in DCT-2 and MTS transformation is used to predict the RD cost of the last sub-CU. (3) Use the relationship between the sum of the RD costs of the child CU and the RD cost of the parent CU to determine whether the current CU needs to be partitioned. If it does not need to be partitioned, the MTS process can be skipped directly. (4) Use the coding information in the neighboring CUs of the current CU to rearrange the MTS candidate list of the current CU, thereby affecting the subsequent RD check process and reducing the intra-frame coding time.

2. The intra-frame fast MTS method based on multi-kernel transform and neighborhood coding information as described in claim 1, characterized in that... For different CU partitioning scenarios, the correlation between sub-CUs and the relationship between the RD costs of each transformation are used to predict the RD cost of the last sub-CU. Specifically, if the current CU is partitioned into a binary tree and the ratio of the information entropy of the next sub-CU to that of the previous sub-CU is within the range of 0.9 to 1.1, then the product of the RD cost of the previous sub-CU and the ratio of their information entropies is directly used as the predicted value of the RD cost of the next sub-CU. Otherwise, a DCT-2 transformation is performed on the last sub-CU and the subsequent RD cost is used as its predicted value. Finally, the relationship between the sum of the RD costs of the sub-CUs and the RD cost of the parent CU is used to determine whether the current CU should be partitioned. Specifically, if the sum of the RD costs of the sub-CUs is less than the RD cost of the parent CU, then the current CU needs to be partitioned; otherwise, the current CU does not need to be partitioned.

3. The intra-frame fast MTS method based on multi-kernel transform and neighborhood coding information as described in claim 1, characterized in that... The frequency of each transform used in the neighboring CUs of the current CU is counted, and the MTS candidate list is reordered according to the frequency of each transform from high to low (the unused transform set is placed after the used transform set in the original order).

Images