A decision method for forced partitioning of coding units

Abstract

This invention provides a method for forcibly partitioning coding units (CUs). The method analyzes the characteristics of the predicted residual data, uses a quadtree, and identifies the consistency of detail richness of high-frequency coefficients in the four sub-blocks after Hadamard transform of the residual data predicted by the coding unit (CU). This determines whether the current CU block size will cause detail loss or detail spread. For CUs with complex residual data (i.e., inconsistent detail richness in the quadtree-partitioned sub-blocks), forced partitioning is performed, and over-partitioning is corrected. This method supplements and corrects the CU decision-making process for pass rate distortion optimization, significantly improving the effects of detail loss and detail spread.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of video coding, in particular to a method for determining the forced partition of a coding unit. BACKGROUND

[0002] In H265 / HEVC protocol, a coding unit CU is composed of a luma CB and corresponding chroma CB and their related syntax elements. The size of CU is variable, from large to small, 64x64, 32x32, 16x16, 8x8. Large CU can greatly improve the coding efficiency of flat area, while small CU can well handle the details of local image. In addition, a quadtree is often used in the analysis and classification of spatial data, which is a tree structure, and each node has four sub-blocks. Hadamard transform is a common feature transform in image automatic classification, which is a process of transforming time domain data to frequency domain using Hadamard matrix as the transform matrix. Rate-distortion model is an important content of Shannon information theory, which describes the relationship between encoder output code rate R, distortion D, quantization step QP and image content. Its basic idea is that reducing distortion will increase code rate, and reducing code rate will increase distortion.

[0003] The conventional CU partition is determined by comparing the rate-distortion cost of the CU and the four sub-blocks, and the rate-distortion cost is calculated using the following formula:

[0004] J = D + λ·R

[0005] Where D is the sum of the square of the difference between the reconstructed image and the original image, R is the code stream length after the predicted residual is transformed, quantized and entropy encoded, and λ is the Lagrange multiplier.

[0006] The conventional CU partition is determined by selecting the partition mode with the minimum rate-distortion cost. The minimum rate-distortion cost refers to obtaining the minimum distortion under a given code rate or obtaining the minimum code rate within the allowable distortion range, which is a trade-off between distortion and code rate. Generally, reducing distortion corresponds to increasing code rate, and vice versa.

[0007] The conventional CU partition is mainly determined according to the minimum objective data rate-distortion cost. However, in actual monitoring market applications, the rate-distortion cost as an objective factor for CU partition evaluation is applied to some areas where the human eye is sensitive to image distortion, and the market response is poor. For areas where the image distortion is not sensitive, increasing the distortion does not have much market response. Since the conventional CU partition technology is increasingly concerned about the impact of subjective perception in image detail retention and detail diffusion, relying solely on the rate-distortion cost as an objective data for CU partition is not fully applicable to market demand, and visual perception technology is urgently needed to participate in CU partition and other coding technologies. SUMMARY

[0008] To address the aforementioned issues, this method aims to analyze the characteristics of the predicted residual data, focusing on identifying the consistency of detail richness of the high-frequency coefficients in the four sub-blocks after Hadamard transformation. This will help determine whether the current CU block size will cause detail loss or detail diffusion, and in such cases, a forced CU partitioning approach will be adopted.

[0009] Specifically, the present invention provides a method for determining the forced partitioning of coding units. The method analyzes the characteristics of the predicted residual data and identifies the consistency of the detail richness of the high-frequency coefficients of the residual data after the Hadamard transform of the four sub-blocks. This determines whether the current CU block size will cause detail loss or detail diffusion. The method performs forced partitioning for CUs with complex residual data, i.e., inconsistent detail richness of the quadtree partitioned sub-blocks, and corrects over-partitioning.

[0010] The method includes the following steps:

[0011] S1, Residual Characteristic Analysis and Eigenvalue Calculation:

[0012] The residual data resi obtained after prediction by the coding unit CU is divided into quadtrees, and Hadamard transform is performed on the four sub-blocks to obtain the transform coefficients satd_coef[sub_resi_idx][N*N]. The absolute error and sad, the maximum and minimum high-frequency coefficient deviation diff, and the average high-frequency coefficient avg of the high-frequency coefficients after transformation of each sub-block are calculated.

[0013] Further calculate the fluctuation value sad_sad of sad for the four sub-blocks, the fluctuation value diff_sad of the maximum and minimum deviations, and the fluctuation value avg_sad of the average value, where N represents the width and height of the residual sub-block, and sub_resi_idx represents the index value of the residual sub-block.

[0014] S2, perform CU-forced partitioning on sub-blocks with inconsistent detail richness in residual sub-blocks; otherwise, pass the rate-distortion optimization decision.

[0015] Set threshold:

[0016] sad_sad_thrd = 12;

[0017] diff_sad_thrd = 15;

[0018] avg_sad_thrd = 10;

[0019] The feature value results obtained in step S1 are compared with the set threshold to obtain the consistency marker of the feature value fluctuation of each of the four sub-blocks:

[0020] flg sad =sad_sad<sad_sad_thrd

[0021] flag diff =diff_sad<diff_sad_thrd

[0022] flag avg =avg_sad<avg_sad_thrd

[0023] Based on the above conditions, determine whether the four sub-blocks of the current block are blocks with consistent residual detail richness; if the residual feature values ​​are all within the given threshold, it is considered that the high-frequency information of the residual data has high consistency and no forced partitioning is required; otherwise, forced partitioning is required.

[0024] The CU forced partitioning flag, split_flag, can be obtained from the following formula:

[0025] split_flag = ~(flag) sad &&flag diff &&flag avg );

[0026] S3, Correction for over-partitioning caused by step S2:

[0027] For CU blocks in the original image raw[sub_idx][N*N] that are complex and satisfy the forced segmentation condition split_flag in step S2, there will be no noticeable phenomenon of detail diffusion and loss. Therefore, split_flag needs to be modified. That is, when flagcplx is true, forced segmentation is not performed, and split_flag is set to 0.

[0028] In step S1

[0029] Average value of high-frequency coefficients of residual sub-blocks:

[0030] High-frequency coefficient fluctuation values ​​of residual sub-blocks:

[0031] Maximum deviation of high-frequency coefficients in residual sub-blocks:

[0032] diff j =satd_coef_max j -satd_coef_min j j = 0, 1, 2, 3;

[0033] Among them, satd_coef_maxj and satd_coef_minj are the maximum and minimum values ​​of the high-frequency coefficients of the respective residual sub-block;

[0034] avg j The average high-frequency detail intensity of the j-th sub-block describes the amount of detail carried by the residual data; sad j The diff represents the fluctuation of the high-frequency coefficients in the j-th sub-block, describing the degree of detail balance; j Characterize the strongest and weakest detail deviations of the 4th sub-block, describing the strength of the detail deviations;

[0035] Fluctuation value of the fluctuation:

[0036] The fluctuation value of the maximum deviation:

[0037] Fluctuation of the average value:

[0038] The larger the value of the fluctuation eigenvalue, the greater the difference in detail richness; the more consistent the eigenvalues ​​of the four sub-blocks, that is, the smaller the eigenvalues, the higher the consistency of the residual detail distribution, and the less inclined to the forced partitioning of CU.

[0039] In step S3

[0040] Average value of sub-blocks in the original CU image:

[0041] Fluctuations in sub-blocks of the original CU image:

[0042]

[0043] Take the smallest raw_sad among the 4 sub-blocks j Let it be denoted as sad_min, when the fluctuation value is raw_sad j When the smallest sub-block deviation exceeds a certain threshold sad_max_thrd, let's set the threshold sad_max_thrd = 32. Mark the current CU as a content-complex block.

[0044] Is the raw image data a complex block? (flag) cplx =sad_min>sad_max_thrd.

[0045] In the method, the sizes of the encoding unit CU, prediction unit PU, and transformation unit TU are set to be equal.

[0046] The method is applicable to the H.265 / HEVC protocol.

[0047] Therefore, the beneficial effects of this application are as follows: the present invention plays a supplementary and corrective role in the CU decision of pass rate distortion optimization, and has a significant improvement effect on the phenomena of detail loss and detail diffusion. Attached Figure Description

[0048] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0049] Figure 1 This is a flowchart illustrating the method of the present invention. Detailed Implementation

[0050] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0051] This invention is mainly used to determine the CU (Code Encoding Unit) partitioning of I-frames and analyze the complexity of the data to be encoded from the perspective of residuals. Forced partitioning is performed on CUs with complex residual data, i.e., inconsistent detail richness in the quadtree sub-blocks. The entire process is divided into three parts: residual characteristic analysis, partitioning condition determination, and over-partition correction. The sizes of the coding unit (CU), prediction unit (PU), and transform unit (TU) are set to be equal.

[0052] This invention relates to a method for determining mandatory partitioning of coding units. The method is applicable to the H.265 / HEVC protocol. By analyzing the characteristics of the predicted residual data, the method identifies the consistency of detail richness of the high-frequency coefficients of the residual data after prediction by the coding unit (CU) in the four sub-blocks after Hadamard transform. This determines whether the current CU block size will cause detail loss or detail diffusion. For CUs with complex residual data, i.e., inconsistent detail richness in the quadtree-partitioned sub-blocks, mandatory partitioning is performed, and over-partitioning is corrected.

[0053] Specifically, such as Figure 1 As shown below:

[0054] S1. Residual property analysis and eigenvalue calculation:

[0055] The residual data `resi` obtained after CU prediction is partitioned into a quadtree, and a Hadamard transform is performed on each of the four sub-blocks to obtain the transform coefficients `satd_coef[sub_resi_idx][N*N]`. The `sad`, maximum and minimum high-frequency coefficient deviations `diff`, and average high-frequency coefficients `avg` are calculated for each sub-block. Further calculations are then performed on the fluctuation values ​​of `sad_sad`, `diff_sad` (maximum and minimum deviation fluctuations), and `avg_sad` (average fluctuations). Here, `N` represents the width and height of the residual sub-block, and `sub_resi_idx` represents the index of the residual sub-block.

[0056] Average value of high-frequency coefficients of residual sub-blocks:

[0057] High-frequency coefficient fluctuation values ​​of residual sub-blocks:

[0058] Maximum deviation of high-frequency coefficients in residual sub-blocks:

[0059] diff j =satd_coef_max j -satd_coef_min j j = 0, 1, 2, 3

[0060] Among them, satd_coef_max j and satd_coef_min j It is the maximum and minimum value among the high-frequency coefficients of the residual sub-block.

[0061] avgj can represent the average high-frequency detail intensity of the j-th sub-block, describing the amount of detail carried by the residual data; sad j The diff represents the fluctuation of the high-frequency coefficients in the j-th sub-block, describing the degree of detail balance; j Characterizes the strongest and weakest detail deviations of the 4th sub-block, describing the strength of the detail deviation.

[0062] Fluctuation value of the fluctuation:

[0063] The fluctuation value of the maximum deviation:

[0064] Fluctuation of the average value:

[0065] The larger the value of the fluctuation eigenvalue, the greater the difference in detail richness; the more consistent the eigenvalues ​​of the four sub-blocks, that is, the smaller the eigenvalues, the higher the consistency of the residual detail distribution, and the less inclined to the forced partitioning of CU.

[0066] S2. Perform CU-forced partitioning on sub-blocks with inconsistent detail richness in residual sub-blocks; otherwise, pass the rate-distortion optimization decision.

[0067] Set threshold:

[0068] sad_sad_thrd = 12;

[0069] diff_sad_thrd = 15;

[0070] avg_sad_thrd = 10;

[0071] The feature value results obtained in step 1 are compared with the given threshold to obtain the consistency marker of the feature value fluctuation of each of the four sub-blocks.

[0072] flag sad =sad_sad<sad_sad_thrd

[0073] flag diff =diff_sad<diff_sad_thrd

[0074] flag avg =avg_sad<avg_sad_thrd

[0075] Based on the above conditions, determine whether the four sub-blocks of the current block are blocks with consistent residual detail richness. If the residual feature values ​​are all within a given threshold, the high-frequency information of the residual data is considered to have high consistency, and no forced partitioning is required; otherwise, forced partitioning is required. The CU forced partitioning flag `split_flag` can be obtained by the following formula:

[0076] split_flag = ~(flag) sad &&flag diff &&flag avg );

[0077] S3, Correction for over-partitioning caused by step S2:

[0078] For CU blocks in the original image raw[sub_idx][N*N] that are internally complex and satisfy the forced segmentation condition split_flag in step 2, there will be no noticeable phenomenon of detail diffusion and loss of detail. Therefore, split_flag needs to be modified. That is, when flagcplx is true, forced segmentation is not performed, and split_flag is set to 0.

[0079] Average value of sub-blocks in the original CU image:

[0080] Fluctuations in sub-blocks of the original CU image:

[0081]

[0082] Take the smallest raw_sad among the 4 sub-blocks j Let it be denoted as sad_min, when the fluctuation value is raw_sad j When the smallest sub-block deviation is greater than a certain threshold sad_max_thrd, the threshold sad_max_thrd is set to 32, and the current CU is marked as a content complex block.

[0083] Is the raw image data a complex block? (flag) cplx =sad_min>sad_max_thrd.

[0084] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A decision method of coding unit forced partitioning, characterized in that, The method judges whether the current CU block size will cause detail loss or detail diffusion phenomenon by analyzing the predicted residual data characteristics, identifying the consistency degree of the detail richness of the high frequency coefficients of the four sub-blocks after Hadamard transform of the residual data of the coding unit (CU) after prediction, and forcibly dividing the CU for the residual data complex, i.e., the detail richness of the four sub-blocks is inconsistent, and correcting the over-division. S1, residual characteristic analysis and characteristic value calculation: The residual data resi obtained after prediction of the coding unit (CU) is quad-tree divided, and Hadamard transform is performed on the four sub-blocks to obtain the transform coefficients satd_coef[sub_resi_idx][N*N]; and the absolute error sum sad, the maximum minimum high frequency coefficient deviation diff, and the average value avg of the high frequency coefficients of each sub-block after transform are calculated. Further, the fluctuation value sad_sad of the sad of the four sub-blocks, the fluctuation value diff_sad of the maximum minimum deviation, and the fluctuation value avg_sad of the average value are calculated, wherein N represents the width and height value of the residual sub-block, and sub_resi_idx represents the index value of the residual sub-block. S2, CU forced division is performed on the sub-blocks with inconsistent residual sub-block detail richness, otherwise rate-distortion optimization decision is made: The threshold values are set as follows: sad_sad_thrd=12; diff_sad_thrd=15; avg_sad_thrd=10; The characteristic value results obtained in step S1 are compared with the set threshold values to obtain the fluctuation consistency flags of the characteristic values of the four sub-blocks: flag sad = sad_sad < sad_sad_thrd flag diff = diff_sad < diff_sad_thrd flag avg = avg_sad < avg_sad_thrd According to the above conditions, it is judged whether the four sub-blocks of the current block are blocks with consistent residual detail richness; if the residual characteristic values are within the given threshold values, it is considered that the high frequency information consistency of the residual data is high, and forced division is not needed; otherwise, forced division is needed. The CU forced division flag split_flag can be obtained by the following formula: split_flag = ~ (flag sad && flag diff && flag avg ) ; S3, correction of over-division caused by step S2: For the CU block in the original image raw[sub_idx][N*N] with complex content and satisfying the forced split condition split_flag in step S2, no noticeable phenomenon of detail diffusion and detail loss will occur, and split_flag needs to be modified, i.e., when the complex content block flag cplx is true, no forced split is performed, and split_flag is set to 0.

2. The decision method for forced partitioning of coding units according to claim 1, characterized in that, In the step S1, residual sub-block high frequency coefficient average value: Residual sub-block high frequency coefficient fluctuation value: The maximum deviation value of the high frequency coefficients of the residual sub-block: diff j = satd_coef_max j - satd_coef_min j j = 0, 1, 2, 3; where satd_coef_max j and satd_coef_min j are the maximum minimum values among the high frequency coefficients of the respective residual sub-blocks; avg j represents the average high-frequency detail intensity of the jth sub-block, which describes the amount of details carried by the residual data; sad j represents the fluctuation of high-frequency coefficients of the jth sub-block, which describes the balance degree of details; diff j represents the strongest and weakest detail deviation of the 4th sub-block, which describes the strength and weakness of the detail deviation; fluctuation value of the fluctuation: Maximum deviation from volatility: fluctuation value of the average value: The greater the numerical value of the fluctuation characteristic value, the greater the difference in detail richness; the more consistent the characteristic values of the four sub-blocks, i.e., the smaller the characteristic values, the higher the consistency of the residual detail distribution, and the less inclined to forced division of the CU.

3. The decision method for forced partitioning of coding units according to claim 1, characterized in that, In the step S3, Average of CU original picture sub-block: The fluctuation of the CU original image sub-block: taking the minimum raw_sad in the 4 sub-blocks j denoted as sad_min, when the fluctuation value raw_sad j when the minimum sub-block deviation is greater than a certain threshold, marking the current CU as a content complex block; Is the original image data a content complex block: flag cplx = sad_min > sad_max_thrd.

4. The method of Claim 3, wherein The threshold value is sad_max_thrd, and the threshold value sad_max_thrd=32.

5. The decision method for forced partitioning of coding units according to claim 1, characterized in that, In the method, the size of the coding unit (CU), the prediction unit (PU), and the transform unit (TU) is set to be equal.

6. The method of claim 1, wherein the coding unit is forcedly partitioned into the sub-coding units, and the sub-coding units are coded in the order of the sub-coding units from the top to the bottom and from the left to the right. The method is applicable to the H265 / HEVC protocol.

Images