A method for determining TU partition depth

Abstract

The present invention provides a method for determining the TU division depth, which analyzes the characteristics of the prediction residual data, recursively upward from 4x4 blocks, and according to the principle of merging TUs based on local complexity consistency, determines the TU division depth corresponding to the current CU, and performs TU division on the area where the local complexity of the residual data is inconsistent. The method acts on the TU division with residuals after inter-frame prediction, and forcibly performs TU division on the CU with inconsistent local complexity of the residual data, and obtains the TU division depth by recursively merging a quadtree from bottom to top; the whole process is divided into three parts: analyzing residual characteristics, obtaining sub-block merge conditions, and determining TU depth. The present invention determines the TU division depth by the consistency between the local complexity of the inter-frame prediction residual and the overall complexity. It can well identify the situation of inconsistent local fluctuations, while the situation of small overall fluctuations and large overall fluctuations is not affected.

Description

Technical Field

[0001] The present invention relates to the technical field of video coding, and in particular to a method for determining TU division depth. Background Art

[0002] With the development of the times, the application of video coding technology has become more and more common. In the prior art, when deciding the TU division depth, the rate-distortion optimization criterion is mainly used to determine it. It is necessary to first perform the transformation-quantization-inverse quantization-inverse transformation reconstruction process, calculate the difference between the reconstructed block and the original pixel, and other calculation processes before selecting the TU division depth.

[0003] According to the local variation characteristics of the prediction residual, the large-block TU mode can better concentrate the energy, and the small-block TU mode can preserve more image details. This flexible segmentation structure can fully compress the transformed residual energy and improve the coding gain.

[0004] In particular, paper attacks are one of the most commonly used and convenient attack methods, and the attack cost is also low, so it has become a common concern. Practice has proved that visible light liveness detection cannot absolutely distinguish between live and non-live objects by relying on one method alone, and multiple methods need to be used in combination to achieve the best effect.

[0005] However, the existing technology only considers the optimal overall objective data of TU division. In actual market applications, some areas with excessive flatness and complex details may be judged as larger TU sizes under the optimal objective data judgment. Such areas are prone to high-frequency diffusion noise-like phenomena, which affects the subjective perception of the video image.

[0006] Common technical terms in the prior art include:

[0007] In the H265 / HEVC protocol, the transform unit TU is a basic unit that independently completes transformation and quantization. It achieves energy concentration through transformation from time domain to frequency domain to improve compression gain. Its size can be flexibly changed, supporting four transform sizes of 4x4, 8x8, 16x16, and 32x32. The size of TU depends on the coding unit CU, and the size of TU cannot be larger than the CU size. Larger TUs are used in flat areas and areas with slowly changing content, and smaller TUs are used in areas with texture details. The optimal division method can be determined by the rate-distortion optimization criterion.

[0008] A quadtree is a tree structure with four sub-blocks on each node.

[0009] The quadtree recursion is that each node can be further divided into four sub-blocks until it cannot be divided anymore.

[0010] The recursive merging of quadtrees from bottom to top is the process of merging four sub-regions that meet the conditions into one parent block until the conditions for merging are no longer met. Summary of the invention

[0011] In order to solve the above technical problems, the purpose of the present invention is to determine the TU division depth by the consistency between the local complexity and the overall complexity of the inter-frame prediction residual. TU division is performed on the area where the local complexity of the residual data is inconsistent to reduce the noise-like phenomenon formed by the high frequency of the complex content in the area diffusing to the flat area; and improve the subjective perception of the video.

[0012] Specifically, the present invention provides a method for determining the TU division depth, which analyzes the characteristics of the predicted residual data, recursively upward from 4x4 blocks, and based on the principle of merging TUs based on local complexity consistency, determines the TU division depth corresponding to the current CU, and performs TU division on the area where the local complexity of the residual data is inconsistent.

[0013] The method is applied to TU partition with residual after inter-frame prediction, and for CU with inconsistent local complexity of residual data, TU partition is forced, and the TU partition depth is obtained by recursive merging of quadtree from bottom to top; the whole process is divided into three parts: analyzing residual characteristics, obtaining sub-block merge conditions, and judging TU depth.

[0014] The method further comprises the following steps:

[0015] S1, analyze the residual characteristics: take the residual data fluctuation value sad as the complexity measurement condition: calculate the residual data fluctuation value sad, the maximum and minimum deviation values ​​diff, and the data average value avg for the sub-blocks divided by the quadtree as the conditions for judging whether the local data complexity of the block is consistent;

[0016] S1.1, calculate the sad, diff, and avg of the four sub-blocks respectively, and determine the local complexity:

[0017]

[0018] diff=res max -resi min ;

[0019]

[0020] Among them, resi represents residual data;

[0021] S1.2, calculate the sad_sad, diff_sad, and avg_sad of the parent block to determine the consistency between the local and overall complexity:

[0022]

[0023]

[0024]

[0025] sad_min=MIN(sad0,sad1,sad2,sad3);

[0026] S2, obtain the conditions for merging sub-blocks, and merge sub-blocks with the same local and overall complexity upwards:

[0027] S2.1, set threshold:

[0028] sad_sad_min_thr=12;

[0029] diff_sad_min_thr=15;

[0030] avg_sad_min_thr = 10;

[0031] max_sad_thr = 60;

[0032] S2.2, the data results of the above analysis are sad_sad, diff_sad, avg_sad, sad_min

[0033] Compare with the given threshold above to obtain the mark of the consistency of the overall and local complexity of the residual data;

[0034] flag sad =sad_sad <sad_sad_min_thrd;

[0035] flag diff =diff_sad <diff_sad_min_thrd;

[0036] flag avg =avg_sad <avg_sad_min_thrd;

[0037] flag cplx =sad_min>sad_max_thrd;

[0038] S2.3, according to the above conditions, mark whether the four sub-blocks of the current block have the same complexity:

[0039] merge_flag=(flag sad &&flag diff &&flag avg )||flagcplx ;

[0040] S3, determine TU depth:

[0041] S3.1, store the TU partition depth to be used by the current CU in tu_depth[N4], where N4 represents the number of 4x4 partitions that can be used in the CU;

[0042] S3.2, recursively divide the residual data of the current CU in a quadtree manner until it is divided into 4x4, and determine whether the sub-block is merged upward by the method in step S2;

[0043] S3.3, if merge, the sub-block is assigned a value in the corresponding storage area of ​​tu_depth[] by looking up the block size table; the TU partition depth result of the current CU is obtained recursively in this way.

[0044] In step S1, the larger the sad fluctuation value is, the more unfavorable it is for the transformation to concentrate the residual data energy. For blocks with large local fluctuations and small overall fluctuations, objective data judges that the TU is not divided.

[0045] In summary, the advantages that can be achieved by applying the method of the present application are: the present invention determines the TU division depth by the consistency between the local complexity and the overall complexity of the inter-frame prediction residual, and can well identify the situation of inconsistent local fluctuations, while the situation of small overall fluctuations and large overall fluctuations is not affected. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0047] Figure 1 is a schematic diagram of the process of this method.

[0048] Figure 2 It is a further flow chart of step S1 of the method. DETAILED DESCRIPTION

[0049] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0050] The present invention mainly acts on TU partitioning with residuals after inter-frame prediction. For CUs with inconsistent local complexity of residual data, TU partitioning is forced, and the TU partitioning depth is obtained by recursive merging of quadtrees from bottom to top. The whole process is divided into three parts: analyzing residual characteristics, obtaining sub-block merge conditions, and determining TU depth.

[0051] 1. Analyze residual characteristics

[0052] In the present invention, the fluctuation value sad of the residual data is used as a complexity measurement condition. The larger the sad fluctuation value, the more unfavorable it is for the transformation to concentrate the residual data energy. For blocks with large local fluctuations and small overall fluctuations, it is easy to be judged by objective data as TU undivided. The present invention can well identify the situation of inconsistent local fluctuations, while the situations of small overall fluctuations and large overall fluctuations are not affected.

[0053] The residual data fluctuation value sad, the maximum and minimum deviation values ​​diff, and the data average value avg are calculated for the sub-blocks divided by the quadtree, and are used as conditions for determining whether the local data complexity of the block is consistent.

[0054] 1) Calculate the sad, diff, and avg of the four sub-blocks respectively to determine the local complexity.

[0055]

[0056] diff=res max -resi min

[0057]

[0058] resi represents residual data

[0059] 2) Calculate the sad_sad, diff_sad, and avg_sad of the parent block to determine the consistency between the local and overall complexity.

[0060]

[0061]

[0062]

[0063] sad_min=MIN(sad0,sad1,sad2,sad3)

[0064] 2. Obtain the conditions for merging sub-blocks, and set the threshold for merging upwards sub-blocks with consistent local and overall complexity:

[0065] sad_sad_min_thr=12;

[0066] diff_sad_min_thr=15;

[0067] avg_sad_min_thr = 10;

[0068] max_sad_thr = 60;

[0069] Compare the data results sad_sad, diff_sad, avg_sad, and sad_min analyzed above with the given thresholds above to obtain a mark of the consistency of the overall and local complexity of the residual data.

[0070] flag sad =sad_sad <sad_sad_min_thrd

[0071] flag diff =diff_sad <diff_sad_min_thrd

[0072] flag avg =avg_sad <avg_sad_min_thrd

[0073] flag cplx =sad_min>sad_max_thrd

[0074] According to the above conditions, mark whether the four sub-blocks of the current block have the same complexity.

[0075] merge_flag=(flag sad &&flag diff &&flag avg )||flag cplx

[0076] 3. Determine TU depth

[0077] The TU division depth that the current CU will use is stored in tu_depth[N4], where N4 represents the number of 4x4 partitions that can be divided in the CU. The residual data of the current CU is recursively divided in a quadtree manner until it is divided into 4x4, and the method in step 2 is used to determine whether the sub-block is merged upward. If merged, the sub-block is assigned a value by looking up the block size table in the corresponding storage area of ​​tu_depth[]. In this way, the TU division depth result of the current CU is obtained recursively. Taking CU32 as an example, its corresponding TU division depth can be shown in the following table:

[0078] blk_8x8 blk_16x16 blk_32x32 blk_64x64 merge 3 2 1 1 Non-merge 4 3 2 1

[0079] TU Depth Chart

[0080]

[0081] TU partition depth result corresponding to CU

[0082] When performing CU transform coding, if the CU depth is less than the TU depth, a TU recursive partitioning transformation is performed until the CU sub-residual block depth is equal to the TU depth, thereby achieving the purpose of controlling TU partitioning.

[0083] Further systematization is as follows: Figure 1 As shown, the method of the present invention comprises the following steps:

[0084] S1, analyze the residual characteristics: take the residual data fluctuation value sad as the complexity measurement condition: calculate the residual data fluctuation value sad, the maximum and minimum deviation values ​​diff, and the data average value avg for the sub-blocks divided by the quadtree as the conditions for judging whether the local data complexity of the block is consistent;

[0085] like Figure 2 As shown, S1.1 calculates the sad, diff, and avg of the four sub-blocks respectively to determine the local complexity:

[0086]

[0087] diff=res max -resi min ;

[0088]

[0089] Among them, resi represents residual data;

[0090] S1.2, calculate the sad_sad, diff_sad, and avg_sad of the parent block to determine the consistency between the local and overall complexity:

[0091]

[0092]

[0093]

[0094] sad_min=MIN(sad0,sad1,sad2,sad3);

[0095] S2, obtain the conditions for merging sub-blocks, and merge sub-blocks with consistent local and overall complexity upwards: S2.1, set the threshold:

[0096] sad_sad_min_thr=12;

[0097] diff_sad_min_thr=15;

[0098] avg_sad_min_thr = 10;

[0099] max_sad_thr = 60;

[0100] S2.2, compare the data results sad_sad, diff_sad, avg_sad, sad_min analyzed above with the given thresholds above to obtain the mark of the consistency of the overall and local complexity of the residual data;

[0101] flag sad =sad_sad <sad_sad_min_thrd;

[0102] flag diff =diff_sad <diff_sad_min_thrd;

[0103] flag avg =avg_sad <avg_sad_min_thrd;

[0104] flag cplx =sad_min>sad_max_thrd;

[0105] S2.3, according to the above conditions, mark whether the four sub-blocks of the current block have the same complexity: merge_flag = (flag sad &&flag diff &&flag avg )||flag cplx ;

[0106] S3, determine TU depth:

[0107] S3.1, store the TU partition depth to be used by the current CU in tu_depth[N4], where N4 represents the number of 4x4 partitions that can be used in the CU;

[0108] S3.2, recursively divide the residual data of the current CU in a quadtree manner until it is divided into 4x4, and determine whether the sub-block is merged upward by the method in step S2;

[0109] S3.3, if merge, the sub-block is assigned a value in the corresponding storage area of ​​tu_depth[] by looking up the block size table; the TU partition depth result of the current CU is obtained recursively in this way.

[0110] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for determining TU partition depth, characterized in that: The method analyzes the characteristics of the prediction residual data, adopts a recursive method upward from 4x4 blocks, and according to the principle of merging TUs based on local complexity consistency, determines the TU division depth corresponding to the current CU, and performs TU division on the area where the local complexity of the residual data is inconsistent. The method acts on the TU division with residuals after inter-frame prediction, and forcibly performs TU division on the CU with inconsistent local complexity of the residual data, and obtains the TU division depth by recursively merging the quadtree from bottom to top; the whole process is divided into three parts: analyzing the residual characteristics, obtaining the sub-block merge conditions, and determining the TU depth; the method further includes the following steps: S1, analyze the residual characteristics: take the residual data fluctuation value sad as the complexity measurement condition; calculate the residual data fluctuation value sad, the maximum and minimum deviation values ​​diff, and the data average value avg for the sub-blocks divided by the quadtree as the conditions for judging whether the local data complexity of the block is consistent; S1.1, calculate the sad, diff, and avg of the four sub-blocks respectively, and determine the local complexity: ； ; ; Among them, resi represents residual data, N=4; S1.2, calculate the sad_sad, diff_sad, and avg_sad of the parent block to determine the consistency between the local and overall complexity: ； ； ； ； S2, obtain the conditions for merging sub-blocks, and merge sub-blocks with the same local and overall complexity upwards: S2.1, set threshold: sad_sad_min_thrd=12; diff_sad_min_thrd=15; avg_sad_min_thrd=10; max_sad_thrd=60; S2.2, compare the data results sad_sad, diff_sad, avg_sad, sad_min analyzed above with the given thresholds above to obtain the mark of the consistency of the overall and local complexity of the residual data; ； ； ； ； S2.3, mark whether the four sub-blocks of the current block have the same complexity according to the above conditions: ； S3, determine TU depth: S3.1, store the TU partition depth to be used by the current CU in tu_depth[N4], where N4 represents the number of 4x4 partitions that can be used in the CU; S3.2, recursively divide the residual data of the current CU in a quadtree manner until it is divided into 4x4, and determine whether the sub-block is merged upward by the method in step S2; S3.3, if merge, the sub-block is assigned a value in the corresponding storage area of ​​tu_depth[] by looking up the block size table; the TU partition depth result of the current CU is obtained recursively in this way.

2. The method for determining TU division depth according to claim 1, characterized in that: In step S1, the larger the sad fluctuation value is, the more unfavorable it is for the transformation to concentrate the residual data energy. For blocks with large local fluctuations and small overall fluctuations, objective data judges that the TU is not divided.

3. The method for determining TU division depth according to claim 1, characterized in that: In step S3, when performing CU transform coding, if the CU depth is less than the TU depth, a TU recursive division transform is performed until the CU sub-residual block depth is equal to the TU depth, thereby achieving the purpose of controlling TU division.

4. The method for determining TU division depth according to claim 1, characterized in that: The CU is CU32, and its corresponding TU division depth can be shown in the following table: ； The TU division depth results corresponding to the CU are as follows: 。

Images