Coding unit division method and device, computer equipment, storage medium and computer program product

Abstract

The invention relates to a coding unit division method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a to-be-processed coding unit, and acquiring a non-division cost of the to-be-processed coding unit; obtaining the texture complexity of the coding unit to be processed; and dividing the to-be-processed coding unit under the conditions that the size of the to-be-processed coding unit is not the target size, the texture complexity is greater than a target texture complexity threshold and the non-division cost is greater than a non-division cost threshold. By adopting the method, the overhead of hardware resources can be reduced, and the coding speed is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a coding unit division method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] The HEVC coding standard stipulates that a CTU (Coding Tree Unit, maximum coding unit) can be divided into several CUs (Coding Unit, coding units).

[0003] HEVC uses a quadtree CU partitioning method. The maximum size of a CTU is 64x64, and the maximum size of a CU is 64x64 and the minimum is 8x8. From a two-dimensional image point of view, it is equivalent to dividing a 64x64 rectangular image block into several smaller rectangular image blocks. The accuracy of CU partitioning determines the basic trend of the encoder's encoding quality. Therefore, how to use a suitable CU partitioning algorithm when hardware resources are limited is a problem worth studying.

[0004] In traditional methods, CU division generally needs to be decided through RDO (Rate Distortion Optimization) cost, but a complete RDO algorithm consumes a lot of hardware resources and encoding time.

[0005] Therefore, if the CU division is determined directly based on the RDO cost, it will have a significant impact on the hardware resource overhead and encoding speed. Summary of the invention

[0006] Based on this, it is necessary to provide a coding unit division method, device, computer equipment, computer-readable storage medium and computer program product that can reduce the overhead of hardware resources and improve the coding speed in order to address the above technical problems.

[0007] In a first aspect, the present application provides a coding unit division method, the method comprising:

[0008] Obtaining a coding unit to be processed, and obtaining a non-division cost of the coding unit to be processed;

[0009] Obtaining the texture complexity of the coding unit to be processed;

[0010] When the coding unit to be processed is not of a target size, the texture complexity is greater than a target texture complexity threshold, and the non-division cost is greater than a non-division cost threshold, the coding unit to be processed is divided.

[0011] In one embodiment, the method further comprises:

[0012] Obtaining a partition cost of the coding unit to be processed;

[0013] In the case where the texture complexity is less than or equal to the target texture complexity threshold or the non-division cost is less than or equal to the target non-division cost threshold, determining whether the division cost is less than the non-division cost;

[0014] When the division cost is less than the non-division cost, the coding unit to be processed is divided.

[0015] In one embodiment, the method further comprises:

[0016] When the coding unit to be processed is of the target size, or the division cost is greater than or equal to the non-division cost, the coding unit to be processed is not divided.

[0017] In one embodiment, the method further comprises:

[0018] Obtaining a target non-division cost threshold based on a fitting result of the quantization parameter value and the non-division cost threshold and a quantization parameter value corresponding to the coding unit to be processed;

[0019] The fitting result is obtained by testing different sample video sequences to obtain the correspondence between different quantization parameter values ​​and the non-division cost threshold, and performing curve fitting based on the correspondence between the different quantization parameter values ​​and the non-division cost threshold.

[0020] In one embodiment, the method further comprises:

[0021] Based on the quantization parameter value corresponding to the coding unit to be processed, a mapping relationship table between the quantization parameter value and the non-division cost threshold is queried to obtain the target non-division cost threshold; the mapping relationship table between the quantization parameter value and the non-division cost threshold is obtained based on the fitting results of the quantization parameter value and the non-division cost threshold and each quantization parameter value; the fitting result is obtained by testing based on different sample video sequences to obtain the correspondence between different quantization parameter values ​​and the non-division cost threshold, and curve fitting is performed based on the correspondence between the different quantization parameter values ​​and the non-division cost threshold.

[0022] In one embodiment, the method further comprises:

[0023] Based on the fitting results of the quantization parameter value and the texture complexity threshold and the quantization parameter value corresponding to the coding unit to be processed, the target texture complexity threshold is obtained; the fitting result is tested based on different sample video sequences to obtain the corresponding relationship between different quantization parameter values ​​and the texture complexity threshold; and based on the corresponding relationship between the different quantization parameter values ​​and the texture complexity threshold, the curve fitting is performed.

[0024] In one embodiment, the method further comprises:

[0025] Based on the quantization parameter value corresponding to the coding unit to be processed, a mapping relationship table between the quantization parameter value and the texture complexity threshold is queried to obtain a target texture complexity threshold; the mapping relationship table between the quantization parameter value and the texture complexity threshold is obtained based on the fitting results of the quantization parameter value and the texture complexity threshold and each quantization parameter value; the fitting results are obtained by testing different sample video sequences to obtain the corresponding relationship between different quantization parameter values ​​and the texture complexity threshold, and curve fitting is performed based on the corresponding relationship between the different quantization parameter values ​​and the texture complexity threshold.

[0026] In one of the embodiments, before obtaining the texture complexity of the coding unit to be processed, the method further includes:

[0027] Obtaining a pixel mean based on the pixel value of each pixel in the coding unit to be processed;

[0028] Obtaining a pixel variance of the coding unit to be processed based on a pixel value of each pixel in the coding unit to be processed and the pixel mean;

[0029] The pixel variance is used as the texture complexity of the coding unit to be processed.

[0030] In one of the embodiments, before obtaining the texture complexity of the coding unit to be processed, the method further includes:

[0031] Obtaining pre-division results of the coding unit to be processed, wherein the pre-division results include coding units with successively increasing sizes;

[0032] Determine an initial coding unit based on the size, and calculate each texture complexity of each initial coding unit;

[0033] Based on the size, the next coding unit is determined as each current coding unit, and based on the texture complexity of each coding unit of the previous size, the texture complexity of each current coding unit is calculated until the texture complexity of each coding unit of the size is obtained.

[0034] In a second aspect, the present application further provides a coding unit division device, the device comprising:

[0035] A coding unit acquisition module, used to acquire a coding unit to be processed, and acquire a non-division cost of the coding unit to be processed;

[0036] A texture complexity acquisition module, used to acquire the texture complexity of the coding unit to be processed;

[0037] The division module is used to divide the coding unit to be processed when the coding unit to be processed is not of a target size, the texture complexity is greater than a target texture complexity threshold, and the non-division cost is greater than a non-division cost threshold.

[0038] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0039] In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0040] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0041] The above-mentioned coding unit division method, device, computer equipment, storage medium and computer program product obtain the coding unit to be processed, and obtain the cost of not dividing the coding unit to be processed; obtain the texture complexity of the coding unit to be processed; when the coding unit to be processed is not the target size, the texture complexity is greater than the target texture complexity threshold and the cost of not dividing is greater than the cost threshold of not dividing, the coding unit to be processed is divided. In this way, there is no need to calculate the division cost of the coding unit to be processed when making judgments, thereby reducing the calculation of the division cost of the coding unit, thereby reducing the overhead of hardware resources and improving the encoding speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A schematic diagram of a flow chart of a coding unit division method in one embodiment;

[0044] Figure 2 A schematic diagram of coding unit division and non-division in one embodiment;

[0045] Figure 3 is a schematic diagram of an encoding unit in one embodiment;

[0046] Figure 4 is a flowchart of the decision logic in one embodiment;

[0047] Figure 5 is a fitting curve diagram of a quantization parameter and a texture complexity threshold in one embodiment;

[0048] Figure 6 A fitting curve diagram of a quantization parameter and a non-divided threshold in one embodiment;

[0049] Figure 7 is a hardware schematic diagram in one embodiment;

[0050] Figure 8 A schematic diagram of a texture complexity calculation sequence in one embodiment;

[0051] Fig. 9 is a structural block diagram of a coding unit division device in one embodiment;

[0052] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] In one embodiment, Figure 1 As shown, a coding unit division method is provided. This embodiment uses the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server.

[0055] In this embodiment, the method includes the following steps:

[0056] S102: Obtain a coding unit to be processed, and obtain a cost of not dividing the coding unit to be processed.

[0057] The coding unit to be processed is a coding unit that needs to be divided at present, that is, a CU or a CTU.

[0058] The non-division cost is the prediction mode cost when the coding unit to be processed is not divided. The non-division cost can be calculated using algorithms such as SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transformed Difference, based on the sum of absolute errors of the Hadamard transform), and RDO (Rate Distortion Optimization). This application does not limit the calculation method of the non-division cost.

[0059] Given a CU, assuming its size is 2Nx2N pixels, the horizontal coordinate in the CU is i (0≤i≤2N-1), and the vertical coordinate is the original pixel value at position j (0≤j≤2N-1), denoted by O i，j .

[0060] Specifically, combined Figure 2 As shown, if CU is not divided, the size of CU is 2Nx2N, denoted as CU 2N×2N If the CU needs to be divided, it is divided into 4 NxN CUs, denoted as CU N×N_0 , CU N×N_1 , CU N×N_2 , CU N×N_3 .

[0061] Through the motion estimation algorithm, the predicted pixels and prediction mode costs of CU division and non-division are obtained respectively, including:

[0062] Get CU 2N×2N The predicted pixel, the predicted pixel block size is the same as 2Nx2N, the horizontal coordinate in the predicted pixel block is i (0≤i≤2N-1), and the vertical coordinate is the predicted pixel value at position j (0≤j≤2N-1), denoted as Prediction mode cost, denoted as ModeCost_CU 2N×2N .

[0063] Get CU N×N_0 , CU N×N_1 , CU N×N_2 , CU N×N_3 The predicted pixel blocks of the four CUs are all of the same size of NxN. The horizontal coordinate in the predicted pixel block is i (0≤i≤N-1), and the vertical coordinate is the predicted pixel value at position j (0≤j≤N-1), which are recorded as Prediction mode cost, denoted as ModeCost_CU N×N_0 、ModeCost_CU N×N_1 、ModeCost_CU N×N_2 、ModeCost_CU N×N_3 .

[0064] Since motion estimation is not the focus of this application, how to obtain the predicted pixels and the prediction mode cost is not described in detail here.

[0065] Calculate the CU without dividing the cost, that is, calculate Cost_CU 2N×2N The cost, denoted as NoSplitCost, is calculated as follows:

[0066]

[0067] Calculate the cost of CU splitting, denoted as SplitCost, and the calculation steps are as follows:

[0068] First calculate CU separately Split0 , CU Split1 , CU Split2 , CU Split3 The cost is denoted as Cost_CU N×N_0 、Cost_CU N×N_1 、Cost_CU N×N_2 、Cost_CU N×N_3 :

[0069]

[0070]

[0071]

[0072]

[0073] Then calculate the total cost of the above four CUs, which is the partition cost, recorded as SplitCost:

[0074] SplitCost=Cost_CU N×N_0 +Cost_CU N×N_1 +Cost_CU N×N_2 +Cost_CU N×N_3

[0075] S104: Obtain the texture complexity of the coding unit to be processed.

[0076] Among them, there is the following relationship between CU division and image texture: the richer the image texture and the more detailed areas, the more complex the image texture; the more monotonous the image texture and the more flat areas, the simpler the image texture.

[0077] The relationship between the complexity of image texture and the degree of CU division is as follows: the more complex the image texture is, the more likely it is that the CU will be divided into smaller parts; the simpler the image texture is, the more likely it is that the CU will be divided into larger parts.

[0078] By using the above characteristics, the mathematical relationship between the image texture and the CU division tendency is obtained, so that the mathematical relationship is used to assist the CU division decision.

[0079] In one of the optional embodiments, the method for calculating the texture complexity, that is, before obtaining the texture complexity of the coding unit to be processed, also includes: a step of calculating the texture complexity, and the step of calculating the texture complexity includes: obtaining a pixel mean based on the pixel value of each pixel in the coding unit to be processed; obtaining a pixel variance of the coding unit to be processed based on the pixel value of each pixel in the coding unit to be processed and the pixel mean; and using the pixel variance as the texture complexity of the coding unit to be processed.

[0080] Texture complexity refers to the complexity of image texture. Here, variance is used to measure the complexity of image texture. The steps for calculating the variance of the 2Nx2N size coding unit CU to be processed are as follows:

[0081] The first step is to calculate the pixel mean A of the coding unit CU to be processed

[0082]

[0083] The second step is to calculate the pixel variance S of the coding unit CU to be processed

[0084]

[0085] The relationship between the variance S calculated by the above formula and the texture complexity of the coding unit CU to be processed is as follows:

[0086] When the variance S is small, it means that the fluctuation of the pixel value in the coding unit CU to be processed is small and the distribution is relatively concentrated, which means that the simpler the pixel texture of the coding unit CU to be processed is, the more likely the coding unit CU to be processed is to not be divided.

[0087] When the variance S is large, it means that the fluctuation of the pixel values ​​in the coding unit CU to be processed is large and the distribution is relatively dispersed, which means that the more complex the pixel texture of the coding unit CU to be processed is, the more likely the coding unit CU to be processed is to be divided.

[0088] S106: When the coding unit to be processed is not of the target size, the texture complexity is greater than the target texture complexity threshold, and the non-division cost is greater than the non-division cost threshold, the coding unit to be processed is divided.

[0089] The target size is the minimum size of the coding unit to be processed. When the coding unit to be processed is the target size, it is definitely not necessary to divide it. For example, for HEVC encoding, the minimum size of the coding unit to be processed CU is 8x8, and at this time, it is definitely not necessary to divide the coding unit to be processed CU.

[0090] When the coding unit to be processed is not the target size, the texture complexity is used to determine whether the coding unit to be processed needs to be divided. Specifically, the variance S is used to determine whether to divide the coding unit to be processed into CUs. Specifically, the threshold T is introduced. Split As a quantitative indicator, when S>T Split , it is considered that the coding unit CU to be processed tends to be divided.

[0091] However, the complexity of the texture of the coding unit CU to be processed does not necessarily mean that it must be divided. In some scenarios, although the texture of the coding unit CU to be processed is very complex, the coding unit CU to be processed may be a moving whole. In this case, the motion information of the entire coding unit CU to be processed is consistent, so it is better not to divide the coding unit CU to be processed. Figure 3 As shown, the first block represents the coding unit CU1 to be processed, and the second block represents the coding unit CU2 to be processed. The video image has the following characteristics: the entire background is still, and the female host tosses the ball back and forth with both hands, so both hands and the ball are moving, and the transport direction and speed are inconsistent.

[0092] The coding unit CU2 to be processed is part of the background. Although it has more texture details, it is static. Therefore, the coding unit CU2 to be processed can find an optimal reference block of the same size in the reference frame, so the probability of not being divided is relatively high.

[0093] The coding unit CU1 to be processed also has more texture details and complex motion. Therefore, it is better to divide the coding unit CU1 to be processed into smaller coding units CU to be processed, because the motion information of each small coding unit CU to be processed is inconsistent. After dividing, each small coding unit CU to be processed can find the optimal reference block at different positions in the reference frame.

[0094] Generally speaking, if the current coding unit CU to be processed is a moving whole, the cost obtained by the motion estimation algorithm will not be very large. Therefore, this application introduces a second threshold value, which does not divide the cost threshold T NoSplit , when no split cost NoSplitCost>T NoSplit , it is considered that the coding unit CU to be processed tends to be divided, otherwise, it tends not to be divided.

[0095] Therefore, in this application, when the coding unit to be processed is not the target size, it is first determined whether the texture complexity is greater than the target texture complexity threshold. If so, it is further determined whether the non-division cost is greater than the non-division cost threshold. If so, the coding unit to be processed is divided.

[0096] The above-mentioned coding unit division method obtains the coding unit to be processed and obtains the cost of not dividing the coding unit to be processed; obtains the texture complexity of the coding unit to be processed; when the coding unit to be processed is not the target size, the texture complexity is greater than the target texture complexity threshold and the cost of not dividing is greater than the cost threshold of not dividing, the coding unit to be processed is divided. In this way, there is no need to calculate the division cost of the coding unit to be processed when making judgments, thereby reducing the calculation of the division cost of the coding unit, thereby reducing the overhead of hardware resources and improving the encoding speed.

[0097] In one of the optional embodiments, the method also includes: obtaining the partition cost of the coding unit to be processed; when the texture complexity is less than or equal to the target texture complexity threshold or the non-partition cost is less than or equal to the target non-partition cost threshold, judging whether the partition cost is less than the non-partition cost; when the partition cost is less than the non-partition cost, partitioning the coding unit to be processed.

[0098] In one of the optional embodiments, the method further includes: when the coding unit to be processed is of a target size, or the division cost is greater than or equal to the non-division cost, not dividing the coding unit to be processed.

[0099] The specific calculation method of the partitioning cost can be found above.

[0100] Combination Figure 4 As shown, the decision logic of the present application includes: first, let the initialization split flag, for example, SplitFlag = 0, and then determine whether the coding unit to be processed is the target size. If the coding unit to be processed is the target size, the split flag is set to not split. If the coding unit to be processed is not the target size, then continue to determine whether the texture complexity is greater than the target texture complexity threshold and whether the non-split cost is greater than the target non-split cost threshold, that is, determine S>T Split And NoSplitCost>T NoSplit Is it satisfied? If so, the split flag is set to split, otherwise continue to compare whether the split cost is less than the non-split cost, that is, whether SplitCost<NoSplitCost is satisfied. If the split cost is less than the non-split cost, the split flag is set to split, that is, SplitFlag=1, otherwise the split flag is set to not split, that is, SplitFlag=0.

[0101] In one of the optional embodiments, the method also includes: obtaining the target non-division cost threshold based on the fitting result of the quantization parameter value and the non-division cost threshold and the quantization parameter value corresponding to the coding unit to be processed; wherein the fitting result is based on testing different sample video sequences to obtain the correspondence between different quantization parameter values ​​and the non-division cost threshold, and based on the correspondence between different quantization parameter values ​​and the non-division cost threshold, curve fitting is performed.

[0102] In one of the optional embodiments, the method also includes: obtaining a target texture complexity threshold based on a fitting result of a quantization parameter value and a texture complexity threshold and a quantization parameter value corresponding to the coding unit to be processed; the fitting result is obtained by testing different sample video sequences to obtain a correspondence between different quantization parameter values ​​and texture complexity thresholds; and curve fitting is performed based on the correspondence between different quantization parameter values ​​and texture complexity thresholds.

[0103] Among them, the CU division tendency is also strongly correlated with the quantization parameter QP: the larger the QP, the coarser the encoding, so the CU tends not to be divided; the smaller the QP, the finer the encoding, so the CU tends to be divided.

[0104] According to this, different QPs correspond to T Split With T NoSplit The value of should be different. First, obtain some typical video sequences, and then do a lot of parameter training for different QP values ​​of each video sequence, and then obtain QP and T respectively through curve fitting. Split , T NoSplit relationship.

[0105] Specifically, combined Figure 5 As shown, Figure 5 To intercept the optimal T for 10 different test sequences when QP = 22, 26, 30, 34 Split Value. It can be seen that T Split The changing trends of are roughly the same, and for most test sequences, under the same QP, their optimal T Split The value is relatively concentrated.

[0106] In this application, a binary linear function is used to fit the curve:

[0107] T Split =α0×Qp 2 +β0×Qp+γ0

[0108] Where Qp represents the value of the quantization parameter QP, T Split represents the optimal threshold, α0, β0, and γ0 are three constant parameters.

[0109] Due to different encoders, the trained curves are definitely different, which means that the values ​​of α0, β0, and γ0 are definitely different. Therefore, this application does not make specific restrictions on the values ​​of α0, β0, and γ0.

[0110] Combination Figure 6 As shown, Figure 6 To intercept the optimal T for 17 different test sequences when QP = 22, 26, 30, 34 NoSplit Value. It can be seen that T NoSplit The changing trends of are roughly the same, and for most test sequences, under the same QP, their optimal T Noplit The value is relatively concentrated.

[0111] In this application, a binary linear function is used to fit the curve:

[0112] T NoSplit =α1×Qp 2 +β1×Qp+γ1

[0113] according to Figure 6 The curve trend can also be fitted with a natural exponential function:

[0114] T NoSplit =a×e b×Qp

[0115] Where Qp represents the value of the quantization parameter QP, T NoSplit represents the optimal threshold, α1, β1, γ1, a, and b are all constant parameters.

[0116] Due to different encoders, the trained curves are definitely different, which means that the values ​​of α1, β1, γ1, a, and b are definitely different. Therefore, this application does not restrict the selection of the fitting function, nor does it make specific restrictions on the values ​​of α1, β1, γ1, a, and b.

[0117] For ease of understanding, combined Figure 7 As shown, Figure 7 Schematic diagram of hardware in one embodiment. In this embodiment, the memory is used to store data such as original pixels, reference pixels, thresholds, etc.

[0118] The motion estimation module is used to calculate the cost of the 2Nx2N CU, including: the cost of not dividing the CU, that is, the cost of the 2Nx2N CU and the cost of dividing the CU, that is, the cost of 4 NxN CUs.

[0119] Texture analysis is used to calculate the texture features of a CTU, that is, the variance of CUs of various sizes within a CTU.

[0120] The CU decision module uses the relationship between the variance and the threshold, and the relationship between the cost, to decide whether to divide the CU.

[0121] In some of the optional embodiments, before obtaining the texture complexity of the coding unit to be processed, it also includes: obtaining each pre-division result of the coding unit to be processed, the pre-division result includes each coding unit with increasing size in sequence; determining the initial coding unit based on the size, and calculating each texture complexity of each initial coding unit; determining the next coding unit as each current coding unit based on the size, and calculating each texture complexity of each current coding unit based on the texture complexity of each coding unit of the previous size, until the texture complexity of the coding unit of each size is obtained.

[0122] Texture analysis mainly performs texture analysis on the original pixel values, that is, calculates the variance of the original pixel values.

[0123] In order to simplify the calculation logic of the hardware, the variance formula above is transformed as follows:

[0124]

[0125] make:

[0126]

[0127]

[0128] but:

[0129]

[0130] For HEVC, the maximum size of a CTU is 64x64 pixels. If the variance of 64x64 pixels is calculated directly, the amount of calculation is relatively large. Therefore, the variance is calculated in units of 16x16 pixels. For example, Figure 8 As shown:

[0131] According to the scan order of the dotted line in the first figure, the P and Q of the 16x16 CU are calculated in sequence. The P and Q of the 16 16x16 CUs are recorded as P0~P15 and Q0~Q15 respectively. represents the original pixel of the kth (0≤k≤15) 16x16CU, then

[0132]

[0133]

[0134] The variances of the 16 16x16 CUs are recorded as S0 to S15, respectively, then:

[0135]

[0136] The P and Q of the four 32x32 CUs are recorded as PP0~PP3, QQ0~QQ3 respectively. After calculating the P and Q of each 16x16 CU, the PP is accumulated. t and QQ t (0≤t≤3)

[0137] t=k>>2,(0≤k≤15)

[0138]

[0139]

[0140] The variances of the four 32x32 CUs are recorded as SS0 to SS3.

[0141]

[0142] The P and Q of a 64x64 CU are recorded as PPP and QQQ respectively. After calculating the P and Q of each 32x32 CU, PPP and QQQ are accumulated.

[0143]

[0144]

[0145] The variance of a 64x64 CU is recorded as SSS

[0146]

[0147] So far, the variances of 16 16x16 CUs, 4 32x32 CUs, and 1 64x64 CU are calculated in turn, where

[0148] S k (0≤k≤15), used to assist in deciding whether the kth 16x16 CU should be divided into four 8x8 CUs.

[0149] SS t (0≤t≤3), used to assist in deciding whether the t-th 32x32 CU should be divided into four 16x16 CUs.

[0150] SSS is used to assist in deciding whether a 64x64 CU should be divided into four 32x32 CUs.

[0151] In one of the optional embodiments, the method also includes: based on the quantization parameter value corresponding to the coding unit to be processed, querying the mapping relationship table between the quantization parameter value and the non-division cost threshold to obtain the target non-division cost threshold; the mapping relationship table between the quantization parameter value and the non-division cost threshold is obtained based on the fitting results of the quantization parameter value and the non-division cost threshold and each quantization parameter value; the fitting result is based on testing different sample video sequences to obtain the correspondence between different quantization parameter values ​​and the non-division cost threshold, and based on the correspondence between different quantization parameter values ​​and the non-division cost threshold, curve fitting is performed.

[0152] In one of the optional embodiments, the method also includes: based on the quantization parameter value corresponding to the coding unit to be processed, querying the mapping relationship table between the quantization parameter value and the texture complexity threshold to obtain the target texture complexity threshold; the mapping relationship table between the quantization parameter value and the texture complexity threshold is obtained based on the fitting results of the quantization parameter value and the texture complexity threshold and each quantization parameter value; the fitting result is based on testing different sample video sequences to obtain the correspondence between different quantization parameter values ​​and the texture complexity threshold, and based on the correspondence between different quantization parameter values ​​and the texture complexity threshold, curve fitting is performed.

[0153] Where T Split , T NoSplit The relationship between QP and QP can be fitted into a function model. However, in video coding, the value range of QP is a finite set of positive integers. For example, in HEVC encoder, the value range of QP is a positive integer between 0 and 51, with a total of 52 values. In this embodiment, the threshold T Split , T NoSplit The relationship with QP is made into two tables to avoid complicated calculations by the hardware.

[0154] At the same time, these two tables can be configured by software instead of being hard-coded in the HW, which increases flexibility and scalability.

[0155] The specific operation includes the following steps:

[0156] Train T separately Split , T NoSplit The relationship between and QP is obtained to obtain the fitting function.

[0157] Substitute the QP value into the above fitting function to obtain T at each QP value Split , T NoSplit value.

[0158] For the same QP value, T Split With T NoSplit Each occupies 16 bits, and the two T Split With T NoSplitThe 32-bit data is combined into a 32-bit data, and a 32-bit data is called a DW (Double Word). In other embodiments, other bit numbers can be used for storage, and no specific limitation is made here.

[0159] Assuming that QP has N values, there are N DW data in total. The software driver writes the N DW data into the memory, and the hardware encoder reads the N DW data from the memory and writes it into the internal SRAM. When the T corresponding to a certain QP value is to be used, Split With T NoSplit When the threshold is reached, the QP value is used as the index to read the data of one DW from the SRAM. The data of this DW contains the T corresponding to the QP. Split With T NoSplit Threshold.

[0160] After adding the CU division method mentioned in this solution to the HEVC hardware encoder, the coding efficiency is improved, especially for large QP segments. 31 video sequences were tested in this application, and the specific test results are as follows:

[0161] For the small QP (0≤QP≤18) segment, the maximum improvement is 13.84% and the average improvement is 2.21%.

[0162] For the middle QP segment (18<QP≤34), the maximum improvement is 4.43% and the average improvement is 1.71%.

[0163] For the large QP (34<QP≤51) segment, the maximum improvement is 59.20% and the average improvement is 35.40%.

[0164] This application fully considers the impact of CU division on hardware encoding speed and resource occupancy, and is relatively friendly to hardware implementation. This application supports software preset thresholds, and different threshold parameters can be set according to needs, with strong scalability. This application only uses the HEVC encoder as an example to illustrate the processing method, and other coding standards can also use this method.

[0165] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0166] Based on the same inventive concept, the embodiment of the present application also provides a coding unit division device for implementing the coding unit division method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more coding unit division device embodiments provided below can refer to the limitations of the coding unit division method above, and will not be repeated here.

[0167] In an exemplary embodiment, Fig. 9 As shown, a coding unit division device is provided, comprising: a coding unit acquisition module 901, a texture complexity acquisition module 902 and a division module 903, wherein:

[0168] The coding unit acquisition module 901 is used to acquire the coding unit to be processed and acquire the undivided cost of the coding unit to be processed;

[0169] The texture complexity acquisition module 902 is used to acquire the texture complexity of the coding unit to be processed;

[0170] The division module 903 is used to divide the coding unit to be processed when the coding unit to be processed is not of the target size, the texture complexity is greater than the target texture complexity threshold, and the non-division cost is greater than the non-division cost threshold.

[0171] In one of the optional embodiments, the above device further includes:

[0172] A partition cost calculation module, used to obtain the partition cost of the coding unit to be processed;

[0173] The division module 903 is also used to determine whether the division cost is less than the non-division cost when the texture complexity is less than or equal to the target texture complexity threshold or the non-division cost is less than or equal to the target non-division cost threshold; when the division cost is less than the non-division cost, the coding unit to be processed is divided.

[0174] In one of the optional embodiments, the above-mentioned division module 903 is further used for not dividing the coding unit to be processed when the coding unit to be processed is of the target size or the division cost is greater than or equal to the non-division cost.

[0175] In one of the optional embodiments, the above device further includes:

[0176] The first target non-division cost threshold generation module is used to obtain the target non-division cost threshold based on the fitting result of the quantization parameter value and the non-division cost threshold and the quantization parameter value corresponding to the coding unit to be processed; wherein the fitting result is based on testing different sample video sequences to obtain the correspondence between different quantization parameter values ​​and the non-division cost threshold, and based on the correspondence between different quantization parameter values ​​and the non-division cost threshold, curve fitting is performed.

[0177] In one of the optional embodiments, the above device further includes:

[0178] The second target non-division cost threshold generation module is used to query the mapping relationship table between the quantization parameter value and the non-division cost threshold based on the quantization parameter value corresponding to the coding unit to be processed, and obtain the target non-division cost threshold; the mapping relationship table between the quantization parameter value and the non-division cost threshold is obtained based on the fitting results of the quantization parameter value and the non-division cost threshold and each quantization parameter value; the fitting result is based on testing different sample video sequences to obtain the corresponding relationship between different quantization parameter values ​​and the non-division cost threshold, and based on the corresponding relationship between different quantization parameter values ​​and the non-division cost threshold, curve fitting is performed.

[0179] In one of the optional embodiments, the above device further includes:

[0180] The first target texture complexity threshold generation module is used to obtain the target texture complexity threshold based on the fitting result of the quantization parameter value and the texture complexity threshold and the quantization parameter value corresponding to the coding unit to be processed; the fitting result is based on different sample video sequences for testing to obtain the corresponding relationship between different quantization parameter values ​​and the texture complexity threshold; and based on the corresponding relationship between different quantization parameter values ​​and the texture complexity threshold, the curve is fitted.

[0181] In one of the optional embodiments, the above device further includes:

[0182] The second target texture complexity threshold generation module is used to query the mapping relationship table between the quantization parameter value and the texture complexity threshold based on the quantization parameter value corresponding to the coding unit to be processed, so as to obtain the target texture complexity threshold; the mapping relationship table between the quantization parameter value and the texture complexity threshold is obtained based on the fitting results of the quantization parameter value and the texture complexity threshold and the quantization parameter values; the fitting results are obtained by testing different sample video sequences to obtain the corresponding relationship between different quantization parameter values ​​and the texture complexity threshold, and curve fitting is performed based on the corresponding relationship between different quantization parameter values ​​and the texture complexity threshold.

[0183] In one of the optional embodiments, the above device further includes:

[0184] The texture complexity calculation module is used to obtain a pixel mean based on the pixel value of each pixel in the coding unit to be processed; obtain a pixel variance of the coding unit to be processed based on the pixel value of each pixel in the coding unit to be processed and the pixel mean; and use the pixel variance as the texture complexity of the coding unit to be processed.

[0185] In one of the optional embodiments, the texture complexity calculation module is specifically used to obtain the pre-division results of the coding unit to be processed, the pre-division results include coding units of increasing sizes; determine the initial coding unit based on the size, and calculate the texture complexities of each initial coding unit; determine the next coding unit as the current coding unit based on the size, and calculate the texture complexities of each current coding unit based on the texture complexities of each coding unit of the previous size, until the texture complexity of the coding unit of each size is obtained.

[0186] Each module in the above coding unit division device can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0187] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.10As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a coding unit division method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0188] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0189] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0190] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0191] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0192] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0193] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0194] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A coding unit division method, characterized in that: The method comprises: Obtaining a coding unit to be processed, and obtaining a non-division cost of the coding unit to be processed; Obtaining the texture complexity of the coding unit to be processed; When the coding unit to be processed is not of a target size, the texture complexity is greater than a target texture complexity threshold, and the non-division cost is greater than a non-division cost threshold, the coding unit to be processed is divided.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining a partition cost of the coding unit to be processed; In the case where the texture complexity is less than or equal to the target texture complexity threshold or the non-division cost is less than or equal to the target non-division cost threshold, determining whether the division cost is less than the non-division cost; When the division cost is less than the non-division cost, the coding unit to be processed is divided.

3. The method according to claim 2, characterized in that The method further comprises: When the coding unit to be processed is of the target size, or the division cost is greater than or equal to the non-division cost, the coding unit to be processed is not divided.

4. The method according to claim 2, characterized in that: The method further comprises: Obtaining a target non-division cost threshold based on a fitting result of the quantization parameter value and the non-division cost threshold and a quantization parameter value corresponding to the coding unit to be processed; The fitting result is obtained by testing different sample video sequences to obtain the correspondence between different quantization parameter values ​​and the non-division cost threshold, and performing curve fitting based on the correspondence between the different quantization parameter values ​​and the non-division cost threshold.

5. The method according to claim 2, characterized in that: The method further comprises: Based on the quantization parameter value corresponding to the coding unit to be processed, a mapping relationship table between the quantization parameter value and the non-division cost threshold is queried to obtain the target non-division cost threshold; the mapping relationship table between the quantization parameter value and the non-division cost threshold is obtained based on the fitting results of the quantization parameter value and the non-division cost threshold and each quantization parameter value; the fitting result is obtained by testing based on different sample video sequences to obtain the correspondence between different quantization parameter values ​​and the non-division cost threshold, and curve fitting is performed based on the correspondence between the different quantization parameter values ​​and the non-division cost threshold.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Based on the fitting results of the quantization parameter value and the texture complexity threshold and the quantization parameter value corresponding to the coding unit to be processed, the target texture complexity threshold is obtained; the fitting result is tested based on different sample video sequences to obtain the corresponding relationship between different quantization parameter values ​​and the texture complexity threshold; and based on the corresponding relationship between the different quantization parameter values ​​and the texture complexity threshold, the curve fitting is performed.

7. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Based on the quantization parameter value corresponding to the coding unit to be processed, a mapping relationship table between the quantization parameter value and the texture complexity threshold is queried to obtain a target texture complexity threshold; the mapping relationship table between the quantization parameter value and the texture complexity threshold is obtained based on the fitting results of the quantization parameter value and the texture complexity threshold and each quantization parameter value; the fitting results are obtained by testing different sample video sequences to obtain the corresponding relationship between different quantization parameter values ​​and the texture complexity threshold, and curve fitting is performed based on the corresponding relationship between the different quantization parameter values ​​and the texture complexity threshold.

8. The method according to claim 1, characterized in that Before obtaining the texture complexity of the coding unit to be processed, the method further includes: Obtaining a pixel mean based on the pixel value of each pixel in the coding unit to be processed; Obtaining a pixel variance of the coding unit to be processed based on a pixel value of each pixel in the coding unit to be processed and the pixel mean; The pixel variance is used as the texture complexity of the coding unit to be processed.

9. The method according to claim 1, characterized in that: Before obtaining the texture complexity of the coding unit to be processed, the method further includes: Obtaining pre-division results of the coding unit to be processed, wherein the pre-division results include coding units with successively increasing sizes; Determine an initial coding unit based on the size, and calculate each texture complexity of each initial coding unit; Based on the size, the next coding unit is determined as each current coding unit, and based on the texture complexity of each coding unit of the previous size, the texture complexity of each current coding unit is calculated until the texture complexity of each coding unit of the size is obtained.

10. A coding unit division device, characterized in that: The device comprises: A coding unit acquisition module, used to acquire a coding unit to be processed, and acquire a non-division cost of the coding unit to be processed; A texture complexity acquisition module, used to acquire the texture complexity of the coding unit to be processed; The division module is used to divide the coding unit to be processed when the coding unit to be processed is not of a target size, the texture complexity is greater than a target texture complexity threshold, and the non-division cost is greater than a non-division cost threshold.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images