Method and system for accelerating division and selection of 32*32 coding units in universal video coding

Abstract

The invention discloses a method and a system for accelerating division and selection of 32 * 32 coding units in general video coding, and the method comprises the following steps: S1, forming an extraction matrix of texture features by taking 4 * 4 coding units as elements for feature extraction, the range Range, the variance Var, the absolute average deviation MAD and the maximum local texture of the extraction matrix are used as features for judging continuous division of the coding units; s2, using the features obtained in the step S1 as input, and constructing a random forest classifier model used for judging whether the coding units need to be continuously divided or not and a heuristic selection model of division types; and S3, judging whether the coding unit needs to be continuously divided by using the random forest classifier model obtained in the step S2, and if so, obtaining the division type of the coding unit by using the heuristic selection model in the step S2. According to the method, the division process of the coding units is predicted and accelerated by using a method of combining machine learning and data statistics, so that the coding efficiency can be considered while the time is saved.

Description

Technical Field

[0001] The present invention belongs to the technical field of video coding, and in particular relates to a method and a system for accelerating a 32×32 coding unit division process in general video coding. Background Art

[0002] In recent years, with the development of video acquisition technology and network technology, the amount of high-definition and ultra-high-definition video data has shown explosive growth, and there is an urgent need for more efficient and fast video coding technology. The Joint Video Exploration Team (JVET) completed the formulation of the General Video Coding (H.266 / VVC) standard in July 2020. Compared with the previous generation of High Efficiency Video Coding (HEVC) standard, this standard uses quadtrees and nested multi-type trees (QTMT) in the coding structure. This structure can divide complex image content into more flexible coding units, improving compression efficiency by about 50%. However, while improving the coding quality, it brings a huge computational burden, which limits its development, adoption and application.

[0003] QTMT is one of the most effective, complex and time-consuming coding techniques in VVC. There are five methods in the partition structure that can be used to partition the coding unit, including quadtree (QT), horizontal binary tree (HBT), vertical binary tree (VBT), horizontal ternary tree (HTT) and vertical ternary tree (VTT). HBT, VBT, HTT and VTT are collectively referred to as multi-type trees (MTT). The leaf nodes of QT can continue to be partitioned by QTMT. However, the leaf nodes of MTT can only be partitioned by MTT, regardless of whether the node is square or not. In order to achieve the highest possible compression performance, the coding tree unit (CTU) is usually recursively divided into smaller coding units by QTMT, and the encoder also needs to calculate the rate-distortion (RD) cost of each partition mode. In this recursive traversal mode, top-down partitioning and bottom-up decision-making will lead to very large computational complexity, which not only consumes a lot of time, but also has a lot of redundancy. Therefore, it is necessary to terminate the partitioning process early to reduce complexity and save encoding time.

[0004] The earliest method used to accelerate the division of coding units was based on data statistics. It used edge detection operators to extract edge features, calculate the variance of coding units, calculate absolute conversion differences and other data features to determine whether coding units need to be divided. Later, researchers found that they could pre-train decision models through machine learning methods and then predict the division of coding units. Therefore, some prediction models based on the principles of machine learning algorithms such as support vector machines, decision trees, and random forests were proposed. With the rapid development of neural network models, researchers have proposed more complex models to predict the division of coding units and achieved improved prediction accuracy. However, neural network models also have disadvantages such as high computational overhead and difficulty in interpretation. Therefore, in practical applications, methods based on the combination of data statistics and traditional machine learning to accelerate the division of coding units are still very popular, which can take into account both efficiency and accuracy.

[0005] In the VVC encoding process, since the maximum allowed size of the binary tree (BT) root node and the ternary tree (TT) root node is 32, there are 6 types of options for the coding unit of size 32×32 (no division, QT, HBT, VBT, HTT, VTT); and the coding unit of size 64×64 can only choose between QT division or no division. Therefore, the division problem of the 64×64 coding unit is reduced to a binary decision of division or no division, and the acceleration gain generated by accelerating this division step is not high for the benefit of the entire division encoding process. Therefore, the present invention does not focus on the division problem of the coding unit of size 64×64, but focuses on the division problem of the 32×32 coding unit that takes the most time, that is, how to accelerate the division process of the 32×32 coding unit and select the appropriate division type. When the size of the coding unit mentioned later is not specified, it is defaulted to the 32×32 coding unit. Based on this, the present invention provides a method and system for accelerating the division selection of 32×32 coding units in general video encoding. Summary of the invention

[0006] In view of the problem that redundancy exists in the process of dividing 32×32 coding units in general video coding, which leads to excessive computational burden, the present invention provides a method and system for accelerating the selection of 32×32 coding unit division. The present invention predicts and accelerates the division process of coding units by combining machine learning with data statistics, which can save time while taking into account coding efficiency.

[0007] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0008] A method for accelerating 32×32 coding unit partitioning selection in general video coding comprises the following steps:

[0009] S1, the smallest 4×4 coding unit is used as the primitive for feature extraction to form a texture feature extraction matrix, and the range (Range), variance (Var), mean absolute deviation (MAD), and maximum local texture of this matrix are used as features to determine whether the coding unit needs to be further divided;

[0010] S2, using the features obtained in step S1 as input, constructing a random forest classifier model for determining whether the coding unit needs to be further divided and a heuristic model for how to select the division type;

[0011] S3, using the random forest classifier model obtained in step S2 to determine whether the coding unit needs to be further divided; if it needs to be further divided, the heuristic selection model obtained in step S2 is used to directly obtain the division type of the coding unit, that is, the division result.

[0012] Preferably, in step S1, the feature calculation method specifically includes the following steps:

[0013] S11, reducing the dimension of the 32×32 coding unit to an 8×8 feature extraction matrix, that is, mapping each 4×4 minimum coding unit in the original coding unit to a primitive, calculating the average brightness value of 16 pixels in each minimum coding unit as the value of the primitive, thereby reducing the dimension of the 32×32 input matrix to an 8×8 feature extraction matrix Ave;

[0014] The specific calculation expression is:

[0015]

[0016] Among them, Ave (m,n) represents the value of a single computing unit, (m,n) represents the position of the computing unit in the feature extraction matrix, and P (i,j) Represents the brightness value at (i, j) in the original coding unit;

[0017] S12, calculating the statistical features of the 8×8 feature extraction matrix Ave, including range (Range), variance (Var), and mean absolute deviation (MAD).

[0018] The specific expression is as follows:

[0019] Range=max(Ave (m,n) )-min(Ave (m,n) )

[0020]

[0021] Among them, w and h represent the width and height of the feature extraction matrix respectively, represents the average value of the entire feature extraction matrix, and abs(·) represents the absolute value of the calculated difference;

[0022] S13, calculating the maximum local texture intensity of the 8×8 feature extraction matrix Ave based on the search box method, that is, the local texture maximum variance (MVar) and the local texture maximum absolute mean deviation (MMAD).

[0023] Furthermore, specifically, in order to maximize the search for the largest local texture feature, the search box starts from the upper left corner of the feature extraction matrix, and the moving step length each time is one primitive row (or column), resulting in 25 overlapping search boxes, each containing 16 primitives. For the calculation process of the search box, it is necessary to first calculate the mean of the primitives inside it, then calculate its Var and MAD, and then take the maximum value of the Var value and MAD value of the 25 search boxes as MVar and MMAD. Finally, these two eigenvalues ​​are used to express the largest local texture.

[0024] The specific expression is as follows:

[0025]

[0026] Preferably, in step S2, the random forest classifier model of whether the 32×32 coding unit needs to be further divided and the heuristic model construction method of how to divide it are specifically as follows:

[0027] S21, select some frames of the VVC standard video sequence and divide and encode them using the standard encoding scheme, then count the division results of the 32×32 coding units, and extract features according to the method of step S1 (that is, use 4×4 coding units as primitives for feature extraction to form an extraction matrix of texture features), and finally construct a data set according to <feature, division result>;

[0028] S22, build a random classifier model with the following parameters;

[0029] Table 1. Random forest classifier model parameter settings

[0030]

[0031] S23, divide the data set into a training set and a test set, use the training set to drive the construction of the random forest classifier model for training, and use the test set to perform performance testing until the results converge or the maximum number of training rounds is reached.

[0032] In order to construct the heuristic selection model, the MAD of the undivided sub-coding blocks generated after the 32×32 coding unit is divided is counted to determine the threshold in the selection model. Furthermore, there are 5 sizes of sub-coding blocks that need to be counted, namely 16×16, 8×32, 32×8, 16×32 and 32×16; these 5 sizes are divided into 256 and 512 categories according to the number of points. Their statistical process is similar to that of the 32×32 coding unit. They all need to be converted into a feature extraction matrix first, and then the MAD of the feature extraction matrix is ​​calculated and the division results are recorded.

[0033] Preferably, when the actual 32×32 coding unit is divided, step S3 is specifically as follows:

[0034] S31, using the feature extraction method given in S1 (i.e., using 4×4 coding units as primitives for feature extraction to form an extraction matrix of texture features) to calculate five features of the current 32×32 coding unit;

[0035] S32, inputting the five features into the random forest classification model obtained in step S23, and judging whether further classification is needed according to the output result of the random forest classification model;

[0036] S33, for the 32×32 coding unit that needs to be further divided, first pre-assume division is performed on the coding unit, and the MAD of each sub-coding block under different pre-division types is calculated;

[0037] S34, comparing the MAD value of the sub-coding block with the threshold of the selection model to determine whether the coding unit meets the early exit judgment process;

[0038] S35, for samples that do not conform to the selection model (i.e., samples that do not conform to the three judgment modes), the sub-coding block MADs obtained by the pre-hypothesis partitioning are added, and the partition type corresponding to the minimum value is taken as the final partition selection, and the expression is as follows:

[0039] HB=MAD HBT1 +MAD HBT2

[0040] VB=MAD VBT1 +MAD VBT2

[0041] HT=MAD HTT1 +MAD HTT2 +MAD HTT3

[0042] VT=MAD VTT1 +MAD VTT2 +MAD VTT3

[0043] Q=MAD QT1+MAD QT2 +MAD QT3 +MAD QT4

[0044] Among them, HB, VB, HT, VT and Q represent the sum of the sub-coding block MAD after different pre-hypothesis divisions, MAD HBT1 and MAD HBT2 represents the MAD of the two sub-coding blocks after the pre-assumption level 2 division, MAD VBT1 and MAD VBT2 It represents the MAD of the two sub-coding blocks after the pre-assumed vertical two-division, MAD HTT1 、MAD HTT2 and MAD HTT3 represents the MAD of the three sub-coding blocks after the pre-assumption level three division, MAD VTT1 、MAD VTT2 and MAD VTT3 It represents the MAD of the three sub-coding blocks after the pre-assumed vertical three-division, MAD QT1 、MAD QT2 、MAD QT3 and MAD QT4 Represents the MAD of the four sub-coding blocks after the pre-assumed quadtree QT partition.

[0045] Preferably, in the process of constructing the heuristic selection model, the MAD and the division results of the two undivided sub-coding blocks with different numbers of points generated after the 32×32 coding unit division are counted to determine the threshold in the selection model. The specific process is as follows:

[0046] S241, grouping according to the QP parameter value during encoding, sorting the MAD values ​​in each group from small to large, calculating the average of the 50%-75% ranking part as the threshold for whether the sub-coding block of the type corresponding to the QP parameter needs to be further divided.

[0047] S242, different QP parameters will generate corresponding thresholds, and the thresholds and parameters are combined into a linear parametric equation, thereby deriving the relationship between the thresholds and the QP parameters. The equation expression is as follows:

[0048] TH1=α×QP+β

[0049] TH2=γ×QP+δ

[0050] The number of sub-coding block points corresponding to TH1 is 512, and the number of sub-coding block points corresponding to TH2 is 256.

[0051] S243, the final derived parameters are α=12.10, β=6.38; y=1.48, δ=16.95.

[0052] Preferably, the specific process of step S33 is as follows:

[0053] S331, when performing pre-hypothesis partitioning, a total of three types of pre-hypothesis partitioning are required, namely, horizontal partitioning, vertical partitioning and quadtree QT partitioning. When the first type of partitioning cannot jump out of the selection model, subsequent pre-hypothesis partitioning judgments will be performed again in order;

[0054] S332, the pre-assumption partitioning needs to calculate the MAD of the sub-coding blocks of the coding unit, and the horizontal pre-assumption partitioning and the vertical pre-assumption partitioning are calculated first. The sub-coding block MAD calculation process of the two is similar; due to the establishment of the search box in step S13, the MAD (1, 1), MAD (1, 5), MAD (5, 1), and MAD (5, 5) calculated in this step correspond to the MAD of the four sub-coding blocks divided by the pre-assumption QT; the specific calculation expression of the sub-coding block MAD generated by the horizontal pre-assumption partitioning is as follows:

[0055]

[0056] Where w and h represent the width and height of the feature extraction matrix, respectively; h / 2 and h / 4 represent the heights of different sub-coding blocks after pre-hypothesis division, respectively. and They respectively represent the average values ​​of the internal computing units of different sub-coding blocks after the pre-hypothesis division;

[0057] S333, after all the pre-hypothesized divisions, the MAD of all sub-coding blocks can be calculated: MAD HBT1 , MAD HBT2 , MAD HTT1 , MAD HTT2 , MAD HTT3 , MAD VBT1 , MAD VBT2 , MAD VTT2 , MAD VTT1 , MAD VTT3 , MAD QT1 , MAD QT2 , MAD QT3 , MAD QT4 .

[0058] Preferably, step S34 is a multi-stage process of jumping out of the model in advance by comparing with the threshold value, and the specific process is as follows:

[0059] S341, in step S24, the MAD and division results of the two coding blocks with different points are counted, so whether the coding blocks of the two points are further divided can be regarded as a binary classification problem, and the possibility of their division can be judged by comparing with the thresholds TH1 and TH2.

[0060] S342, the conditions for the horizontal pre-hypothesis partition to jump out of the selection model in advance are:

[0061] 1. If MAD HBT1 <TH1 and MAD HBT2 <TH1 is established, then horizontal two division is performed;

[0062] 2. If (MAD HBT1 <TH1 or MAD HBT2 <TH1) But MAD HTT2 >TH1, then perform horizontal two-division;

[0063] 3. If (MAD HBT1 <TH1 or MAD HBT2 <TH1) But MAD HTT2 <TH1, then perform horizontal three-division;

[0064] 4. If (MAD HBT1 >TH1 and MAD HBT2 >TH1) and (MAD HTT1 <TH2 or MAD HTT2 <TH1 or MAD HTT3 <TH2), then perform horizontal three-division.

[0065] S343, the conditions for vertical pre-hypothesis partitioning to jump out of the selection model in advance are:

[0066] 1. If MAD VBT1 <TH1 and MAD VBT2 <TH1 is established, then vertical two-division is performed;

[0067] 2. If (MAD VBT1 <TH1 or MAD VBT2 <TH1) But MAD VTT2 >TH1, then vertical two-division is performed;

[0068] 3. If (MAD VBT1 <TH1 or MAD VBT2 <TH1) But MAD VTT2 <TH1, then vertical three-division is performed;

[0069] 4. If (MAD VBT1 >TH1 and MAD VBT2 >TH1) and (MAD VTT1 <TH2 or MAD VTT2 <TH1 or MAD VTT3 <TH2), a vertical three-division is performed.

[0070] S344, QT pre-assumption partition early jump out of the selection model conditions are:

[0071] If MAD QT1 、MAD QT2 、MAD QT3 、MAD QT4 If any of is less than TH2, QT division is performed.

[0072] The present invention also discloses a system for accelerating 32×32 coding unit partitioning and selection in general video coding, which is used to execute the above method and includes the following modules:

[0073] Feature extraction module: A texture feature extraction matrix is ​​constructed using 4×4 coding units as primitives for feature extraction, and the range, variance, absolute mean deviation, and maximum local texture of the extraction matrix are used as features for determining whether to further divide the coding unit.

[0074] Model building module: using the obtained features as input, constructing a random forest classifier model for determining whether the coding unit needs to be further divided and a heuristic selection model for the division type;

[0075] Coding unit division module: The random forest classifier model is used to determine whether the coding unit needs to be further divided. If so, the heuristic selection model is used to obtain the division type of the coding unit.

[0076] Compared with the prior art, the present invention has the following characteristics and beneficial effects:

[0077] By adopting the technical solution of the present invention, the 32×32 coding unit is converted to a 4×4 minimum partition block as the primitive, and the dimension is reduced to an 8×8 feature extraction matrix, thereby reducing the complexity of feature calculation; then the overall and local texture features are input into the random forest classifier model to determine whether the current coding unit needs to be further divided; finally, for the coding unit that needs to be further divided, a heuristic selection model is used to select the optimal division type. Compared with the VVC standard coding unit division process, the present invention can save time while taking into account coding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0079] Figure 1A flowchart of a method for accelerating 32×32 coding unit partitioning and selection according to a preferred embodiment of the present invention;

[0080] Figure 2 A schematic diagram of converting a coding unit into a computing unit in a preferred embodiment of the present invention;

[0081] Figure 3 A schematic diagram of a local texture search box according to a preferred embodiment of the present invention;

[0082] Figure 4 A schematic diagram of five pre-assumption division types in a preferred embodiment of the present invention;

[0083] Figure 5 A schematic diagram showing that a sub-coding block generated by a division of a coding unit in a preferred embodiment of the present invention does not need to be further divided;

[0084] Figure 6 A block diagram of a system for accelerating 32×32 coding unit partition selection according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0085] In order to explain the purpose, technical scheme and process of the present invention in more detail, the present invention will be further described with reference to the accompanying drawings.

[0086] Embodiment 1:

[0087] like Figure 1 As shown, this embodiment proposes a method for accelerating 32×32 coding unit partition selection in general video coding, including the following steps:

[0088] S1, using the smallest 4×4 coding unit as the primitive for feature extraction to form a texture feature extraction matrix, using the range (Range), variance (Var), mean absolute deviation (MAD), and maximum local texture of this matrix as features to determine whether the coding unit needs to be further divided.

[0089] S2, using the features obtained in step S1 as input, constructs a random forest classifier model for determining whether the coding unit needs to be further divided and a heuristic model for selecting the division type.

[0090] S3, using the random forest classifier model obtained in step S2 to determine whether the coding unit needs to be further divided. If it needs to be further divided, the heuristic model obtained in step S2 is used to directly obtain the division type of the coding unit, that is, the division result.

[0091] The following is a more detailed description of each step of this embodiment.

[0092] For step 1:

[0093] The first is how to characterize the texture complexity of the coding unit. The preferred embodiment of the present invention uses the range, variance, and mean absolute deviation (MAD) in statistics to jointly characterize the overall texture complexity of the coding unit. In order to fully describe the details of the entire coding unit, it is also necessary to find its details. Therefore, this embodiment also adds a texture search box to find the largest local texture. This embodiment is dedicated to the division of coding units with a size of 32×32; it contains 1024 brightness value points. If feature calculations are performed directly, this will generate a large amount of calculations, which does not meet the invention purpose of the present invention to speed up the encoding process. Therefore, it is also necessary to first reduce the dimension of the coding unit and then calculate the features. The conversion is as follows: Figure 2 As shown in the figure, a coding unit of size 32×32 is divided into 64 4×4 calculation units (division principle: the size of the smallest coding unit generated during the encoding process is 4×4, which means that no matter how the difference in the brightness value inside this smallest unit is, it will not be further divided). In subsequent calculations, the calculation unit is used as the calculation data. The conversion formula is as follows:

[0094]

[0095] Among them, Ave(m,n) represents the value of a single computing unit, (m,n) represents the position of the computing unit in the feature extraction matrix, and P (i，j) Represents the brightness value at (i, j) in the original coding unit. According to the obtained feature matrix, the required overall features are calculated. The calculation expression is as follows:

[0096] Range=max(Ave (m，n) )-min(Ave (m，n) )

[0097]

[0098] Among them, Range, Var, and MAD represent the range, variance, and absolute mean deviation of the entire feature extraction matrix, respectively; max(·) represents the maximum value; min(·) represents the minimum value; and abs(·) represents the absolute value of the calculated difference.

[0099] How to set up a search box, such as Figure 3 As shown, in this embodiment, the size of the search box is 4×4 (including 16 calculation units). The search box moves left and right from the upper left corner of the coding unit in steps of one calculation unit, and a total of 25 search boxes are generated. Only the largest Var and MAD in these search boxes are needed to express the largest local texture feature. The calculation expression is as follows:

[0100]

[0101] Among them, Mean (x,y) 、Var (x，y) and MAD (x，y) The mean, variance and absolute mean deviation of the search box are shown respectively, and (x, y) represents the position of the search box.

[0102] When expressing texture intensity, the variance and mean absolute deviation are used at the same time, which can both be used to express the degree of discreteness of a set of data. The reason is as follows: the variance and the mean absolute deviation can both express the degree of discreteness of a set of data; however, when there are individual extreme values ​​in the data, the calculation process of the variance will expand the influence of this extreme value to the entire set of data, and the data variance at this time cannot effectively express the degree of discreteness of the data. Introducing the mean absolute deviation can well compensate for this defect.

[0103] For step 2:

[0104] The video sequences used for statistical data sets are proven to have good spatial and temporal information distribution, which ensures the reliability of the data set. When statistically analyzing data, it is also necessary to ensure that the number of different types of data in the same category remains relatively consistent, which ensures the fairness of the data set and the generalization ability of the model.

[0105] Select some frames of the VVC standard video sequence and divide them into codes using the standard coding scheme. Count the division results of the 32×32 coding units, extract features according to the method in step S1, and finally construct a data set according to <features, division results>.

[0106] Construct and train a random forest classifier model, and the number of trees in the forest is set to 100; the maximum number of splits for each tree is 5 to prevent overfitting; each leaf node contains at least 1 sample; since this model is a classification task, the splitting principle uses the Gini coefficient. Before training begins, the data set needs to be preprocessed to remove some extreme and special case data. The prediction accuracy of the test set constructed by the classifier model in this embodiment is shown in Table 2. It can be seen that the prediction accuracy for the coding units that need to be further divided reaches 97.46%, and the prediction accuracy for the coding units that do not need to be divided reaches 81.37%; both can achieve very high prediction accuracy.

[0107] Table 2 Model prediction results for the test set

[0108]

[0109] like Figure 1As shown, the heuristic selection model of the second part is a multi-level jump-out model. If the precondition is met, the model can be jumped out in advance to execute the subsequent encoding steps. Regarding how to determine the precondition, this embodiment counts a large number of coding unit division results and finds that the sub-coding blocks generated in some divisions do not need to be further divided, such as Figure 5 As shown, if the sub-coding blocks that do not need to be further divided can be detected in advance, the division type of the coding unit can be directly determined.

[0110] The execution order of the judgment is determined according to the time cost of each division type in the standard coding scheme. The specific order is: horizontal two-division, horizontal three-division, vertical two-division, vertical three-division, and four-division. When all five judgments cannot be met, the final comprehensive decision is entered, the MAD and value of the sub-coding block are calculated, and the division type corresponding to the minimum value is taken as the division type selection for this time.

[0111] For step 3:

[0112] By inputting the five features into the random forest classifier, it can be concluded whether the coding unit needs to be further divided; the coding unit that does not need to be further divided can end the division process and enter the next coding stage; for the coding unit that still needs to be further divided, it will continue to enter the second selection model. In the second selection model, it is necessary to pre-hypothesize the division of the coding unit, such as Figure 4 As shown in the figure, the order of pre-hypothesis partitioning is consistent with the order of early exit of the model; when the judgment conditions of the five types cannot be met, the five types of segmentation have been pre-hypothesized, and the sum of the sub-coding block MAD under each pre-hypothesis partitioning type is calculated, and the partitioning type corresponding to the minimum sum is selected as the final partitioning selection.

[0113] The calculation expression for the pre-assumed horizontal partition type is as follows ( Figure 4 (a) and (b)):

[0114]

[0115] Among them, MAD HBT1 , MAD HBT2 , MAD HTT1 , MAD HTT2 and MAD HTT3 Represent the sub-coding blocks of horizontal two-partition and three-partition, and The MAD calculation process of the sub-coding blocks generated by the subsequent pre-assumed vertical partitioning is similar to the above.

[0116] This embodiment proposes a method for accelerating the division and selection of coding units in general video coding, which reduces the complexity of feature calculation by converting coding units into feature extraction matrices. Specifically, the coding unit is divided into several 4×4 minimum coding blocks, and the mean of the internal brightness value of the minimum coding block is used as the value of the calculation unit, and then subsequent feature calculations are performed. In order to be able to express the texture information of the coding unit well, the present invention uses overall texture information and local texture information to jointly characterize its texture intensity. These texture features are input into the trained random forest classifier model to determine whether the current coding unit needs to be divided. Finally, through a heuristic selection model, one of the five division types is selected.

[0117] The two models are integrated into the encoder to encode the video sequence, and the encoding time and Delta Bit Rate (BDBR). The experimental results are shown in Table 3. It can be seen that the embodiment of the present invention can save an average of 53.38% of the encoding time, while the increase of BDBR is only 1.63%, which proves that the effect of the embodiment of the present invention is very excellent and can well balance the reduction of computational complexity and the loss of encoding quality. Therefore, compared with the standard VVC encoding scheme, the present invention has significant progress.

[0118] Table 3 Experimental results

[0119]

[0120] Embodiment 2:

[0121] like Figure 6 As shown, this embodiment discloses a system for accelerating 32×32 coding unit partitioning selection in general video coding, which is used to execute the method of the above embodiment, and includes the following modules:

[0122] Feature extraction module: A texture feature extraction matrix is ​​constructed using 4×4 coding units as primitives for feature extraction, and the range, variance, absolute mean deviation, and maximum local texture of the extraction matrix are used as features for determining whether to further divide the coding unit.

[0123] Model building module: using the obtained features as input, constructing a random forest classifier model for determining whether the coding unit needs to be further divided and a heuristic selection model for the division type;

[0124] Coding unit division module: The random forest classifier model is used to determine whether the coding unit needs to be further divided. If so, the heuristic selection model is used to obtain the division type of the coding unit.

[0125] For other contents of this embodiment, please refer to the above method embodiment.

[0126] The above is a description of the preferred embodiments of the present invention. For ordinary technicians in this field, without paying any creative work, they can also obtain other drawings and other implementation methods based on the drawings of the above embodiments, which all belong to the protection scope of the present invention.

Claims

1. A method for accelerating 32×32 coding unit partitioning and selection in general video coding, characterized in that: The steps include: S1, using 4×4 coding units as primitives for feature extraction to form a texture feature extraction matrix, and using the range Range, variance Var, absolute mean deviation MAD and maximum local texture of the extraction matrix as features for determining whether to further divide the coding unit; S2, using the features obtained in step S1 as input, constructing a random forest classifier model for determining whether the coding unit needs to be further divided and a heuristic selection model for the division type; S3, using the random forest classifier model obtained in step S2 to determine whether the coding unit needs to be further divided, if so, using the heuristic selection model in step S2 to obtain the division type of the coding unit.

2. The method for accelerating 32×32 coding unit partitioning and selection in general video coding according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11, reducing the dimension of the 32×32 coding unit to an 8×8 feature extraction matrix, that is, mapping each 4×4 minimum coding unit in the original coding unit to a primitive, calculating the average brightness value of 16 pixels in each minimum coding unit as the value of the primitive, thereby reducing the dimension of the 32×32 input matrix to an 8×8 feature extraction matrix Ave; The specific expression is: Among them, Ave (m,n) represents the value of a single computing unit, (m,n) represents the position of the computing unit in the feature extraction matrix, and P (i,j) Represents the brightness value at (i, j) in the original coding unit; S12, calculating the statistical features of the 8×8 feature extraction matrix Ave, including the range Range, variance Var, and absolute mean deviation MAD; The specific expression is as follows: Range=max(Ave (m,n) )-min(Ave (m,n) ) Among them, w and h represent the width and height of the feature extraction matrix respectively, represents the average value of the entire feature extraction matrix, and abs(·) represents the absolute value of the calculated difference; S13, calculating the maximum local texture intensity of the 8×8 feature extraction matrix Ave based on the search box method, that is, the local texture maximum variance MVar and the local texture maximum absolute mean deviation MMAD.

3. The method for accelerating 32×32 coding unit partitioning and selection in general video coding as claimed in claim 2, characterized in that: In step S13, the search box starts from the upper left of the feature extraction matrix, and the moving step length each time is one primitive row or column, so that 25 mutually overlapping search boxes are obtained, and each search box contains 16 primitives; the calculation process of the search box: first calculate the primitive mean inside it, then calculate Var and MAD, and then take the maximum value of the Var value and MAD value of the 25 search boxes as MVar and MMAD respectively, and finally, use these two eigenvalues ​​to express the maximum local texture; The specific expression is as follows: Among them, Mean (x,y) 、Var (x,y) and MAD (x,y) The mean, variance and absolute mean deviation of the search box are shown respectively, and (x, y) represents the position of the search box.

4. The method for accelerating 32×32 coding unit partitioning and selection in general video coding as claimed in claim 2 or 3, characterized in that: In step S2, the random forest classifier model construction method is specifically as follows: S21, select some frames of the VVC standard video sequence and divide and encode them using the standard encoding scheme, count the division results of 32×32 coding units, and use 4×4 coding units as primitives for feature extraction to form an extraction matrix of texture features, and finally construct a data set according to <feature, division result>; S22, build a random forest classifier model with the following parameters; Number of forests: 100 Method: Classification Maximum number of splits: 5 Minimum number of leaf nodes: 1 Classification principle: Gini coefficient S23, dividing the data set of step S21 into a training set and a test set, training the random forest classifier model driven by the training set, and performing performance testing using the test set until the results converge or the maximum number of training rounds is reached.

5. The method for accelerating 32×32 coding unit partitioning and selection in general video coding as claimed in claim 4, characterized in that: In step S2, the heuristic selection model construction method is specifically as follows: The MAD of the undivided sub-coding blocks generated after the 32×32 coding unit division is counted to determine the threshold in the selection model; there are 5 sizes of sub-coding blocks counted, namely 16×16, 8×32, 32×8, 16×32 and 32×16; these 5 sizes are divided into 256 and 512 categories according to the number of points. The statistical process needs to be converted into a feature extraction matrix first, and then the MAD of the feature extraction matrix is ​​calculated and the division results are recorded.

6. The method for accelerating 32×32 coding unit partitioning and selection in general video coding as claimed in claim 5, characterized in that: Step S3 is as follows: S31, using a 4×4 coding unit as a primitive for feature extraction to form a texture feature extraction matrix, and calculating five features of the current 32×32 coding unit; S32, inputting the five features of step S31 into the random forest classification model obtained in step S23, and judging whether to continue the classification according to the output result of the random forest classification model; S33, for the 32×32 coding unit that needs to be further divided, first pre-assume division is performed on the coding unit, and the MAD of each sub-coding block under different pre-division types is calculated; S34, comparing the MAD value of the sub-coding block with the threshold of the selection model to determine whether the coding unit meets the early exit determination process; S35, for samples that do not conform to the selection model, add the sub-coding block MADs obtained by the pre-hypothesis partitioning, and take the partition type corresponding to the minimum value as the final partition selection. The expression is as follows: HB=MAD HBT1 +MAD HBT2 VB=FOOD VBT1 +FOOD VBT2 HT=FOOD HTT1 +FOOD HTT2 +FOOD HTT3 VT=FOOD VTT1 +FOOD VTT2 +FOOD VTT3 Q=FOOD QT1 +FOOD QT2 +FOOD QT3 +FOOD QT4 Among them, HB, VB, HT, VT and Q represent the sum of the sub-coding block MAD after different pre-hypothesis divisions, MAD HBT1 and MAD HBT2 represents the MAD of the two sub-coding blocks after the pre-assumption level 2 division, MAD VBT1 and MAD VBT2 It represents the MAD of the two sub-coding blocks after the pre-assumed vertical two-division, MAD HTT1 、MAD HTT2 and MAD HTT3 represents the MAD of the three sub-coding blocks after the pre-assumption level three division, MAD VTT1 、MAD VTT2 and MAD VTT3 It represents the MAD of the three sub-coding blocks after the pre-assumed vertical three-division, MAD WT1 、MAD QT2 、MAD QT3 and MAD QT4 Represents the MAD of the four sub-coding blocks after the pre-assumed quadtree QT partition.

7. The method for accelerating 32×32 coding unit partitioning and selection in general video coding as claimed in claim 6, characterized in that: In the process of building the heuristic selection model, the MAD and division results of the two undivided sub-coding blocks with different numbers of points generated after the 32×32 coding unit division are statistically determined to determine the threshold in the selection model, as follows: S241, grouping according to the QP value of the encoding parameter, sorting the MAD values ​​in each group from small to large, and calculating the average of the 50%-75% ranking part as the threshold value for determining whether the sub-coding block of the type corresponding to the QP parameter needs to be further divided; S242, different parameters QP will generate corresponding thresholds, and the threshold and the parameter are combined into a linear parametric equation, thereby deducing the relationship between the threshold and the parameter QP. The equation expression is as follows: TH1=α×QP+β TH2=γ×QP+δ Among them, the number of sub-coding block points corresponding to TH1 is 512, and the number of sub-coding block points corresponding to TH2 is 256; S243, derived parameters α = 12.10, β = 6.38; γ = 1.48, δ = 16.

95.

8. The method for accelerating 32×32 coding unit partitioning and selection in general video coding as claimed in claim 7, characterized in that: The specific process of step S33 is as follows: S331, when performing pre-hypothesis partitioning, a total of three types of pre-hypothesis partitioning are required, namely, horizontal partitioning, vertical partitioning and quadtree QT partitioning. When the first type of partitioning cannot jump out of the selection model, subsequent pre-hypothesis partitioning judgments are performed in sequence; S332, the pre-hypothesis partitioning needs to calculate the MAD of the sub-coding blocks of the coding unit, and the horizontal pre-hypothesis partitioning and the vertical pre-hypothesis partitioning are calculated first; the MAD calculated in this step (1,1) 、MAD (1,5) 、MAD (5,1) 、MAD (5,5) The MAD of the four sub-coding blocks corresponding to the pre-assumption QT division; the specific calculation expression of the sub-coding block MAD generated by the horizontal pre-assumption division is as follows: Where w and h represent the width and height of the feature extraction matrix, respectively; h / 2 and h / 4 represent the heights of different sub-coding blocks after pre-hypothesis division, respectively. and They respectively represent the average values ​​of the internal computing units of different sub-coding blocks after the pre-hypothesis division; S333, after all the pre-hypothesized divisions, the MAD of all sub-coding blocks is calculated: MAD HBT1 、MAD HBT2 、MAD HTT1 、MAD HTT2 、MAD HTT3 、MAD VBT1 、MAD VBT2 、MAD VTT2 、MAD VTT1 、MAD VTT3 、MAD QT1 、MAD QT2 、MAD Q13 、MAD QT4 .

9. The method for accelerating 32×32 coding unit partitioning and selection in general video coding as claimed in claim 8, characterized in that: The specific process of step S34 is as follows: S341, determining the probability of their division by comparing with thresholds TH1 and TH2; S342, the conditions for the horizontal pre-hypothesis partition to jump out of the selection model in advance are: 1) If MAD HBT1 < TH1 and MAD HBT2 < TH1 holds, then perform horizontal two-way partitioning; 2) If MAD HBT1 <TH1 or MAD HBT2 <TH1, but MAD HTT2 > TH1, then perform horizontal two-way partitioning; 3) If MAD HBT1 <TH1 or MAD HBT2 <TH1, but MAD HTT2 <TH1, then perform a horizontal three-way partition; 4) If MAD HBT1 > TH1 and MAD HBT2 > TH1, and MAD HTT1 < TH2 or MAD HTT2 < TH1 or MAD HTT3 < TH2, then perform horizontal three-way partitioning; S343, the conditions for vertical pre-hypothesis partitioning to jump out of the selection model in advance are: 1) If MAD VBT1 < TH1 and MAD VBT2 < TH1 holds, then perform vertical bisection; 2) If MAD VBT1 <TH1 or MAD VBT2 <TH1, but MAD VTT2 > TH1, then perform vertical bisection; 3) If MAD VBT1 <TH1 or MAD VBT2 <TH1, but MAD VTT2 <TH1, then perform vertical three-partitioning; 4) If MAD VBT1 > TH1 and MAD VBT2 > TH1, and MAD VTT1 < TH2 or MAD VTT2 < TH1 or MAD VTT3 < TH2, then perform vertical three-partitioning; S344, QT pre-assumption partition early jump out of the selection model conditions are: If MAD QT1 、MAD QT2 、MAD QT3 、MAD QT4 If any of is less than TH2, QT division is performed.

10. A system for accelerating 32×32 coding unit partitioning and selection in general video coding, used to execute the method according to any one of claims 1 to 9, characterized in that: Includes the following modules: Feature extraction module: A texture feature extraction matrix is ​​constructed using 4×4 coding units as primitives for feature extraction, and the range, variance, absolute mean deviation, and maximum local texture of the extraction matrix are used as features for determining whether to further divide the coding unit. Model building module: using the obtained features as input, constructing a random forest classifier model for determining whether the coding unit needs to be further divided and a heuristic selection model for the division type; Coding unit division module: The random forest classifier model is used to determine whether the coding unit needs to be further divided. If so, the heuristic selection model is used to obtain the division type of the coding unit.

Images