A method and system for CTU division within a high-efficiency video coding frame

Abstract

The present invention proposes a method and system for efficiently dividing CTU within a video coding frame, relates to the technical field of video coding unit division, solves the problems of high computational complexity and long coding time of the current intra-frame CTU division method in HEVC, produces video sets with different resolutions based on an image data set, extracts the intra-frame coding unit CU size division under different quantization parameter QP settings, obtains a division threshold T based on the established data set, determines 64x64 size CU division according to the size relationship between the mean absolute deviation M and T, trains two neural networks through a training set, respectively determines whether a 32x32 size CU and a 16x16 size CU continue to be divided, and finally determines the final CU size for subsequent coding processing, thereby reducing the computational complexity of HEVC intra-frame coding and shortening the coding time.

Description

Technical Field

[0001] The present invention relates to the technical field of video coding unit division, and in particular to a method and system for high-efficiency video coding intra-frame CTU division. Background Art

[0002] The original Coding Tree Unit (CTU) division algorithm in High Efficiency Video Coding (HEVC) is: a CTU contains one or more Coding Units (CU), and the CU size is four types, namely 64x64, 32x32, 16x16, and 8x8. The CTU quadtree traversal algorithm can find the best division method for each CTU and determine the best CU partition, but it greatly increases the encoding complexity. As people's requirements for video quality become higher and higher, it brings challenges to video encoding and decoding of mobile devices. Therefore, it is necessary to reduce the encoding complexity through algorithm optimization so that mobile devices can apply HEVC with a lower threshold.

[0003] There are three main solutions for the current CU fast optimization algorithm in HEVC: traditional methods, machine learning-based methods, and deep learning-based methods. Traditional methods generally calculate the mean and variance of pixels, which can determine the size of the CU in advance and reduce the coding complexity. For example, Jae Myung Ha et al. proposed a texture-based HEVC intra-frame coding CU size fast decision algorithm. The texture feature selects the mean and variance of the pixels in the current CU block. Although this method can reduce the coding complexity, the computational complexity is too large, which has a great impact on the coding performance; the machine learning-based method is simple and direct, and mainly extracts some effective image features to judge the CU complexity. For example, LIU et al. established a three-classification structure for CU scale decision based on support vector machine, which are complex CU, homogeneous CU, and uncertain CU. Although the decision effect is good, the computational complexity is increased; the deep learning-based method is efficient in reducing coding complexity. This method introduces a convolutional network structure to automatically extract good feature values. For example, arid Zaki et al. proposed a CTU fast partitioning framework based on the ResNet-18 convolutional network, and proposed different network structures for three different CU sizes: 64x64, 32x32, and 16x16. Although it can greatly reduce the encoding complexity, the introduced network layers are deeper, which increases the complexity of the network model itself. In addition, the 64x64CU size network of this method is prone to overfitting when training data, which limits the network from performing well.

[0004] Existing patent disclosures also disclose a method for dividing CTUs within a HEVC frame, which uses a quadtree neural network model to process video frames to obtain the quadtree structure of all CTUs in the entire frame, and uses an optimized encoder to encode and divide the structures of all CTU quadtrees in the current frame. The CTUs within the HEVC frame are divided according to the division results, so that the correlation of each division result is strengthened, thereby improving the efficiency of data processing and reducing the encoding time. However, when dividing the CTUs within the HEVC frame, neural network prediction is used in the process of segmenting CUs of different sizes, which has high computational complexity and long encoding time, making it difficult for HEVC to be better applied to mobile devices. Summary of the invention

[0005] In order to solve the problems of high computational complexity and long encoding time of the current CTU division method in HEVC, the present invention proposes a method and system for CTU division within a high-efficiency video coding frame. Based on the correlation between texture complexity and deep learning, the division threshold and neural network are used to judge the CU segmentation situation, thereby reducing the computational complexity of HEVC intra-frame coding and shortening the encoding time.

[0006] In order to achieve the above technical effects, the technical solution of the present invention is as follows:

[0007] A method for high-efficiency video coding intra-frame CTU division includes the following steps:

[0008] S1. Collect image datasets and use them to create video sets of different resolutions;

[0009] S2. Set different quantization parameters QP, extract four cases of the size division of the video intra-coding unit CU in the video set under different quantization parameter QP settings, namely 64x64 size CU, 32x32 size CU, 16x16 size CU, and 8x8 size CU;

[0010] S3. The texture complexity of the current 64x64 CU is characterized by the mean absolute deviation M, and the relationship between the situation where the 64x64 CU is split into 32x32 CUs under different quantization parameter QP settings and M is determined to obtain the division threshold T of the 64x64 CU, and execute step S4;

[0011] S4. Set the initial value of the coding unit depth depth to 0, determine whether M is less than or equal to T, if so, the 64x64 size CU is not split, and the 64x64 size CU is output as the final CU size; otherwise, the coding unit depth depth is increased by 1, the 64x64 size CU is split into 4 32x32 size CUs, and step S5 is executed;

[0012] S5. Build and train the A neural network, input each 32x32 CU into the trained A neural network, and use the A neural network to determine whether the 32x32 CU is split. If so, increase the coding unit depth by one, split the 32x32 CU into four 16x16 CUs, and execute step S6; otherwise, output the 32x32 CU as the final CU size decision;

[0013] S6. Build and train a B neural network, input each 16x16 CU into the trained B neural network, and use the B neural network to determine whether the 16x16 CU is split. If so, increase the coding unit depth by one, split the 16x16 CU into four 8x8 CUs, and output the 8x8 CU as the final CU size decision; otherwise, output the 16x16 CU as the final CU size decision;

[0014] S7. The final CU size obtained in S4 to S6 is determined for subsequent encoding processing.

[0015] In this technical solution, the mean absolute deviation M is first used to characterize the texture complexity of the current 64x64 CU. Based on the relationship between the situation where the 64x64 CU is split into a 32x32 CU under different quantization parameter QP settings and the mean absolute deviation M, the division threshold T of the 64x64 CU is obtained; then, the size comparison relationship between M and T is used to determine whether the 64x64 CU is split into a 32x32 CU, which can terminate the segmentation of the 64x64 CU with simple texture, thereby reducing the computational complexity of the algorithm; then, the correlation between texture complexity and coding unit depth is used to construct and train the A neural network and the B neural network respectively, and the CU is split into 32x32 CUs and the trained A neural network is input, and the A neural network is used to perform the segmentation. A neural network determines whether a 32x32 CU is segmented into a 16x16 CU, and then inputs the segmented 16x16 CU into the trained B neural network. The B neural network is used to determine whether the 16x16 CU is segmented into an 8x8 CU. Different methods are proposed for the segmentation of different CU sizes. The segmentation of a 64x64 CU is determined by the partition threshold T, the segmentation of a 32x32 CU is determined by the A neural network, and the segmentation of a 16x16 CU is determined by the B neural network, thereby ensuring the accuracy of the segmentation of different CU sizes. Finally, the final CU size obtained according to the segmentation of different CU sizes is used for subsequent encoding processing, thereby reducing the computational complexity of HEVC intra-frame encoding and shortening the encoding time.

[0016] Preferably, in step S1, the collected image data set is a RAISE ultra-high definition image set, and four YUV format video sets with different resolutions are produced using the RAISE ultra-high definition image set, and the resolutions are 4928x3264, 2560x1600, 1536x1024, and 704x576, respectively.

[0017] Preferably, in step S3, different quantization parameters QP are set to correspond to different thresholds, 4 video sequences with different resolutions and different video contents are selected, and 4 quantization parameter QP values ​​are used for intra-frame coding to obtain 16 video coding results;

[0018] The average absolute deviation of the 64x64 size CU is divided into the global average absolute deviation, the row pixel average absolute deviation, and the column pixel average absolute deviation. The global average absolute deviation calculation formula of the 64x64 size CU is as follows:

[0019]

[0020] in, MAD It is expressed as the global mean absolute deviation of the 64x64 size CU, X Represented as the pixel value of each row, Y Represented as the pixel value of each column, p ( x,y ) is expressed as ( x,y ), mean Represented as the mean value of all pixels in a 64x64 size CU;

[0021] The calculation formula for the row pixel average absolute value deviation is as follows:

[0022]

[0023] in, Expressed as the row pixel mean absolute deviation, mean y Represented as the mean value of all rows of pixels in a 64x64 size CU;

[0024] The calculation formula for the average absolute value deviation of column pixels is as follows:

[0025]

[0026] in, Expressed as the column pixel mean absolute deviation, mean x It is expressed as the mean of all column pixels in the 64x64 size CU. The final average absolute value deviation formula of the 64x64 size CU is as follows:

[0027]

[0028] in, M Expressed as the mean absolute deviation in the final 64x64 size CU;

[0029] When the 64x64 CU is not segmented, the texture of the CU area is smooth in the horizontal and vertical directions. When the 64x64 CU is segmented, the texture of the CU area is complex in the horizontal and vertical directions. The average absolute value deviation of the current 64x64 CU is used to determine whether the 64x64 CU is segmented, and the critical state of the average absolute deviation value between segmentation and non-segmentation is obtained. The size of the average absolute deviation value when the 64x64 CU is not segmented and the average absolute deviation value when the 64x64 CU is segmented is confirmed, and a value between the two is used as the current selection threshold. The segmentation of the 64x64 CU is different from the standard comparison result and is regarded as an experimental error. The current selection threshold is ±0.05, and the value with the smallest experimental error is selected as the division threshold T; the standard is the 64x64 CU segmentation when the HEVC standard reference software HM16.20 is used to encode the video, which can improve the judgment accuracy of the division threshold T. When the division threshold T is compared with the average absolute deviation M, the segmentation of the 64x64 CU can be made more accurate.

[0030] Preferably, in step S5, the A neural network includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a fully connected layer and an output layer which are sequentially connected; the first convolution block is composed of one convolution layer, and each of the second convolution block, the third convolution block, the fourth convolution block and the fifth convolution block is composed of two convolution layers, and the parameters of the two convolution layers are set the same; the first convolution block is set to 64 convolution kernels, and the size of the convolution kernel is 7x7; the second convolution block is set to 64 convolution kernels, and the size of the convolution kernel is 3x3; the third convolution block is set to 128 convolution kernels, and the size of the convolution kernel is 3x3; the fourth convolution block is set to 256 convolution kernels, and the size of the convolution kernel is 3x3; the fifth convolution block is set to 512 convolution kernels, and the size of the convolution kernel is 3x3;

[0031] The fully connected layer includes two hidden layers. The output layer is activated by the SoftMax function so that the output value of the output layer is mapped between (0, 1). The output value of the output layer is the probability of 32x32 CU segmentation. The category labels of the output value of the output layer are "0" and "1". "0" means no segmentation and "1" means segmentation. The category label corresponding to the maximum output value probability is selected as the prediction result. If the prediction result is segmentation, the coding unit depth depth is increased by one, and the 32x32 CU is segmented into 4 16x16 CUs; ​​otherwise, the 32x32 CU is not segmented.

[0032] Here, by utilizing the correlation between texture complexity and deep learning, an A neural network is introduced to predict whether the 32x32 CU is segmented, thereby avoiding the situation where a threshold is used again to determine the segmentation of the 32x32 CU, and ensuring the accuracy of the judgment of the complex texture area of ​​the 32x32 CU.

[0033] Preferably, in step S5, the video set is randomly divided into a 90% training set, a 5% validation set and a 5% test set, the A neural network is trained by the training set, the A neural network is validated by the validation set, and the A neural network is tested by the test set;

[0034] The loss function used when training A neural network is Loss function ( ) is the classification cross entropy, and the formula is as follows:

[0035]

[0036] Where N is the number of CU blocks in training A neural network, is the i-th prediction of the A neural network model, is the target corresponding to the i-th prediction of the A neural network model; the trained A neural network is tested with the test set, and the prediction accuracy threshold is set. When the A neural network is tested with the test set, if the prediction accuracy of the A neural network is greater than the prediction accuracy threshold, the training is stopped to obtain the trained A neural network. Using the loss function ( ) Train the A neural network. The value of the loss function reflects the gap between the A neural network and the actual data. The lower the loss value, the smaller the gap between the current predicted value and the label value, which facilitates code modification and subsequent optimization.

[0037] Preferably, in step S6, the B neural network includes a first convolution layer, a second convolution layer, a pooling layer, a third convolution layer, a fourth convolution layer, a fully connected layer and an output layer; the first convolution layer is set to 32 convolution kernels, and the size of the convolution kernel is 3x3; the second convolution layer is set to 64 convolution kernels, and the size of the convolution kernel is 3x3; the pooling layer is set to AvgPool operation, and the pooling kernel size is 2x2; the third convolution layer is set to 64 convolution kernels, and the size of the convolution kernel is 2x2; the fourth convolution layer is set to 128 convolution kernels, and the size of the convolution kernel is 2x2;

[0038] The fully connected layer includes two hidden layers, and a random loss with a probability of 50% is made between the second hidden layer and the output layer: the output layer is activated by the Sigmoid function so that the output value of the output layer is between (0, 1). The output value of the output layer is the probability of 16x16 CU segmentation. To determine whether the segmentation probability is greater than 50%, the coding unit depth is increased by one, and the 16x16 CU is segmented into 4 8x8 CUs; ​​otherwise, the 16x16 CU is not segmented.

[0039] Here, by utilizing the correlation between texture complexity and deep learning, a B neural network is introduced to predict whether the 16x16 CU is segmented, thereby avoiding the situation where a threshold is used again to determine the segmentation of the 16x16 CU, and ensuring the accuracy of the judgment of the complex texture area of ​​the 16x16 CU.

[0040] Preferably, in step 6, the video set is randomly divided into a 90% training set, a 5% validation set and a 5% test set, the B neural network is trained by the training set, the B neural network is validated by the validation set, and the B neural network is tested by the test set;

[0041] The loss function formula used when training the B neural network is as follows:

[0042]

[0043] Wherein, n is the number of CU blocks in the training B neural network, x[i] is the i-th target value of the B neural network model, and y[i] is the i-th predicted value of the B neural network model. The trained B neural network is tested with a test set, and a prediction accuracy threshold is set. When the B neural network is tested with the test set, if the prediction accuracy of the B neural network is greater than the prediction accuracy threshold, the training is stopped to obtain a trained B neural network.

[0044] Preferably, the subsequent encoding process is performed in HEVC standard reference software HM16.20.

[0045] The present invention also provides a system for high-efficiency video coding intra-frame CTU division, the system comprising:

[0046] A data set making module, used to collect image data sets and make video sets of different resolutions with the image data sets;

[0047] The coding unit division module is used to set different quantization parameters QP, and extract four cases of the size division of the coding unit CU in the video frame of the video set under different quantization parameter QP settings, namely 64x64 size CU, 32x32 size CU, 16x16 size CU, and 8x8 size CU;

[0048] The division threshold determination module uses the mean absolute deviation M to characterize the texture complexity of the current 64x64 CU, determines the relationship between the situation where the 64x64 CU is split into 32x32 CUs under different quantization parameter QP settings and M, and obtains the division threshold T of the 64x64 CU;

[0049] A judgment module is used to set the initial value of the coding unit depth depth to 0, and judge whether M is less than or equal to T. If so, the 64x64 size CU is not split, and the 64x64 size CU is output as the final CU size decision; otherwise, the coding unit depth depth is increased by one, and the 64x64 size CU is split into four 32x32 size CUs;

[0050] A neural network construction processing module is used to construct and train the A neural network. Each 32x32 CU is input into the trained A neural network. The A neural network is used to determine whether the 32x32 CU is split. If so, the coding unit depth depth is increased by one, and the 32x32 CU is split into four 16x16 CUs; ​​otherwise, the 32x32 CU is output as the final CU size decision;

[0051] The B neural network construction processing module is used to construct and train the B neural network. Each 16x16 size CU is input into the trained B neural network. The B neural network is used to determine whether the 16x16 size is split. If so, the coding unit depth depth is increased by one, the 16x16 size CU is split into four 8x8 size CUs, and the 8x8 size CU is output as the final CU size decision; otherwise, the 16x16 size CU is output as the final CU size decision;

[0052] The encoding processing module is used to perform subsequent encoding processing on the final CU size obtained in the judgment module, the A neural network construction processing module, and the A neural network construction processing module.

[0053] Preferably, the encoding processing module is encapsulated in HEVC standard reference software HM16.20.

[0054] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0055] The present invention proposes a method and system for CTU division within a high-efficiency video coding frame. Compared with the current CTU division method in HEVC, the method proposed by the present invention obtains the division threshold T of the 64x64 CU based on the relationship between the situation that a 64x64 CU is split into a 32x32 CU under different quantization parameter QP settings and the mean absolute deviation M; then, the size comparison relationship between M and T is used to determine whether the 64x64 CU is split into a 32x32 CU, and the 64x64 CU with simple texture can be terminated for segmentation, thereby reducing the computational complexity of the algorithm; then, the correlation between texture complexity and coding unit depth depth is used to construct and train A neural network and B neural network respectively, and the 32x32 CU input is split into a trained CU. A neural network is used to determine whether a 32x32 CU is segmented into a 16x16 CU. The segmented CU is then input into the trained B neural network. The B neural network is used to determine whether the 16x16 CU is segmented into an 8x8 CU. Different methods are proposed for the segmentation of different CU sizes. The segmentation of a 64x64 CU is determined by the partition threshold T, the segmentation of a 32x32 CU is determined by the A neural network, and the segmentation of a 16x16 CU is determined by the B neural network, thereby ensuring the accuracy of the segmentation of different CU sizes. Finally, the final CU size is determined based on the segmentation of different CU sizes for subsequent encoding processing, thereby reducing the computational complexity of HEVC intra-frame encoding and shortening the encoding time. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram showing a flow chart of a method for high-efficiency video coding intra-frame CTU division proposed in Embodiment 1 of the present invention;

[0057] Figure 2 It represents the 64x64 size CU under different quantization parameters QP set in embodiment 1 of the present invention.

[0058] Threshold curve graph;

[0059] Figure 3 A neural network structure diagram of A proposed in Example 2 of the present invention is shown;

[0060] Figure 4 A diagram showing relevant parameters of the A neural network proposed in Example 2 of the present invention;

[0061] Figure 5 A diagram showing the structure of a neural network B proposed in Embodiment 3 of the present invention;

[0062] Figure 6 A structural diagram showing a system for high-efficiency intra-frame CTU division in video coding proposed in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0063] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0064] In order to better illustrate the present embodiment, some parts of the drawings may be omitted, enlarged or reduced, and do not represent actual sizes. The description of the directions of parts such as "upper" and "lower" does not limit the present patent;

[0065] It is understandable to those skilled in the art that some well-known contents may be omitted in the drawings;

[0066] The positional relationships described in the drawings are only for illustrative purposes and should not be construed as limiting the present patent.

[0067] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0068] Example 1

[0069] Figure 1 A flowchart of a method for high-efficiency video coding intra-frame CTU division proposed in an embodiment of the present invention is shown, comprising the following steps:

[0070] S1. Collect image datasets and use them to create video sets of different resolutions;

[0071] There are many methods for producing a video set. In this embodiment, the collected image data set is a RAISE ultra-high-definition image set. Four YUV format video sets with different resolutions are produced using the RAISE ultra-high-definition image set, and the resolutions are 4928x3264, 2560x1600, 1536x1024, and 704x576, respectively; a number of 4928x3264 ultra-high-definition images are selected, and then some photos are downsampled to three resolutions of 2560x1600, 1536x1024, and 704x576, and the YUV format video set of each resolution is randomly divided into 90% training set, 5% validation set, and 5% test set; for the HEVC reference software HM16.20, the All_intra configuration is used, and its quantization parameter QP is set to 22, 27, 32, and 37, and the coding unit division under these four different QP settings is extracted to construct a data set.

[0072] S2. Set the quantization parameter QP to 22, 27, 32, and 37, and extract four cases of the size division of the video intra-coding unit CU in the video set with the quantization parameter QP of 22, 27, 32, and 37, which are 64x64 size CU, 32x32 size CU, 16x16 size CU, and 8x8 size CU respectively;

[0073] S3. The texture complexity of the current 64x64 CU is characterized by the mean absolute deviation M, and the relationship between the situation where the 64x64 CU is split into a 32x32 CU when the quantization parameter QP is 22, 27, 32, and 37 and M is determined to obtain the division threshold T of the 64x64 CU, and execute step S4;

[0074] See also Figure 2 , when the quantization parameter QP is 22, the corresponding threshold is set to 3.112, when the quantization parameter QP is 27, the corresponding threshold is set to 3.592, when the quantization parameter QP is 32, the corresponding threshold is set to 4.056, and when the quantization parameter QP is 37, the corresponding threshold is set to 4.356. Video sequences with resolutions of 4928x3264, 2560x1600, 1536x1024, and 704x576 and different video contents are selected. Four quantization parameter QP values ​​of 22, 27, 32, and 37 are used for intra-frame coding, and 16 video encoding results are obtained;

[0075] The average absolute deviation of the 64x64 size CU is divided into the global average absolute deviation, the row pixel average absolute deviation, and the column pixel average absolute deviation. The global average absolute deviation calculation formula of the 64x64 size CU is as follows:

[0076]

[0077] in, MAD It is expressed as the global mean absolute deviation of the 64x64 size CU, X Represented as the pixel value of each row, Y Represented as the pixel value of each column, p ( x,y ) is expressed as ( x,y ), mean Represented as the mean value of all pixels in a 64x64 size CU;

[0078] The calculation formula for the row pixel average absolute value deviation is as follows:

[0079]

[0080] in, Expressed as the row pixel mean absolute deviation, mean y Represented as the mean value of all rows of pixels in a 64x64 size CU;

[0081] The calculation formula for the average absolute value deviation of column pixels is as follows:

[0082]

[0083] in, Expressed as the column pixel mean absolute deviation, mean x It is expressed as the mean of all column pixels in the 64x64 size CU. The final average absolute value deviation formula of the 64x64 size CU is as follows:

[0084]

[0085] in, M Expressed as the mean absolute deviation in the final 64x64 size CU;

[0086] When the 64x64 CU is not segmented, the texture of the CU area is smooth in the horizontal and vertical directions. When the 64x64 CU is segmented, the texture of the CU area is complex in the horizontal and vertical directions. The average absolute deviation of the current 64x64 CU is used to determine whether the 64x64 CU is segmented, and the critical state of the average absolute deviation between segmentation and non-segmentation is obtained. The size of the average absolute deviation when the 64x64 CU is not segmented and the average absolute deviation when it is segmented is confirmed. The value between the two is used as the current selection threshold. The segmentation of the 64x64 CU is different from the standard comparison result and is regarded as an experimental error. The current selection threshold is ±0.05, and the value with the smallest experimental error is selected as the segmentation threshold T; the standard is the segmentation of the 64x64 CU when the HEVC standard reference software HM16.20 is used to encode the video. The judgment accuracy of the segmentation threshold T is improved, and when the segmentation threshold T is compared with the average absolute deviation M, the segmentation of the 64x64 CU can be made more accurate.

[0087] Under the setting of QP=32, 5 video test sequences with complex textures and simple textures were selected to test the accuracy of threshold judgment. The average accuracy was 73.2% for video sequences with complex textures, and 78.65% for video sequences with simpler textures.

[0088] S4. Set the initial value of the coding unit depth depth to 0, determine whether M is less than or equal to T, if so, the 64x64 size CU is not split, the current coding unit depth depth value is 0, and the 64x64 size CU is output as the final CU size decision; otherwise, the coding unit depth depth is increased by 1, the current coding unit depth depth value becomes 1, the 64x64 size CU is split into 4 32x32 size CUs, and step S5 is executed;

[0089] S5. Build and train the A neural network, input each 32x32 CU into the trained A neural network, and use the A neural network to determine whether the 32x32 CU is split. If so, first add 1 to the coding unit depth of 1, and then the coding unit depth becomes 2. Then split 1 CU of 32x32 into 4 CUs of 16x16, and the remaining 3 CUs of 32x32 implement the same splitting method. Finally, the CUs of 16x16 are input into the B network, and step S6 is executed; otherwise, the current coding unit depth is still 1, and the 32x32 CU is directly output to determine the final CU size.

[0090] S6. Build and train the B neural network, input each 16x16 CU into the trained B neural network, and use the B neural network to determine whether the 16x16 CU is split. If so, first add 1 to the coding unit depth of 2, and the coding unit depth becomes 3. Then split 1 CU of 16x16 into 4 CUs of 8x8, and the other 3 CUs of 16x16 are split in the same way. Finally, the CU split into 8x8 is directly output to determine the final CU size; otherwise, the current coding unit depth is still 2, and the CU of 16x16 is directly output to determine the final CU size.

[0091] S7. The final CU size obtained in S4 to S6 is determined to undergo subsequent encoding processing, and the subsequent encoding processing is performed in the HEVC standard reference software HM16.20.

[0092] In this technical solution, the mean absolute deviation M is first used to characterize the texture complexity of the current 64x64 CU. Based on the relationship between the situation where the 64x64 CU is split into a 32x32 CU under different quantization parameter QP settings and the mean absolute deviation M, the division threshold T of the 64x64 CU is obtained; then, the size comparison relationship between M and T is used to determine whether the 64x64 CU is split into a 32x32 CU, which can terminate the segmentation of the 64x64 CU with simple texture, thereby reducing the computational complexity of the algorithm; then, the correlation between texture complexity and coding unit depth is used to construct and train the A neural network and the B neural network respectively, and the CU is split into 32x32 CUs and the trained A neural network is input, and the A neural network is used to perform the segmentation. A neural network determines whether a 32x32 CU is segmented into a 16x16 CU, and then inputs the segmented 16x16 CU into the trained B neural network. The B neural network is used to determine whether the 16x16 CU is segmented into an 8x8 CU. Different methods are proposed for the segmentation of different CU sizes. The segmentation of a 64x64 CU is determined by the partition threshold T, the segmentation of a 32x32 CU is determined by the A neural network, and the segmentation of a 16x16 CU is determined by the B neural network, thereby ensuring the accuracy of the segmentation of different CU sizes. Finally, the final CU size obtained according to the segmentation of different CU sizes is used for subsequent encoding processing, thereby reducing the computational complexity of HEVC intra-frame encoding and shortening the encoding time.

[0093] Example 2

[0094] Figure 3 It represents the structure diagram of the A neural network proposed in the embodiment of the present invention, such as Figure 3 As shown, the A neural network includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a fully connected layer and an output layer connected in sequence; the first convolution block consists of one convolution layer, and each of the second convolution block, the third convolution block, the fourth convolution block and the fifth convolution block consists of two convolution layers, and the parameters of these two convolution layers are set the same; see Figure 4 , the first convolution block is set to 64 convolution kernels, the size of the convolution kernel is 7x7; the second convolution block is set to 64 convolution kernels, the size of the convolution kernel is 3x3; the third convolution block is set to 128 convolution kernels, the size of the convolution kernel is 3x3; the fourth convolution block is set to 256 convolution kernels, the size of the convolution kernel is 3x3; the fifth convolution block is set to 512 convolution kernels, the size of the convolution kernel is 3x3;

[0095] The fully connected layer includes two hidden layers. The output layer is activated by the SoftMax function so that the output value of the output layer is mapped between (0, 1). The output value of the output layer is the probability of 32x32 CU segmentation. The category labels of the output value of the output layer are "0" and "1". "0" means no segmentation and "1" means segmentation. The category label corresponding to the maximum output value probability is selected as the prediction result. If the prediction result is segmentation, the coding unit depth depth is increased by one, and the 32x32 CU is segmented into 4 16x16 CUs; ​​otherwise, the 32x32 CU is not segmented. By utilizing the correlation between texture complexity and deep learning, an A neural network is introduced to predict whether the 32x32 CU is segmented, thereby avoiding the situation where a threshold is used again to judge the segmentation of the 32x32 CU, thereby ensuring the accuracy of the judgment of the complex texture area of ​​the 32x32 CU. In this embodiment, the accuracy of the A neural network test set is 78.32%.

[0096] See also Figure 1 In step S1, the video set is randomly divided into 90% training set, 5% validation set and 5% test set. In step S5, the A neural network is trained by the training set, the A neural network is validated by the validation set, and the A neural network is tested by the test set. The loss function used in training the A neural network is Loss function ( ) is the classification cross entropy, and the formula is as follows:

[0097]

[0098] Where N is the number of CU blocks in training A neural network, is the i-th prediction of the A neural network model, is the target corresponding to the i-th prediction of the A neural network model; the trained A neural network is tested with the test set, and the prediction accuracy threshold is set. When the A neural network is tested with the test set, if the prediction accuracy of the A neural network is greater than the prediction accuracy threshold, the training is stopped to obtain the trained A neural network. Using the loss function ( ) Train the A neural network. The value of the loss function reflects the gap between the A neural network and the actual data. The lower the loss value, the smaller the gap between the current predicted value and the label value, which facilitates code modification and subsequent optimization.

[0099] Example 3

[0100] like Figure 5As shown in the figure, the B neural network includes the first convolution layer, the second convolution layer, the pooling layer, the third convolution layer, the fourth convolution layer, the fully connected layer and the output layer; the first convolution layer is set to 32 convolution kernels, and the size of the convolution kernel is 3x3; the second convolution layer is set to 64 convolution kernels, and the size of the convolution kernel is 3x3; the pooling layer is set to AvgPool operation, and the pooling kernel size is 2x2; the third convolution layer is set to 64 convolution kernels, and the size of the convolution kernel is 2x2; the fourth convolution layer is set to 128 convolution kernels, and the size of the convolution kernel is 2x2;

[0101] The fully connected layer includes two hidden layers, and there is a 50% probability of random loss between the second hidden layer and the output layer: the output layer is activated by the Sigmoid function, so that the output value of the output layer is between (0, 1), and the output value of the output layer is the probability of 16x16 CU segmentation. It is judged whether the segmentation probability is greater than 50%. If the output layer output value probability is greater than 50%, if so, let the coding unit depth depth increase by one, and the 16x16 CU is segmented into 4 8x8 CUs; ​​otherwise, the 16x16 CU is not segmented. Using the correlation between texture complexity and deep learning, the B neural network is introduced to predict and judge whether the 16x16 CU is segmented, avoiding the situation of taking the threshold to judge the segmentation of the 16x16 CU again, and ensuring the accuracy of the judgment of the complex texture area of ​​the 16x16 CU. In this embodiment, the accuracy of the B neural network test set is 72.16.

[0102] See also Figure 1 , in step S1, the video set is randomly divided into 90% training set, 5% validation set and 5% test set, the B neural network is trained by the training set, the B neural network is validated by the validation set, and the B neural network is tested by the test set;

[0103] The loss function used when training the B neural network is Loss function ( ) is the classification cross entropy, and the formula is as follows:

[0104]

[0105] Wherein, n is the number of CU blocks in the training B neural network, x[i] is the i-th target value of the B neural network model, and y[i] is the i-th predicted value of the B neural network model. The trained B neural network is tested with a test set, and a prediction accuracy threshold is set. When the B neural network is tested with the test set, if the prediction accuracy of the B neural network is greater than the prediction accuracy threshold, the training is stopped to obtain a trained B neural network.

[0106] Example 4

[0107] like Figure 6As shown, a system for efficiently dividing CTUs within a video coding frame includes:

[0108] A data set making module 11 is used to collect image data sets and make video sets of different resolutions with the image data sets;

[0109] The coding unit division module 12 is used to set different quantization parameters QP, and extract four cases of the size division of the coding unit CU in the video frame of the video set under different quantization parameter QP settings, namely, 64x64 size CU, 32x32 size CU, 16x16 size CU, and 8x8 size CU;

[0110] The division threshold determination module 13 uses the mean absolute deviation M to characterize the texture complexity of the current 64x64 size CU, determines the relationship between the situation where the 64x64 size CU is split into 32x32 size CUs under different quantization parameter QP settings and M, and obtains the division threshold T of the 64x64 size CU;

[0111] The judging module 14 is used to set the initial value of the coding unit depth depth to 0, and judge whether M is less than or equal to T. If so, the 64x64 size CU is not split, and the 64x64 size CU is output as the final CU size; otherwise, the coding unit depth depth is increased by one, and the 64x64 size CU is split into four 32x32 size CUs;

[0112] A neural network construction processing module 15 is used to construct and train the A neural network, input each 32x32 size CU into the trained A neural network, and use the A neural network to determine whether the 32x32 size CU is split. If so, the coding unit depth depth is increased by one, and the 32x32 size CU is split into four 16x16 size CUs; ​​otherwise, the 32x32 size CU is output as the final CU size decision;

[0113] The B neural network construction processing module 16 is used to construct and train the B neural network, input each 16x16 size CU into the trained B neural network, and use the B neural network to determine whether the 16x16 size is split. If so, the coding unit depth depth is increased by one, the 16x16 size CU is split into four 8x8 size CUs, and the 8x8 size CU is output as the final CU size decision; otherwise, the 16x16 size CU is output as the final CU size decision;

[0114] The encoding processing module 17 is used to perform subsequent encoding processing on the final CU size obtained in the judgment module, the A neural network construction processing module, and the A neural network construction processing module. The encoding processing module is encapsulated in the HEVC standard reference software HM16.20.

[0115] Obviously, the above embodiments of the present invention are only examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for efficient intra-frame CTU division in video coding. The following steps are involved: S1. Collect image datasets and use them to create video sets of different resolutions; S2. Set different quantization parameters QP, extract four cases of the size division of the video intra-coding unit CU in the video set under different quantization parameter QP settings, namely 64x64 size CU, 32x32 size CU, 16x16 size CU, and 8x8 size CU; S3. The texture complexity of the current 64x64 CU is characterized by the mean absolute deviation M, and the relationship between the situation where the 64x64 CU is split into 32x32 CUs under different quantization parameter QP settings and M is determined to obtain the division threshold T of the 64x64 CU, and execute step S4; In step S3, different quantization parameters QP are set to correspond to different thresholds, 4 video sequences with different resolutions and different video contents are selected, and 4 quantization parameter QP values ​​are used for intra-frame coding to obtain 16 video coding results; The average absolute deviation of the 64x64 size CU is divided into the global average absolute deviation, the row pixel average absolute deviation, and the column pixel average absolute deviation. The global average absolute deviation calculation formula of the 64x64 size CU is as follows: in, MAD It is expressed as the global mean absolute deviation of the 64x64 size CU, X Represented as the pixel value of each row, Y Represented as the pixel value of each column, p ( x,y ) is expressed as ( x,y ), mean Represented as the mean value of all pixels in a 64x64 size CU; The calculation formula for the row pixel average absolute value deviation is as follows: in, Expressed as the row pixel mean absolute deviation, mean y Represented as the mean value of all rows of pixels in a 64x64 size CU; The calculation formula for the average absolute value deviation of column pixels is as follows: in, Expressed as the column pixel mean absolute deviation, mean x It is expressed as the mean of all column pixels in the 64x64 size CU. The final average absolute value deviation formula of the 64x64 size CU is as follows: in, M Expressed as the mean absolute deviation in the final 64x64 size CU; When the 64x64 CU is not segmented, the texture of the CU area is smooth in the horizontal and vertical directions. When the 64x64 CU is segmented, the texture of the CU area is complex in the horizontal and vertical directions. The average absolute value deviation of the current 64x64 CU is used to determine whether the 64x64 CU is segmented, and the critical state of the average absolute deviation value between segmentation and non-segmentation is obtained. The size of the average absolute deviation value when the 64x64 CU is not segmented and the average absolute deviation value when it is segmented is confirmed, and the value between the two is used as the current selection threshold. The segmentation of the 64x64 CU is different from the standard comparison result as an experimental error, and the current selection threshold is ±0.05, and the value with the smallest experimental error is selected as the division threshold T; the standard is the 64x64 CU segmentation when the HEVC standard reference software HM16.20 is used to encode the video; S4. Set the initial value of the coding unit depth depth to 0, determine whether M is less than or equal to T, if so, the 64x64 size CU is not split, and the 64x64 size CU is output as the final CU size; otherwise, the coding unit depth depth is increased by 1, the 64x64 size CU is split into 4 32x32 size CUs, and step S5 is executed; S5. Build and train the A neural network, input each 32x32 CU into the trained A neural network, and use the A neural network to determine whether the 32x32 CU is split. If so, increase the coding unit depth by one, split the 32x32 CU into four 16x16 CUs, and execute step S6; otherwise, output the 32x32 CU as the final CU size decision; S6. Build and train a B neural network, input each 16x16 CU into the trained B neural network, and use the B neural network to determine whether the 16x16 CU is split. If so, increase the coding unit depth by one, split the 16x16 CU into four 8x8 CUs, and output the 8x8 CU as the final CU size decision; otherwise, output the 16x16 CU as the final CU size decision; S7. The final CU size obtained in S4 to S6 is determined for subsequent encoding processing.

2. The method for dividing CTU within a high-efficiency video coding frame according to claim 1, It is characterized in that In step S1, the collected image data set is a RAISE ultra-high-definition image set, and four YUV format video sets with different resolutions are produced using the RAISE ultra-high-definition image set, and the resolutions are 4928x3264, 2560x1600, 1536x1024, and 704x576, respectively.

3. The method for high-efficiency video coding intra-frame CTU division according to claim 2, It is characterized in that In step S5, the A neural network includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a fully connected layer and an output layer which are connected in sequence; the first convolution block is composed of one convolution layer, and each of the second convolution block, the third convolution block, the fourth convolution block and the fifth convolution block is composed of two convolution layers, and the parameters of the two convolution layers are set the same; the first convolution block is set to 64 convolution kernels, and the size of the convolution kernel is 7x7; the second convolution block is set to 64 convolution kernels, and the size of the convolution kernel is 3x3; the third convolution block is set to 128 convolution kernels, and the size of the convolution kernel is 3x3; the fourth convolution block is set to 256 convolution kernels, and the size of the convolution kernel is 3x3; the fifth convolution block is set to 512 convolution kernels, and the size of the convolution kernel is 3x3; The fully connected layer includes two hidden layers. The output layer is activated by the SoftMax function so that the output value of the output layer is mapped between (0, 1). The output value of the output layer is the probability of 32x32 CU segmentation. The category labels of the output value of the output layer are "0" and "1". "0" means no segmentation, and "1" means segmentation. The category label corresponding to the maximum output value probability is selected as the prediction result. If the prediction result is segmentation, the coding unit depth depth is increased by one, and the 32x32 CU is segmented into 4 16x16 CUs; ​​otherwise, the 32x32 CU is not segmented.

4. The method for dividing CTU within a high-efficiency video coding frame according to claim 2, It is characterized in that In step S5, the video set is randomly divided into a 90% training set, a 5% validation set, and a 5% test set, the A neural network is trained by the training set, the A neural network is validated by the validation set, and the A neural network is tested by the test set; The loss function used when training A neural network is Loss function ( L f ) is the classification cross entropy, and the formula is as follows: Where N is the number of CU blocks in training A neural network, is the i-th prediction of the A neural network model, is the target corresponding to the i-th prediction of the A neural network model; the trained A neural network is tested with the test set, and the prediction accuracy threshold is set. When the A neural network is tested with the test set, if the prediction accuracy of the A neural network is greater than the prediction accuracy threshold, the training is stopped to obtain the trained A neural network.

5. The method for high-efficiency video coding intra-frame CTU division according to claim 2, It is characterized in that In step S6, the B neural network includes a first convolution layer, a second convolution layer, a pooling layer, a third convolution layer, a fourth convolution layer, a fully connected layer and an output layer; the first convolution layer is set to 32 convolution kernels, and the size of the convolution kernel is 3x3; the second convolution layer is set to 64 convolution kernels, and the size of the convolution kernel is 3x3; the pooling layer is set to AvgPool operation, and the pooling kernel size is 2x2; the third convolution layer is set to 64 convolution kernels, and the size of the convolution kernel is 2x2; the fourth convolution layer is set to 128 convolution kernels, and the size of the convolution kernel is 2x2; The fully connected layer includes two hidden layers, and a random loss with a probability of 50% is made between the second hidden layer and the output layer: the output layer is activated by the Sigmoid function so that the output value of the output layer is between (0, 1). The output value of the output layer is the probability of 16x16 CU segmentation. It is judged whether the segmentation probability is greater than 50%. If so, the coding unit depth depth is increased by one, and the 16x16 CU is segmented into 4 8x8 CUs; ​​otherwise, the 16x16 CU is not segmented.

6. The method for high-efficiency video coding intra-frame CTU division according to claim 5, It is characterized in that In step 6, the video set is randomly divided into a 90% training set, a 5% validation set, and a 5% test set. The B neural network is trained by the training set, the B neural network is validated by the validation set, and the B neural network is tested by the test set. The loss function formula used when training the B neural network is as follows: Wherein, n is the number of CU blocks in the training B neural network, x[i] is the i-th target value of the B neural network model, and y[i] is the i-th predicted value of the B neural network model. The trained B neural network is tested with a test set, and a prediction accuracy threshold is set. When the B neural network is tested with the test set, if the prediction accuracy of the B neural network is greater than the prediction accuracy threshold, the training is stopped to obtain a trained B neural network.

7. The method for high-efficiency video coding intra-frame CTU division according to claim 1, It is characterized in that Subsequent encoding processing is performed in the HEVC standard reference software HM16.

20.

8. A system for efficient intra-frame CTU division in video coding. It is characterized in that The system comprises: A data set making module, used to collect image data sets and make video sets of different resolutions with the image data sets; The coding unit division module is used to set different quantization parameters QP, and extract four cases of the size division of the coding unit CU in the video frame of the video set under different quantization parameter QP settings, namely 64x64 size CU, 32x32 size CU, 16x16 size CU, and 8x8 size CU; The division threshold determination module uses the mean absolute deviation M to characterize the texture complexity of the current 64x64 CU, determines the relationship between the situation where the 64x64 CU is split into 32x32 CUs under different quantization parameter QP settings and M, and obtains the division threshold T of the 64x64 CU; Among them, different quantization parameter QP settings correspond to different thresholds, 4 video sequences with different resolutions and different video contents are selected, and 4 quantization parameter QP values ​​are used for intra-frame coding, and 16 video coding results are obtained; The average absolute deviation of the 64x64 size CU is divided into the global average absolute deviation, the row pixel average absolute deviation, and the column pixel average absolute deviation. The global average absolute deviation calculation formula of the 64x64 size CU is as follows: in, MAD It is expressed as the global mean absolute deviation of the 64x64 size CU, X Represented as the pixel value of each row, Y Represented as the pixel value of each column, p ( x,y ) is expressed as ( x,y ), mean Represented as the mean value of all pixels in a 64x64 size CU; The calculation formula for the row pixel average absolute value deviation is as follows: in, Expressed as the row pixel mean absolute deviation, mean y Represented as the mean value of all rows of pixels in a 64x64 size CU; The calculation formula for the average absolute value deviation of column pixels is as follows: in, Expressed as the column pixel mean absolute deviation, mean x It is expressed as the mean of all column pixels in the 64x64 size CU. The final average absolute value deviation formula of the 64x64 size CU is as follows: in, M Expressed as the mean absolute deviation in the final 64x64 size CU; When the 64x64 CU is not segmented, the texture of the CU area is smooth in the horizontal and vertical directions. When the 64x64 CU is segmented, the texture of the CU area is complex in the horizontal and vertical directions. The average absolute value deviation of the current 64x64 CU is used to determine whether the 64x64 CU is segmented, and the critical state of the average absolute deviation value between segmentation and non-segmentation is obtained. The size of the average absolute deviation value when the 64x64 CU is not segmented and the average absolute deviation value when it is segmented is confirmed, and the value between the two is used as the current selection threshold. The segmentation of the 64x64 CU is different from the standard comparison result as an experimental error, and the current selection threshold is ±0.05, and the value with the smallest experimental error is selected as the division threshold T; the standard is the 64x64 CU segmentation when the HEVC standard reference software HM16.20 is used to encode the video; A judgment module is used to set the initial value of the coding unit depth depth to 0, and judge whether M is less than or equal to T. If so, the 64x64 size CU is not split, and the 64x64 size CU is output as the final CU size decision; otherwise, the coding unit depth depth is increased by one, and the 64x64 size CU is split into four 32x32 size CUs; A neural network construction processing module is used to construct and train the A neural network. Each 32x32 CU is input into the trained A neural network. The A neural network is used to determine whether the 32x32 CU is split. If so, the coding unit depth depth is increased by one, and the 32x32 CU is split into four 16x16 CUs; ​​otherwise, the 32x32 CU is output as the final CU size decision; The B neural network construction processing module is used to construct and train the B neural network. Each 16x16 size CU is input into the trained B neural network, and the B neural network is used to determine whether the 16x16 size is split. If so, the coding unit depth depth is increased by one, and the 16x16 size CU is split into four 8x8 size CUs, and the 8x8 size CU is output as the final CU size decision; otherwise, the 16x16 size CU is output as the final CU size decision; The encoding processing module is used to perform subsequent encoding processing on the final CU size obtained in the judgment module, the A neural network construction processing module, and the A neural network construction processing module.

9. The system for high-efficiency video coding intra-frame CTU division according to claim 8, It is characterized in that The encoding processing module is encapsulated in the HEVC standard reference software HM16.20.

Images