A machine learning based vvc fast intra coding mode decision method

Abstract

The application claims a machine learning-based VVC fast intra coding mode decision method, belonging to the field of video coding, which comprises the following steps: obtaining the texture information, sub-block texture information and domain information of the current to-be-coded CU for LightGBM model prediction; firstly, using the trained LightGBM model to predict the probabilities of three categories, namely 0 (angle prediction mode), 1 (ISP mode) and 2 (MIP mode); comparing the probabilities with the threshold value, and removing the categories with probabilities lower than the threshold value, so as to reduce the number of candidate modes that need to perform complete rate-distortion optimization process; in addition, in the case that category 0 is retained, further using another LightGBM model to predict the probabilities of 67 angle prediction modes, and removing the prediction modes with lower probabilities. The application can significantly save the intra coding time of H.266 / VVC under the premise of ensuring the video quality, and is suitable for video coding application scenarios with high real-time requirements.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of video coding, more specifically, a fast intra coding mode decision method suitable for Versatile Video Coding (VVC) can be applied to video coding scenarios with high real-time requirements. BACKGROUND

[0002] With the popularity of Internet applications, users' expectations for video are constantly increasing, and they hope to obtain more realistic and clear visual experience, and expect higher level of smooth performance of video playback. In order to meet the needs of users, various emerging video applications have appeared, such as Ultra High Definition (UHD), High Frame Rate (HFR), High Dynamic Range (HDR), Wide Color Gamut (WCG), Virtual Reality (VR) and Augmented Reality (AR). However, the surge of video data has put tremendous pressure on limited communication transmission bandwidth and storage resources, so the Joint Video Experts Team (JVET) developed the next generation video coding standard VVC (Versatile Video Coding). Compared with its predecessor, High Efficiency Video Coding (HEVC), VVC adopts a variety of new coding technologies, such as Quad-tree Plus Multi-type Tree (QTMT) based Coding Unit (CU) partition structure, more angle prediction modes (increased from 35 in HEVC to 67 in VVC), Wide Angle Intra Prediction (WAIP), Position Dependent Intra Prediction Combination (PDPC), Multiple Reference Line (MRL) prediction, Intra Sub-Partitions Mode (ISP) and Matrix weighted Intra Prediction (MIP) and other technologies. These new coding technologies make the coding efficiency of VVC increase by nearly 50%, but at the same time the coding complexity also increases significantly. There are two main reasons for the significant increase in coding complexity, QTMT based CU partition structure and Rate Distortion Optimization (RDO) based intra coding mode search process, and the present invention focuses on reducing the complexity of intra coding mode search.The RDO-based intra coding mode search process performs a complete RDO process (including prediction, transformation, quantization and entropy coding processes) on each mode in the candidate mode list (CML), and calculates the rate-distortion cost of each mode, wherein the mode with the lowest rate-distortion cost is selected as the optimal intra coding mode. This brute-force search process greatly increases the encoding complexity. Therefore, the present application reduces the number of modes in the candidate mode list to reduce the encoding complexity.

[0003] The method LightGBM model of the present application predicts the intra coding mode, removes the redundant intra coding mode using the prediction result, reduces the calculation complexity of the RDO process, and significantly reduces the encoding time of VVC intra coding. First, the original encoder is used to encode the video sequence, and the feature data and intra coding mode are recorded in this process, and the corresponding data set is established for training the Light Gradient Boosting Machine (LightGBM) model; secondly, the hyperparameters of the LightGBM model training are determined, and the data set is loaded to train the model; finally, the CU intra coding mode is predicted using the encoder loaded model, and the redundant RDO process is removed using the prediction result. The present application can be used for encoding under the configuration of VVC intra coding, and significantly reduces the encoding time of VVC under the premise of ensuring little loss of encoding efficiency. SUMMARY

[0004] The present application aims to solve the above problems of the prior art. A VVC fast intra coding mode decision method based on machine learning is proposed. The technical solution of the present application is as follows:

[0005] A VVC fast intra coding mode decision method based on machine learning, comprising the following steps:

[0006] S1, selecting the CU feature for training the Light Gradient Boosting Machine (LightGBM) model 1, which realizes the probability prediction of 3 categories, i.e. angle prediction mode, Intra Sub-Partitions Mode (ISP) mode, and Matrix weighted Intra Prediction (MIP) mode; the CU feature includes 2 categories of features, i.e. texture information and sub-block texture information;

[0007] S2, select CU features for training the LightGBM model 2, which realizes the probability prediction of 67 prediction modes (including DC, planar and 65 different angles) in the angle prediction mode; the CU features include 3 categories of features, which are texture information, sub-block texture information and context information;

[0008] S3, encode the video sequence using the modified VVC test reference model encoder, collect the texture and intra coding mode data of the CU, establish the training data set, and train the LightGBM model 1 and the LightGBM model 2;

[0009] S4, use the trained LightGBM model 1 and the LightGBM model 2 in step S3 to predict the CU intra coding mode, skip the rate distortion optimization (RDO) process of the redundant CU coding mode, and realize the fast intra coding mode decision of the CU.

[0010] Further, in the step S1, the texture information includes horizontal and vertical gradients, gradient ratio, variance, entropy and kurtosis of the CU;

[0011] The horizontal and vertical gradients are calculated using the Sobel operator, and the calculation formula is as follows:

[0012]

[0013] G h (x,y) and G v (x,y) in formula (1) represent the horizontal gradient and the vertical gradient of the pixel at coordinate (x,y); A(x,y) represents the pixel matrix centered at coordinate (x,y); then, the average gradient of the current CU pixel in the horizontal direction and the vertical direction is calculated by formula (2);

[0014]

[0015] |G h (x,y)| and |G v (x,y)| in formula (2) are the absolute values of the horizontal gradient and the vertical gradient of the pixel at coordinate (x,y); Gradient x and Gradient y are the horizontal average gradient and the vertical average gradient of the current CU, respectively;

[0016] The ratio of the horizontal and vertical gradients is calculated as follows:

[0017]

[0018] GradientRatio = GradientY / GradientX (1) wherein, GradientRatio represents the ratio of the horizontal gradient and the vertical gradient of the current CU;

[0019] The variance calculation formula is as follows:

[0020]

[0021] wherein, P(x, y) is the pixel value of the luminance component of the CU at (x, y) coordinates; W and H represent the width and height of the CU respectively; Mean is the average value of the pixels of the CU; Variance is the pixel variance of the CU;

[0022] The entropy calculation formula is as follows:

[0023]

[0024] wherein, L is the total number of gray levels in the luminance component of the CU; P(i) is the normalized probability of the gray level i; Entropy is the entropy of the CU image;

[0025] The kurtosis calculation formula is as follows:

[0026]

[0027] wherein, N is the total number of pixels in the CU; L is the total number of gray levels; x i is the number of pixels of the gray level i; μ is the average value of the gray levels of the CU; σ is the standard deviation of the gray levels of the CU; Kurtosis is the kurtosis value of the CU.

[0028] Further, in the step S1, the sub-block texture information includes: sub-block variance and variance of the sub-block variance; the sub-block variance is obtained by dividing the current CU to be encoded into 4 blocks horizontally and vertically, and calculating the variance of each sub-block by using formula (4), and the calculation formula of the variance of the sub-block variance is as follows:

[0029]

[0030] wherein, Variance i represents the variance of the i-th sub-block after the CU is divided into 4 blocks horizontally / vertically; Mean sub-block represents the average value of the sub-block variance; Variance sub-block represents the variance of the sub-block variance.

[0031] Further, in the step S2, the texture information and the sub-block texture information are the same as those in the step S1, and the context information includes: the average values of the pixels of the upper and left reference lines, and the calculation formula is as follows:

[0032]

[0033] wherein, N represents the total number of reference line pixels; P i represents the i-th pixel value; Mean left represents the pixel mean of the left reference line of the current CU; Mean top represents the pixel mean of the upper reference line of the current CU.

[0034] Further, in the step S3, the encoding data of the CU is obtained by encoding the picture or video sequence using the modified VTM encoder, and the feature data and the intra coding mode of the CU are recorded in the process; wherein the picture sequence used for collecting samples contains 6 resolutions, which are: 416x240, 832x480, 1280x720, 1920x1080, 2560x1600 and 3840x2160.

[0035] Further, in the step S3, the training parameters of the LightGBM model are set as: num_leaves=31, learning_rate=0.1, early_stop_rounds=5; wherein, num_leaves is the maximum number of leaves of each tree, which is used to control the complexity of the model, a higher value can improve the fitting ability of the model, and a lower value can reduce the overfitting of the model; learning_rate is the learning rate of the model training, which is used to control the learning speed of the model; early_stop_rounds is the early stop round number of the model training, when the loss of the model does not change in the specified round of training process, the training process of the model is stopped.

[0036] Further, in the step S4, the trained LightGBM model is used to predict the intra coding mode of the CU, and the optimal intra coding mode of the CU or the redundant intra coding mode of the CU is selected, and the specific process of the fast intra coding mode decision of the CU is as follows:

[0037] S41, obtain a candidate mode list through the coarse screening process of the original encoder, and enter step S42;

[0038] S42, check whether the current CU to be encoded is a luminance component, and whether the size of the CU meets the following size requirements: 32x32, 32x16, 16x32, 32x8, 8x32, 32x16, 16x32, 16x16, 16x8, 8x16, 8x8; if it meets, enter step S43; otherwise, enter step S45;

[0039] S43, extract the texture information of the current CU, the sub-block texture information as a feature, use the LightGBM model 1 to predict the probability of the angle mode, the ISP mode and the MIP mode three categories; compare the probabilities of the three categories with the threshold value respectively, if less than the threshold value, remove the mode of the corresponding category in the candidate mode list; otherwise, keep the mode of the corresponding category, enter step S44;

[0040] S44, if the probability of the angle mode category is greater than the threshold value, extract the texture information of the current CU, the sub-block texture information and the context texture information as a feature, use the LightGBM model 2 to predict the probability of 67 angle prediction modes; and compare the angle mode in the candidate mode list obtained in step S41 with the probability, only keep the mode located in Top-3, enter step S45;

[0041] S45, use the complete rate-distortion optimization process of the original encoder to calculate the rate-distortion cost of each mode in the candidate mode list, and select the mode with the lowest rate-distortion cost as the optimal intra coding mode, enter step S46;

[0042] S46, end the intra coding mode search process of the current CU.

[0043] A storage medium internally stores a computer program, which is read by a processor to execute the machine learning-based VVC intra fast coding mode decision method of any one of the above.

[0044] The advantages and beneficial effects of the present application are as follows:

[0045] The present application proposes a machine learning-based VVC fast intra coding mode decision method aiming at the problem of high intra coding complexity and long encoding time of the video encoder implemented by the VVC standard. First, the original encoder is used to encode the video sequence, and the feature data and intra coding mode are recorded in this process, and the corresponding data set is established for training the Light Gradient Boosting Machine (LightGBM) model; secondly, the hyperparameters of the LightGBM model training are determined, and the data set is loaded to train the model; finally, the encoder loads the model to predict the CU intra coding mode, and uses the prediction result to remove the redundant RDO process. Through the above steps, the VVC fast intra coding mode decision can be realized.

[0046] The method of the present application completes the processes of feature design, data set construction, model training and candidate mode list pruning. The method adopted uses two trained LightGBM models to predict the CU intra mode, and uses the prediction results to remove the intra coding modes with low possibility, which can effectively simplify the CU intra coding process and reduce the coding complexity. The present application makes full use of the texture information of the CU as the feature to train the LightGBM model, and effectively uses two LightGBM models to complete different prediction tasks, which can more accurately remove the redundant intra coding modes, while ensuring the video coding quality, significantly reducing the coding time, therefore, the method in the present application has strong innovation and application value. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a flowchart of a VVC fast intra coding mode decision method based on machine learning provided by the present application;

[0048] Figure 2 is a VVC intra coding mode decision flowchart. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. The described embodiments are only a part of the embodiments of the present application.

[0050] The technical solution of the present application to solve the above technical problems is:

[0051] Figure 1 is a flowchart of a VVC fast intra coding mode decision method based on machine learning in the present application, the method of the present application comprises the following steps:

[0052] S1, select the CU feature for training the LightGBM model 1, the model realizes the probability prediction of 3 categories, i.e. angle prediction mode, intra sub-partition (ISP) mode, and matrix weighted intra prediction (MIP) mode; the CU feature includes 2 categories of features, which are texture information and sub-block texture information;

[0053] S2, select the CU feature for training the LightGBM model 2, the model realizes the probability prediction of 67 prediction modes in the angle prediction mode (including DC, plane and 65 different angles); the CU feature includes 3 categories of features, which are texture information, sub-block texture information and context information;

[0054] S3, encode the video sequence using the modified VVC test reference model encoder, collect data such as texture and intra coding mode of CU, establish training data set, and train two LightGBM models;

[0055] S4, use the LightGBM model trained in step S3 to predict the intra coding mode of CU, skip the rate distortion optimization (RDO) process of redundant CU coding mode, and realize fast intra coding mode decision of CU.

[0056] Preferably, in step S1, the texture information includes horizontal and vertical gradients, gradient ratio, variance, entropy and kurtosis of the CU.

[0057] The horizontal and vertical gradients are calculated using Sobel operator, and the calculation formula is as follows:

[0058]

[0059] G h (x,y) and G v (x,y) respectively represent the horizontal gradient and the vertical gradient of the pixel at the coordinate (x, y); A(x, y) represents the pixel matrix centered at the coordinate (x, y); then, the average gradient of the current CU in the horizontal direction and the vertical direction is calculated by using formula (2);

[0060]

[0061] |G h (x,y)| and |G v (x,y)| respectively represent the absolute value of the horizontal gradient and the vertical gradient of the pixel at the coordinate (x, y); Gradient x and Gradient y respectively represent the horizontal average gradient and the vertical average gradient of the current CU;

[0062] The ratio of the horizontal and vertical gradients is calculated as follows:

[0063]

[0064] GradientRatio represents the ratio of the horizontal and vertical gradients of the current CU;

[0065] The variance calculation formula is as follows:

[0066]

[0067] where P(x, y) is the pixel value of the luma component of the CU at (x, y) coordinate; W and H represent the width and height of the CU respectively; Mean is the pixel mean value of the CU; Variance is the pixel variance of the CU;

[0068] The calculation formula of the entropy is as follows:

[0069]

[0070] where L is the total number of gray levels in the luma component of the CU; P(i) is the normalized probability of the gray level i; and Entropy is the entropy of the CU image.

[0071] The calculation formula of the kurtosis is as follows:

[0072]

[0073] where N is the total number of pixels in the CU; L is the total number of gray levels; x i is the number of pixels of the gray level i; μ is the gray mean value of the CU; σ is the gray standard deviation of the CU; and Kurtosis is the kurtosis value of the CU.

[0074] Preferably, in the step S1, the sub-block texture information includes: sub-block variance and variance of the sub-block variance; the sub-block variance is obtained by dividing the current CU to be encoded into 4 blocks horizontally and vertically, and calculating the variance of each sub-block by using the formula (4); and the calculation formula of the variance of the sub-block variance is as follows:

[0075]

[0076] where Variance i represents the variance of the i-th sub-block after the CU is divided into 4 blocks horizontally / vertically; Mean sub-block represents the mean value of the sub-block variance; and Variance sub-block represents the variance of the sub-block variance.

[0077] Preferably, in the step S2, the texture information and the sub-block texture information are the same as those in the step S1, and the context information includes: the mean values of the pixels of the upper and left reference lines, and the calculation formula is as follows:

[0078]

[0079] where N represents the total number of the pixels of the reference line; P i represents the i-th pixel value; Mean left represents the mean value of the pixels of the left reference line of the current CU; and Mean top represents the mean value of the pixels of the upper reference line of the current CU.

[0080] Preferably, in the step S3, the encoding data of the CU is acquired by encoding the picture or video sequence using the modified VTM encoder, and the feature data and the intra coding mode of the CU are recorded in the process; wherein the picture sequence used for collecting samples contains 6 resolutions, which are 416x240, 832x480, 1280x720, 1920x1080, 2560x1600 and 3840x2160 respectively;

[0081] In the step S3, the training parameters of the LightGBM model are set as: num_leaves=31, learning_rate=0.1, early_stop_rounds=5; wherein num_leaves is the maximum number of leaves of each tree, which is used to control the complexity of the model, a higher value can improve the fitting ability of the model, and a lower value can reduce the overfitting of the model; learning_rate is the learning rate of the model training, which is used to control the learning speed of the model; early_stop_rounds is the early stop round number of the model training, when the loss of the model does not change in the specified round of training process, the training process of the model is stopped.

[0082] Preferably, in the step S4, the trained LightGBM model is used to predict the intra coding mode of the CU, and the optimal intra coding mode of the CU or the redundant intra coding mode of the CU is selected, and the specific process of the fast intra coding mode decision of the CU is as follows:

[0083] S41, obtain a candidate mode list through the coarse screening process of the original encoder, and enter step S42;

[0084] S42, check whether the current CU to be encoded is a luminance component and whether the size of the CU meets the following size requirements: 32x32, 32x16, 16x32, 32x8, 8x32, 32x16, 16x32, 16x16, 16x8, 8x16, 8x8; if yes, enter step S43; otherwise, enter step S45;

[0085] S43, extract the texture information and sub-block texture information of the current CU as features, and use the LightGBM model 1 to predict the probabilities of the angle mode, the ISP mode and the MIP mode; compare the probabilities of the three categories with the threshold value respectively, if less than the threshold value, remove the mode of the corresponding category in the candidate mode list; otherwise, keep the mode of the corresponding category, and enter step S44;

[0086] S44, if the probability of the angle mode category is greater than a threshold, extracting the texture information of the current CU, the sub-block texture information and the context texture information as features, using a LightGBM model 2 to predict the probabilities of 67 angle prediction modes; and comparing the angle modes in the candidate mode list obtained in step S41 with the probabilities, only keeping the modes in the Top-3, entering step S45;

[0087] S45, using the complete rate-distortion optimization process of the original encoder to calculate the rate-distortion cost of each mode in the candidate mode list, and selecting the one with the lowest rate-distortion cost as the optimal intra coding mode, entering step S46;

[0088] S46, ending the intra coding mode search process of the current CU.

[0089] The systems, apparatuses, modules or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions.

[0090] The computer readable medium includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition in this paper, the computer readable medium does not include transitory computer readable medium, such as modulated data signal and carrier wave.

[0091] It should also be noted that the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, product or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, product or device. Without more limitation, the element defined by the statement "including a" does not exclude the existence of other identical elements in the process, method, product or device including the element.

[0092] The above examples are to be understood only as illustrative of the present application and not restrictive of the scope of the present application. After reading the foregoing disclosure, many modifications and variations of the present application will be apparent to the skilled person. Such variations and modifications are to be considered as falling within the scope of the present application as defined by the claims.

Claims

1. A method for machine learning based VVC fast intra coding mode decision, characterized in that, The method comprises the following steps: S1, selecting CU features for training a LightGBM model 1, which realizes probability prediction of 3 categories, i.e., angle prediction mode, ISP mode and MIP mode; the CU features include 2 categories of features, i.e., texture information and sub-block texture information; the texture information includes horizontal and vertical gradients, gradient ratio, variance, entropy and kurtosis of the CU; the sub-block texture information includes sub-block variance and variance of the sub-block variance; the context information includes average values of reference row pixels on the top and left sides; S2, selecting CU features for training a LightGBM model 2, which realizes probability prediction of 67 prediction modes in the angle prediction mode; the 67 prediction modes include DC, planar and 65 different angles; the CU features include 3 categories of features, i.e., texture information, sub-block texture information and context information; S3, using a modified VVC test reference model encoder to encode a video sequence, collecting texture and intra coding mode data of CUs, establishing a training data set, and training the LightGBM model 1 and the LightGBM model 2; S4, using the trained LightGBM model 1 and the LightGBM model 2 in step S3 to predict the intra coding mode of the CU, skipping the rate-distortion optimization (RDO) process of the redundant CU coding mode, and realizing fast intra coding mode decision of the CU; In the step S3, the training parameters of the LightGBM model are set as follows: num_leaves=31, learning_rate=0.1, early_stop_rounds=5; wherein, num_leaves is the maximum number of leaves of each tree, which is used to control the complexity of the model, a higher value can improve the fitting ability of the model, and a lower value can reduce the overfitting of the model; learning_rate is the learning rate of the model training, which is used to control the learning speed of the model; early_stop_rounds is the early stop round number of the model training, when the loss of the model does not change in the specified round of training process, the training process of the model is stopped; In the step S4, the trained LightGBM model is used to predict the intra coding mode of the CU, the optimal CU intra coding mode or the redundant CU intra coding mode is selected, and the specific process of realizing the fast intra coding mode decision of the CU is as follows: S41, obtaining a candidate mode list through a coarse screening process of the original encoder, and entering step S42; S42, checking whether the current CU to be encoded is a luminance component and whether the CU size meets the following size requirements: 32x32, 32x16, 16x32, 32x8, 8x32, 32x16, 16x32, 16x16, 16x8, 8x16, 8x8; if yes, entering step S43; otherwise, entering step S45; S43, extract the texture information and sub-block texture information of the current CU as features, and use the LightGBM model 1 to predict the probabilities of the three categories of angle mode, ISP mode and MIP mode; compare the probabilities of the three categories with the threshold value respectively, if less than the threshold value, remove the modes of the corresponding category in the candidate mode list; otherwise, keep the modes of the corresponding category, and enter step S44; S44, if the probability of the angle mode category is greater than the threshold value, extract the texture information, sub-block texture information and context texture information of the current CU as features, and use the LightGBM model 2 to predict the probabilities of 67 angle prediction modes; and compare the angle modes in the candidate mode list obtained in step S41 with the probabilities, and only keep the modes in the Top-3, and enter step S45; S45, use the complete rate-distortion optimization process of the original encoder to calculate the rate-distortion cost of each mode in the candidate mode list, and select the mode with the lowest rate-distortion cost as the optimal intra coding mode, and enter step S46; S46, end the intra coding mode search process of the current CU.

2. The method of claim 1, wherein the method is based on machine learning. In the step S1, the horizontal and vertical gradients are calculated using the Sobel operator, and the calculation formula is as follows: G in Equation (1) h (x,y) and G v (x,y) respectively represent the horizontal gradient and the vertical gradient of the pixel at the coordinate (x,y); A(x,y) represents a pixel matrix centered at the coordinate (x,y); then, the average gradient of the current CU in the horizontal direction and the vertical direction is calculated by using Equation (2); |G h (x,y) and |G v (x,y) are absolute values of horizontal and vertical gradients of the pixel at (x,y) coordinates, respectively; Gradient x and Gradient y are horizontal and vertical average gradients of the current CU, respectively; The ratio of the horizontal and vertical gradients is calculated as follows: Wherein, GradientRatio represents the ratio of the horizontal and vertical gradients of the current CU; The variance calculation formula is as follows: Wherein, P(x, y) is the pixel value of the brightness component of the CU at (x, y) coordinates; W and H represent the width and height of the CU respectively; Mean is the average value of the CU pixels; Variance is the pixel variance of the CU; The calculation formula of entropy is as follows: Wherein, L is the total number of gray levels in the brightness component of the CU; P(i) is the normalized probability of gray level i; Entropy is the entropy of the CU image; The calculation formula of kurtosis is as follows: where N is the total number of pixels in the CU; L is the total number of gray levels; x i is the number of pixels of gray level i; μ is the mean of the gray scale of the CU; σ is the standard deviation of the gray scale of the CU; Kurtosis is the kurtosis value of the CU.

3. The method of claim 1, wherein the method is based on machine learning. In the step S1, the sub-block variance is calculated by dividing the current CU to be coded into 4 blocks horizontally and vertically, and the variance of each sub-block is calculated using formula (4). The calculation formula of the sub-block variance is as follows: wherein Variance i represents the variance of the i-th sub-block after the CU is equally divided into 4 blocks horizontally / vertically; Mean sub-block represents the mean of the variance of the sub-blocks; Variance sub-block represents the variance of the variance of the sub-blocks.

4. The method of claim 1, wherein the method is based on machine learning. In the step S2, the texture information and sub-block texture information are the same as in step S1, and the calculation formula is as follows: where N represents the total number of reference line pixels; P i represents the i-th pixel value; Mean left represents the pixel mean of the left reference line of the current CU; Mean top represents the pixel mean of the upper reference line of the current CU.

5. The method of claim 1, wherein, In the step S3, the encoding data of the CU is obtained by using the modified VTM encoder to encode the picture or video sequence, and the feature data and intra coding mode of the CU are recorded in this process; wherein the picture sequence used to collect samples contains 6 resolutions, which are: 416*240, 832*480, 1280*720, 1920*1080, 2560*1600 and 3840*2160.

6. A storage medium, which stores a computer program in the inside, characterized by The computer program is read by the processor, and the machine learning based VVC intra fast coding mode decision method of any one of claims 1-5 is executed.

Images