Video coding and block division method and device, equipment and storage medium

Abstract

The invention provides a video coding and block division method and device, equipment and a storage medium, and belongs to the technical field of information processing. The method comprises the following steps: in the process of coding a video image, under the condition that a coding result of a pixel block in the video image is obtained every time, generating training data based on the coding result, and performing online learning on a classification model corresponding to each block division hierarchy by adopting the training data to obtain a classification model corresponding to each block division hierarchy; the obtained classification model corresponding to each block division hierarchy meets a training ending condition; in the process of coding uncoded pixel blocks in a video image, before block division is carried out each time, recursive division prediction is carried out by adopting a classification model corresponding to the current division hierarchy, and the next hierarchy division is carried out on the current pixel block under the condition that the prediction result is that recursive division is needed, so that the next hierarchy division is carried out. And terminating the block division process under the condition that the prediction result is that recursive division is not needed. The method is used for improving block division efficiency and reducing coding complexity.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a video encoding and block division method, device, equipment and storage medium. Background Art

[0002] Video compression plays an indispensable role in multimedia applications. At present, various video codec standards are widely used, such as HEVC (High Efficiency Video Coding), VP9 (Next Gen OpenVideo) and AVS (Audio Video Coding Standard), all of which aim to achieve high compression efficiency while limiting computational complexity. AV1 is an open source video compression standard launched by the Alliance for Open Media (AOM). Its performance is greatly improved compared to HEVC, especially in processing ultra-high-definition videos such as 4K resolution and 8K resolution. Compared with other high-performance decoders, it has achieved considerable compression gain and greatly reduced the bit rate. However, in order to adapt to the complex texture of ultra-high-definition videos, the block division of its encoding has also become extremely complex. Improving the efficiency of block division is one of the main bottlenecks in reducing encoding complexity. Summary of the invention

[0003] The present disclosure provides a video encoding and block division method, apparatus, device and storage medium.

[0004] A first aspect of the present disclosure provides a block division method in video coding, comprising:

[0005] In the process of encoding the video image, when each encoding result of a pixel block in the video image is obtained, training data is generated based on the encoding result, and the classification model corresponding to each block division level is online learned using the training data until the obtained classification model corresponding to each block division level meets the training end condition;

[0006] In the process of encoding uncoded pixel blocks in a video image, before each block division, a classification model corresponding to the current division level is used to perform recursive division prediction; when the prediction result is that recursive division is required, the current pixel block is recursively divided; when the prediction result is that recursive division is not required, the recursive division process is terminated, and block division is performed according to a block division method other than recursive division corresponding to the current division level.

[0007] A second aspect of the present disclosure provides a video encoding method, including the block division method in video encoding described in the first aspect.

[0008] A third aspect of the present disclosure provides a block division device in video coding, comprising:

[0009] A training module is used for generating training data based on the encoding result of each pixel block in the video image during encoding of the video image, and using the training data to perform online learning on the classification model corresponding to each block division level until the obtained classification model corresponding to each block division level meets the training end condition;

[0010] A partitioning module is used to perform recursive partitioning prediction using a classification model corresponding to a current partitioning level before each block partitioning during the process of encoding an uncoded pixel block in a video image; recursively partition the current pixel block when the prediction result indicates that recursive partitioning is required; and terminate the recursive partitioning process when the prediction result indicates that recursive partitioning is not required, and perform block partitioning according to a block partitioning method other than recursive partitioning corresponding to the current partitioning level.

[0011] A fourth aspect of the present disclosure provides an electronic device, including:

[0012] at least one processor;

[0013] A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to the first aspect or the second aspect;

[0014] At least one I / O interface is connected between the processor and the memory and is configured to implement information interaction between the processor and the memory.

[0015] A fifth aspect of the present disclosure provides a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to the first aspect or the second aspect is implemented.

[0016] The present disclosure has the following advantages:

[0017] In the case of obtaining the encoding result of a pixel block in the video image each time, generating training data based on the encoding result, using the training data to perform online learning on the classification model corresponding to each block division level, until the obtained classification model corresponding to each block division level meets the training end condition, so that the classification model corresponding to each block division level is obtained by online learning in the process of encoding the video image, without pre-training the classification model in advance, so that the classification model can adapt to the changes of the video frame sequence, and the training module is directly iterated based on the encoding result to form a closed-loop system. Compared with offline learning, it can reduce training data, quickly adapt to the feature changes of new training data, and dynamically update the classification model, so that the classification model has better adaptability; and Moreover, in the subsequent encoding process, before each block division, the classification model corresponding to the current division level is used to perform recursive division prediction. When the prediction result is that recursive division is required, the current pixel block is recursively divided. When the prediction result is that recursive division is not required, the recursive division process is terminated, and block division is performed according to the block division method other than recursive division corresponding to the current division level. This allows the recursive division to be terminated in advance when the classification model predicts that recursion is not required, and only the block division needs to be performed according to the division method other than recursive division corresponding to the current division level, thereby avoiding unnecessary recursive division processes, reducing the complexity of block division, and thereby improving block division efficiency and reducing encoding complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of a block partitioning method provided in an embodiment of the present disclosure;

[0019] Figure 2 A schematic diagram of block division in a process of edge feature detection using a DEA algorithm provided in an embodiment of the present disclosure;

[0020] Figure 3 A schematic diagram of a hybrid coding and decoding framework provided in an embodiment of the present disclosure;

[0021] Figure 4 A flowchart of an online block partitioning solution based on LR+FTRL provided in an embodiment of the present disclosure;

[0022] Figure 5 A schematic diagram of a block division process in an AV1 encoder provided in an embodiment of the present disclosure;

[0023] Figure 6 A schematic diagram of the structure of a block division device in video coding provided in an embodiment of the present disclosure;

[0024] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0026] As used in this disclosure, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0027] The terms used in the present disclosure are only used to describe specific embodiments and are not intended to limit the present disclosure.As used in the present disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0028] When the terms “comprising” and / or “made of…” are used in the present disclosure, it specifies the existence of the stated features, integers, steps, operations, elements and / or components, but does not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0029] Unless otherwise defined, all terms (including technical and scientific terms) used in this disclosure have the same meaning as those commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined in this disclosure.

[0030] The algorithms used in video coding include but are not limited to HEVC, VVC (Versatile Video Coding), AV1 (Alliance for Open Media Video 1, an open source, royalty-free video coding format designated by the Alliance for Open Media), etc. Considering that the block division of AV1 is more complex, AV1 is used as an example in the following examples. AV1 supports 10 different block divisions. In order to be more suitable for high-resolution video coding, its maximum block processing size has been expanded to 128x128. For all NxN blocks, recursive execution will continue until the smallest 4x4 processing unit is reached. Therefore, AV1 has a total of five division levels: 128x128, 64x64, 32x32, 16x16, and 8x8, and each level has 10 other different division methods. Traditional encoders need to calculate the recursive distortion rate (RDO) and scan all block combinations to find the best value. This method is extremely complex and is also common in other codecs, which makes improving block division efficiency one of the main bottlenecks in reducing coding complexity.

[0031] Based on this, an embodiment of the present disclosure provides a block division method for video coding to improve block division efficiency and reduce coding complexity. The method can be applied to any video coding device. Figure 1 The block division process diagram is shown in FIG. 1 . The block division method for video coding mainly includes the following steps:

[0032] Step 101, in the process of encoding the video image, when each encoding result of a pixel block in the video image is obtained, training data is generated based on the encoding result, and the classification model corresponding to each block division level is online learned using the training data until the classification model corresponding to each block division level meets the training end condition.

[0033] In some embodiments, training data is generated based on the encoding result, including: obtaining the block division method ultimately adopted in the encoding in the encoding result, and obtaining the recursive division label of each division level in the ultimately adopted block division method, wherein the recursive division label is used to indicate whether the corresponding division level is recursively divided; obtaining the feature vector of the pixel block of each division level in the ultimately adopted block division method; and combining the feature vector of the pixel block of each division level and the recursive division label of the corresponding division level into training data.

[0034] In some embodiments, obtaining the feature vector of the pixel block at each division level in the block division method finally adopted by the encoding includes: obtaining the variance eigenvalue and edge eigenvalue corresponding to the pixel block at each division level in the block division method finally adopted by the encoding, and using the vector obtained by the variance eigenvalue and the edge eigenvalue as the feature vector of the corresponding pixel block. In this embodiment, considering that the block division of video encodings such as AV1 is extremely complex, in order to consider the accuracy of its division in many aspects and adapt to the online learning process, the variance value and edge eigenvalue of the pixel are used as training parameters.

[0035] In some embodiments, obtaining the variance characteristic value of the pixel blocks at each division level in the block division method ultimately adopted by the encoding includes: performing the following process on any pixel block at each division level in the block division method ultimately adopted by the encoding: obtaining the average brightness value corresponding to any one of the pixel blocks; obtaining the square value of the difference between the brightness value corresponding to each pixel of the any one of the pixel blocks and the corresponding average brightness value, and taking the average of the square values ​​corresponding to each pixel of the any one of the pixel blocks as the variance characteristic value of the any one of the pixel blocks.

[0036] The variance eigenvalue is an important feature that reflects the complexity of the image. Here, the variance eigenvalue of the pixel block is used to determine whether to perform recursive division. The variance eigenvalue of the pixel block is expressed as χ var , its expression:

[0037]

[0038] Where P L (n,m) represents the brightness value of the pixel in the nth row and mth column of the pixel block. represents the average brightness value of the pixel block, and B represents the size of the pixel block, that is, the number of rows or columns of the pixel block. The number of rows is the same as the number of columns.

[0039] In some embodiments, obtaining edge feature values ​​corresponding to pixel blocks of each division level in the block division method finally adopted by the encoding includes: performing the following process on any pixel block of each division level in the block division method finally adopted by the encoding as a target pixel block: performing data envelopment analysis (DEA) on the target pixel block to obtain edge feature values ​​of the target pixel block. Sobel operator, Robert operator and Canny operator are commonly used edge detection algorithms, among which Sobel operator and Robert operator are simple to calculate, but the accuracy effect is not good, and the feature extraction process of Canny operator is relatively complicated and not suitable for online learning process. DEA algorithm is used here to better detect edge features.

[0040] In some embodiments, data envelopment analysis is performed on the target pixel block to obtain edge feature values ​​of the target pixel block, the envelopment: the target pixel block is evenly divided into four sub-pixel blocks to obtain an upper left sub-pixel block, an upper right sub-pixel block, a lower left sub-pixel block, and a lower right sub-pixel block; the average pixel values ​​corresponding to the upper left sub-pixel block, the upper right sub-pixel block, the lower left sub-pixel block, and the lower right sub-pixel block are obtained; a first absolute value obtained by subtracting the average pixel value of the upper right sub-pixel block from the average pixel value of the upper left sub-pixel block and taking the absolute value, a second absolute value obtained by subtracting the average pixel value of the lower right sub-pixel block from the average pixel value of the lower left sub-pixel block and taking the absolute value, and a first sum value obtained by summing the first absolute value and the second absolute value; a third absolute value obtained by summing the average pixel value of the lower left sub-pixel block and the pixel value of the center pixel point of the current pixel block and taking the absolute value, and the pixel value of the center pixel point of the current pixel block minus the average pixel value of the upper right sub-pixel block. a fourth absolute value obtained by taking the absolute value of the average pixel value, and obtaining a second sum value obtained by summing the third absolute value and the fourth absolute value; obtaining a fifth absolute value obtained by subtracting the average pixel value of the upper left sub-pixel block from the average pixel value of the lower left sub-pixel block, and obtaining a sixth absolute value obtained by subtracting the average pixel value of the upper right sub-pixel block from the average pixel value of the lower right sub-pixel block, and obtaining a third sum value obtained by summing the fifth absolute value and the sixth absolute value; obtaining a seventh absolute value obtained by subtracting the pixel value of the central pixel point of the current pixel block from the average pixel value of the upper right sub-pixel block, and obtaining an eighth absolute value obtained by subtracting the pixel value of the central pixel point of the current pixel block from the average pixel value of the upper left sub-pixel block, and obtaining a fourth sum value obtained by summing the seventh absolute value and the eighth absolute value; and using the first sum value, the second sum value, the third sum value and the fourth sum value as the edge feature value of the target pixel block.

[0041] For example, Figure 2 The figure shows a block division diagram in the process of edge feature detection using the DEA algorithm. The N*N pixel block is evenly divided into four N / 2*N / 2 sub-pixel blocks, namely the upper left sub-pixel block al, the upper right sub-pixel block ar, the lower left sub-pixel block bl, and the lower right sub-pixel block br, with the center point c. "*" indicates multiplication. The center point of the block is c.

[0042] The average pixel value of the upper left sub-pixel block al is represented by P al , the average pixel value of the upper right sub-pixel block ar is expressed as P ar , the average pixel value of the lower left sub-pixel block bl is expressed as P bl , the average pixel value of the lower right sub-pixel block br is expressed as P br The average pixel value of the center point is expressed as Pc , and its calculation formula is:

[0043]

[0044]

[0045]

[0046]

[0047]

[0048] Among them, P (i,j) Indicates the pixel value at the pixel coordinate (i, j). After obtaining the average pixel value in each direction, calculate the DEA value of each angle respectively: DEA al 、DEA ar 、DEA bl 、DEA br , that is, the first sum, the second sum, the third sum and the fourth sum are given by the following formulas respectively:

[0049] DEA al =|P ar -P al |+|P br -P bl |

[0050] DEA ar =|P bl -P c |+|P c -P ar |

[0051] DEA bl =|P bl -P al |+|P br -P ar |

[0052] DEA br =|P br -P c |+|P c -P al |

[0053] The edge features of the pixel block are determined by the features of the four DEAs, and the edge feature value is χ al =DEA αl ,

[0054] χ ar =DEA ar , χ bl =DEA bl, χ br =DEA br .

[0055] Combining the variance eigenvalue and edge eigenvalue of the pixel block, the eigenvector of the pixel block can be obtained as χ i ={ var , χ al , χ ar , χ bl , χ br}.

[0056] In some embodiments, the training data is used to perform online learning on the classification model corresponding to each block division level, including: using an online learning framework composed of an LR model (logistic regression model) and an online logistic regression algorithm as a FTRL (Follow the Regularized Leader, online learning rate optimization algorithm) model to learn the training data, obtain the optimal solution for the parameters to be learned in the LR model, and replace the parameters of the LR model to obtain the classification model.

[0057] The traditional LR model is suitable for large-scale data and can obtain a classification model with high accuracy. The video encoding process involves a large number of data samples. Loading a large number of samples into the memory requires a high level of computer memory. If the number of samples exceeds the computer memory, the model cannot be trained. The LR model usually uses cross entropy as the loss function, which is a high-order differentiable continuous convex function. It can be solved using the gradient descent method and the Newton method. The mathematical expression of the cross entropy loss function is:

[0058]

[0059] l(ω) represents the loss function, y i represents the label value of sample i, ω represents the parameters to be learned in the classification model, and x represents the feature vector of sample i.

[0060] Here, the FTRL model is used to train the classification model. The classification model is updated once after each sample is trained, which reduces the requirement for computer memory. The FTRL model is used to replace the gradient descent to update the parameters of the LR model, and an online learning logistic regression (LR+FTRL) model is obtained. FTRL is essentially an online training idea that can dynamically adjust parameters so that the model can continuously adapt to the environment and achieve the effect of online classification.

[0061] It should be noted that machine learning algorithms such as OGD (Online Gradient Decent), FOBOS (Forward-Backward Splitting), RDA (Regularized Dual Average), neural networks, etc. can be used to replace the online learning algorithm of LR+FTRL.

[0062] Step 102, in the process of encoding the uncoded pixel blocks in the video image, before each block division, a classification model corresponding to the current division level is used to perform recursive division prediction; when the prediction result is that recursive division is required, the current pixel block is recursively divided; when the prediction result is that recursive division is not required, the recursive division process is terminated, and block division is performed according to the block division method corresponding to the current division level except recursive division.

[0063] In some embodiments, after recursively dividing the current pixel block when the prediction result indicates that recursive division is required, it also includes: determining whether the size of the pixel block after the recursive division is the smallest size pixel block; if not, using the classification model corresponding to the division level after the recursive division to perform recursive division prediction; if so, terminating the block division process.

[0064] Among them, the process of using the classification model corresponding to the division level after recursive division to perform recursive division prediction is: when the prediction result is that recursive division is required, the pixel blocks of the division level continue to be recursively divided; when the prediction result is that recursive division is not required, the recursive division process is terminated.

[0065] In some embodiments, assuming that the encoding algorithm has N block sizes from the largest pixel block to the smallest pixel block, there are N-1 division levels, that is, except for the smallest pixel block, before recursive division, the classification model of the corresponding division level is used to perform recursive division prediction for the pixel blocks of other sizes, and a binary classification result of whether to recursively divide is obtained.

[0066] In the case where the binary classification result corresponding to the pixel block of the current division level requires recursive division, the N*N pixel block of the current division level is divided into the pixel block of the next division level of M*M size, M=N / 2, and the classification model corresponding to the next division level is called for recursive division prediction, and there is no need to use other division methods in the current division level for block division. If the binary classification result corresponding to the pixel block of the current division level does not require recursive division, the block is divided according to the various preset block division methods corresponding to the current division level.

[0067] When the recursive partitioning is terminated, various block partitioning methods of the partitioning level at the time of termination and the recursive partitioning methods of the partitioning levels above it are obtained, and a combination of block partitioning methods with the best RDO (Rate Distortion Optimation) distortion rate is selected therefrom.

[0068] In an exemplary embodiment, Figure 3 The figure shows a hybrid coding and decoding framework diagram provided by an embodiment of the present disclosure, taking AV1 coding as an example. Figure 3 The block division module adopts the block division scheme in the video coding provided by the embodiment of the present disclosure. The input image frame is the image frame to be encoded. The block division module uses a trained classification model to determine whether recursive division is required before each block division until the block division process is completed; the block division result and the residual of the prediction result of the image frame to be encoded are transformed and quantized, and then entropy coding is performed to obtain the encoding result of the input image frame; in the decoding process, the encoding result is inversely quantized and inversely transformed, and then superimposed with the prediction result, loop filtering and adaptive compensation are performed to obtain the decoded image frame, and the output image frame is output after the motion estimation result is superimposed in the output image frame module for compensation.

[0069] Figure 4 The flowchart of the online block partitioning scheme based on LR+FTRL in the embodiment of the present disclosure is shown. First, the video sequence at the input time t is preprocessed, and the feature vector is obtained by feature extraction. At the same time, the original AV1 encoding is used to generate an online learning training set based on the extracted feature vector and the true value of the original encoding (that is, the block partitioning method finally selected by the original encoding). When the number of iterations has not reached the maximum number of iterations, the training set is used to perform FTRL online learning on the LR model. Through learning and feedback during multiple iterations, when the number of iterations reaches the maximum number of iterations, the trained classifier is used as the terminated classification model and the model is output. Among them, in the process of online learning using the training set, five classifiers are trained online at each division level 128x128, 64x64, 32x32, 16x16, and 8x8. Each group of classifiers determines whether to perform recursive division. If the judgment result is 0, it means no recursion, and if the judgment result is 1, it means recursion. Due to the online training process, the AV1 original encoder continuously gives feedback on the correct division results. Through FTRL online learning and adjustment of the parameters to be learned of the LR model, the classifier corresponding to the LR model can obtain the correct judgment result. Repeat the iterative training process, and when the preset maximum number of iterations is reached, the classification model is output for block division of subsequent frames.

[0070] Figure 5The block partitioning process in the AV1 encoder provided in the embodiment of the present disclosure is shown as a schematic diagram. Here, the AV1 encoder is used as an example. It supports two macroblock sizes of 128x128 and 64x64, and can be recursively divided into blocks of 4x4 size in sequence. There are eight partitioning methods for 128x128 blocks, 4 partitioning methods for 8x8 blocks, and 10 partitioning methods for the remaining levels. When entering each level, the classification model of the corresponding level is preferentially used to determine whether to use recursive partitioning. If recursive partitioning is not required, the recursive partitioning process is terminated, and other block partitioning methods corresponding to the level are used for partitioning, and the traditional RDO distortion rate is used to determine the optimal block partitioning method to achieve the purpose of accelerating the partitioning.

[0071] Assuming that the classification model of the corresponding division level predicts that recursive division is not required for the 128*128 pixel block, the 9 block division methods in the box pointed to by the N arrow corresponding to c128*128 are traversed, and the block division method combination with the best RDO is selected for the final encoding; assuming that the classification model of the corresponding division level predicts that recursive division is required for the 128*128 pixel block, the recursive division method specified by the Y arrow is used to divide it into pixel blocks of size c64*64, and there is no need to use the 9 block division methods pointed to by the N arrow for block division.

[0072] Similarly, assuming that the classification model of the corresponding division level is used to predict the c64*64 pixel block and no recursive division is required, the 9 block division methods in the box pointed to by the corresponding N arrow are traversed, and the block division method combination with the best RDO is selected for the final encoding; assuming that the classification model of the corresponding division level is used to predict that recursive division is required for the c64*64 pixel block, the recursive division method specified by the Y arrow is used to divide it into pixel blocks of size c32*32, and there is no need to use the 9 block division methods pointed to by the N arrow for block division.

[0073] Similarly, assuming that the classification model of the corresponding division level is used to predict the c32*32 pixel block and no recursive division is required, the 9 block division methods in the box pointed to by the corresponding N arrow are traversed, and the block division method combination with the best RDO is selected for the final encoding; assuming that the classification model of the corresponding division level is used to predict the c32*32 pixel block and recursive division is required, the recursive division method specified by the Y arrow is used to divide it into c16*16 pixel blocks, and there is no need to use the 9 block division methods pointed to by the N arrow for block division.

[0074] Similarly, assuming that the classification model of the corresponding division level is used to predict the c16*16 pixel block and no recursive division is required, the 9 block division methods in the box pointed to by the corresponding N arrow are traversed, and the block division method combination with the best RDO is selected for the final encoding; assuming that the classification model of the corresponding division level is used to predict that recursive division is required for the c16*16 pixel block, the recursive division method specified by the Y arrow is used to divide it into c8*8 pixel blocks, and there is no need to use the 9 block division methods pointed to by the N arrow for block division.

[0075] Similarly, assuming that the classification model of the corresponding partitioning level is used to predict the c8*8 pixel block, no recursive partitioning is required. The three block partitioning methods in the box pointed by the corresponding N arrow are used to traverse, and the block partitioning method combination with the best RDO is selected for the final encoding; assuming that the classification model of the corresponding partitioning level is used to predict the c8*8 pixel block, recursive partitioning is used to divide it into c4*4 pixel blocks, and the three block partitioning methods pointed by the N arrow are not required for block partitioning. So far, the minimum size of the partitioning level has been reached, and no further partitioning is required.

[0076] In the embodiment of the present disclosure, when the encoding result of a pixel block in a video image is obtained each time, training data is generated based on the encoding result, and the classification model corresponding to each block division level is learned online using the training data until the classification model corresponding to each block division level meets the training end condition, so that the classification model corresponding to each block division level is learned online in the process of encoding the video image, and there is no need to pre-train the classification model in advance, so that the classification model can adapt to the changes in the video frame sequence, and the training module is directly iterated based on the encoding result to form a closed-loop system. Compared with offline learning, it can reduce training data, quickly adapt to the feature changes of new training data, and dynamically update the classification model, so that the classification model has better adaptability; and in the subsequent encoding process, before each block division, the classification model corresponding to the current division level is used to perform recursive division prediction, and when the prediction result is that recursive division is required, the current pixel block is divided into the next level, and when the prediction result is that recursive division is not required, the block division process is terminated, so that the recursive division can be terminated in advance when the classification model predicts that recursion is not required, thereby avoiding unnecessary block division processes, improving block division efficiency, and reducing encoding complexity.

[0077] The block division method in video coding provided by the embodiment of the present disclosure uses online classification learning to establish the classification model of each division level of video coding, and uses the classification model of the corresponding level to perform block recursive division judgment before recursively dividing the pixel blocks of the corresponding level. In the case of no recursive division, the block division method corresponding to the division level is directly used to traverse to obtain the optimal block division method, and the subsequent encoding process can be carried out, so that the recursive operation of block division can be terminated in time, the block division efficiency is improved, and the encoding complexity is reduced. The online learning method can overcome the shortcomings of batch learning, convert the newly generated data instances into sample data in real time, and update the classification model in real time. Compared with the block division method in the traditional encoding algorithm, the block division method based on online learning avoids the huge amount of calculation caused by continuous recursive traversal through reasonable training and online learning, reduces the complexity of block division by terminating the recursive operation of the current level in advance, and ensures the reliability of the traditional method through the training of the binary classification model of small samples, and ensures the encoding quality of the video. It is essentially an accelerated algorithm on the traditional block division method, and also reduces the error caused by completely replacing it with an online learning algorithm.

[0078] Among them, the variance and edge features of the image pixels of the pixel block are used as feature vectors, and small batches of training data can be obtained in real time for online learning. An online learning model is established, feedback information is collected from the traditional encoder in real time, and the training model is iterated to form a closed-loop system. Compared with offline learning, it can reduce training data, quickly adapt to feature changes in new data, dynamically update the model, and have better adaptability.

[0079] The step division of the above various methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the protection scope of this disclosure. Adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the protection scope of this disclosure.

[0080] An embodiment of the present disclosure provides a video encoding method, including the block division method in video encoding described above.

[0081] An embodiment of the present disclosure provides a block division device in video encoding. The specific implementation of the device can refer to the relevant description of the method embodiment, which will not be repeated here. Figure 6 The schematic diagram of the structure of the device is shown, which mainly includes:

[0082] The training module 601 is used to generate training data based on the encoding result of each pixel block in the video image during encoding of the video image, and use the training data to perform online learning on the classification model corresponding to each block division level until the classification model corresponding to each block division level meets the training end condition;

[0083] The division module 602 is used to perform recursive division prediction using a classification model corresponding to a current division level before each block division in the process of encoding unencoded pixel blocks in a video image; recursively divide the current pixel block when the prediction result indicates that recursive division is required; and terminate the recursive division process when the prediction result indicates that recursive division is not required, and perform block division according to a block division method other than recursive division corresponding to the current division level.

[0084] The functions or modules included in the device provided in the embodiments of the present disclosure can be used to execute the method described in the method embodiments. The specific implementation and technical effects thereof can be referred to the description of the method embodiments above, and will not be described again here for the sake of brevity.

[0085] It should be noted that all modules involved in this embodiment are logic modules. In practical applications, a logic unit may be a physical unit, or a part of a physical unit, or may be implemented as a combination of multiple physical units. In addition, in order to highlight the innovative part of the present disclosure, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by the present disclosure, but this does not mean that there are no other units in this embodiment.

[0086] Reference Figure 7 , an embodiment of the present disclosure provides an electronic device, comprising:

[0087] at least one processor 701;

[0088] A memory 702 storing at least one program, when the at least one program is executed by the at least one processor, the at least one processor implements the above method;

[0089] At least one I / O interface 703 is connected between the processor and the memory and is configured to implement information exchange between the processor and the memory.

[0090] Among them, the processor 701 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 702 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 703 is connected between the processor 701 and the memory 702, and can realize information interaction between the processor 701 and the memory 702, including but not limited to a data bus (Bus), etc.

[0091] In some embodiments, the processor 701 , the memory 702 , and the I / O interface 703 are connected to each other through a bus, and further connected to other components of the computing device.

[0092] This embodiment further provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the method provided in this embodiment is implemented. To avoid repeated description, the specific steps of the method are not repeated here.

[0093] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods applied for above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0094] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0095] Those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present embodiment and to form different embodiments.

[0096] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present disclosure, but the present disclosure is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and substance of the present disclosure, and these modifications and improvements are also considered to be within the scope of protection of the present disclosure.

Claims

1. A block division method in video coding, characterized in that: include: In the process of encoding the video image, when each encoding result of a pixel block in the video image is obtained, training data is generated based on the encoding result, and the classification model corresponding to each block division level is online learned using the training data until the obtained classification model corresponding to each block division level meets the training end condition; In the process of encoding uncoded pixel blocks in a video image, before each block division, a classification model corresponding to the current division level is used to perform recursive division prediction; when the prediction result is that recursive division is required, the current pixel block is recursively divided; when the prediction result is that recursive division is not required, the recursive division process is terminated, and block division is performed according to a block division method other than recursive division corresponding to the current division level.

2. The block division method in video coding according to claim 1, characterized in that: The generating training data based on the encoding result comprises: Obtaining a block partitioning method finally adopted in the encoding result, and obtaining a recursive partitioning tag of each partitioning level in the finally adopted block partitioning method, wherein the recursive partitioning tag is used to indicate whether the corresponding partitioning level is recursively partitioned; Obtaining a feature vector of a pixel block at each division level in the block division method finally adopted by the encoding; The feature vectors of the pixel blocks at each partition level and the recursive partition labels of the corresponding partition level are used to form training data.

3. The block division method in video coding according to claim 2, characterized in that: The obtaining of the feature vector of the pixel block at each division level in the block division method finally adopted for the encoding comprises: The variance eigenvalue and the edge eigenvalue corresponding to the pixel block of each division level in the block division method finally adopted for the encoding are obtained, and a vector obtained by the variance eigenvalue and the edge eigenvalue is used as a eigenvector of the corresponding pixel block.

4. The block division method in video coding according to claim 3, characterized in that: Obtaining the variance characteristic value of the pixel block at each division level in the block division method finally adopted by the encoding, including: The following process is performed on any pixel block in each division level of the pixel blocks in the block division method finally adopted by the encoding: Obtaining an average brightness value corresponding to any pixel block; The square value of the difference between the brightness value corresponding to each pixel of the any pixel block and the corresponding average brightness value is obtained, and the average value of the square values ​​corresponding to each pixel of the any pixel block is used as the variance eigenvalue of the any pixel block.

5. The block division method in video coding according to claim 3, characterized in that: Obtaining edge feature values ​​corresponding to pixel blocks at each division level in the block division method finally adopted by the encoding, including: The following process is performed on any pixel block in each division level of the block division method finally adopted for the encoding as a target pixel block: data envelopment analysis is performed on the target pixel block to obtain edge feature values ​​of the target pixel block.

6. The block division method in video coding according to claim 5, characterized in that: The data envelopment analysis is performed on the target pixel block to obtain the edge feature value of the target pixel block, the envelope: Dividing the target pixel block into four sub-pixel blocks evenly to obtain an upper left sub-pixel block, an upper right sub-pixel block, a lower left sub-pixel block and a lower right sub-pixel block; Obtaining average pixel values ​​corresponding to the upper left sub-pixel block, the upper right sub-pixel block, the lower left sub-pixel block, and the lower right sub-pixel block; Obtain a first absolute value obtained by subtracting the average pixel value of the upper left sub-pixel block from the average pixel value of the upper right sub-pixel block, and a second absolute value obtained by subtracting the average pixel value of the lower left sub-pixel block from the average pixel value of the lower right sub-pixel block, and obtain a first sum value obtained by summing the first absolute value and the second absolute value; Obtain a third absolute value obtained by taking the absolute values ​​of the average pixel value of the lower left sub-pixel block and the pixel value of the central pixel point of the current pixel block, and a fourth absolute value obtained by subtracting the average pixel value of the upper right sub-pixel block from the pixel value of the central pixel point of the current pixel block, and obtaining a second sum value obtained by summing the third absolute value and the fourth absolute value; Obtain a fifth absolute value obtained by subtracting the average pixel value of the upper left sub-pixel block from the average pixel value of the lower left sub-pixel block, and a sixth absolute value obtained by subtracting the average pixel value of the upper right sub-pixel block from the average pixel value of the lower right sub-pixel block, and obtain a third sum value obtained by summing the fifth absolute value and the sixth absolute value; Obtain a seventh absolute value obtained by subtracting the pixel value of the center pixel point of the current pixel block from the average pixel value of the upper right sub-pixel block and taking the absolute value thereof, and an eighth absolute value obtained by subtracting the average pixel value of the upper left sub-pixel block from the pixel value of the center pixel point of the current pixel block and taking the absolute value thereof, and obtain a fourth sum value obtained by summing the seventh absolute value and the eighth absolute value; The first sum value, the second sum value, the third sum value and the fourth sum value are used as edge feature values ​​of the target pixel block.

7. The block division method in video coding according to claim 1, characterized in that: After recursively dividing the current pixel block when the prediction result indicates that recursive division is required, the method further includes: Determine whether the size of the pixel block after recursive partitioning is the minimum size of the pixel block. If not, use the classification model corresponding to the partition level after recursive partitioning to perform recursive partitioning prediction; if so, terminate the block partitioning process.

8. A video encoding method, characterized in that: The invention comprises the block division method in video coding as described in any one of claims 1-7.

9. A block division device in video coding, characterized in that: include: A training module is used for generating training data based on the encoding result of each pixel block in the video image during encoding of the video image, and using the training data to perform online learning on the classification model corresponding to each block division level until the obtained classification model corresponding to each block division level meets the training end condition; A partitioning module is used to, in the process of encoding uncoded pixel blocks in a video image, before each block partitioning, use a classification model corresponding to a current partitioning level to perform recursive partitioning prediction; when the prediction result is that recursive partitioning is required, recursive partitioning is performed on the current pixel block; when the prediction result is that recursive partitioning is not required, the recursive partitioning process is terminated, and block partitioning is performed according to a block partitioning method other than recursive partitioning corresponding to the current partitioning level.

10. An electronic device, characterized in that: include: at least one processor; A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7 or implement the method according to any one of claim 8; At least one I / O interface is connected between the processor and the memory and is configured to implement information interaction between the processor and the memory.

11. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the program implements the method according to any one of claims 1 to 7 or the method according to any one of claims 8.

Images