Fast decision method and storage medium for VVC multi-tree structure based on texture characteristics
Through the fast decision-making method of VVC multi-forktree structure based on texture characteristics, the edge detection operator and texture direction analysis are used to skip unnecessary multi-forktree division, which solves the problem of high complexity of VVC encoder and realizes coding efficiency and Improved quality, suitable for real-time video applications.
Patent Information
- Application Number
- CN202210822273.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-12
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-07-12
AI Technical Summary
The computational complexity of VVC encoder in the intra-coding module is too high, resulting in an increase in encoding time, hindering its development in real-time video applications, and the prior art has failed to effectively reduce the complexity of CU division.
The VVC multi-forktree structure fast decision-making method based on texture characteristics. By reading the encoding information of the CU, the edge detection operator extracts gradient features, judges the texture direction, and analyzes the variance of edge pixels of adjacent sub-blocks, skips unnecessary Multi-forktree division reduces the partition mode traversal of coding units.
It significantly reduces the computational complexity of CU division, improves coding efficiency, and reduces coding time, and is suitable for real-time video communication applications.
Smart Images

Figure CN115278259B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of video coding, and more specifically, relates to a VVC multi-branch tree structure fast decision method based on texture characteristics and a storage medium. Background Art
[0002] With the implementation of 5G technology, the low-latency and high-bandwidth transmission characteristics of 5G have injected vitality into the application of high-definition video technology. The video industry has thus ushered in unprecedented growth opportunities, and ultra-high-definition video (UHD) represented by 4K and 8K has begun to be widely used. Ultra-high-definition video has ultra-high resolution and high frame rate, which can effectively improve the fineness and smoothness of the image, thus bringing an excellent viewing experience, but it also brings a sharp increase in video traffic. Taking 8K video with a frame rate of 120fps as an example, the size of an uncompressed video of 1 second reaches 119Gbps. The huge amount of data has brought new challenges to the storage and real-time transmission of videos. In order to cope with the new pain points arising from the development of the video industry, the International Telecommunication Union (ITU) and the International Organization for Standardization (ISO) jointly established the Joint Video Experts Group (JVET) in 2016 and began to explore new key technologies for the next-generation video coding technology standard, namely the VVC standard. In July 2020, the new video coding standard VVC was officially released.
[0003] Compared with H.265 / HEVC, VVC has developed many new coding technologies, which involve various modules of video coding. For example, in the quantization and transformation modules, VVC has added large-scale transformation with high-frequency zeroing and adaptive multi-core transformation technology, and the encoder can adaptively select the optimal transformation method. In the prediction coding module, in order to improve the prediction accuracy, VVC has added intra-frame multi-line prediction, cross-component linear model prediction, intra-frame sub-block prediction, position-adaptive intra-frame joint prediction, affine motion compensation prediction, intra-frame and inter-frame joint prediction, symmetric motion vector difference and other technologies.
[0004] The intra-frame coding module in the video encoder effectively compresses the video information mainly by removing spatial redundancy. The new technology of VVC is also developed to remove spatial redundancy more efficiently. In terms of partition structure, VVC inherits the block partition structure of previous video coding standards, but unlike previous standards that can only divide coding units into square blocks, VVC adopts a quadtree with nested multi-type tree (QTMT) structure, which allows the CU to be adaptively divided into square or rectangular blocks according to the video content. In terms of intra-frame prediction, VVC expands the 35 prediction modes of the previous generation HEVC standard and adopts 67 intra-frame prediction modes to further reduce the spatial redundancy of intra-frame prediction. However, in the process of deciding the best mode, it is necessary to traverse all prediction modes and perform rate-distortion optimization, which is extremely complex and time-consuming. In the full intra-frame configuration, the intra-frame coding efficiency of the VVC encoder is 41% higher than that of HEVC, but the encoding time is 31 times that of HEVC.
[0005] By adopting new coding technology, VVC can reduce the code rate of the encoder output by 50% when processing the same video compared with the previous generation video coding standard HEVC, while the video coding quality is almost not lost. However, it also comes at the huge cost of exponentially increased computational complexity, which hinders the development of VVC in real-time video applications. Therefore, how to significantly reduce the coding complexity of VVC without affecting the video coding quality and reducing the coding efficiency is the research focus of many research institutions, experts and scholars, and video application-related companies. According to the test results of existing literature, if the deep division of the QTMT structure in the VVC intra-frame coding module is not performed, the encoder can save up to 80% of the coding time. Therefore, conducting research on the intra-frame coding module and proposing an effective intra-frame coding acceleration algorithm is an important direction to improve the coding speed of VVC and reduce the application cost.
[0006] CN112929657A, a H.266 / VVC fast CU partitioning method based on gradient and variance, the steps are: first, evaluate the homogeneity of the current CU according to the variance value, and judge whether the current CU can be terminated in advance to be further divided. Then use the Sobel operator to extract the gradient features of the current CU, and judge whether the current CU can be QT partitioned, thereby skipping BT and TT partitions. Finally, use the Canny operator to extract the edge features of the current CU texture, exclude the MT partition in the vertical or horizontal partition direction according to the texture trend of the current CU, take the MT partition in the other direction as a candidate, and take the partitioning method with the smallest RDO-cost as the optimal partitioning method. The present invention makes decisions on CU partitioning step by step, accelerates the CU partitioning process by early termination and early skipping, and significantly reduces the complexity of CU partitioning while ensuring the coding quality, greatly improving the coding efficiency. However, after excluding the multi-tree (MT) partition in the vertical or horizontal partitioning direction by the texture trend of the current CU, the patent does not continue to make further rapid judgments on the binary tree partitioning and ternary tree partitioning under another MT partition, so the CU partitioning acceleration performance of the invention still has room for further improvement.
[0007] After determining the division direction of the current CU based on the gradient characteristics, the patent in this article decides to skip the vertical or horizontal division method in advance; then the structural characteristics of the current CU are analyzed by using the difference in edge pixels of adjacent sub-blocks of the current CU multi-tree structure. If the edge difference of adjacent sub-blocks divided by the binary tree is greater than the minimum value of the edge difference of adjacent sub-blocks divided by the ternary tree, the current CU skips the horizontal ternary tree or vertical ternary tree division method. Therefore, the method proposed in this patent can minimize the computational complexity of CU division and greatly speed up video encoding. Summary of the invention
[0008] The present invention aims to solve the above problems of the prior art. A VVC multi-tree structure fast decision method and storage medium based on texture characteristics are proposed. The technical solution of the present invention is as follows:
[0009] The VVC multi-tree structure fast decision method based on texture characteristics includes the following steps:
[0010] 101. Read the coding information of the current CU (coding unit) to be divided in the input video frame, and determine whether the current CU is the default mandatory division CU of VVC (Versatile Video Coding). If so, the current CU is determined by the VVC original platform CU division method to determine the best division mode, otherwise, execute step 102;
[0011] 102. Use edge detection operator to extract gradient features of the current CU;
[0012] 103. Compare the average gradient value in the horizontal direction of the current CU with the average gradient value in the horizontal direction of the current coded frame, and the average gradient value in the vertical direction of the current CU with the average gradient value in the vertical direction of the current coded frame, to determine whether the multi-tree (MTT) partitioning of the current CU is terminated in advance. If the multi-tree partitioning is terminated in advance, the current CU skips all MTT partitioning processes, otherwise, execute step 104;
[0013] 104. According to the gradient feature value of the current CU, determine the main texture direction of the current CU. If the texture direction of the current CU is fuzzy, use the VVC original platform CU division method to determine the best division mode. If the current CU texture is mainly horizontal, choose to skip the vertical division method; if the current CU texture is mainly vertical, choose to skip the horizontal division method, and then execute step 105;
[0014] 105. Calculate the variance of adjacent edge pixels of the current CU sub-block. Evaluate the edge difference of adjacent sub-blocks by using the absolute difference of the variance of the edge pixels of adjacent sub-blocks. By comparing the edge difference of adjacent sub-blocks divided by binary tree (BT) with the edge difference of adjacent sub-blocks divided by ternary tree (TT), decide whether to skip the ternary tree division method;
[0015] 106. Update the partitioning mode candidate list of the current CU.
[0016] Furthermore, in step 101, the encoding information of the CU to be divided is read, including information such as the division depth, position, size and channel of the current CU.
[0017] The judgment of whether the current CU is the VVC default mandatory partition CU refers to the CU at the coding frame boundary and the CU that generates partition redundancy. The CU at the coding frame boundary refers to the CU that exceeds the right or bottom boundary of the image; partition redundancy refers to the situation where the same CU has the same coding structure despite different partitioning methods and orders.
[0018] Furthermore, the step 102 uses an edge detection operator to extract the gradient features of the current CU, specifically including:
[0019] Use one of the operators such as Sobel, Laplace, Scharr, Canny, etc. to perform planar convolution with the CU pixel matrix through the operator template in the horizontal and vertical directions respectively, take the absolute value of the calculated value and divide it by the number of CU pixels to obtain the average gradient value Grad in the horizontal direction of the CU h And the average gradient value Grad in the vertical direction v .
[0020] Furthermore, in step 103, the horizontal average gradient value of the current coded frame and the vertical average gradient value of the current coded frame refer to the horizontal average gradient value and the vertical average gradient value of the current coded frame calculated by the video encoder using a certain edge detection operator. Taking the Scharr operator as an example, the calculation method of the horizontal average gradient value and the vertical average gradient value of the current coded frame is shown in formulas (1) and (2);
[0021]
[0022]
[0023] In the above formula, P is the pixel matrix of the current coded frame; Gh pic Gv is the average gradient value in the horizontal direction of the current coded frame; pic is the average gradient value in the vertical direction of the current coded frame; W and H are the width and height of the current coded frame respectively.
[0024] Furthermore, in step 103, whether the current CU is to be divided into multiple trees is determined if the horizontal average gradient value Grad h Less than the average horizontal gradient value Gh of the current coded frame pic , and the vertical average gradient value Grad of the current CU v Less than the average vertical gradient value Gv of the current coded frame pic When , the current CU no longer performs multi-tree division; otherwise, the current CU performs multi-tree division.
[0025] Furthermore, in step 104, the main texture direction of the current CU is determined by formula (3), and the obtained main texture direction is marked by the division direction flag dir_flag;
[0026]
[0027] Among them, ThD is a threshold for judging the direction of the current CU texture, which can be obtained through experimental statistics or learning.
[0028] If dir_flag=0, the current CU texture is mainly vertical, and the horizontal direction (including HBT and HTT) division method is skipped; if dir_flag=1, the current CU texture is mainly horizontal, and the vertical direction (including VBT and VTT) division method is skipped; if dir_flag=-1, it indicates that the current CU texture direction is fuzzy, and the main direction of the current CU texture cannot be determined. The VVC original platform CU division method is used to determine the best division mode.
[0029] Furthermore, in step 105, the variance of the edge pixels of the adjacent sub-blocks of the current CU is calculated, specifically including: when the current CU is divided horizontally or vertically, the variance of the adjacent edge pixels of the CU sub-block is calculated. If it is determined that the current CU texture is mainly horizontal, the variance of the adjacent edge pixels of the adjacent sub-blocks of the horizontal ternary tree and binary tree is calculated. For the three sub-blocks from top to bottom of the horizontal ternary tree, the variance of the edge pixels of the bottom row of the first sub-block is calculated to obtain Calculate the variance of edge pixels in the top row and the bottom row of the second sub-block, and get and Calculate the variance of the edge pixels in the top row of the third sub-block and get For the two sub-blocks from top to bottom of the horizontal binary tree, calculate the variance of the edge pixels in the bottom row of the first sub-block and get Calculate the variance of the edge pixels in the top row of the second sub-block and get If it is determined that the current CU texture is mainly vertical, calculate the variance of the adjacent edge pixels of the adjacent sub-blocks of the vertical ternary tree and binary tree. For the three sub-blocks from left to right in the vertical ternary tree, calculate the variance of the edge pixels in the rightmost column of the first sub-block, and get Calculate the variance of the edge pixels in the leftmost column and the rightmost column of the second sub-block, and get and Calculate the variance of the edge pixels in the leftmost column of the third sub-block and get For the two sub-blocks from left to right in the vertical binary tree, calculate the variance of the edge pixels in the rightmost column of the first sub-block and get Calculate the variance of the edge pixels in the leftmost column of the second sub-block and get
[0030] Furthermore, in step 105, evaluating the edge difference of adjacent sub-blocks by using the absolute difference of the pixel variances of the edges of adjacent sub-blocks means calculating the absolute difference of the pixel variances of the adjacent edges of adjacent sub-blocks in the horizontal direction by formula (4), or calculating the absolute difference of the pixel variances of the adjacent edges of adjacent sub-blocks in the vertical direction by formula (5), and evaluating the edge difference of adjacent sub-blocks by using the magnitude of the absolute difference.
[0031]
[0032]
[0033] In the above formula, is the absolute difference of pixel variances of adjacent edges of adjacent sub-blocks in the horizontal binary tree; and are the absolute differences of pixel variances of adjacent edges of adjacent sub-blocks in the horizontal ternary tree; is the absolute difference of the pixel variances of adjacent edges of adjacent sub-blocks in the vertical binary tree; and are the absolute differences of pixel variances of adjacent edges of adjacent sub-blocks in the vertical ternary tree.
[0034] Furthermore, in step 105, by comparing the edge difference of adjacent sub-blocks divided by binary tree (BT) with the edge difference of adjacent sub-blocks divided by ternary tree (TT), it is decided whether to skip the ternary tree division method. For the horizontal multi-tree, the judgment is made by formula (6), and for the vertical multi-tree, the judgment is made by formula (7). If the edge difference of adjacent sub-blocks divided by horizontal binary tree (HBT) or vertical binary tree (VBT) is greater than the minimum value of the edge difference of adjacent sub-blocks divided by horizontal ternary tree (HTT) or vertical ternary tree (VTT), the current CU skips the horizontal ternary tree or vertical ternary tree division method;
[0035]
[0036]
[0037] In the above formula, tt skip A flag indicating whether to skip the ternary tree partition. skip =1, indicating that the current CU skips the horizontal ternary tree or vertical ternary tree partitioning method; if tt skip =0, indicating that the current CU does not skip the horizontal ternary tree or vertical ternary tree partitioning method.
[0038] A storage medium stores a computer program therein, wherein when the computer program is read by a processor, any of the above methods is executed.
[0039] The advantages and beneficial effects of the present invention are as follows:
[0040] In view of the problems that the current VVC encoder has high computational complexity, high implementation cost and difficulty in practical application, the present invention proposes a method and storage medium for realizing a VVC multi-branch tree structure fast decision-making by utilizing the texture characteristics of the video itself, skipping the traversal process of unnecessary CU division modes. First, the texture complexity is judged based on the gradient characteristics detected by the current CU. If the texture of the current CU is smooth, the multi-branch tree division process is terminated. Then, based on the judgment of the main texture direction of the current CU, it is decided whether to skip the vertical or horizontal division method in advance; finally, the difference between the edge pixels of adjacent sub-blocks in the current CU multi-branch tree structure is used to decide whether to skip the three-branch division method. When the VVC video encoder adopts the present invention, the encoding calculation complexity can be greatly reduced while ensuring that the encoding efficiency and encoding quality are slightly reduced, and it can be used in video communication application scenarios with high real-time requirements.
[0041] The CU partitioning method of the present invention is used to terminate or skip some CU partitioning processes of the encoder in advance, so as to realize the rapid decision of the CU structure. For CU with smooth texture, all multi-tree partitioning processes are terminated in advance; for CU with complex texture, when it is judged that the current CU texture is horizontal, the multi-tree partitioning in the vertical direction is skipped; when it is judged that the current CU texture is vertical, the multi-tree partitioning in the horizontal direction is skipped; the difference between the edge pixels of adjacent sub-blocks of the binary tree and the ternary tree structure is analyzed to decide whether to skip the ternary partitioning method, which can further reduce the coding complexity. At present, in the literature related to the study of VVC fast CU partitioning algorithm, skipping the ternary tree partitioning method in the multi-tree is rarely considered. The main reason is that if the sub-block characteristics of the current CU cannot be accurately analyzed, skipping the ternary tree partitioning process will cause a large loss in coding efficiency. The present invention can accurately judge the structural characteristics of the binary tree and the ternary tree by analyzing the difference of the edge pixels of the current CU sub-block, skip the ternary tree partitioning process, and the coding efficiency loss of the video encoder is not large. Since the present invention only uses a conventional edge detection operator to analyze the gradient characteristics of the current CU and the coded frame, it can terminate the current CU multi-tree partitioning, or skip the multi-tree partitioning process in a certain direction. In terms of evaluating the sub-block edge difference, it is only necessary to calculate the pixel variance of the adjacent sub-block edges of the current CU. Therefore, the algorithm logic of the present invention is simple, easy to implement, and has obvious encoding acceleration effect, and the cost of encoding efficiency and encoding quality is also very small. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 It is a flowchart of a VVC multi-tree structure fast decision method based on texture characteristics according to a preferred embodiment of the present invention;
[0043] Figure 2 Schematic diagram of adjacent edges of sub-blocks in the horizontal and vertical directions of a CU in the present invention; DETAILED DESCRIPTION
[0044] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.
[0045] The technical solution of the present invention to solve the above technical problems is:
[0046] Figure 1 The present invention provides a flowchart of a VVC multi-tree structure fast decision method based on texture characteristics, which specifically includes the following steps:
[0047] 101. Read the encoding information of the current CU to be divided, and determine whether the current CU is the VVC default mandatory division CU. If so, the current CU is determined by the VVC original platform CU division method to determine the best division mode, otherwise, execute the next step.
[0048] 102. Use edge detection operator to extract gradient features of the current CU.
[0049] 103. By comparing the horizontal gradient value of the current CU with the horizontal gradient value of the current coded frame, and the vertical gradient value of the current CU with the vertical gradient value of the current coded frame, it is determined whether the current CU terminates the multi-tree division in advance. If the multi-tree division is terminated in advance, the current CU skips all the multi-tree division processes, otherwise, executes the next step.
[0050] 104. According to the gradient feature value of the current CU, determine the main texture direction of the current CU. If the texture direction of the current CU is fuzzy, use the VVC original platform CU division method to determine the best division mode. If the current CU texture is mainly horizontal, choose to skip the vertical division method; if the current CU texture is mainly vertical, choose to skip the horizontal division method, and then execute the next step.
[0051] 105. Calculate the variance of adjacent edge pixels of the current CU sub-block. Evaluate the edge difference of adjacent sub-blocks by using the absolute difference of the variance of the edge pixels of adjacent sub-blocks. By comparing the edge difference of adjacent sub-blocks divided by binary tree (BT) with the edge difference of adjacent sub-blocks divided by ternary tree (TT), decide whether to skip the ternary tree division method;
[0052] 106. Update the candidate list of partitioning modes of the current CU and perform the next CU partitioning.
[0053] Preferably, in step 101, the encoding information of the CU to be divided is read, including information including the division depth, position, size and channel of the current CU;
[0054] Preferably, in step 101, judging whether the current CU is a VVC default mandatory partition CU refers to a CU at the coding frame boundary and a CU that may generate partition redundancy. The CU at the coding frame boundary is a CU that exceeds the right boundary or the bottom boundary of the image. Partition redundancy refers to the situation where the same coding structure is generated for the same CU although the partitioning method and order are different.
[0055] Preferably, in step 102, the edge detection operator is used to extract the gradient features of the current CU, which means that a certain operator such as Sobel, Laplace, Scharr, Canny, etc. is used to perform a plane convolution with the CU pixel matrix through the operator template in the horizontal and vertical directions, and the calculated value is taken as the absolute value and then divided by the number of CU pixels to obtain the average gradient value in the corresponding direction of the CU. Taking the Scharr operator to extract the horizontal and vertical gradients as an example, the method for calculating the horizontal and vertical gradient values of the current CU is shown in formulas (1) and (2), Grad h is the calculated average gradient value in the horizontal direction of the current CU; Grad v It is the average gradient value in the vertical direction of the current CU.
[0056]
[0057]
[0058] Among them, A is the pixel matrix of CU; width and height correspond to the width and height of the current CU respectively.
[0059] Preferably, in step 103, the horizontal gradient value of the current coded frame and the vertical gradient value of the current coded frame are the average gradient values of the horizontal and vertical directions of the current coded frame calculated by the edge detection operator after the video encoder obtains the current coded frame. Taking the Scharr operator as an example, the calculation method of the horizontal and vertical gradient values of the current coded frame is shown in formulas (3) and (4).
[0060]
[0061]
[0062] Where P is the pixel matrix of the current coded frame; Gh pic Gv is the average gradient value in the horizontal direction of the current coded frame; pic is the average gradient value in the vertical direction of the current coded frame; W and H are the width and height of the current coded frame respectively.
[0063] Preferably, in step 103, judging whether the current CU is to be divided into multiple trees means that if the horizontal average gradient value Grad hLess than the horizontal average gradient value Gh of the current coded frame pic , and the vertical average gradient value Grad of the current CU v Less than the vertical average gradient value Gv of the current coded frame pic When , the current CU no longer performs multi-tree division; otherwise, the current CU performs multi-tree division.
[0064] Preferably, in step 104, the main texture direction of the current CU is determined by formula (5), and after the main texture direction is obtained, it is marked by a division direction flag dir_flag.
[0065]
[0066] Among them, ThD is a threshold for judging the direction of the current CU texture, which can be obtained through experimental statistics or learning.
[0067] If dir_flag=0, the current CU texture is mainly vertical, and the horizontal direction (including HBT and HTT) division method is skipped; if dir_flag=1, the current CU texture is mainly horizontal, and the vertical direction (including VBT and VTT) division method is skipped; if dir_flag=-1, it indicates that the current CU texture direction is fuzzy, and the main direction of the current CU texture cannot be determined. The VVC original platform CU division method is used to determine the best division mode.
[0068] Preferably, in step 105, calculating the variance of the edge pixels of the adjacent sub-blocks of the current CU means that the CU is calculated according to Figure 1 When dividing horizontally or vertically as shown, the variance of two adjacent edge pixels of adjacent sub-blocks is calculated. If the current CU texture is mainly horizontal, the calculation is as follows Figure 2 The variance of adjacent edge pixels of adjacent sub-blocks of the horizontal ternary tree and binary tree shown in Figure (a) and Figure (b) respectively obtains six edge pixel variance values and If the current CU texture is mainly vertical, the calculation is as follows Figure 2 The variance of adjacent edge pixels of adjacent sub-blocks of the ternary tree and binary tree in the vertical direction shown in Figure (c) and Figure (d) is used to obtain the six edge pixel variance values. and
[0069] Preferably, in step 105, evaluating the feature of the difference between the edges of adjacent sub-blocks by using the absolute difference of the pixel variances of the edges of adjacent sub-blocks means calculating the absolute difference of the pixel variances of the adjacent edges of adjacent sub-blocks in the horizontal direction by formula (6), or calculating the absolute difference of the pixel variances of the adjacent edges of adjacent sub-blocks in the vertical direction by formula (7), and evaluating the difference between the edges of adjacent sub-blocks by using the magnitude of the absolute difference;
[0070]
[0071]
[0072] in, is the absolute difference of pixel variances of adjacent edges of adjacent sub-blocks in the horizontal binary tree; and are the absolute differences of pixel variances of adjacent edges of adjacent sub-blocks in the horizontal ternary tree; is the absolute difference of the pixel variances of adjacent edges of adjacent sub-blocks in the vertical binary tree; and are the absolute differences of pixel variances of adjacent edges of adjacent sub-blocks in the vertical ternary tree.
[0073] Preferably, in step 105, by comparing the edge difference of adjacent sub-blocks divided by binary tree (BT) with the edge difference of adjacent sub-blocks divided by ternary tree (TT), it is determined whether to skip the ternary tree division method. For the horizontal multi-tree, the judgment is made by formula (8), and for the vertical multi-tree, the judgment is made by formula (9). If the edge difference of adjacent sub-blocks divided by horizontal binary tree (HBT) or vertical binary tree (VBT) is greater than the minimum value of the edge difference of adjacent sub-blocks divided by horizontal ternary tree (HTT) or vertical ternary tree (VTT), the current CU skips the horizontal ternary tree or vertical ternary tree division method.
[0074]
[0075]
[0076] Among them, tt skip A flag indicating whether to skip the ternary tree partition. skip =1, indicating that the current CU skips the horizontal ternary tree or vertical ternary tree partitioning method; if tt skip =0, indicating that the current CU does not skip the horizontal ternary tree or vertical ternary tree partitioning method.
[0077] Preferably, in step 106, updating the candidate list of the partition mode of the current CU means that if it is determined that the current CU can terminate the multi-tree partition in advance, only QT is added to the candidate list of the current CU partition mode; if the texture of the current CU is mainly horizontal, if the HTT partition can be skipped, only QT and HBT are added to the candidate list of the current CU partition mode, otherwise QT, HBT and HTT are added to the candidate list of the current CU partition mode; if the texture of the current CU is mainly vertical, if the VTT partition can be skipped, only QT and VBT are added to the candidate list of the current CU partition mode, otherwise QT, VBT and VTT are added to the candidate list of the current CU partition mode;
[0078] A storage medium stores a computer program therein, wherein when the computer program is read by a processor, any of the above methods is executed.
[0079] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0080] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0081] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0082] The above embodiments should be understood to be only used to illustrate the present invention and not to limit the protection scope of the present invention. After reading the contents of the present invention, technicians can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.
Claims
1. A VVC multi-tree structure fast decision method based on texture characteristics, characterized in that: The following steps are involved:
101. Read the coding information of the current coding unit (CU) to be divided in the input video frame, and determine whether the current CU is the default mandatory division CU of VVC multifunctional video coding. If so, the current CU is determined by the VVC original platform CU division method to determine the best division mode, otherwise, execute step 102; 102. Use edge detection operator to extract gradient features of the current CU; 103. Compare the average gradient value in the horizontal direction of the current CU with the average gradient value in the horizontal direction of the current coded frame, and the average gradient value in the vertical direction of the current CU with the average gradient value in the vertical direction of the current coded frame, and determine whether the multi-tree (MTT) partitioning of the current CU is terminated in advance. If the multi-tree partitioning is terminated in advance, the current CU skips all MTT partitioning processes. Otherwise, execute step 104.
104. According to the gradient feature value of the current CU, the main texture direction of the current CU is determined. If the texture direction of the current CU is fuzzy, the VVC original platform CU division method is used to determine the best division mode; if the current CU texture is mainly horizontal, the vertical division method is skipped; if the current CU texture is mainly vertical, the horizontal division method is skipped, and then step 105 is executed; 105. Calculate the variance of adjacent edge pixels of the current CU sub-block; evaluate the edge difference of adjacent sub-blocks by using the absolute difference of the variance of the edge pixels of adjacent sub-blocks; and decide whether to skip the ternary tree division method by comparing the edge difference of adjacent sub-blocks divided by the binary tree (BT) and the edge difference of adjacent sub-blocks divided by the ternary tree (TT); 106. Update the partition mode candidate list of the current CU; In step 105, by comparing the edge difference of adjacent sub-blocks divided by the binary tree BT with the edge difference of adjacent sub-blocks divided by the ternary tree TT, it is determined whether to skip the ternary tree division mode; for the horizontal multi-tree, the judgment is made by formula (6), and for the vertical multi-tree, the judgment is made by formula (7); if the edge difference of adjacent sub-blocks divided by the horizontal binary tree (HBT) or the vertical binary tree (VBT) is greater than the minimum value of the edge difference of adjacent sub-blocks divided by the horizontal ternary tree (HTT) or the vertical ternary tree (VTT), the current CU skips the horizontal ternary tree or the vertical ternary tree division mode; In the above formula, tt skip A flag indicating whether to skip ternary tree partitioning; if tt skip =1, indicating that the current CU skips the horizontal ternary tree or vertical ternary tree partitioning method; if tt skip =0, indicating that the current CU does not skip the horizontal ternary tree or vertical ternary tree partitioning mode; In step 105, evaluating the edge difference of adjacent sub-blocks by using the absolute difference of the pixel variances of the edges of adjacent sub-blocks means calculating the absolute difference of the pixel variances of adjacent edges of adjacent sub-blocks in the horizontal direction by formula (4), or calculating the absolute difference of the pixel variances of adjacent edges of adjacent sub-blocks in the vertical direction by formula (5), and evaluating the edge difference of adjacent sub-blocks by using the magnitude of the absolute difference; In the above formula, is the absolute difference of pixel variances of adjacent edges of adjacent sub-blocks in the horizontal binary tree; and are the absolute differences of pixel variances of adjacent edges of adjacent sub-blocks in the horizontal ternary tree; is the absolute difference of the pixel variances of adjacent edges of adjacent sub-blocks in the vertical binary tree; and are the absolute differences of pixel variances of adjacent edges of adjacent sub-blocks in the vertical ternary tree.
2. The VVC multi-tree structure fast decision method based on texture characteristics according to claim 1, characterized in that: In step 101, the coding information of the CU to be divided is read, including the division depth, position, size and channel information of the current CU; The judgment of whether the current CU is the VVC default forced division CU refers to the CU at the coding frame boundary and the CU that generates division redundancy; the CU at the coding frame boundary refers to the CU that exceeds the right boundary or the bottom boundary of the image; division redundancy refers to the situation where the same coding structure is generated for the same CU although the division method and order are different.
3. The VVC multi-tree structure fast decision method based on texture characteristics according to claim 1, characterized in that: The step 102 uses an edge detection operator to extract the gradient features of the current CU, specifically including: Use one of the operators of Sobel, Laplace, Scharr, and Canny to perform planar convolution with the CU pixel matrix through the operator template in the horizontal and vertical directions, take the absolute value of the calculated value and divide it by the number of CU pixels to obtain the average gradient value Grad in the horizontal direction of the CU h And the average gradient value Grad in the vertical direction v .
4. The VVC multi-tree structure fast decision method based on texture characteristics according to claim 1, characterized in that: In step 103, the horizontal average gradient value of the current coded frame and the vertical average gradient value of the current coded frame refer to the video encoder using a certain edge detection operator to calculate the horizontal average gradient value and the vertical average gradient value of the current coded frame. When the Scharr operator is used, the calculation method of the horizontal average gradient value and the vertical average gradient value of the current coded frame is as shown in formulas (1) and (2). In the above formula, P is the pixel matrix of the current coded frame; Gh pic Gv is the average horizontal gradient value of the current coded frame; pic is the average vertical gradient value of the current coded frame; W and H are the width and height of the current coded frame respectively.
5. The VVC multi-tree structure fast decision method based on texture characteristics according to claim 1, characterized in that: In step 103, it is determined whether the current CU is to be divided into multiple trees, that is, if the horizontal average gradient value Grad h Less than the average horizontal gradient value Gh of the current coded frame pic , and the vertical average gradient value Grad of the current CU v Less than the average vertical gradient value Gv of the current coded frame pic When , the current CU no longer performs multi-tree division, otherwise, the current CU performs multi-tree division.
6. The VVC multi-tree structure fast decision method based on texture characteristics according to claim 1, characterized in that: In step 104, the main texture direction of the current CU is determined by formula (3). After the main texture direction is obtained, it is marked by the division direction flag dir_flag; Among them, ThD is the threshold for judging the direction of the current CU texture, which can be obtained through experimental statistics or learning; If dir_flag=0, the current CU texture is mainly vertical, and the horizontal HBT and HTT division methods are skipped; if dis_flag=1, the current CU texture is mainly horizontal, and the vertical VBT and VTT division methods are skipped; if dir_flag=-1, it indicates that the current CU texture direction is fuzzy, and the main direction of the current CU texture cannot be determined. The VVC original platform CU division method is used to determine the best division mode.
7. The VVC multi-tree structure fast decision method based on texture characteristics according to claim 1, characterized in that: In step 105, the variance of the edge pixels of the adjacent sub-blocks of the current CU is calculated, specifically including: when the current CU is divided in the horizontal direction or the vertical direction, the variance of the adjacent edge pixels of the CU sub-block is calculated; if it is determined that the current CU texture is mainly horizontal, the variance value of the adjacent edge pixels of the adjacent sub-blocks of the horizontal ternary tree and the binary tree is calculated; for the three sub-blocks from top to bottom of the horizontal ternary tree, the variance value of the edge pixels of the bottom row of the first sub-block is calculated, and the variance value of the edge pixels of the bottom row of the first sub-block is obtained. Calculate the variance of edge pixels in the top row and the bottom row of the second sub-block, and get and Calculate the variance of the edge pixels in the top row of the third sub-block and get For the two sub-blocks from top to bottom of the horizontal binary tree, calculate the variance of the edge pixels in the bottom row of the first sub-block and get Calculate the variance of the edge pixels in the top row of the second sub-block and get If it is determined that the current CU texture is mainly vertical, the variance values of adjacent edge pixels of adjacent sub-blocks of the vertical ternary tree and binary tree are calculated; for the three sub-blocks from left to right in the vertical ternary tree, the variance value of the edge pixels in the rightmost column of the first sub-block is calculated, and the obtained Calculate the variance of the edge pixels in the leftmost column and the rightmost column of the second sub-block, and get and Calculate the variance of the edge pixels in the leftmost column of the third sub-block and get For the two sub-blocks from left to right in the vertical binary tree, calculate the variance of the edge pixels in the rightmost column of the first sub-block and get Calculate the variance of the edge pixels in the leftmost column of the second sub-block and get 8. A storage medium, characterized in that: The storage medium stores a computer program therein, and when the computer program is read by a processor, the method of any one of claims 1 to 7 is executed.
