AVS3 intra-frame prediction mode rapid judgment method
By optimizing the rate distortion cost calculation formula and mode selection of AVS3 intra prediction mode, SAD and Bins are used instead of traditional methods, reducing the computational complexity and improving the encoding speed, and suitable for real-time encoding and transcoding applications.
Patent Information
- Application Number
- CN202510700390.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-28
- Publication Date
- 2025-07-18
AI Technical Summary
The calculation complexity of the AVS3 intra prediction mode judgment process is high, resulting in slow operation of the software and difficult to meet the real-time encoding requirements.
By optimizing the rate distortion cost calculation formula, SAD is used instead of SATD and Bins is used instead of Rate, rate distortion cost calculation is simplified, and different number of modes are selected according to the predicted block size to enter the RDO process, reducing the calculation steps and time.
It effectively reduces the computational complexity of AVS3 intra prediction mode judgment, improves encoding speed, and is suitable for real-time encoding and transcoding application scenarios.
Smart Images
Figure CN120343260A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to, in particular, a fast decision method for intra prediction modes in AVS3. Background Art
[0002] The digital audio and video coding standard AVS3 has 66 intra prediction modes, including DC, Plane, Bilinear, IPCM, and 62 angular modes [1] , such as Figure 2 shown.
[0003] With the development of AVS2 to the third generation, i.e., AVS3, the algorithm complexity of intra prediction is also getting higher and higher. In terms of the number of intra prediction modes, it has doubled from 33 in AVS2 to 66. Selecting the optimal prediction mode from these 66 prediction modes to encode the prediction block, the computational complexity has also increased exponentially. The official encoder reference software of AVS3 [2] is often used for algorithm research and academic exchanges, and not much optimization has been done on the encoding speed. The decision of AVS3 intra prediction modes refers to selecting an optimal one from the above 66 prediction modes as the final prediction mode to predict the current prediction block and obtain the predicted value. In the selection of 66 intra prediction modes, the common approach is as follows: 1) Coarse mode decision (RMD) is to select the top three prediction modes with the minimum rate-distortion cost from 65 prediction modes except the IPCM mode as the coarse modes. In the rate-distortion cost formula used in this process, Distortion is calculated using SATD, and Rate is the number of bits consumed by the prediction mode calculated through entropy coding. SATD requires performing Hadamard transform on the residuals, and entropy coding for the prediction mode requires a more complex calculation process, which greatly increases the calculation time and is not conducive to encoding acceleration; 2) Construct the most probable prediction mode (MPM) list, which contains two most probable prediction modes; 3) Combine the top three coarse modes in step 1) with the two most probable prediction modes in the most probable prediction mode (MPM) list, and send them into the complete RDO process, and then select an optimal one as the final prediction mode. In this process, for all prediction block sizes, such as prediction blocks of 32x32, 16x16, and 8x8 sizes, five prediction modes are passed in for rate-distortion optimization (RDO). This approach has a long calculation cycle because for larger-sized prediction blocks, the errors of predicted values brought by different prediction modes are not so sensitive, and fewer prediction modes can be passed in to select the best one, while for smaller-sized prediction blocks, the deviation of prediction modes has a greater impact on the predicted value, and more accurate selection is required, and more prediction modes can be passed in for selection of the best one.
[0004] The main problems existing in the background technology are as follows: The official encoder reference software of AVS3 selects an optimal prediction mode from 66 intra prediction modes, which takes a large amount of computing time, resulting in slow software operation speed. There are still many places that can be optimized, which is also the problem that this patent hopes to solve.
[0005] The difficulty in solving the above problems and defects is that the intra prediction mode decision process is lengthy, including rough mode selection, constructing the MPM list, mode refinement, etc., and the image block is recursively divided into prediction blocks of different sizes downward [3] For prediction blocks of different sizes, 66 prediction modes need to be traversed. From what perspective to reduce the process or the complexity in the process is the difficulty faced in solving the above problems.
[0006] The significance of solving the above problems and defects is that it can effectively reduce the computational complexity of AVS3 intra prediction mode decision. Select different numbers of prediction modes for prediction blocks of different sizes to enter the RDO process, reduce the prediction process, which is beneficial to realizing real-time encoding of the AVS3 encoder and promoting the application scenario of AVS3 real-time transcoding. Summary of the Invention
[0007] The present invention provides a fast decision method for AVS3 intra prediction mode, which reduces the computational complexity by optimizing the rate-distortion cost calculation formula, thereby accelerating the rough mode selection process, reducing the overall RDO time, and achieving the purpose of quickly deciding the optimal prediction mode.
[0008] The technical solution of the present invention is as follows: The fast decision method for AVS3 intra prediction mode of the present invention includes the following steps: S1. Select three intra prediction modes step by step from 66 intra prediction modes of the digital audio and video coding standard AVS3. In each step, the rate-distortion cost value is calculated by using the rate-distortion cost formula, and several optimal prediction modes are selected to enter the next step by comparing the rate-distortion cost values until the optimal three prediction modes are selected; S2. Construct the most probable prediction mode (MPM) list and obtain two most probable prediction modes and load them into the MPM list; S3. Optimize the quantity of the three optimal prediction modes selected in step S1 and the two most probable prediction modes in the most probable prediction mode (MPM) list in step S2, a total of five prediction modes.
[0009] Optionally, in the above AVS3 intra prediction mode fast decision method, in step S1, during the process of calculating the rate-distortion cost value, for the rate-distortion cost formula RDCost = SATD + lambda * Rate, where RDCost is the rate-distortion cost value, SATD is the sum of the absolute values of the differences after transformation, Lambda is the Lagrange coefficient, and Rate is the number of bits consumed when encoding the prediction mode, replace SATD with the sum of absolute differences SAD, and replace Rate with the number of binary digits Bins of the prediction mode, that is, RDCost = SAD + lambda * Bins, which reduces the time for calculating the RDCost of each prediction mode in each step.
[0010] Optionally, in the above AVS3 intra prediction mode fast decision method, 18 prediction modes are evenly selected from 66 prediction modes {0, 1, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32}, the rate-distortion cost value is calculated, and the top 10 prediction modes with the smallest rate-distortion cost value are selected. These 10 prediction modes are extended to both neighboring modes, and for the first time, they are extended to 20 modes; continue to calculate the rate-distortion cost value, select the top 6 prediction modes with the smallest rate-distortion cost value, and for the second time, they are extended to 16 modes; continue to calculate the rate-distortion cost value, select the top 3, and output them to the subsequent rate-distortion optimization process.
[0011] Optionally, in the above AVS3 intra prediction mode fast decision method, in step S3, the rate-distortion cost value is calculated again for five prediction modes. Among them, the three optimal prediction modes selected in step S1 have already been calculated and do not need to be recalculated. Only the rate-distortion cost values of the two most likely prediction modes in the most probable mode (MPM) list need to be calculated. The rate-distortion cost values of the five prediction modes are sorted in ascending order, and according to different prediction block sizes, the top several prediction modes with the smallest rate-distortion cost value are selected and sent to the rate-distortion optimization (RDO) process for complete rate-distortion optimization (RDO).
[0012] According to the technical solution of the present invention, the beneficial effects are as follows: Regarding the distortion variable D, that is, Distortion, in the rate-distortion cost formula using SAD instead of SATD eliminates the Hadamard transform process and saves calculation time.
[0013] Regarding the rate-distortion cost formula The prediction mode in [it] consumes the number of bits R, that is, Rate. Instead of using the complex process of calculating the number of bits consumed by the prediction mode through entropy coding, it directly uses the number of bins of the binary string after binarizing the prediction mode, which not only reduces the computational complexity but also reduces the data dependence for subsequent hardware parallel pipelining and software multi-threaded parallel coding, greatly improving the coding speed.
[0014] Regarding the problem that five prediction modes are uniformly selected to enter the complete RDO process, and the RDO process is also the most time-consuming, according to the size of different prediction blocks, different numbers of prediction modes are selected to enter the complete RDO process. For larger-sized prediction blocks, fewer prediction modes are selected to enter the complete RDO process, and for smaller-sized prediction blocks, more prediction modes are selected to enter the complete RDO process. This hierarchical processing method greatly improves the efficiency of the RDO process and saves RDO time.
[0015] To better understand and illustrate the concept, working principle, and inventive effect of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and through specific embodiments as follows: Description of the Drawings
[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings required for describing the specific embodiments or the prior art will be briefly introduced below.
[0017] Figure 1 is the flowchart of the fast decision method for the AVS3 intra prediction mode of the present invention; Figure 2 There are 66 existing AVS3 intra prediction modes; Figure 3 is a schematic diagram of the process of calculating the predicted value for intra prediction; Figure 4 is a schematic diagram of the positions of the current MxN-sized prediction block and its left and upper prediction units (PUs) in constructing the most probable prediction mode (MPM) list. Detailed Embodiments
[0018] To make the purpose, technical method, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific examples. These examples are merely illustrative and not limiting to the present invention.
[0019] The present invention mainly optimizes the selection process of the above 66 intra prediction modes, reduces the computational complexity, and reduces the calculation process, so as to achieve the purpose of accelerating the selection of the optimal intra prediction mode.
[0020] The fast decision method for the AVS3 intra prediction mode of the present invention includes the following steps: S1. Select three intra prediction modes step by step from 66 intra prediction modes of the digital audio and video coding standard AVS3. In each step, the rate-distortion cost value RDCost is calculated using the rate-distortion cost formula. By comparing the RDCost values, several optimal prediction modes are selected to enter the next step until the optimal three prediction modes are selected, and then enter the RDO process.
[0021] Specifically, for the distortion variable D, that is, Distortion, in the rate-distortion cost formula SAD is used instead of SATD, eliminating the Hadamard transform process and saving calculation time. For the number of bits R consumed by the prediction mode, that is, Rate, in the rate-distortion cost formula the number of binary strings Bins after binarization of the prediction mode is directly used instead of the complex process of calculating the number of bits consumed by the prediction mode using entropy coding. This not only reduces the calculation complexity but also reduces the data dependence for subsequent hardware parallel pipelining and software multi-threaded parallel coding, greatly improving the coding speed.
[0022] Select 18 from 66 prediction modes evenly {0, 1, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32}, and calculate RDCost = SATD + lambda * Rate, where RDCost is the rate-distortion cost value, SATD is the sum of the absolute values of the differences after transformation, Lambda is the Lagrange coefficient, and Rate is the number of bits consumed when encoding the prediction mode. During the calculation of RDCost, SATD is replaced by the sum of absolute errors SAD, and Rate is replaced by the number of binary digits Bins of the prediction mode, that is, RDCost = SAD + lambda * Bins. This improvement greatly reduces the time for calculating the RDCosts of each prediction mode in each step and improves the coding speed. Select the top 10 with the smallest RDCost, and these 10 prediction modes are extended to the neighboring modes on both sides. For example, Figure 2 the neighbors of prediction mode 4 are 3 and 5, the neighbors of prediction mode 6 are 5 and 7, and so on. It is extended to 20 for the first time. Continue to calculate RDCost = SATD + lambda * Rate, select the top 6 with the smallest RDCost, and extend to 16 for the second time. Continue to calculate RDCost = SATD + lambda * Rate, select the top 3, and output them to the subsequent RDO process.
[0023] The schematic diagram of the process of calculating the predicted value in intra prediction can be demonstrated by Figure 3 taking prediction mode 18 and prediction block size 8×8 as an example.
[0024] The prediction mode 18 exactly belongs to the angular prediction from the upper left to the lower right. By referring to the upper reference pixel and the left neighbor reference pixel and performing weighted calculation, the predicted value of each pixel point in the prediction block can be calculated.
[0025] This step mainly simplifies the rate-distortion cost calculation formula. It uses the sum of absolute differences (SAD) to replace the sum of the absolute values of the differences after transformation (SATD), and uses the number of binary digits Bins of the prediction mode to replace the Rate value, that is, RDCost = SAD + lambda * Bins. This simplification brings an improvement in the speed of the entire rough mode selection process with a relatively small BD-Rate loss. In practical application scenarios where the coding speed is the key concern, such as live broadcast, live relay, UAV video transmission, etc., it has important practical value. Specifically, the SAD calculation rule is shown in the following formula.
[0026] Among them, f(x, y) and f’(x, y) are the original pixel value and the predicted value of the image at the image coordinate (x, y), and MxN is the size of the image block.
[0027] The calculation process of the number of binary digits Bins of the prediction mode is as follows.
[0028] The optimal prediction mode ModeC of the current prediction block appears in the most probable prediction mode (MPM) list. If ModeC = MPM0, then ipm_code = -2; If ModeC = MPM1, then ipm_code = -1; The optimal prediction mode ModeC of the current prediction block does not appear in the most probable prediction mode (MPM) list. (1) Sort the most probable prediction mode (MPM) list in ascending order; (2) Compare ModeC with the candidate prediction modes in the most probable prediction mode (MPM) list respectively: if ModeC < MPM0, then ipm_code = ModeC; otherwise, if MPM0 <= ModeC < MPM1, then ipm_code = ModeC - 1; otherwise, if ModeC >= MPM1, then ipm_code = ModeC - 2; Finally, entropy coding is performed on the prediction mode ipm_code.
[0029] Before performing entropy coding on ipm_code, it needs to be binarized. The corresponding relationship between ipm_code and the binarized string is shown in Table 1 below.
[0030] Table 1 Corresponding relationship between ipm_code and binarized string It can be seen that binIdx represents the subscript of the binary bit after the prediction mode is binarized. If the optimal prediction mode ModeC of the current prediction block is not in the MPM list, the length of the binary string after binarizing ipm_code is at most Bins = 7, i.e., 0x1x2x3x4x5x6. If the optimal prediction mode ModeC of the current prediction block appears in the MPM list, the length of the binary string after binarizing ipm_code is at most Bins = 2, i.e., 1 x1. In this way, the rate-distortion cost formula can be optimized as follows: when the optimal prediction mode ModeC of the current prediction block is not in the MPM list, RDCost = SAD + labmda * 7; when the optimal prediction mode ModeC of the current prediction block appears in the MPM list, RDCost = SAD + labmda * 2. This saves the process of calculating the number of bits Rate consumed by each prediction mode using the entropy coding algorithm and improves the calculation efficiency.
[0031] S2. Construct the Most Probable Modes (MPM) list and obtain the two most probable prediction modes, which are loaded into the MPM list. The specific construction method is as follows: (1) If the optimal prediction modes of the left prediction unit (PU) and the upper prediction unit (PU) of the current prediction block are equal, then MPM0 = DC; if they are equal and both are the DC mode, then MPM1 = Bilinear, otherwise MPM1 = MAX(IPM_L, IPM_U), where IPM_L represents the optimal prediction mode of the left prediction unit (PU) of the current prediction block, and IPM_U represents the optimal prediction mode of the upper prediction unit (PU) of the current prediction block; (2) Otherwise, MPM0 = MIN(IPM_L, IPM_U), MPM1 = MAX(IPM_L, IPM_U); The current MxN-sized prediction block ( Figure 4 the current PU in Figure 4 ), and the schematic diagrams of the positions of its left prediction block (PU) and upper prediction block (PU) are as
[0032] shown.
[0033] The RDO calculation process is relatively complex and requires the reconstruction of the current prediction block. The reconstruction process includes prediction, transformation, quantization, inverse quantization, and inverse transformation to obtain the residual value. Adding the residual value to the predicted value gives the reconstructed pixel, and this process has a long and cumbersome loop. Usually, the three prediction modes roughly selected plus the two prediction modes in the MPM list, a total of five prediction modes, are sent into the RDO process for a complete RDO calculation. Each prediction mode goes through the RDO, that is, the above-mentioned reconstruction loop, which includes prediction, transformation, quantization, inverse quantization, and inverse transformation to obtain the residual value. Adding the residual value to the predicted value gives the reconstructed pixel, and this process has a long and cumbersome loop.
[0034] According to different prediction block sizes, the present invention selects different numbers of prediction modes for RDO. Such hierarchical processing can improve the efficiency of the RDO process and reduce the time of RDO. The specific number allocation of RDO candidate prediction modes is shown in Table 2 below.
[0035] Table 2 Number Allocation of RDO Candidate Prediction Modes Predicted block size Number of RDO candidate modes 64x64 1 32x32 2 16x16 2 8x8 3 4x4 3 The specific operations of step S3 include: recalculating the rate-distortion cost value RDCost for the above five prediction modes. Among them, the five prediction modes include the three prediction modes selected by the rough mode and the two most likely prediction modes in the MPM list. The three selected by the rough mode have been calculated and do not need to be recalculated. Only the RDCost of the two most likely prediction modes in the MPM list needs to be calculated. Sort the rate-distortion cost values (RDCost) of these five prediction modes in ascending order, and according to different prediction block sizes, select the first few with the smallest RDCost and send them into the RDO process for a complete RDO.
[0036] The present invention mainly optimizes the process of traversing 66 prediction modes during the RMD rough mode selection in step S1, which consumes a large amount of encoding time. By optimizing the rate-distortion cost calculation formula, the calculation complexity is reduced, thereby accelerating the rough mode selection process. Optimize the selection of five prediction modes to enter the RDO process in step S1. According to the sizes of different prediction blocks, select different numbers of prediction modes to enter the RDO process, reduce the overall RDO time, and achieve the purpose of quickly determining the optimal prediction mode. The present invention can effectively reduce the calculation complexity of the AVS3 intra-frame prediction mode decision. Select different numbers of prediction modes to enter the RDO process according to different sizes of prediction blocks, reduce the prediction process, and this method is beneficial to realizing real-time encoding of the AVS3 encoder and promoting the application scenario of AVS3 real-time transcoding.
[0037] The above description is the best embodiment based on the inventive concept and working principle of the invention. The above embodiments should not be construed as limiting the scope of protection of the claims. Combinations of other implementation manners and implementation methods according to the inventive concept of the present invention all fall within the scope of protection of the present invention.
[0038] References: [1]. M. Wang, F. Luo, Z. Wang, et al. “AVS3 Intra Prediction Mode Extension”, AVS-M4780, Jun. 2019. [2]. “AVS3 software repository,” / Public / codec / video_codec / HPM / HPM-6.0, Jan. 2020. [3]. J. Yao, B. Zhang, H. Wang, et al. “Intra Prediction Mode Decision Device and Method for AVS3”, Chinese Patent No. CN202411334238.8 2024.
Claims
1. A fast decision method for AVS3 intra prediction mode, characterized in that It includes the following steps: S1. Step by step, select three intra prediction modes from 66 intra prediction modes of the digital audio-visual coding standard AVS3. In each step, the rate-distortion cost value is calculated using the rate-distortion cost formula, and several optimal prediction modes are selected by comparing the rate-distortion cost values to enter the next step until the optimal three prediction modes are selected; S2. Construct the Most Probable Modes (MPM) list and obtain two most probable prediction modes; S3. Optimize the quantity of a total of five prediction modes, namely the three optimal prediction modes selected in step S1 and the two most probable prediction modes in the Most Probable Modes (MPM) list in step S2.
2. The AVS3 intra prediction mode fast decision method according to claim 1, characterized in that, In step S1, during the calculation of the rate-distortion cost value, for the rate-distortion cost formula RDCost = SATD + lambda * Rate, where RDCost is the rate-distortion cost value, SATD is the sum of the absolute values of the differences after transformation, Lambda is the Lagrange coefficient, and Rate is the number of bits consumed when encoding the prediction mode, replace SATD with the sum of absolute differences SAD and Rate with the number of binary digits Bins of the prediction mode, that is, RDCost = SAD + lambda * Bins, which reduces the time for calculating the RDCost of each prediction mode in each step.
3. The AVS3 intra prediction mode fast decision method according to claim 2, characterized in that, Evenly select 18 from 66 prediction modes {0, 1, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32}, calculate the rate-distortion cost value, select the top 10 with the smallest rate-distortion cost value, and expand these 10 prediction modes to the two neighboring modes. The first expansion results in 20; continue to calculate the rate-distortion cost value, select the top 6 with the smallest rate-distortion cost value, and the second expansion results in 16; continue to calculate the rate-distortion cost value, select the top 3, and output them to the subsequent rate-distortion optimization process.
4. The fast decision method for the intra prediction mode of AVS3 according to claim 1, characterized in that In step S3, calculate the rate-distortion cost value of the five prediction modes again. Among them, the three optimal prediction modes selected in step S1 have already been calculated and do not need to be recalculated. Only the rate-distortion cost values of the two most probable prediction modes in the Most Probable Modes (MPM) list need to be calculated. Sort the rate-distortion cost values of the five prediction modes in ascending order, and select the top several with the smallest rate-distortion cost value according to different prediction block sizes and send them into the Rate-Distortion Optimization (RDO) process for complete rate-distortion optimization (RDO).