Sample adaptive offset mode decision method and apparatus, medium, and terminal device

Abstract

The application belongs to the technical field of video coding, and particularly relates to a sample adaptive offset mode decision method and device, a computer readable storage medium and a terminal device. In the method, an optimal non-parametric fusion offset mode can be selected from various non-parametric fusion offset modes according to the division depth of a tree coding unit, and then an optimal offset mode can be selected from the optimal non-parametric fusion offset mode and a parametric fusion offset mode. Through the method, the optimal non-parametric fusion offset mode can be selected according to the division depth of the tree coding unit, the decision time in the non-parametric fusion offset mode is saved, the efficiency of the sample adaptive offset mode decision method is improved, and the method has strong usability and practicality.

Description

Technical Field

[0001] This application belongs to the field of video coding technology, and in particular relates to a sample adaptive compensation mode decision method, apparatus, computer-readable storage medium and terminal equipment. Background Technology

[0002] With the development of information technology, people have increasingly diverse ways of obtaining information. As an effective information transmission medium in modern society, video has also seen a continuous increase in people's demands for video quality. When video data is large, it is usually compressed before transmission. During the video encoding or decoding process, a "ringing effect" can easily occur. This effect occurs because image edges lose a lot of high-frequency information during the transformation / quantization process, resulting in a wavy effect around the image edges after the inverse transformation / inverse quantization.

[0003] To reduce the "ringing effect," High Efficiency Video Coding (HEVC) has introduced a new Sample Adjust Offset (SAO) technique. SAO starts at the pixel domain, classifying reconstructed pixels into different categories based on their characteristics, and then applying different compensation values ​​to each category. In the specific processing of SAO, the decision-making process for the compensation mode is the most crucial and time-consuming step; however, existing methods for determining the SAO mode are inefficient. Summary of the Invention

[0004] In view of this, embodiments of this application provide a sample point adaptive compensation mode decision-making method, apparatus, computer-readable storage medium, and terminal device to solve the problem of low efficiency in existing sample point adaptive compensation mode decision-making methods.

[0005] A first aspect of this application provides a sample point adaptive compensation mode decision-making method, which may include:

[0006] Obtain the partition depth of the tree coding unit;

[0007] Based on the aforementioned division depth, the optimal nonparametric fusion compensation mode is selected from various nonparametric fusion compensation modes;

[0008] The optimal compensation mode is selected from the optimal non-parametric fusion compensation mode and the optimal parametric fusion compensation mode.

[0009] In one specific implementation of the first aspect, selecting the optimal nonparametric fusion compensation mode from various nonparametric fusion compensation modes based on the partitioning depth may include:

[0010] If the partitioning depth is 0, then the optimal nonparametric fusion compensation mode is selected from the horizontal boundary compensation mode, the vertical boundary compensation mode, and the sideband compensation mode.

[0011] In one specific implementation of the first aspect, selecting the optimal nonparametric fusion compensation mode from various nonparametric fusion compensation modes based on the partitioning depth may include:

[0012] If the partitioning depth is not 0, then obtain the minimum number of intra-frame prediction units of the tree coding unit;

[0013] The optimal nonparametric fusion compensation mode is selected from various nonparametric fusion compensation modes based on the number of minimum intra-frame prediction units.

[0014] In one specific implementation of the first aspect, selecting the optimal nonparametric fusion compensation mode from various nonparametric fusion compensation modes based on the number of the minimum intra-frame prediction units may include:

[0015] If the number of the minimum intra-frame prediction units is greater than or equal to a preset number threshold, then the first rate-distortion cost of each of the minimum intra-frame prediction units in various intra-frame angle prediction modes is obtained respectively.

[0016] Based on the first rate-distortion cost, at least one candidate boundary compensation mode is selected from various boundary compensation modes.

[0017] The optimal nonparametric fusion compensation mode is selected from the candidate boundary compensation mode and the sideband compensation mode.

[0018] In one specific implementation of the first aspect, selecting at least one candidate boundary compensation mode from various boundary compensation modes based on the first rate-distortion cost may include:

[0019] Calculate the second rate-distortion cost of each of the minimum intra-frame prediction units under various boundary compensation modes based on the first rate-distortion cost;

[0020] The third rate-distortion cost of the tree coding unit under each of the various boundary compensation modes is calculated based on the second rate-distortion cost.

[0021] At least one of the candidate boundary compensation modes is selected from the various boundary compensation modes based on the third rate distortion cost.

[0022] In one specific implementation of the first aspect, calculating the second rate-distortion cost of each of the minimum intra-prediction units under various boundary compensation modes based on the first rate-distortion cost may include:

[0023] Calculate the average value of the first rate-distortion cost of the target minimum intra-prediction unit under various intra-angle prediction modes corresponding to the target boundary compensation mode; wherein, the target minimum intra-prediction unit is any one of the minimum intra-prediction units, and the target boundary compensation mode is any one of the boundary compensation modes.

[0024] The average value is determined as the second rate-distortion cost of the target minimum intra-frame prediction unit in the target boundary compensation mode.

[0025] In one specific implementation of the first aspect, selecting at least one candidate boundary compensation mode from various boundary compensation modes based on the third rate-distortion cost may include:

[0026] The boundary compensation mode with the lowest third rate distortion cost is selected as the candidate boundary compensation mode.

[0027] Calculate the ratio between the third rate distortion cost and the baseline cost for each of the various unselected boundary compensation modes; wherein the baseline cost is the minimum third rate distortion cost.

[0028] The boundary compensation mode whose ratio is less than a preset ratio threshold is selected as the candidate boundary compensation mode.

[0029] In one specific implementation of the first aspect, after obtaining the minimum number of intra-frame prediction units of the tree coding unit, the method may further include:

[0030] If the number of minimum intra-frame prediction units is less than the number threshold, then the optimal nonparametric fusion compensation mode is selected from various nonparametric fusion compensation modes.

[0031] A second aspect of the embodiments of this application provides a sample point adaptive compensation device, which may include:

[0032] The depth acquisition module is used to obtain the partitioning depth of the tree coding unit;

[0033] The first selection module is used to select the optimal nonparametric fusion compensation mode among various nonparametric fusion compensation modes according to the division depth.

[0034] The second selection module is used to select the optimal compensation mode from the optimal non-parametric fusion compensation mode and the parameter fusion compensation mode.

[0035] In one specific implementation of the second aspect, the first selection module may include:

[0036] The selection unit is used to select the optimal nonparametric fusion compensation mode among the horizontal boundary compensation mode, the vertical boundary compensation mode, and the sideband compensation mode if the division depth is 0.

[0037] In one specific implementation of the second aspect, the first selection module may further include:

[0038] The number acquisition unit is used to acquire the minimum intra-frame prediction unit number of the tree coding unit if the partitioning depth is not 0.

[0039] The compensation mode selection unit is used to select the optimal nonparametric fusion compensation mode from various nonparametric fusion compensation modes based on the number of the minimum intra-frame prediction units.

[0040] In one specific implementation of the second aspect, the compensation mode selection unit may include:

[0041] The cost acquisition subunit is used to acquire the first rate-distortion cost of each of the minimum intra-frame prediction units under various intra-frame angle prediction modes if the number of the minimum intra-frame prediction units is greater than or equal to a preset number threshold.

[0042] The compensation mode selection sub-unit is used to select at least one candidate boundary compensation mode from various boundary compensation modes based on the first rate distortion cost.

[0043] The fusion compensation mode selection sub-unit is used to select the optimal nonparametric fusion compensation mode from the candidate boundary compensation mode and the sideband compensation mode.

[0044] In one specific implementation of the second aspect, the compensation mode selection subunit may include:

[0045] The first cost calculation subunit is used to calculate the second rate-distortion cost of each of the minimum intra-frame prediction units under various boundary compensation modes based on the first rate-distortion cost.

[0046] The second cost calculation subunit is used to calculate the third rate-distortion cost of the tree coding unit under various boundary compensation modes based on the second rate-distortion cost.

[0047] The compensation mode selection subunit is used to select at least one of the candidate boundary compensation modes from various boundary compensation modes based on the third rate distortion cost.

[0048] In one specific implementation of the second aspect, the first cost calculation subunit may include:

[0049] The average value calculation subunit is used to calculate the average value of the first rate-distortion cost of the target minimum intra-prediction unit under various intra-angle prediction modes corresponding to the target boundary compensation mode; wherein, the target minimum intra-prediction unit is any one of the minimum intra-prediction units, and the target boundary compensation mode is any one of the boundary compensation modes.

[0050] A cost determination subunit is used to determine the average value as the second rate-distortion cost of the target minimum intra-frame prediction unit in the target boundary compensation mode.

[0051] In one specific implementation of the second aspect, the compensation mode selection subunit may include:

[0052] The first mode selection sub-unit is used to select the boundary compensation mode with the lowest rate distortion cost as the candidate boundary compensation mode.

[0053] The cost ratio calculation subunit is used to calculate the ratio between the third rate distortion cost and the reference cost for each of the various unselected boundary compensation modes; wherein the reference cost is the minimum third rate distortion cost.

[0054] The second mode selection subunit is used to select the boundary compensation mode whose ratio is less than a preset ratio threshold as the candidate boundary compensation mode.

[0055] In one specific implementation of the second aspect, the sample point adaptive compensation device may further include:

[0056] The mode selection module is used to select the optimal nonparametric fusion compensation mode among various nonparametric fusion compensation modes if the number of the minimum intra-frame prediction units is less than the number threshold.

[0057] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described sample adaptive compensation mode decision-making methods.

[0058] A fourth aspect of this application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described sample adaptive compensation mode decision-making methods.

[0059] The fifth aspect of this application provides a computer program product that, when run on a terminal device, causes the terminal device to execute the steps of any of the above-described sample adaptive compensation mode decision-making methods.

[0060] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment can select the optimal non-parametric fusion compensation mode from various non-parametric fusion compensation modes based on the partitioning depth of the tree coding unit. Then, the optimal compensation mode can be selected from the optimal non-parametric fusion compensation mode and the parametric fusion compensation mode. Through this application embodiment, the optimal non-parametric fusion compensation mode can be selected based on the partitioning depth of the tree coding unit, saving decision time in non-parametric fusion compensation modes, improving the efficiency of the sample point adaptive compensation mode decision-making method, and possessing strong usability and practicality. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 A schematic diagram of the overall process of HEVC video encoding;

[0063] Figure 2 This is a flowchart of one embodiment of a sample point adaptive compensation mode decision-making method in this application.

[0064] Figure 3 This is a schematic diagram showing the partitioning depth of a tree-based coding unit.

[0065] Figure 4 A schematic flowchart illustrating various boundary compensation modes;

[0066] Figure 5 This is a schematic diagram of the sidebands in the sideband compensation mode;

[0067] Figure 6 A statistical chart of the optimal nonparametric fusion compensation mode in surveillance video scenarios;

[0068] Figure 7 A schematic flowchart illustrating the selection process of candidate boundary compensation modes;

[0069] Figure 8 This is a schematic diagram of 33 intra-frame angle prediction modes;

[0070] Figure 9 This is a schematic diagram of a tree-shaped coding unit in the parameter fusion compensation mode;

[0071] Figure 10 This is a structural diagram of one embodiment of a sample point adaptive compensation device in this application.

[0072] Figure 11 This is a schematic block diagram of a terminal device in an embodiment of this application. Detailed Implementation

[0073] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0074] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0075] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0076] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0077] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0078] Furthermore, in the description of this application, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0079] With the development of information technology, people have increasingly diverse ways of obtaining information. As an effective information transmission medium in modern society, video has also seen a continuous increase in people's demands for video quality. When video data is large, it is usually compressed before transmission. During the video encoding or decoding process, a "ringing effect" can easily occur. This effect occurs because image edges lose a lot of high-frequency information during the transformation / quantization process, resulting in a wavy effect around the image edges after the inverse transformation / inverse quantization.

[0080] To reduce the "ringing effect," High Efficiency Video Coding (HEVC) incorporates Sample Adjust Offset (SAO) technology as a sub-module within the loop post-processing module of the video encoding and decoding process. The overall HEVC video encoding workflow can be found by referring to... Figure 1 It mainly includes several modules such as intra-frame prediction, inter-frame prediction (motion estimation and motion compensation), transform / quantization, inverse transform / inverse quantization, loop post-processing module (deblocking filtering and sample adaptive compensation) and entropy coding. Among them, the loop post-processing performs post-processing on the reconstructed image of the current encoded image during the video encoding process to improve the image quality and use it as a reference frame for subsequent images to be encoded, thereby improving the overall quality of video encoding.

[0081] The specific steps involved in sample-point adaptive compensation are as follows: (1) counting the number of pixels of different categories and their corresponding distortions within the Coding Tree Unit (CTU); (2) determining the optimal compensation mode and corresponding compensation value for sample-point adaptive compensation through rate-distortion optimization; and (3) performing pixel-by-pixel compensation on the CTU using the optimal compensation mode and corresponding compensation value. Among these steps, determining the compensation mode is the most important and time-consuming step; however, existing sample-point adaptive compensation mode decision-making methods are inefficient.

[0082] In view of this, embodiments of this application provide a sample-point adaptive compensation mode decision-making method, apparatus, computer-readable storage medium, and terminal device to solve the problem of low efficiency in existing sample-point adaptive compensation mode decision-making methods. Through embodiments of this application, the optimal non-parametric fusion compensation mode can be selected based on the partitioning depth of the tree coding unit, saving decision-making time in the non-parametric fusion compensation mode, improving the efficiency of the sample-point adaptive compensation mode decision-making method, and possessing strong ease of use and practicality.

[0083] It should be noted that the subject of execution of the method in this application is a terminal device, which can be a common computing device such as a desktop computer, a laptop, or a handheld computer, or other computing devices.

[0084] Please see Figure 2 One embodiment of the sample point adaptive compensation mode decision-making method in this application may include:

[0085] Step S201: Obtain the partitioning depth of the tree coding unit.

[0086] In the embodiments of this application, sample adaptive compensation can be performed on a tree coding unit basis.

[0087] It should be noted that an image can be divided into several non-overlapping tree coding units, and the internal structure of each tree coding unit can be further divided into coding units (CUs) using a cyclic hierarchical structure based on quadtrees. The partitioning depth can be 0, 1, 2, or 3. Please refer to [reference needed]. Figure 3 The tree coding units shown can have sizes of 64, 32, 16, and 8 corresponding to different partition depths. Coding units at the same level have the same partition depth. A tree coding unit may contain only one coding unit (i.e., no partitioning) or multiple coding units (i.e., multiple partitioning). Larger coding units can improve coding efficiency in flat regions, while smaller coding units can better handle local image details, thus making predictions of complex images more accurate. Therefore, tree coding units in image regions with complex textures or motion typically have larger partition depths to improve prediction accuracy, while tree coding units in image regions with simple textures or motion have smaller partition depths to improve coding efficiency.

[0088] In this embodiment, a compensation mode decision for adaptive compensation of sample points can be made based on the division depth of the tree coding unit. Here, the tree coding unit can be any tree coding unit in the image to be encoded.

[0089] Step S202: Select the optimal nonparametric fusion compensation mode from various nonparametric fusion compensation modes according to the division depth.

[0090] In this embodiment of the application, the optimal nonparametric fusion compensation mode can be selected from four edge offset (EO) modes and one band offset (BO) mode based on the partitioning depth of the tree coding unit.

[0091] It should be noted that the non-parametric fusion compensation modes in sample adaptive compensation can include a skip mode, four boundary compensation modes, and one sideband compensation mode. The skip mode means that no compensation is performed on any pixels of the tree coding unit. The boundary compensation mode classifies the current pixel by comparing its value with the values ​​of its two adjacent pixels, and then compensates pixels of the same category with the same value. Specifically, based on the position of adjacent pixels, the boundary compensation mode can be divided into four angular boundary compensation modes: horizontal boundary compensation mode (EO_0), vertical boundary compensation mode (EO_1), 135-degree boundary compensation mode (EO_2), and 45-degree boundary compensation mode (EO_3). Please refer to [link to relevant documentation]. Figure 4 Where c is the pixel value of the current pixel, and pixels a and b are the pixel values ​​of two adjacent pixels of c. Under any directional angle, all pixels of a tree-coded unit can be divided into 5 categories according to the following rules, and compensation values ​​are determined based on the categories. The division criteria for the 5 categories can be found in the table below:

[0092] type condition 1 c < a and c < b 2 c < a and c == b, or, c == a and c < b 3 c>a and c==b, or c==a and c>b 4 c>a and c>b 0 other

[0093] HEVC specifies that compensation values ​​for categories 1 and 2 must be greater than or equal to 0, while compensation values ​​for categories 3 and 4 must be less than or equal to 0. Category 0 receives no compensation. The sideband compensation mode categorizes pixels based on their intensity values, dividing the range of pixel intensity values ​​into 32 equal sidebands. (For example, see [reference needed]). Figure 5 For an 8-bit image, the pixel range is 0 to 255, so each sideband can contain 8 pixel values, and the pixels in each sideband can use the same compensation value for sample adaptive compensation.

[0094] In this embodiment, if the partitioning depth of the tree coding unit is 0, the optimal non-parametric fusion compensation mode can be selected from several preset non-parametric fusion compensation modes through a rate-distortion optimization process. Specifically, when the partitioning depth of the tree coding unit is 0, the number of times each non-parametric fusion compensation mode is selected as the optimal non-parametric fusion compensation mode in a specific application scenario can be counted, and the optimal non-parametric fusion compensation mode can be directly selected from the non-parametric fusion compensation modes that are selected as the optimal non-parametric fusion compensation mode most frequently in subsequent applications.

[0095] It is understood that the method of this application can be applied to specific scenarios, and the number of times each nonparametric fusion compensation mode is selected as the optimal nonparametric fusion compensation mode for sample adaptive compensation can be counted according to the specific application scenario. For example, if the method of this application needs to be specifically applied to scenario A, then in scenario A, the number of times each nonparametric fusion compensation mode is selected as the optimal nonparametric fusion compensation mode when the partition depth of the tree coding unit is 0 can be counted, that is, the number of times the skip mode, the four boundary compensation modes and the sideband compensation mode are selected as the optimal nonparametric fusion compensation mode can be counted respectively.

[0096] In this embodiment, the sample-point adaptive compensation mode decision-making method can be applied to surveillance video scenarios. Therefore, it is possible to count the number of times each non-parametric fusion compensation mode is selected as the optimal non-parametric fusion compensation mode under different time periods and monitoring environments. Specifically, in four specific scenarios—indoor daytime, outdoor daytime, indoor nighttime, and outdoor nighttime—the number of times each non-parametric fusion compensation mode is selected as the optimal non-parametric fusion compensation mode can be counted. The statistical results can be as follows: Figure 6 As shown (the skip mode is not included in the statistics because it requires relatively less computational resources), it can be seen that when the partitioning depth of the tree coding unit is 0, in surveillance video applications, the most frequently selected optimal nonparametric fusion compensation modes are the horizontal boundary compensation mode, the vertical boundary compensation mode, and the sideband compensation mode. Therefore, in this embodiment, if the partitioning depth of the tree coding unit is 0, the optimal nonparametric fusion compensation mode can be selected from the horizontal boundary compensation mode, the vertical boundary compensation mode, and the sideband compensation mode.

[0097] It is understandable that in engineering practice, the statistical period, environment, and number of statistical times can be specified and contextualized according to the actual application scenario of the sample point adaptive compensation mode decision-making method, including but not limited to the implementation methods of the embodiments of this application.

[0098] It should be noted that each coding unit in a tree coding unit can be divided into several prediction units (PUs) for prediction operations. In the embodiments of this application, when the partitioning depth of the tree coding unit is not 0, the optimal nonparametric fusion compensation mode can be selected based on the relationship between the minimum number of intra-frame prediction units of the tree coding unit and a preset number threshold.

[0099] Specifically, when the partitioning depth of the tree coding unit is not 0, the number of the smallest intra-prediction units in the tree coding unit can be obtained, and it can be determined whether the number of the smallest intra-prediction units is greater than or equal to a preset number threshold. If the number of the smallest intra-prediction units is greater than or equal to the number threshold, it can be considered that the angular prediction mode direction of the intra-prediction units in the current tree coding unit can better reflect the image texture direction in the current tree coding unit area. At this time, the first rate-distortion cost of each smallest intra-prediction unit under various intra-angle prediction modes can be obtained, and at least one candidate boundary compensation mode can be selected from various boundary compensation modes based on the first rate-distortion cost. The number threshold can be specified and contextualized according to actual needs. In this embodiment, the number threshold can preferably be set to half of the total number of the smallest prediction units in the tree coding unit.

[0100] Please see Figure 7 In this embodiment of the application, the process of selecting at least one candidate boundary compensation mode from various boundary compensation modes based on the first rate distortion cost may include the following steps:

[0101] Step S701: Calculate the second rate-distortion cost of each minimum intra-frame prediction unit under various boundary compensation modes based on the first rate-distortion cost.

[0102] It should be noted that before performing sample adaptive compensation, HEVC performs intra-frame prediction and obtains intra-frame prediction modes, which can include DC prediction mode, Plannar prediction mode, and 33 intra-frame angle prediction modes. The 33 intra-frame angle prediction modes are numbered 2 to 34. For details, please refer to [reference needed]. Figure 8 .

[0103] In this embodiment, it is preferable to calculate the first rate-distortion cost of each smallest intra-prediction unit under the 33 intra-angle prediction modes obtained during intra-frame prediction. Since the predicted pixels obtained by intra-frame prediction are calculated by determining the optimal angle prediction mode after comparing the cost of each angle prediction mode during rate-distortion optimization, the direction of the intra-angle prediction mode can reflect the texture direction of the current predicted image region to a certain extent. Furthermore, since the selection of the angle direction of the boundary compensation mode in the decision of the sample adaptive compensation mode is also closely related to the texture direction of the image region, in this embodiment, the information of the 33 intra-frame angle prediction modes of intra-frame prediction can be used to determine the optimal boundary compensation direction template in advance, thereby improving the efficiency of subsequently selecting the optimal non-parametric fusion compensation mode.

[0104] Specifically, if the angular direction of a certain intra-frame angle prediction mode is close to the angular direction of the boundary compensation mode, the angular direction of the intra-frame angle prediction mode can be mapped to the angular direction of the boundary compensation mode. Here, the angular directions of intra-frame angle prediction modes 6-14 can be mapped to the horizontal direction (0 degrees) of the boundary compensation mode, the angular directions of intra-frame angle prediction modes 22-30 can be mapped to the vertical direction (90 degrees) of the boundary compensation mode, the angular directions of intra-frame angle prediction modes 15-21 can be mapped to the 135-degree direction of the boundary compensation mode, and the angular directions of intra-frame angle prediction modes 2-5 and 31-34 can be mapped to the 45-degree direction of the boundary compensation mode. Then, the second rate-distortion cost of each smallest intra-frame prediction unit in the angular direction of the four boundary compensation modes can be obtained according to the first rate-distortion cost of the 33 intra-frame angle prediction modes according to the above mapping rules.

[0105] For ease of explanation, this section will use any one of the smallest intra-prediction units (denoted as the target smallest intra-prediction unit) and any one of the boundary compensation modes (denoted as the target boundary compensation mode) as examples to illustrate the calculation process of the second rate-distortion cost. For the target smallest intra-prediction unit, the average value of the first rate-distortion cost of the intra-angle prediction modes that have a mapping relationship with the target boundary compensation mode can be calculated. Then, this average value can be used as the second rate-distortion cost of the target smallest intra-prediction unit under the target boundary compensation mode. For example, if the target boundary compensation mode is the horizontal boundary compensation mode, then the intra-angle prediction modes that have a mapping relationship with the horizontal boundary compensation mode are intra-angle prediction modes 6 to 14. In this case, the average value of the first rate-distortion cost of intra-angle prediction modes 6 to 14 can be calculated, and this average value can be used as the second rate-distortion cost of the target smallest intra-prediction unit under the horizontal boundary compensation mode.

[0106] It is understandable that by traversing the various boundary compensation modes in the target intra-frame minimum prediction unit and repeating the above process, the second rate distortion cost of the target intra-frame minimum prediction unit under various boundary compensation modes can be obtained; by traversing each intra-frame minimum prediction unit and repeating the above process, the second rate distortion cost of each minimum intra-frame prediction unit under various boundary compensation modes can be obtained.

[0107] Step S702: Calculate the third rate-distortion cost of the tree coding unit under various boundary compensation modes based on the second rate-distortion cost.

[0108] In the embodiments of this application, the third rate-distortion cost of the tree coding unit under various boundary compensation modes can be calculated based on the second rate-distortion cost of each minimum intra-frame prediction unit under various boundary compensation modes.

[0109] Specifically, the second rate-distortion cost of each minimum intra-frame prediction unit under a certain boundary compensation mode can be summed to obtain the third rate-distortion cost of the tree coding unit under that boundary compensation mode. For example, the second rate-distortion cost of each minimum intra-frame prediction unit under the vertical boundary compensation mode can be summed to obtain the third rate-distortion cost of the tree coding unit under the vertical boundary compensation mode.

[0110] It is understandable that by traversing the various boundary compensation modes of each smallest intra-frame prediction unit and repeating the above process, the third rate distortion cost of the tree coding unit under various boundary compensation modes can be obtained.

[0111] Step S703: Select at least one candidate boundary compensation mode from various boundary compensation modes based on the third rate distortion cost.

[0112] In this embodiment of the application, if the third rate distortion cost of a certain boundary compensation mode meets the preset threshold condition, then the boundary compensation mode can be selected as a candidate boundary compensation mode.

[0113] Specifically, the boundary compensation mode with the smallest third rate-distortion cost can be selected as a candidate boundary compensation mode, and the smallest third rate-distortion cost can be used as the benchmark cost. Then, the ratio between the third rate-distortion cost of the unselected boundary compensation modes and the benchmark cost can be calculated. From these, the boundary compensation modes with a ratio less than a preset ratio threshold can be selected as candidate boundary compensation modes. The ratio threshold can be set according to actual needs and contextualization. This application does not make specific limitations on this. Here, it is preferable to set the ratio threshold to 1.2 times the smallest third rate-distortion cost. That is, among the unselected boundary compensation modes, the boundary compensation modes with a ratio less than 1.2 times the smallest third rate-distortion cost can be selected as candidate boundary compensation modes.

[0114] Understandably, since the boundary compensation mode with the lowest third-rate distortion cost is usually selected as the candidate boundary compensation mode, at least one candidate boundary compensation mode can be selected from various boundary compensation modes based on the third-rate distortion cost.

[0115] In this embodiment, the optimal nonparametric fusion compensation mode can be selected from candidate boundary compensation modes and sideband compensation modes. Specifically, in the tree coding unit, rate-distortion optimization calculation can be performed based on the distortion between the original pixels and the reconstructed pixels, the coding compensation mode, and the number of coded bits of its corresponding compensation value. The calculation formula can be:

[0116] J=D+λR

[0117] Where J represents the rate-distortion cost, D represents the distortion between the original and reconstructed pixels, λ is a constant related to the quantization parameters, and R represents the number of encoded bits for the coding compensation mode and its corresponding compensation value. Based on this, the rate-distortion costs of the candidate boundary compensation modes and the sideband compensation modes can be obtained, and the non-parametric fusion compensation mode with the lowest rate-distortion cost is selected as the optimal non-parametric fusion compensation mode for the tree coding unit. For example, both the horizontal boundary compensation mode and the 135-degree boundary compensation mode are candidate boundary compensation modes. Among the horizontal boundary compensation mode, the 135-degree boundary compensation mode, and the sideband compensation mode, the tree coding unit has the lowest rate-distortion cost in the 135-degree boundary compensation mode. Therefore, the 135-degree boundary compensation mode can be selected as the optimal non-parametric fusion compensation mode.

[0118] In this embodiment, after obtaining the number of minimum intra-prediction units of the current tree coding unit, if the number of minimum intra-prediction units is found to be less than a threshold, it can be considered that the angular direction of various intra-angle prediction modes of the tree coding unit is difficult to reflect the texture direction of the tree coding unit. If the decision on the optimal non-parametric fusion compensation mode is made based on various intra-angle prediction modes, it may affect the performance of HEVC. Therefore, at this time, the optimal non-parametric fusion compensation mode can be selected from various non-parametric fusion compensation modes. Specifically, rate-distortion optimization calculation can be performed based on the distortion between the original pixels and reconstructed pixels of the tree coding unit, the coding compensation mode, and the number of coded bits of its corresponding compensation value. The rate-distortion costs of the four boundary compensation modes and the rate-distortion costs of the sideband compensation mode are obtained, and the non-parametric fusion compensation mode with the smallest rate-distortion cost is selected as the optimal non-parametric fusion compensation mode.

[0119] It should be noted that a tree coding unit can contain a luminance component and a chrominance component. By performing the above process on the luminance component and the chrominance component in the tree coding unit respectively, the optimal nonparametric fusion compensation mode corresponding to the luminance component and the optimal nonparametric fusion compensation mode corresponding to the chrominance component can be obtained respectively.

[0120] Step S203: Select the optimal compensation mode from the optimal non-parametric fusion compensation mode and the parameter fusion compensation mode.

[0121] In this embodiment of the application, the optimal compensation mode of the tree coding unit can be selected based on the rate-distortion cost of the optimal non-parametric fusion compensation mode and the rate-distortion cost of the parameter fusion compensation mode.

[0122] It should be noted that the parameter fusion compensation mode refers to the current tree coding unit directly using the sample adaptive compensation parameters of its left-adjacent or top-adjacent tree coding unit when performing sample adaptive compensation. (See reference here.) Figure 9Where C is the current tree coding unit, and tree coding units A and B are the left and top adjacent tree coding units of the current tree coding unit, respectively. When using the parameter fusion compensation mode for sample adaptive compensation, it is only necessary to encode the flags of the sample adaptive compensation mode parameters of the adjacent tree coding units.

[0123] In this embodiment of the application, the rate-distortion cost of the optimal non-parametric fusion compensation mode and the rate-distortion cost of the parameter fusion compensation mode can be compared, and the compensation mode with the smaller rate-distortion cost can be selected as the optimal compensation mode.

[0124] It is understood that, in the embodiments of this application, after determining the optimal compensation mode of the tree coding unit, the optimal compensation mode and its corresponding compensation value can be input into the entropy coding module for encoding during the encoding process, and sample point adaptive compensation can be performed on the tree coding unit to obtain the compensated coding unit.

[0125] It should be noted that for the luminance and chrominance components of a tree coding unit, if the non-parametric fusion compensation mode is selected as the optimal compensation mode, the luminance and chrominance components can determine their compensation mode and compensation value according to the characteristics of their own pixels; if the parametric fusion compensation mode is selected as the optimal compensation mode, the luminance and chrominance components can simultaneously use the compensation parameters of the left adjacent or top adjacent tree coding units.

[0126] In this embodiment of the application, by traversing the tree coding units in the image to be encoded and repeatedly executing the above process, the sample point adaptive compensation of the image to be encoded can be completed.

[0127] It is understandable that, in practical applications, the sample adaptive compensation mode decision method of this application embodiment can reduce the overall time of the sample adaptive compensation module while reducing a certain rate distortion performance (BD performance). When the partition depth of the tree coding unit is 0, the 135-degree direction boundary compensation mode and the 45-degree direction boundary compensation mode can be excluded from the non-parametric fusion compensation modes; while when the partition depth of the tree coding unit is not 0, the optimal non-parametric fusion compensation mode can be selected from various non-parametric fusion compensation modes. As a reference only, under experimental conditions, this application embodiment can reduce the overall time of the sample adaptive compensation module by 42% while reducing BD performance by only 0.8%. The specific application scenarios and performance statistics of the dataset can be found in the table below:

[0128]

[0129] Wherein, BDBR is the increase in bit rate under the same peak signal-to-noise ratio (PSNR); when the value of BDBR is positive, it indicates that the BD performance has decreased; conversely, it indicates that the BD performance has improved; ΔT represents the increase in the overall time of the sample adaptive compensation module.

[0130] In summary, in this embodiment, the optimal non-parametric fusion compensation mode can be selected from various non-parametric fusion compensation modes based on the partitioning depth of the tree coding unit. Then, the optimal compensation mode can be selected from the optimal non-parametric fusion compensation mode and the parametric fusion compensation mode. Through this embodiment, the optimal non-parametric fusion compensation mode can be selected based on the partitioning depth of the tree coding unit, saving decision-making time in non-parametric fusion compensation modes, improving the efficiency of the sample point adaptive compensation mode decision-making method, and possessing strong usability and practicality.

[0131] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0132] Corresponding to the sample point adaptive compensation mode decision-making method described in the above embodiments, Figure 10 This diagram illustrates a structural diagram of an embodiment of a sample point adaptive compensation device provided in this application.

[0133] In this embodiment, a sample point adaptive compensation device may include:

[0134] The depth acquisition module 1001 is used to acquire the partitioning depth of the tree coding unit;

[0135] The first selection module 1002 is used to select the optimal nonparametric fusion compensation mode among various nonparametric fusion compensation modes according to the division depth.

[0136] The second selection module 1003 is used to select the optimal compensation mode from the optimal non-parametric fusion compensation mode and the parameter fusion compensation mode.

[0137] In one specific implementation of this application embodiment, the first selection module may include:

[0138] The selection unit is used to select the optimal nonparametric fusion compensation mode among the horizontal boundary compensation mode, the vertical boundary compensation mode, and the sideband compensation mode if the division depth is 0.

[0139] In one specific implementation of this application embodiment, the first selection module may further include:

[0140] The number acquisition unit is used to acquire the minimum intra-frame prediction unit number of the tree coding unit if the partitioning depth is not 0.

[0141] The compensation mode selection unit is used to select the optimal nonparametric fusion compensation mode from various nonparametric fusion compensation modes based on the number of the minimum intra-frame prediction units.

[0142] In one specific implementation of this application embodiment, the compensation mode selection unit may include:

[0143] The cost acquisition subunit is used to acquire the first rate-distortion cost of each of the minimum intra-frame prediction units under various intra-frame angle prediction modes if the number of the minimum intra-frame prediction units is greater than or equal to a preset number threshold.

[0144] The compensation mode selection sub-unit is used to select at least one candidate boundary compensation mode from various boundary compensation modes based on the first rate distortion cost.

[0145] The fusion compensation mode selection sub-unit is used to select the optimal nonparametric fusion compensation mode from the candidate boundary compensation mode and the sideband compensation mode.

[0146] In one specific implementation of this application embodiment, the compensation mode selection subunit may include:

[0147] The first cost calculation subunit is used to calculate the second rate-distortion cost of each of the minimum intra-frame prediction units under various boundary compensation modes based on the first rate-distortion cost.

[0148] The second cost calculation subunit is used to calculate the third rate-distortion cost of the tree coding unit under various boundary compensation modes based on the second rate-distortion cost.

[0149] The compensation mode selection subunit is used to select at least one of the candidate boundary compensation modes from various boundary compensation modes based on the third rate distortion cost.

[0150] In one specific implementation of this application embodiment, the first cost calculation subunit may include:

[0151] The average value calculation subunit is used to calculate the average value of the first rate-distortion cost of the target minimum intra-prediction unit under various intra-angle prediction modes corresponding to the target boundary compensation mode; wherein, the target minimum intra-prediction unit is any one of the minimum intra-prediction units, and the target boundary compensation mode is any one of the boundary compensation modes.

[0152] A cost determination subunit is used to determine the average value as the second rate-distortion cost of the target minimum intra-frame prediction unit in the target boundary compensation mode.

[0153] In one specific implementation of this application embodiment, the compensation mode selection subunit may include:

[0154] The first mode selection sub-unit is used to select the boundary compensation mode with the lowest rate distortion cost as the candidate boundary compensation mode.

[0155] The cost ratio calculation subunit is used to calculate the ratio between the third rate distortion cost and the reference cost for each of the various unselected boundary compensation modes; wherein the reference cost is the minimum third rate distortion cost.

[0156] The second mode selection subunit is used to select the boundary compensation mode whose ratio is less than a preset ratio threshold as the candidate boundary compensation mode.

[0157] In one specific implementation of this application embodiment, the sample point adaptive compensation device may further include:

[0158] The mode selection module is used to select the optimal nonparametric fusion compensation mode among various nonparametric fusion compensation modes if the number of the minimum intra-frame prediction units is less than the number threshold.

[0159] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0160] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0161] Figure 11 A schematic block diagram of a terminal device provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0162] like Figure 11 As shown, the terminal device 11 in this embodiment includes: a processor 110, a memory 111, and a computer program 112 stored in the memory 111 and executable on the processor 110. When the processor 110 executes the computer program 112, it implements the steps in the various sample adaptive compensation mode decision-making method embodiments described above, for example... Figure 2Steps S201 to S203 are shown. Alternatively, when the processor 110 executes the computer program 112, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 10 The functions of modules 1001 to 1003 are shown.

[0163] For example, the computer program 112 may be divided into one or more modules / units, which are stored in the memory 111 and executed by the processor 110 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 112 in the terminal device 11.

[0164] The terminal device 11 can be a desktop computer, laptop, handheld computer, or other computing device. Those skilled in the art will understand that... Figure 11 This is merely an example of terminal device 11 and does not constitute a limitation on terminal device 11. It may include more or fewer components than shown, or combine certain components, or different components. For example, terminal device 11 may also include input / output devices, network access devices, buses, etc.

[0165] The processor 110 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0166] The memory 111 can be an internal storage unit of the terminal device 11, such as a hard disk or memory of the terminal device 11. The memory 111 can also be an external storage device of the terminal device 11, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal device 11. Furthermore, the memory 111 can include both internal and external storage units of the terminal device 11. The memory 111 is used to store the computer program and other programs and data required by the terminal device 11. The memory 111 can also be used to temporarily store data that has been output or will be output.

[0167] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0168] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0169] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0170] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0171] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0172] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0173] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0174] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A sample-point adaptive compensation mode decision-making method, characterized in that, include: Obtain the partition depth of the tree coding unit; If the partitioning depth is not 0, then obtain the minimum number of intra-frame prediction units of the tree coding unit; If the number of the minimum intra-frame prediction units is greater than or equal to a preset number threshold, then the first rate-distortion cost of each of the minimum intra-frame prediction units in various intra-frame angle prediction modes is obtained respectively. Based on a preset mapping rule, at least one candidate boundary compensation mode is selected from various boundary compensation modes according to the first rate-distortion cost; wherein, the mapping rule is used to map various intra-frame angle prediction modes to corresponding boundary compensation modes. The optimal nonparametric fusion compensation mode is selected from the candidate boundary compensation mode and the sideband compensation mode. The optimal compensation mode is selected from the optimal non-parametric fusion compensation mode and the optimal parametric fusion compensation mode.

2. The sample point adaptive compensation mode decision-making method according to claim 1, characterized in that, Also includes: If the partitioning depth is 0, then the optimal nonparametric fusion compensation mode is selected from the horizontal boundary compensation mode, the vertical boundary compensation mode, and the sideband compensation mode.

3. The sample point adaptive compensation mode decision-making method according to claim 1, characterized in that, The step of selecting at least one candidate boundary compensation mode from various boundary compensation modes based on the first rate-distortion cost includes: Calculate the second rate-distortion cost of each of the minimum intra-frame prediction units under various boundary compensation modes based on the first rate-distortion cost; The third rate-distortion cost of the tree coding unit under each of the various boundary compensation modes is calculated based on the second rate-distortion cost. At least one of the candidate boundary compensation modes is selected from the various boundary compensation modes based on the third rate distortion cost.

4. The sample point adaptive compensation mode decision-making method according to claim 3, characterized in that, The step of calculating the second rate-distortion cost of each of the minimum intra-frame prediction units under various boundary compensation modes based on the first rate-distortion cost includes: Calculate the average value of the first rate-distortion cost of the target minimum intra-prediction unit under various intra-angle prediction modes corresponding to the target boundary compensation mode; wherein, the target minimum intra-prediction unit is any one of the minimum intra-prediction units, and the target boundary compensation mode is any one of the boundary compensation modes. The average value is determined as the second rate-distortion cost of the target minimum intra-frame prediction unit in the target boundary compensation mode.

5. The sample point adaptive compensation mode decision-making method according to claim 3, characterized in that, The step of selecting at least one candidate boundary compensation mode from various boundary compensation modes based on the third rate-distortion cost includes: The boundary compensation mode with the lowest third rate distortion cost is selected as the candidate boundary compensation mode. Calculate the ratio between the third rate distortion cost and the baseline cost for each of the various unselected boundary compensation modes; wherein the baseline cost is the minimum third rate distortion cost. The boundary compensation mode whose ratio is less than a preset ratio threshold is selected as the candidate boundary compensation mode.

6. The sample point adaptive compensation mode decision-making method according to any one of claims 1 to 5, characterized in that, After obtaining the minimum number of intra-prediction units of the tree coding unit, the method further includes: If the number of the minimum intra-frame prediction units is less than the number threshold, then the optimal nonparametric fusion compensation mode is selected from various nonparametric fusion compensation modes.

7. A sample point adaptive compensation device, characterized in that, include: The depth acquisition module is used to obtain the partitioning depth of the tree coding unit; The first selection module is configured to: if the partitioning depth is not 0, obtain the number of minimum intra-prediction units of the tree coding unit; if the number of minimum intra-prediction units is greater than or equal to a preset number threshold, obtain the first rate-distortion cost of each minimum intra-prediction unit under various intra-angle prediction modes; select at least one candidate boundary compensation mode from various boundary compensation modes based on a preset mapping rule and the first rate-distortion cost; wherein the mapping rule is used to map various intra-angle prediction modes to corresponding boundary compensation modes; and select the optimal non-parametric fusion compensation mode from the candidate boundary compensation modes and the sideband compensation mode. The second selection module is used to select the optimal compensation mode from the optimal non-parametric fusion compensation mode and the parameter fusion compensation mode.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the sample adaptive compensation mode decision method as described in any one of claims 1 to 6.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the sample adaptive compensation mode decision method as described in any one of claims 1 to 6.

Images