Image decoding device, image decoding method, and program

Abstract

An image decoding device 200 according to an embodiment includes an intra prediction unit 204 that replaces a part or all of intra prediction with PDP in TIMD and applies the PDP on the basis of a template cost calculated by a prediction method having a smaller calculation amount than the PDP.

Description

Image Decoding Device, Image Decoding Method, and Program

[0005]

[0001] Cross - reference to related applications This application claims priority based on Japanese Patent Application No. 2024 - 186181 filed on October 22, 2024, and all the contents of that patent application are incorporated herein by reference.

[0002] The present disclosure relates to an image decoding device, an image decoding method, and a program.

[0003] Intra - prediction is disclosed in Non - Patent Document 1. Extended methods of the intra - prediction disclosed in Non - Patent Document 1 are disclosed in Non - Patent Documents 2 to 4.

[0004] Also, Template - based intra Mode Derivation (TIMD) is disclosed in Non - Patent Documents 1 and 2.

[0005] Also, Position Dependent Prediction (PDP) is disclosed in Non - Patent Documents 1 and 3, which holds weights obtained by pre - learning as a Matrix (matrix) table and performs a sum - of - products operation with neighboring pixels of a decoding target block to obtain predicted pixels.

[0006] Further, Non - Patent Document 4 discloses applying PDP to TIMD.

[0007] M. Coban, et al., “Algorithm description of Enhanced Compression Model 13 (ECM 13),” JVET-AH2025, 2024.K. Cao et al., “JVET-W0123: Fusion for template-based intra mode derivation,” JVET-W0123, 2021.B. Ray, et al., “EE2-2.13: Matrix based intra prediction replacing conventional intra modes,” JVET-AH0209, 2024.Y. Kidani, et al., “Non-EE2: Matrix-based position dependent intra prediction for TIMD,” JVET-AI0085, 2024.

[0008] The image decoding device according to this embodiment is an image decoding device comprising an intra prediction unit that replaces part or all of the intra prediction with a PDP in TIMD, and applies the PDP based on a template cost calculated using a prediction method that requires less computation than the PDP.

[0009] The image decoding method according to this embodiment is an image decoding method comprising the step of replacing part or all of the intra prediction with a PDP in TIMD, and applying the PDP based on a template cost calculated using a prediction method that requires less computation than the PDP.

[0010] The program according to this embodiment is a program that causes a computer to function as an image decoding device, wherein the image decoding device comprises an intra-prediction unit that replaces part or all of the intra-prediction with a PDP and applies the PDP based on a template cost calculated using a prediction method that requires less computation than the PDP.

[0011] Figure 1 shows an example of the functional block of the image decoding device 200 according to the first embodiment. Figure 2 shows an example of the template cost calculated by the intra prediction unit 204 of the image decoding device 200 according to the first embodiment. Figure 3 shows an example of the range of targets for calculating the template cost by PDP. Figure 4 shows an example of the range of targets for calculating the template cost by PDP.

[0012] The following description of this embodiment will be made with reference to the drawings. Note that the components in the following embodiment can be replaced with existing components as appropriate, and various variations are possible, including combinations with other existing components. Therefore, the following description of the embodiment does not limit the scope of the claims.

[0013] Non-patent document 4 uses PDP in part of TIMD, but PDP has the problem of being computationally intensive and therefore burdensome to process.

[0014] In particular, the decoding process has the problem of being significantly affected by increased processing load because the proportion of computational time described in Non-Patent Document 4 to the overall process is larger compared to the encoding process.

[0015] In view of the above-mentioned problems, this disclosure provides an image decoding device, an image decoding method, and a program with high encoding efficiency.

[0016] <First Embodiment> The image decoding device 200 according to this embodiment will be described below with reference to Figures 1 to 4.

[0017] The image decoding device 200 according to the first embodiment is designed for a variety of image signals (hereinafter referred to as "images"). For example, the image decoding device 200 according to this embodiment is designed for YUV (YCbCr) images composed of luminance pixels and chrominance pixels, RGB images composed of RGB pixels, and monochrome images. Here, each pixel constituting an image has discrete values ​​(pixel values) with a predetermined bit width.

[0018] Furthermore, in the following descriptions of the functions of each part, the term "pixel" may refer to a block (unit) composed of pixels, a tree block which is the largest block size, or a slice, tile, or image (picture) larger than a tree block.

[0019] Figure 1 is a diagram showing an example of the functional block of the image decoding device 200 according to this embodiment.

[0020] As shown in Figure 1, the image decoding device 200 includes a decoding unit 201, an inverse quantization unit 202, an inverse transformation unit 203, an intra prediction unit 204, a motion compensation unit 205, an adder 206, and an accumulation unit 207.

[0021] The decoding unit 201 is configured to decode control information and quantization values ​​from the coded information encoded by the image encoding device. For example, the decoding unit 201 is configured to output control information and quantization values ​​by performing variable-length decoding on such coded information.

[0022] Here, the quantized values ​​are sent to the inverse quantization unit 202, and the control information is sent to the intra prediction unit 204 and the motion compensation unit 205. This control information includes information necessary for controlling the inverse quantization unit 202, the intra prediction unit 204, and the motion compensation unit 205, and may also include header information such as a sequence parameter set, a picture parameter set, a picture header, and a slice header.

[0023] The inverse quantization unit 202 is configured to inverse quantize the quantized values ​​sent from the decoding unit 201 to obtain conversion coefficients. These conversion coefficients are then sent to the inverse conversion unit 203.

[0024] The inverse transformation unit 203 is configured to inversely transform the transformation coefficients sent from the inverse quantization unit 202 to obtain the predicted residual. This predicted residual is then sent to the adder 206.

[0025] The intra-prediction unit 204 is configured to generate a first prediction pixel for addition with the prediction residual in the adder 206, based on the decoded pixel obtained via the adder 206 and the control information decoded by the decoding unit 201. This first prediction pixel is sent to the adder 206.

[0026] The motion compensation unit 205 is configured to generate a second predicted pixel for addition with the predicted residual in the adder 206, based on the decoded pixel obtained by referring to the storage unit 207 and the control information decoded by the decoding unit 201. This second predicted pixel is sent to the adder 206.

[0027] The storage unit 207 is configured to cumulatively store the decoded pixels sent from the adder 206. These decoded pixels are referenced by the motion compensation unit 205 via the storage unit 207.

[0028] The adder 206 is configured to obtain decoded pixels by adding the predicted residual sent from the inverse transformer 203 with the first predicted pixels or second predicted pixels input from the intra-prediction unit 204 or the motion compensation unit 205, respectively. These decoded pixels are output as an image and sent to the storage unit 207 and the intra-prediction unit 204.

[0029] (Intra Prediction Unit 204) The intra prediction unit 204, which is a characteristic configuration of the image decoding device 200 according to this embodiment, will be described below. Specifically, an example of a method for decoding the first predicted pixel in the intra prediction unit 204 will be described.

[0030] Firstly, the intra-prediction unit 204 replaces part or all of the intra-prediction used in TIMD as an extended intra-prediction method with PDP.

[0031] Here, the intra-prediction unit 204 calculates the template cost in intra-prediction, regardless of whether or not the PDP is replaced.

[0032] Figure 2 shows an example of the template cost calculated by the intra prediction unit 204.

[0033] In Figure 2, the intra-prediction unit 204 calculates the template cost for the intra-prediction mode limited to MPM (Most Probable Mode).

[0034] Since the angle mode of TIMD is extended to twice that of the angle mode of intra-prediction, the MPM of intra-prediction is limited to even numbers. In this case, calculating the template cost using PDP would be computationally intensive, so the intra-prediction unit 204 calculates the template cost using intra-prediction even in modes that have been replaced by PDP.

[0035] Next, the intra prediction unit 204 calculates the template cost using PDP only for modes that have a cost below the threshold T1 (first threshold).

[0036] Here, it is desirable for the intra-prediction unit 204 to set a threshold T1 based on the minimum cost.

[0037] For example, if the threshold T1 is N1 times the minimum cost, and the minimum cost is 900 and N1 is 2, the intra prediction unit 204 calculates the template cost by PDP only for modes among those replaced by PDP that have a cost of threshold T1 = 1800 or less (see Figure 3).

[0038] While N1 can be set to any value, it is desirable to set it to a value that can be expressed as a combination of powers of two, such as 1.5, 1.25, or 1.125, in order to reduce the processing load. With this configuration, the calculation process for the PDP template cost can be kept to a minimum, resulting in faster processing.

[0039] Alternatively, the intra-prediction unit 204 may calculate template costs using PDP only for up to N2 (a predetermined number) modes surrounding the mode with the lowest cost.

[0040] For example, if the mode with the lowest cost is 100 and N2 has 10 values, the intra-prediction unit 204 calculates the template cost using PDP only for the modes from 90 to 110 that have been replaced by PDP (see Figure 4). With this configuration, the calculation process for the PDP template cost can be kept to the bare minimum, resulting in faster processing.

[0041] Alternatively, the intra prediction unit 204 may use the template cost of intra prediction multiplied by a threshold value T2 (second threshold value) as the template cost of the PDP.

[0042] For example, when the threshold value T2 is 0.9, the intra prediction unit 204 multiplies all the template costs of the conventional intra prediction modes to be replaced by the PDP by 0.9. As a result, in the calculation process of the template cost required for deriving the intra prediction mode within the TIMD, the calculation process of the template cost of the PDP can be made unnecessary, and thus an effect of achieving high speed can be obtained.

[0043] The intra prediction unit 204 may set at least one of the above-described T1, N2, and T2 as a constant, or may set at least one of the above-described T1, N2, and T2 based on control information.

[0044] As such control information, the intra prediction unit 204 can use a block size, a block aspect ratio, or a quantization parameter.

[0045] Alternatively, the intra prediction unit 204 may set at least one of the above-described T1, N2, and T2 based on a mode in which the template cost is the minimum value.

[0046] For example, when the mode in which the template cost is the minimum value is the mode replaced by the PDP, since such a mode is highly likely to be selected, it is desirable to set T1, N2, and T2 to be small.

[0047] Conversely, when the mode in which the template cost is the minimum value is a mode not replaced by the PDP, since a mode replaced by another PDP is highly likely to be selected, it is desirable to set T1, N2, and T2 to be large.

[0048] Alternatively, for the Planar mode and the DC mode of the PDP, the intra prediction unit 204 may calculate the template cost by the PDP only for the smaller one of the template costs by the Planar mode and the DC mode of intra prediction.

[0049] This configuration allows for the omission of one step in calculating the template cost using the PDP, resulting in faster processing.

[0050] Alternatively, if the intra-prediction unit 204 determines that the template cost of the intra-prediction in Planar mode or DC mode exceeds a minimum cost threshold T3 (third threshold) multiple, it may omit calculating the corresponding template cost of the PDP in Planar mode or DC mode.

[0051] Here, it is desirable for the intra-prediction unit 204 to set such threshold T3 based on the minimum value of the template cost.

[0052] For example, if the threshold T3 is set to N3 times the minimum template cost, and the minimum template cost is 900 and N3 is 2, the intra-prediction unit 204 calculates the template cost using PDP only for modes in the Planar mode and DC mode of intra-prediction that have a cost of threshold T3 = 1800 or less.

[0053] Furthermore, if the calculation of template costs is omitted in both the Planar mode and DC mode of intra-prediction, the intra-prediction unit 204 does not need to be used for weighted averaging.

[0054] Furthermore, if one of the Planar mode or DC mode of the intra-prediction is omitted, the intra-prediction unit 204 uses the one that is not omitted for the weighted average, and compares the omitted one with the template cost of the angle mode of the intra-prediction.

[0055] This configuration allows for the omission of up to two template cost calculations using PDP, resulting in faster processing.

[0056] PDP can be applied not only to TIMD but also to other methods. For example, PDP can be applied when deriving the Most Probable Mode (MPM) for intra-prediction.

[0057] Alternatively, PDP can be applied to the partitioning determination of the Geometric Partitioning Mode (GPM) and / or to the MPM of the intra-prediction of the GPM.

[0058] Alternatively, PDP can be applied to Spatial Geometric Partitioning Mode (SGPM), Template-based Multiple Reference Line (TMRL), Combined Inter-Intra Prediction (CIIP), MPM for intra-prediction of color difference components, and fusion of color difference components.

[0059] This embodiment can also be applied to the calculation of template costs for those PDPs to improve processing speed.

[0060] According to the image decoding device 200 of this embodiment, when applying PDP to TIMD, the processing speed can be improved by limiting the calculation of template costs using PDP, depending on relatively simple template cost calculations, etc.

[0061] Furthermore, according to the image decoding device 200 of this embodiment, when part or all of the intra prediction is replaced with PDP in TIMD, the mode for calculating the template cost by PDP based on the template cost by intra prediction is restricted, thereby reducing the processing load.

[0062] The image decoding device 200 described above may be implemented as a program that causes a computer to execute each function (each process).

[0063] According to this embodiment, it is possible to provide an image decoding device, an image decoding method, and a program with high encoding efficiency.

[0064] Furthermore, according to this embodiment, for example, it is possible to achieve an overall improvement in service quality in video communication, thereby contributing to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote sustainable industrialization and expand innovation."

[0065] 200...Image decoding unit 201...Decoding unit 202...Inverse quantization unit 203...Inverse transformation unit 204...Intra prediction unit 205...Motion compensation unit 206...Adder 207...Storage unit

Claims

1. An image decoding device comprising an intra prediction unit in which, in TIMD, a part or all of the intra prediction is replaced with a PDP, and the PDP is applied based on a template cost calculated using a prediction method that requires less computation than the PDP.

2. The image decoding apparatus according to claim 1, wherein the intra prediction unit calculates the template cost by intra prediction, regardless of whether or not the PDP is replaced.

3. The image decoding apparatus according to claim 1, wherein the intra prediction unit calculates the template cost using the PDP only for modes having a cost within a first threshold.

4. The image decoding apparatus according to claim 3, wherein the intra prediction unit sets the first threshold based on the minimum cost.

5. The image decoding apparatus according to claim 1, wherein the intra prediction unit calculates the template cost using the PDP only for a predetermined number of modes surrounding the mode with the minimum cost.

6. The image decoding apparatus according to claim 1, wherein the intra-prediction unit obtains the template cost of the PDP by multiplying the template cost calculated by intra-prediction by a second threshold.

7. The image decoding apparatus according to claim 3, wherein the intra prediction unit sets the first threshold based on control information.

8. The image decoding device according to claim 5, wherein the intra prediction unit sets the predetermined number based on control information.

9. The image decoding apparatus according to claim 6, wherein the intra prediction unit sets the second threshold based on control information.

10. The image decoding apparatus according to claim 3, wherein the intra prediction unit sets a first threshold based on the mode in which the template cost is the minimum value.

11. The image decoding apparatus according to claim 5, wherein the intra prediction unit sets the predetermined number based on the mode in which the template cost is the minimum value.

12. The image decoding apparatus according to claim 6, wherein the intra prediction unit sets the second threshold based on the mode in which the template cost is the minimum value.

13. The image decoding apparatus according to claim 1, wherein the intra prediction unit calculates the template cost in the PDP using only the PDP with the smaller template cost for the Planar mode and DC mode of the intra prediction.

14. The image decoding apparatus according to claim 1, wherein the intra prediction unit omits calculating the template cost of the corresponding PDP in Planar mode or DC mode if the template cost of the intra prediction in Planar mode or DC mode exceeds a third threshold multiple of the minimum cost.

15. The image decoding apparatus according to claim 14, wherein the intra prediction unit sets the third threshold based on the minimum value of the template cost calculated by the intra prediction.

16. An image decoding method comprising the step of replacing part or all of the intra prediction with a PDP in TIMD, and applying the PDP based on a template cost calculated using a prediction method that requires less computation than the PDP.

17. A program that causes a computer to function as an image decoding device, wherein the image decoding device comprises an intra-prediction unit that replaces part or all of an intra-prediction with a PDP and applies the PDP based on a template cost calculated using a prediction method that requires less computation than the PDP.

Images