Image decoding device, image decoding method, and program

Abstract

To improve coding efficiency.SOLUTION: An image decoding device 200 according to an invention comprises: an intra-prediction part 204 which generates first predictive pixels based on decoded pixels and control information; a motion compensation part 205 which generates second predictive pixels based on the decoded pixels and the control information; and a color difference prediction part 206 which predicts the color difference of the first predictive pixels and / or the second predictive pixels by applying a plurality of prediction formulas using a prediction coefficient, in accordance with a threshold based on the luminance of the first predictive pixels and / or the second predictive pixels.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image decoding apparatus, an image decoding method, and a program.

Background Art

[0002] Non-Patent Document 1 and Non-Patent Document 2 disclose a multi-model cross component linear model (MMLM).

[0003] The MMLM predicts a color difference pixel signal (hereinafter referred to as color difference) of a block to be decoded by applying two different linear models to a decoded luminance pixel signal (hereinafter referred to as luminance).

[0004] Specifically, in the MMLM, as shown in FIG. 2, first, the average of the luminance Y' in the vicinity of the block to be decoded is obtained and used as a threshold Th.

[0005] Second, the neighboring luminance Y' is divided into a neighboring luminance Y'0 greater than the threshold Th and a neighboring luminance Y'1 less than or equal to the threshold Th.

[0006] Third, coefficients (α0, β0) are obtained by the least squares method from the color difference C'0 corresponding to the luminance Y'0 and the luminance Y'0. Similarly, coefficients (α1, β1) are obtained by the least squares method from the color difference C'1 corresponding to the luminance Y'1 and the luminance Y'1.

[0007] On the other hand, the luminance Y of the block to be decoded is divided into a luminance Y0 of the block to be decoded greater than the threshold Th and a luminance Y1 of the block to be decoded less than or equal to the threshold Th.

[0008] Fourth, predicted values C0 (= α0 × Y0 + β0) and C1 (= α1 × Y1 + β1) are calculated by applying the coefficients (α0, β0) and (α1, β1) to the luminance Y0 and Y1 of the block to be decoded, respectively, and used as the predicted value C of the color difference of the block to be decoded.

Prior Art Documents

Non-Patent Documents

[0009] [Non-Patent Document 1] ITU-T H.266 VVC [Non-Patent Document 2] M. Coban, et al., “Algorithm description of Enhanced Compression Model 10 (ECM 10)”, JVET-AF2025, 2023 [Summary of the Invention] [Problems to be Solved by the Invention]

[0010] However, in Non-Patent Document 1 and Non-Patent Document 2, since the target for applying the two models is selected by the average of the neighboring luminance, the luminance distribution of the decoding target block is not considered, and there is a problem that it is not necessarily optimal.

[0011] Also, near the threshold at which the two models switch, discontinuities occur because the coefficients (α0, β0) and (α1, β1) are different, and there is room for improvement in improving the coding efficiency.

[0012] Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide an image decoding apparatus, an image decoding method, and a program with high coding efficiency. [Means for Solving the Problems]

[0013] A first feature of the present invention is an image decoding apparatus, which includes a decoding unit that decodes control information and quantization values, an inverse quantization unit that inverse quantizes the quantization values to obtain transform coefficients, an inverse transform unit that inverse transforms the transform coefficients to obtain prediction residuals, an intra prediction unit that generates a first predicted pixel based on decoded pixels and the control information, an accumulation unit that accumulates the decoded pixels, a motion compensation unit that generates a second predicted pixel based on the decoded pixels accumulated in the accumulation unit and the control information, and a color difference prediction unit that predicts a color difference of at least one of the first predicted pixel and the second predicted pixel by applying a plurality of prediction formulas using prediction coefficients according to a threshold based on the luminance of at least one of the first predicted pixel and the second predicted pixel.

[0014] A second feature of the present invention is an image decoding method, which includes a step of decoding control information and quantization values, a step of inverse quantizing the quantization values to obtain transform coefficients, a step of inverse transforming the transform coefficients to obtain prediction residuals, a step of generating a first predicted pixel based on decoded pixels and the control information, a step of accumulating the decoded pixels, a step of generating a second predicted pixel based on the accumulated decoded pixels and the control information, and a step of predicting a color difference of at least one of the first predicted pixel and the second predicted pixel by applying a plurality of prediction formulas using prediction coefficients according to a threshold based on the luminance of at least one of the first predicted pixel and the second predicted pixel.

[0015] A third feature of the present invention is a program that causes a computer to function as an image decoding device, where the image decoding device includes a decoding unit that decodes control information and quantization values, an inverse quantization unit that inverse quantizes the quantization values to obtain transform coefficients, an inverse transform unit that inverse transforms the transform coefficients to obtain prediction residuals, an intra prediction unit that generates a first prediction pixel based on decoded pixels and the control information, an accumulation unit that accumulates the decoded pixels, a motion compensation unit that generates a second prediction pixel based on the decoded pixels accumulated in the accumulation unit and the control information, and a color difference prediction unit that predicts a color difference of at least one of the first prediction pixel and the second prediction pixel by applying a plurality of prediction formulas using prediction coefficients according to a threshold value based on the luminance of at least one of the first prediction pixel and the second prediction pixel.

Advantages of the Invention

[0016] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program with high coding efficiency.

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components or the like, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0019] <First Embodiment> Hereinafter, with reference to FIGS. 1 to 8, the image decoding apparatus 200 according to the present embodiment will be described. FIG. 1 is a diagram showing an example of the functional blocks of the image decoding apparatus 200 according to the present embodiment.

[0020] As shown in FIG. 1, the image decoding apparatus 200 includes a code input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transform unit 203, an intra prediction unit 204, a motion compensation unit 205, a color difference prediction unit 206, an adder 207, an accumulation unit 208, and an image output unit 220.

[0021] The code input unit 210 is configured to acquire code information encoded by an image encoding apparatus.

[0022] The decoding unit 201 is configured to decode control information and quantization values from the coded information input from the code input unit 210. For example, the decoding unit 201 is configured to output control information and quantization values by performing variable-length decoding on such coded information.

[0023] Here, the quantization values are sent to the inverse quantization unit 202, and the control information is sent to the intra prediction unit 204, the motion compensation unit 205, and the color difference prediction unit 206. Note that such control information includes information necessary for controlling the inverse quantization unit 202, the intra prediction unit 204, the motion compensation unit 205, the color difference prediction unit 206, etc., and may include header information such as a sequence parameter set, a picture parameter set, a picture header, and a slice header.

[0024] The inverse quantization unit 202 is configured to inverse-quantize the quantization values sent from the decoding unit 201 to obtain transform coefficients. Such transform coefficients are sent to the inverse transform unit 203.

[0025] The inverse transform unit 203 is configured to inverse-transform the transform coefficients sent from the inverse quantization unit 202 to obtain a prediction residual. Such a prediction residual is sent to the adder 207.

[0026] The intra prediction unit 204 is configured to generate a first prediction pixel for adding to the prediction residual by the adder 207 based on the decoded pixel obtained via the adder 207 and the control information decoded by the decoding unit 201. Such a first prediction pixel is sent to the adder 207 or the color difference prediction unit 206.

[0027] The motion compensation unit 205 is configured to generate a second prediction pixel for adding to the prediction residual by the adder 207 based on the decoded pixel obtained by referring to the storage unit 208 and the control information decoded by the decoding unit 201. Such a second prediction pixel is sent to the adder 207 or the color difference prediction unit 206.

[0028] The accumulation unit 208 is configured to cumulatively accumulate the decoded pixels sent from the adder 207. Such decoded pixels receive a reference from the motion compensation unit 205 via the accumulation unit 208.

[0029] The adder 207 is configured to add the prediction residual sent from the inverse transform unit 203 and either the first predicted pixel or the second predicted pixel input from the intra prediction unit 204 and the motion compensation unit 205 to obtain a decoded pixel. Such decoded pixels are sent to the image output unit 220, the accumulation unit 208, and the intra prediction unit 204.

[0030] The image output unit 220 outputs the decoded pixels sent from the adder 207.

[0031] (Color difference prediction unit 206) The color difference prediction unit 206 is configured to change the color difference between the first predicted pixel and the second predicted pixel based on the first predicted pixel (including color difference and luminance) and the second predicted pixel (including color difference and luminance) respectively received from the intra prediction unit 204 and the motion compensation unit 205, and return them to the intra prediction unit 204 and the motion compensation unit 205 respectively.

[0032] Hereinafter, an example of a method for predicting the color difference of the first predicted pixel or the second predicted pixel by the color difference prediction unit 206 will be described.

[0033] The role of the color difference prediction unit 206 is to derive an optimal prediction coefficient for the color difference of the block to be decoded in order to accurately compensate the color difference of the block to be decoded in the subsequent adder 207, and predict the luminance of the input predicted pixel according to the prediction coefficient.

[0034] The color difference prediction unit 206 performs color difference prediction by applying a plurality of prediction formulas to the luminance.

[0035] For example, when the color difference prediction unit 206 generates a predicted value C(x, y) of the color difference at coordinates (x, y) within the block to be decoded from the luminance Y(x, y), it defines a prediction formula using prediction coefficients (α0, β0) and (α1, β1). The following formula is an example of defining a prediction formula with two types of prediction coefficients using a threshold Th.

[0036] C(x, y) = α0 × Y(x, y) + β0, Y(x, y) > Th C(x, y) = α1 × Y(x, y) + β1, Y(x, y) ≤ Th Alternatively, the color difference prediction unit 206 may define a prediction formula using a plurality of luminances Y(x, y), Y(x - 1, y), … Y(x + n, y + n). The following formula is an example of defining a prediction formula using five luminances including the upper, lower, left, and right around the coordinates (x, y).

[0037] C(x, y) = α0 × Y(x, y) + β0 × Y(x, y - 1) + γ0 × (x - 1, y) + Δ0 × Y(x + 1, y) + ε0 × Y(x, y + 1) + η0, Y(x, y) > Th C(x, y) = α1 × Y(x, y) + β1 × Y(x, y - 1) + γ1 × (x - 1, y) + Δ1 × Y(x + 1, y) + ε1 × Y(x, y + 1) + η1, Y(x, y) ≤ Th FIG. 3 shows an example illustrating the above formula. However, for the sake of brevity of notation, Y(x + i, y + j) is represented as Y i,j here.

[0038] As the threshold Th, N options can be prepared and a specific value can be selected.

[0039] For example, when N = 3, the color difference prediction unit 206 prepares three types: the representative value (average value, median value, etc.) Y’a of the luminance of neighboring pixels, the average value (Y’a + Y’max) / 2 of the representative value Y’a and the maximum value Y’max, and the average value (Y’a + Y’min) / 2 of the representative value Y’a and the minimum value Y’min. Then, it divides the neighboring pixels into two using each as the threshold Th, derives the prediction coefficients respectively, and can select the threshold with a small prediction error when applied to the block to be decoded.

[0040] Alternatively, the color difference prediction unit 206 may equally divide the luminance distribution of neighboring pixels into N options, or may set the number of pixels at the time of division to be equal.

[0041] Alternatively, for the threshold value Th, the color difference prediction unit 206 may limit it to K (K > N) types according to the luminance of the block to be decoded from N types of options.

[0042] For example, the color difference prediction unit 206 can also limit it to two threshold values close to the average value Ya of the luminance of the block to be decoded with K = 2.

[0043] Alternatively, the color difference prediction unit 206 may determine it to be a threshold value close to the average value Ya of the luminance of the block to be decoded with K = 1.

[0044] In any case, by restricting the options of the threshold value, an effect of reducing the processing load can be obtained. Also, by reflecting the luminance distribution of the block to be decoded, an effect of improving the prediction accuracy can be obtained.

[0045] As shown in FIG. 4, the color difference prediction unit 206 derives prediction coefficients from the luminance and color difference in the vicinity of the block to be decoded.

[0046] Specifically, the color difference prediction unit 206 derives prediction coefficients α and β such that the luminance Y' in the vicinity of the block to be decoded and the color difference C' in the vicinity match by prediction (FIG. 4(1)).

[0047] For example, the color difference prediction unit 206 defines an error function E of the following formula and derives prediction coefficients α and β that minimize E.

[0048] E = Σ(C'(x, y) - (α × Y'(x, y) + β)) 2 When deriving such prediction coefficients, the color difference prediction unit 206 may use the least squares method or the like. Alternatively, when deriving such prediction coefficients, the color difference prediction unit 206 may use robust estimation such as principal component regression or partial least squares regression in order to reduce the influence of outliers.

[0049] The color difference prediction unit 206 applies the derived prediction coefficients α and β to the luminance Y of the block to be decoded (Fig. 4(2)) and obtains the predicted value C of the color difference of the block to be decoded (Fig. 4(3)).

[0050] According to the above configuration, by deriving the prediction coefficients from the neighboring pixels, it is not necessary to decode the prediction coefficients, so that the effect of improving the coding efficiency can be obtained.

[0051] Here, the color difference prediction unit 206 uses all or part of the neighboring pixels within a certain distance range for the neighboring luminance Y' and color difference C' used to derive the prediction coefficients.

[0052] For example, as shown in Fig. 5(1), the color difference prediction unit 206 may be set to use the neighboring pixels within 6 lines from the block to be decoded when deriving the prediction coefficients.

[0053] Conversely, the color difference prediction unit 206 may limit the range of the neighboring pixels used when deriving the prediction coefficients. For example, when deriving the prediction coefficients, the color difference prediction unit 206 may use only the neighboring pixels in the region located above the block to be decoded (Fig. 5(2)), or may use only the neighboring pixels in the region located to the left of the block to be decoded (Fig. 5(3)).

[0054] Note that by making it possible to select a plurality of ranges for specifying the neighboring pixels, an appropriate prediction coefficient can be derived, and the effect of improving the coding efficiency can be obtained.

[0055] Alternatively, depending on the color space, since the number of luminance pixels and the number of color difference pixels are different, downsampling for associating luminance pixels and color difference pixels is used, but the color difference prediction unit 206 may use the pixels after downsampling or may use the pixels before downsampling.

[0056] The range of the neighboring pixels for deriving the prediction coefficients may be given fixedly or may be set variably.

[0057] For example, depending on the threshold value, the number of pixels for deriving the prediction coefficient may be reduced. Therefore, the color difference prediction unit 206 may set the range of neighboring pixels for deriving the prediction coefficient to be equal to or greater than a certain number of pixels.

[0058] Alternatively, the color difference prediction unit 206 may change the range of neighboring pixels for deriving the prediction coefficient according to the block size and aspect ratio.

[0059] Further, the color difference prediction unit 206 may perform color difference prediction by calculating a weighted average for a plurality of predicted values C(x, y) of color differences. FIG. 6 shows an example of a weighted average for a plurality of prediction results.

[0060] As shown in FIG. 6, first, the color difference prediction unit 206 calculates a threshold value Th from at least one of the luminance in the vicinity of the block to be decoded and the luminance of the block to be decoded.

[0061] Second, the color difference prediction unit 206 derives coefficients α0 and β0 from the luminance greater than the threshold value among the luminance in the vicinity of the block to be decoded and the color difference in the vicinity of the block to be decoded corresponding to such luminance by using the least squares method or the like.

[0062] Similarly, the color difference prediction unit 206 derives coefficients α1 and β1 from the luminance less than or equal to the threshold value among the luminance in the vicinity of the block to be decoded and the color difference corresponding to such luminance by using the least squares method or the like.

[0063] Third, the color difference prediction unit 206 sets a width M to which the weighted average is applied.

[0064] Fourth, the color difference prediction unit 206 calculates prediction coefficients α and β between the threshold value Th - M and the threshold value Th + M. Here, when calculating the prediction coefficients α and β, it is desirable for the color difference prediction unit 206 to set the predicted value at the threshold value Th - M and the predicted value at the threshold value Th + M to be continuous.

[0065] Fifthly, the color difference prediction unit 206 calculates a predicted value C of the color difference by applying α0 and β0 to the luminance greater than the threshold Th + M among the luminance values of the block to be decoded, applying α1 and β1 to the luminance less than or equal to the threshold Th - M, and applying α and β to the luminance between the threshold Th - M and the threshold Th + M.

[0066] Multiple options can be prepared for the width M, and a specific value can be selected. For example, M = 32, 64, 128 can be prepared, and after deriving the prediction coefficients α and β with each as the width M, the width M with the smallest prediction error when applied to the block to be decoded can be selected.

[0067] It is desirable to change the value of M according to the bit depth of the luminance value. By applying different widths according to the distribution of the luminance of the block to be decoded, the effect of improving the prediction accuracy can be obtained. Also, the effect of improving the prediction accuracy can be obtained by preventing discontinuity in the color difference corresponding to the luminance near the threshold.

[0068] The color difference prediction unit 206 may specify the width M by control information, or may specify the width M by the block size or the block aspect ratio.

[0069] The color difference prediction unit 206 can also be specified by control information while restricting the options of the width M by the block size or the block aspect ratio.

[0070] Alternatively, the color difference prediction unit 206 can set the width M asymmetrically. That is, the color difference prediction unit 206 may set the widths M0 and M1, and calculate the predicted value C of the color difference by applying α and β to the luminance between the threshold Th + M0 and the threshold Th - M1.

[0071] Alternatively, the color difference prediction unit 206 may apply a weighted average that changes the weight w according to the position information without deriving the prediction coefficients α and β. The weight w may be set linearly or non-linearly.

[0072] The following is an example that defines a weighted average by applying a linear weight w calculated for each coordinate (x, y).

[0073] C(x, y) = (1 - w)×(α0×Y(x, y)+β0)+w×(α1×Y(x, y)+β1) w(x, y) = a×x + b×y + c According to such a configuration, by deriving the coefficients a, b, and c from neighboring pixels by the least squares method or the like, an effect can be obtained in which the prediction accuracy can be improved while suppressing the amount of the coefficients a, b, and c.

[0074] Alternatively, the color difference prediction unit 206 may apply a weighted average in which the weight w is changed according to the difference d (= Th - Y(x, y)) from the threshold Th without deriving the prediction coefficients α and β. The weight w may be set linearly or non-linearly.

[0075] The following is an example that defines a weighted average with a linear weight w in a range of width M centered on the threshold Th.

[0076] The weight w in the following formula is set to 0 when it is -M, set to 1 when it is +M, and set to connect 0 and 1 with a straight line (to internally divide) when it is between -M and +M.

[0077] C(x, y) = (1 - w)×(α0×Y(x, y)+β0)+w×(α1×Y(x, y)+β1) w = 0, d ≤ -M w = (d + M) / 2M, -M < d < +M w = 1, d ≥ +M According to such a configuration, when there are many prediction coefficients, an effect can be obtained in which the processing load can be reduced.

[0078] Hereinafter, the control information decoded by the decoding unit 201 regarding the color difference prediction method will be described.

[0079] The coded information input to the image decoding apparatus 200 can include a sequence parameter set (SPS) that aggregates control information in units of sequences. Further, such coded information can include a picture parameter set (PPS) or a picture header (PH) that aggregates control information in units of pictures. Such coded information may include a slice header (SH) that aggregates control information in units of slices.

[0080] Hereinafter, with reference to FIG. 7, a method for setting a method for predicting a color difference in units of sequences will be described.

[0081] As shown in FIG. 7, in step S101, the decoding unit 201 determines whether sps_lm_enabled_flag is 1. sps_lm_enabled_flag is a syntax that controls the presence or absence of color difference prediction. When sps_lm_enabled_flag is 1, it indicates that color difference prediction is effective, and when sps_lm_enabled_flag is 0, it indicates that color difference prediction is ineffective.

[0082] If Yes, this process proceeds to step S102, and if No, this process ends.

[0083] In step S102, the decoding unit 201 decodes sps_lm_mode. sps_lm_mode is a syntax that controls the method for predicting a color difference.

[0084] By using sps_lm_mode, it is possible to change the prediction method according to the image characteristics in units of sequences, so that an effect of maximizing the coding efficiency can be expected.

[0085] For example, for a sequence composed of CG, since the luminance change is often steep, it can be set to narrow the width, and for a sequence composed of natural images, since the luminance change is often not steep, it can be set to widen the width, thereby maximizing the coding efficiency.

[0086] When setting the method for predicting the color difference in picture units, the decoding unit 201 also decodes pps_lm_enabled_flag and pps_lm_mode in the picture parameter set or the picture header in the same way.

[0087] By using pps_lm_mode, the method for prediction according to the image characteristics can be set and changed in picture units, so an effect of maximizing the coding efficiency can be expected.

[0088] For example, for a picture composed of CG, since the luminance change is often steep, it can be set to narrow the width, and for a picture composed of natural images, since the luminance change is often not steep, it can be set to widen the width, and the maximization of the coding efficiency can be achieved.

[0089] When setting the method for predicting the color difference in slice units, the decoding unit 201 also decodes sh_lm_enabled_flag and sh_lm_mode in the slice header in the same way.

[0090] By using sh_lm_mode, the method for prediction according to the image characteristics can be set and changed in slice units, so an effect of maximizing the coding efficiency can be expected.

[0091] For example, for a slice area composed of CG, since the luminance change is often steep, it can be set to narrow the width, and for a slice area composed of natural images, since the luminance change is often not steep, it can be set to widen the width, and the maximization of the coding efficiency can be achieved.

[0092] It is also possible to suppress an increase in the amount of code by setting only in the upper layer, or adaptive control can be performed by setting in the lower layer and giving priority to the setting in the lower layer.

[0093] Alternatively, when the method for predicting the color difference is set in advance, the decoding of such a method for predicting the color difference itself can be omitted.

[0094] In the above example, the method of setting the color difference prediction method in sequence units, picture units, or slice units was described. However, without setting these, the method may be directly selected in block units described later. In this case, the increase in the above header information can be avoided.

[0095] Hereinafter, with reference to FIG. 8, the method of setting the method of applying color difference prediction in block units will be described.

[0096] As shown in FIG. 8, in step S201, the decoding unit 201 determines whether sps_lm_enabled_flag, pps_lm_enabled_flag, or sh_lm_enabled_flag is 1.

[0097] If any of sps_lm_enabled_flag, pps_lm_enabled_flag, or sh_lm_enabled_flag is 1, this process proceeds to step S202. If none of sps_lm_enabled_flag, pps_lm_enabled_flag, or sh_lm_enabled_flag is 1, this process ends.

[0098] In step S202, the decoding unit 201 decodes cu_lm_mode, which is a control signal representing the color difference prediction method.

[0099] In step S203, the decoding unit 201 determines whether the color difference prediction method is of one type.

[0100] If Yes, this process ends. If No, this process proceeds to step S204.

[0101] In step S204, the decoding unit 201 decodes cu_lm_idx, which is a control signal for specifying the color difference prediction method from among a plurality of prediction methods.

[0102] cu_lm_idex is decoded to identify one of the prediction methods set by the lowest layer lm_mode applied to the block to be decoded.

[0103] According to the present embodiment, since decoding is performed by predicting the color difference using an adaptive threshold or weighted average using at least one of the luminance in the vicinity of the block to be decoded and the luminance of the block to be decoded, the coding efficiency can be improved.

[0104] The above-described image decoding apparatus 200 may be a program that causes a computer to execute each function (each process) and may be realized.

Industrial Applicability

[0105] According to the present embodiment, for example, since an improvement in overall service quality can be realized in moving image communication, it is possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Build resilient infrastructure, promote sustainable industrialization, and foster innovation."

Explanation of Signs

[0106] 200... Image decoding apparatus 201... Decoding unit 202... Inverse quantization unit 203... Inverse transform unit 204... Intra prediction unit 205... Motion compensation unit 206... Color difference prediction unit 207... Adder 208... Accumulation unit 210... Symbol input unit 220... Image output unit

Claims

1. An image decoding apparatus, comprising: a decoding unit that decodes control information and quantization values; an inverse quantization unit that inverse quantizes the quantization values to obtain transform coefficients; an inverse transform unit that inverse transforms the transform coefficients to obtain prediction residuals; an intra prediction unit that generates a first predicted pixel based on decoded pixels and the control information; an accumulation unit that accumulates the decoded pixels; a motion compensation unit that generates a second predicted pixel based on the decoded pixels accumulated in the accumulation unit and the control information; a color difference prediction unit that predicts at least one color difference of the first predicted pixel and the second predicted pixel by applying a plurality of prediction formulas using prediction coefficients according to a threshold value based on at least one luminance of the first predicted pixel and the second predicted pixel.

2. The image decoding apparatus according to claim 1, wherein the color difference prediction unit prepares a plurality of options as the threshold value and selects the threshold value from among the plurality of options.

3. The image decoding apparatus according to claim 1, wherein the color difference prediction unit selects, as the threshold value, a threshold value with a small prediction error when applied to a decoding target block.

4. The image decoding apparatus according to claim 2, wherein the color difference prediction unit restricts options of the threshold value according to the luminance of a decoding target block.

5. The image decoding apparatus according to claim 4, wherein the color difference prediction unit restricts the options of the threshold value to a predetermined number of threshold values close to an average value of the luminance of the decoding target block.

6. The image decoding apparatus according to claim 5, wherein the predetermined number is 1.

7. The image decoding apparatus according to claim 1, wherein the color difference prediction unit variably sets a range of neighboring pixels for deriving the prediction coefficients.

8. The image decoding apparatus according to claim 1, wherein the color difference prediction unit sets neighboring pixels used for deriving the prediction coefficients to be equal to or more than a certain number of pixels.

9. The image decoding apparatus according to claim 1, wherein the color difference prediction unit calculates a weighted average for a plurality of predicted values of the color difference.

10. The image decoding apparatus according to claim 9, wherein the color difference prediction unit sets a width for applying the weighted average.

11. The image decoding apparatus according to claim 9, wherein the color difference prediction unit performs the weighted average so that predicted values of the color difference are continuous.

12. The image decoding apparatus according to claim 10, wherein the color difference prediction unit prepares a plurality of options for the width and selects a specific value from among the options.

13. The image decoding apparatus according to claim 12, wherein the color difference prediction unit selects, from among the options, a width with a small prediction error when applied to the block to be decoded.

14. The image decoding apparatus according to claim 9, wherein the color difference prediction unit changes a weight in the weighted average according to position information.

15. The image decoding apparatus according to claim 9, wherein the color difference prediction unit changes a weight in the weighted average according to a difference between luminance and the threshold value.

16. An image decoding method, comprising: a step of decoding control information and quantization values; a step of inverse quantizing the quantization values to obtain transform coefficients; a step of inverse transforming the transform coefficients to obtain prediction residuals; a step of generating a first predicted pixel based on the decoded pixels and the control information; a step of accumulating the decoded pixels; a step of generating a second predicted pixel based on the accumulated decoded pixels and the control information; a step of predicting at least one color difference of the first predicted pixel and the second predicted pixel by applying a plurality of prediction formulas using prediction coefficients according to a threshold value based on at least one luminance of the first predicted pixel and the second predicted pixel.

17. A program for causing a computer to function as an image decoding apparatus, the image decoding apparatus comprising: a decoding unit that decodes control information and quantization values; an inverse quantization unit that inverse quantizes the quantization values to obtain transform coefficients; an inverse transform unit that inverse transforms the transform coefficients to obtain prediction residuals; an intra prediction unit that generates a first predicted pixel based on the decoded pixels and the control information; an accumulation unit that accumulates the decoded pixels; a motion compensation unit that generates a second predicted pixel based on the decoded pixels accumulated in the accumulation unit and the control information; a color difference prediction unit that predicts at least one color difference of the first predicted pixel and the second predicted pixel by applying a plurality of prediction formulas using prediction coefficients according to a threshold value based on at least one luminance of the first predicted pixel and the second predicted pixel. ​

Images