Image decoding device, image decoding method and program

Abstract

To enhance encoding efficiency.SOLUTION: An image decoding device 200 includes a first intra-frame prediction unit 204 for generating a first predicted pixel based on decoded pixels and control information, a second intra-frame prediction unit 205 for generating a second predicted pixel by performing a weighted average on the decoded pixels, an accumulation unit 208 for accumulating the decoded pixels, an inter-frame prediction unit 206 for generating a third predicted pixel based on the accumulated decoded pixels and control information, and an adder 207 for adding a prediction residual and the first to third predicted pixels to obtain a decoded pixel.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image decoding device, an image decoding method, and a program. [Background technology]

[0002] Non-Patent Documents 1 to 3 disclose intra block copy (IBC).

[0003] In IBC, pixels are referenced from the decoded pixel area of ​​the frame to be decoded and used as predicted pixels for the block to be decoded. Specifically, in IBC, pixels are copied from the reference block indicated by the block vector of the block to be decoded and used as predicted pixels.

[0004] Non-Patent Document 4 discloses that IBC holds two block vectors. When IBC holds two block vectors, it refers to pixels from the decoded pixel area and performs simple averaging to obtain a predicted value of the block to be decoded, which is used as a predicted pixel. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] ITU-T H.266 / VVC [Non-Patent Document 2] X. [Non-Patent Document 3] M. Coban et al., “Algorithm description of Enhanced Compression Model 8 (ECM 8)”, JVET-AC2025, 2023 [Non-Patent Document 4] Y. Kidani et al., “Non-EE2: Bi-predictive IBC for natural and screen content”, JVET-AD0134, 2023. Summary of the Invention [Problem to be solved by the invention]

[0006] However, in Non-Patent Documents 1 to 3, there is only one block vector, so there is a problem that the prediction accuracy of the referenced pixel is not necessarily high.

[0007] In Non-Patent Document 4, by dividing the block vector into two, the respective reference pixels are averaged, so that when the prediction residuals of the two reference pixels are independent of each other, the variance of the prediction residuals of the reference pixels can be halved, and as a result, the prediction accuracy is improved, and the above-mentioned problem can be partially improved. However, since it is a simple average, there is still room for improvement in the coding performance.

[0008] Therefore, the present invention has been made in consideration of the above-mentioned problems, and an object of the present invention is to provide an image decoding device, an image decoding method, and a program with high coding efficiency. [Means for solving the problem]

[0009] A first feature of the present invention is an image decoding device comprising: a decoding unit that decodes control information and quantized values; an inverse quantization unit that inversely quantizes the quantized values ​​to obtain transform coefficients; an inverse transform unit that inversely transforms the transform coefficients to obtain predictive residuals; a first intra-frame prediction unit that generates a first predicted pixel based on decoded pixels and the control information; a second intra-frame prediction unit that generates a second predicted pixel by performing a weighted averaging on the decoded pixels; a storage unit that accumulates the decoded pixels; an inter-frame prediction unit that generates a third predicted pixel based on the accumulated decoded pixels and the control information; and an adder that adds the predictive residual and the first to third predicted pixels to obtain the decoded pixel.

[0010] A second feature of the present invention is an image decoding method comprising the steps of: decoding control information and quantized values; inversely quantizing the quantized values ​​to obtain transform coefficients; inversely transforming the transform coefficients to obtain predictive residuals; generating a first predicted pixel based on decoded pixels and the control information; generating a second predicted pixel by performing a weighted average on the decoded pixels; accumulating the decoded pixels; generating a third predicted pixel based on the accumulated decoded pixels and the control information; and obtaining the decoded pixel by adding the predictive residuals to the first to third predicted pixels.

[0011] A third feature of the present invention is a program for causing a computer to function as an image decoding device, the image decoding device comprising: a decoding unit that decodes control information and quantized values; an inverse quantization unit that inversely quantizes the quantized values ​​to obtain transform coefficients; an inverse transform unit that inversely transforms the transform coefficients to obtain predictive residuals; a first intra-frame prediction unit that generates a first predicted pixel based on decoded pixels and the control information; a second intra-frame prediction unit that generates a second predicted pixel by performing a weighted averaging on the decoded pixels; a storage unit that accumulates the decoded pixels; an inter-frame prediction unit that generates a third predicted pixel based on the accumulated decoded pixels and the control information; and an adder that adds the predictive residual and the first to third predicted pixels to obtain the decoded pixel. Effect of the Invention

[0012] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program with high coding efficiency. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing an example of functional blocks of an image decoding device 200 according to an embodiment. [Diagram 2]FIG. 2 is a diagram illustrating an example of a BV input to the second intra-frame predictor 205 of the image decoding device 200 according to an embodiment. [Diagram 3] FIG. 3 is a flowchart showing an example of a method for setting a correction method on a sequence basis. [Figure 4] FIG. 4 is a flowchart showing an example of a method for applying two BVs on a block-by-block basis. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0015] First Embodiment An image decoding device 200 according to this embodiment will be described below with reference to Figures 1 to 4. Figure 1 is a diagram showing an example of functional blocks of the image decoding device 200 according to this embodiment.

[0016] As shown in FIG. 1, the image decoding device 200 includes a code input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transformation unit 203, a first intra-frame prediction unit 204, a second intra-frame prediction unit 205, an inter-frame prediction unit 206, an adder 207, an accumulation unit 208, and an image output unit 220.

[0017] The code input unit 210 is configured to obtain code information coded by an image coding device.

[0018] The decoding unit 201 is configured to decode the control information and the quantized value from the code information input from the code input unit 210. For example, the decoding unit 201 is configured to output the control information and the quantized value by performing variable length decoding on the code information.

[0019] Here, the quantized value is sent to the inverse quantization unit 202, and the control information is sent to the first intra-frame prediction unit 204, the second intra-frame prediction unit 205, and the inter-frame prediction unit 206. Note that such control information includes information necessary for controlling the first intra-frame prediction unit 204, the second intra-frame prediction unit 205, the inter-frame prediction unit 206, etc., and may include header information such as a sequence parameter set, a picture parameter set, a picture header, a slice header, etc.

[0020] The inverse quantization unit 202 is configured to inverse quantize the quantized values ​​sent from the decoding unit 201 to generate transform coefficients. The transform coefficients are sent to the inverse transform unit 203.

[0021] The inverse transform unit 203 is configured to inversely transform the transform coefficients sent from the inverse quantization unit 202 to generate prediction residuals. The prediction residuals are sent to an adder 207.

[0022] The first intra-frame prediction unit 204 is configured to generate a first predicted pixel to be added to the prediction residual by the adder 207, based on the decoded pixel obtained via the adder 207 and the control information decoded by the decoding unit 201. The first predicted pixel is sent to the adder 207.

[0023] The inter-frame prediction unit 206 is configured to generate a third predicted pixel to be added to the prediction residual by the adder 207, based on the decoded pixel obtained by referring to the storage unit 208 and the control information decoded by the decoding unit 201. The third predicted pixel is sent to the adder 207.

[0024] The storage unit 208 is configured to cumulatively store the decoded pixels sent from the adder 207. The inter-frame prediction unit 206 refers to the decoded pixels via the storage unit 208.

[0025] The adder 207 is configured to add the prediction residual sent from the inverse transform unit 203 to any one of the first to third predicted pixels sent from the first intra-frame prediction unit 204, the second intra-frame prediction unit 205, and the inter-frame prediction unit 206 to obtain a decoded pixel. The decoded pixel is sent to the image output unit 220, the storage unit 208, the first intra-frame prediction unit 204, and the second intra-frame prediction unit 205.

[0026] (Second frame intra prediction unit 205) An example of a prediction method performed by the second intra-frame prediction section 205 will be described below.

[0027] The role of the second intra-frame prediction unit 205 is to derive one or more block vectors (hereinafter referred to as BV) for the block to be decoded, and to predict the pixels of the block referenced by the BV (second intra-frame prediction), in order to compensate the block to be decoded with high accuracy in the downstream adder 207.

[0028] An example of the second intra-frame prediction is intra-block copy (hereinafter, referred to as IBC) disclosed in Non-Patent Document 4.

[0029] In the following, as an example, a case will be described in which one of two BVs is in IBC BVP mode and the other is in IBC merge mode, as shown in FIG. 2, and input to second intra-frame prediction unit 205.

[0030] First, the BV in the IBC BVP mode is expressed as the following equation as a combination of the block vector prediction (hereinafter, abbreviated as BVP) and the block vector difference (hereinafter, abbreviated as BVD) of the BV.

[0031] BV=BVP+BVD In this case, in the IBC BVP mode coding, the image coding device constructs a BVP candidate list with the BVs held by neighboring blocks and BVs used in the past as BVP candidates, and signals the BVP as an index of the BVP candidate list. The BVD is derived and signaled as the difference of the BVP with respect to the BV, i.e., BVD=(BV-BVP).

[0032] On the other hand, in the IBC merge mode, the image encoding apparatus constructs a BVP candidate list from BVs held by neighboring blocks and BVs used in the past, and signals the BV as an index of the BVP candidate list.

[0033] In the IBC merge mode, the BVP candidate indicated by the index is derived (decoded) as the BV. In this way, the IBC merge mode can reduce the amount of coding because there is no BVD, but since the BV is limited, the prediction accuracy generally tends not to be higher than that of the IBC BVP mode.

[0034] Next, the second intra-frame prediction unit 205 is configured to generate a second predicted pixel by performing a weighted average on the regions of reference pixels referred to by each of the two BVs, BV1 and BV2.

[0035] Here, a plurality of weights for the weighted average are prepared in advance, and a weight index indicating the weight is signaled.

[0036] The second intra-frame prediction unit 205 is configured to derive the weight wi specified by the weight index i for the second predicted pixel P' by applying it to the pixel (reference pixel) P1 referenced by BV1 and the pixel (reference pixel) P2 referenced by BV2, as shown in the following equation.

[0037] P'=wi×P1+(1-wi)×P2 For example, when wi=4 / 8, 3 / 8, 5 / 8, 10 / 8, -2 / 8, a weight index i representing one of the five types needs to be signaled in the block to be decoded.

[0038] At this time, in order to reduce the amount of coding of the weight index, the second intra-frame predictor 205 changes at least one of the number and type of weights according to the priorities of BV1 and BV2.

[0039] Specifically, the second intra-frame predictor 205 can set such priorities depending on the likelihood of errors occurring between the reference pixels P1 and P2.

[0040] For example, since the reference pixel P1 in the IBC BVP mode has a higher prediction accuracy than the reference pixel P2 in the IBC merge mode, the second intra frame predictor 205 applies the larger of wi and (1-wi) to the reference pixel P1 with the higher prediction accuracy.

[0041] In other words, the second intra-frame prediction unit 205 can reduce the amount of coding for the weight index by limiting the number of weights so that only the larger weights (wi=4 / 8, 5 / 8, 10 / 8) among the selectable weights are applied to the high priority reference pixel P1.

[0042] Since three types of weight index i are signaled, it is expected that the coding efficiency will be improved.

[0043] When both BVs are in IBC BVP mode or IBC merge mode, the second intra-frame prediction unit 205 may decode using five types of weight indexes without limiting the weights, or may set the five types of weights to a different combination (e.g., wi=4 / 8, 2 / 8, 6 / 8, 9 / 8, -1 / 8), or may increase or decrease the number of weight types (e.g., wi=4 / 8, 1 / 8, 2 / 8, 3 / 8, 5 / 8, 6 / 8, 7 / 8).

[0044] Alternatively, the second intra-frame predictor 205 may decode the weight index in accordance with a priority order, which will be described later.

[0045] During encoding, since the IBC merge mode does not have a BVD, the reference points are limited. Therefore, it is desirable to first determine the BV in the IBC merge mode, derive the weighted difference between the pixel P to be encoded and the reference pixel P2, and then determine the BV and weights for the IBC BVP mode that minimize the error with this difference.

[0046] As a modification, the second intra-frame predictor 205 may set the priority of BV1 and BV2 to the length of BV.

[0047] For example, considering that local correlation in images is generally high, it is expected that a short BV will have higher prediction accuracy than a long BV, so the second intra-frame prediction unit 205 may be configured to set the priority inversely proportional to the length of the BV and apply the larger of wi and (1-wi) to the reference pixels of the short BV.

[0048] As another modification, the second intra-frame predictor 205 can set the priority of BV1 and BV2 to the length of the BVD.

[0049] For example, considering that a long BVD is selected even when the coding amount of the BVD is taken into account in rate-distortion optimization (RDO), it is expected that a long BVD will have higher prediction accuracy than a short BVD, so the second intra-frame prediction unit 205 may be configured to make the priority proportional to the length of the BVD and apply the larger of wi and (1-wi) to the reference pixels of the long BVD.

[0050] As another modification, the second intra-frame predictor 205 may set the priorities of BV1 and BV2 to the coordinates of BV.

[0051] For example, in text areas, etc., the same character is likely to exist directly above or directly to the side of the block in vertical and horizontal writing, so BVs near the x-axis and y-axis are expected to have high prediction accuracy. Therefore, the second intra-frame prediction unit 205 may be configured to set the priority inversely proportional to the distance from the axis, and apply the larger of wi and (1-wi) to the reference pixels of BVs close to y=0 or x=0.

[0052] Hereinafter, the control information decoded by the decoding unit 201 for the second intra-frame prediction will be described.

[0053] The input code information may include a sequence parameter set (SPS) that summarizes control information for each sequence.

[0054] The input coded information may also include a picture parameter set (PPS) or a picture header (PH) that summarizes control information for each picture.

[0055] Furthermore, the input coded information may include a slice header (SH) that summarizes control information for each slice.

[0056] A method for setting the correction method on a sequence-by-sequence basis will be described below. Figure 3 shows a flowchart illustrating an example of such a method.

[0057] As shown in FIG. 3, in step S101, the decoding unit 201 determines whether or not sps_biibc_enabled_flag is 1 in the sequence parameter set.

[0058] Here, sps_biibc_enabled_flag is a syntax element that controls the presence or absence of bi-predictive IBC. When sps_biibc_enabled_flag is 1, this indicates that the bi-predictive IBC is enabled, and when sps_biibc_enabled_flag is 0, this indicates that the bi-predictive IBC is disabled.

[0059] If sps_biibc_enabled_flag is 1, the operation proceeds to step S102, and if sps_biibc_enabled_flag is 0, the operation ends.

[0060] In step S202, the decoding unit 201 decodes sps_bcwibc_mode.

[0061] Here, sps_bcwibc_mode is a syntax element that controls the bi-prediction IBC method.

[0062] By using sps_bcwibc_mode, the bi-predictive IBC method can be changed according to image characteristics on a sequence-by-sequence basis, which is expected to maximize coding efficiency.

[0063] For example, for sequences composed of CG, the pixel distribution is often composed of the same values, so the number of bi-predictive IBC weight combinations can be set to be small, and for sequences composed of natural images, the pixel distribution is diverse, so the number of bi-predictive IBC weight combinations can be set to be large, thereby maximizing coding efficiency.

[0064] When the correction method is set on a picture-by-picture basis, the decoding unit 201 similarly decodes pps_bcwibc_enabled_flag and pps_bcwibc_mode in the picture parameter set or the picture header.

[0065] By using pps_bcwibc_mode, the correction method can be changed on a picture-by-picture basis according to the image characteristics, which is expected to maximize coding efficiency.

[0066] For example, for pictures composed of CG, the pixel distribution is often composed of the same values, so the number of bi-predictive IBC weight combinations can be set to be small, and for pictures composed of natural images, the pixel distribution is diverse, so the number of bi-predictive IBC weight combinations can be set to be large, thereby maximizing coding efficiency.

[0067] When the correction method is set on a slice-by-slice basis, the decoding unit 201 similarly decodes sh_bcwibc_enabled_flag and sh_bcwibc_mode in the slice header.

[0068] By using sh_bcwibc_mode, the correction method can be changed according to the image characteristics on a slice-by-slice basis, which is expected to maximize coding efficiency.

[0069] For example, for slice regions composed of CG, the pixel distribution is often composed of the same values, so the number of bi-predictive IBC weight combinations can be set to be small, and for slice regions composed of natural images, the pixel distribution is diverse, so the number of bi-predictive IBC weight combinations can be set to be large, thereby maximizing coding efficiency.

[0070] By setting only the upper layers, it is possible to suppress an increase in the amount of code, and by setting the lower layers as well and then prioritizing the settings in the lower layers, adaptive control is possible.

[0071] Alternatively, if the correction method is set in advance, decoding of the correction method itself can be omitted.

[0072] In the above example, the method of setting the bi-predictive IBC method on a sequence basis, a picture basis, or a slice basis has been described, but the method may be directly selected on a block basis (described later) without setting these. In this case, the increase in header information described above can be avoided.

[0073] A method for applying two BVs on a block-by-block basis will be described below. Figure 4 shows a flowchart of an example of such a method.

[0074] As shown in FIG. 4, in step S201, the decoding unit 201 determines whether any one of sps_biibc_enabled_flag, pps_biibc_enabled_flag, and sh_biibc_enabled_flag is 1 or not.

[0075] If neither is 1, the operation ends, and if either is 1, the operation proceeds to step S202.

[0076] In step S202, the decoding unit 201 determines whether the current block to be decoded is two BVs.

[0077] If Yes, the operation proceeds to step S203; if No, the operation ends.

[0078] In step S203, the decoding unit 201 decodes cu_bcwibc_mode, which is a control signal indicating weights for the reference pixels of the two BVs.

[0079] In step S204, the decoding unit 201 decodes cu_bcwibc_index, which is a control signal representing the weight.

[0080] According to this embodiment, by setting weights according to priorities in decoding of BVs in bi-predictive IBC, decoding is performed using a relatively small amount of code, thereby making it possible to improve coding efficiency.

[0081] The above-described image decoding device 200 may be realized as a program that causes a computer to execute each function (each process). [Industrial Applicability]

[0082] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, which is to "build resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]

[0083] 200...Image decoding device 201…Decoding section 202...Inverse quantization section 203…Inverse conversion unit 204...First frame intra prediction unit 205...Second frame intra prediction unit 206...Interframe prediction unit 207…Adder 208…Storage section 210...Code input section 220...Image output unit

Claims

1. An image decoding device, comprising: a decoding unit for decoding the control information and the quantization value; an inverse quantization unit that inversely quantizes the quantized values ​​to generate transform coefficients; an inverse transform unit that inversely transforms the transform coefficients to generate prediction residuals; a first intra-frame prediction unit that generates a first predicted pixel based on a decoded pixel and the control information; a second intra-frame prediction unit that generates a second predicted pixel by performing a weighted average on the decoded pixels; a storage unit that stores the decoded pixels; an inter-frame prediction unit that generates a third predicted pixel based on the accumulated decoded pixels and the control information; an adder that adds the prediction residual and the first to third predicted pixels to obtain the decoded pixel.

2. 2. The image decoding device according to claim 1, wherein the second intra-frame prediction section limits the application of the weighted average to blocks of a specific size.

3. The image decoding device according to claim 1 , characterized in that the second intra-frame prediction unit generates the second predicted pixel by performing a weighted average on a region of reference pixels that are respectively referenced by two or more block vectors.

4. The image decoding device according to claim 1 , characterized in that the second intra-frame prediction unit changes at least one of the number and type of weights for reference pixels referenced by each of the multiple block vectors according to priority.

5. The image decoding device according to claim 4 , wherein the second intra-frame prediction section sets the priority based on a prediction accuracy of the reference pixels.

6. The image decoding device of claim 4, characterized in that the second intra-frame prediction unit limits the weights so that only weights that are greater than the weights of reference pixels with low priority are applied to reference pixels with high priority compared to reference pixels with low priority among the reference pixels referenced by each of the multiple block vectors, among the selectable weights.

7. 5. The image decoding device according to claim 4, wherein the second intra-frame prediction section does not limit the weights when there is no difference in the priorities.

8. 5. The image decoding device according to claim 4, wherein the second intra-frame predictor sets the priority to a length of the block vector.

9. 5. The image decoding device according to claim 4, wherein the second intra-frame predictor sets the priority in inverse proportion to a length of the block vector.

10. The image decoding device according to claim 4 , wherein the second intra-frame predictor sets the priority to a length of a block vector difference.

11. 5. The image decoding device according to claim 4, wherein the second intra-frame predictor sets the priority in proportion to a length of a block vector difference D.

12. The image decoding device according to claim 4 , wherein the second intra-frame predictor sets the priority to coordinates of the block vector.

13. 5. The image decoding device according to claim 4, wherein the second intra-frame prediction section sets the priority in inverse proportion to a distance between the block vector and a coordinate axis.

14. 1. An image decoding method, comprising: decoding the control information and the quantized values; dequantizing the quantized values ​​to obtain transform coefficients; inverse transforming the transform coefficients to obtain a prediction residual; generating a first predicted pixel based on the decoded pixel and the control information; generating a second predicted pixel by performing a weighted average on the decoded pixels; storing the decoded pixels; generating a third predicted pixel based on the accumulated decoded pixels and the control information; and adding the prediction residual to the first to third predicted pixels to obtain the decoded pixel.

15. A program for causing a computer to function as an image decoding device, The image decoding device comprises: a decoding unit for decoding the control information and the quantization value; an inverse quantization unit that inversely quantizes the quantized values ​​to generate transform coefficients; an inverse transform unit that inversely transforms the transform coefficients to generate prediction residuals; a first intra-frame prediction unit that generates a first predicted pixel based on a decoded pixel and the control information; a second intra-frame prediction unit that generates a second predicted pixel by performing a weighted average on the decoded pixels; a storage unit that stores the decoded pixels; an inter-frame prediction unit that generates a third predicted pixel based on the accumulated decoded pixels and the control information; an adder that adds the prediction residual and the first to third predicted pixels to obtain the decoded pixel.

Images