Image decoding device, image decoding method, and program

Abstract

To increase coding efficiency.SOLUTION: An image decoding device 200 according to an invention comprises a composing part 280 which is configured to generate a prediction pixel based on prediction information and output the prediction pixel. The composing part 280 is configured to assign different intra-prediction to each of a plurality of subregions obtained by dividing a unit block of a block to be decoded.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image decoding device, an image decoding method, and a program. [Background technology]

[0002] Non-Patent Documents 1 and 2 disclose GPM (Geometric Partitioning Mode).

[0003] GPM divides a rectangular block diagonally into two parts and performs motion compensation on each part. Specifically, the two divided parts are motion compensated using a merge vector and then combined by weighted averaging. There are 64 patterns of diagonal divisions available, depending on the angle and position. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] ITU-T H.266 VVC [Non-patent document 2] CE4:Summary report on inter prediction with geometric partioing, JVET-Q0024 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in Non-Patent Document 1 and Non-Patent Document 2, the target is limited to a merge mode involving motion compensation, and therefore there is a problem that there is room for improvement in improving coding performance.

[0006] Therefore, the present invention has been made in consideration of the above-mentioned problems, and an object of the present invention is to provide an image decoding device, an image decoding method, and a program that can further improve coding efficiency. [Means for solving the problem]

[0007] A first feature of the present invention is an image decoding device comprising a synthesis unit configured to generate and output predicted pixels based on prediction information, and the synthesis unit is configured to assign different intra predictions to each of a plurality of small regions obtained by dividing a unit block of a block to be decoded.

[0008] A second feature of the present invention is an image decoding method comprising a step of generating and outputting predicted pixels based on prediction information, wherein in the step, different intra predictions are assigned to each of a plurality of small regions obtained by dividing a unit block of a block to be decoded.

[0009] A third feature of the present invention is a program that causes a computer to function as an image decoding device, the image decoding device comprising a synthesis unit configured to generate and output predicted pixels based on prediction information, and the synthesis unit configured to assign different intra predictions to each of a plurality of small regions obtained by dividing a unit block of a block to be decoded. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program that can further improve coding efficiency. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 2 is a diagram illustrating an example of functional blocks of an image decoding device 200 according to an embodiment. [Figure 2] 10 is a diagram showing an example of a case in which a unit block is divided into a small region A and a small region B in an image decoding device 200 according to an embodiment. FIG. [Figure 3] FIG. 10 is a diagram showing an example of a case where a unit block is divided into small regions A to C in an image decoding device 200 according to an embodiment. [Figure 4]FIG. 3 is a diagram showing an example of a case where intra prediction and motion compensation are assigned to small regions A and B in FIG. 2, respectively, to generate predicted pixels, in an image decoding device 200 according to an embodiment. [Figure 5] FIG. 4 is a diagram showing an example of a case where intra prediction, motion compensation, and intra prediction are assigned to each of the small regions A to C in FIG. 3 to generate predicted pixels in the image decoding device 200 according to an embodiment. [Figure 6] FIG. 6 is a diagram showing an example of a calculation method using weighting coefficients and weighted averaging processing. [Figure 7] FIG. 7 is a diagram showing an example of a calculation method using weighting coefficients and weighted averaging processing. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0013] (First embodiment) An image decoding device 200 according to a first embodiment of the present invention will be described below with reference to Figures 1 to 7. Figure 1 is a diagram showing an example of functional blocks of the image decoding device 200 according to this embodiment.

[0014] As shown in FIG. 1, the image decoding device 200 includes a code input unit 201, a decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an adder 240, a storage unit 250, an intra prediction unit 260, a motion compensation unit 270, a synthesis unit 280, and an image output unit 290.

[0015] The code input unit 201 is configured to acquire and output code information output by an image coding device.

[0016] The decoding unit 210 is configured to generate and output quantized values ​​and prediction information by performing variable length decoding processing on the code information output by the code input unit 201. Here, as shown in Fig. 1 , the decoding unit 210 is configured to output the quantized values ​​to the inverse quantization unit 220, and output the prediction information to the intra prediction unit 260, the motion compensation unit 270, and the synthesis unit 280.

[0017] The inverse quantization unit 220 is configured to generate and output transform coefficients by performing inverse quantization processing on the quantized values ​​output by the decoding unit 210. Here, as shown in FIG. 1 , the inverse quantization unit 220 is configured to output the transform coefficients to the inverse transform unit 230.

[0018] The inverse transform unit 230 is configured to generate and output a prediction residual by performing an inverse transform process on the transform coefficients output by the inverse quantization unit 220. Here, as shown in FIG. 1 , the inverse transform unit 230 is configured to output the prediction residual to the adder 240.

[0019] The adder 240 is configured to obtain the prediction residual output by the inverse transform unit 230 and the prediction pixel output by the synthesis unit 280. Here, the adder 240 is configured to add the prediction residual and the prediction pixel to obtain and output a decoded pixel. Here, as shown in FIG. 1 , the adder 240 is configured to output the decoded pixel to the accumulation unit 250, the intra prediction unit 260, and the image output unit 290.

[0020] The storage unit 250 is configured to cumulatively store the decoded pixels output by the adder 240. The storage unit 250 is configured to output the decoded pixels in response to a request from the motion compensation unit 270.

[0021] The intra prediction unit 260 is configured to generate predicted pixels as approximations of input pixels in a small region set by a synthesis unit 280 (described later) based on the decoded pixels output by the adder 240 and the prediction information output by the decoding unit 210. Here, as shown in FIG. 1 , the intra prediction unit 260 is configured to output predicted pixels to the synthesis unit 280.

[0022] The motion compensation unit 270 is configured to generate predicted pixels as approximations of input pixels in a small region set by a synthesis unit 280 (described later) based on decoded pixels obtained by referring to the storage unit 250 and prediction information output by the decoding unit 210. Here, as shown in FIG. 1 , the motion compensation unit 270 is configured to output predicted pixels to the synthesis unit 280.

[0023] The image output unit 290 is configured to output the decoded pixels output by the adder 240 .

[0024] An example of the function of the synthesis unit 280 will be described below with reference to FIGS.

[0025] The synthesis unit 280 is configured to generate and output predicted pixels based on the prediction information output by the decoding unit 210, the predicted pixels output by the intra prediction unit 260, and the predicted pixels output by the motion compensation unit 270.

[0026] The synthesis unit 280 is also configured to generate the predicted pixels by performing a weighted averaging process using weighting coefficients on the predicted pixels output by the intra prediction unit 260 and the predicted pixels output by the motion compensation unit 270.

[0027] Here, the role of the synthesis unit 280 is to select weighting factors for a plurality of prediction pixels that are optimal for the block to be decoded, and to synthesize the plurality of prediction pixels obtained from the intra prediction unit 260 and the motion compensation unit 270 according to the weighting factors, in order to compensate the block to be decoded with high precision in the subsequent adder 240.

[0028] It should be noted that, for the weighting coefficients, multiple patterns are prepared in which arbitrary values ​​are set in advance for each pixel of the unit block, and the synthesis unit 280 is configured to identify the weighting coefficient to be used for each pixel from the multiple weighting coefficient patterns in accordance with the prediction information obtained from the decoding unit 210.

[0029] The sum of the weighting factors for multiple predicted pixels is designed to be 1 for each pixel, and the combined result obtained by performing weighted averaging on multiple predicted pixels using these weighting factors is used as the predicted pixel by the combining unit 280.

[0030] For pixels with a weighting factor other than 0, the predicted pixel is adopted, and for pixels with a weighting factor of 0, the predicted pixel is not used. Therefore, conceptually, this is equivalent to dividing a unit block into multiple small regions, and determining which of the multiple predicted pixels to apply to where and in what proportion.

[0031] Here, it is desirable to distribute the weighting coefficients in a non-rectangular shape, since a rectangular distribution, such as a bisecting distribution, can be expressed in smaller unit blocks.

[0032] FIG. 2 shows an example in which unit blocks are distributed in a diagonal shape, and the unit blocks are divided into small regions A and B by diagonal straight lines.

[0033] In the example of Fig. 3, the unit block is divided by a plurality of straight lines into small areas A, B, and C. The number of divisions may be any number.

[0034] 3, predicted pixels may be generated by assigning intra prediction and / or motion compensation to each of the small regions A to C. That is, the synthesis unit 280 may be configured to use predicted pixels generated by the intra prediction unit 260 and / or predicted pixels generated by the motion compensation unit 270 in each of the plurality of small regions A to C obtained by dividing the unit blocks of the block to be decoded.

[0035] In this case, if the number of divisions increases, it becomes necessary to encode prediction information such as intra-prediction, which leads to a problem that the coding efficiency cannot be improved. To solve this problem, a procedure is taken in which a prediction mode corresponding to the division of small regions is used.

[0036] 4 shows an example in which intra prediction and motion compensation are assigned to small region A and small region B in FIG. 2, respectively, to generate predicted pixels. Any combination can be used, such as assigning different intra predictions to both small region A and small region B. Furthermore, with regard to intra prediction, either pre-deblocking filter application or post-deblocking filter application may be used. It is desirable to determine which method to use in advance.

[0037] For intra prediction, the intra prediction mode may be limited depending on the division shape of the small region. That is, when intra prediction is assigned to a small region, the synthesis unit 280 may be configured to use the intra prediction mode depending on the division shape of the small region.

[0038] For example, as shown in FIG. 4, the synthesis unit 280 can limit the intra prediction modes to only one type of mode in a direction parallel to the direction of division by the straight lines that form the division shape.

[0039] Alternatively, the synthesizing unit 280 may limit the intra prediction modes to only two types: a mode in a direction parallel to the direction of division by the straight lines that form the division shape, and a mode in a direction perpendicular to the direction.

[0040] In this way, by limiting the intra prediction mode to one that corresponds to the direction of the division shape, it is possible to obtain the effect of adaptively reducing the amount of code representing the intra prediction mode.

[0041] The decoding unit 210 may be configured to adaptively decode prediction modes limited according to the type of division of the small regions.

[0042] For example, when limiting the intra prediction modes to directions parallel and perpendicular to the direction of division by the straight lines that form the division shape, the decoding unit 210 may be configured to decode the intra prediction mode as the parallel direction if the code input to the decoding unit 210 is "0," and to decode the intra prediction mode as the perpendicular direction if the code input to the decoding unit 210 is "1." With this configuration, prediction information for the intra prediction mode can be expressed in 1 bit, thereby achieving the effect of significantly improving coding efficiency.

[0043] The synthesis unit 280 may be configured to include a specific intra prediction mode as a prediction mode option. For example, the synthesis unit 280 may be configured to always add a specific intra prediction mode that does not depend on direction, such as a planar mode or a DC mode, to the options.

[0044] For example, the synthesis unit 280 may limit the intra prediction modes to only three types: a mode in a direction parallel to the direction of division by the straight lines that make up the division shape, a mode in a direction perpendicular to the direction of division, and a planar mode that is independent of the direction.

[0045] Alternatively, the synthesis unit 280 may limit the intra prediction modes to only four types: a mode parallel to the direction of division by the straight lines that make up the division shape, a mode perpendicular to the direction of division, a planar mode that is independent of the direction, and a DC mode.

[0046] In addition, when an MRL is enabled that selects reference pixels to be used for generating predicted pixels from multiple reference pixel lines adjacent to the block to be decoded, as disclosed in Non-Patent Document 1, the synthesis unit 280 may use the reference pixels selected by the MRL as reference pixels to be referenced in the intra prediction mode according to the above-mentioned division shape.

[0047] In FIG. 5, intra prediction, motion compensation, and intra prediction are assigned to small regions A to C in FIG. 3, respectively.

[0048] In each intra prediction, options may include all or some of the intra prediction modes corresponding to the direction of division by the multiple straight lines that make up the small regions A to C. In other words, the synthesis unit 280 may be configured to include, as options, at least some of the intra prediction modes corresponding to the direction of division by the straight lines that make up the division shape of the small regions.

[0049] The synthesizing unit 280 may determine the number of intra prediction modes to be included in the options based on the size of the block to be decoded.

[0050] For example, when the block size of the unit block is small, the synthesis unit 280 may generate predicted pixels using one intra prediction mode according to the direction in which the small area is divided by the straight lines that form the division shape of the small area and predicted pixels generated by multiple intra prediction modes near that intra prediction mode.

[0051] Furthermore, when the block size of a unit block is large, the synthesis unit 280 may generate predicted pixels using only one intra prediction mode according to the direction of division by the straight lines that form the division shape of the small region.

[0052] Since the position of each reference pixel (the distance between each reference pixel) between adjacent intra-prediction modes changes depending on the size of the unit block, in large-sized blocks where the positions of each reference pixel are likely to be far apart as described above, by limiting the division direction to, for example, one parallel intra-prediction mode, it is possible to avoid the possibility of degradation of prediction performance occurring when pixels referenced by prediction modes near such parallel intra-prediction mode, i.e., reference pixels far from the division lines that make up the division shape, are used to generate predicted pixels.

[0053] On the other hand, in small-sized blocks where the reference pixel positions described above are unlikely to be separated, improved prediction performance can be expected by using, for example, pixels referenced by one parallel intra-prediction mode and a prediction mode close to this parallel intra-prediction mode in the division direction to generate predicted pixels.

[0054] The synthesizing unit 280 may determine the number of intra prediction modes to be included in the options based on the aspect ratio of the block to be decoded.

[0055] For example, when the aspect ratio of a unit block is small, the synthesis unit 280 may generate predicted pixels using one intra prediction mode according to the direction in which the small area is divided by the straight lines that form the division shape of the small area, and predicted pixels generated by multiple intra prediction modes near that intra prediction mode.

[0056] Furthermore, when the aspect ratio of a unit block is large, the synthesizing unit 280 may generate predicted pixels using only one intra prediction mode according to the direction of division by the straight lines that form the division shape of the small region.

[0057] Since the aspect ratio of the unit block changes the positions of each reference pixel (the distance between each reference pixel) between adjacent intra-prediction modes, as described above, in non-square blocks where the positions of each reference pixel are likely to be far apart, by limiting the division direction to, for example, one parallel intra-prediction mode, it is possible to avoid the possibility of degradation of prediction performance occurring when pixels referenced by prediction modes near such parallel intra-prediction mode, i.e., reference pixels far from the division lines that form the division shape, are used to generate predicted pixels.

[0058] On the other hand, in square blocks where the positions of the reference pixels mentioned above are unlikely to be separated, improved prediction performance can be expected by using, for example, pixels referenced by one parallel intra-prediction mode and a prediction mode close to this parallel intra-prediction mode in the division direction to generate predicted pixels.

[0059] The predicted pixels of each small area A to C are calculated by performing a weighted averaging process using a weighting factor according to the division shape. The small areas A to C are combined by setting a weighting factor for each pixel and performing a weighted averaging process using the weighting factor on multiple predicted pixels.

[0060] FIG. 6 shows an example of a calculation method using weighting factors and weighted averaging.

[0061] Furthermore, when intra prediction is combined with motion compensation, the prediction accuracy of intra prediction is relatively low in the lower right corner, so in addition to the weighting factors in Figure 6, as shown in Figure 7, the weighting factors for intra prediction can be designed taking into account the fact that they become smaller toward the lower right corner of the unit block. That is, the synthesis unit 280 may be configured to decrease the weighting factors toward the lower right corner of the unit block of the current block to be decoded. In this case, explicit small region division can be omitted by uniformly setting the weighting factor w to 0.5.

[0062] Furthermore, in regions where intra prediction is generally selected, intra prediction is often selected for neighboring unit blocks as well, so the weighting coefficients may be adaptively changed depending on whether the neighboring unit blocks are intra predicted or not. That is, the synthesis unit 280 may be configured to adaptively change the weighting coefficients of neighboring unit blocks of the unit block to be decoded, based on whether the neighboring unit blocks are intra predicted or not.

[0063] For example, the weighting factor may be proportional to the number of intra-predictions of neighboring unit blocks. That is, the synthesis unit 280 may be configured to determine the weighting factor so as to be proportional to the number of unit blocks to which intra-predictions are assigned among unit blocks neighboring the unit block of the block to be decoded. When there are many intra-predictions in the neighborhood, the weight r is increased to strengthen the influence of the intra-predictions, and when there are few intra-predictions in the neighborhood, the weight r is decreased to reduce the influence of the intra-predictions, which is expected to improve prediction accuracy.

[0064] Furthermore, the above-described image decoding device 200 may be realized as a program that causes a computer to execute each function (each step). [Industrial Applicability]

[0065] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which will contribute to the achievement of Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]

[0066] 200...Image decoding device 201...Code input section 210...Decoding unit 220...Inverse quantization section 230...Inverse conversion unit 240...adder 250...Storage section 260...Intra prediction unit 270...Motion compensation unit 280...Synthesis section 290...Image output unit

Claims

1. An image decoding device, a synthesis unit configured to generate and output predicted pixels based on prediction information; the synthesis unit is configured to assign different intra predictions to each of a plurality of small regions obtained by dividing a unit block of a block to be decoded, The unit block is divided into the small regions by diagonal straight lines, The image decoding device, wherein the synthesis unit is configured to use a prediction mode according to the division shape of the small region.

2. The image decoding device described in Claim 1, characterized in that the synthesis unit is configured to use the prediction mode according to the direction of the straight lines that make up the division shape.

3. An image decoding method, comprising: generating and outputting predicted pixels based on the prediction information; In the step, different intra predictions are assigned to a plurality of small regions obtained by dividing a unit block of a block to be decoded, The unit block is divided into the small regions by diagonal straight lines, An image decoding method, wherein in the step, a prediction mode according to the division shape of the small region is used.

4. A program that causes a computer to function as an image decoding device, the image decoding device includes a synthesis unit configured to generate and output predicted pixels based on prediction information; the synthesis unit is configured to assign different intra predictions to each of a plurality of small regions obtained by dividing a unit block of a block to be decoded, The unit block is divided into the small regions by diagonal straight lines, The program is characterized in that the synthesis unit is configured to use a prediction mode according to the division shape of the small region.

Images