Image decoding device, image decoding method, and program

Abstract

To improve the decoding efficiency in a GPM.SOLUTION: An image decoding device 200 according to the present invention is provided with: a decoding unit 201 that decodes control information and a quantization value; an inverse quantization unit 202 that performs inverse quantization on the quantization value into a conversion factor; an inverse transformation unit 203 that performs inverse transformation on the conversion factor into a predictive residual; an intra-prediction unit 204 that generates first prediction pixels based on decoded pixels and the control information; an accumulation unit 205 that accumulates the decoded pixels; a motion compensation unit 206 that generates second prediction pixels based on the decoded pixels and the control information; a composition unit 207 that prepares a plurality of weighting factors different in a width of a division boundary and applies weighted averaging to at least one of the first prediction pixels and the second prediction pixels to generate third prediction pixels having the controlled width of the division boundary; and an addition unit 208 that adds up the predictive residual and the third prediction pixels to obtain the decoded pixels.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image decoding device, an image decoding method, and a program. [Background technology]

[0002] Non-Patent Documents 1 and 2 disclose a geometric partitioning mode (GPM).

[0003] GPM divides a rectangular block diagonally into two parts and motion compensates each part. Specifically, the two divided parts are motion compensated using merge vectors and then combined using weighted averaging. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] ITU-T H.266 / VVC [Non-patent document 2] CE4:Summary report on Inter prediction with geometric partitioning, JVET-Q0024 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the techniques disclosed in Non-Patent Document 1 and Non-Patent Document 2 have a problem in that the weighted average patterns are limited, and there is room for improvement in encoding performance. Therefore, the present invention has been made in consideration of the above-mentioned problem, and an object of the present invention is to provide an image decoding device, an image decoding method, and a program that can improve encoding efficiency in GPM. [Means for solving the problem]

[0006] a motion compensation unit that generates a second predicted pixel based on the stored decoded pixels and the decoded control information; a synthesis unit that prepares a plurality of weighting factors with different widths of a division boundary for at least one of the first predicted pixel and the second predicted pixel and generates a third predicted pixel by controlling the width of the division boundary by a weighted average; and an adder that adds the decoded prediction residual and the third predicted pixel to obtain the decoded pixel.

[0007] a step of generating a second predicted pixel based on the decoded pixels and the decoded control information; a step of preparing a plurality of weighting factors having different widths of a division boundary for at least one of the first predicted pixel and the second predicted pixel, and a step of controlling the width of the division boundary by a weighted average; and a step of adding the decoded prediction residual and the third predicted pixel to obtain the decoded pixel.

[0008] a storage unit that stores the decoded pixels; a motion compensation unit that generates a second predicted pixel based on the stored decoded pixels and the decoded control information; a synthesis unit that prepares a plurality of weighting factors with different widths of a division boundary for at least one of the first predicted pixel and the second predicted pixel and generates a third predicted pixel by controlling the width of the division boundary by a weighted average; and an addition unit that adds the decoded prediction residual and the third predicted pixel to obtain the decoded pixel. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program that can improve the coding efficiency in GPM. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of functional blocks of an image decoding device 200 according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of a case where a rectangular unit block is divided into two small regions A and B by a dividing boundary. [Figure 3] FIG. 3 is a diagram showing an example of three patterns of weighting coefficients assigned to the division boundaries of the small region B shown in FIG. [Figure 4] FIG. 4 is a diagram showing an example in which the weighting factor w of pattern (2) is applied to an 8×8 block. [Figure 5] FIG. 5 is a diagram showing an example in which the weighting factor w of pattern (1) is applied to an 8×8 block. [Figure 6]FIG. 6 is a diagram showing an example in which the weighting factor w of pattern (3) is applied to an 8×8 block. [Figure 7] FIG. 7 is a flowchart illustrating an example of a process for setting weighting factors by the synthesis unit 207 in the first embodiment. [Figure 8] FIG. 8 is a flowchart illustrating an example of a process for setting a weighting factor by the synthesis unit 207 in the second embodiment. [Figure 9] FIG. 9 is a diagram for explaining the second embodiment. [Figure 10] FIG. 10 is a diagram for explaining the second embodiment. [Figure 11] FIG. 11 is a flowchart illustrating an example of a process for setting a weighting factor by the synthesis unit 207 in the third embodiment. [Figure 12] FIG. 12 is a diagram for explaining an example in which the weighting coefficient is defined based on the distance from the division boundary. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0012] First Embodiment An image decoding device 200 according to this embodiment will be described below with reference to Figures 1 to 7. Figure 1 is a diagram showing an example of functional blocks of the image decoding device 200 according to this embodiment.

[0013] As shown in FIG. 1, the image decoding device 200 includes a code input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transform unit 203, an intra prediction unit 204, an accumulation unit 205, a motion compensation unit 206, a synthesis unit 207, an addition unit 208, and an image output unit 220.

[0014] The code input unit 210 is configured to obtain code information coded by an image coding device.

[0015] The decoding unit 201 is configured to decode the control information and the quantized value from the code information input from the code input unit 210. For example, the decoding unit 201 is configured to output the control information and the quantized value by performing variable length decoding on the code information.

[0016] Here, the quantized value is sent to the inverse quantization unit 202, and the control information is sent to the motion compensation unit 206, the intra prediction unit 204, and the synthesis unit 207. Note that this control information includes information necessary for controlling the motion compensation unit 206, the intra prediction unit 204, the synthesis unit 207, etc., and may also include header information such as a sequence parameter set, a picture parameter set, a picture header, or a slice header.

[0017] The inverse quantization unit 202 is configured to inverse quantize the quantized values ​​sent from the decoding unit 201 to generate decoded transform coefficients. The transform coefficients are sent to the inverse transform unit 203.

[0018] The inverse transform unit 203 is configured to inverse transform the transform coefficients sent from the inverse quantization unit 202 to generate decoded prediction residuals. The prediction residuals are sent to the adder 208.

[0019] The intra prediction unit 204 is configured to generate a first predicted pixel based on the decoded pixel and the control information sent from the decoding unit 201. Here, the decoded pixel is obtained via the adder 208 and stored in the storage unit 205. The first predicted pixel is a predicted pixel as an approximation of an input pixel in a small region set by the synthesis unit 207. The first predicted pixel is sent to the synthesis unit 207.

[0020] The storage unit 205 is configured to cumulatively store the decoded pixels sent from the addition unit 208. The motion compensation unit 206 references these decoded pixels via the storage unit 205.

[0021] The motion compensation unit 206 is configured to generate second predicted pixels based on the decoded pixels stored in the storage unit 205 and the control information sent from the decoding unit 201. Here, the second predicted pixels are predicted pixels as approximations of input pixels in a small region set by the synthesis unit 207. The second predicted pixels are sent to the synthesis unit 207.

[0022] The adder 208 is configured to add the prediction residual sent from the inverse transformer 203 and the third predicted pixel sent from the synthesizer 207 to obtain a decoded pixel. The decoded pixel is sent to the image output unit 220, the storage unit 205, and the intra prediction unit 204.

[0023] The synthesis unit 207 is configured to prepare multiple weighting coefficients with different widths of the division boundary for at least one of the first predicted pixel sent from the intra prediction unit 204 and the second predicted pixel sent from the motion compensation unit 206, and to generate a third predicted pixel in which the width of the division boundary is controlled by weighted averaging.

[0024] The role of the synthesis unit 207 is to select weighting factors for a plurality of prediction pixels that are optimal for the block to be decoded, and to synthesize the input prediction pixels in accordance with the weighting factors, in order to compensate the block to be decoded with high precision in the subsequent addition unit 208.

[0025] Any partitioning mode can be used in which the block to be decoded is divided into multiple small regions. In the following, we will explain the case where the geometric block partitioning mode (GPM) disclosed in Non-Patent Document 1 and Non-Patent Document 2 is used as an example of a partitioning mode.

[0026] For the weighting coefficients, a plurality of patterns in which arbitrary values ​​are set in advance for each pixel of the unit block are prepared, and one of the patterns is applied. In other words, the synthesis unit 207 may be configured to select and apply one of the plurality of weighting coefficients.

[0027] According to this configuration, by preparing a lookup table or the like in which a plurality of weighting factors are set, the synthesis unit 207 does not need to calculate the weighting factor every time.

[0028] The sum of weighting factors for a plurality of predicted pixels is designed to be 1 for each pixel, and the result of combining a plurality of predicted pixels by weighted averaging using the weighting factors is used as the predicted pixel by the combining unit 207.

[0029] A pixel with a weighting factor of 1 (i.e., the maximum value) adopts the input predicted pixel, and a pixel with a weighting factor of 0 (i.e., the minimum value) does not use the input predicted pixel. Therefore, conceptually, this is equivalent to dividing a unit block into multiple small regions, and determining which pixel of multiple input predicted pixels to apply to where and in what proportion.

[0030] Here, it is desirable to distribute the weighting coefficients in a non-rectangular shape, since a rectangular distribution, such as a bisecting distribution, can be expressed in smaller unit blocks.

[0031] The example in Fig. 2 shows a case where unit blocks are distributed in a diagonal shape. In the example in Fig. 2, a rectangular unit block is divided into two small areas A and B by a dividing boundary.

[0032] In each of the small regions A / B, predicted pixels are generated by any method such as intra prediction or motion compensation.

[0033] In this case, even if the shape of the division is determined, if the weighting coefficients near the division boundaries are fixed, the diversity of the division boundaries cannot be expressed, and therefore there is a problem that the coding efficiency cannot be improved.

[0034] For example, if a small region is an area with a lot of movement, blurring occurs during imaging, so it is desirable to blur a plurality of small regions over a wide area and perform weighted averaging on the division boundary.

[0035] Conversely, if the small region is an artificially edited region such as a subtitle, blurring will not occur, so it is preferable to limit the division boundary to a narrow region and perform a weighted average so that multiple small regions are simply adjacent to each other.

[0036] To solve this problem, in this embodiment, a procedure is taken in which a plurality of weighting factors with different widths of the dividing boundaries of the small regions are prepared and selected.

[0037] Fig. 3 shows an example of three patterns of weighting coefficients assigned to the division boundary of small area B shown in Fig. 2. In Fig. 3, the horizontal axis represents the distance in pixels from the position of the division boundary, and the vertical axis represents the weighting coefficient.

[0038] Specifically, we have prepared pattern (1) in which a weighting factor of [0,1] is assigned to the range [a,b] for pixel-unit distances a and b from a preset division boundary position; pattern (2) in which distances a and b are similarly doubled and a weighting factor of [0,1] is assigned to the range [2a,2b]; and pattern (3) in which distances a and b are similarly halved and a weighting factor of [0,1] is assigned to the range [a / 2,b / 2]. These weighting coefficients are calculated based on the distance d(x c ,y c ) is uniquely determined by xc,ycWhen this is defined as follows, it is equivalent to preparing multiple patterns (variable values) instead of a limited pattern (fixed value) for the width of the division boundary of the small area in FIG. 12, that is, the width τ where the weighting coefficient is other than the minimum or maximum value. Here, x c ,y c are coordinates within the block to be decoded.

[0039] That is, the synthesis unit 207 may be configured to set a plurality of weighting factors according to the inter-pixel distance from the division boundary.

[0040] According to this configuration, the width of the boundary can be varied by being proportional to the distance from the dividing boundary, and changes from the conventional calculation formula shown in FIG. 12 can be minimized.

[0041] Note that a weighting factor may be set symmetrically with respect to the division boundary, i.e., a=b. That is, the synthesis unit 207 may be configured to set a weighting factor symmetrically with respect to the division boundary as the weighting factor. With this configuration, b becomes unnecessary, and therefore the amount of code can be reduced.

[0042] Alternatively, weighting coefficients may be set asymmetrically with respect to the division boundary, with a ≠ b. That is, the synthesis unit 207 may be configured to set weighting coefficients asymmetrically with respect to the division boundary as the weighting coefficients. With this configuration, highly accurate prediction is possible when there are different degrees of blur on both sides of the boundary.

[0043] Furthermore, the weighting coefficients can be set for multiple line segments, etc., by increasing the number of the weighting coefficients other than two, a and b. That is, the synthesis unit 207 may be configured to set weighting coefficients for multiple line segments according to the inter-pixel distance from the division boundary. With this configuration, highly accurate prediction is possible when nonlinear blurring occurs.

[0044] 4 to 6 show examples in which each weighting factor w is applied to an 8×8 block. The weighting factor w in FIGS. 4 to 6 takes values ​​from 0 to 8, and is synthesized using the following equation.

[0045] (w×small area A+(8-w)×small area B+4)>>3 In this way, by setting multiple weighting coefficients w according to the inter-pixel distance from the division boundary, it is possible to obtain the effect of uniformly deriving a weighted average for various block sizes, such as 8x8 or 64x16. The type, shape, and number of patterns can be set arbitrarily. For example, while the above description has described multiple patterns of 2 and 1 / 2 times the distances a and b, they may also be 4 or 1 / 4 times. Furthermore, in the above formula, the weighting coefficients are set to values ​​between 0 and 8, but they can also be set to other values, such as 0 to 16 or 0 to 32. In particular, when the inter-pixel distance from the division boundary is 2 or 4 times, the maximum value of the weighting coefficient can be increased to improve the accuracy of the weighted average for each pixel.

[0046] An example of the process of setting the weighting coefficients by the synthesis unit 207 will be described below with reference to FIG.

[0047] 7, in step S101, the synthesis unit 207 determines whether any of sps_div_enabled_flag, pps_div_enabled_flag, and sh_div_enabled_flag included in the above-mentioned control information is 1. If the result is No (if none of them is 1), the process proceeds to step S102, and if the result is Yes, the process proceeds to step S103.

[0048] In step S102, the synthesis unit 207 does not apply weighted averaging using a weighting factor to the current block to be decoded.

[0049] In step S103, the synthesis unit 207 determines whether GPM is applied to the block to be decoded. If the result is No, the process proceeds to step S102, and if the result is Yes, the process proceeds to step S104.

[0050] In step S104, the synthesis unit 207 decodes cu_div_blending_idx included in the above-mentioned control information.

[0051] If cu_div_blending_idx is 0, the operation proceeds to step S105; if cu_div_blending_idx is 1, the operation proceeds to step S106; if cu_div_blending_idx is 2, the operation proceeds to step S107.

[0052] In step S105, the synthesis unit 207 selects and applies the weighting factor of pattern (1) from among patterns (1) to (3).

[0053] In step S106, the synthesis unit 207 selects and applies the weighting factor of pattern (2) from among patterns (1) to (3).

[0054] In step S107, the synthesis unit 207 selects and applies the weighting factor of pattern (3) from among patterns (1) to (3).

[0055] Note that, when the chrominance pixel components of the block to be decoded are not downsampled with respect to the luminance component of the block to be decoded, the synthesis unit 207 may be configured to use the weighting coefficients derived for the luminance component of the block to be decoded, which determine the width of the division boundaries, as weighting coefficients for determining the width of the division boundaries of the chrominance components of the block to be decoded. With this configuration, it is possible to reduce the process of deriving weighting coefficients for the chrominance components of the block to be decoded.

[0056] Furthermore, when the chrominance components of the block to be decoded are not downsampled with respect to the luminance component of the block to be decoded, the synthesis unit 207 may derive weighting factors for determining the width of the division boundaries of the chrominance components of the block to be decoded, for example, in a manner similar to that described above, rather than using the weighting factors for determining the width of the division boundaries derived for the luminance component of the block to be decoded as they are, as weighting factors for determining the width of the division boundaries of the chrominance components of the block to be decoded. With this configuration, weighting factors for the chrominance components of the block to be decoded can be derived independently, which is expected to improve coding performance.

[0057] On the other hand, when the chrominance components of the block to be decoded are downsampled with respect to the luminance component of the block to be decoded, the synthesis unit 207 may take the downsampling method into consideration and derive a weighting coefficient that determines the width of the division boundary for the chrominance components of the block to be decoded from the width of the division boundary for the luminance component of the block to be decoded. With this configuration, the same effect as that obtained for the luminance component of the block to be decoded can be obtained for the downsampled chrominance components of the block to be decoded.

[0058] Furthermore, when the synthesis unit 207 uses control information such as a header to determine the width of the division boundary of the luminance component of the block to be decoded, this is not necessary for the chrominance component of the block to be decoded, which is expected to improve coding performance.

[0059] For example, when the chrominance components of the block to be decoded are downsampled to half in both the horizontal and vertical directions relative to the luminance component of the block to be decoded, the synthesis unit 207 may derive a weighting factor that determines the width of the partition boundary such that the width is half the width of the partition boundary derived for the luminance component of the block to be decoded, as the weighting factor that determines the width of the partition boundary for the chrominance components of the block to be decoded.

[0060] For example, when the chrominance components of the block to be decoded are downsampled to half the width in either the horizontal or vertical direction relative to the luminance component of the block to be decoded, the synthesis unit 207 may derive a weighting factor that determines the width of the partition boundary of the chrominance components of the block to be decoded, such that the width of the partition boundary is equal to or half the width of the partition boundary derived for the luminance component of the block to be decoded.

[0061] Second Embodiment Hereinafter, the second embodiment of the present invention will be described with reference to FIGS. 3 and 8 to 10, focusing on the differences from the first embodiment described above.

[0062] In this embodiment, the code length is reduced by specifying a weighting coefficient pattern without requiring direct control information.

[0063] Therefore, in this embodiment, the synthesis unit 207 is configured to generate a third predicted pixel by weighted averaging of at least one of the first predicted pixel and the second predicted pixel described above using a weighting factor that is uniquely selected from a plurality of weighting factors based on indirect control information.

[0064] That is, in this embodiment, the synthesis unit 207 is configured to select (uniquely identify) a weighting factor from among a plurality of weighting factors in accordance with indirect control information.

[0065] Here, the synthesis unit 207 may be configured to prepare a plurality of weighting factors with different widths of the division boundaries of the small regions, and to select a weighting factor from among the plurality of weighting factors.

[0066] Specifically, the synthesis unit 207 may be configured to select a weighting factor from among a plurality of weighting factors according to the shape of the block to be decoded, which serves as indirect control information.

[0067] For example, the synthesis unit 207 may be configured to select a weighting factor from among a plurality of weighting factors depending on at least one of the short side of the block to be decoded, the long side of the block to be decoded, the aspect ratio of the block to be decoded, the partitioning mode, and the number of pixels of the block to be decoded.

[0068] For example, when using the short side of the block to be decoded as the shape of the block to be decoded, if the short side of the block to be decoded is small, performing a weighted average over a wide area will result in the same result as simple bidirectional prediction, so it is desirable to exclude weight coefficients for patterns with wide division boundaries from the options.

[0069] For example, in the example of Figure 3, the synthesis unit 207 selects the weighting factor of pattern (3) when the short side of the block to be decoded is less than or equal to the threshold, and selects the weighting factor of pattern (2) when the short side of the block to be decoded is greater than the threshold, thereby increasing the number of patterns while eliminating the need for pattern control information and achieving the effect of improving coding efficiency.

[0070] An example of the weighting factor setting process performed by the synthesis unit 207 will be described below with reference to FIG.

[0071] 8, in step S201, the synthesis unit 207 determines whether any of sps_div_enabled_flag, pps_div_enabled_flag, and sh_div_enabled_flag included in the above-mentioned control information is 1. If the result is No (if none of them is 1), the process proceeds to step S202, and if the result is Yes, the process proceeds to step S203.

[0072] In step S202, the synthesis unit 207 does not apply weighted averaging using a weighting factor to the current block to be decoded.

[0073] In step S203, the synthesis unit 207 determines whether GPM has been applied to the block to be decoded. If the result is No, the process proceeds to step S202, and if the result is Yes, the process proceeds to step S204.

[0074] In step S204, the synthesis unit 207 determines whether the shorter side of the block to be decoded is equal to or less than a preset threshold value 1. If the result is No, the process proceeds to step S205, and if the result is Yes, the process proceeds to step S208.

[0075] In step S205, the synthesis unit 207 determines whether the shorter side of the block to be decoded is equal to or less than a preset threshold value 2. Here, threshold value 2 is greater than threshold value 1. If the result is No, the process proceeds to step S206, and if the result is Yes, the process proceeds to step S207.

[0076] In step S206, the synthesis unit 207 selects and applies the weighting factor of pattern (2) from among patterns (1) to (3).

[0077] In step S207, the synthesis unit 207 selects and applies the weighting factor of pattern (1) from among patterns (1) to (3).

[0078] In step S208, the synthesis unit 207 selects and applies the weighting factor of pattern (3) from among patterns (1) to (3).

[0079] Similarly, when the shape of the block to be decoded is determined by factors such as the long side of the block to be decoded, the aspect ratio of the block to be decoded, the partition mode, and the number of pixels of the block to be decoded, performing a weighted average over a wide area results in an effect that is no different from simple bi-prediction, and therefore it is desirable to exclude weight coefficients for patterns with wide partition boundaries from the options.

[0080] That is, in steps S204 and S205 of the flowchart shown in FIG. 8, the short side of the block to be decoded may be replaced with the long side of the block to be decoded, the aspect ratio of the block to be decoded, the partition mode, or the number of pixels of the block to be decoded.

[0081] Also, in the flowchart shown in Figure 8, in step S204, the synthesis unit 207 may determine whether the short side of the block to be decoded is less than a predetermined threshold value 1, and in step S205, the synthesis unit 207 may determine whether the short side of the block to be decoded is less than a predetermined threshold value 2.

[0082] As a modification of the above, the shape of the block to be decoded may be determined by using the short side of the block to be decoded, the aspect ratio of the block, or the division mode (angle of the division boundary).

[0083] For example, if the short side of the block to be decoded is small, the aspect ratio of the block is large (e.g., length:width = 4:1), and the angle of the division boundary is 45 degrees or more, weighting coefficients for patterns with wide division boundaries may be excluded from the options.

[0084] Conversely, if the short side of the block to be decoded is small, the aspect ratio of the block is large (e.g., length:width = 4:1), and the angle of the division boundary is less than 45 degrees, weighting coefficients with patterns in which the width of the division boundary is narrow may be excluded from the options.

[0085] This makes it possible to select the width of the division boundary taking into account the shape of the block, which is expected to improve coding performance.

[0086] The synthesizing unit 207 may also be configured to select the weighting factor in accordance with the motion vector.

[0087] Specifically, the synthesizing unit 207 may be configured to use the motion vectors of the small regions and select the weighting coefficients described above in accordance with the length of the motion vectors of the small regions or the resolution of the motion vectors of the small regions.

[0088] The larger the motion vector, the more blurred the boundary between the divisions, so it is desirable to widen the distribution of the weighting factors.Similarly, the coarser the resolution of the motion vector, the more blurred the boundary between the divisions, so it is desirable to widen the distribution of the weighting factors.

[0089] Furthermore, the synthesizing unit 207 may be configured to select the weighting factor described above in accordance with the difference between the motion vectors of the small area A and the small area B.

[0090] Here, the difference in motion vector is the difference in the reference frames of the motion vectors of small area A and small area B, or the difference in the motion vectors themselves.

[0091] For example, the synthesis unit 207 may be configured to select the above-mentioned weighting factor so as to narrow the distribution of the weighting factor if the difference between the motion vectors of the small area A and the small area B is equal to or greater than a predetermined threshold (e.g., 1 pixel), and to select the above-mentioned weighting factor so as to widen the distribution of the weighting factor if the difference between the motion vectors of the small area A and the small area B is less than a predetermined threshold (e.g., 1 pixel).

[0092] According to this configuration, highly accurate prediction can be made in accordance with image edges that may occur near division boundaries (such as boundaries between background and foreground that have different movements).

[0093] Alternatively, the synthesis unit 207 may be configured to select the above-mentioned weighting factor so as to widen the distribution of weighting factors if the difference between the motion vectors of small area A and small area B is equal to or greater than a predetermined threshold (e.g., 1 pixel), and to select the above-mentioned weighting factor so as to narrow the distribution of weighting factors if the difference between the motion vectors of small area A and small area B is less than a predetermined threshold (e.g., 1 pixel).

[0094] According to this configuration, it is possible to make a highly accurate prediction in accordance with the magnitude of motion blur near the division boundary. Here, the synthesis unit 207 may be configured to select a weighting factor that can be selected depending on the relationship between the angle between the motion vector and the division boundary.

[0095] For example, as shown in FIG. 9, the synthesis unit 207 may be configured to select the above-mentioned weighting coefficient according to the absolute value |x×u+y×v| of the dot product of the motion vector (x, y) and the unit normal vector (u, v) of the division boundary.

[0096] Alternatively, the synthesis unit 207 may be configured to select a weighting factor that is selectable depending on the exposure time or the frame rate.

[0097] Blurring is likely to occur when the exposure time is long or the frame rate is low, but blurring is unlikely when the exposure time is short or the frame rate is high, so this configuration makes it possible to select an appropriate width.

[0098] For example, the synthesis unit 207 is configured to select 2, which has a wide width, in the former case, and select 3, which has a narrow width, in the latter case.

[0099] Furthermore, the synthesis unit 207 may be configured to select a weighting factor that can be selected depending on the prediction method for the small region.

[0100] Since intra prediction and motion compensation are assumed as prediction methods, this configuration makes it possible to improve prediction accuracy by setting them according to the characteristics of each method.

[0101] Furthermore, the synthesis unit 207 may be configured to select a selectable weighting factor depending on the quantization parameter.

[0102] Since the larger the value of the quantization parameter, the narrower the width is likely to be selected, this configuration makes it possible to improve prediction accuracy by adding the quantization parameter to the decision criteria.

[0103] Furthermore, the synthesis unit 207 may be configured to select a weighting factor for the block to be decoded in accordance with control information of blocks neighboring the block to be decoded, rather than the control information of the block to be decoded.

[0104] For example, since small regions tend to be continuous across multiple blocks, the synthesis unit 207 may be configured to select a weighting factor for the block to be decoded depending on the weighting factors of adjacent already-decoded blocks.

[0105] FIG. 10 is a diagram showing an example of blocks adjacent to the left, upper left, upper, and upper right of the current block to be decoded.

[0106] Although partition boundaries also exist in the blocks to the left and top left adjacent to the block to be decoded, the synthesis unit 207 does not select these because they are not contiguous with the partition boundary of the block to be decoded, and can instead select the width of the partition boundary of the block above the block to be decoded, whose partition boundary is contiguous, for the block to be decoded.

[0107] Similarly, the synthesis unit 207 may be configured to derive a pattern of weighting factors of blocks adjacent to the block to be decoded as an internal parameter corresponding to the merge index used when decoding the merge vector of each small region, and select the pattern as the weighting factor of each small region of the block to be decoded.

[0108] The synthesis unit 207 may be configured to select the width of a division boundary of a predetermined pattern (e.g., pattern (1)) for a sub-region of the block to be decoded when there is no merge vector corresponding to each sub-region.

[0109] Here, when each small region is in intra prediction mode, the synthesis unit 207 may be configured to select the width of the division boundary of a predetermined pattern (e.g., pattern (1)) for the small region of the block to be decoded.

[0110] According to these configurations, prediction accuracy can be improved by inheriting the width of neighboring blocks having similar motion.

[0111] Third Embodiment Hereinafter, the third embodiment of the present invention will be described with reference to FIGS. 3 and 8 to 11, focusing on the differences from the first and second embodiments described above.

[0112] In this embodiment, the synthesis unit 207 is configured to generate a third predicted pixel by weighted averaging of at least one of the first predicted pixel and the second predicted pixel using any of the weighting factors limited based on the decoded control information.

[0113] That is, the synthesis unit 207 is configured to limit the combinations of weighting factors that can be selected according to the indirect control information, and then select a weighting factor to be applied based on the decoded control information from the limited combinations of weighting factors.

[0114] Here, the synthesis unit 207 may be configured to prepare a plurality of weighting factors with different widths of the dividing boundaries of the small regions and select the weighting factor.

[0115] The synthesis unit 207 may be configured to limit the combinations of weighting factors that can be selected according to the shape of the block to be decoded, which serves as indirect control information.

[0116] For example, the synthesis unit 207 may be configured to limit the selectable weighting factors depending on at least one of the size of the block to be decoded (the short side of the block to be decoded, the long side of the block to be decoded, etc.), the aspect ratio of the block to be decoded, the partition mode, and the number of pixels of the block to be decoded.

[0117] Here, when using the short side of the block to be decoded as the shape of the block to be decoded, if the short side of the block to be decoded is small, performing a weighted average over a wide area will result in a result that is no different from simple bidirectional prediction, so it is desirable to exclude weight coefficients with patterns in which the width of the division boundary is wide from the options (combinations of selectable weight coefficients).

[0118] For example, in the example of Figure 3, when the short side of the block to be decoded is equal to or smaller than a threshold, the synthesis unit 207 limits the selectable combination of weighting factors to the weighting factors of pattern (1) / (3), and when the short side of the block to be decoded is greater than the threshold, it limits the selectable combination of weighting factors to the weighting factors of pattern (1) / (2).This increases the number of patterns while reducing the amount of coding for the control information of the patterns, thereby achieving the effect of improving coding efficiency.

[0119] Here, the threshold value for the short side of the block to be decoded may be set to, for example, 8 pixels or 16 pixels.

[0120] An example of the weighting factor setting process performed by the synthesis unit 207 will be described below with reference to FIG.

[0121] 11, in step S301, the synthesis unit 207 determines whether any of sps_div_enabled_flag, pps_div_enabled_flag, and sh_div_enabled_flag included in the above-mentioned control information is 1. If the result is No (if none of them is 1), the process proceeds to step S302, and if the result is Yes, the process proceeds to step S303.

[0122] In step S302, the synthesizer 207 does not apply weighted averaging using a weighting factor to the current block to be decoded.

[0123] In step S303, the synthesis unit 207 determines whether GPM has been applied to the block to be decoded. If the result is No, the process proceeds to step S302, and if the result is Yes, the process proceeds to step S304.

[0124] In step S304, the synthesizing unit 207 determines whether the shorter side of the block to be decoded is equal to or less than a preset threshold.

[0125] If the answer is No, the process proceeds to step S305, and if the answer is Yes, the process proceeds to step S306. Here, if the answer is No, the synthesis unit 207 limits the selectable weighting factor combinations to pattern (1) / (2), and if the answer is Yes, the synthesis unit 207 limits the selectable weighting factor combinations to pattern (1) / (3).

[0126] In step S305, the synthesis unit 207 decodes cu_div_blending_idx (direct control information) included in the above-mentioned control information.

[0127] If cu_div_blending_idx is not 0, the operation proceeds to step S307, and if cu_div_blending_idx is 0, the operation proceeds to step S308.

[0128] Similarly, in step S306, if cu_div_blending_idx is not 0, the operation proceeds to step S309, and if cu_div_blending_idx is 0, the operation proceeds to step S310.

[0129] In step S307, the synthesis unit 207 selects and applies the weighting factor of pattern (1) from pattern (1) / (2) as the weighting factor.

[0130] In step S308, the synthesis unit 207 selects and applies the weighting factor of pattern (2) from pattern (1) / (2) as the weighting factor.

[0131] In step S309, the synthesis unit 207 selects and applies the weighting factor of pattern (1) from patterns (1) / (3) as the weighting factor.

[0132] In step S310, the synthesis unit 207 selects and applies the weighting factor of pattern (3) from among patterns (1) / (3).

[0133] Similarly, when the shape of the block to be decoded is determined by factors such as the long side of the block to be decoded, the aspect ratio of the block to be decoded, the partition mode, and the number of pixels of the block to be decoded, performing a weighted average over a wide area results in an effect that is no different from simple bi-prediction, and therefore it is desirable to exclude weight coefficients for patterns with wide partition boundaries from the options.

[0134] That is, in step S304 of the flowchart shown in FIG. 11, the short side of the block to be decoded may be replaced with the long side of the block to be decoded, the aspect ratio of the block to be decoded, the partition mode, or the number of pixels of the block to be decoded.

[0135] In the flowchart shown in FIG. 11, in step S304, the synthesizing unit 207 may determine whether the shorter side of the block to be decoded is less than a preset threshold value.

[0136] As a modification of the above, the shape of the block to be decoded may be determined by using the short side of the block to be decoded, the aspect ratio of the block, or the division mode (angle of the division boundary).

[0137] For example, if the short side of the block to be decoded is small, the aspect ratio of the block is large (e.g., length:width = 4:1), and the angle of the division boundary is 45 degrees or more, weighting coefficients for patterns with wide division boundaries may be excluded from the options.

[0138] Conversely, if the short side of the block to be decoded is small, the aspect ratio of the block is large (e.g., length:width = 4:1), and the angle of the division boundary is less than 45 degrees, weighting coefficients with patterns in which the width of the division boundary is narrow may be excluded from the options.

[0139] This makes it possible to select the width of the division boundary taking into account the shape of the block, which is expected to improve coding performance.

[0140] Furthermore, the synthesizing unit 207 may be configured to limit the combinations of weighting factors that can be selected depending on the motion vector.

[0141] Specifically, the synthesis unit 207 may be configured to use the motion vectors of the small regions and limit the combinations of selectable weighting factors depending on the length of the motion vectors of the small regions or the resolution of the motion vectors of the small regions.

[0142] The larger the motion vector, the more blurred the boundary between the divisions, so it is desirable to widen the distribution of the weighting factors.Similarly, the coarser the resolution of the motion vector, the more blurred the boundary between the divisions, so it is desirable to widen the distribution of the weighting factors.

[0143] Furthermore, the synthesis unit 207 may be configured to limit the combinations of weighting factors that can be selected according to the difference between the motion vectors of the small area A and the small area B.

[0144] Here, the difference in motion vector is the difference in the reference frames of the motion vectors of small area A and small area B, or the difference in the motion vectors themselves.

[0145] For example, the synthesis unit 207 may be configured to limit the selectable combinations of weighting factors so as to narrow the distribution of weighting factors if the difference between the motion vectors of small area A and small area B is equal to or greater than a predetermined threshold (e.g., 1 pixel), and to limit the selectable combinations of weighting factors so as to widen the distribution of weighting factors if the difference between the motion vectors of small area A and small area B is less than a predetermined threshold (e.g., 1 pixel).

[0146] According to this configuration, highly accurate prediction can be made in accordance with image edges that may occur near division boundaries (such as boundaries between background and foreground that have different movements).

[0147] Alternatively, the synthesis unit 207 may be configured to limit the combination of selectable weighting factor patterns so as to widen the distribution of weighting factors if the difference between the motion vectors of small area A and small area B is equal to or greater than a predetermined threshold (e.g., 1 pixel), and to limit the combination of selectable weighting factor patterns so as to narrow the distribution of weighting factors if the difference between the motion vectors of small area A and small area B is less than a predetermined threshold (e.g., 1 pixel).

[0148] According to this configuration, it is possible to make a highly accurate prediction in accordance with the magnitude of motion blur near the division boundary.

[0149] Here, the synthesizing unit 207 may be configured to limit the combinations of weighting factors that can be selected depending on the relationship between the angle between the motion vector and the division boundary.

[0150] For example, as shown in FIG. 9, the synthesis unit 207 may be configured to limit the combinations of selectable weighting factors depending on the absolute value |x×u+y×v| of the dot product of the motion vector (x, y) and the unit normal vector (u, v) of the division boundary.

[0151] Alternatively, the combiner 207 may be configured to limit the selectable weighting factors depending on the exposure time or the frame rate.

[0152] Blurring is likely to occur when the exposure time is long or the frame rate is low, but blurring is unlikely when the exposure time is short or the frame rate is high, so this configuration makes it possible to select an appropriate width.

[0153] For example, the synthesis unit 207 is configured to select 2, which has a wide width, in the former case, and select 3, which has a narrow width, in the latter case.

[0154] Furthermore, the synthesis unit 207 may be configured to limit the combinations of weighting factors that can be selected depending on the prediction method for the small region.

[0155] Since intra prediction and motion compensation are assumed as prediction methods, this configuration makes it possible to improve prediction accuracy by setting them according to the characteristics of each method.

[0156] Furthermore, the synthesis unit 207 may be configured to limit the combinations of weighting factors that can be selected depending on the quantization parameter.

[0157] Since the larger the value of the quantization parameter, the narrower the width is likely to be selected, this configuration makes it possible to improve prediction accuracy by adding the quantization parameter to the decision criteria.

[0158] In addition, the synthesis unit 207 may be configured to limit the selectable combinations of weighting factors for the block to be decoded, not only in accordance with the control information of the block to be decoded, but also in accordance with the control information of blocks neighboring the block to be decoded.

[0159] For example, since small regions tend to be continuous across multiple blocks, the synthesis unit 207 may be configured to limit the combinations of weighting factors of the block to be decoded that can be selected depending on the weighting factors of adjacent decoded blocks.

[0160] FIG. 10 is a diagram showing an example of blocks adjacent to the left, upper left, upper, and upper right of the current block to be decoded.

[0161] Although partition boundaries also exist in the blocks to the left and top left adjacent to the block to be decoded, the synthesis unit 207 does not include these boundaries in the combination of weighting factors for the block to be decoded because they are not contiguous with the partition boundaries of the block to be decoded. Instead, the synthesis unit 207 can include the width of the partition boundary of the block above the block to be decoded, which has a contiguous partition boundary, in the combination of weighting factors for the block to be decoded.

[0162] In addition, when limiting the selectable combinations of weighting factors for the block to be decoded, the synthesis unit 207 may be configured to limit the combinations in stages, rather than being limited to the binary choice of whether or not to include the combination.

[0163] For example, the decoding unit 201 improves the coding efficiency by assigning different code lengths according to the selection probability of the weighting coefficients and decoding the data.

[0164] In the above example, the decoding unit 201 can set the weighting factor pattern used by the adjacent decoded block as a short code length, and set other patterns as long code lengths.

[0165] The above-described image decoding device 200 may be realized as a program that causes a computer to execute each function (each step). [Industrial Applicability]

[0166] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which will contribute to the achievement of Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]

[0167] 200...Image decoding device 201...Decoding section 202...Inverse quantization section 203...Inverse conversion unit 204...Intra prediction unit 205...Storage section 206...Motion compensation unit 207...Synthesis section 208...Addition section 210...Code input section 220...Image output unit

Claims

1. An image decoding device, a decoding unit for decoding the control information and the quantized value; an inverse quantization unit that inverse quantizes the decoded quantized values ​​to obtain decoded transform coefficients; an inverse transform unit that inversely transforms the decoded transform coefficients to generate decoded prediction residuals; an intra prediction unit that generates a first predicted pixel based on a decoded pixel and the decoded control information; a storage unit that stores the decoded pixels; a motion compensation unit that generates a second predicted pixel based on the accumulated decoded pixels and the decoded control information; a synthesis unit that generates a third predicted pixel by controlling the width of the division boundary by a weighted average using a plurality of weighting coefficients having different widths of the division boundary for at least one of the first predicted pixel and the second predicted pixel; an adder that adds the decoded prediction residual and the third predicted pixel to obtain the decoded pixel; the control information includes a syntax for specifying a pattern of the weighting coefficients; The image decoding device, wherein the synthesis unit selects and applies a weighting factor of the pattern specified by the syntax from the plurality of weighting factors.

2. The image decoding device according to claim 1 , wherein the synthesis unit sets the weighting factors symmetrical with respect to the division boundary.

3. The image decoding device according to claim 1 , wherein the synthesis unit sets, as the weighting coefficient, a weighting coefficient that is asymmetric with respect to the division boundary.

4. The image decoding device according to claim 1 , wherein the synthesis unit sets the plurality of weighting factors in accordance with inter-pixel distances from the division boundary.

5. The image decoding device according to claim 3 , wherein the synthesis unit sets the weighting coefficients for a plurality of line segments according to inter-pixel distances from the division boundary.

6. An image decoding method, comprising: a step A of decoding the control information and the quantized values; a step B of dequantizing the decoded quantized values ​​to obtain decoded transform coefficients; a step C of inverse transforming the decoded transform coefficients to obtain decoded prediction residuals; a step D of generating a first predicted pixel based on the decoded pixel and the decoded control information; a step E of storing said decoded pixels; a step F of generating a second predicted pixel based on the stored decoded pixel and the decoded control information; a step G of preparing a plurality of weighting coefficients having different widths of a division boundary for at least one of the first predicted pixel and the second predicted pixel, and generating a third predicted pixel by controlling the width of the division boundary by a weighted average; and a step H of adding the decoded prediction residual and the third predicted pixel to obtain the decoded pixel; the control information includes a syntax for specifying a pattern of the weighting coefficients; In the step G, a weighting factor of the pattern designated by the syntax is selected and applied from the plurality of weighting factors.

7. A program that causes a computer to function as an image decoding device, The image decoding device comprises: a decoding unit for decoding the control information and the quantized value; an inverse quantization unit that inverse quantizes the decoded quantized values ​​to obtain decoded transform coefficients; an inverse transform unit that inversely transforms the decoded transform coefficients to generate decoded prediction residuals; an intra prediction unit that generates a first predicted pixel based on a decoded pixel and the decoded control information; a storage unit that stores the decoded pixels; a motion compensation unit that generates a second predicted pixel based on the accumulated decoded pixels and the decoded control information; a synthesis unit that generates a third predicted pixel by controlling the width of the division boundary by a weighted average using a plurality of weighting coefficients having different widths of the division boundary for at least one of the first predicted pixel and the second predicted pixel; an adder that adds the decoded prediction residual and the third predicted pixel to obtain the decoded pixel; the control information includes a syntax for specifying a pattern of the weighting coefficients; The program, wherein the synthesis unit selects and applies a weighting factor for the pattern specified by the syntax from the plurality of weighting factors.

Images