Image decoding device, image decoding method, and program

Abstract

To improve coding efficiency in GPM. [Solution] The image decoding device 200 according to the present invention comprises a decoding unit 201 that decodes control information and quantization values, an inverse quantization unit 202 that inversely quantizes the quantization values ​​to obtain conversion coefficients, an inverse conversion unit 203 that inversely converts the conversion coefficients to obtain prediction residuals, an intra-prediction unit 204 that generates first prediction pixels based on decoded pixels and control information, an accumulation unit 205 that accumulates decoded pixels, a motion compensation unit 206 that generates second prediction pixels based on decoded pixels and control information, a synthesis unit 207 that generates a third prediction pixel by weighted averaging using weight coefficients uniquely selected from a plurality of weight coefficients based on indirect control information for at least one of the first prediction pixels and the second prediction pixels, and an addition unit 208 that adds the prediction residuals and the third prediction pixels to obtain decoded pixels.

Description

Technical Field

[0001] The present invention relates to an image decoding apparatus, an image decoding method, and a program.

Background Art

[0002] In Non-Patent Document 1 and Non-Patent Document 2, a Geometric Partitioning Mode (GPM) is disclosed.

[0003] GPM divides a rectangular block diagonally into two parts and performs motion compensation on each part. Specifically, the two divided regions are motion-compensated by merge vectors and synthesized by weighted averaging.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the technologies disclosed in Non-Patent Document 1 and Non-Patent Document 2, since the pattern of weighted averaging is limited, there is a problem that there is room for improvement in coding performance. Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide an image decoding apparatus, an image decoding method, and a program capable of improving the coding efficiency in GPM.

Means for Solving the Problems

[0006] The first feature of the present invention is an image decoding device comprising: a decoding unit that decodes control information and quantization values; an inverse quantization unit that inversely quantizes the decoded quantization values ​​to obtain decoded conversion coefficients; an inverse transformation unit that inversely transforms the decoded conversion coefficients to obtain decoded prediction residuals; an intra-prediction unit that generates first prediction pixels based on decoded pixels and the decoded control information; an accumulation unit that accumulates the decoded pixels; a motion compensation unit that generates second prediction pixels based on the accumulated decoded pixels and the decoded control information; a synthesis unit that generates a third prediction pixel by weighted averaging using weight coefficients uniquely selected from a plurality of weight coefficients based on indirect control information for at least one of the first prediction pixels and the second prediction pixels; and an addition unit that adds the decoded prediction residuals and the third prediction pixels to obtain the decoded pixels.

[0007] A second feature of the present invention is an image decoding method comprising the steps of: decoding control information and quantization values; inverse quantization of the decoded quantization values ​​to obtain decoded conversion coefficients; inverse transformation of the decoded conversion coefficients to obtain decoded prediction residuals; generating first prediction pixels based on decoded pixels and the decoded control information; accumulating the decoded pixels; generating second prediction pixels based on the accumulated decoded pixels and the decoded control information; generating a third prediction pixel by weighted averaging of at least one of the first prediction pixels and the second prediction pixels using weight coefficients uniquely selected from a plurality of weight coefficients based on indirect control information; and adding the decoded prediction residuals and the third prediction pixels to obtain the decoded pixels.

[0008] A third feature of the present invention is a program that causes a computer to function as an image decoding device, wherein the image decoding device comprises: a decoding unit that decodes control information and quantization values; an inverse quantization unit that dequantizes the decoded quantization values ​​to obtain decoded conversion coefficients; an inverse transformation unit that inversely transforms the decoded conversion coefficients to obtain decoded prediction residuals; an intra-prediction unit that generates first prediction pixels based on decoded pixels and the decoded control information; an accumulation unit that accumulates the decoded pixels; a motion compensation unit that generates second prediction pixels based on the accumulated decoded pixels and the decoded control information; a synthesis unit that generates a third prediction pixel by weighted averaging using weight coefficients uniquely selected from a plurality of weight coefficients based on indirect control information for at least one of the first prediction pixels and the second prediction pixels; and an addition unit that adds the decoded prediction residuals and the third prediction pixels to obtain the decoded pixels. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program that can improve coding efficiency in GPM. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 shows an example of the functional block of an image decoding device 200 according to one embodiment. [Figure 2] Figure 2 shows an example of a case where a rectangular unit block is divided into two sub-regions, A and B, by a dividing boundary. [Figure 3] Figure 3 shows an example of three patterns of weight coefficients that can be assigned to the division boundary of sub-region B shown in Figure 2. [Figure 4] Figure 4 shows an example of applying the weight coefficient w of pattern (2) to an 8x8 block. [Figure 5] Figure 5 shows an example of applying the weight coefficient w of pattern (1) to an 8x8 block. [Figure 6]FIG. 6 is a diagram showing an example in which the weight coefficient w of the pattern (3) is applied to an 8×8 block. [Figure 7] FIG. 7 is a flowchart for explaining an example of the weight coefficient setting process by the synthesizing unit 207 in the first embodiment. [Figure 8] FIG. 8 is a flowchart for explaining an example of the weight coefficient setting process by the synthesizing unit 207 in the second embodiment. [Figure 9] FIG. 9 is a diagram for explaining the second embodiment. [Figure 10] FIG. 10 is a diagram for explaining the second embodiment. [Figure 11] FIG. 11 is a flowchart for explaining an example of the weight coefficient setting process by the synthesizing unit 207 in the third embodiment. [Figure 12] FIG. 12 is a diagram for explaining an example in which the weight coefficient is defined based on the distance from the division boundary. MODE FOR CARRYING OUT THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components or the like, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0012] <First Embodiment> Hereinafter, the image decoding apparatus 200 according to the present embodiment will be described with reference to FIGS. 1 to 7. FIG. 1 is a diagram showing an example of the functional blocks of the image decoding apparatus 200 according to the present embodiment.

[0013] As shown in FIG. 1, the image decoding apparatus 200 includes a code input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transformation unit 203, an intra prediction unit 204, an accumulation unit 205, a motion compensation unit 206, a synthesis unit 207, an addition unit 208, and an image output unit 220.

[0014] The code input unit 210 is configured to acquire coded information coded by an image encoding apparatus.

[0015] The decoding unit 201 is configured to decode control information and quantization values from the coded information input from the code input unit 210. For example, the decoding unit 201 is configured to output control information and quantization values by performing variable length decoding on such coded information.

[0016] Here, the quantization values are sent to the inverse quantization unit 202, and the control information is sent to the motion compensation unit 206, the intra prediction unit 204, and the synthesis unit 207. Note that such control information includes information necessary for control of the motion compensation unit 206, the intra prediction unit 204, the synthesis unit 207, etc., and may include header information such as a sequence parameter set, a picture parameter set, a picture header, a slice header, etc.

[0017] The inverse quantization unit 202 is configured to inverse quantize the quantization values sent from the decoding unit 201 to obtain decoded transform coefficients. Such transform coefficients are sent to the inverse transformation unit 203.

[0018] The inverse transformation unit 203 is configured to inverse transform the transform coefficients sent from the inverse quantization unit 202 to obtain decoded prediction residuals. Such prediction residuals are sent to the addition unit 208.

[0019] The intra-prediction unit 204 is configured to generate a first prediction pixel based on the decoded pixels and control information sent from the decoding unit 201. Here, the decoded pixels are obtained via the addition unit 208 and stored in the storage unit 205. The first prediction pixel is a prediction pixel that is an approximation of the input pixels in a small region set by the synthesis unit 207. The first prediction pixel is sent to the synthesis unit 207.

[0020] The storage unit 205 is configured to cumulatively store the decoded pixels sent from the addition unit 208. These decoded pixels are referenced by the motion compensation unit 206 via the storage unit 205.

[0021] The motion compensation unit 206 is configured to generate a second predicted pixel based on the decoded pixels stored in the storage unit 205 and the control information sent from the decoding unit 201. Here, the second predicted pixel is a predicted pixel that is an approximation of the input pixel in a small region set by the synthesis unit 207. The second predicted pixel is sent to the synthesis unit 207.

[0022] The addition unit 208 is configured to obtain a decoded pixel by adding the predicted residual sent from the inverse transformation unit 203 and the third predicted pixel sent from the synthesis unit 207. This decoded pixel is then sent to the image output unit 220, the storage unit 205, and the intra-prediction unit 204.

[0023] The synthesis unit 207 is configured to generate a third prediction pixel by controlling the width of the division boundary using a weighted average, by preparing a plurality of weight coefficients with different division boundary widths for at least one of the first prediction pixel sent from the intra prediction unit 204 and the second prediction pixel sent from the motion compensation unit 206.

[0024] The role of the synthesis unit 207 is to select weight coefficients for multiple prediction pixels that are optimal for the block to be decoded, and to synthesize the input multiple prediction pixels according to the weight coefficients, in order to accurately compensate the block to be decoded in the subsequent addition unit 208.

[0025] Any partitioning mode can be used to divide the block to be decoded into multiple sub-regions, but below we will describe the case using the Geometric Partitioning Mode (GPM), which is disclosed in Non-Patent Documents 1 and 2, as an example of a partitioning mode.

[0026] Regarding the weight coefficients, multiple patterns are prepared with arbitrary values ​​set in advance for each pixel of the unit block, and one of these patterns is applied. In other words, the compositing unit 207 may be configured to select and apply one of the multiple weight coefficients.

[0027] With this configuration, by preparing a lookup table or the like with multiple weight coefficients set, the synthesis unit 207 does not need to calculate the weight coefficients each time.

[0028] The sum of the weight coefficients for multiple prediction pixels is designed to be 1 for each pixel, and the result of combining the multiple prediction pixels using these weight coefficients through weighted averaging is used as the prediction pixel by the synthesis unit 207.

[0029] Pixels with a weight coefficient of 1 (i.e., the maximum value) are adopted as input prediction pixels, while pixels with a weight coefficient of 0 (i.e., the minimum value) are not used as input prediction pixels. Conceptually, this is equivalent to dividing a unit block into multiple sub-regions, and determining which of the multiple input prediction pixels to apply to which region and in what proportion.

[0030] Here, it is desirable to distribute the weight coefficients in a non-rectangular shape, since a rectangular distribution such as bisecting allows them to be represented by smaller unit blocks.

[0031] Figure 2 illustrates an example where the unit blocks are distributed in a diagonal shape. In Figure 2, a rectangular unit block is divided into two sub-regions, A and B, by a dividing boundary.

[0032] In each sub-region A / B, predicted pixels are generated using any method, such as intra-prediction or motion compensation.

[0033] In this case, even if the shape of the partition is determined, if the weight coefficients near the partition boundary are fixed, it is not possible to represent the diversity of the partition boundary, and therefore the coding efficiency cannot be improved.

[0034] For example, if a small region is an area with rapid movement, blurring occurs during imaging. Therefore, it is preferable to blur multiple small regions over a wide area and weight the resulting boundary.

[0035] Conversely, if the small area is an artificially edited area like a text overlay, blurring will not occur. Therefore, it is preferable to limit the division boundary to a narrow area and simply weight the average to make multiple small areas adjacent to each other.

[0036] To solve this problem, in this embodiment, a procedure is taken in which multiple weight coefficients with different widths for the sub-region division boundaries are prepared and selected.

[0037] Figure 3 shows an example of three patterns of weight coefficients that can be assigned to the division boundary of sub-region B shown in Figure 2. In Figure 3, the horizontal axis represents the distance in pixels from the position of the division boundary, and the vertical axis represents the weight coefficient.

[0038] Specifically, we have prepared three patterns: (1) in which the weight coefficient [0,1] is assigned to the range [a,b] for pixel-level distances a and b from a predetermined division boundary position; (2) in which distances a and b are similarly doubled and the weight coefficient [0,1] is assigned to the range [2a,2b]; and (3) in which distances a and b are similarly halved and the weight coefficient [0,1] is assigned to the range [a / 2,b / 2]. In these patterns, the weight coefficient is assigned to the distance d(x) from the division boundary (black solid line) as shown in Figure 12. c ,y c γ, which is uniquely determined by ) xc,ycWhen defined as such, it is equivalent to providing multiple patterns (variable values) rather than a limited pattern (fixed value) for the width of the sub-region division boundary in Figure 12, i.e., the width τ where the weight coefficient is other than the minimum or maximum value. Here, x c ,y c This represents the coordinates within the block to be decrypted.

[0039] In other words, the synthesis unit 207 may be configured to set a plurality of weighting coefficients according to the inter-pixel distance from the division boundary.

[0040] This configuration allows for variable boundary width by being proportional to the distance from the dividing boundary, while minimizing changes from the conventional calculation formula shown in Figure 12.

[0041] Alternatively, a=b may be used to set a weight coefficient that is symmetric with respect to the division boundary. That is, the synthesis unit 207 may be configured to set a weight coefficient that is symmetric with respect to the division boundary as the weight coefficient described above. With such a configuration, b becomes unnecessary, and therefore the amount of sign can be reduced.

[0042] Alternatively, a weight coefficient that is asymmetric with respect to the dividing boundary may be set, where a ≠ b. In other words, the synthesis unit 207 may be configured to set a weight coefficient that is asymmetric with respect to the dividing boundary as the weight coefficient described above. With such a configuration, it is possible to make predictions with high accuracy when there are different degrees of blurring on both sides of the boundary.

[0043] Furthermore, the number of weighting coefficients can be increased beyond just two, a and b, to include multiple line segments. In other words, the synthesis unit 207 may be configured to set weighting coefficients using multiple line segments according to the inter-pixel distance from the division boundary. With such a configuration, high-precision prediction is possible when blurring occurs non-linearly.

[0044] Figures 4 to 6 show examples of applying each weight coefficient w to an 8x8 block. The weight coefficient w in Figures 4 to 6 can take values ​​from 0 to 8 and are combined using the following formula.

[0045] (w×small area A+(8-w)×small area B+4)>>3 In this way, by setting multiple weight coefficients w according to the inter-pixel distance from the division boundary, the effect of being able to derive a uniform result for various block sizes such as 8x8 and 64x16 can be obtained. The type, shape, and number of patterns can be set arbitrarily. For example, above, we explained that the multiple patterns were twice and half the distances a and b, but they could also be four times or a quarter of the distance. Also, in the above formula, the weight coefficients were set to values ​​from 0 to 8, but they can also be set to other values ​​such as 0 to 16 or 0 to 32. In particular, when the inter-pixel distance from the division boundary is twice or four times, increasing the maximum value of the weight coefficient can improve the accuracy of the pixel-level weighted average.

[0046] Below, with reference to Figure 7, an example of the weight coefficient setting process by the synthesis unit 207 will be explained.

[0047] As shown in Figure 7, in step S101, the synthesis unit 207 determines whether any of sps_div_enabled_flag, pps_div_enabled_flag, and sh_div_enabled_flag included in the control information described above is 1. If No (none of them is 1), the process proceeds to step S102; if Yes, the process proceeds to step S103.

[0048] In step S102, the synthesis unit 207 does not apply a weighted average using weight coefficients to the blocks to be decoded.

[0049] In step S103, the synthesis unit 207 determines whether or not GPM is applied to the block to be decoded. If No, the process proceeds to step S102; if Yes, the process proceeds to step S104.

[0050] In step S104, the blending unit 207 decodes cu_div_blending_idx included in the control information described above.

[0051] If cu_div_blending_idx is 0, the operation proceeds to step S105; if cu_div_blending_idx is 1, the operation proceeds to step S106; and if cu_div_blending_idx is 2, the operation proceeds to step S107.

[0052] In step S105, the synthesis unit 207 selects and applies the weight coefficient of pattern (1) from patterns (1) to (3) as the weight coefficient.

[0053] In step S106, the synthesis unit 207 selects and applies the weight coefficient of pattern (2) from patterns (1) to (3) as the weight coefficient.

[0054] In step S107, the synthesis unit 207 selects and applies the weight coefficient of pattern (3) from patterns (1) to (3) as the weight coefficient.

[0055] Furthermore, the synthesis unit 207 may be configured to use the weighting coefficients derived for determining the width of the division boundary of the color difference component

[0056] Furthermore, if the color difference component of the block to be decoded is not downsampled relative to the luminance component of the block to be decoded, the synthesis unit 207 may not directly use the weight coefficients derived for determining the division boundary width for the luminance component of the block to be decoded as weight coefficients for determining the division boundary width for the color difference component of the block to be decoded, but may instead derive weight coefficients for determining the division boundary width for the color difference component of the block to be decoded in the same manner as described above. With such a configuration, the weight coefficients for the color difference component of the block to be decoded can be derived independently, and an improvement in encoding performance can be expected.

[0057] On the other hand, if the color difference component of the block to be decoded is downsampled relative to the luminance component of the block to be decoded, the synthesis unit 207 may take the downsampling method into consideration and derive a weighting coefficient for determining the width of the division boundary of the color difference component of the block to be decoded from the width of the division boundary of the luminance component of the block to be decoded. With this configuration, the same effect obtained with the luminance component of the block to be decoded can be obtained with respect to the color difference component of the block to be decoded that is downsampled.

[0058] Furthermore, if the synthesis unit 207 uses control information such as a header to determine the width of the division boundary of the luminance component of the block to be decoded, it becomes unnecessary for the color difference component of the block to be decoded, thus improving encoding performance can be expected.

[0059] For example, if the color difference component of the block to be decoded is downsampled by half both horizontally and vertically with respect to the luminance component of the block to be decoded, the synthesis unit 207 may derive a weighting coefficient that determines the width of the division boundary such that it is half the width of the division boundary derived for the luminance component of the block to be decoded, as a weighting coefficient that determines the width of the division boundary for the color difference component of the block to be decoded.

[0060] For example, if the color difference component of the block to be decoded is downsampled by half in either the horizontal or vertical direction relative to the luminance component of the block to be decoded, the synthesis unit 207 may derive a weighting coefficient that determines the width of the division boundary so that it is equal to or half the width of the division boundary derived for the luminance component of the block to be decoded, as a weighting coefficient that determines the width of the division boundary for the color difference component of the block to be decoded.

[0061] <Second Embodiment> Hereinafter, with reference to Figures 3 and 8 to 10, a second embodiment of the present invention will be described, focusing on the differences from the first embodiment described above.

[0062] In this embodiment, code length is reduced by identifying patterns of weight coefficients while eliminating the need for direct control information.

[0063] Therefore, in this embodiment, the synthesis unit 207 is configured to generate a third predicted pixel by weighting and averaging of at least one of the first and second predicted pixels described above, using weight coefficients uniquely selected from a plurality of weight coefficients based on indirect control information.

[0064] In other words, in this embodiment, the synthesis unit 207 is configured to select (uniquely identify) a weight coefficient from among a plurality of weight coefficients according to indirect control information.

[0065] Here, the synthesis unit 207 may be configured to prepare a plurality of weight coefficients with different widths for the sub-region division boundaries and to select a weight coefficient from among these plurality of weight coefficients.

[0066] Specifically, the synthesis unit 207 may be configured to select weight coefficients from among a plurality of weight coefficients according to the shape of the block to be decoded as indirect control information.

[0067] For example, the synthesis unit 207 may be configured to select a weighting coefficient from a plurality of weighting coefficients according to at least one of the short side of the block to be decoded, the long side of the block to be decoded, the aspect ratio of the block to be decoded, the division mode, and the number of pixels in the block to be decoded.

[0068] For example, when using the shorter side of the block to be decoded as the shape of the block to be decoded, if the shorter side of the block to be decoded is small, weighting and averaging over a wide area will result in a result no different from simple bidirectional prediction. Therefore, it is desirable to exclude weight coefficients for patterns with wide partition boundaries from the options.

[0069] For example, in the example shown in Figure 3, the synthesis unit 207 selects the weight coefficient of pattern (3) when the short side of the block to be decoded is less than or equal to the threshold, and selects the weight coefficient of pattern (2) when the short side of the block to be decoded is greater than the threshold. This increases the number of patterns while eliminating the need for pattern control information, thereby improving coding efficiency.

[0070] The following describes an example of the weight coefficient setting process by the synthesis unit 207, with reference to Figure 8.

[0071] As shown in Figure 8, in step S201, the synthesis unit 207 determines whether any of sps_div_enabled_flag, pps_div_enabled_flag, and sh_div_enabled_flag included in the control information described above is 1. If No (none of them are 1), the process proceeds to step S202; if Yes, the process proceeds to step S203.

[0072] In step S202, the synthesis unit 207 does not apply a weighted average using weight coefficients to the blocks to be decoded.

[0073] In step S203, the synthesis unit 207 determines whether or not GPM is applied to the block to be decoded. If No, the process proceeds to step S202; if Yes, the process proceeds to step S204.

[0074] In step S204, the synthesis unit 207 determines whether the short side of the block to be decoded is less than or equal to a preset threshold of 1. If the result is No, the process proceeds to step S205; if the result is Yes, the process proceeds to step S208.

[0075] In step S205, the synthesis unit 207 determines whether the shorter side of the block to be decoded is less than or equal to a preset threshold of 2. Here, threshold 2 is greater than threshold 1. If No, the process proceeds to step S206; if Yes, the process proceeds to step S207.

[0076] In step S206, the synthesis unit 207 selects and applies the weight coefficient of pattern (2) from patterns (1) to (3) as the weight coefficient.

[0077] In step S207, the synthesis unit 207 selects and applies the weight coefficient of pattern (1) from patterns (1) to (3) as the weight coefficient.

[0078] In step S208, the synthesis unit 207 selects and applies the weight coefficient of pattern (3) from patterns (1) to (3) as the weight coefficient.

[0079] Similarly, when using the longest side of the block to be decoded, the aspect ratio of the block to be decoded, the division mode, and the number of pixels in the block to be decoded as the shape of the block to be decoded, applying a weighted average over a wide area will result in no different from simple biprediction. Therefore, it is desirable to exclude weight coefficients for patterns with wide division boundaries from the options.

[0080] In other words, in steps S204 and S205 of the flowchart shown in Figure 8, the shorter side of the block to be decoded may be replaced with the longer side of the block to be decoded, the aspect ratio of the block to be decoded, the division mode, or the number of pixels of the block to be decoded.

[0081] Furthermore, in the flowchart shown in Figure 8, in step S204, the merging unit 207 may determine whether the short side of the block to be decoded is less than a preset threshold of 1, and in step S205, the merging unit 207 may determine whether the short side of the block to be decoded is less than a preset threshold of 2.

[0082] Here, as an example of the above modifications, the shape of the block to be decoded may be determined by the shorter side of the block, the aspect ratio of the block, or the division mode (angle of the division boundary).

[0083] For example, if the shorter side of the block to be decrypted is small, the aspect ratio of the block is large (e.g., height:width = 4:1), and the angle of the division boundary is 45 degrees or more, then the weight coefficients for patterns with wide division boundaries may be excluded from the options.

[0084] Conversely, if the shorter side of the block to be decoded is small, the aspect ratio of the block is large (e.g., height:width = 4:1), and the angle of the division boundary is less than 45 degrees, then the weight coefficients for patterns with narrow division boundaries may be excluded from the options.

[0085] This allows for the selection of boundary widths that take the block shape into consideration, which is expected to improve encoding performance.

[0086] Furthermore, the synthesis unit 207 may be configured to select the weight coefficients described above according to the motion vector.

[0087] Specifically, the synthesis unit 207 may be configured to use the motion vectors of a small region and select the weight coefficients described above according to the length of the motion vectors of the small region or the resolution of the motion vectors of the small region.

[0088] Since a larger motion vector contributes to blurring the division boundary, it is desirable to broaden the distribution of weight coefficients. Similarly, since a coarser resolution of the motion vector contributes to blurring the division boundary, it is desirable to broaden the distribution of weight coefficients.

[0089] Furthermore, the synthesis unit 207 may be configured to select the weight coefficients described above according to the difference between the motion vectors of sub-region A and sub-region B.

[0090] Here, the difference in motion vectors is the difference in the reference frames of the motion vectors of sub-region A and sub-region B, or the difference in the motion vectors themselves.

[0091] For example, the synthesis unit 207 may be configured to select the weight coefficients described above to narrow the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is greater than or equal to a predetermined threshold (e.g., 1 pixel), and to select the weight coefficients described above to widen the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is less than a predetermined threshold (e.g., 1 pixel).

[0092] With this configuration, it is possible to make highly accurate predictions that match image edges that may occur near the division boundary (such as the boundary between the background and foreground, which have different movements).

[0093] Alternatively, the synthesis unit 207 may be configured to select the weight coefficients described above to broaden the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is greater than or equal to a predetermined threshold (e.g., 1 pixel), and to narrow the distribution of weight coefficients described above if the difference between the motion vectors of sub-region A and sub-region B is less than a predetermined threshold (e.g., 1 pixel).

[0094] With this configuration, it is possible to make highly accurate predictions based on the magnitude of motion blur near the division boundary. Here, the synthesis unit 207 may be configured to select a weight coefficient that can be selected based on the relationship between the angle between the motion vector and the division boundary.

[0095] For example, as shown in Figure 9, the synthesis unit 207 may be configured to select the weight coefficients described above according to the absolute value |x×u+y×v| of the dot product of the motion vector (x,y) and the unit normal vector (u,v) of the division boundary.

[0096] Alternatively, the synthesis unit 207 may be configured to select a weighting coefficient that can be selected according to the exposure time or frame rate.

[0097] Since blurring is more likely with long exposure times or low frame rates, and less likely with short exposure times or high frame rates, this configuration allows for the selection of an appropriate width.

[0098] For example, the composite section 207 is configured to select width 2 in the former case and width 3 in the latter case.

[0099] Furthermore, the synthesis unit 207 may be configured to select weight coefficients that can be chosen according to the prediction method for the small region.

[0100] The prediction method is assumed to be intraprediction and motion compensation, and with this configuration, prediction accuracy can be improved by setting the appropriate parameters for each.

[0101] Furthermore, the synthesis unit 207 may be configured to select selectable weight coefficients according to the quantization parameters.

[0102] Since larger quantization parameter values ​​tend to result in the selection of narrower widths, this configuration allows for improved prediction accuracy by incorporating the quantization parameter into the decision-making process.

[0103] Furthermore, the synthesis unit 207 may be configured to select the weight coefficient of the block to be decoded not only according to the control information of the block to be decoded, but also according to the control information of blocks adjacent to the block to be decoded.

[0104] For example, since small regions tend to be continuous across multiple blocks, the merging unit 207 may be configured to select the weight coefficient of the block to be decoded according to the weight coefficients of the adjacent decoded blocks.

[0105] Figure 10 shows an example of blocks adjacent to the block to be decrypted: to the left, upper left, top, and upper right.

[0106] Although partition boundaries also exist in the blocks adjacent to the decryption target block, such as the blocks to the left and upper left, the merging unit 207 does not select them because they are not continuous with the partition boundaries of the decryption target block. Instead, it can select the width of the partition boundary of the block above the decryption target block, where the partition boundary is continuous, for the decryption target block.

[0107] Similarly, the synthesis unit 207 may be configured to derive a pattern of weight coefficients of blocks adjacent to the block to be decoded as an internal parameter corresponding to the merge index used when decoding the merge vector of each sub-region, and to select it as the weight coefficient of each sub-region of the block to be decoded.

[0108] The merging unit 207 may be configured to select the width of the division boundary of a pre-set pattern (for example, pattern (1)) for the sub-region of the block to be decoded if there is no merge vector corresponding to each sub-region.

[0109] Here, the synthesis unit 207 may be configured to select the width of the division boundary of a pre-set pattern (for example, pattern (1)) for the sub-region of the block to be decoded when each sub-region is in intra-prediction mode.

[0110] These configurations allow for improved prediction accuracy by inheriting the width of neighboring blocks with similar movements.

[0111] <Third Embodiment> Hereinafter, with reference to Figures 3 and 8 to 11, a third embodiment of the present invention will be described, focusing on the differences from the first and second embodiments described above.

[0112] In this embodiment, the synthesis unit 207 is configured to generate a third predicted pixel by weighting an average of at least one of the first and second predicted pixels using one of the limited weight coefficients based on the decoded control information.

[0113] In other words, the synthesis unit 207 is configured to limit the combinations of weight coefficients that can be selected according to indirect control information, and then select the weight coefficient to be applied from among the limited combinations of weight coefficients based on the decoded control information.

[0114] Here, the synthesis unit 207 may be configured to prepare a plurality of weight coefficients with different widths for the sub-region division boundaries and to select the aforementioned weight coefficient.

[0115] The synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients according to the shape of the block to be decoded as indirect control information.

[0116] For example, the synthesis unit 207 may be configured to limit the selectable weight coefficients according to at least one of the following: the size of the block to be decoded (such as the short side or long side of the block to be decoded), the aspect ratio of the block to be decoded, the division mode, and the number of pixels in the block to be decoded.

[0117] Here, if the shorter side of the block to be decoded is used as the shape of the block to be decoded, then if the shorter side of the block to be decoded is small, weighting and averaging over a wide area will result in no different from simple bidirectional prediction. Therefore, it is desirable to exclude weight coefficients for patterns with wide partition boundaries from the options (combinations of selectable weight coefficients).

[0118] For example, in the example shown in Figure 3, the synthesis unit 207 limits the selectable weight coefficient combinations to the weight coefficients of pattern (1) / (3) when the short side of the block to be decoded is less than or equal to the threshold, and limits the selectable weight coefficient combinations to the weight coefficients of pattern (1) / (2) when the short side of the block to be decoded is greater than the threshold. This increases the number of patterns while reducing the amount of code for the control information of the patterns, thereby improving coding efficiency.

[0119] Here, the threshold for the short side of the block to be decrypted may be set to, for example, 8 pixels or 16 pixels.

[0120] The following describes an example of the weight coefficient setting process by the synthesis unit 207, with reference to Figure 11.

[0121] As shown in Figure 11, in step S301, the synthesis unit 207 determines whether any of sps_div_enabled_flag, pps_div_enabled_flag, and sh_div_enabled_flag included in the control information described above is 1. If No (none of them are 1), the process proceeds to step S302; if Yes, the process proceeds to step S303.

[0122] In step S302, the synthesis unit 207 does not apply a weighted average using weight coefficients to the blocks to be decoded.

[0123] In step S303, the synthesis unit 207 determines whether or not GPM is applied to the block to be decoded. If No, the process proceeds to step S302; if Yes, the process proceeds to step S304.

[0124] In step S304, the synthesis unit 207 determines whether the short side of the block to be decoded is less than or equal to a preset threshold.

[0125] If the answer is No, the process proceeds to step S305; if the answer is Yes, the process proceeds to step S306. Here, if the answer is No, the synthesis unit 207 limits the selectable combinations of weight coefficients to patterns (1) / (2); if the answer is Yes, the synthesis unit 207 limits the selectable combinations of weight coefficients to patterns (1) / (3).

[0126] In step S305, the blending unit 207 decodes cu_div_blending_idx (direct control information) included in the control information described above.

[0127] If cu_div_blending_idx is not 0, the process proceeds to step S307; if cu_div_blending_idx is 0, the process proceeds to step S308.

[0128] Similarly, in step S306, if cu_div_blending_idx is not 0, the operation proceeds to step S309; ​​if cu_div_blending_idx is 0, the operation proceeds to step S310.

[0129] In step S307, the synthesis unit 207 selects and applies the weight coefficient of pattern (1) from among patterns (1) and (2) as the weight coefficient.

[0130] In step S308, the synthesis unit 207 selects and applies the weight coefficient of pattern (2) from among patterns (1) and (2) as the weight coefficient.

[0131] In step S309, the synthesis unit 207 selects and applies the weight coefficient of pattern (1) from among patterns (1) and (3) as the weight coefficient.

[0132] In step S310, the synthesis unit 207 selects and applies the weight coefficient of pattern (3) from among patterns (1) and (3) as the weight coefficient.

[0133] Similarly, when using the longest side of the block to be decoded, the aspect ratio of the block to be decoded, the division mode, and the number of pixels in the block to be decoded as the shape of the block to be decoded, applying a weighted average over a wide area will result in no different from simple biprediction. Therefore, it is desirable to exclude weight coefficients for patterns with wide division boundaries from the options.

[0134] In other words, in step S304 of the flowchart shown in Figure 11, the shorter side of the block to be decoded may be replaced with the longer side of the block to be decoded, the aspect ratio of the block to be decoded, the division mode, or the number of pixels of the block to be decoded.

[0135] Furthermore, in the flowchart shown in Figure 11, in step S304, the synthesis unit 207 may determine whether the short side of the block to be decoded is less than a preset threshold.

[0136] Here, as an example of the above modifications, the shape of the block to be decoded may be determined by the shorter side of the block, the aspect ratio of the block, or the division mode (angle of the division boundary).

[0137] For example, if the shorter side of the block to be decrypted is small, the aspect ratio of the block is large (e.g., height:width = 4:1), and the angle of the division boundary is 45 degrees or more, then the weight coefficients for patterns with wide division boundaries may be excluded from the options.

[0138] Conversely, if the shorter side of the block to be decoded is small, the aspect ratio of the block is large (e.g., height:width = 4:1), and the angle of the division boundary is less than 45 degrees, then the weight coefficients for patterns with narrow division boundaries may be excluded from the options.

[0139] This allows for the selection of boundary widths that take the block shape into consideration, which is expected to improve encoding performance.

[0140] Furthermore, the synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients according to the motion vector.

[0141] Specifically, the synthesis unit 207 may be configured to utilize the motion vectors of a small region and limit the selectable combinations of weight coefficients according to the motion vector length of the small region or the resolution of the motion vectors of the small region.

[0142] Since a larger motion vector contributes to blurring the division boundary, it is desirable to broaden the distribution of weight coefficients. Similarly, since a coarser resolution of the motion vector contributes to blurring the division boundary, it is desirable to broaden the distribution of weight coefficients.

[0143] Furthermore, the synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients according to the difference between the motion vectors of sub-region A and sub-region B.

[0144] Here, the difference in motion vectors is the difference in the reference frames of the motion vectors of sub-region A and sub-region B, or the difference in the motion vectors themselves.

[0145] For example, the synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients so as to narrow the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is greater than or equal to a predetermined threshold (e.g., 1 pixel), and to limit the selectable combinations of weight coefficients so as to widen the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is less than a predetermined threshold (e.g., 1 pixel).

[0146] With this configuration, it is possible to make highly accurate predictions that match image edges that may occur near the division boundary (such as the boundary between the background and foreground, which have different movements).

[0147] Alternatively, the synthesis unit 207 may be configured to limit the combination of selectable weight coefficient patterns so as to broaden the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is greater than or equal to a predetermined threshold (e.g., 1 pixel), and to limit the combination of selectable weight coefficient patterns so as to narrow the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is less than a predetermined threshold (e.g., 1 pixel).

[0148] With this configuration, it is possible to make highly accurate predictions based on the magnitude of motion blur near the division boundary.

[0149] Here, the synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients based on the relationship between the motion vector and the division boundary.

[0150] For example, as shown in Figure 9, the synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients according to the absolute value |x×u+y×v| of the dot product of the motion vector (x,y) and the unit normal vector (u,v) of the division boundary.

[0151] Alternatively, the synthesis unit 207 may be configured to limit the selectable weighting coefficients according to the exposure time or frame rate.

[0152] Since blurring is more likely with long exposure times or low frame rates, and less likely with short exposure times or high frame rates, this configuration allows for the selection of an appropriate width.

[0153] For example, the composite section 207 is configured to select width 2 in the former case and width 3 in the latter case.

[0154] Furthermore, the synthesis unit 207 may be configured to limit the combination of weight coefficients that can be selected depending on the prediction method for the small region.

[0155] The prediction method is assumed to be intraprediction and motion compensation, and with this configuration, prediction accuracy can be improved by setting the appropriate parameters for each.

[0156] Furthermore, the synthesis unit 207 may be configured to limit the combination of weight coefficients that can be selected according to the quantization parameters.

[0157] Since larger quantization parameter values ​​tend to result in the selection of narrower widths, this configuration allows for improved prediction accuracy by incorporating the quantization parameter into the decision-making process.

[0158] Furthermore, the synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients for blocks to be decoded not only according to the control information of the blocks to be decoded, but also according to the control information of blocks adjacent to the blocks to be decoded.

[0159] For example, since small regions tend to be continuous across multiple blocks, the synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients for the blocks to be decoded, depending on the weight coefficients of the adjacent decoded blocks.

[0160] Figure 10 shows an example of blocks adjacent to the block to be decrypted: to the left, upper left, top, and upper right.

[0161] Although partition boundaries also exist in the blocks to the left and upper left of the block to be decoded, the blending unit 207 is not continuous with the partition boundary of the block to be decoded, so it is not included in the combination of weight coefficients for the block to be decoded. Instead, the width of the partition boundary of the block above the block to be decoded, where the partition boundary is continuous, can be included in the combination of weight coefficients for the block to be decoded.

[0162] Furthermore, when the synthesis unit 207 limits the combinations of weight coefficients of selectable decryption target blocks, it may be configured to limit them stepwise, rather than being limited to a simple choice of whether or not to include a block in such a combination.

[0163] For example, the decoding unit 201 improves coding efficiency by assigning different code lengths according to the selection probability of the weight coefficients mentioned above and decoding accordingly.

[0164] In the example described above, the decoding unit 201 can set the weight coefficient pattern used by adjacent decoded blocks as a short code length and other patterns as a long code length.

[0165] The image decoding device 200 described above may be implemented as a program that causes a computer to execute each function (each process). [Industrial applicability]

[0166] Furthermore, according to this embodiment, for example, it is possible to achieve an overall improvement in service quality in video communication, thereby contributing to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of Symbols]

[0167] 200…Image decoding device 201...Decoding section 202...Inverse quantization section 203...Inverse Transformation Section 204...Intra Prediction Unit 205...Storage section 206...Motion compensation unit 207...Synthesis section 208... Addition section 210... Code input section 220...Image output unit

Claims

1. An image decoding device to which a geometric division mode is applied, which divides a rectangular unit block to be decoded into a first sub-region and a second sub-region, When the aforementioned geometric mode is applied, In the first subregion, predictive pixels are generated by intraprediction. In the second sub-region, predictive pixels are generated by intraprediction. The number of selectable combinations of weight coefficients is limited based on the shape of the block to be decoded. A unique weight coefficient is selected from a limited set of patterns of combinations of the aforementioned weight coefficients. An image decoding device characterized by applying a weighted average using the weight coefficients uniquely selected to the blocks to be decoded.

2. An image decoding method to which a geometric division mode can be applied, which divides a rectangular unit block to be decoded into a first sub-region and a second sub-region, When the aforementioned geometric mode is applied, In the first subregion, predictive pixels are generated by intraprediction. In the second sub-region, predictive pixels are generated by intraprediction. The number of selectable combinations of weight coefficients is limited based on the shape of the block to be decoded. A unique weight coefficient is selected from a limited set of patterns of combinations of the aforementioned weight coefficients. An image decoding method characterized by applying a weighted average using the weight coefficients uniquely selected to the blocks to be decoded.

3. A program that causes a computer to function as an image decoding device to which a rectangular unit block to be decoded is applicable in a geometric division mode that divides the decoded block into a first sub-region and a second sub-region, When the geometric mode is applied, the image decoding device, In the first subregion, predictive pixels are generated by intraprediction. In the second sub-region, predictive pixels are generated by intraprediction. The number of selectable combinations of weight coefficients is limited based on the shape of the block to be decoded. A unique weight coefficient is selected from a limited set of patterns of combinations of the aforementioned weight coefficients. A program characterized by applying a weighted average to the decryption target block using the weight coefficients uniquely selected.

Images