Image decoding device, image decoding method and program

Abstract

To improve encoding efficiency in a GPM.SOLUTION: An image decoding device 200 comprises a decoding part 201 which controls decoding of syntax controlling a border width of a geometric block division mode of a sequence to be decoded, and the decoding part 201 can decode the syntax controlling whether the border width can be varied, and change the pattern or the number of the border width of the geometric block division mode at a header of each block to be decoded.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image decoding device, an image decoding method, and a program. [Background technology]

[0002] Non-Patent Documents 1 and 2 disclose a geometric partitioning mode (GPM).

[0003] GPM divides a rectangular block diagonally into two parts and motion compensates each part. Specifically, the two divided parts are motion compensated using merge vectors and then combined using weighted averaging. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] ITU-T H.266 / VVC [Non-patent document 2] CE4:Summary report on Inter prediction with geometric partitioning, JVET-Q0024 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the techniques disclosed in Non-Patent Document 1 and Non-Patent Document 2 have a problem in that the weighted average patterns are limited, and there is room for improvement in encoding performance. Therefore, the present invention has been made in consideration of the above-mentioned problem, and an object of the present invention is to provide an image decoding device, an image decoding method, and a program that can improve encoding efficiency in GPM. [Means for solving the problem]

[0006] A first feature of the present invention is an image decoding device including a decoding unit that controls the decoding of syntax that controls the boundary width of a geometric block partitioning mode of a sequence to be decoded, wherein the decoding unit decodes syntax that controls whether the boundary width can be changed, and is capable of changing the pattern or number of the boundary width of the geometric block partitioning mode in the header of each block to be decoded.

[0007] A second feature of the present invention is an image decoding method including a step of controlling the decoding of syntax that controls the boundary width of a geometric block partitioning mode of a sequence to be decoded, wherein in the step, syntax that controls whether or not the boundary width can be changed is decoded, and the pattern or number of the boundary width of the geometric block partitioning mode can be changed in the header of each block to be decoded.

[0008] A third feature of the present invention is a program that causes a computer to function as an image decoding device, the image decoding device including a decoding unit that controls the decoding of syntax that controls the boundary width of a geometric block partitioning mode of a sequence to be decoded, the decoding unit decodes syntax that controls whether the boundary width can be changed, and is capable of changing the pattern or number of the boundary width of the geometric block partitioning mode in a header for each block to be decoded. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program that can improve the coding efficiency in GPM. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of functional blocks of an image decoding device 200 according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of a case where a rectangular unit block is divided into two small regions A and B by a dividing boundary. [Figure 3]FIG. 3 is a diagram showing an example of three patterns of weighting coefficients assigned to the division boundaries of the small region B shown in FIG. [Figure 4] FIG. 4 is a diagram showing an example in which the weighting factor w of pattern (2) is applied to an 8×8 block. [Figure 5] FIG. 5 is a diagram showing an example in which the weighting factor w of pattern (1) is applied to an 8×8 block. [Figure 6] FIG. 6 is a diagram showing an example in which the weighting factor w of pattern (3) is applied to an 8×8 block. [Figure 7] FIG. 7 is a flowchart showing an example of a method for decoding control information by decoding section 201. [Figure 8] FIG. 8 is a flowchart illustrating an example of a process for setting weighting factors by the synthesis unit 207 in the first embodiment. [Figure 9] FIG. 9 is a flowchart illustrating an example of a process for setting weighting factors by the synthesis unit 207 in the second embodiment. [Figure 10] FIG. 10 is a diagram for explaining the second embodiment. [Figure 11] FIG. 11 is a diagram for explaining the second embodiment. [Figure 12] FIG. 12 is a flowchart illustrating an example of a process for setting a weighting factor by the synthesis unit 207 in the third embodiment. [Figure 13] FIG. 13 is a diagram for explaining an example in which the weighting coefficient is defined based on the distance from the division boundary. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0012] First Embodiment An image decoding device 200 according to this embodiment will be described below with reference to Figures 1 to 8. Figure 1 is a diagram showing an example of functional blocks of the image decoding device 200 according to this embodiment.

[0013] As shown in FIG. 1, the image decoding device 200 includes a code input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transform unit 203, an intra prediction unit 204, an accumulation unit 205, a motion compensation unit 206, a synthesis unit 207, an addition unit 208, and an image output unit 220.

[0014] The code input unit 210 is configured to obtain code information coded by an image coding device.

[0015] The decoding unit 201 is configured to control the decoding of syntax that controls the boundary width of the division mode of the sequence to be decoded.

[0016] Here, the syntax for controlling the boundary width of the division mode of the sequence to be decoded indicates sps_div_blending_flag, sps_div_num_blending_list, sps_div_blending_list, pps_div_blending_flag, pps_div_num_blending_list, pps_div_blending_list, sh_div_blending_flag, sh_div_num_blending_list, sh_div_blending_list, etc., which will be described later.

[0017] Specifically, the decoding unit 201 is configured to decode the control information and the quantized value from the code information input from the code input unit 210. For example, the decoding unit 201 is configured to output the control information and the quantized value by performing variable length decoding on the code information.

[0018] Here, the quantized value is sent to the inverse quantization unit 202, and the control information is sent to the motion compensation unit 206, the intra prediction unit 204, and the synthesis unit 207. Note that this control information includes information necessary for controlling the motion compensation unit 206, the intra prediction unit 204, the synthesis unit 207, etc., and may also include header information such as a sequence parameter set, a picture parameter set, a picture header, or a slice header.

[0019] The inverse quantization unit 202 is configured to inverse quantize the quantized values ​​sent from the decoding unit 201 to generate decoded transform coefficients. The transform coefficients are sent to the inverse transform unit 203.

[0020] The inverse transform unit 203 is configured to inverse transform the transform coefficients sent from the inverse quantization unit 202 to generate decoded prediction residuals. The prediction residuals are sent to the adder 208.

[0021] The intra prediction unit 204 is configured to generate a first predicted pixel based on the decoded pixel and the control information sent from the decoding unit 201. Here, the decoded pixel is obtained via the adder 208 and stored in the storage unit 205. The first predicted pixel is a predicted pixel as an approximation of an input pixel in a small region set by the synthesis unit 207. The first predicted pixel is sent to the synthesis unit 207.

[0022] The storage unit 205 is configured to cumulatively store the decoded pixels sent from the addition unit 208. The motion compensation unit 206 references these decoded pixels via the storage unit 205.

[0023] The motion compensation unit 206 is configured to generate second predicted pixels based on the decoded pixels stored in the storage unit 205 and the control information sent from the decoding unit 201. Here, the second predicted pixels are predicted pixels as approximations of input pixels in a small region set by the synthesis unit 207. The second predicted pixels are sent to the synthesis unit 207.

[0024] The adder 208 is configured to add the prediction residual sent from the inverse transformer 203 and the third predicted pixel sent from the synthesizer 207 to obtain a decoded pixel. The decoded pixel is sent to the image output unit 220, the storage unit 205, and the intra prediction unit 204.

[0025] The synthesis unit 207 is configured to prepare multiple weighting coefficients with different widths of the division boundary for at least one of the first predicted pixel sent from the intra prediction unit 204 and the second predicted pixel sent from the motion compensation unit 206, and to generate a third predicted pixel in which the width of the division boundary is controlled by weighted averaging.

[0026] The role of the synthesis unit 207 is to select weighting factors for a plurality of prediction pixels that are optimal for the block to be decoded, and to synthesize the input prediction pixels in accordance with the weighting factors, in order to compensate the block to be decoded with high precision in the subsequent addition unit 208.

[0027] Any partitioning mode can be used in which the block to be decoded is divided into multiple small regions. In the following, we will explain the case where the geometric block partitioning mode (GPM) disclosed in Non-Patent Document 1 and Non-Patent Document 2 is used as an example of a partitioning mode.

[0028] For the weighting coefficients, a plurality of patterns in which arbitrary values ​​are set in advance for each pixel of the unit block are prepared, and one of the patterns is applied. In other words, the synthesis unit 207 may be configured to select and apply one of the plurality of weighting coefficients.

[0029] The sum of weighting factors for a plurality of predicted pixels is designed to be 1 for each pixel, and the result of combining a plurality of predicted pixels by weighted averaging using the weighting factors is used as the predicted pixel by the combining unit 207.

[0030] A pixel with a weighting factor of 1 (i.e., the maximum value) adopts the input predicted pixel, and a pixel with a weighting factor of 0 (i.e., the minimum value) does not use the input predicted pixel. Therefore, conceptually, this is equivalent to dividing a unit block into multiple small regions, and determining which pixel of multiple input predicted pixels to apply to where and in what proportion.

[0031] Here, it is desirable to distribute the weighting coefficients in a non-rectangular shape, since a rectangular distribution, such as a bisecting distribution, can be expressed in smaller unit blocks.

[0032] The example in Fig. 2 shows a case where unit blocks are distributed in a diagonal shape. In the example in Fig. 2, a rectangular unit block is divided into two small areas A and B by a dividing boundary.

[0033] In each of the small regions A / B, predicted pixels are generated by any method such as intra prediction or motion compensation.

[0034] In this case, even if the shape of the division is determined, if the weighting coefficients near the division boundaries are fixed, the diversity of the division boundaries cannot be expressed, and therefore there is a problem that the coding efficiency cannot be improved.

[0035] For example, if a small region is an area with a lot of movement, blurring occurs during imaging, so it is desirable to blur a plurality of small regions over a wide area and perform weighted averaging on the division boundary.

[0036] Conversely, if the small region is an artificially edited region such as a subtitle, blurring will not occur, so it is preferable to limit the division boundary to a narrow region and perform a weighted average so that multiple small regions are simply adjacent to each other.

[0037] To solve this problem, in this embodiment, a procedure is taken in which a plurality of weighting factors with different widths of the dividing boundaries of the small regions are prepared and selected.

[0038] Fig. 3 shows an example of three patterns of weighting coefficients assigned to the division boundary of small area B shown in Fig. 2. In Fig. 3, the horizontal axis represents the distance in pixels from the position of the division boundary, and the vertical axis represents the weighting coefficient.

[0039] Specifically, we have prepared pattern (1) in which a weighting factor of [0,1] is assigned to the range [a,b] for pixel-unit distances a and b from a preset division boundary position; pattern (2) in which distances a and b are similarly doubled and a weighting factor of [0,1] is assigned to the range [2a,2b]; and pattern (3) in which distances a and b are similarly halved and a weighting factor of [0,1] is assigned to the range [a / 2,b / 2].

[0040] These weighting coefficients are determined by the distance d(x c ,y c ) is uniquely determined by xc,yc When this is defined as follows, it is equivalent to preparing multiple patterns instead of a limited pattern (fixed value) for the range τ in which the weight coefficient in FIG. 13 is other than the minimum or maximum value. c ,y c are coordinates within the block to be decoded.

[0041] That is, the synthesis unit 207 may be configured to set a plurality of weighting factors according to the inter-pixel distance from the division boundary.

[0042] Note that weighting coefficients may be set symmetrically with respect to the division boundary, with a=b. That is, the synthesis unit 207 may be configured to set weighting coefficients symmetrically with respect to the division boundary as the weighting coefficients described above.

[0043] Alternatively, weighting coefficients may be set that are asymmetric with respect to the division boundary, with a≠b. That is, the synthesis unit 207 may be configured to set weighting coefficients that are asymmetric with respect to the division boundary as the weighting coefficients described above.

[0044] Furthermore, the weighting coefficients can be set not only by two, a and b, but also by increasing the number of line segments, etc. In other words, the synthesis unit 207 may be configured to set weighting coefficients for multiple line segments according to the inter-pixel distance from the division boundary.

[0045] 4 to 6 show examples in which each weighting factor w is applied to an 8×8 block. The weighting factor w in FIGS. 4 to 6 takes values ​​from 0 to 8, and is synthesized using the following equation.

[0046] (w×small area A+(8-w)×small area B+4)>>3 In this way, by setting multiple weighting coefficients w according to the pixel-to-pixel distance from the division boundary, it is possible to obtain the effect of uniformly deriving a weighted average for various block sizes, such as 8x8 or 64x16. The type, shape, and number of patterns can be set arbitrarily. For example, while the above description has been given of multiple patterns of 2 and 1 / 2 times the distances a and b, they may also be 4 or 1 / 4 times. Furthermore, in the above formula, the weighting coefficients are set to values ​​between 0 and 8, but they can also be set to other values, such as 0 to 16 or 0 to 32. In particular, when the distance is 2 or 4 times, increasing the maximum value of the weighting coefficient can improve the accuracy of the weighted average for each pixel.

[0047] An example of the control information decoding process performed by decoding section 201 will be described below.

[0048] The coded information input to the decoding unit 201 may include a sequence parameter set (SPS) that is a collection of control information for each sequence.

[0049] The coded information input to the decoding unit 201 may also include a picture parameter set (PPS) or a picture header (PH) that summarizes control information for each picture.

[0050] Furthermore, the coded information input to the decoding unit 201 may include a slice header (SH) that summarizes control information for each slice.

[0051] Here, a method in which the decoding unit 201 sets the boundary width in units of sequences will be described with reference to FIG.

[0052] 7, in step S11, decoding unit 201 determines whether sps_div_enabled_flag in the sequence parameter set is 1. If yes, the process proceeds to step S12, and if no, the process ends.

[0053] Here, sps_div_enabled_flag is a syntax that controls whether or not GPM is present. When sps_div_enabled_flag is 1, it indicates that GPM is enabled, and when sps_div_enabled_flag is 0, it indicates that GPM is disabled.

[0054] In step S12, the decoding unit 201 decodes sps_div_blending_flag.

[0055] Here, sps_div_blending_flag is a syntax that controls whether or not the boundary width is changed. When sps_div_blending_flag is 1, it indicates that the boundary width change is enabled, and when sps_div_blending_flag is 0, it indicates that the boundary width change is disabled.

[0056] Next, the decoding unit 201 determines whether or not sps_div_blending_flag is 1. If YES, the process proceeds to step S13, and if NO, the process ends.

[0057] In step S13, the decoding unit 201 decodes sps_div_num_blending_list.

[0058] Here, sps_div_num_blending_list is a syntax that controls the number of patterns of the border width.

[0059] In step S14, the decoding unit 201 decodes sps_div_blending_list representing the boundary width patterns as many times as the number set in sps_div_num_blending_list.

[0060] In the example of FIG. 3, sps_div_num_blending_list is set to 3, and the weighting factors corresponding to patterns (1) to (3) of sps_div_blending_list are used as boundary width options.

[0061] In this way, by using sps_div_num_blending_list and sps_div_blending_list, the number of boundary width patterns and the pattern type can be changed in sequence units according to the image characteristics, which is expected to maximize the coding efficiency.

[0062] For example, for a sequence consisting of natural images captured by a camera, multiple patterns (1) to (3) can be set as shown in the example of Figure 3. For a sequence consisting of screen images such as a computer screen or game screen, for which a pattern with a small distribution of weighting coefficients is effective, one type of pattern with a short boundary width can be set, thereby maximizing coding efficiency.

[0063] When setting the boundary width on a picture-by-picture basis, the decoding unit 201 decodes pps_div_enabled_flag instead of sps_div_enabled_flag, decodes pps_div_blending_flag instead of sps_div_blending_flag, decodes pps_div_num_blending_list instead of sps_div_num_blending_list, and decodes pps_div_blending_list instead of sps_div_blending_list in pps or ph.

[0064] In this way, by using pps_div_num_blending_list and pps_div_blending_list, the number of boundary width patterns and the pattern type can be changed in units of pictures according to the image characteristics, which is expected to maximize the coding efficiency.

[0065] For example, for pictures consisting of natural images captured by a camera, multiple patterns (1) to (3) can be set as shown in the example of Figure 3, and for pictures consisting of screen images such as computer screens or game screens, for which a pattern with a small distribution of weighting coefficients is effective, one type of pattern with a short boundary width can be set, thereby maximizing coding efficiency.

[0066] When setting the boundary width in slice units, the decoding unit 201 decodes, in sh, sh_div_enabled_flag instead of sps_div_enabled_flag, sh_div_blending_flag instead of sps_div_blending_flag, sh_div_num_blending_list instead of sps_div_num_blending_list, and sh_div_blending_list instead of sps_div_blending_list.

[0067] In this way, by using sh_div_num_blending_list and sh_div_blending_list, the number of boundary width patterns and the pattern type can be changed in slice units according to the image characteristics, which is expected to maximize the coding efficiency.

[0068] For example, for a slice area consisting of a natural image captured by a camera, multiple patterns (1) to (3) can be set as shown in the example of Figure 3, and for a slice area consisting of a screen image such as a computer screen or game screen, where a pattern with a small distribution of weighting coefficients is effective, one type of pattern with a short boundary width can be set, thereby maximizing coding efficiency.

[0069] By setting only the upper layer, it is possible to suppress an increase in the amount of coding, and by setting the lower layer as well and then prioritizing the setting in the lower layer, adaptive control is possible.

[0070] Alternatively, if the pattern of the width of the division boundary is set in advance, the decoding unit 201 can omit decoding the pattern of the division boundary width itself.

[0071] In the above example, the number of weighting coefficient patterns and the pattern type are set in units of sequences, pictures, or slices, but it is also possible to directly select a pattern in units of blocks, as will be described later, without setting these. In this case, the degree of freedom in setting the number of weighting coefficient patterns and the pattern type is reduced, but the increase in the header information described above can be avoided.

[0072] An example of the weighting factor setting process performed by the synthesis unit 207 will be described below with reference to FIG.

[0073] 8, in step S101, the synthesis unit 207 determines whether any of sps_div_enabled_flag, pps_div_enabled_flag, and sh_div_enabled_flag included in the above-mentioned control information is 1. If the result is No (if none of them is 1), the process proceeds to step S102, and if the result is Yes, the process proceeds to step S103.

[0074] In step S102, the synthesis unit 207 does not apply weighted averaging using a weighting factor to the current block to be decoded.

[0075] In step S103, the synthesis unit 207 determines whether GPM is applied to the block to be decoded. If the result is No, the process proceeds to step S102, and if the result is Yes, the process proceeds to step S104.

[0076] In step S104, the synthesis unit 207 decodes cu_div_blending_idx, which is included in the above-mentioned control information and specifies the width of the division boundary.

[0077] If cu_div_blending_idx is 0, the operation proceeds to step S105; if cu_div_blending_idx is 1, the operation proceeds to step S106; if cu_div_blending_idx is 2, the operation proceeds to step S107.

[0078] In step S105, the synthesis unit 207 selects and applies the weighting factor of pattern (1) from among patterns (1) to (3).

[0079] In step S106, the synthesis unit 207 selects and applies the weighting factor of pattern (2) from among patterns (1) to (3).

[0080] In step S107, the synthesis unit 207 selects and applies the weighting factor of pattern (3) from among patterns (1) to (3).

[0081] Figure 8 shows an example in which the synthesis unit 207 decodes one cu_div_blending_idx when the pattern of the division boundary width is symmetric (a = b), but the synthesis unit 207 may also decode two, cu_div_blending_idx0 and cu_div_blending_idx1, when the pattern of the division boundary width is asymmetric (a ≠ b).

[0082] Here, cu_div_blending_idx is decoded to identify one of the div_blending_lists in the lowest layer that is applied to the current block.

[0083] Therefore, when div_num_blending_list in the lowest layer applied to the block to be decoded is 1, decoding of cu_div_blending_idx can be omitted, and coding efficiency can be improved by estimating cu_div_blending_idx to be 0.

[0084] If the div_num_blending_list in the lowest layer applied to the block to be decoded is greater than 1, cu_div_blending_idx may be decoded as a fixed length, or cu_div_blending_idx may be decoded as a variable length. When cu_div_blending_idx is decoded as a variable length, the value of cu_div_blending_idx may be assigned in order of the highest selection rate.

[0085] Note that, when the chrominance components of the block to be decoded are not downsampled relative to the luminance component of the block to be decoded, the synthesis unit 207 may derive weighting coefficients that determine the width of the division boundaries of the chrominance components of the block to be decoded without using control information for deriving weighting coefficients that determine the width of the division boundaries of the luminance component of the block to be decoded (i.e., by separately using control information for deriving weighting coefficients that determine the width of the division boundaries of the chrominance components of the block to be decoded).With this configuration, weighting coefficients for the chrominance components of the block to be decoded can be derived independently, which is expected to improve coding performance.

[0086] On the other hand, when the chrominance component of the block to be decoded is downsampled relative to the luminance component of the block to be decoded, the synthesis unit 207 may use the control information for deriving a weighting factor that determines the width of the division boundary of the luminance component of the block to be decoded (i.e., without using separate control information for deriving a weighting factor that determines the width of the division boundary of the chrominance component of the block to be decoded) to derive a weighting factor that determines the width of the division boundary of the chrominance component of the block to be decoded.

[0087] This configuration achieves the same effect on the downsampled chrominance components as on the luminance components. Furthermore, when control information such as a header is used to determine the width of the division boundary for the luminance components of the block to be decoded, this control information is not required for the chrominance components, which is expected to improve coding performance.

[0088] Second Embodiment Hereinafter, the second embodiment of the present invention will be described with reference to FIGS. 3 and 9 to 11, focusing on the differences from the first embodiment described above.

[0089] In this embodiment, the code length is reduced by specifying a weighting coefficient pattern without requiring direct control information.

[0090] Therefore, in this embodiment, the synthesis unit 207 is configured to generate a third predicted pixel by weighted averaging of at least one of the first predicted pixel and the second predicted pixel described above using a weighting factor that is uniquely selected from a plurality of weighting factors based on indirect control information.

[0091] That is, in this embodiment, the synthesis unit 207 is configured to select (uniquely identify) a weighting factor from among a plurality of weighting factors in accordance with indirect control information.

[0092] Here, the synthesis unit 207 may be configured to prepare a plurality of weighting factors with different widths of the division boundaries of the small regions, and to select a weighting factor from among the plurality of weighting factors.

[0093] Specifically, the synthesis unit 207 may be configured to select a weighting factor from among a plurality of weighting factors according to the shape of the block to be decoded, which is indirect control information.

[0094] For example, the synthesis unit 207 may be configured to select a weighting factor from among a plurality of weighting factors depending on at least one of the short side of the block to be decoded, the long side of the block to be decoded, the aspect ratio of the block to be decoded, the partitioning mode, and the number of pixels of the block to be decoded.

[0095] For example, when using the short side of the block to be decoded as the shape of the block to be decoded, if the short side of the block to be decoded is small, performing a weighted average over a wide area will result in the same result as simple bidirectional prediction, so it is desirable to exclude weight coefficients for patterns with wide division boundaries from the options.

[0096] For example, in the example of Figure 3, the synthesis unit 207 selects the weighting factor of pattern (3) when the short side of the block to be decoded is less than or equal to the threshold, and selects the weighting factor of pattern (2) when the short side of the block to be decoded is greater than the threshold, thereby increasing the number of patterns while eliminating the need for pattern control information and achieving the effect of improving coding efficiency.

[0097] An example of the process of setting the weighting coefficients by the synthesis unit 207 will be described below with reference to FIG.

[0098] 9, in step S201, the synthesis unit 207 determines whether any of sps_div_enabled_flag, pps_div_enabled_flag, and sh_div_enabled_flag included in the above-mentioned control information is 1. If the result is No (if none of them is 1), the process proceeds to step S202, and if the result is Yes, the process proceeds to step S203.

[0099] In step S202, the synthesis unit 207 does not apply weighted averaging using a weighting factor to the current block to be decoded.

[0100] In step S203, the synthesis unit 207 determines whether GPM has been applied to the block to be decoded. If the result is No, the process proceeds to step S202, and if the result is Yes, the process proceeds to step S204.

[0101] In step S204, the synthesis unit 207 determines whether the shorter side of the block to be decoded is equal to or less than a preset threshold value 1. If the result is No, the process proceeds to step S205, and if the result is Yes, the process proceeds to step S208.

[0102] In step S205, the synthesis unit 207 determines whether the shorter side of the block to be decoded is equal to or less than a preset threshold value 2. Here, threshold value 2 is greater than threshold value 1. If the result is No, the process proceeds to step S206, and if the result is Yes, the process proceeds to step S207.

[0103] In step S206, the synthesis unit 207 selects and applies the weighting factor of pattern (2) from among patterns (1) to (3).

[0104] In step S207, the synthesis unit 207 selects and applies the weighting factor of pattern (1) from among patterns (1) to (3).

[0105] In step S208, the synthesis unit 207 selects and applies the weighting factor of pattern (3) from among patterns (1) to (3).

[0106] Similarly, when the shape of the block to be decoded is determined by using the long side of the block to be decoded, the aspect ratio of the block to be decoded, the partition mode of the block to be decoded, the number of pixels of the block to be decoded, etc., performing weighted averaging over a wide area results in the same result as simple bi-prediction, so it is desirable to exclude weight coefficients for patterns with wide partition boundaries from the options.

[0107] That is, in steps S204 and S205 of the flowchart shown in FIG. 9, the short side of the block to be decoded may be replaced with the long side of the block to be decoded, the aspect ratio of the block to be decoded, the division mode of the block to be decoded, or the number of pixels of the block to be decoded.

[0108] Also, in the flowchart shown in Figure 9, in step S204, the synthesis unit 207 may determine whether the short side of the block to be decoded is less than a predetermined threshold value 1, and in step S205, the synthesis unit 207 may determine whether the short side of the block to be decoded is less than a predetermined threshold value 2.

[0109] As a modification of the above, the shape of the block to be decoded may be determined by using the short side of the block to be decoded, the aspect ratio of the block, or the division mode (angle of the division boundary).

[0110] For example, if the short side of the block to be decoded is small, the aspect ratio of the block is large (e.g., length:width = 4:1), and the angle of the division boundary is 45 degrees or more, weighting coefficients for patterns with wide division boundaries may be excluded from the options.

[0111] Conversely, if the short side of the block to be decoded is small, the aspect ratio of the block is large (e.g., length:width = 4:1), and the angle of the division boundary is less than 45 degrees, weighting coefficients with patterns in which the width of the division boundary is narrow may be excluded from the options.

[0112] This makes it possible to select the width of the division boundary taking into account the shape of the block, which is expected to improve coding performance.

[0113] The synthesizing unit 207 may also be configured to select the weighting factor in accordance with the motion vector.

[0114] Specifically, the synthesizing unit 207 may be configured to use the motion vectors of the small regions and select the weighting coefficients described above in accordance with the length of the motion vectors of the small regions or the resolution of the motion vectors of the small regions.

[0115] The larger the motion vector, the more blurred the boundary between the divisions, so it is desirable to widen the distribution of the weighting factors.Similarly, the coarser the resolution of the motion vector, the more blurred the boundary between the divisions, so it is desirable to widen the distribution of the weighting factors.

[0116] Furthermore, the synthesizing unit 207 may be configured to select the weighting factor described above in accordance with the difference between the motion vectors of the small area A and the small area B.

[0117] Here, the difference in motion vector is the difference in the reference frames of the motion vectors of small area A and small area B, or the difference in the motion vectors themselves.

[0118] For example, the synthesis unit 207 may be configured to select the above-mentioned weighting factor so as to narrow the distribution of the weighting factor if the difference between the motion vectors of the small area A and the small area B is equal to or greater than a predetermined threshold (e.g., 1 pixel), and to select the above-mentioned weighting factor so as to widen the distribution of the weighting factor if the difference between the motion vectors of the small area A and the small area B is less than a predetermined threshold (e.g., 1 pixel).

[0119] According to this configuration, highly accurate prediction can be made in accordance with image edges that may occur near division boundaries (such as boundaries between background and foreground that have different movements).

[0120] Alternatively, the synthesis unit 207 may be configured to select the above-mentioned weighting factor so as to widen the distribution of weighting factors if the difference between the motion vectors of small area A and small area B is equal to or greater than a predetermined threshold (e.g., 1 pixel), and to select the above-mentioned weighting factor so as to narrow the distribution of weighting factors if the difference between the motion vectors of small area A and small area B is less than a predetermined threshold (e.g., 1 pixel).

[0121] According to this configuration, it is possible to make a highly accurate prediction in accordance with the magnitude of motion blur near the division boundary.

[0122] Here, the synthesis unit 207 may be configured to select a weighting factor that can be selected depending on the relationship between the angle between the motion vector and the division boundary.

[0123] For example, as shown in FIG. 10, the synthesis unit 207 may be configured to select the above-mentioned weighting coefficient according to the absolute value |x×u+y×v| of the dot product of the motion vector (x, y) and the unit normal vector (u, v) of the division boundary.

[0124] Alternatively, the combiner 207 may be configured to select a selectable weighting factor depending on the exposure time or the frame rate.

[0125] Furthermore, the synthesis unit 207 may be configured to select a weighting factor that can be selected depending on the prediction method for the small region.

[0126] Furthermore, the synthesis unit 207 may be configured to select a selectable weighting factor depending on the quantization parameter.

[0127] Furthermore, the synthesis unit 207 may be configured to select a weighting factor for the block to be decoded in accordance with control information of blocks neighboring the block to be decoded, rather than the control information of the block to be decoded.

[0128] For example, since small regions tend to be continuous across multiple blocks, the synthesis unit 207 may be configured to select a weighting factor for the block to be decoded depending on the weighting factors of adjacent already-decoded blocks.

[0129] FIG. 11 is a diagram showing an example of blocks adjacent to the left, upper left, upper, and upper right of the current block to be decoded.

[0130] Although partition boundaries also exist in the blocks to the left and top left adjacent to the block to be decoded, the synthesis unit 207 does not select these because they are not contiguous with the partition boundary of the block to be decoded, and can instead select the width of the partition boundary of the block above the block to be decoded, whose partition boundary is contiguous, for the block to be decoded.

[0131] Similarly, the synthesis unit 207 may be configured to derive a pattern of weighting factors of blocks adjacent to the block to be decoded as an internal parameter corresponding to the merge index used when decoding the merge vector of each small region, and select the pattern as the weighting factor of each small region of the block to be decoded.

[0132] The synthesis unit 207 may be configured to adopt the width of the division boundary of a predetermined pattern (e.g., pattern (1)) for the small region of the block to be decoded when there is no merge vector corresponding to each small region.

[0133] Here, the synthesis unit 207 may be configured to adopt the width of the division boundary of a predetermined pattern (e.g., pattern (1)) for the small regions of the block to be decoded when each small region is in intra prediction mode.

[0134] Third Embodiment Hereinafter, the third embodiment of the present invention will be described with reference to FIGS. 3 and 8 to 12, focusing on the differences from the first and second embodiments described above.

[0135] In this embodiment, the synthesis unit 207 is configured to generate a third predicted pixel by weighted averaging of at least one of the first predicted pixel and the second predicted pixel using any of the weighting factors limited based on the decoded control information.

[0136] That is, the synthesis unit 207 is configured to limit the combinations of weighting factors that can be selected according to the indirect control information, and then select a weighting factor to be applied based on the decoded control information from the limited combinations of weighting factors.

[0137] Here, the synthesis unit 207 may be configured to prepare a plurality of weighting factors with different widths of the dividing boundaries of the small regions and select the weighting factor.

[0138] The synthesis unit 207 may be configured to limit the combinations of weighting factors that can be selected according to the shape of the block to be decoded, which serves as indirect control information.

[0139] For example, the synthesis unit 207 may be configured to limit the selectable weighting factors depending on at least one of the size of the block to be decoded (such as the short side of the block to be decoded or the long side of the block to be decoded), the aspect ratio of the block to be decoded, the division mode of the block to be decoded, and the number of pixels of the block to be decoded.

[0140] Here, when using the short side of the block to be decoded as the shape of the block to be decoded, if the short side of the block to be decoded is small, performing weighted averaging over a wide area will result in a result that is no different from simple bidirectional prediction, so it is desirable to exclude weighting elements with patterns in which the width of the division boundary is wide from the options (combinations of selectable weighting factors).

[0141] For example, in the example of Figure 3, when the short side of the block to be decoded is equal to or smaller than a threshold, the synthesis unit 207 limits the selectable combination of weighting factors to the weighting factors of pattern (1) / (3), and when the short side of the block to be decoded is greater than the threshold, it limits the selectable combination of weighting factors to the weighting factors of pattern (1) / (2).This increases the number of patterns while reducing the amount of coding for the control information of the patterns, thereby achieving the effect of improving coding efficiency.

[0142] Here, the threshold value for the short side of the block to be decoded may be set to, for example, 8 pixels or 16 pixels.

[0143] An example of the weighting factor setting process performed by the synthesis unit 207 will be described below with reference to FIG.

[0144] 12, in step S301, the synthesis unit 207 determines whether any of sps_div_enabled_flag, pps_div_enabled_flag, and sh_div_enabled_flag included in the above-mentioned control information is 1. If the result is No (if none of them is 1), the process proceeds to step S302, and if the result is Yes, the process proceeds to step S303.

[0145] In step S302, the synthesizer 207 does not apply weighted averaging using a weighting factor to the current block to be decoded.

[0146] In step S303, the synthesis unit 207 determines whether GPM has been applied to the block to be decoded. If the result is No, the process proceeds to step S302, and if the result is Yes, the process proceeds to step S304.

[0147] In step S304, the synthesizing unit 207 determines whether the shorter side of the block to be decoded is equal to or less than a preset threshold.

[0148] If the answer is No, the process proceeds to step S305, and if the answer is Yes, the process proceeds to step S306. Here, if the answer is No, the synthesis unit 207 limits the selectable weighting factor combinations to pattern (1) / (2), and if the answer is Yes, the synthesis unit 207 limits the selectable weighting factor combinations to pattern (1) / (3).

[0149] In step S305, the synthesis unit 207 decodes cu_div_blending_idx (direct control information) included in the above-mentioned control information.

[0150] If cu_div_blending_idx is not 0, the operation proceeds to step S307, and if cu_div_blending_idx is 0, the operation proceeds to step S308.

[0151] Similarly, if cu_div_blending_idx is not 0, the operation proceeds to step S309, and if cu_div_blending_idx is 0, the operation proceeds to step S310.

[0152] In step S307, the synthesis unit 207 selects and applies the weighting factor of pattern (1) from pattern (1) / (2) as the weighting factor.

[0153] In step S308, the synthesis unit 207 selects and applies the weighting factor of pattern (2) from pattern (1) / (2) as the weighting factor.

[0154] In step S309, the synthesis unit 207 selects and applies the weighting factor of pattern (1) from patterns (1) / (3) as the weighting factor.

[0155] In step S310, the synthesis unit 207 selects and applies the weighting factor of pattern (3) from among patterns (1) / (3).

[0156] As described above, when the short side of the block to be decoded as indirect control information is equal to or smaller than a threshold value, the synthesis unit 207 limits the selectable combination of weighting factors among the weighting factors of patterns (1) to (3) to the weighting factors of pattern (1) / (3), and selects a weighting factor of one pattern from the weighting factors of the limited patterns by making a judgment on cu_div_blending_idx as direct control information.

[0157] On the other hand, when the shorter side of the block to be decoded is larger than the threshold value, the synthesis unit 207 limits the selectable combination of weighting factors from the weighting factors of patterns (1) to (3) to the weighting factors of pattern (1) / (2), and selects a weighting factor of one pattern from the weighting factors of the limited patterns by making a judgment on cu_div_blending_idx as direct control information.

[0158] That is, in the example where three types of weighting factors for patterns (1) to (3) are stored in the sps_div_blending_list, the synthesis unit 207 can limit the combinations of weighting factors for patterns selectable by cu_div_blending_idx to two types using indirect control information.

[0159] Then, when the shorter side of the block to be decoded is equal to or less than the threshold, if cu_div_blending_idx is 0, the synthesis unit 207 selects the weighting factor of pattern (2), and if cu_div_blending_idx is 1, selects the weighting factor of pattern (1).

[0160] On the other hand, if the shorter side of the block to be decoded is greater than the threshold, the synthesis unit 207 selects the weighting factor of pattern (3) if cu_div_blending_idx is 0, and selects the weighting factor of pattern (1) if cu_div_blending_idx is 1.

[0161] According to the above configuration, even if the same cu_div_blending_idx is 0, the pattern of weighting factors to be selected is controlled in accordance with other control signals, thereby achieving the effect of improving coding efficiency.

[0162] Similarly, when using the long side of the block to be decoded, the aspect ratio of the block to be decoded, the partition mode of the block to be decoded, the number of pixels of the block to be decoded, etc. as the shape of the block to be decoded, if weighted averaging is performed over a wide area, it will be no different from simple bi-prediction, so it is desirable to exclude weighting elements with patterns having wide partition boundaries from the options.

[0163] That is, in step S304 of the flowchart shown in FIG. 12, the short side of the block to be decoded may be replaced with the long side of the block to be decoded, the aspect ratio of the block to be decoded, the division mode of the block to be decoded, or the number of pixels of the block to be decoded.

[0164] In the flowchart shown in FIG. 12, in step S304, the synthesizing unit 207 may determine whether the shorter side of the block to be decoded is less than a preset threshold value.

[0165] As a modification of the above, the shape of the block to be decoded may be determined by using the short side of the block to be decoded, the aspect ratio of the block, or the division mode (angle of the division boundary).

[0166] For example, if the short side of the block to be decoded is small, the aspect ratio of the block is large (e.g., length:width = 4:1), and the angle of the division boundary is 45 degrees or more, weighting coefficients for patterns with wide division boundaries may be excluded from the options.

[0167] Conversely, if the short side of the block to be decoded is small, the aspect ratio of the block is large (e.g., length:width = 4:1), and the angle of the division boundary is less than 45 degrees, weighting coefficients with patterns in which the width of the division boundary is narrow may be excluded from the options.

[0168] This makes it possible to select the width of the division boundary taking into account the shape of the block, which is expected to improve coding performance.

[0169] Furthermore, the synthesizing unit 207 may be configured to limit the combinations of weighting factors that can be selected depending on the motion vector.

[0170] Specifically, the synthesis unit 207 may be configured to use the motion vectors of the small regions and limit the combinations of selectable weighting factors depending on the length of the motion vectors of the small regions or the resolution of the motion vectors of the small regions.

[0171] The larger the motion vector, the more blurred the boundary between the divisions, so it is desirable to widen the distribution of the weighting factors.Similarly, the coarser the resolution of the motion vector, the more blurred the boundary between the divisions, so it is desirable to widen the distribution of the weighting factors.

[0172] Furthermore, the synthesis unit 207 may be configured to limit the combinations of weighting factors that can be selected according to the difference between the motion vectors of the small area A and the small area B.

[0173] Here, the difference in motion vector is the difference in the reference frames of the motion vectors of small area A and small area B, or the difference in the motion vectors themselves.

[0174] For example, the synthesis unit 207 may be configured to limit the selectable combinations of weighting factors so as to narrow the distribution of weighting factors if the difference between the motion vectors of small area A and small area B is equal to or greater than a predetermined threshold (e.g., 1 pixel), and to limit the selectable combinations of weighting factors so as to widen the distribution of weighting factors if the difference between the motion vectors of small area A and small area B is less than a predetermined threshold (e.g., 1 pixel).

[0175] According to this configuration, highly accurate prediction can be made in accordance with image edges that may occur near division boundaries (such as boundaries between background and foreground that have different movements).

[0176] Alternatively, the synthesis unit 207 may be configured to limit the combination of selectable weighting factor patterns so as to widen the distribution of weighting factors if the difference between the motion vectors of small area A and small area B is equal to or greater than a predetermined threshold (e.g., 1 pixel), and to limit the combination of selectable weighting factor patterns so as to narrow the distribution of weighting factors if the difference between the motion vectors of small area A and small area B is less than a predetermined threshold (e.g., 1 pixel).

[0177] According to this configuration, it is possible to make a highly accurate prediction in accordance with the magnitude of motion blur near the division boundary.

[0178] Here, the synthesizing unit 207 may be configured to limit the combinations of weighting factors that can be selected depending on the relationship between the angle between the motion vector and the division boundary.

[0179] For example, as shown in FIG. 10, the synthesis unit 207 may be configured to limit the selectable weighting coefficients depending on the absolute value |x×u+y×v| of the dot product of the motion vector (x, y) and the unit normal vector (u, v) of the division boundary.

[0180] Alternatively, the synthesis unit 207 may be configured to limit the combinations of selectable weighting factors depending on the exposure time or the frame rate.

[0181] Furthermore, the synthesis unit 207 may be configured to limit the combinations of weighting factors that can be selected depending on the prediction method for the small region.

[0182] Furthermore, the synthesis unit 207 may be configured to limit the combinations of weighting factors that can be selected depending on the quantization parameter.

[0183] In addition, the synthesis unit 207 may be configured to limit the selectable combinations of weighting factors for the block to be decoded, not only in accordance with the control information of the block to be decoded, but also in accordance with the control information of blocks neighboring the block to be decoded.

[0184] For example, since small regions tend to be continuous across multiple blocks, the synthesis unit 207 may be configured to limit the combinations of weighting factors of the block to be decoded that can be selected depending on the weighting factors of adjacent decoded blocks.

[0185] FIG. 11 is a diagram showing an example of blocks adjacent to the left, upper left, upper, and upper right of the current block to be decoded.

[0186] Although partition boundaries also exist in the blocks to the left and top left adjacent to the block to be decoded, the synthesis unit 207 does not include these boundaries in the combination of weighting factors for the block to be decoded because they are not contiguous with the partition boundaries of the block to be decoded. Instead, the synthesis unit 207 can include the width of the partition boundary of the block above the block to be decoded, which has a contiguous partition boundary, in the combination of weighting factors for the block to be decoded.

[0187] In addition, when limiting the selectable combinations of weighting factors for the block to be decoded, the synthesis unit 207 may be configured to limit the combinations in stages, rather than being limited to the binary choice of whether or not to include the combination.

[0188] For example, the decoding unit 201 improves the coding efficiency by assigning different code lengths according to the selection probability of the weighting coefficients and decoding the data.

[0189] In the above example, the decoding unit 201 can set the weighting factor pattern used by the adjacent decoded block as a short code length, and set other patterns as long code lengths.

[0190] The above-described image decoding device 200 may be realized as a program that causes a computer to execute each function (each step). [Industrial Applicability]

[0191] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which will contribute to the achievement of Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]

[0192] 200...Image decoding device 201...Decoding section 202...Inverse quantization section 203...Inverse conversion unit 204...Intra prediction unit 205...Storage section 206...Motion compensation unit 207...Synthesis section 208...Addition section 210...Code input section 220...Image output unit

Claims

1. a decoding unit that controls decoding of a syntax that controls a boundary width of a geometric block division mode of a sequence to be decoded; The image decoding device is characterized in that the decoding unit decodes syntax that controls whether the boundary width can be changed, and can change the pattern or number of the boundary width in the geometric block division mode in the header of each block to be decoded.

2. The image decoding device according to claim 1 , wherein the decoding unit omits decoding of the pattern of the boundary width when the pattern of the boundary width is set in advance.

3. The image decoding device according to claim 1 , wherein the decoding unit changes a value of a syntax that controls a pattern of the boundary width in accordance with another syntax.

4. The image decoding device described in claim 7, characterized in that the decoding unit uses at least one of block size, motion vector, coding mode, division shape, and quantization parameter as syntax for changing the value of syntax that controls the boundary width pattern.

5. The method includes a step of controlling the decoding of a syntax for controlling a boundary width of a geometric block division mode of a sequence to be decoded, An image decoding method characterized in that, in the above process, a syntax that controls whether or not the boundary width can be changed is decoded, and the pattern or number of the boundary width in the geometric block division mode can be changed in the header of each block to be decoded.

6. A program that causes a computer to function as an image decoding device, the image decoding device includes a decoding unit that controls decoding of a syntax that controls a boundary width of a geometric block partition mode of a sequence to be decoded; The decoding unit decodes syntax that controls whether the boundary width can be changed, and the program is characterized in that the pattern or number of the boundary width in the geometric block division mode can be changed in the header of each block to be decoded.

Images