Image decoding device, image decoding method, and program

Abstract

To improve encoding efficiency in a GPM.SOLUTION: An image decoding device 200 comprises a decoding part 201 which controls decoding of syntax controlling a border width of a geometric block division mode of a sequence to be decoded, and the decoding part 201 can decode the syntax controlling whether the border width can be varied, and change the pattern or the number of the border width of the geometric block division mode at a header of each block to be decoded.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image decoding apparatus, an image decoding method, and a program.

Background Art

[0002] In Non-Patent Document 1 and Non-Patent Document 2, a Geometric Partitioning Mode (GPM) is disclosed.

[0003] GPM divides a rectangular block diagonally into two parts and performs motion compensation on each part. Specifically, the two divided regions are motion-compensated by merge vectors and synthesized by weighted averaging.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the technologies disclosed in Non-Patent Document 1 and Non-Patent Document 2, since the pattern of weighted averaging is limited, there is a problem that there is room for improvement in encoding performance. Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide an image decoding apparatus, an image decoding method, and a program capable of improving the encoding efficiency in GPM.

Means for Solving the Problems

[0006] The first feature of the present invention is an image decoding device comprising a decoding unit that controls the decoding of syntax that controls the boundary width of the geometric block division mode of the sequence to be decoded, wherein the decoding unit decodes syntax that controls whether or not the boundary width can be changed, and the pattern or number of boundary widths of the geometric block division mode can be changed in the header for each block to be decoded.

[0007] A second feature of the present invention is an image decoding method comprising a step of controlling the decoding of a syntax that controls the boundary width of the geometric block division mode of the sequence to be decoded, wherein in the step, a syntax that controls whether or not the boundary width can be changed is decoded, and the pattern or number of the boundary widths of the geometric block division mode can be changed in the header for each block to be decoded.

[0008] A third feature of the present invention is a program that causes a computer to function as an image decoding device, wherein the image decoding device includes a decoding unit that controls the decoding of syntax that controls the boundary width of the geometric block division mode of the sequence to be decoded, and the decoding unit decodes syntax that controls whether or not the boundary width can be changed, and the pattern or number of boundary widths of the geometric block division mode can be changed in the header for each block to be decoded. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program that can improve coding efficiency in GPM. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 shows an example of the functional block of an image decoding device 200 according to one embodiment. [Figure 2] Figure 2 shows an example of a case where a rectangular unit block is divided into two sub-regions, A and B, by a dividing boundary. [Figure 3]Figure 3 shows an example of three patterns of weight coefficients that can be assigned to the division boundary of sub-region B shown in Figure 2. [Figure 4] Figure 4 shows an example of applying the weight coefficient w of pattern (2) to an 8x8 block. [Figure 5] Figure 5 shows an example of applying the weight coefficient w of pattern (1) to an 8x8 block. [Figure 6] Figure 6 shows an example of applying the weight coefficient w of pattern (3) to an 8x8 block. [Figure 7] Figure 7 is a flowchart showing an example of a method for decoding control information by the decoding unit 201. [Figure 8] Figure 8 is a flowchart illustrating an example of the weight coefficient setting process by the synthesis unit 207 in the first embodiment. [Figure 9] Figure 9 is a flowchart illustrating an example of the weight coefficient setting process by the synthesis unit 207 in the second embodiment. [Figure 10] Figure 10 is a diagram illustrating a second embodiment. [Figure 11] Figure 11 is a diagram illustrating a second embodiment. [Figure 12] Figure 12 is a flowchart illustrating an example of the weight coefficient setting process by the synthesis unit 207 in the third embodiment. [Figure 13] Figure 13 illustrates an example where the weighting coefficients are determined based on the distance from the division boundary. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will be described below with reference to the drawings. Note that the components in the following embodiments can be replaced with existing components as appropriate, and various variations are possible, including combinations with other existing components. Therefore, the description of the following embodiments does not limit the content of the invention as described in the claims.

[0012] <First Embodiment> Hereinafter, the image decoding apparatus 200 according to the present embodiment will be described with reference to FIGS. 1 to 8. FIG. 1 is a diagram showing an example of the functional blocks of the image decoding apparatus 200 according to the present embodiment.

[0013] As shown in FIG. 1, the image decoding apparatus 200 includes a code input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transformation unit 203, an intra prediction unit 204, an accumulation unit 205, a motion compensation unit 206, a synthesis unit 207, an addition unit 208, and an image output unit 220.

[0014] The code input unit 210 is configured to acquire coded information encoded by an image encoding apparatus.

[0015] The decoding unit 201 is configured to control the decoding of syntax for controlling the boundary width of the division mode of the decoding target sequence.

[0016] Here, the syntax for controlling the boundary width of the division mode of the decoding target sequence indicates, for example, sps_div_blending_flag, sps_div_num_blending_list, sps_div_blending_list, pps_div_blending_flag, pps_div_num_blending_list, pps_div_blending_list, sh_div_blending_flag, sh_div_num_blending_list, sh_div_blending_list, etc., which will be described later.

[0017] Specifically, the decoding unit 201 is configured to decode control information and quantization values from the coded information input from the code input unit 210. For example, the decoding unit 201 is configured to output control information and quantization values by performing variable length decoding on such coded information.

[0018] Here, the quantized values ​​are sent to the inverse quantization unit 202, and the control information is sent to the motion compensation unit 206, the intra prediction unit 204, and the synthesis unit 207. This control information includes information necessary for controlling the motion compensation unit 206, the intra prediction unit 204, and the synthesis unit 207, and may also include header information such as a sequence parameter set, a picture parameter set, a picture header, and a slice header.

[0019] The inverse quantization unit 202 is configured to inversely quantize the quantized values ​​sent from the decoding unit 201 to obtain the decoded conversion coefficients. These conversion coefficients are then sent to the inverse transformation unit 203.

[0020] The inverse transformation unit 203 is configured to inversely transform the transformation coefficients sent from the inverse quantization unit 202 to obtain the decoded predicted residual. This predicted residual is then sent to the summing unit 208.

[0021] The intra-prediction unit 204 is configured to generate a first prediction pixel based on the decoded pixels and control information sent from the decoding unit 201. Here, the decoded pixels are obtained via the addition unit 208 and stored in the storage unit 205. The first prediction pixel is a prediction pixel that is an approximation of the input pixels in a small region set by the synthesis unit 207. The first prediction pixel is sent to the synthesis unit 207.

[0022] The storage unit 205 is configured to cumulatively store the decoded pixels sent from the addition unit 208. These decoded pixels are referenced by the motion compensation unit 206 via the storage unit 205.

[0023] The motion compensation unit 206 is configured to generate a second predicted pixel based on the decoded pixels stored in the storage unit 205 and the control information sent from the decoding unit 201. Here, the second predicted pixel is a predicted pixel that is an approximation of the input pixel in a small region set by the synthesis unit 207. The second predicted pixel is sent to the synthesis unit 207.

[0024] The addition unit 208 is configured to obtain a decoded pixel by adding the predicted residual sent from the inverse transformation unit 203 and the third predicted pixel sent from the synthesis unit 207. This decoded pixel is then sent to the image output unit 220, the storage unit 205, and the intra-prediction unit 204.

[0025] The synthesis unit 207 is configured to generate a third prediction pixel by controlling the width of the division boundary using a weighted average, by preparing a plurality of weight coefficients with different division boundary widths for at least one of the first prediction pixel sent from the intra prediction unit 204 and the second prediction pixel sent from the motion compensation unit 206.

[0026] The role of the synthesis unit 207 is to select weight coefficients for multiple prediction pixels that are optimal for the block to be decoded, and to synthesize the input multiple prediction pixels according to the weight coefficients, in order to accurately compensate the block to be decoded in the subsequent addition unit 208.

[0027] Any partitioning mode can be used to divide the block to be decoded into multiple sub-regions, but below we will describe the case using the Geometric Partitioning Mode (GPM), which is disclosed in Non-Patent Documents 1 and 2, as an example of a partitioning mode.

[0028] Regarding the weight coefficients, multiple patterns are prepared with arbitrary values ​​set in advance for each pixel of the unit block, and one of these patterns is applied. In other words, the compositing unit 207 may be configured to select and apply one of the multiple weight coefficients.

[0029] The sum of the weight coefficients for multiple prediction pixels is designed to be 1 for each pixel, and the result of combining the multiple prediction pixels using these weight coefficients through weighted averaging is used as the prediction pixel by the synthesis unit 207.

[0030] Pixels with a weight coefficient of 1 (i.e., the maximum value) are adopted as input prediction pixels, while pixels with a weight coefficient of 0 (i.e., the minimum value) are not used as input prediction pixels. Conceptually, this is equivalent to dividing a unit block into multiple sub-regions, and determining which of the multiple input prediction pixels to apply to which region and in what proportion.

[0031] Here, it is desirable to distribute the weight coefficients in a non-rectangular shape, since a rectangular distribution such as bisecting allows them to be represented by smaller unit blocks.

[0032] Figure 2 illustrates an example where the unit blocks are distributed in a diagonal shape. In Figure 2, a rectangular unit block is divided into two sub-regions, A and B, by a dividing boundary.

[0033] In each sub-region A / B, predicted pixels are generated using any method, such as intra-prediction or motion compensation.

[0034] In this case, even if the shape of the partition is determined, if the weight coefficients near the partition boundary are fixed, it is not possible to represent the diversity of the partition boundary, and therefore the coding efficiency cannot be improved.

[0035] For example, if a small region is an area with rapid movement, blurring occurs during imaging. Therefore, it is preferable to blur multiple small regions over a wide area and weight the resulting boundary.

[0036] Conversely, if the small area is an artificially edited area like a text overlay, blurring will not occur. Therefore, it is preferable to limit the division boundary to a narrow area and simply weight the average to make multiple small areas adjacent to each other.

[0037] To solve this problem, this embodiment takes the procedure of preparing multiple weight coefficients with different widths for the sub-region division boundaries and selecting one.

[0038] Figure 3 shows examples of three patterns of weight coefficients that can be assigned to the division boundary of sub-region B shown in Figure 2. In Figure 3, the horizontal axis represents the distance in pixels from the position of the division boundary, and the vertical axis represents the weight coefficient.

[0039] Specifically, we have prepared three patterns: (1) in which a weight coefficient [0,1] is assigned to the range [a,b] for pixel-level distances a and b from a pre-set division boundary position; (2) in which distances a and b are similarly doubled and weight coefficients [0,1] are assigned to the range [2a,2b]; and (3) in which distances a and b are similarly halved and weight coefficients [0,1] are assigned to the range [a / 2,b / 2].

[0040] These weighting coefficients are determined by the distance d(x) from the dividing boundary (black solid line), as shown in Figure 13. c ,y c γ, which is uniquely determined by ) xc,yc When defined as such, it is equivalent to providing multiple patterns, rather than a limited pattern (fixed value), for the width τ in Figure 13 where the weight coefficient is other than the minimum or maximum value. Here, x c ,y c This represents the coordinates within the block to be decrypted.

[0041] In other words, the synthesis unit 207 may be configured to set a plurality of weighting coefficients according to the inter-pixel distance from the division boundary.

[0042] Alternatively, a=b may be used to set a weight coefficient that is symmetric with respect to the dividing boundary. In other words, the synthesis unit 207 may be configured to set a weight coefficient that is symmetric with respect to the dividing boundary as the weight coefficient described above.

[0043] Alternatively, a weight coefficient that is asymmetric with respect to the division boundary may be set such that a ≠ b. In other words, the synthesis unit 207 may be configured to set a weight coefficient that is asymmetric with respect to the division boundary as the weight coefficient described above.

[0044] Furthermore, the number of weighting coefficients can be increased beyond just two, a and b, to include multiple line segments. In other words, the compositing unit 207 may be configured to set weighting coefficients using multiple line segments according to the inter-pixel distance from the division boundary.

[0045] Figures 4 to 6 show examples of applying each weight coefficient w to an 8x8 block. The weight coefficient w in Figures 4 to 6 can take values ​​from 0 to 8 and are combined using the following formula.

[0046] (w×small area A+(8-w)×small area B+4)>>3 In this way, by setting multiple weight coefficients w according to the inter-pixel distance from the division boundary, the effect of being able to derive a uniform result for various block sizes such as 8x8 and 64x16 can be obtained. The type, shape, and number of patterns can be set arbitrarily. For example, above, we explained multiple patterns as twice and half the distances a and b, but four times and a quarter of the distances could also be used. Also, in the above formula, the weight coefficients were set to values ​​from 0 to 8, but they can also be set to other values ​​such as 0 to 16 or 0 to 32. In particular, when the distance is twice or four times, increasing the maximum value of the weight coefficient can improve the accuracy of the weighted average per pixel.

[0047] The following describes an example of the control information decoding process performed by the decoding unit 201.

[0048] The coded information input to the decoding unit 201 may include a sequence parameter set (SPS) that summarizes control information for each sequence.

[0049] Furthermore, the coded information input to the decoding unit 201 may include a picture parameter set (PPS) or picture header (PH) that summarizes control information for each picture.

[0050] Furthermore, the code information input to the decoding unit 201 may include a slice header (SH) that summarizes the control information for each slice.

[0051] Now, referring to Figure 7, we will explain how the decoding unit 201 sets the boundary width on a sequence-by-sequence basis.

[0052] As shown in Figure 7, in step S11, the decoding unit 201 determines whether sps_div_enabled_flag is 1 in the sequence parameter set. If yes, the process proceeds to step S12; if no, the process terminates.

[0053] Here, sps_div_enabled_flag is a syntax that controls the presence or absence of GPM; if sps_div_enabled_flag is 1, it indicates that GPM is enabled, and if sps_div_enabled_flag is 0, it indicates that GPM is disabled.

[0054] In step S12, the decoding unit 201 decodes sps_div_blending_flag.

[0055] Here, sps_div_blending_flag is syntax that controls whether or not the boundary width is changed. If sps_div_blending_flag is 1, it indicates that changing the boundary width is enabled, and if sps_div_blending_flag is 0, it indicates that changing the boundary width is disabled.

[0056] Next, the decoding unit 201 determines whether sps_div_blending_flag is 1 or not. If yes, the process proceeds to step S13; otherwise, the process terminates.

[0057] In step S13, the decoding unit 201 decodes sps_div_num_blending_list.

[0058] Here, sps_div_num_blending_list is syntax that controls the number of boundary width patterns.

[0059] In step S14, the decoding unit 201 decodes sps_div_blending_list, which represents boundary width patterns, a number of times specified in sps_div_num_blending_list.

[0060] In the example in Figure 3, sps_div_num_blending_list is set to 3, and the weight coefficients corresponding to patterns (1) to (3) are used as options for the boundary width in sps_div_blending_list.

[0061] Thus, by using sps_div_num_blending_list and sps_div_blending_list, the number of boundary width patterns and the pattern type can be changed on a sequence-by-sequence basis according to the image characteristics, which is expected to maximize coding efficiency.

[0062] For example, for sequences consisting of natural images captured by a camera, multiple patterns (1) to (3) can be set, as shown in the example in Figure 3. For sequences consisting of screen images such as computer screens or game screens, where patterns with a small distribution of weight coefficients are effective, only one type of pattern with a short boundary width can be set, thereby maximizing encoding efficiency.

[0063] When setting the boundary width on a picture-by-picture basis, the decoding unit 201 decodes pps or ph, decodes pps_div_enabled_flag instead of sps_div_enabled_flag, decodes pps_div_blending_flag instead of sps_div_blending_flag, decodes pps_div_num_blending_list instead of sps_div_num_blending_list, and decodes pps_div_blending_list instead of sps_div_blending_list.

[0064] In this way, by using pps_div_num_blending_list and pps_div_blending_list, the number of boundary width patterns and the pattern type can be changed on a picture-by-picture basis according to the image characteristics, which is expected to maximize encoding efficiency.

[0065] For example, for pictures composed of natural images captured by a camera, multiple patterns (1) to (3) can be set, as shown in the example in Figure 3. For pictures composed of screen images such as computer screens or game screens, where patterns with a small distribution of weight coefficients are effective, only one type of pattern with a short boundary width can be set, thereby maximizing encoding efficiency.

[0066] When setting the boundary width on a slice-by-slice basis, the decoding unit 201 decodes sh_div_enabled_flag instead of sps_div_enabled_flag, sh_div_blending_flag instead of sps_div_blending_flag, sh_div_num_blending_list instead of sps_div_num_blending_list, and sh_div_blending_list instead of sps_div_blending_list.

[0067] In this way, by using sh_div_num_blending_list and sh_div_blending_list, the number of boundary width patterns and the pattern type can be changed on a slice-by-slice basis according to the image characteristics, which is expected to maximize coding efficiency.

[0068] For example, for slice regions composed of natural images captured by a camera, multiple patterns (1) to (3) can be set, as shown in the example in Figure 3. For slice regions composed of screen images such as computer screens or game screens, where patterns with a small distribution of weight coefficients are effective, only one type of pattern with a short boundary width can be set, thereby maximizing encoding efficiency.

[0069] By setting the encoding only in the upper layers, the increase in the encoding amount can be suppressed, or by setting it in the lower layers as well and prioritizing the settings in the lower layers, adaptive control can be achieved.

[0070] Alternatively, if the pattern of the width of the division boundary is set in advance, the decoding unit 201 can omit decoding the pattern of the division boundary width itself.

[0071] In the examples above, we described how to set the number of weight coefficient patterns and pattern types on a sequence, picture, or slice basis. However, you can also directly select patterns on a block basis, as described later, without setting these parameters. In this case, the degree of freedom in setting the number of weight coefficient patterns and pattern types will decrease, but the increase in header information mentioned above can be avoided.

[0072] The following describes an example of the weight coefficient setting process by the synthesis unit 207, with reference to Figure 8.

[0073] As shown in Figure 8, in step S101, the synthesis unit 207 determines whether any of sps_div_enabled_flag, pps_div_enabled_flag, and sh_div_enabled_flag included in the control information described above is 1. If No (none of them is 1), the process proceeds to step S102; if Yes, the process proceeds to step S103.

[0074] In step S102, the synthesis unit 207 does not apply a weighted average using weight coefficients to the blocks to be decoded.

[0075] In step S103, the synthesis unit 207 determines whether or not GPM is applied to the block to be decoded. If No, the process proceeds to step S102; if Yes, the process proceeds to step S104.

[0076] In step S104, the blending unit 207 decodes cu_div_blending_idx, which is included in the control information described above and specifies the width of the division boundary.

[0077] If cu_div_blending_idx is 0, the operation proceeds to step S105; if cu_div_blending_idx is 1, the operation proceeds to step S106; and if cu_div_blending_idx is 2, the operation proceeds to step S107.

[0078] In step S105, the synthesis unit 207 selects and applies the weight coefficient of pattern (1) from patterns (1) to (3) as the weight coefficient.

[0079] In step S106, the synthesis unit 207 selects and applies the weight coefficient of pattern (2) from patterns (1) to (3) as the weight coefficient.

[0080] In step S107, the synthesis unit 207 selects and applies the weight coefficient of pattern (3) from patterns (1) to (3) as the weight coefficient.

[0081] Figure 8 shows an example where the blending unit 207 decodes one cu_div_blending_idx when the pattern of the division boundary width is symmetric (a=b). However, the blending unit 207 may decode two cu_div_blending_idx, cu_div_blending_idx0 and cu_div_blending_idx1, when the pattern of the division boundary width is asymmetric (a≠b).

[0082] Here, cu_div_blending_idx is decoded to identify one of the lowest-level div_blending_lists applied to the block to be decoded.

[0083] Therefore, if the lowest-level div_num_blending_list applied to the block to be decoded is 1, the decoding of cu_div_blending_idx can be omitted, and coding efficiency can be improved by estimating cu_div_blending_idx as 0.

[0084] If the lowest-level `div_num_blending_list` applied to the block to be decrypted is greater than 1, `cu_div_blending_idx` may be decrypted with a fixed length, or it may be decrypted with a variable length. If `cu_div_blending_idx` is decrypted with a variable length, the values ​​of `cu_div_blending_idx` may be assigned in order of decreasing selectivity.

[0085] Furthermore, if the color difference component of the block to be decoded is not downsampled relative to the luminance component of the block to be decoded, the synthesis unit 207 may derive the weight coefficient for determining the width of the division boundary of the color difference component of the block to be decoded without using control information for deriving the weight coefficient for determining the width of the division boundary of the luminance component of the block to be decoded (i.e., by separately using control information for deriving the weight coefficient for determining the width of the division boundary of the division boundary of the color difference component of the block to be decoded). With this configuration, the weight coefficient for the color difference component of the block to be decoded can be derived independently, and an improvement in encoding performance can be expected.

[0086] On the other hand, if the color difference component of the block to be decoded is downsampled relative to the luminance component of the block to be decoded, the synthesis unit 207 may use this as control information for deriving a weighting coefficient that determines the width of the division boundary of the luminance component of the block to be decoded (i.e., without separately using control information for deriving a weighting coefficient that determines the width of the division boundary of the color difference component of the block to be decoded).

[0087] With this configuration, the same effect obtained with the luminance component can be obtained with respect to the downsampled chrominance component. Furthermore, if control information such as a header is used to determine the width of the division boundary of the luminance component of the block to be decoded, such control information becomes unnecessary for the chrominance component, thus improving encoding performance can be expected.

[0088] <Second Embodiment> The second embodiment of the present invention will be described below with reference to Figures 3 and 9 to 11, focusing on the differences from the first embodiment described above.

[0089] In this embodiment, code length is reduced by identifying patterns of weight coefficients while eliminating the need for direct control information.

[0090] Therefore, in this embodiment, the synthesis unit 207 is configured to generate a third predicted pixel by weighting averaging using weight coefficients uniquely selected from a plurality of weight coefficients based on indirect control information for at least one of the first predicted pixel and the second predicted pixel described above.

[0091] In other words, in this embodiment, the synthesis unit 207 is configured to select (uniquely identify) a weight coefficient from among a plurality of weight coefficients according to indirect control information.

[0092] Here, the synthesis unit 207 may be configured to prepare a plurality of weight coefficients with different widths for the sub-region division boundaries and to select a weight coefficient from among these plurality of weight coefficients.

[0093] Specifically, the synthesis unit 207 may be configured to select a weight coefficient from among a plurality of weight coefficients according to the shape of the block to be decoded as indirect control information.

[0094] For example, the synthesis unit 207 may be configured to select a weighting coefficient from a plurality of weighting coefficients according to at least one of the short side of the block to be decoded, the long side of the block to be decoded, the aspect ratio of the block to be decoded, the division mode, and the number of pixels of the block to be decoded.

[0095] For example, when using the shorter side of the block to be decoded as the shape of the block to be decoded, if the shorter side of the block to be decoded is small, weighting and averaging over a wide area will result in a result no different from simple bidirectional prediction. Therefore, it is desirable to exclude weight coefficients for patterns with wide partition boundaries from the options.

[0096] For example, in the example shown in Figure 3, the synthesis unit 207 selects the weight coefficient of pattern (3) when the short side of the block to be decoded is less than or equal to the threshold, and selects the weight coefficient of pattern (2) when the short side of the block to be decoded is greater than the threshold. This increases the number of patterns while eliminating the need for pattern control information, thereby improving coding efficiency.

[0097] The following describes an example of the weight coefficient setting process by the synthesis unit 207, with reference to Figure 9.

[0098] As shown in Figure 9, in step S201, the synthesis unit 207 determines whether any of sps_div_enabled_flag, pps_div_enabled_flag, and sh_div_enabled_flag included in the control information described above is 1. If No (none of them is 1), the process proceeds to step S202; if Yes, the process proceeds to step S203.

[0099] In step S202, the synthesis unit 207 does not apply a weighted average using weight coefficients to the blocks to be decoded.

[0100] In step S203, the synthesis unit 207 determines whether or not GPM is applied to the block to be decoded. If No, the process proceeds to step S202; if Yes, the process proceeds to step S204.

[0101] In step S204, the synthesis unit 207 determines whether the short side of the block to be decoded is less than or equal to a preset threshold of 1. If the result is No, the process proceeds to step S205; if the result is Yes, the process proceeds to step S208.

[0102] In step S205, the synthesis unit 207 determines whether the shorter side of the block to be decoded is less than or equal to a preset threshold of 2. Here, threshold 2 is greater than threshold 1. If No, the process proceeds to step S206; if Yes, the process proceeds to step S207.

[0103] In step S206, the synthesis unit 207 selects and applies the weight coefficient of pattern (2) from patterns (1) to (3) as the weight coefficient.

[0104] In step S207, the synthesis unit 207 selects and applies the weight coefficient of pattern (1) from patterns (1) to (3) as the weight coefficient.

[0105] In step S208, the synthesis unit 207 selects and applies the weight coefficient of pattern (3) from patterns (1) to (3) as the weight coefficient.

[0106] Similarly, when using the longest side of the block to be decoded, the aspect ratio of the block to be decoded, the division mode of the block to be decoded, and the number of pixels in the block to be decoded as the shape of the block to be decoded, if a weighted average is applied over a wide area, it will become no different from simple biprediction. Therefore, it is desirable to exclude weight coefficients for patterns with wide division boundaries from the options.

[0107] In other words, in steps S204 and S205 of the flowchart shown in Figure 9, the shorter side of the block to be decoded may be replaced with the longer side of the block to be decoded, the aspect ratio of the block to be decoded, the division mode of the block to be decoded, or the number of pixels of the block to be decoded.

[0108] Furthermore, in the flowchart shown in Figure 9, in step S204, the merging unit 207 may determine whether the short side of the block to be decoded is less than a preset threshold of 1, and in step S205, the merging unit 207 may determine whether the short side of the block to be decoded is less than a preset threshold of 2.

[0109] Here, as an example of the above modifications, the shape of the block to be decoded may be determined by the shorter side of the block, the aspect ratio of the block, or the division mode (angle of the division boundary).

[0110] For example, if the shorter side of the block to be decrypted is small, the aspect ratio of the block is large (e.g., height:width = 4:1), and the angle of the division boundary is 45 degrees or more, then the weight coefficients for patterns with wide division boundaries may be excluded from the options.

[0111] Conversely, if the shorter side of the block to be decoded is small, the aspect ratio of the block is large (e.g., height:width = 4:1), and the angle of the division boundary is less than 45 degrees, then the weight coefficients for patterns with narrow division boundaries may be excluded from the options.

[0112] This allows for the selection of boundary widths that take the block shape into consideration, which is expected to improve encoding performance.

[0113] Furthermore, the synthesis unit 207 may be configured to select the weight coefficients described above according to the motion vector.

[0114] Specifically, the synthesis unit 207 may be configured to use the motion vectors of a small region and select the weight coefficients described above according to the length of the motion vectors of the small region or the resolution of the motion vectors of the small region.

[0115] Since a larger motion vector contributes to blurring the division boundary, it is desirable to broaden the distribution of weight coefficients. Similarly, since a coarser resolution of the motion vector contributes to blurring the division boundary, it is desirable to broaden the distribution of weight coefficients.

[0116] Furthermore, the synthesis unit 207 may be configured to select the weight coefficients described above according to the difference between the motion vectors of sub-region A and sub-region B.

[0117] Here, the difference in motion vectors is the difference in the reference frames of the motion vectors of sub-region A and sub-region B, or the difference in the motion vectors themselves.

[0118] For example, the synthesis unit 207 may be configured to select the weight coefficients described above to narrow the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is greater than or equal to a predetermined threshold (e.g., 1 pixel), and to select the weight coefficients described above to widen the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is less than a predetermined threshold (e.g., 1 pixel).

[0119] With this configuration, it is possible to make highly accurate predictions that match image edges that may occur near the division boundary (such as the boundary between the background and foreground, which have different movements).

[0120] Alternatively, the synthesis unit 207 may be configured to select the weight coefficients described above to broaden the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is greater than or equal to a predetermined threshold (e.g., 1 pixel), and to narrow the distribution of weight coefficients described above if the difference between the motion vectors of sub-region A and sub-region B is less than a predetermined threshold (e.g., 1 pixel).

[0121] With this configuration, it is possible to make highly accurate predictions based on the magnitude of motion blur near the division boundary.

[0122] Here, the synthesis unit 207 may be configured to select a weight coefficient that can be selected based on the relationship between the angle between the motion vector and the division boundary.

[0123] For example, as shown in Figure 10, the synthesis unit 207 may be configured to select the weight coefficients described above according to the absolute value |x×u+y×v| of the dot product of the motion vector (x,y) and the unit normal vector (u,v) of the dividing boundary.

[0124] Alternatively, the compositing unit 207 may be configured to select a weighting coefficient that can be selected according to the exposure time or frame rate.

[0125] Furthermore, the synthesis unit 207 may be configured to select weight coefficients that can be chosen according to the prediction method for the small region.

[0126] Furthermore, the synthesis unit 207 may be configured to select selectable weight coefficients according to the quantization parameters.

[0127] Furthermore, the synthesis unit 207 may be configured to select the weight coefficient of the block to be decoded not only according to the control information of the block to be decoded, but also according to the control information of blocks adjacent to the block to be decoded.

[0128] For example, since small regions tend to be continuous across multiple blocks, the merging unit 207 may be configured to select the weight coefficient of the block to be decoded according to the weight coefficients of the adjacent decoded blocks.

[0129] Figure 11 shows an example of blocks adjacent to the block to be decrypted: to the left, upper left, top, and upper right.

[0130] Although partition boundaries also exist in the blocks adjacent to the decryption target block, such as the blocks to the left and upper left, the merging unit 207 does not select them because they are not continuous with the partition boundaries of the decryption target block. Instead, it can select the width of the partition boundary of the block above the decryption target block, where the partition boundary is continuous, for the decryption target block.

[0131] Similarly, the synthesis unit 207 may be configured to derive a pattern of weight coefficients of blocks adjacent to the block to be decoded as an internal parameter corresponding to the merge index used when decoding the merge vector of each sub-region, and to select it as the weight coefficient of each sub-region of the block to be decoded.

[0132] The merging unit 207 may be configured to adopt the width of a pre-set pattern (for example, pattern (1)) for the sub-region of the block to be decoded if there is no merge vector corresponding to each sub-region.

[0133] Here, the synthesis unit 207 may be configured to adopt the width of the division boundary of a preset pattern (for example, pattern (1)) for the sub-region of the block to be decoded when each sub-region is in intra prediction mode.

[0134] <Third Embodiment> Hereinafter, with reference to Figures 3 and 8 to 12, a third embodiment of the present invention will be described, focusing on the differences from the first and second embodiments described above.

[0135] In this embodiment, the synthesis unit 207 is configured to generate a third predicted pixel by weighting an average of at least one of the first and second predicted pixels using one of the limited weight coefficients based on the decoded control information.

[0136] In other words, the synthesis unit 207 is configured to limit the combinations of weight coefficients that can be selected according to indirect control information, and then select the weight coefficient to be applied from among the limited combinations of weight coefficients based on the decoded control information.

[0137] Here, the synthesis unit 207 may be configured to prepare a plurality of weight coefficients with different widths for the sub-region division boundaries and to select the aforementioned weight coefficient.

[0138] The synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients according to the shape of the block to be decoded as indirect control information.

[0139] For example, the synthesis unit 207 may be configured to limit the selectable weight coefficients according to at least one of the following: the size of the block to be decoded (such as the short side or long side of the block to be decoded), the aspect ratio of the block to be decoded, the division mode of the block to be decoded, and the number of pixels in the block to be decoded.

[0140] Here, if the shorter side of the block to be decoded is used as the shape of the block to be decoded, then if the shorter side of the block to be decoded is small, weighting and averaging over a wide area will result in a result no different from simple bidirectional prediction. Therefore, it is desirable to exclude weighting systems with wide partition boundary patterns from the options (combinations of selectable weight coefficients).

[0141] For example, in the example shown in Figure 3, the synthesis unit 207 limits the selectable weight coefficient combinations to the weight coefficients of pattern (1) / (3) when the short side of the block to be decoded is less than or equal to the threshold, and limits the selectable weight coefficient combinations to the weight coefficients of pattern (1) / (2) when the short side of the block to be decoded is greater than the threshold. This increases the number of patterns while reducing the amount of code for the control information of the patterns, thereby improving coding efficiency.

[0142] Here, the threshold for the short side of the block to be decrypted may be set to, for example, 8 pixels or 16 pixels.

[0143] Below, with reference to Figure 12, an example of the weight coefficient setting process by the synthesis unit 207 will be described.

[0144] As shown in Figure 12, in step S301, the synthesis unit 207 determines whether any of sps_div_enabled_flag, pps_div_enabled_flag, and sh_div_enabled_flag included in the control information described above is 1. If No (none of them is 1), the process proceeds to step S302; if Yes, the process proceeds to step S303.

[0145] In step S302, the synthesis unit 207 does not apply a weighted average using weight coefficients to the blocks to be decoded.

[0146] In step S303, the synthesis unit 207 determines whether or not GPM is applied to the block to be decoded. If No, the process proceeds to step S302; if Yes, the process proceeds to step S304.

[0147] In step S304, the synthesis unit 207 determines whether the short side of the block to be decoded is less than or equal to a preset threshold.

[0148] If the answer is No, the process proceeds to step S305; if the answer is Yes, the process proceeds to step S306. Here, if the answer is No, the synthesis unit 207 limits the selectable combinations of weight coefficients to patterns (1) / (2); if the answer is Yes, the synthesis unit 207 limits the selectable combinations of weight coefficients to patterns (1) / (3).

[0149] In step S305, the blending unit 207 decodes cu_div_blending_idx (direct control information) included in the control information described above.

[0150] If cu_div_blending_idx is not 0, the process proceeds to step S307; if cu_div_blending_idx is 0, the process proceeds to step S308.

[0151] Similarly, if cu_div_blending_idx is not 0, the process proceeds to step S309; ​​if cu_div_blending_idx is 0, the process proceeds to step S310.

[0152] In step S307, the synthesis unit 207 selects and applies the weight coefficient of pattern (1) from among patterns (1) and (2) as the weight coefficient.

[0153] In step S308, the synthesis unit 207 selects and applies the weight coefficient of pattern (2) from among patterns (1) and (2) as the weight coefficient.

[0154] In step S309, the synthesis unit 207 selects and applies the weight coefficient of pattern (1) from among patterns (1) and (3) as the weight coefficient.

[0155] In step S310, the synthesis unit 207 selects and applies the weight coefficient of pattern (3) from among patterns (1) and (3) as the weight coefficient.

[0156] As described above, if the shorter side of the decoded block, which is the indirect control information, is less than or equal to a threshold, the synthesis unit 207 limits the selectable combination of weight coefficients from patterns (1) to (3) to the weight coefficients of pattern (1) / (3), and by making a determination on cu_div_blending_idx, which is the direct control information, it selects one pattern's weight coefficient from among the limited pattern's weight coefficients.

[0157] On the other hand, if the shorter side of the block to be decoded is greater than the threshold, the synthesis unit 207 limits the selectable combination of weight coefficients from patterns (1) to (3) to the weight coefficients of pattern (1) / (2), and by making a determination on cu_div_blending_idx as direct control information, selects one pattern's weight coefficient from the limited number of pattern weight coefficients.

[0158] In other words, in the example where the sps_div_blending_list contains three types of weight coefficients for patterns (1) to (3), the blending unit 207 can limit the combinations of weight coefficients for patterns selectable by cu_div_blending_idx to two types through indirect control information.

[0159] Then, the synthesis unit 207 selects the weighting coefficient of pattern (2) if the short side of the block to be decoded is less than or equal to the threshold and cu_div_blending_idx is 0, and selects the weighting coefficient of pattern (1) if cu_div_blending_idx is 1.

[0160] On the other hand, the blending unit 207 selects the weighting coefficient of pattern (3) if the shorter side of the block to be decoded is greater than the threshold, and if cu_div_blending_idx is 0, it selects the weighting coefficient of pattern (1) if cu_div_blending_idx is 1.

[0161] According to the above configuration, even when cu_div_blending_idx is 0, the coding efficiency can be improved by controlling the pattern of weight coefficients selected in accordance with other control signals.

[0162] Similarly, when using the longest side of the block to be decoded, the aspect ratio of the block to be decoded, the division mode of the block to be decoded, and the number of pixels in the block to be decoded as the shape of the block to be decoded, if a weighted average is applied over a wide area, it will become no different from simple biprediction. Therefore, it is desirable to exclude weighting elements with wide division boundary patterns from the options.

[0163] In other words, in step S304 of the flowchart shown in Figure 12, the shorter side of the block to be decoded may be replaced with the longer side of the block to be decoded, the aspect ratio of the block to be decoded, the division mode of the block to be decoded, or the number of pixels of the block to be decoded.

[0164] Furthermore, in the flowchart shown in Figure 12, in step S304, the synthesis unit 207 may determine whether the short side of the block to be decoded is less than a preset threshold.

[0165] Here, as an example of the above modifications, the shape of the block to be decoded may be determined by the shorter side of the block, the aspect ratio of the block, or the division mode (angle of the division boundary).

[0166] For example, if the shorter side of the block to be decrypted is small, the aspect ratio of the block is large (e.g., height:width = 4:1), and the angle of the division boundary is 45 degrees or more, then the weight coefficients for patterns with wide division boundaries may be excluded from the options.

[0167] Conversely, if the shorter side of the block to be decoded is small, the aspect ratio of the block is large (e.g., height:width = 4:1), and the angle of the division boundary is less than 45 degrees, then the weight coefficients for patterns with narrow division boundaries may be excluded from the options.

[0168] This allows for the selection of boundary widths that take the block shape into consideration, which is expected to improve encoding performance.

[0169] Furthermore, the synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients according to the motion vector.

[0170] Specifically, the synthesis unit 207 may be configured to utilize the motion vectors of a small region and limit the selectable combinations of weight coefficients according to the motion vector length of the small region or the resolution of the motion vectors of the small region.

[0171] Since a larger motion vector contributes to blurring the division boundary, it is desirable to broaden the distribution of weight coefficients. Similarly, since a coarser resolution of the motion vector contributes to blurring the division boundary, it is desirable to broaden the distribution of weight coefficients.

[0172] Furthermore, the synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients according to the difference between the motion vectors of sub-region A and sub-region B.

[0173] Here, the difference in motion vectors is the difference in the reference frames of the motion vectors of sub-region A and sub-region B, or the difference in the motion vectors themselves.

[0174] For example, the synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients so as to narrow the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is greater than or equal to a predetermined threshold (e.g., 1 pixel), and to limit the selectable combinations of weight coefficients so as to widen the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is less than a predetermined threshold (e.g., 1 pixel).

[0175] With this configuration, it is possible to make highly accurate predictions that match image edges that may occur near the division boundary (such as the boundary between the background and foreground, which have different movements).

[0176] Alternatively, the synthesis unit 207 may be configured to limit the combination of selectable weight coefficient patterns so as to broaden the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is greater than or equal to a predetermined threshold (e.g., 1 pixel), and to limit the combination of selectable weight coefficient patterns so as to narrow the distribution of weight coefficients if the difference between the motion vectors of sub-region A and sub-region B is less than a predetermined threshold (e.g., 1 pixel).

[0177] With this configuration, it is possible to make highly accurate predictions based on the magnitude of motion blur near the division boundary.

[0178] Here, the synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients based on the relationship between the motion vector and the division boundary.

[0179] For example, as shown in Figure 10, the synthesis unit 207 may be configured to limit the selectable weight coefficients according to the absolute value |x×u+y×v| of the dot product of the motion vector (x,y) and the unit normal vector (u,v) of the division boundary.

[0180] Alternatively, the synthesis unit 207 may be configured to limit the combination of weighting coefficients that can be selected according to the exposure time or frame rate.

[0181] Furthermore, the synthesis unit 207 may be configured to limit the combination of weight coefficients that can be selected depending on the prediction method for the small region.

[0182] Furthermore, the synthesis unit 207 may be configured to limit the combination of weight coefficients that can be selected according to the quantization parameters.

[0183] Furthermore, the synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients for blocks to be decoded not only according to the control information of the blocks to be decoded, but also according to the control information of blocks adjacent to the blocks to be decoded.

[0184] For example, since small regions tend to be continuous across multiple blocks, the synthesis unit 207 may be configured to limit the selectable combinations of weight coefficients for the blocks to be decoded, depending on the weight coefficients of the adjacent decoded blocks.

[0185] Figure 11 shows an example of blocks adjacent to the block to be decrypted: to the left, upper left, top, and upper right.

[0186] Although partition boundaries also exist in the blocks to the left and upper left of the block to be decoded, the blending unit 207 is not continuous with the partition boundary of the block to be decoded, so it is not included in the combination of weight coefficients for the block to be decoded. Instead, the width of the partition boundary of the block above the block to be decoded, where the partition boundary is continuous, can be included in the combination of weight coefficients for the block to be decoded.

[0187] Furthermore, when the synthesis unit 207 limits the combinations of weight coefficients of selectable decryption target blocks, it may be configured to limit them stepwise, rather than being limited to a simple choice of whether or not to include a block in such a combination.

[0188] For example, the decoding unit 201 improves coding efficiency by assigning different code lengths according to the selection probability of the weight coefficients mentioned above and decoding accordingly.

[0189] In the example described above, the decoding unit 201 can set the weight coefficient pattern used by adjacent decoded blocks as a short code length and other patterns as a long code length.

[0190] The image decoding device 200 described above may be implemented as a program that causes a computer to execute each function (each process). [Industrial applicability]

[0191] Furthermore, according to this embodiment, for example, it is possible to achieve an overall improvement in service quality in video communication, thereby contributing to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]

[0192] 200…Image Decoding Device 201...Decoding section 202...Inverse quantization section 203...Inverse Transformation Section 204...Intra Prediction Unit 205...Storage section 206...Motion compensation unit 207...Synthesis section 208... Addition section 210... Code input section 220...Image output unit

Claims

1. It includes a decoding unit that controls the decoding of the syntax that controls the boundary width of the geometric block partitioning mode of the sequence to be decoded, The image decoding device is characterized in that the decoding unit decodes syntax that controls whether or not the boundary width can be changed, and the pattern or number of boundary widths in geometric block division mode can be changed in the header for each block to be decoded.

2. The image decoding apparatus according to claim 1, characterized in that the decoding unit omits decoding the boundary width pattern itself when the boundary width pattern is set in advance.

3. The image decoding apparatus according to claim 1, characterized in that the decoding unit changes the value of the syntax that controls the boundary width pattern according to other syntax.

4. The image decoding apparatus according to claim 1, characterized in that the decoding unit uses at least one of block size, motion vector, coding mode, division shape, and quantization parameter as a syntax for changing the value of the syntax that controls the pattern of the boundary width.

5. The process includes a step of controlling the decoding of a syntax that controls the boundary width of the geometric block partitioning mode of the sequence to be decoded. An image decoding method characterized in that, in the above step, a syntax that controls whether or not the boundary width can be changed is decoded, and the pattern or number of boundary widths in the geometric block division mode can be changed in the header for each block to be decoded.

6. A program that makes a computer function as an image decoding device, The image decoding device includes a decoding unit that controls the decoding of the syntax that controls the boundary width of the geometric block division mode of the sequence to be decoded. The decoding unit is a program characterized by decoding syntax that controls whether or not the boundary width can be changed, and being able to change the pattern or number of boundary widths in geometric block division mode in the header for each block to be decoded.

Images