Image decoding device, image decoding method, and program

Abstract

To prevent a search range of template matching having a motion vector as an origin from exceeding a referenceable area including a reference image after a motion vector that references the outside of the referenceable area is clipped to the outside of the referenceable area.SOLUTION: In an image decoding device 200 according to the present invention, when a motion vector refers to a region outside a referenceable area including a reference image, a motion compensation unit 208 controls a clip position of the motion vector depending on whether template matching is valid or not.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image decoding device, an image decoding method, and a program. [Background technology]

[0002] Non-patent document 1 defines an area (i.e., a repeated padding area) in which pixel values ​​at the screen boundary are duplicated (padded) by a predetermined number of pixels outward from the reference image boundary, so that motion compensation can be performed even when the motion vector references an area outside the reference image.

[0003] Furthermore, Non-Patent Document 1 discloses a technique for translating (clipping) the reference position of a motion vector into the referenceable area when the motion vector references an area outside the referenceable area, including a reference image or a repeated padding area.

[0004] Non-Patent Document 2 discloses template matching for correcting motion vectors. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] ITU-T H.266 / VVC [Non-patent document 2] Algorithm description of Enhanced Compression Model 6 (ECM 6), JVET-AA2025 Summary of the Invention [Problem to be solved by the invention]

[0006] When the template matching disclosed in Non-Patent Document 2 is effective and a motion vector references an area outside the referenceable area (referenceable area) including a reference image, there is a problem in that the search range of template matching with the clipped motion vector as the origin exceeds the referenceable area. Therefore, the present invention has been made in consideration of the above-mentioned problem, and aims to provide an image decoding device, an image decoding method, and a program that can prevent the search range of template matching with the motion vector as the origin from exceeding the referenceable area including the reference image after a motion vector that references an area outside the referenceable area is clipped. [Means for solving the problem]

[0007] A first feature of the present invention is an image decoding device comprising: a decoding unit that decodes control information; an intra prediction unit that generates a first predicted pixel based on decoded pixels and the control information; a storage unit that accumulates the decoded pixels; and a motion compensation unit that generates a second predicted pixel based on the accumulated decoded pixels and the control information, wherein the motion compensation unit controls a clip position of the motion vector depending on whether template matching is valid when the motion vector references a region outside a referenceable region that includes a reference image.

[0008] A second feature of the present invention is an image decoding device including: a decoding unit that decodes control information; an intra prediction unit that generates a first predicted pixel based on decoded pixels and the control information; a storage unit that accumulates the decoded pixels; and a motion compensation unit that generates a second predicted pixel based on the accumulated decoded pixels and the control information, wherein the motion compensation unit limits a search range for the template matching depending on whether template matching is valid and on the width or height of a block to be decoded when a motion vector references a region outside a referenceable region including a reference image.

[0009] A third feature of the present invention is an image decoding method comprising: a step A of decoding control information; a step B of generating a first predicted pixel based on decoded pixels and the control information; a step C of accumulating the decoded pixels; and a step D of generating a second predicted pixel based on the accumulated decoded pixels and the control information, wherein in step D, when a motion vector references a region outside a referenceable region including a reference image, a clip position of the motion vector is controlled depending on whether template matching is valid.

[0010] A fourth feature of the present invention is a program that causes a computer to function as an image decoding device, the image decoding device including: a decoding unit that decodes control information; an intra prediction unit that generates a first predicted pixel based on decoded pixels and the control information; a storage unit that accumulates the decoded pixels; and a motion compensation unit that generates a second predicted pixel based on the accumulated decoded pixels and the control information, wherein the motion compensation unit controls a clip position of the motion vector depending on whether template matching is valid when the motion vector references a region outside a referenceable region that includes a reference image. [Effects of the Invention]

[0011] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program that can prevent the search range of template matching with the motion vector as the origin from exceeding the referenceable area including the reference image after the motion vector that references outside the referenceable area is clipped outside the referenceable area. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing an example of functional blocks of an image decoding device 200 according to an embodiment. [Figure 2] FIG. 2 is a diagram for explaining template matching (TM). [Figure 3]FIG. 3 is a diagram showing an example of a referenceable area when the motion compensation unit 208 disclosed in Non-Patent Document 1 and Non-Patent Document 2 derives a motion vector. [Figure 4] FIG. 4 is a diagram for explaining a specific example of generation of a motion compensation padding area in FIG. [Figure 5] FIG. 5 is a diagram showing an example of a clipping method when a motion vector refers to an area outside the referenceable area on the right side of the reference image, with respect to the referenceable area disclosed in Non-Patent Document 1. [Figure 6] FIG. 6 is a diagram showing an example of a clipping method when a motion vector refers to an area outside the referenceable area on the left side of the reference image, as disclosed in Non-Patent Document 1. In FIG. [Figure 7] FIG. 7 is a diagram showing an example of a case where template matching is enabled and a motion vector references an area outside the referenceable area, and the motion vector is clipped. [Figure 8] FIG. 8 is a diagram showing an example of a case where template matching is enabled and a motion vector references an area outside the referenceable area, and the motion vector is clipped. [Figure 9] FIG. 9 is a diagram showing an example of a case where template matching is enabled and a motion vector references an area outside the referenceable area, and the motion vector is clipped. [Figure 10] FIG. 10 is a diagram showing an example of a case where template matching is enabled and a motion vector references an area outside the referenceable area, and the motion vector is clipped. [Figure 11] FIG. 11 is a diagram for explaining the first solution. [Figure 12] FIG. 12 is a diagram for explaining the second solution. [Figure 13] FIG. 13 is a diagram for explaining the second solution. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0014] First Embodiment An image decoding device 200 according to this embodiment will be described below with reference to Figures 1 to 13. Figure 1 is a diagram showing an example of functional blocks of the image decoding device 200 according to this embodiment.

[0015] As shown in FIG. 1, the image decoding device 200 includes a code input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transform unit 203, an intra prediction unit 204, an adder 206, a storage unit 207, a motion compensation unit 208, and an image output unit 220.

[0016] The code input unit 210 is configured to obtain code information coded by an image coding device.

[0017] The decoding unit 201 is configured to decode the control information and the quantized value from the code information input from the code input unit 210. For example, the decoding unit 201 is configured to output the control information and the quantized value by performing variable length decoding on the code information.

[0018] Here, the quantized value is sent to the inverse quantization unit 202, and the control information is sent to the motion compensation unit 208 and the intra prediction unit 204. Note that this control information includes information necessary for controlling the inverse quantization unit 202, the motion compensation unit 208, the intra prediction unit 204, etc., and may include header information such as a sequence parameter set, a picture parameter set, a picture header, or a slice header.

[0019] The inverse quantization unit 202 is configured to inverse quantize the quantized values ​​sent from the decoding unit 201 to generate decoded transform coefficients. The transform coefficients are sent to the inverse transform unit 203.

[0020] The inverse transform unit 203 is configured to inverse transform the transform coefficients sent from the inverse quantization unit 202 to generate decoded prediction residuals. The prediction residuals are sent to an adder 206.

[0021] The intra prediction unit 204 is configured to generate a first predicted pixel based on the decoded pixel and the control information sent from the decoding unit 201. Here, the decoded pixel is obtained via the adder 206 and stored in the storage unit 207. The first predicted pixel is a predicted pixel to be added to the prediction residual in the adder 206. The first predicted pixel is sent to the adder 206.

[0022] The storage unit 207 is configured to cumulatively store the decoded pixels sent from the adder 206. The motion compensation unit 208 references these decoded pixels via the storage unit 207.

[0023] The motion compensation unit 208 is configured to generate second predicted pixels based on the decoded pixels stored in the storage unit 207 and the control information sent from the decoding unit 201. Here, the second predicted pixels are predicted pixels to be added to the prediction residual in the adder 206. The second predicted pixels are sent to the adder 206.

[0024] The adder 206 is configured to add the prediction residual sent from the inverse transform unit 203 to either the input first predicted pixel or second predicted pixel to obtain a decoded pixel. The decoded pixel is sent to the image output unit 220, the storage unit 207, and the intra prediction unit 204.

[0025] The image output unit 220 is configured to output the decoded pixels sent from the adder 206 .

[0026] The motion compensation unit 208, which is a characteristic component of the image decoding device 200 according to this embodiment, will be described below.

[0027] <Method of deriving motion vectors and method of generating inter-predicted images> The motion compensation unit 208 derives a combination of a motion vector and a reference image required to generate an inter-predicted pixel (that is, to perform motion compensation from a reference image).

[0028] Here, the combination of the motion vector and the reference image derived by the motion compensation unit 208 may be configured from a maximum of two different combinations of the motion vector and the reference image disclosed in Non-Patent Document 1.

[0029] Alternatively, the combination of the motion vector and the reference image derived by the motion compensation unit 208 may be composed of a combination of two or more (e.g., three or four) motion vectors and reference images, such as Multiple Hypothesis Prediction (MHP) or Overlapped Block Motion Compensation (OBMC) disclosed in Non-Patent Document 2.

[0030] Furthermore, the motion compensation unit 208 may use the Adaptive Motion Vector Prediction (AMVP) mode or merge mode disclosed in Non-Patent Document 1 as a method for deriving a motion vector and a reference image.

[0031] When using the AMVP mode, the motion compensation unit 208 generates candidates for motion vector prediction (MVP) using methods such as spatial AMVP, temporal AMVP, or history-based AMVP disclosed in Non-Patent Document 1 and Non-Patent Document 2 in order to derive the motion vector of the block to be decoded, and selects MVP candidates according to the control information sent from the decoding unit 201.

[0032] In addition, when using AMVP mode, the motion compensation unit 208 adds the motion vector difference (MVD) included in the control information sent from the decoding unit 201 to the selected MVP candidate to derive the final motion vector and reference image.

[0033] When using merge mode, the motion compensation unit 208 generates merge candidates (combinations of motion vectors and reference images) from spatial merge, temporal merge, non-adjacent spatial merge, history-based merge, pairwise average merge, etc. disclosed in Non-Patent Document 1 and Non-Patent Document 2 in order to derive the motion vector of the block to be decoded, and registers them in a merge list.

[0034] Furthermore, when the merge mode is used, the motion compensation unit 208 uses a merge index included in the control information sent from the decoding unit 201 to derive a final motion vector and reference image from the merge candidates.

[0035] Furthermore, the motion compensation unit 208 may apply template matching for correcting motion vectors, as disclosed in Non-Patent Document 2, to the motion vectors derived in the above-mentioned AMVP mode or merge mode.

[0036] Specifically, as shown in Figure 2, template matching is a technique in which the motion vector (Initial MV) of the block to be decoded is used as the origin, a reference position (i.e., MVD) that minimizes the template cost within a specified search range is derived, and this is added to the motion vector (Initial MV) of the block to be decoded to generate a final motion vector (Final MV).

[0037] Here, the template cost is the sum of absolute differences (SAD) or the sum of absolute transformed differences (SATD) between the block to be decoded and one or more lines of decoded pixels adjacent to the left and / or top of the reference block.

[0038] Furthermore, the motion compensation unit 208 may apply the above-described template cost comparison not only to the generation of the final motion vector, but also to the selection of MVP candidates in the AMVP mode.

[0039] That is, in the AMVP mode, the motion compensation unit 208 may select the MVP that minimizes the template cost.

[0040] The search range for template matching may be configured as a range of 8 pixels (±8 pixels) in the up, down, left and right directions, with the motion vector (Initial MV) defined in FIG. 2 and Non-Patent Document 2 as the origin.

[0041] Alternatively, the search range for template matching may be configured as a range smaller than the above range (for example, ±2, ±4, ±6 pixels) or a range larger than the above range (for example, ±10, ±12, ±14 pixels).

[0042] Alternatively, the search range for template matching may be configured as an asymmetric search range in the up / down and left / right directions.

[0043] Furthermore, the search accuracy of MVD in template matching may be controlled taking into consideration the accuracy of motion vectors for the AMVP mode and merge mode, as disclosed in FIG. 2 and Non-Patent Document 2.

[0044] For example, for the AMVP mode, if the motion vector accuracy is 4-pixel accuracy or integer pixel accuracy in AMVR (Adaptive Motion Vector Resolution) disclosed in Non-Patent Document 1, the motion compensation unit 208 may perform searches only at each pixel accuracy.

[0045] In merge mode, if the alternative interpolation filter (Switchable Interpolation Filter) for motion compensation disclosed in Non-Patent Document 1 is enabled, the motion compensation unit 208 may search only with half-pixel accuracy and integer pixel accuracy, as shown in Figure 2.

[0046] Finally, the motion compensation unit 208 generates inter-predicted pixels for the current block to be decoded using the motion vectors derived as described above and the reference image.

[0047] <Area where motion vectors can be referenced> The referenceable area of ​​the motion vector derived by the motion compensation unit 208 will be described below.

[0048] FIG. 3 is a diagram showing an example of a referenceable area when the motion compensation unit 208 disclosed in Non-Patent Document 1 and Non-Patent Document 2 derives a motion vector.

[0049] In video scenes where the entire screen moves quickly (video in which the camera pans quickly left and right or up and down), the person, object, background, etc. to be predicted may be outside the reference image (outside the reference image), i.e., the motion vector derived by motion compensation unit 208 may refer to an area outside the reference image.

[0050] To deal with such cases, Non-Patent Documents 1 and 2 define a referenceable area of ​​a predetermined number of pixels outside the reference image.

[0051] Specifically, Non-Patent Document 1 defines a repetitive padding area outside the reference image, as shown in FIG. 3(a), whose size is the maximum size of the width or height of the block to be decoded (block to be coded) (i.e., the width or height of the coding tree block) (maxCUwidth in the example of FIG. 3) plus 16 pixels. A fixed value other than 16 pixels may be set as the size of the repetitive padding area (the number of pixels from the boundary of the reference image). Specific examples will be described later.

[0052] Such a repeated padding area is generated by duplicating (padding) pixel values ​​located on the boundary of the reference image toward the outside of the reference image.

[0053] When a motion vector references a reference image or an area outside the referenceable area including the repeated padding area, the motion compensation unit 208 translates (clips) the reference position of the motion vector into the repeated padding area.

[0054] Next, in Non-Patent Document 2, as shown in FIG. 3(b), a motion compensation padding area is defined between the boundary of the reference image and the repeated padding area.

[0055] To improve the accuracy of motion compensation in the motion compensation unit 208, the motion compensation padding area assigns reference pixel values ​​of the reference image, rather than duplicating pixel values ​​located on the reference image boundary as in the repeated padding area.

[0056] FIG. 4 shows a specific example of how a motion compensation padding area is generated.

[0057] For example, as shown in Figure 4, when the reference blocks of a block to be decoded (BBlk) within the image to be decoded and an adjacent block (MCP Blk) outside the image to be decoded, which are separated by a boundary between the image to be decoded, are within the reference image, the reference block for the adjacent block contains pixel values ​​within the reference image, rather than a copy of the boundary pixel values ​​of the reference image.

[0058] In such cases, by retaining reference pixel values ​​that are not copies of pixel values ​​at the reference image boundary as the motion compensation padding area, the accuracy of motion compensation is improved when the motion vector references outside the reference image but within the motion compensation padding area compared to when a repeated padding area is defined outside the reference image.

[0059] The size of the motion compensation padding area may be 16 pixels as disclosed in Non-Patent Document 2, or may be 64 pixels as shown in Fig. 3. Alternatively, the size of the motion compensation padding area may be other than these, such as 4 pixels, 8 pixels, 32 pixels, 128 pixels, or 256 pixels, which is the minimum size, maximum size, or possible size value (number of pixels that is a power of 2) of the block to be decoded.

[0060] If the motion vector references an area outside the referenceable area including the reference image, the motion compensation padding area, or the repeated padding area, the motion compensation unit 208 translates (clips) the reference position of the motion vector within the repeated padding area.

[0061] <Clipping method when a motion vector references an area outside the referenceable area> The clipping method used by the motion compensation unit 208 when a motion vector refers to a region outside the referenceable area will be described below.

[0062] First, a method of clipping a motion vector when template matching is disabled and the motion vector references a region outside the referenceable area will be described with reference to FIGS.

[0063] FIG. 5 shows an example of a clipping method in which a motion vector references an area outside the referenceable area (consisting of a reference image and a repeated padding area) disclosed in Non-Patent Document 1 to the right of the reference image.

[0064] In Non-Patent Document 1, as shown in Figure 5, when a motion vector (Intitial MV) references an area outside the referenceable area, i.e., when the reference block (RefBlk) is outside the referenceable area, the motion compensation unit 208 translates (clips) the motion vector indicating the upper left corner of the reference block to a position 8 pixels away from the boundary of the reference image.

[0065] This clipping operation also applies to the case where a motion vector in the downward direction of the reference image refers to a picture outside the reference image.

[0066] FIG. 6 shows an example of a clipping method when a motion vector references an area outside the referenceable area (consisting of a reference image and a repeated padding area) disclosed in Non-Patent Document 1 to the left of the reference image.

[0067] In Non-Patent Document 1, as shown in Figure 6, when a motion vector references an area outside the referenceable area, i.e., when the reference block (RefBlk) is outside the referenceable area, the motion compensation unit 208 translates (clips) the motion vector (Intitial MV) indicating the upper left corner of the reference block to a position that is 8 pixels away from the boundary of the reference image plus the maximum width (maxCUwidth) or maximum height of the block to be decoded.

[0068] This clipping operation also applies to the case where a motion vector in the upward direction of the reference image refers to a picture outside the reference image.

[0069] In Non-Patent Document 1, the motion vector is clipped to a position 8 pixels away from the boundary of the reference image or the boundary of the referenceable area, so that even if the width or height of the reference block (RefBlk) is the maximum width or height of the block to be decoded, as shown in Figures 5 and 6, the reference block (RefBlk) is contained within the referenceable area, as shown in Figures 5 and 6.

[0070] Furthermore, since 8 pixels are secured in the reference image direction from the boundary of the referenceable area, even when the motion vector references a decimal pixel precision position, reference pixels are secured to the left, right, and top and bottom of the reference block for the motion compensation interpolation filter (8 taps in Non-Patent Document 1, 12 taps in Non-Patent Document 2) applied when generating motion compensation pixels (in the case of 8 taps, 3 pixels are required on the left and top of the block, and 4 pixels are required on the right and bottom; in the case of 12 taps, 5 pixels are required on the left and top of the block, and 6 pixels are required on the right and bottom).

[0071] The motion compensation unit 208 may change the number of pixels in the repeated padding area of ​​the referenceable area according to the number of taps (filter length) of the motion compensation interpolation filter.

[0072] Specifically, the motion compensation unit 208 may specify the size of the repeated padding area as a multiple (power) of the minimum width or height of the coding block (block to be decoded), other than the above-mentioned 16 pixels.

[0073] For example, when the minimum width of the coding block (block to be decoded) is 4 pixels, the motion compensation unit 208 may set the size of the repeated padding area to a fixed value such as 8 pixels, 32 pixels, or 64 pixels.

[0074] Alternatively, the number of pixels in the width or height of the repeated padding area may be configured by adding a number of pixels greater than the number of taps (filter length) T of the motion compensation interpolation filter, i.e., T+α, to the maximum width or maximum height of the block to be decoded.

[0075] The number of pixels T+α, which is larger than the number of taps (filter length) T of this motion compensation interpolation filter, may be set to a fixed value between 1.0 and 2.0 times T, such as T+α=kT (e.g., k=1.1, 1.2, 1.3, 1.4, 1.5, 2.0). In addition, when referencing outside the motion vector referenceable area, the motion compensation unit 208 may change the settings in advance depending on the number of taps (filter length) of the motion compensation interpolation filter so that the clip position of the motion vector from the boundary of the reference image or the boundary of the referenceable area (8 pixels in the examples of Figures 5 and 6) can be accommodated by the number of taps of the motion compensation interpolation filter.

[0076] Specifically, when the number of pixels in the width or height of the repeated padding area is calculated by adding a number of pixels greater than the number of taps (filter length) of the motion compensation interpolation filter to the maximum width or maximum height of the block to be decoded, as shown in Figures 5 and 6 above, the motion compensation unit 208 may clip the clip position of the motion vector to a position that is (T + α) / 2 pixels away from the boundary of the reference image or the boundary of the referenceable area, respectively.

[0077] Secondly, a method of clipping a motion vector when template matching is enabled and the motion vector references an area outside the referenceable area will be described with reference to FIGS.

[0078] 7 to 10 are diagrams showing an example of clipping a motion vector when template matching is enabled and the motion vector references an area outside the referenceable area.

[0079] Figures 7 and 8 show examples of clipping methods when a motion vector references an area outside the referenceable area (consisting of a reference image and a repeated padding area) to the right of the reference image, as disclosed in Non-Patent Document 1.

[0080] In the cases shown in Figures 7 and 8, template matching is valid in both cases, so unlike the case shown in Figure 5, the reference position of the motion vector (Initial MV) is not translated (clipped) to a position 8 pixels away from the boundary of the reference image, but rather the left end position of the search range of template matching with the motion vector as the origin is translated (clipped) to the same position.

[0081] This clipping operation also applies to the case where a motion vector in the downward direction of the reference image refers to a picture outside the reference image.

[0082] Here, in the case shown in Figure 7, the template matching search range (8 pixels x 2 + width of RefBlk) is the number of pixels that can be accommodated from the clip position of the repeated padding area to the boundary of the referenceable area, that is, the template matching search range does not exceed the boundary of the referenceable area in the outer direction, as shown in Figure 7, so the template matching search can be performed.

[0083] On the other hand, in the case shown in Figure 8, the template matching search range (8 pixels x 2 + width of RefBlk) is a number of pixels that cannot be accommodated from the clip position of the repeated padding area to the boundary of the referenceable area, that is, the template matching search range exceeds the boundary of the referenceable area in the outward direction, as shown in Figure 8, so template matching search cannot be performed.

[0084] In the case shown in FIG. 8, when the reference position of the motion vector is a position with decimal pixel accuracy, the search range of template matching may extend beyond the boundary of the referenceable area in the direction outside the boundary, as in the case of FIG. 7, depending on the number of taps of the motion compensation interpolation filter.

[0085] The operation of the search range when template matching is enabled is the same even when the motion vector refers to the outside of the reference image in the downward direction of the reference image.

[0086] Figures 9 and 10 show examples of clipping methods when a motion vector references an area outside the referenceable area (consisting of a reference image and a repeated padding area) to the left of the reference image, as disclosed in Non-Patent Document 1.

[0087] In the cases shown in Figures 9 and 10, template matching is enabled in both cases, so unlike the case shown in Figure 6, the reference position of the motion vector is not translated (clipped) to a position 8 pixels away from the boundary of the referenceable area, but rather the left end position of the template matching search range with the motion vector as the origin is translated (clipped) to the same position.

[0088] This clipping operation also applies to the case where a motion vector in the upward direction of the reference image refers to a picture outside the reference image.

[0089] Here, in the case shown in Figure 9, the template matching search range (8 pixels x 2 + width of RefBlk) is the number of pixels that can be accommodated from the clip position of the repeated padding area to the boundary of the reference image area, that is, the template matching search range does not extend inward toward the boundary of the reference image, as shown in Figure 9, so template matching search can be performed.

[0090] On the other hand, in the case shown in Figure 10, the template matching search range (8 pixels x 2 + width of RefBlk) is a number of pixels that cannot be contained from the clip position of the repeated padding area to the boundary of the reference image area. In other words, the template matching search range extends beyond the inside of the reference image boundary as shown in Figure 10. Therefore, template matching search can be performed, but only in the area within the template matching search range that extends beyond the inside of the reference image boundary will the pixel values ​​at the reference image boundary have pixel values ​​that are not repeated.

[0091] In other words, for pixel values ​​within the search range of template matching, non-uniformity in pixel values ​​occurs in the left-right direction inside the boundary of the reference image mentioned above, so a motion vector difference (MVD) in the left-right direction is generated, and the motion vector is corrected in the left-right direction by template matching (in this case, if the direction does not go beyond the inside boundary of the reference image, only MVD is generated in the up-down direction).

[0092] The operation of the search range when template matching is enabled is the same even when the motion vector refers to the outside of the reference image in the upward direction of the reference image.

[0093] Figures 6 to 10 explain the issues that arise when template matching is enabled and the motion vector references an area outside the referenceable area, based on the case where there is no motion compensation padding area as shown in Non-Patent Document 2 described in Figure 3.

[0094] On the other hand, the same problem occurs in the case of the motion compensation padding area in Non-Patent Document 2. This is because in Non-Patent Document 2, the position at which the motion vector is clipped when the motion vector references an area outside the referenceable area is set on the assumption that a motion compensation padding area exists.

[0095] 5, the motion vector is clipped not to the boundary of the reference image but to a position 8 pixels away from the boundary of the motion compensation padding area toward the boundary of the referenceable area. Note that this clipping operation is also the same when the motion vector references an area outside the reference image in the upward direction of the reference image. 6, the motion vector is clipped not at the boundary of the reference image but at a position that is 8 pixels away from the boundary of the motion compensation padding area plus the maximum width of the block to be decoded in the direction of the boundary of the referenceable area (i.e., a position that is 8 pixels away from the boundary of the referenceable area in the direction of the reference image). Note that this clipping operation is also the same when the motion vector references an area outside the reference image in the upward direction of the reference image.

[0096] To solve this problem, the motion compensation unit 208 selects at least one of the following solutions 1 to 3.

[0097] <Solution 1: Controlling / Limiting the Search Range of Template Matching> When a motion vector references a region outside the referenceable region, the motion compensation unit 208 controls the search range of template matching depending on whether template matching is valid or not.

[0098] Alternatively, when a motion vector references an area outside the referenceable area, the motion compensation unit 208 may control the search range of template matching depending on whether template matching is valid and the width, height, or size of the reference block.

[0099] Specifically, the motion compensation unit 208 may control the search range of template matching by comparing the width, height, or size of the reference block and the search range of template matching with a predetermined number of pixels.

[0100] Here, the specified number of pixels may be, for example, the maximum width, maximum height, or maximum size of the block to be coded (block to be decoded), or may be a predetermined number of pixels T+α (8 pixels in the examples shown in Figures 5 to 10 above).

[0101] Specifically, the motion compensation unit 208 maintains the search range of template matching when the following formula is satisfied, and limits the search range of template matching when the following formula is not satisfied.

[0102] CuWidth+TmPmWidth <maxCUwidth+(T+α) / 2 Here, CuWidth is the width of the block to be decoded (reference block), TmPmWidth is the search width of template matching (±8 pixels = 16 pixels in Non-Patent Document 2), and maxCUwidth is the maximum width of the block to be decoded (reference block). Also, T+α is the value obtained by adding a predetermined number of pixels α to the number of taps (filter length) of the interpolation filter, as described above.

[0103] When the motion compensation unit 208 uses the method of reducing the search range of template matching, it prohibits all searches in directions in which the motion vector references outside the referenceable area, as shown in Figure 11 (in the example of Figure 11, the search width in the left and right directions is limited).

[0104] That is, the search range of template matching is limited to the vertical direction, and therefore the MVD generated by template matching has components only in the vertical direction, as shown in FIG.

[0105] FIG. 11 shows an example in which the motion vector references the right side outside the referenceable area. However, even in the case in which the motion vector references the left side outside the referenceable area, by limiting the search range of template matching in the same way, it is possible to avoid cases in which template matching is performed beyond the boundary of the reference image with the motion vector as the origin.

[0106] Similarly, the motion compensation unit 208 can deal with cases where the motion vector references the upper or lower side outside the referenceable area by applying the same method.

[0107] Alternatively, the motion compensation unit 208 may reduce the number of pixels in the search range of template matching, rather than restricting (prohibiting) the search range of template matching in a lump only in the reference direction outside the referenceable area as described above.

[0108] Furthermore, the motion compensation unit 208 may control the reduction number of pixels in template matching according to the width, height, or size (number of pixels) of the block to be decoded (reference block).

[0109] <Solution 2: Correct the clip position of the motion vector> When a motion vector references an area outside the referenceable area, the motion compensation unit 208 controls the clip position of the motion vector depending on whether template matching is valid or not.

[0110] Specifically, when template matching is not effective, the motion compensation unit 208 translates (clips) the reference position of the motion vector to the position described above with reference to FIGS.

[0111] On the other hand, when template matching is valid, the motion compensation unit 208 translates (clips) the reference position of the motion vector to the positions shown in Figures 12 and 13 (i.e., the boundary of the reference image in Figure 2, and the boundary of the referenceable area in Figure 13).

[0112] Here, if each clip position has sub-pixel accuracy, the motion compensation unit 208 may (always) round the clip position to a position with integer pixel accuracy, which eliminates the need for a motion compensation interpolation filter and allows the template search range to be accommodated within the repeated padding area, as shown in Figures 12 and 13.

[0113] In the examples of Figures 12 and 13, the search range for template matching is 16 pixels (±8 pixels), and the number of pixels in the area excluding the maximum width of the block to be coded (block to be decoded) in the repeated padding area is 16 pixels (repeated padding reserve area). However, if the search range for template matching is more than 16 pixels, the same solution can be achieved by setting the repeated padding reserve area to 16 pixels or more to match the search range for template matching.

[0114] Alternatively, the motion compensation unit 208 may clip the motion vector to the above-mentioned position, regardless of whether template matching is valid or not.

[0115] <Solution 3: Expanding the repeat padding area> The motion compensation unit 208 extends the repeated padding area if template matching is effective.

[0116] Specifically, the motion compensation unit 208 expands the repeated padding area by the number of pixels in the search range of template matching relative to the specified area when template matching is disabled, in accordance with the search range of template matching.

[0117] According to the image decoding device 200 of this embodiment, it is possible to prevent template matching in which the motion vector goes beyond the referenceable area including the reference image.

[0118] The motion compensation unit 208 may determine whether template matching is effective based on control information decoded or estimated by the decoding unit 201 .

[0119] Specifically, the control information may include a flag for each block to be coded (block to be decoded) for specifying whether template matching is valid or not, and the motion compensation unit 208 can make a determination based on the value of this flag.

[0120] For example, the motion compensation unit 208 may determine that template matching is enabled when the value of the flag is 1, and that template matching is disabled when the value of the flag is 0.

[0121] In addition, the control information may include a flag for specifying whether template matching is enabled or disabled on a sequence basis, a picture basis, or a slice basis before the unit of the block to be coded (block to be decoded), and the decoding unit 201 may control whether to decode based on the decoding result of the flag of the previous layer (slice for blocks, picture for slices).

[0122] The above-described image decoding device 200 may be realized as a program that causes a computer to execute each function (each step). [Industrial Applicability]

[0123] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which will contribute to the achievement of Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]

[0124] 200...Image decoding device 201...Decoding unit 202...Inverse quantization section 203...Inverse conversion unit 204...Intra prediction unit 206...adder 207...Storage section 208...Motion compensation unit 210...Code input section 220...Image output unit

Claims

1. An image decoding device, a decoding unit that decodes the control information; an intra prediction unit that generates a first predicted pixel based on a decoded pixel and the control information; a storage unit that stores the decoded pixels; a motion compensation unit that generates a second predicted pixel based on the stored decoded pixel and the control information, An image decoding device characterized in that the motion compensation unit controls the clip position of the motion vector depending on whether template matching is valid or not when the motion vector references a reference area outside the referenceable area including the reference image.

2. An image decoding device, a decoding unit that decodes the control information; an intra prediction unit that generates a first predicted pixel based on a decoded pixel and the control information; a storage unit that stores the decoded pixels; a motion compensation unit that generates a second predicted pixel based on the stored decoded pixel and the control information, The image decoding device is characterized in that, when a motion vector references a region outside a referenceable region including a reference image, the motion compensation unit limits the search range of the template matching depending on whether template matching is valid and the width or height of the block to be decoded.

3. 2. The image decoding device according to claim 1, wherein the motion compensation unit always rounds the reference position of the motion vector to an integer pixel precision position when controlling the clip position of the motion vector depending on whether the template matching is valid or not.

4. An image decoding method, comprising: a step A of decoding control information; a step B of generating a first predicted pixel based on the decoded pixel and the control information; a step C of storing the decoded pixels; a step D of generating a second predicted pixel based on the stored decoded pixel and the control information, An image decoding method characterized in that in step D, when a motion vector references a region outside a referenceable area including a reference image, the clip position of the motion vector is controlled depending on whether template matching is valid or not.

5. A program that causes a computer to function as an image decoding device, The image decoding device comprises: a decoding unit that decodes the control information; an intra prediction unit that generates a first predicted pixel based on a decoded pixel and the control information; a storage unit that stores the decoded pixels; a motion compensation unit that generates a second predicted pixel based on the stored decoded pixel and the control information, A program characterized in that the motion compensation unit controls the clip position of the motion vector depending on whether template matching is valid or not when the motion vector references a region outside the referenceable region including the reference image.

Images