Image decoding device, image decoding method, and program

Abstract

To improve encoding efficiency.SOLUTION: In an image decoding device 200 according to the present invention, an inter-prediction unit 205 performs chained motion vector prediction of recursively adding all motion vectors derived by searching for motion vectors accumulated in a decoding picture buffer 207, in motion vectors of positions that are spatially or temporally adjacent or proximate to each other, to a decoding target block stored in a merge candidate list that stores candidates for a motion vector, thereby generating a chained motion vector, and deriving the chained motion vector as a candidate for a motion vector (merge candidate) of the decoding target block.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image decoding apparatus, an image decoding method, and a program.

Background Art

[0002] Non-Patent Document 1 and Non-Patent Document 2 disclose inter prediction.

[0003] In inter prediction, predicted pixels of a block to be decoded are generated from decoded pixels (reference pixels) in a decoded picture (reference picture) different from the picture to be decoded.

[0004] Also, in inter prediction, using control information, a motion vector of a block to be decoded necessary for generating predicted pixels of the block to be decoded is selected from candidates of a plurality of motion vectors at positions spatially or temporally adjacent or close to the block to be decoded.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] In the inter prediction in Non-Patent Document 1 and Non-Patent Document 2, candidates of motion vectors that can be selected are limited to motion vectors at positions spatially or temporally adjacent or close to the block to be decoded, so there is a problem that there is room for improvement in coding efficiency.

[0007] Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide an image decoding apparatus, an image decoding method, and a program with high coding efficiency.

Means for Solving the Problems

[0008] A first feature of the present invention is an image decoding apparatus including a decoding unit that variably decodes coded information and outputs a quantization value and control information, an inverse quantization unit that inverse quantizes the quantization value and outputs a transform coefficient, an inverse transform unit that inverse transforms the transform coefficient and outputs a prediction residual pixel, an intra prediction unit that generates an intra prediction pixel from the control information and decoded pixels, a decoded picture buffer that stores the decoded pixels, an inter prediction unit that generates an inter prediction pixel from the control information and the decoded pixels stored in the decoded picture buffer, and an adder that adds at least one of the intra prediction pixel and the inter prediction pixel to the prediction residual pixel to generate the decoded pixel. The inter prediction unit recursively searches for motion vectors stored in the decoded picture buffer for motion vectors at positions spatially or temporally adjacent or close to a decoding target block stored in a merge candidate list storing motion vector candidates, and adds all the motion vectors derived thereby to generate a chained motion vector, and the gist is to perform chained motion vector prediction to derive the chained motion vector as a candidate (merge candidate) for the motion vector of the decoding target block.

[0009] A second feature of the present invention is an image decoding method, comprising: step A of performing variable-length decoding on coded information to output a quantization value and control information; step B of inverse quantizing the quantization value to output a transform coefficient; step C of inverse-transforming the transform coefficient to output a prediction residual pixel; step D of generating an intra prediction pixel from the control information and decoded pixels; step E of storing the decoded pixels in a decoded picture buffer; step F of generating an inter prediction pixel from the control information and the decoded pixels stored in the decoded picture buffer; and step G of adding at least one of the intra prediction pixel and the inter prediction pixel to the prediction residual pixel to generate the decoded pixel. In step F, a chain-like motion vector prediction is performed, in which all motion vectors derived by recursively searching for motion vectors at positions spatially or temporally adjacent or close to a decoding target block stored in a merge candidate list storing motion vector candidates are added to generate a chain-like motion vector, and the chain-like motion vector is derived as a candidate (merge candidate) for the motion vector of the decoding target block.

[0010] A third feature of the present invention is a program that causes a computer to function as an image decoding device, the image decoding device including: a decoding unit that variably decodes code information and outputs a quantization value and control information; an inverse quantization unit that inverse quantizes the quantization value and outputs a transform coefficient; an inverse transform unit that inverse transforms the transform coefficient and outputs a prediction residual pixel; an intra prediction unit that generates an intra prediction pixel from the control information and decoded pixels; a decoded picture buffer that stores the decoded pixels; an inter prediction unit that generates an inter prediction pixel from the control information and the decoded pixels stored in the decoded picture buffer; and an adder that adds at least one of the intra prediction pixel and the inter prediction pixel to the prediction residual pixel to generate the decoded pixel. The inter prediction unit recursively searches for motion vectors stored in the decoded picture buffer for motion vectors at positions spatially or temporally adjacent or close to a decoding target block stored in a merge candidate list storing motion vector candidates, and adds all the derived motion vectors to generate a chained motion vector, and the gist is to perform chained motion vector prediction to derive the chained motion vector as a candidate (merge candidate) for the motion vector of the decoding target block.

Advantages of the Invention

[0011] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program with high encoding efficiency.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components or the like, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0014] <First Embodiment> Hereinafter, with reference to FIGS. 1 to 24, an image decoding apparatus 200 according to the present embodiment will be described.

[0015] The image decoding apparatus 200 according to the first embodiment of the present invention targets various image signals (hereinafter, images). For example, the image decoding apparatus 200 according to the present embodiment targets a YUV (YCbCr) image composed of luminance pixels and chrominance pixels, an RGB image composed of RGB pixels, or a monochrome image. Here, each pixel constituting the image has a discrete value (pixel value) with a predetermined bit width.

[0016] FIG. 1 is a diagram showing an example of a functional block of the image decoding apparatus 200 according to the present embodiment.

[0017] As shown in FIG. 1, the image decoding apparatus 200 includes a code input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transformation unit 203, an intra prediction unit 204, an inter prediction unit 205, an adder 206, a decoded picture buffer 207, and an image output unit 220.

[0018] Hereinafter, in the description of the functions of each unit, the places described as "pixel" may be blocks (units) composed of pixels, tree blocks which are the maximum size of the blocks, slices, tiles, or images (pictures) larger than the tree blocks.

[0019] Although not shown in FIG. 1, the image decoding apparatus 200 may include a loop filter for correcting the pixel values of the decoded pixels between the adder 206 and the decoded picture buffer 207.

[0020] The code input unit 210 is configured to acquire the coded information coded by the image coding apparatus.

[0021] The decoding unit 201 is configured to decode the control information and the quantization values from the coded information input from the code input unit 210. For example, the decoding unit 201 is configured to output the control information and the quantization values by performing variable length decoding on such coded information.

[0022] Here, the quantization values are sent to the inverse quantization unit 202, and the control information is sent to the decoded picture buffer 207, the intra prediction unit 204, and the inter prediction unit 205. Note that such control information (syntax) includes information necessary for controlling the decoded picture buffer 207, the intra prediction unit 204, the inter prediction unit 205, etc., and may include header information such as a sequence parameter set, a picture parameter set, a picture header, and a slice header.

[0023] The inverse quantization unit 202 is configured to inverse quantize the quantization values sent from the decoding unit 201 to obtain inverse quantized transform coefficients. Such transform coefficients are sent to the inverse transformation unit 203.

[0024] The inverse transformation unit 203 is configured to perform inverse transformation on the transformation coefficients sent from the inverse quantization unit 202 to obtain a decoded prediction residual. Such a prediction residual is sent to the adder 206.

[0025] The intra prediction unit 204 is configured to generate an intra prediction pixel based on the decoded pixel and the control information sent from the decoding unit 201. Here, the decoded pixel is obtained via the adder 206 and accumulated in the decoded picture buffer 207. Also, the intra prediction pixel is a prediction pixel for addition with the prediction residual by the adder 206. Note that the intra prediction pixel is sent to the adder 206.

[0026] Here, the prediction performed by the intra prediction unit 204 may be the intra prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2. Alternatively, the prediction performed by the intra prediction unit 204 may be the intra block copy or intra template matching prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0027] The decoded picture buffer 207 cumulatively stores the decoded pixel sent from the adder 206 and the inter prediction information corresponding to the decoded pixel sent from the inter prediction unit 205.

[0028] Here, the decoded pixel stored in the decoded picture buffer 207 and the inter prediction information corresponding to such a decoded pixel are referred to by the inter prediction unit 205. Details of the inter prediction information will be described later.

[0029] The inter prediction unit 205 is configured to generate an inter prediction pixel based on the decoded pixel and the inter prediction information obtained by referring to the decoded picture buffer 207 and the control information sent from the decoding unit 201.

[0030] Furthermore, the inter prediction unit 205 sends the generated inter prediction pixels to the adder 206, and sends the inter prediction information used for the generation of such inter prediction pixels to the decoded picture buffer 207 as corresponding to the pixels to be decoded.

[0031] Here, the prediction performed by the inter prediction unit 205 may be the inter prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2. Alternatively, the prediction performed by the inter prediction unit 205 may be the intra block copy disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0032] The adder 206 is configured to calculate decoded pixels by adding at least one of the prediction residual sent from the inverse transform unit 203 and the input intra prediction pixels and inter prediction pixels. Such decoded pixels are sent to the image output unit 220, the decoded picture buffer 207, and the intra prediction unit 204.

[0033] The image output unit 220 is configured to output the decoded pixels transmitted from the adder 206.

[0034] Hereinafter, the roles of the inter prediction unit 205 and the decoded picture buffer 207, which are characteristic configurations of the image decoding apparatus 200 according to the present embodiment, will be described by taking as an example the case where the prediction performed by the inter prediction unit 205 is inter prediction.

[0035] <Basic Roles of Inter Prediction Unit 205 and Decoded Picture Buffer 207> The basic role of the inter prediction unit 205 is to derive one or more motion vectors (Mv: Motion vector) for the block to be decoded in order to accurately predict the block to be decoded in the subsequent adder 206, and to perform an inter prediction, that is, to predict the pixels (hereinafter, referred to as reference pixels) of the decoded block (hereinafter, referred to as reference block) referred to by Mv in a decoded picture (hereinafter, referred to as reference picture) different from the decoded picture to which the block to be decoded belongs.

[0036] FIG. 2 shows a conceptual diagram of inter prediction. Specifically, FIG. 2 shows two motion vectors MvL0 derived by the inter prediction unit 205 in order to generate inter prediction pixels of a decoding target block (CurBlk) within a decoding target picture (CurPic) in inter prediction. k and MvL1 k Based on these, two different reference pictures (RerPicL0 k and RerPicL1 k ) are shown as examples where the reference blocks (RerBlkL0 k and RerBlkL1 k ) within are referenced.

[0037] Here, L0 and L1 indicate the numbers of lists (reference picture lists) in which reference pictures that can be referenced from the decoding target picture are described, and k corresponds to the value of control information (index) for specifying the motion vector described in the motion vector derivation method described later.

[0038] Similar to Non-Patent Document 1 and Non-Patent Document 2, the image decoding apparatus 200 according to the present embodiment may derive list 0 (L0) and list 1 (L1) as two different reference picture lists for each decoding target picture.

[0039] Here, the decoding picture buffer 207 may derive the reference picture list for each decoding target picture based on the control information decoded by the decoding unit 201.

[0040] For example, the control information may include information such as the number of reference pictures to be included in the reference picture list and candidates for reference pictures (picture reference structure) for the decoding target picture.

[0041] Details of the motion vector derivation method in the inter prediction unit 205 will be described later. The outline of the inter prediction pixel generation method in the inter prediction unit 205 is as follows.

[0042] The inter prediction unit 205 may generate an inter prediction pixel from a single reference block, or as shown in FIG. 2, may generate an inter prediction pixel using two reference blocks, or may generate an inter prediction pixel using two or more reference blocks.

[0043] Here, as a method for the inter prediction unit 205 to generate an inter prediction pixel using two reference blocks, the following method may be applied.

[0044] As an example, the inter prediction unit 205 may generate an inter prediction pixel by simply averaging the pixel values of two reference blocks.

[0045] As another example, the inter prediction unit 205 may generate an inter prediction pixel by weighted averaging the pixel values of two reference blocks in block units using a predetermined weight value.

[0046] As a technique of such block unit weighted averaging, the block unit weighted bi-prediction (BCW) disclosed in Non-Patent Document 1 and Non-Patent Document 2 may be applied to the inter prediction unit 205.

[0047] Note that in Non-Patent Documents 1 and 2, in BCW, a plurality of weight values including a simple average, that is, a weight value of 1:1, can be selected, and such a weight value is specified by the value of an internal parameter (bcwIdx, the third internal parameter).

[0048] Here, the inter prediction unit 205 derives bcwIdx by inheritance from control information or the decoded picture buffer 207. Details of the inheritance will be described later.

[0049] As another example, the inter prediction unit 205 may generate an inter prediction pixel by dividing the block to be decoded into two by a predetermined straight line, and performing weighted averaging on the pixel values of the reference blocks corresponding to the divided regions using a weight value corresponding to the distance from the dividing line.

[0050] The inter prediction unit 205 may apply a Geometric Partitioning Mode (GPM) disclosed in Non-Patent Document 1 and Non-Patent Document 2 as a technique of weighted average for each region obtained by linearly bisecting such a block.

[0051] Here, when the motion vector refers to a fractional pixel accuracy position within the reference block, the inter prediction unit 205 may generate an inter prediction pixel using an interpolation filter for deriving the reference pixel value at such a fractional pixel accuracy position from the reference pixel values at the integer pixel accuracy positions in the vertical, horizontal, upper, and lower directions of such a fractional pixel position.

[0052] Furthermore, when generating an inter prediction pixel, the inter prediction unit 205 may adaptively apply two interpolation filters with different cut-off frequencies for inter prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0053] Among such two interpolation filters, the interpolation filter with a low cut-off frequency is called a Switchable Interpolation Filter (SIF).

[0054] Similar to Non-Patent Document 1 and Non-Patent Document 2, when the motion vector refers to a half pixel accuracy position and an internal parameter (hpelIfIdx, the first internal parameter) for specifying whether or not to apply the SIF indicates that the SIF is to be applied, the inter prediction unit 205 may apply the SIF.

[0055] Note that the inter prediction unit 205 derives hpelIfIdx by inheritance from control information or the decoded picture buffer 207. Details of the inheritance will be described later.

[0056] On the other hand, when the motion vector refers to a fractional pixel accuracy position, the inter prediction unit 205 may round (clip) the reference destination of the motion vector to the nearest integer pixel accuracy position of such fractional pixel accuracy position, and generate an inter prediction pixel without applying an interpolation filter.

[0057] Also, when generating the final inter prediction pixel, the inter prediction unit 205 may apply local illumination compensation (LIC) disclosed in Non-Patent Document 2.

[0058] LIC is a technique for correcting a reference pixel value in a reference block by linear prediction based on decoded pixels adjacent to each of the block to be decoded and the reference block.

[0059] Similar to Non-Patent Document 1 and Non-Patent Document 2, the inter prediction unit 205 may apply LIC when an internal parameter (licFlag, the second internal parameter) for controlling whether or not to apply LIC indicates that LIC is to be applied.

[0060] Note that the inter prediction unit 205 derives the licFlag by inheritance from control information or the decoded picture buffer 207. Details of the inheritance will be described later.

[0061] Also, when generating the final inter prediction pixel, the inter prediction unit 205 may apply multi-hypothesis prediction (MHP) disclosed in Non-Patent Document 2.

[0062] MHP is a technique for generating a final inter prediction pixel by weighted average from reference pixel values of at least three or more reference blocks.

[0063] Similar to Non-Patent Document 2, the inter prediction unit 205 may apply MHP when an internal parameter (mhpFlag, the fourth internal parameter) for specifying whether or not to apply MHP indicates that MHP is to be applied.

[0064] Note that the inter prediction unit 205 derives the mhpFlag based on control information or inheritance from the decoded picture buffer 207. Details of the inheritance will be described later.

[0065] The inter prediction unit 205 sends the inter predicted pixels for the block to be decoded to the adder 206, and sends to the decoded picture buffer 207 the motion vector used to generate such inter predicted pixels, an index (reference index) for specifying the reference picture referred to by the motion vector within the reference picture list, and a counter (POC: Picture Order Count) representing the picture output order of the reference picture corresponding to the reference index. Hereinafter, the motion vector, reference index, and POC corresponding to the reference index used to generate these inter predicted pixels are referred to as inter prediction information. Further, the inter prediction unit 205 may send such inter prediction information including at least any one or all of the internal parameters bcwIdx, hpelIfIdx, licFlag, or mhpFlag that change the above-described method for generating the inter predicted pixels.

[0066] The decoded picture buffer 207 accumulates the inter prediction information corresponding to the decoded pixels of the block to be decoded in addition to the decoded pixels of the block to be decoded sent from the adder 206.

[0067] The decoded picture buffer 207 may accumulate such inter prediction information in a predetermined size (number of pixels). For example, the decoded picture buffer 207 may accumulate such inter prediction information in units of the minimum size (4×4 pixels) of the block to be decoded as disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0068] By the decoded picture buffer 207 accumulating such inter prediction information in the minimum size of the block to be decoded, the inter prediction accuracy when such inter prediction information is referred to from the block to be decoded that is to be decoded in the future can be improved.

[0069] The decoded picture buffer 207 accumulates decoded pixels in units of pictures to be decoded. Alternatively, the decoded picture buffer 207 may accumulate decoded pixels in units of slices to be decoded. Hereinafter, a case where decoded pixels are accumulated in units of pictures to be decoded will be described as an example.

[0070] When it is specified that the accumulated decoded pixels in the unit of the picture to be decoded in the decoded picture buffer 207 and the inter-prediction information corresponding to the decoded pixels are not referred to as a reference picture and a reference block from a future picture to be decoded or a block to be decoded, the decoded pixels in the unit of the picture to be decoded and the inter-prediction information corresponding to the decoded pixels may be sequentially deleted. According to such a configuration, the amount of information accumulated in the decoded picture buffer 207 can be reduced.

[0071] The decoded picture buffer 207 may derive the number of pictures to be decoded to be accumulated and the reference picture structure between the pictures to be decoded based on the control information decoded by the decoding unit 201.

[0072] As a modification, when there are inter-prediction pixels in a predetermined area outside the picture to be decoded in addition to the picture to be decoded, the decoded picture buffer 207 may additionally accumulate such inter-prediction pixels.

[0073] FIG. 3 is a diagram for explaining reference pixel values referred to when an area outside the picture to be decoded accumulated in the decoded picture buffer 207 disclosed in each of Non-Patent Document 1 and Non-Patent Document 2 is referred to from a future block to be decoded.

[0074] As shown in FIG. 3, in Non-Patent Document 1 (VVC: Versatile Video Coding), when an area outside the picture to be decoded is referred to from a future block to be decoded within a predetermined range, a pixel value at the nearest decoded picture boundary is copied to generate a reference pixel value.

[0075] In VVC, as such a predetermined range (padding area), a range obtained by adding 16 pixels to the maximum size (number of pixels) of the width of a block to be decoded is specified.

[0076] In Non-Patent Document 2 (ECM: Enhanced Compression Model), as shown in FIG. 3, between outside the picture to be decoded and the padding area, only a predetermined range is accumulated as an area (inter prediction padding area) where the inter prediction pixel values of the block to be decoded can be referred to from future blocks to be decoded.

[0077] Thus, even if outside the picture to be decoded is referred to from a future block to be decoded, since there are inter prediction pixel values of the block to be decoded, the inter prediction accuracy of the future block to be decoded is improved.

[0078] FIG. 4 is a conceptual diagram in which the inter prediction padding area described in FIG. 3 is generated.

[0079] As shown in FIG. 4, when an adjacent block of a reference block corresponding to a block to be decoded has inter prediction pixels, even if the block to be decoded is in contact with the picture to be decoded, since inter prediction pixels can be generated outside the picture to be decoded, the decoded picture buffer 207 can accumulate the inter prediction padding area shown in FIG. 3.

[0080] <Method for Deriving Motion Vector in Inter Prediction Unit> Hereinafter, with reference to FIGS. 5 to 17, a method for deriving a motion vector in the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment will be described.

[0081] FIG. 5 is a diagram showing an example of a merge candidate list which is a method for deriving a motion vector in the inter prediction unit 205.

[0082] The merge candidate list is a list in which candidates for inter prediction information (i.e., merge candidates) that the inter prediction unit 205 constructs to derive motion vectors in the merge (merge mode) disclosed in Non-Patent Document 1 and Non-Patent Document 2 are stored.

[0083] The inter prediction unit 205 extracts a merge candidate from among a plurality of merge candidates in such a merge candidate list and uses it as inter prediction information for generating an inter prediction pixel based on an index (merge_idx) for specifying the merge candidate in the merge candidate list sent from the decoding unit 201.

[0084] The upper limit of the number of merge candidates stored in the merge candidate list may be a fixed value or a variable value. In the case of a variable value, for example, like Non-Patent Document 1 and Non-Patent Document 2, control information is provided in a sequence parameter set or the like, and based on such control information, such a variable value can be set.

[0085] The inter prediction unit 205 specifies, from within the merge candidate list, the inter prediction information to be used for generating an inter prediction pixel based on one or more merge_idx sent from the decoding unit 201.

[0086] For example, for the above-described GPM, the inter prediction unit 205 may decode one merge_idx for each region obtained by linearly dividing the block to be decoded and derive inter prediction information.

[0087] As shown in FIG. 5, the inter prediction unit 205 stores inter prediction information (merge candidates) corresponding to L0 and L1 of the merge candidate list.

[0088] Note that, in the example of FIG. 5, the inter prediction unit 205 describes only the motion vector and the reference index as the inter prediction information. However, in addition to these, at least one or more or all of bcwIdx, hpelIfIdx, licFlag, or mhpFlag, which are internal parameters for changing the above-described method for generating the inter prediction pixels, may be stored.

[0089] The inter prediction unit 205 may search for and add, as inter prediction information, that is, as merge candidates, the inter prediction information at positions spatially or temporally adjacent or close to the block to be decoded in the merge candidate list.

[0090] As an example, the inter prediction unit 205 may apply a combination of searches for a plurality of types of merge candidates disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0091] FIG. 6 shows the names of a plurality of types of merge candidates in regular merge disclosed in Non-Patent Document 1 and Non-Patent Document 2 and the derivation order of these merge candidates.

[0092] Specifically, in Non-Patent Document 1 (VVC), the inter prediction unit 205 checks the availability of these merge candidates in the order of spatial merge candidates, temporal motion vector prediction (TMVP) candidates, history-based motion vector prediction (HMVP) candidates, and pairwise merge candidates.

[0093] In Non-Patent Document 2 (ECM), in addition to the above-described merge candidates in Non-Patent Document 1, the inter prediction unit 205 checks for the use of non-adjacent (NA) spatial merge candidates between spatial merge candidates and TMVP candidates.

[0094] The outline of each merge candidate is as follows.

[0095] FIG. 7 is a diagram showing search positions of spatial merge candidates and TMVP candidates in a block to be decoded.

[0096] As shown in FIG. 7, in spatial merge candidates, inter prediction information is searched at positions of lower left (A0), left (A1), upper right (B0), upper (B1), and upper left (B2) that are adjacent to the block to be decoded (CurBlk), respectively.

[0097] That is, in spatial merge candidates, inter prediction information at positions spatially adjacent to the block to be decoded (CurBlk) is used.

[0098] Also, in TMVP candidates, in a picture different from the block to be decoded (reference picture), inter prediction information at positions of lower right center (Col) of the block to be decoded (CurBlk) shown in FIG. 7 and lower right (H) outside the block to be decoded (CurBlk) is searched.

[0099] That is, in TMVP candidates, inter prediction information at positions temporally adjacent to the block to be decoded is used.

[0100] In HMVP candidates and non-adjacent spatial merge candidates, inter prediction information at positions spatially distant (proximate) from the block to be decoded is used.

[0101] In particular, for the HMVP candidates, a table capable of storing FIFO-style inter-prediction information is constructed. Each time new inter-prediction information is searched for, the oldest inter-prediction information is discarded from the table, and such new inter-prediction information is added to the table.

[0102] Similar to the HMVP candidates and non-adjacent spatial merge candidates disclosed in Non-Patent Document 1 and Non-Patent Document 2, the inter-prediction unit 205 of the image decoding apparatus 200 according to the present embodiment can design the position for searching inter-prediction information and the size of the table for HMVP candidates.

[0103] In addition, in the search for merge candidates in the above-described normal merge, the inter-prediction unit 205 may re-search for a new reference block position from the peripheral pixels of the reference block of the inter-prediction information to be searched for so that the difference (hereinafter referred to as the template matching cost) between the decoded pixels adjacent to the reference block based on the inter-prediction information to be searched for (template of the reference block) and the decoded pixels adjacent to the block to be decoded (template of the block to be decoded) is reduced, and correct the inter-prediction information (motion vector).

[0104] As an example of this, the inter-prediction unit 205 may apply the template matching merge disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0105] In addition, in the selection of merge candidates from the merge candidate list in the above-described normal merge, the inter-prediction unit 205 derives the L0 inter-prediction information from the control information, and selects, as the L1 merge candidate, the merge candidate for which the difference between the template of the reference block based on the L0 inter-prediction information derived from such control information and the template of the reference block based on the inter-prediction information stored in L1 of the merge candidate list is minimized.

[0106] The inter prediction unit 205 can derive the merge candidates (i.e., inter prediction information) of L1 based on the inter prediction information of L0 derived from such control information. Therefore, in addition to the control information necessary for deriving the inter prediction information of L0, it does not require decoding of merge_idx for deriving the inter prediction information of L1.

[0107] As an example, the inter prediction unit 205 may apply the Adaptive motion vector Prediction-Merge mode disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0108] As a modification example, the inter prediction unit 205 may reverse L0 and L1 in the above example. That is, the inter prediction unit 205 may derive the inter prediction information of L1 from control information and derive the inter prediction information of L0 from the merge candidate list.

[0109] In addition, the inter prediction unit 205 may construct a merge candidate list for merge for deriving inter prediction information in units of sub-blocks obtained by dividing the block to be decoded, and derive the inter prediction information.

[0110] As an example, the inter prediction unit 205 may construct a merge candidate list for Sub-block TMVP (SbTMVP) or Affine merge disclosed in Non-Patent Document 1 and Non-Patent Document 2 and derive the inter prediction information.

[0111] <1. Basic concept of beaded motion vector prediction (derivation of beaded motion vector)> The methods for deriving the inter prediction information (hereinafter, referred to as motion vectors for simplicity of explanation) by the above-described inter prediction unit are all limited to those stored at positions spatially or temporally adjacent or close to the block to be decoded.

[0112] For further improvement of the symbolization efficiency, the inter prediction unit 205 may derive, as the motion vector of the block to be decoded, a motion vector generated by adding a motion vector derived by recursively searching for motion vectors stored in the decoded picture buffer 207 with respect to motion vectors at positions spatially or temporally adjacent or close to the block to be decoded.

[0113] Here, regarding the technique of recursively (in a chain) searching for motion vectors stored in the decoded picture buffer 207 starting from motion vectors at positions spatially or temporally adjacent or close to the block to be decoded to generate a new motion vector (chained motion vector), this specification refers to it as "Chained Motion Vector Prediction (CMVP)". Hereinafter, the technical details of such chained motion vector prediction will be described in detail.

[0114] Using FIGS. 8 to 17, a method for deriving a chained motion vector in chained motion vector prediction will be described.

[0115] FIG. 8 is a diagram showing an example of a method for deriving a chained motion vector in chained motion vector prediction.

[0116] As shown in FIG. 8, the inter prediction unit 205 starts from the motion vector MvL0 k (0) of L0 corresponding to any merge candidate stored in the merge candidate list and the reference index, and recursively (in a chain) searches for motion vectors stored in the decoded picture buffer 207 to generate a new motion vector.

[0117] For example, as shown in (Equation 1) and (Equation 2), the inter prediction unit 205 can derive a new motion vector generated by chained motion vector prediction and a reference picture for the reference block RefBlk k (m) of the reference picture RefPicL0.

[0118] MvL0 k / m =MvL0k (0) + MvL0 k (1) + MvL0 k (2) + … + MvL0 k (m) (Equation 1) RefPicL0 k / m = RefPicL0 k (m) (Equation 2) That is, the new motion vector MvL0 generated by the beaded motion vector prediction k / m can be calculated by adding all the motion vectors derived by searching the motion vectors stored in the decoding picture buffer 207 recursively (in a beaded manner).

[0119] Also, the new reference picture generated by the beaded motion vector prediction is the reference picture referred to by MvL0 k / m

[0120] FIG. 9 is a diagram showing an example of a method for deriving a beaded motion vector in beaded motion vector prediction.

[0121] As shown in FIG. 9, in beaded motion vector prediction, the inter prediction unit 205 may add a block vector (a vector indicating a reference destination within the same picture) to the search target recursively in addition to the search for the motion vector.

[0122] For example, when a block vector (Bv: Block vector) is added, the new motion vector derivation formula generated by beaded motion vector prediction for the reference block RefBlk of the reference picture RefPicL0 k (m) has Bv added as shown in (Equation 3).

[0123] MvL0 k / m = MvL0 k (0) + Bv k (0) + MvL0 k (1) + MvL0 k (2) + … + MvL0 k (m) (Equation 3) ​FIG. 10 is a diagram showing an example of recursive search of motion vectors including dual prediction (i.e., inter prediction using two different motion vectors for L0 and L1). FIG. 11 is a diagram showing an example of a chained motion vector derived by recursive search of motion vectors including dual prediction.

[0124] As shown in FIGS. 10 and 11, in the chained motion vector prediction, the inter prediction unit 205 uses each search list number (L0 or L1) and each search depth (or each search count) of the motion vectors stored in the decoded picture buffer 207 to newly search and derive one or more (in the examples of FIGS. 10 and 11, up to two) motion vectors from one reference motion vector, and adds all the already derived motion vectors to each of the one or more newly derived motion vectors to obtain the chained motion vector MvL0 k / m to generate.

[0125] FIGS. 12 and 13 are diagrams showing an example of searching for chained motion vector prediction candidates after searching for multiple types of merge candidates of normal merge disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0126] As shown in FIGS. 12 and 13, the inter prediction unit 205 may search for chained motion vector prediction candidates after searching for a predetermined merge candidate.

[0127] For example, as shown in FIG. 12, in VVC, the inter prediction unit 205 may search for chained motion vector prediction candidates after searching for TMVP candidates.

[0128] Since the chained motion vector prediction searches for new motion vectors using the motion vectors of the reference picture rather than the picture to be decoded stored in the decoded picture buffer 207, it will not be effective unless TMVP is effective in the sequence to be decoded or the group of pictures to be decoded or the picture to be decoded or the slice to be decoded.

[0129] Therefore, searching for the chain-link motion vector prediction candidates after searching for the TMVP candidates can be said to be a natural design method as a method for deriving the motion vector.

[0130] As a modification example of FIG. 12, the inter prediction unit 205 may search for the chain-link motion vector prediction candidates after searching for the HMVP candidates in the VVC and ECM as shown in FIG. 13.

[0131] Since the chain-link motion vector prediction generates a new motion vector by recursively searching for the motion vector, it must be applied in a state where at least one merge candidate is stored in the merge candidate list.

[0132] As shown in FIG. 13, by searching for the chain-link motion vector prediction candidates after searching for the HMVP candidates where one or more motion vector candidates can be expected to be stored in the merge candidate list, the possibility of generating a new motion vector by the chain-link motion vector increases.

[0133] As another modification example, the inter prediction unit 205 may search for the chain-link motion vector prediction candidates after searching for the spatial merge candidates in the ECM as shown in FIG. 12.

[0134] Although the search position of the spatial merge candidates is shown in FIG. 7, for example, if the search position of the spatial merge candidates is increased, the possibility of storing one or more motion vector candidates in the merge candidate list increases, and it becomes easier to generate a new motion vector candidate by the chain-link merge candidate.

[0135] FIGS. 14 to 17 are diagrams showing an example of the position (index number) in the merge candidate list where the merge candidate which is the derivation target of the chain-link motion vector prediction candidate and the chain-link motion vector prediction candidate are added.

[0136] Note that in FIGS. 14 to 17, only the motion vectors are described as the inter prediction information stored in the merge candidate list in order to simplify the description of the technical content, but the above-described reference indexes are also stored.

[0137] In addition to the motion vectors and the reference indexes, the inter prediction unit 205 may store bcwIdx, hpelIfIdx, licFlag, and mhpFlag, which are internal parameters for changing the above-described inter prediction pixel generation method.

[0138] As shown in FIG. 14, after storing a predetermined merge candidate in the merge candidate list, the inter prediction unit 205 sequentially applies the consecutive motion vector prediction from the first merge candidate stored in the merge candidate list, and may sequentially store all the new merge candidates generated by the consecutive motion vector prediction for each merge candidate in the merge candidate list.

[0139] Here, the predetermined merge candidate may be any of the plurality of types of merge candidates in the normal merge described with reference to FIGS. 12 and 13, or may be a predetermined number (predetermined number of merge_idx) of merge candidates.

[0140] Such a predetermined number may be set as a fixed value, or may be set as a variable value in the decoding target sequence or the decoding target picture group or the decoding target picture or the decoding target slice unit using the control information of the decoding target sequence or the decoding target picture group or the decoding target picture or the decoding target slice unit.

[0141] As a modification example of FIG. 14, as shown in FIG. 15, after storing a predetermined merge candidate in the merge candidate list, the inter prediction unit 205 sequentially applies the consecutive motion vector prediction from the first merge candidate stored in the merge candidate list, and may sequentially store the new consecutive motion vectors (that is, merge candidates) generated at each search depth of the consecutive motion vector prediction for each merge candidate in the merge candidate list.

[0142] As a modification example of FIGS. 14 and 15, as shown in FIG. 16, after the first merge candidate is stored in the merge candidate list, the inter prediction unit 205 sequentially applies the bead string movement vector prediction from the merge candidate, and all of the new bead string movement vectors (that is, merge candidates) generated by the bead string movement vector prediction for each merge candidate may be sequentially stored in the merge candidate list.

[0143] As a modification example of FIGS. 14 to 16, as shown in FIG. 17, after the first merge candidate is stored in the merge candidate list, the inter prediction unit 205 sequentially applies the bead string movement vector prediction from the merge candidate, and the new bead string movement vectors (that is, merge candidates) generated at each search depth of the bead string movement vector prediction for each merge candidate may be sequentially stored in the merge candidate list.

[0144] As a modification example of FIGS. 14 to 17, the inter prediction unit 205 may combine the bead string movement vectors generated using each search depth of the bead string movement vector prediction for each merge candidate list with the bead string movement vectors generated at a search depth different from each such search depth to generate new merge candidates.

[0145] As a modification example of FIGS. 14 to 17, the inter prediction unit 205 may combine the bead string movement vectors generated using each search list number or each search depth of the bead string movement vector prediction for each merge candidate list with the bead string movement vectors generated using a search list number or a search depth different from each such search list number or each search depth to generate new merge candidates.

[0146] <2. Pruning Process for Bead String Movement Vector Prediction Candidates> Using FIG. 18, the pruning process when the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment adds a merge candidate (the above-described bead string movement vector prediction candidate) corresponding to the bead string movement vector derived by the bead string movement vector prediction to the merge candidate list will be described.

[0147] FIG. 18 is a diagram showing an example of a pruning process when the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment adds a merge candidate corresponding to a rosary movement vector derived by rosary movement vector prediction to a merge candidate list.

[0148] The inter prediction unit 205 determines whether to add a merge candidate corresponding to a rosary movement vector derived by rosary movement vector prediction based on a predetermined condition.

[0149] Specifically, as shown in FIG. 18, when the inter prediction unit 205 determines in step S000 that a predetermined condition is satisfied, it determines in step S002 not to add a merge candidate corresponding to the rosary movement vector derived by rosary movement vector prediction.

[0150] On the other hand, when the inter prediction unit 205 determines in step S000 that the predetermined condition is not satisfied, it determines in step S001 to add a merge candidate corresponding to the rosary movement vector derived by rosary movement vector prediction.

[0151] Here, for example, the predetermined condition may be that the reference pictures and motion vectors associated with at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived rosary movement vector match.

[0152] When the inter prediction unit 205 determines that the reference pictures and motion vectors associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector match, the merge candidate corresponding to the bead string motion vector is not added to the merge candidate list, that is, it is determined that the merge candidate corresponding to the bead string motion vector is pruned (to be a pruning target), thereby preventing a case where two or more identical motion vectors are stored in the merge candidate list, reducing the amount of code of merge_idx (merge index) sent to the image decoding apparatus 200, and improving the coding efficiency.

[0153] Alternatively, the predetermined condition may be that all of one or more reference pictures and motion vectors associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector match.

[0154] Alternatively, the predetermined condition may be that all of the reference pictures and motion vectors of merge candidate list number 0 (L0) and merge candidate list number 1 (L1) associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector match.

[0155] Furthermore, the predetermined condition may be that the reference pictures associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector match, and the error (Euclidean distance) of the reference positions of the motion vectors associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector is less than a predetermined threshold.

[0156] As a result, the pruning process of merge candidates corresponding to the beaded movement vector is strengthened, cases where two or more similar movement vectors are stored in the merge candidate list are more prevented, the amount of code of merge_idx sent to the image decoding device 200 is reduced, and the coding efficiency is improved.

[0157] As a further control, the inter prediction unit 205 may set a predetermined threshold value when there is one motion vector to be compared to be larger than a predetermined threshold value when there are two motion vectors to be compared.

[0158] In this way, by setting the predetermined threshold value when there is one motion vector to be compared to be larger than the predetermined threshold value when there are two motion vectors to be compared, a more accurate pruning process becomes possible.

[0159] Here, the inter prediction unit 205 may set the predetermined threshold value when there is one motion vector to be compared to 1.5 pixels.

[0160] Alternatively, the inter prediction unit 205 may set the predetermined threshold value when there is one motion vector to be compared to 1.0 pixel or 2.0 pixels.

[0161] Also, the inter prediction unit 205 may set the predetermined threshold value when there are two motion vectors to be compared to 1.0 pixel.

[0162] Alternatively, the inter prediction unit 205 may set the predetermined threshold value when there are two motion vectors to be compared to 0.5 pixel or 1.5 pixels.

[0163] Also, the above-mentioned predetermined conditions may be that internal parameters for changing the reference picture, the motion vector, and the generation method of a predetermined inter prediction pixel associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived beaded movement vector match.

[0164] Alternatively, the predetermined condition may be that the internal parameter (hpelIfIdx) that specifies the reference picture, motion vector, and whether to apply the switching interpolation filter associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector matches.

[0165] Alternatively, the predetermined condition may be that the internal parameter (licFlag) that controls the reference picture, motion vector, and whether to apply local luminance compensation associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector matches.

[0166] Furthermore, the predetermined condition may be that the internal parameter (bcwIdx) that specifies the reference picture, motion vector, and the weight value of weighted bi-prediction in units of blocks associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector matches.

[0167] By including the above-described internal parameters in the predetermined condition, the inter prediction unit 205 can leave room for improving the inter prediction accuracy by generating inter prediction pixels in different ways when the reference pictures and motion vectors to be compared match but the internal parameters are different, so an improvement in coding efficiency can be expected.

[0168] <3. Control of the reference source and reference destination when deriving the bead string motion vector> With reference to FIGS. 19 to 22, a method for controlling the reference source and reference destination when the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment derives the bead string motion vector in the bead string motion vector prediction will be described.

[0169] First, with reference to FIG. 19, a method for controlling the reference source when deriving the bead string motion vector in the bead string motion vector prediction will be described.

[0170] FIG. 19 is a diagram showing an example of a method for controlling a reference source when deriving a string-of-beads movement vector in the string-of-beads movement vector prediction by the inter prediction unit 205.

[0171] When deriving the string-of-beads movement vector in the string-of-beads movement vector prediction, the inter prediction unit 205 may set the starting point of the first movement vector constituting the string-of-beads movement vector to a part or all of the block to be decoded.

[0172] As an example of setting it to a part of the block to be decoded, as shown in FIG. 19, when deriving the string-of-beads movement vector in the string-of-beads movement vector prediction, the inter prediction unit 205 may set the starting point of the first movement vector constituting the string-of-beads movement vector to the center (C) of the block to be decoded (CurBlk).

[0173] Alternatively, as shown in FIG. 19, when deriving the string-of-beads movement vector in the string-of-beads movement vector prediction, the inter prediction unit 205 may set the starting point of the first movement vector constituting the string-of-beads movement vector to the center (C) of the block to be decoded (CurBlk), and in addition, set it to a predetermined position of the block to be decoded (CurBlk), and search whether a movement vector is accumulated at the reference destination of the movement vector in a predetermined order.

[0174] Specifically, as shown in FIG. 19, when deriving the string-of-beads movement vector in the string-of-beads movement vector prediction, the inter prediction unit 205 may set the starting point of the first movement vector constituting the string-of-beads movement vector to the center (C) of the block to be decoded (CurBlk), and in addition, set it to the four corners of the block to be decoded (CurBlk), and search whether a movement vector is accumulated at the reference destination of the movement vector in the order of the center (C), top left (TL), top right (TR), bottom left (BL), bottom right (BR) of the block to be decoded as a predetermined order.

[0175] The inter prediction unit 205 may generate a string-of-beads movement vector for each movement vector derived by searching at each point.

[0176] As a modification example, the inter prediction unit 205 may compare the differences in the adjacent decoded pixel values between the reference block of each motion vector derived by searching at each point and the block to be decoded, and limit to at least one or more motion vectors with small such differences to generate a connected motion vector.

[0177] As another modification example, the inter prediction unit 205 may derive a new motion vector for generating a connected motion vector by weighted averaging each motion vector derived by searching at each point.

[0178] Specifically, when performing weighted averaging of each motion vector, the inter prediction unit 205 may scale each motion vector based on the distance of the reference picture of each motion vector from the block to be decoded to generate a new motion vector. The scaling destination may be set to the reference picture with the minimum distance from the block to be decoded.

[0179] When deriving the connected motion vector, the inter prediction unit 205 can increase the possibility of deriving the connected motion vector by setting the starting point of the first motion vector constituting the connected motion vector not only at the center of the block to be decoded but also at the four corners of the block to be decoded. As a result, an improvement in coding efficiency can be expected.

[0180] As further control, in the connected motion vector prediction, the inter prediction unit 205 may control the starting point of the first motion vector constituting the connected motion vector according to the block size or block aspect ratio of the block to be decoded when deriving the connected motion vector.

[0181] For example, in the connected motion vector prediction, when deriving the connected motion vector, the inter prediction unit 205 may set the starting point of the first motion vector constituting the connected motion vector only at the center of the block to be decoded when the block size of the block to be decoded is equal to or less than the first size.

[0182] When the block size of the block to be decoded is less than or equal to the first size, when deriving the beaded movement vector, the first movement vector constituting the beaded movement vector is likely to be the same at the center of the block to be decoded and at the four corners of the block to be decoded.

[0183] Therefore, when the block size of the block to be decoded is less than or equal to the first size, when deriving the beaded movement vector, by setting the starting point of the first movement vector constituting the beaded movement vector only at the center of the block to be decoded (without adding the four corners of the block to be decoded), the search processing amount of the beaded movement vector can be reduced.

[0184] Alternatively, in the beaded movement vector prediction, when the inter prediction unit 205 derives the beaded movement vector, when the block size of the block to be decoded is larger than the first size, in addition to the center of the block to be decoded, the starting point of the first movement vector constituting the beaded movement vector is set at the four corners of the block to be decoded, and in the predetermined order of the center, upper left, upper right, lower left, lower right of the block to be decoded, it may be searched whether the movement vector is accumulated at the reference destination of the movement vector.

[0185] When the block size of the block to be decoded is larger than the first size, when deriving the beaded movement vector, the first movement vector constituting the beaded movement vector is likely to be different at the center of the block to be decoded and at the four corners of the block to be decoded.

[0186] Therefore, when the block size of the block to be decoded is larger than the first size, when deriving the beaded movement vector, by setting the starting point of the first movement vector constituting the beaded movement vector at the four corners of the block to be decoded in addition to the center of the block to be decoded, different beaded movement vectors are generated, and an improvement in coding efficiency can be expected.

[0187] Here, since a case where the block to be decoded is non-square is also assumed, the block size (the number of pixels in the block) of the block to be decoded may be replaced with the width and height of the block to be decoded, and the above control may be performed.

[0188] Specifically, when deriving the beaded movement vector in the beaded movement vector prediction, the inter prediction unit 205, when the width or height of the block of the beaded movement vector constituting the first movement vector is larger than a predetermined number of pixels, sets the starting point of the first movement vector constituting the beaded movement vector not only at the center of the block to be decoded but also at the four corners of the block to be decoded, and searches in the predetermined order of the center, upper left, upper right, lower left, and lower right of the block to be decoded to see if movement vectors are accumulated at the reference destinations of the movement vectors.

[0189] The inter prediction unit 205 may set any one of 8 pixels, 16 pixels, or 32 pixels as such a predetermined number of pixels.

[0190] Alternatively, when deriving the beaded movement vector in the beaded movement vector prediction, the inter prediction unit 205, when the aspect ratio of the block of the beaded movement vector constituting the first movement vector is larger than a predetermined ratio, sets the lower left and upper right of the block to be decoded in addition to the center of the block to be decoded, and searches in the predetermined order of the center, lower left, and upper right of the block to be decoded to see if movement vectors are accumulated at the reference destinations of the movement vectors.

[0191] The inter prediction unit 205 may set any one of 4, 8, or 16 as such a predetermined ratio.

[0192] Second, an example of a control method for a reference destination when deriving the beaded movement vector in the beaded movement vector prediction will be described with reference to FIGS. 20 and 21.

[0193] FIG. 20 is a diagram showing an example of a method for controlling a reference destination when deriving a rosary movement vector in rosary movement vector prediction by the inter prediction unit 205.

[0194] As shown in FIG. 20, when the inter prediction unit 205 searches for a recursive motion vector in the process of deriving a rosary motion vector, if the reference position of each motion vector refers to outside the reference picture, the motion vector may be searched at the accumulation location of the motion vector within the nearest reference picture from such a reference position.

[0195] Even if the reference position of the motion vector refers to outside the reference picture, there may be a case where the rosary-connected motion vector from such a motion vector is in a different reference picture. Therefore, a new rosary motion vector can be generated.

[0196] Alternatively, as shown in FIG. 20, when the inter prediction unit 205 searches for a recursive motion vector in the process of deriving a rosary motion vector, if the reference position of each motion vector refers to outside the reference picture, the generation of the rosary motion vector may be terminated.

[0197] When the reference position of the motion vector refers to outside the reference picture, it is highly possible that the rosary-connected motion vector from such a motion vector also refers to outside the reference picture in a different reference picture. Therefore, the amount of decoding processing can be reduced by terminating the generation of the rosary motion vector.

[0198] Alternatively, when the decoding picture buffer 207 stores an inter prediction padding area outside the reference picture, the description of "outside the reference picture" above may be replaced with "outside the inter prediction padding area", and the inter prediction unit 205 may perform the same processing.

[0199] FIG. 21 shows an example of a method for controlling a reference source and a reference destination when deriving a bead-chain motion vector in the bead-chain motion vector prediction by the inter prediction unit 205 in a case where sequential decoder refresh (GDR: Gradual Decoding Refresh), which is known as an encoding / decoding technique for low-latency video transmission, is applied to a group of pictures to be decoded.

[0200] GDR is a technique for reducing the processing delay of video encoding / decoding at a certain bit rate compared to video encoding / decoding in a random access configuration including intra pictures widely used in video transmission by updating (hereinafter referred to as intra update) only a partial area of a screen by encoding / decoding using intra prediction and moving such a partial area for each picture to be decoded.

[0201] FIG. 21 illustrates an intra update area by a slice (Intra slice) and shows an example in which such a slice moves from the upper end to the lower end of a picture to be decoded.

[0202] Here, in this specification, three divided areas by GDR in a picture to be decoded are referred to as an area after intra update, an area during intra update, and an area before intra update.

[0203] As shown in FIG. 21, when sequential decoder refresh is applied to a group of pictures to be decoded, the inter prediction unit 205 may limit the recursive search for the motion vector of the bead-chain motion vector prediction so as not to exceed the area during intra update.

[0204] For example, as shown in FIG. 21, when sequential decoder refresh is applied to a group of pictures to be decoded and the block to be decoded is included in the area after intra update, the inter prediction unit 205 may limit the area that can be referenced in each search for the recursive motion vector of the bead-chain motion vector prediction to only the area after intra update in the reference picture of each search target.

[0205] Alternatively, when decoder updates are sequentially applied to the picture group to be decoded and the block to be decoded is included in the area before intra update, the inter prediction unit 205 may limit the area that can be referred to in each search for the recursive motion vectors of the bead-chain motion vector prediction to only the area before intra update in the reference picture for each search target.

[0206] Thus, when decoder updates are sequentially applied to the picture group to be decoded, the inter prediction unit 205 restricts the search for the recursive motion vectors of the bead-chain motion vector prediction so as not to exceed the area during intra update, thereby enabling the generation of a highly accurate bead-chain motion vector.

[0207] FIG. 22 is a diagram showing an example of controlling the reference destination when the bead-chain motion vector in the bead-chain motion vector prediction is derived by the inter prediction unit 205 in a case where reference picture resampling (RPR) disclosed in Non-Patent Document 1, in which reference to a motion vector to a reference picture with a different resolution (sampling ratio) from the decoded picture is allowed, is applied to the block to be decoded.

[0208] In RPR, the inter prediction unit 205 calculates the ratios of the width and height of the reference picture to the width and height of the decoded picture, respectively, and scales the pixel position and motion vector length of the reference picture with respect to the decoded picture according to the ratios to generate an inter prediction pixel.

[0209] Therefore, for example, as shown in FIG. 22, when the width and height of the reference picture are 0.5 times the width and height of the decoded picture, the pixel position and motion vector length of the reference picture with respect to the decoded picture are scaled by 0.5 times.

[0210] On the other hand, when generating a bead-chain motion vector, the inter prediction unit 205 needs to connect the motion vectors at the same scale as the decoded picture.

[0211] Therefore, as shown in FIG. 22, when the inter prediction unit 205 refers to motion vectors for reference pictures with a resolution (sampling ratio) different from that of the picture to be decoded, recursively searches for motion vectors from such reference pictures, and generates a chained motion vector, the length of the recursively derived motion vector may be inversely scaled according to the ratio between the width and height of the picture to be decoded and the width and height of each reference picture, and then added to generate a chained motion vector.

[0212] As a modification, when motion vector reference for a reference picture with a resolution (sampling ratio) different from that of the picture to be decoded is allowed for the sequence to be decoded or the group of pictures to be decoded, the inter prediction unit 205 may apply the chained motion vector only to a reference picture having the same sampling ratio as the picture to be decoded.

[0213] In other words, when motion vector reference for a reference picture with a resolution (sampling ratio) different from that of the picture to be decoded is allowed for the sequence to be decoded or the group of pictures to be decoded, the inter prediction unit 205 may choose not to apply the chained motion vector to a reference picture having a sampling ratio different from that of the picture to be decoded.

[0214] <4. Inheritance of Inter Prediction Information in Chained Motion Vector Prediction> When deriving a merge candidate by chained motion vector prediction, the inter prediction unit 205 may control whether to inherit a predetermined internal parameter for changing the generation method of inter prediction pixels.

[0215] (Switching Interpolation Filter) As an example, when deriving merge candidates by beaded movement vector prediction, the inter prediction unit 205 may always set the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter corresponding to the merge candidate derived by beaded movement vector prediction without inheriting the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter associated with each recursively searched movement vector so as to indicate that the switching interpolation filter is not applied.

[0216] Alternatively, when deriving merge candidates by beaded movement vector prediction, the inter prediction unit 205 may inherit, as the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter corresponding to the merge candidate derived by beaded movement vector prediction, the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter associated with the first or last movement vector constituting the beaded movement vector among the internal parameters (hpelIfIdx) that specify whether to apply the switching interpolation filter associated with each recursively searched movement vector.

[0217] Alternatively, when deriving merge candidates by beaded movement vector prediction, if, among the internal parameters (hpelIfIdx) that specify whether to apply the switching interpolation filter associated with each recursively searched movement vector, at least one internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter indicates that the switching interpolation filter is applied, the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter corresponding to the merge candidate derived by beaded movement vector prediction may be set to indicate that the switching interpolation filter is applied.

[0218] Furthermore, when the rosary-linked motion vector generated by the inter prediction unit 205 is composed of at least one or more block vectors, the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter corresponding to the merge candidate derived by the rosary-linked motion vector prediction may always be set to indicate that the switching interpolation filter is not applied.

[0219] (Local Luminance Compensation) As an example, when deriving a merge candidate by the rosary-linked motion vector prediction, the inter prediction unit 205 does not inherit the internal parameter (licFlag) that controls whether to apply the local luminance compensation associated with each recursively searched motion vector, and may always set the internal parameter (licFlag) that controls whether to apply the local luminance compensation corresponding to the merge candidate derived by the rosary-linked motion vector prediction to indicate that the local luminance compensation is not applied.

[0220] Alternatively, when deriving a merge candidate by the rosary-linked motion vector prediction, among the internal parameters (licFlag) that control whether to apply the local luminance compensation associated with each recursively searched motion vector, the internal parameter (licFlag) that controls whether to apply the local luminance compensation associated with the first or last motion vector constituting the rosary-linked motion vector may be inherited as the internal parameter (licFlag) that controls whether to apply the local luminance compensation corresponding to the merge candidate derived by the rosary-linked motion vector prediction.

[0221] Alternatively, when deriving a merge candidate by the rosary-linked motion vector prediction, among the internal parameters (licFlag) that control whether to apply the local luminance compensation associated with each recursively searched motion vector, if at least one or more internal parameters (licFlag) that control whether to apply the local luminance compensation indicate that the local luminance compensation is applied, the internal parameter (licFlag) that controls whether to apply the local luminance compensation corresponding to the merge candidate derived by the rosary-linked motion vector prediction may be set to indicate that the local luminance compensation is applied.

[0222] Furthermore, when the rosary movement vector generated by rosary movement vector prediction is composed of at least one or more block vectors, the inter prediction unit 205 may always set an internal parameter (licFlagg) that controls whether to apply local luminance compensation corresponding to a merge candidate derived by rosary movement vector prediction so as to indicate not to apply local luminance compensation.

[0223] (Weighted bi-prediction in block units) As an example, when deriving a merge candidate by rosary movement vector prediction, the inter prediction unit 205 does not inherit an internal parameter (bcwIdx) that specifies the weight value of weighted bi-prediction in block units associated with each recursively searched motion vector, and the internal parameter (bcwIdx) that specifies the weight value of weighted bi-prediction in block units corresponding to the merge candidate derived by rosary movement vector prediction may always be set so as to indicate that the weight value of weighted bi-prediction in block units is a simple average.

[0224] Alternatively, when deriving a merge candidate by rosary movement vector prediction, among the internal parameters (bcwIdx) that specify the weight values of weighted bi-prediction in block units associated with each recursively searched motion vector, the internal parameter (bcwIdx) that specifies the weight value of weighted bi-prediction in block units associated with the first or last motion vector constituting the rosary movement vector may be inherited as the internal parameter (bcwIdx) that specifies the weight value of weighted bi-prediction in block units corresponding to the merge candidate derived by rosary movement vector prediction.

[0225] As a modification example, when the inter prediction unit 205 derives a merge candidate by beaded movement vector prediction, among the internal parameters (bcwIdx) that specify the weighted bi-prediction weight values for each block unit associated with each recursively searched motion vector, if the internal parameter (bcwIdx) that specifies the weighted bi-prediction weight values for at least one or more block units indicates that the weighted bi-prediction weight value for the block unit is a simple average, the internal parameter (bceIdx) that specifies the weighted bi-prediction weight value for the block unit corresponding to the merge candidate derived by beaded movement vector prediction may also indicate that the weighted bi-prediction weight value for the block unit is a simple average.

[0226] (Multiple hypothesis prediction) As an example, when the inter prediction unit 205 derives a merge candidate by beaded movement vector prediction, without inheriting the internal parameters (mhpFlag, the fourth internal parameter) that control the applicability of multiple hypothesis predictions associated with each recursively searched motion vector, the internal parameter (mhpFlag) that controls the applicability of multiple hypothesis predictions corresponding to the merge candidate derived by beaded movement vector prediction may always be set to indicate that multiple hypothesis predictions are not applied.

[0227] As a modification example, when the inter prediction unit 205 derives a merge candidate by beaded movement vector prediction, among the internal parameters (mhpFlag) that control the applicability of multiple hypothesis predictions associated with each recursively searched motion vector, the internal parameter (mhpFlag) that controls the applicability of multiple hypothesis predictions associated with the first or last motion vector that constitutes the beaded movement vector may be inherited as the internal parameter (mhpFlag) that controls the applicability of multiple hypothesis predictions corresponding to the merge candidate derived by beaded movement vector prediction.

[0228] As a modification example, when deriving a merge candidate by beaded string movement vector prediction, among internal parameters (mhpFlag) that control whether or not to apply a plurality of hypothesis predictions associated with each movement vector recursively searched, if an internal parameter (mhpFlag) that controls whether or not to apply at least one or more of the plurality of hypothesis predictions indicates that the plurality of hypothesis predictions are to be applied, it may be determined that an internal parameter (mhpFlag) that controls whether or not to apply the plurality of hypothesis predictions corresponding to the merge candidate derived by beaded string movement vector prediction indicates that the plurality of hypothesis predictions are to be applied.

[0229] As a further modification example, when the beaded string movement vector generated by beaded string movement vector prediction is composed of at least one or more block vectors, the internal parameter (mhpFlag) that controls whether or not to apply the plurality of hypothesis predictions corresponding to the merge candidate derived by beaded string movement vector prediction may always be set to indicate that the plurality of hypothesis predictions are not applied.

[0230] As described above, if the internal parameter that changes the method for generating an inter prediction pixel is not always inherited by beaded string movement vector prediction, the amount of code of the inter prediction information that needs to be stored in the decoded picture buffer 207 is reduced.

[0231] On the other hand, as described above, if the internal parameter that changes the method for generating an inter prediction pixel is inherited by beaded string movement vector prediction, there is a possibility that the inter prediction pixel using the merge candidate generated by beaded string movement vector prediction will be of higher accuracy, and an improvement in coding efficiency can be expected.

[0232] Also, when inheriting the above-described internal parameter, when the beaded string movement vector is composed of at least one or more block vectors, since there is little need to change the method for generating an inter prediction pixel, by setting not to inherit such an internal parameter, that is, not to change the method for generating an inter prediction pixel, an improvement in coding efficiency can be expected.

[0233] <5. Rearrangement of Merge Candidates Containing Bead Connecting Motion Vector Prediction Candidates and Vector Correction> (Rearrangement of Merge Candidates) The later the bead connecting motion vector prediction candidate is derived compared to other merge candidates, the less likely it is to be stored in the merge candidate list.

[0234] As a method to solve this problem, the inter prediction unit 205 calculates the template matching costs of all merge candidates stored in the merge candidate list including the merge candidates derived by bead connecting motion vector prediction, and may rearrange the storage positions of each merge candidate in the merge candidate list in ascending order according to the ascending order of the template matching costs.

[0235] As an example, as a technique for rearranging the storage positions of each merge candidate in the merge candidate list based on the template matching cost of each merge candidate, the adaptive reordering merge candidate (ARMC) disclosed in Non-Patent Document 2 may be used by the inter prediction unit 205.

[0236] Furthermore, since the bead connecting motion vector prediction recursively generates new motion vectors as described above, when other merge candidates are derived after the derivation of the bead connecting motion vector prediction candidates, there is a possibility that the other merge candidates may not be stored in the merge candidate list.

[0237] As a method to solve this problem, the inter prediction unit 205 calculates the template matching costs of all untrimmed merge candidates including the merge candidates derived by bead connecting motion vector prediction, and may store them in the merge candidate list in ascending order according to the ascending order of the template matching costs.

[0238] Alternatively, the inter prediction unit 205 may calculate the template matching costs of all untrimmed merge candidates including the merge candidates derived by the bead string movement vector prediction, and perform a series of processes until they are stored in ascending order in the merge candidate list in ascending order of the template matching costs a predetermined number of times. For example, such a predetermined number of times may be two or three times.

[0239] As a result, merge candidates capable of generating inter prediction pixels with high accuracy are arranged in the merge candidate list, and an improvement in coding efficiency can be expected.

[0240] (Vector correction) The inter prediction unit 205 may correct the motion vector associated with the merge candidate derived by the bead string movement vector prediction by a predetermined method.

[0241] As an example, the inter prediction unit 205 may correct the motion vector associated with the merge candidate derived by the bead string movement vector prediction by template matching.

[0242] Here, the inter prediction unit 205 may apply a technique similar to the template matching disclosed in Non-Patent Document 2 as such template matching.

[0243] As an example, the inter prediction unit 205 may correct the motion vector associated with the merge candidate derived by the bead string movement vector prediction by bilateral matching.

[0244] Here, the inter prediction unit 205 may apply a technique similar to the bilateral matching disclosed in Non-Patent Document 1 and Non-Patent Document 2 as such bilateral matching. For example, the inter prediction unit 205 may apply the decoder-side motion vector refinement (DMVR) disclosed in Non-Patent Document 1.

[0245] As an example, the inter prediction unit 205 may correct the motion vector associated with the merge candidate derived by the bead string motion vector prediction by adding the motion vector difference derived from the control information.

[0246] Here, as control information indicating the motion vector difference, the inter prediction unit 205 may apply a technique similar to the merge motion vector difference (MMVD: Merge with Motion Vector Difference) disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0247] By the inter prediction unit 205 correcting the motion vector associated with the merge candidate derived by the bead string motion vector prediction by a predetermined method, the inter prediction accuracy is improved, and an improvement in coding efficiency can be expected.

[0248] <6. Various restrictions on bead string motion vector prediction (applicability, number of pictures, number of merge candidates)> Using FIG. 23, various restriction methods for the bead string motion vector prediction by the decoding unit 201 and the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment will be described. (Applicability of bead string motion vector prediction) First, regarding the applicability (applicable or not) of the bead string motion vector prediction, as described above, by its nature, when TMVP is applicable, the bead string motion vector prediction is applicable. In other words, when TMVP is not applicable, the bead string motion vector prediction is not applicable.

[0249] Therefore, regarding the applicability or non-applicability of the bead string motion vector prediction, it can be controlled according to the control information (syntax) that controls the applicability or non-applicability of the temporal motion vector prediction TMVP.

[0250] That is, the inter prediction unit 205 may control the applicability or non-applicability of the bead string motion vector prediction based on the control information that controls the applicability or non-applicability of the temporal motion vector prediction (TMVP) sent from the decoding unit 201.

[0251] Here, in Non-Patent Document 1, as a syntax for controlling the applicability of TMVP, the syntax at the sequence unit to be decoded (sps_temporal_mvp_enabled_flag, first control information, first syntax) and the syntax at the picture unit to be decoded (ph_temporal_mvp_enabled_flag, second control information, second syntax) are defined.

[0252] That is, regarding the applicability of the bead-chain motion vector prediction, it can also be controlled in the same unit as TMVP or a unit lower than TMVP.

[0253] For example, the method for controlling the applicability of the bead-chain motion vector prediction in the inter prediction 205 that controls the applicability of the bead-chain motion vector prediction in units of blocks to be decoded is as follows.

[0254] The inter prediction unit 205 may control the applicability of the bead-chain motion vector prediction for the block to be decoded based on the control information that controls the applicability of the temporal motion vector prediction (TMVP) at the sequence unit to be decoded sent from the decoding unit 201.

[0255] More specifically, when the control information that controls the applicability of the temporal motion vector prediction (TMVP) at the sequence unit to be decoded sent from the decoding unit 201 specifies that the temporal motion vector prediction is applicable, the inter prediction unit 205 specifies that the bead-chain motion vector prediction for the block to be decoded is applicable.

[0256] On the other hand, when the above control information specifies that the temporal motion vector prediction is not applicable, the inter prediction unit 205 specifies that the bead-chain motion vector prediction for the block to be decoded is not applicable.

[0257] Alternatively, the inter prediction unit 205 may control whether or not to apply the bead-chain motion vector prediction to the block to be decoded based on the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP) for each picture to be decoded sent from the decoding unit 201.

[0258] More specifically, when the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP) for each picture to be decoded sent from the decoding unit 201 specifies that the temporal motion vector prediction can be applied, the inter prediction unit 205 specifies that the bead-chain motion vector prediction can be applied to the block to be decoded.

[0259] On the other hand, when the above-described control information specifies that the temporal motion vector prediction cannot be applied, the inter prediction unit 205 specifies that the bead-chain motion vector prediction cannot be applied to the block to be decoded.

[0260] The inter prediction unit 205 may control whether or not to apply the bead-chain motion vector prediction in units of the sequence to be decoded or units of the decoding target smaller than the sequence to be decoded based on the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP) for each sequence to be decoded sent from the decoding unit 201.

[0261] Alternatively, the inter prediction unit 205 may control whether or not to apply the bead-chain motion vector prediction in units of the picture to be decoded or units of the decoding target smaller than the picture to be decoded based on the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP) for each picture to be decoded sent from the decoding unit 201.

[0262] Alternatively, before controlling whether or not to apply the bead-chain motion vector prediction for each block to be decoded by the inter prediction unit 205, the decoding unit 201 may control whether or not to apply the bead-chain motion vector prediction based on the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP).

[0263] FIG. 23 is a diagram showing an example of a process in which a decoder 201 determines whether to decode a syntax (sps_chained_mvp_enabled_flag) that controls whether or not to apply a beaded-chain motion vector prediction in a decoding target sequence unit based on a syntax (sps_temporal_mvp_enabled_flag) that controls whether or not to apply a temporal motion vector prediction (TMVP) in the decoding target sequence unit.

[0264] As shown in FIG. 23, in step S100, when the decoder 201 determines that a syntax (sps_temporal_mvp_enabled_flag) that controls whether or not to apply a temporal motion vector prediction (TMVP) in the decoding target sequence unit is 1, in step S101, the decoder 201 decodes a syntax (sps_chained_mvp_enabled_flag, the third syntax) that controls whether or not to apply a beaded-chain motion vector prediction in the decoding target sequence unit.

[0265] On the other hand, in step S100, when the decoder 201 determines that a syntax (sps_temporal_mvp_enabled_flag) that controls whether or not to apply a temporal motion vector prediction (TMVP) in the decoding target sequence unit is not 1, in step S102, the decoder 201 does not decode a syntax (sps_chained_mvp_enabled_flag) that controls whether or not to apply a beaded-chain motion vector prediction in the decoding target sequence unit.

[0266] Here, when the value of sps_temporal_mvp_enabled_flag is 1, the decoding unit 201 determines that temporal motion vector prediction (TMVP) can be applied for the sequence unit to be decoded. When the value of sps_temporal_mvp_enabled_flag is 0, the decoding unit 201 determines that temporal motion vector prediction (TMVP) cannot be applied for the sequence unit to be decoded. When sps_temporal_mvp_enabled_flag has not been decoded, the value of sps_temporal_mvp_enabled_flag is presumed to be 0.

[0267] Also, when the value of sps_chained_mvp_enabled_flag is 1, the decoding unit 201 determines that chained motion vector prediction can be applied for the sequence unit to be decoded. When the value of sps_chained_mvp_enabled_flag is 0, the decoding unit 201 determines that chained motion vector prediction cannot be applied for the sequence unit to be decoded. When sps_chained_mvp_enabled_flag has not been decoded, the value of sps_chained_mvp_enabled_flag is presumed to be 0.

[0268] In this way, by determining whether sps_chained_mvp_enabled_flag needs to be decoded based on the value of sps_temporal_mvp_enabled_flag, in a case where it is obvious that decoding of sps_chained_mvp_enabled_flag is unnecessary, the sps_chained_mvp_enabled_flag does not need to be decoded, so that the amount of code can be reduced.

[0269] When chained motion vector prediction is used in combination with the above-described rearrangement (ARMC: Adaptive Reordering Merge Candidate) by comparing the template matching costs of merge candidates, a greater improvement in coding efficiency can be expected.

[0270] Therefore, as a modified example of the method for controlling the applicability of the bead string movement vector prediction based on the applicability of the above-described TMVP, the inter prediction unit 205 may control the applicability of the bead string movement vector prediction based on both the applicability of the TMVP and the applicability of the ARMC.

[0271] Specifically, when the TMVP is applicable and the ARMC is applicable, the inter prediction unit 205 may specify that the bead string movement vector prediction is applicable. In other cases, the inter prediction unit 205 may specify that the bead string movement vector prediction is not applicable.

[0272] Regarding the applicability of the ARMC, control information similar to that of the TMVP may be provided, and the determination may be made based on the value of such control information.

[0273] (Number of reference pictures and search depth of bead string movement vector prediction) Hereinafter, a method for limiting the number of reference pictures or the search depth that can be referred to during the recursive movement vector search of the bead string movement vector prediction will be described.

[0274] First, a method for limiting the number of reference pictures that can be referred to during the recursive movement vector search of the bead string movement vector prediction will be described.

[0275] As an example, the inter prediction unit 205 may limit the recursive movement vector search of the bead string movement vector prediction to at least one or more reference pictures included in the reference picture list of the block to be decoded.

[0276] By limiting the recursive movement vector search of the bead string movement vector prediction by the inter prediction unit 205 to at least one or more reference pictures included in the reference picture list of the block to be decoded, the decoding processing amount can be reduced compared to the case where such a limitation is not performed (for example, when the inter prediction unit 205 searches all the reference pictures stored in the decoded picture buffer 207).

[0277] Alternatively, the inter prediction unit 205 may limit the search for the recursive motion vector of the beaded string motion vector prediction to only the reference picture referred to by the motion vector of the block to be decoded.

[0278] In other words, the inter prediction unit 205 re-searches only the block vector within the reference picture referred to by the motion vector of the block to be decoded, and generates a beaded string motion vector by beaded string motion vector prediction.

[0279] Second, a method for limiting the search depth of the recursive motion vector of the beaded string motion vector prediction will be described.

[0280] As an example, the inter prediction unit 205 may limit the search depth of the recursive motion vector of the beaded string motion vector prediction in the block to be decoded based on the control information.

[0281] Specifically, the inter prediction unit 205 may limit the search depth of the recursive motion vector of the beaded string motion vector prediction in the block to be decoded based on the control information (sps_max_num_my_chains) that sets the upper limit of the search depth of the recursive motion vector of the beaded string motion vector prediction in the decoding target sequence unit sent from the decoding unit.

[0282] Here, the control information of the decoding target sequence unit may be control information of a picture unit or a slice unit, which is control information lower than the decoding target sequence unit, or may be hierarchical control information combining these.

[0283] Alternatively, the inter prediction unit 205 may set a predetermined fixed value for the search depth of the recursive motion vector of the beaded string motion vector prediction. The inter prediction unit 205 may set 1, 2, or 3 as such a predetermined fixed value.

[0284] By restricting the search depth of the beaded movement vector prediction in this way, the amount of decoding processing required to derive the beaded movement vector can be reduced compared to the case where it is not restricted.

[0285] (Number of merge candidates for beaded movement vector prediction) Hereinafter, a method for restricting the number of merge candidates derived by beaded movement vector prediction will be described.

[0286] As an example, the inter prediction unit 205 may restrict the number of merge candidates derived by beaded movement vector prediction in the block to be decoded based on the control information.

[0287] Specifically, the inter prediction unit 205 may restrict the number of merge candidates derived by beaded movement vector prediction in the block to be decoded based on the control information (sps_max_num_cmvp_cand) that sets the upper limit of the search depth of the number of merge candidates derived by beaded movement vector prediction for each decoding target sequence unit sent from the decoding unit.

[0288] Here, the control information for each decoding target sequence unit may be control information for each picture unit or slice unit, which is control information lower than the decoding target sequence unit, or may be hierarchical control information combining these.

[0289] Alternatively, the inter prediction unit 205 may set a predetermined fixed value for the number of merge candidates derived by beaded movement vector prediction.

[0290] (Restriction of block vector search in beaded movement vector prediction) It has been described above that in the recursive motion vector search of the beaded movement vector prediction, the search for the block vector may be included. However, block vectors referring to within the same picture are easy to find in screen images or game images including characters and computer-generated graphics (CG), but are difficult to find in images captured by a camera.

[0291] Therefore, it is desirable to be able to limit the search for block vectors in the beaded movement vector prediction according to the characteristics of the image.

[0292] As an example, the inter prediction unit 205 may limit the block vector search based on control information that specifies whether or not to search for block vectors in the beaded movement vector prediction for the block to be decoded.

[0293] Specifically, the inter prediction unit 205 may limit the searchability of block vectors in the beaded movement vector prediction for the block to be decoded based on control information (sps_cmvp_bv_search_enabled_flag) that specifies whether or not to search for block vectors in the beaded movement vector prediction in the decoding target sequence unit sent from the decoding unit 201.

[0294] Here, the control information of the decoding target sequence unit may be control information of a picture unit or a slice unit, which is control information lower than the decoding target sequence unit, or may be hierarchical control information combining these. (Restriction of the reference source of the beaded movement vector prediction) In the recursive motion vector search of the beaded motion vector prediction, the method of adding the four corners of the block to be decoded by adding the starting point of the first motion vector to the center of the block to be decoded has been described above.

[0295] However, compared with screen images and game images in which pixel values change rapidly in pixel units, images taken by a camera are less likely to change.

[0296] Therefore, it is desirable to be able to limit the starting point of the first motion vector in the recursive motion vector search of the beaded motion vector prediction according to the characteristics of the image.

[0297] As an example, the inter prediction unit 205 may limit the reference source for the bead-chain motion vector prediction based on control information for specifying whether to add at least one or more predetermined positions of the block to be decoded by adding the starting point of the first motion vector in the bead-chain motion vector prediction for the block to be decoded to the center of the block to be decoded. Such a predetermined position may be the four corners within the block to be decoded described above.

[0298] Specifically, the inter prediction unit 205 may limit whether to add at least one or more predetermined positions of the block to be decoded by adding the starting point of the first motion vector in the bead-chain motion vector prediction for the block to be decoded to the center of the block to be decoded based on the sps_cmvp_multi_position_flag.

[0299] Here, the sps_cmvp_multi_position_flag is control information for specifying whether to add at least one or more predetermined positions of the block to be decoded in addition to the center of the block to be decoded to the starting point of the first motion vector in the bead-chain motion vector prediction for the block to be decoded.

[0300] Here, the control information for the decoding target sequence unit may be control information for a picture unit or a slice unit, which is control information lower than the decoding target sequence unit, or may be hierarchical control information combining these.

[0301] <7. Applicable Target of Bead-Chain Motion Vector Prediction> Describe the technical features when applying the bead-chain motion vector prediction to merges other than the normal merge disclosed in Non-Patent Document 1 and Non-Patent Document 2. (MMVD, GPM, TM Merge) First, MMVD, GPM, and template matching merge (TM merge) disclosed in Non-Patent Document 1 and Non-Patent Document 2 construct a merge list in the same way as normal merge. However, in Non-Patent Document 2, pruning of merge candidates during the construction of these merge candidate lists is strengthened compared to normal merge, and these merge candidate lists are unlikely to be the same as the merge candidates stored in the merge candidate list of normal merge.

[0302] Therefore, when not reusing the merge candidate list of normal merge in MMVD, GPM, and TM merge, in addition to applying the bead-chain motion vector prediction to the merge candidates in the merge candidate list of normal merge, if the bead-chain motion vector prediction is applied to the merge candidates in the respective merge candidate lists of MMVD, GPM, and TM merge, an improvement in coding efficiency can be expected.

[0303] Therefore, in addition to applying the bead-chain motion vector prediction to the merge candidates in the merge candidate list of normal merge, the inter prediction unit 205 may apply the bead-chain motion vector prediction to the merge candidates in the merge candidate list of the merge mode motion vector difference.

[0304] In addition, in addition to applying the bead-chain motion vector prediction to the merge candidates in the merge candidate list of normal merge, the inter prediction unit 205 may apply the bead-chain motion vector prediction to the merge candidates in the merge candidate list of the geometric partitioning mode.

[0305] In addition, in addition to applying the bead-chain motion vector prediction to the merge candidates in the merge candidate list of normal merge, the inter prediction unit 205 may apply the bead-chain motion vector prediction to the merge candidates in the merge candidate list of the template matching merge.

[0306] Here, for the bead-chain motion vectors generated for the merge mode motion vector difference, geometric partitioning mode, and template matching merge, the prediction unit 205 may apply pruning processing at the time of storing the merge candidate list, motion vector correction, and rearrangement of the merge candidates (motion vectors) in the same manner as in normal merge.

[0307] (Adaptive Motion Vector Prediction - Merge Mode) In the adaptive motion vector prediction - merge mode (Adaptive Motion Vector Prediction - Merge mode) disclosed in Non - Patent Document 1 and Non - Patent Document 2 described above, since the motion vectors in either list 0 (L0) or list 1 (L1) of the reference picture list are derived by adaptive motion vector prediction (AMVP), that is, by control information, if the bead - chain motion vector prediction is applied to the merge candidates in the merge candidate list of the reference picture list number not derived by AMVP, the options for selecting merge candidates that can be selected in the adaptive motion vector prediction - merge mode increase. Therefore, the inter - prediction accuracy is improved, and an improvement in coding efficiency can be expected.

[0308] Therefore, in the adaptive motion vector prediction - merge mode, the prediction unit 205 may apply the bead - chain motion vector prediction to the merge candidates in the merge candidate list of the reference picture list number not derived by the adaptive motion vector prediction.

[0309] Furthermore, when the prediction unit 205 applies the bead - chain motion vector prediction to the merge candidates in the merge candidate list of the reference picture list number not derived by the adaptive motion vector prediction in the adaptive motion vector prediction - merge mode, it may limit the generated bead - chain motion vector so as not to refer to the reference picture of the motion vector derived by the adaptive motion vector prediction.

[0310] Here, for the bead-chain motion vectors generated for the adaptive motion vector prediction-merge mode, the inter prediction unit 205 may apply pruning processing during storage in the merge candidate list, motion vector correction, and rearrangement of merge candidates (motion vectors) in the same manner as normal merge.

[0311] (Sub-block merge mode) As disclosed in Non-Patent Document 1 and Non-Patent Document 2 described above, in affine merge, in order to derive the motion vectors for each sub-block within the block to be decoded by affine transformation, as shown in FIG. 24, in the case of single prediction (either L0 or L1), two or three control point motion vectors cpMv are derived, and in the case of dual prediction (both L0 and L1), four or six control point motion vectors cpMv are derived.

[0312] Note that all the control point motion vectors of L0 and L1 for single prediction or dual prediction refer to the same reference picture.

[0313] In affine merge, as disclosed in Non-Patent Document 1 and Non-Patent Document 2, these control point motion vectors are derived from the merge candidates stored in the merge candidate list for affine merge.

[0314] The inter prediction unit 205 may derive the merge candidates to be stored in the merge candidate list for affine merge in the same manner as the method for deriving the merge candidates for affine merge disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0315] For example, when two or three control point motion vectors are accumulated at positions spatially adjacent or close to the block to be decoded in the inter prediction unit 205, they may be added to the merge candidate list for affine merge of the block to be decoded.

[0316] Alternatively, when a plurality of motion vectors are stored at positions spatially adjacent or close to the block to be decoded, the inter prediction unit 205 may generate a control point motion vector for the block to be decoded based on them and add it to the merge candidate list for affine merge.

[0317] Therefore, in the affine merge mode, the inter prediction unit 205 may add the motion vectors derived by recursively searching for motion vectors with respect to at least two or more control point motion vectors stored in the merge candidate list to each of the first control point motion vectors to generate a chained control point motion vector and store it in the merge candidate list.

[0318] Furthermore, in the affine merge mode, when recursively searching for motion vectors with respect to at least two or more control point motion vectors stored in the merge candidate list, the inter prediction unit 205 may limit the reference pictures to be searched for each control point motion vector to the same reference picture.

[0319] Alternatively, in the affine merge mode, when recursively searching for motion vectors with respect to at least two or more control point motion vectors of L0 and L1 stored in the merge candidate list, the inter prediction unit 205 may limit the reference pictures to be searched for each control point motion vector of L0 and L1 to the same reference picture.

[0320] Here, for the chained control point motion vector generated for affine merge, the inter prediction unit 205 may apply pruning processing at the time of storage in the merge candidate list, motion vector correction, and rearrangement of merge candidates (motion vectors) in the same manner as normal merge.

[0321] According to the image decoding apparatus according to the present embodiment, since the inter prediction unit 205 adds all the motion vectors derived by recursively searching the motion vectors stored in the decoded picture buffer 207 for the motion vectors at positions spatially or temporally adjacent or close to the block to be decoded, and derives the generated motion vector as the motion vector of the block to be decoded, the options for selecting motion vectors that can be selected in inter prediction increase, the inter prediction accuracy improves, and the coding efficiency is improved.

[0322] The above-described image decoding apparatus 200 may be realized by a program that causes a computer to execute each function (each process).

Industrial Applicability

[0323] According to the present embodiment, for example, since an improvement in overall service quality can be realized in moving image communication, it is possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Build resilient infrastructure, promote sustainable industrialization and foster innovation."

Explanation of Signs

[0324] 200…Image decoding apparatus 201…Decoder 202…Inverse quantization unit 203…Inverse transformation unit 204…Intra prediction unit 205…Inter prediction unit 206…Adder 207…Decoded picture buffer 210…Code input unit 220…Image output unit

Claims

1. An image decoding device, an inter prediction unit that generates inter predicted pixels from control information and decoded pixels stored in a decoded picture buffer; The image decoding device is characterized in that the inter prediction unit generates a chained motion vector by recursively searching motion vectors stored in the decoding picture buffer for motion vectors that are spatially or temporally adjacent or close to the block to be decoded, which is stored in a merge candidate list that stores motion vector candidates, and adding all the derived motion vectors, and performs chained motion vector prediction to derive the chained motion vector as a candidate motion vector for the block to be decoded.

2. The image decoding device described in claim 1, characterized in that, in the daisy chain motion vector prediction, the inter prediction unit adds block vectors indicating reference destinations within the same picture to the recursive search targets in addition to the motion vectors stored in the decoded picture buffer.

3. The image decoding device described in claim 1, characterized in that, in the daisy chain motion vector prediction, the inter prediction unit newly searches for one or more motion vectors from a single reference motion vector for each search list number of the merge candidate list and each search depth when searching for recursive motion vectors in the daisy chain motion vector prediction, and adds all motion vectors already derived to each of the one or more newly derived motion vectors to generate the daisy chain motion vector.

4. 2. The image decoding device according to claim 1, wherein the inter prediction unit searches for a daisy chain motion vector prediction candidate after searching for a predetermined merge candidate.

5. The image decoding device according to claim 4 , wherein the inter prediction unit searches for the daisy-chain motion vector prediction candidates after searching for the temporal motion vector prediction candidates.

6. The image decoding device according to claim 4 , wherein the inter prediction unit searches for the daisy-chain motion vector prediction candidates after searching for the history-based motion vector prediction candidates.

7. The image decoding device according to claim 4 , wherein the inter prediction unit searches for the daisy-chain motion vector prediction candidates after searching for spatial merge candidates.

8. The inter prediction unit storing the predetermined merge candidates in a merge candidate list; applying the daisy chain motion vector prediction in order from the first merge candidate stored in the merge candidate list; The image decoding device according to claim 4 , wherein new merge candidates generated by the daisy-chain motion vector prediction for each of the merge candidates are stored in the merge candidate list in order.

9. The inter prediction unit storing the predetermined merge candidates in a merge candidate list; applying the daisy chain motion vector prediction in order from the first merge candidate stored in the merge candidate list; The image decoding device according to claim 4 , wherein new merge candidates generated for each of the merge candidates using each search depth in the daisy-chain motion vector prediction are stored in the merge candidate list in order.

10. The inter prediction unit Store the first merge candidate in the merge candidate list. applying the chained motion vector prediction in order from the merge candidate; The image decoding device according to any one of claims 1 to 4, characterized in that new merge candidates generated by the chained motion vector prediction for each of the merge candidates are stored in a merge candidate list in sequence.

11. The inter prediction unit Store the first merge candidate in the merge candidate list. applying the chained motion vector prediction in order from the merge candidate; The image decoding device according to any one of claims 1 to 4, characterized in that new merge candidates generated using each search depth of the chained motion vector prediction for each of the merge candidates are stored in a merge candidate list in sequence.

12. The image decoding device described in any one of claims 1 to 4, characterized in that the inter prediction unit generates new merge candidates by combining a daisy chain motion vector generated for each merge candidate list using each search depth in the daisy chain motion vector prediction with a daisy chain motion vector generated using a search depth different from each search depth.

13. The image decoding device described in any one of claims 1 to 4, characterized in that the inter prediction unit generates a new merge candidate by combining a daisy chain motion vector generated for each merge candidate list using each search list number or each search depth of the daisy chain motion vector prediction with a daisy chain motion vector generated using a search list number or search depth different from each search list number or each search depth.

14. An image decoding method, comprising: generating inter-predicted pixels from control information and decoded pixels stored in a decoded picture buffer; An image decoding method characterized in that generating the inter-predicted pixel includes performing daisy-chain motion vector prediction, in which a daisy-chain motion vector is generated by recursively searching motion vectors stored in the decoding picture buffer for motion vectors at positions spatially or temporally adjacent or close to the block to be decoded, which are stored in a merge candidate list that stores motion vector candidates, and adding all the derived motion vectors, and deriving the daisy-chain motion vector as a candidate motion vector for the block to be decoded.

15. A program that causes a computer to function as an image decoding device, The image decoding device comprises: an inter prediction unit that generates inter predicted pixels from control information and decoded pixels stored in a decoded picture buffer; The inter prediction unit generates a chained motion vector by recursively searching motion vectors stored in the decoding picture buffer for motion vectors located spatially or temporally adjacent or close to the block to be decoded, which is stored in a merge candidate list that stores motion vector candidates, and adding all the derived motion vectors, and performs chained motion vector prediction, in which the chained motion vector is derived as a candidate motion vector for the block to be decoded.

Images