Image decoding device, image decoding method, and program

Abstract

To improve encoding efficiency.SOLUTION: In an image decoding device 200 according to the present invention, an inter-prediction unit 205 performs chained motion vector prediction of recursively adding all motion vectors derived by searching for motion vectors accumulated in a decoding picture buffer, in motion vectors of positions that are spatially or temporally adjacent or proximate to each other, to a decoding target block stored in a merge candidate list that stores candidates for a motion vector, thereby generating a chained motion vector, and deriving the chained motion vector as a candidate for a motion vector of the decoding target block, and determines whether to add a merge candidate corresponding to the chained motion vector derived through the chained motion vector prediction on the basis of a predetermined condition.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image decoding apparatus, an image decoding method, and a program.

Background Art

[0002] Non-Patent Document 1 and Non-Patent Document 2 disclose inter prediction.

[0003] In inter prediction, predicted pixels of a block to be decoded are generated from decoded pixels (reference pixels) in a decoded picture (reference picture) different from the picture to be decoded.

[0004] Also, in inter prediction, using control information, a motion vector of a block to be decoded necessary for generating predicted pixels of the block to be decoded is selected from candidates of a plurality of motion vectors at positions spatially or temporally adjacent or close to the block to be decoded.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] In the inter prediction in Non-Patent Document 1 and Non-Patent Document 2, candidates of motion vectors that can be selected are limited to motion vectors at positions spatially or temporally adjacent or close to the block to be decoded, so there is a problem that there is room for improvement in coding efficiency.

[0007] Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide an image decoding apparatus, an image decoding method, and a program with high coding efficiency.

Means for Solving the Problems

[0008] A first feature of the present invention is an image decoding apparatus, comprising: a decoding unit that variably decodes code information and outputs a quantization value and control information; an inverse quantization unit that inverse quantizes the quantization value and outputs a transform coefficient; an inverse transform unit that inverse transforms the transform coefficient and outputs a prediction residual pixel; an intra prediction unit that generates an intra prediction pixel from the control information and decoded pixels; a decoded picture buffer that stores the decoded pixels; an inter prediction unit that generates an inter prediction pixel from the control information and the decoded pixels stored in the decoded picture buffer; and an adder that adds at least one of the intra prediction pixel and the inter prediction pixel to the prediction residual pixel to generate the decoded pixel. The inter prediction unit recursively searches for motion vectors stored in the decoded picture buffer for motion vectors at positions spatially or temporally adjacent or close to a decoding target block stored in a merge candidate list storing motion vector candidates, and adds all the motion vectors thus derived to generate a chained motion vector, performs chained motion vector prediction to derive the chained motion vector as a candidate for the motion vector of the decoding target block, and the gist is that the inter prediction unit determines whether to add a merge candidate corresponding to the chained motion vector derived by the chained motion vector prediction based on a predetermined condition.

[0009] A second feature of the present invention is an image decoding method, comprising: step A of performing variable-length decoding on coded information to output a quantization value and control information; step B of performing inverse quantization on the quantization value to output a transform coefficient; step C of performing inverse transformation on the transform coefficient to output a prediction residual pixel; step D of generating an intra prediction pixel from the control information and decoded pixels; step E of storing the decoded pixels in a decoded picture buffer; step F of generating an inter prediction pixel from the control information and the stored decoded pixels; and step G of adding at least one of the intra prediction pixel and the inter prediction pixel to the prediction residual pixel to generate the decoded pixel. In step F, all motion vectors derived by recursively searching motion vectors at positions spatially or temporally adjacent or close to a decoding target block stored in a merge candidate list storing motion vector candidates are added to generate a chained motion vector, and chained motion vector prediction is performed to derive the chained motion vector as a candidate for the motion vector of the decoding target block. In step F, it is essential to determine whether to add a merge candidate corresponding to the chained motion vector derived by the chained motion vector prediction based on a predetermined condition.

[0010] A third feature of the present invention is a program that causes a computer to function as an image decoding device, where the image decoding device includes a decoding unit that variably decodes code information and outputs a quantization value and control information, an inverse quantization unit that inverse quantizes the quantization value and outputs a transform coefficient, an inverse transform unit that inverse transforms the transform coefficient and outputs a prediction residual pixel, an intra prediction unit that generates an intra prediction pixel from the control information and decoded pixels, a decoded picture buffer that stores the decoded pixels, an inter prediction unit that generates an inter prediction pixel from the control information and the stored decoded pixels, and an adder that adds at least one of the intra prediction pixel and the inter prediction pixel to the prediction residual pixel to generate the decoded pixel. The inter prediction unit recursively searches for motion vectors stored in the decoded picture buffer for motion vectors at positions spatially or temporally adjacent or close to a decoding target block stored in a merge candidate list that stores motion vector candidates, and adds all the motion vectors derived thereby to generate a chained motion vector. The chained motion vector prediction is performed to derive the chained motion vector as a candidate for the motion vector of the decoding target block. The gist is that the inter prediction unit determines whether to add a merge candidate corresponding to the chained motion vector derived by the chained motion vector prediction based on a predetermined condition.

Advantages of the Invention

[0011] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program with high encoding efficiency.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components and the like, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0014] <First Embodiment> Hereinafter, an image decoding apparatus 200 according to the present embodiment will be described with reference to FIGS. 1 to 24.

[0015] The image decoding apparatus 200 according to the first embodiment of the present invention targets various image signals (hereinafter, images). For example, the image decoding apparatus 200 according to the present embodiment targets a YUV (YCbCr) image composed of luminance pixels and chrominance pixels, an RGB image composed of RGB pixels, or a monochrome image. Here, the pixels constituting each image have discrete values (pixel values) with a predetermined bit width.

[0016] FIG. 1 is a diagram showing an example of a functional block of the image decoding apparatus 200 according to the present embodiment.

[0017] As shown in FIG. 1, the image decoding apparatus 200 includes a code input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transformation unit 203, an intra prediction unit 204, an inter prediction unit 205, an adder 206, a decoded picture buffer 207, and an image output unit 220.

[0018] Hereinafter, in the description of the functions of each unit, the places described as "pixel" may be a block (unit) composed of pixels, a tree block which is the maximum size of the block, a slice, a tile, or an image (picture) larger than the tree block.

[0019] Although not shown in FIG. 1, the image decoding apparatus 200 may include a loop filter for correcting the pixel value of the decoded pixel between the adder 206 and the decoded picture buffer 207.

[0020] The code input unit 210 is configured to acquire the coded information coded by the image coding apparatus.

[0021] The decoding unit 201 is configured to decode the control information and the quantization value from the coded information input from the code input unit 210. For example, the decoding unit 201 is configured to output the control information and the quantization value by performing variable length decoding on such coded information.

[0022] Here, the quantization value is sent to the inverse quantization unit 202, and the control information is sent to the decoded picture buffer 207, the intra prediction unit 204, and the inter prediction unit 205. Note that such control information (syntax) includes information necessary for controlling the decoded picture buffer 207, the intra prediction unit 204, the inter prediction unit 205, etc., and may include header information such as a sequence parameter set, a picture parameter set, a picture header, and a slice header.

[0023] The inverse quantization unit 202 is configured to inverse quantize the quantization value sent from the decoding unit 201 to obtain an inverse quantized transform coefficient. Such a transform coefficient is sent to the inverse transformation unit 203.

[0024] The inverse transformation unit 203 is configured to perform inverse transformation on the transformation coefficients sent from the inverse quantization unit 202 to obtain a decoded prediction residual. Such a prediction residual is sent to the adder 206.

[0025] The intra prediction unit 204 is configured to generate intra prediction pixels based on the decoded pixels and the control information sent from the decoding unit 201. Here, the decoded pixels are obtained via the adder 206 and accumulated in the decoded picture buffer 207. Also, the intra prediction pixels are prediction pixels for addition with the prediction residual by the adder 206. Note that the intra prediction pixels are sent to the adder 206.

[0026] Here, the prediction performed by the intra prediction unit 204 may be the intra prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2. Alternatively, the prediction performed by the intra prediction unit 204 may be the intra block copy or intra template matching prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0027] The decoded picture buffer 207 cumulatively stores the decoded pixels sent from the adder 206 and the inter prediction information corresponding to the decoded pixels sent from the inter prediction unit 205.

[0028] Here, the decoded pixels stored in the decoded picture buffer 207 and the inter prediction information corresponding to such decoded pixels are referred to by the inter prediction unit 205. Details of the inter prediction information will be described later.

[0029] The inter prediction unit 205 is configured to generate inter prediction pixels based on the decoded pixels and the inter prediction information obtained by referring to the decoded picture buffer 207 and the control information sent from the decoding unit 201.

[0030] Furthermore, the inter prediction unit 205 sends the generated inter prediction pixels to the adder 206, and sends the inter prediction information used for generating such inter prediction pixels to the decoded picture buffer 207 corresponding to the pixels to be decoded.

[0031] Here, the prediction performed by the inter prediction unit 205 may be the inter prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2. Alternatively, the prediction performed by the inter prediction unit 205 may be the intra block copy disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0032] The adder 206 is configured to add at least one of the prediction residual sent from the inverse transform unit 203 and the input intra prediction pixels and inter prediction pixels to calculate the decoded pixels. Such decoded pixels are sent to the image output unit 220, the decoded picture buffer 207, and the intra prediction unit 204.

[0033] The image output unit 220 is configured to output the decoded pixels transmitted from the adder 206.

[0034] Hereinafter, the roles of the inter prediction unit 205 and the decoded picture buffer 207, which are characteristic configurations of the image decoding apparatus 200 according to the present embodiment, will be described by taking as an example the case where the prediction performed by the inter prediction unit 205 is inter prediction.

[0035] <Basic Roles of Inter Prediction Unit 205 and Decoded Picture Buffer 207> The basic role of the inter prediction unit 205 is to derive one or more motion vectors (Mv: Motion vector) for the block to be decoded in order to accurately predict the block to be decoded in the subsequent adder 206, and to generate the decoded picture (hereinafter, referred to as the reference picture) different from the decoded picture to which the block to be decoded belongs. Predict the pixels (hereinafter, referred to as reference pixels) of the decoded block (hereinafter, referred to as the reference block) referred to by Mv within, that is, to perform inter prediction.

[0036] Figure 2 shows a conceptual diagram of inter prediction. Specifically, in Figure 2, in inter prediction, in order to generate inter prediction pixels of a decoding target block (CurBlk) within a decoding target picture (CurPic), two motion vectors MvL0 k and MvL1 k derived by the inter prediction unit 205 are used to reference two different reference blocks (RerBlkL0 k and RerBlkL1 k ) within two different reference pictures (RerPicL0 k and RerPicL1 k ). An example is shown.

[0037] Here, L0 and L1 indicate the numbers of lists (reference picture lists) in which reference pictures that can be referenced from the decoding target picture are described, and k corresponds to the value of control information (index) for specifying the motion vector described in the motion vector derivation method described later.

[0038] Similar to Non-Patent Document 1 and Non-Patent Document 2, the image decoding apparatus 200 according to the present embodiment may derive list 0 (L0) and list 1 (L1) as two different reference picture lists for each decoding target picture.

[0039] Here, the decoding picture buffer 207 may derive a reference picture list for each decoding target picture based on the control information decoded by the decoding unit 201.

[0040] For example, the control information may include information such as the number of reference pictures to be included in the reference picture list and candidates for reference pictures (picture reference structure) for the decoding target picture.

[0041] Details of the motion vector derivation method in the inter prediction unit 205 will be described later. The outline of the inter prediction pixel generation method in the inter prediction unit 205 is as follows.

[0042] The inter prediction unit 205 may generate inter prediction pixels from a single reference block, or as shown in FIG. 2, may generate inter prediction pixels using two reference blocks, or may generate inter prediction pixels using two or more reference blocks.

[0043] Here, as a method for the inter prediction unit 205 to generate inter prediction pixels using two reference blocks, the following method may be applied.

[0044] As an example, the inter prediction unit 205 may generate inter prediction pixels by simply averaging the pixel values of two reference blocks.

[0045] As another example, the inter prediction unit 205 may generate inter prediction pixels by weighted averaging the pixel values of two reference blocks in block units using predetermined weight values.

[0046] As a technique of such weighted averaging in block units, the weighted bi-prediction (BCW: Bi-prediction with CU-level Weights) disclosed in Non-Patent Document 1 and Non-Patent Document 2 may be applied to the inter prediction unit 205.

[0047] Note that in Non-Patent Documents 1 and 2, in BCW, a plurality of weight values including simple averaging, that is, a weight value of 1:1, can be selected, and such weight values are specified by the values of internal parameters (bcwIdx, the third internal parameter).

[0048] Here, the inter prediction unit 205 derives bcwIdx by inheritance from control information or the decoded picture buffer 207. Details of the inheritance will be described later.

[0049] As another example, the inter prediction unit 205 may divide the block to be decoded into two by a predetermined straight line, and generate inter prediction pixels by weighted averaging the pixel values of the reference blocks corresponding to the divided regions using weight values corresponding to the distances from the dividing straight line.

[0050] The inter prediction unit 205 may apply a Geometric Partitioning Mode (GPM) disclosed in Non-Patent Document 1 and Non-Patent Document 2 as a technique of weighted average for each region obtained by linearly bisecting such a block.

[0051] Here, when the motion vector refers to a fractional pixel accuracy position within the reference block, the inter prediction unit 205 may generate an inter prediction pixel using an interpolation filter for deriving the reference pixel value at such a fractional pixel accuracy position from the reference pixel values at the integer pixel accuracy positions in the vertical, horizontal, upper, lower, left, and right directions of such a fractional pixel position.

[0052] Furthermore, when generating an inter prediction pixel, the inter prediction unit 205 may adaptively apply two interpolation filters having different cut-off frequencies for inter prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0053] Among such two interpolation filters, the interpolation filter with a low cut-off frequency is called a Switchable Interpolation Filter (SIF).

[0054] Similar to Non-Patent Document 1 and Non-Patent Document 2, when the motion vector refers to a half pixel accuracy position and an internal parameter (hpelIfIdx, the first internal parameter) for specifying whether or not to apply the SIF indicates that the SIF is to be applied, the inter prediction unit 205 may apply the SIF.

[0055] Note that the inter prediction unit 205 derives hpelIfIdx by inheritance from control information or the decoded picture buffer 207. Details of the inheritance will be described later.

[0056] On the other hand, when the motion vector refers to a sub-pixel accuracy position, the inter prediction unit 205 may round (clip) the reference destination of the motion vector to the nearest integer pixel accuracy position of such sub-pixel accuracy position, and generate an inter prediction pixel without applying an interpolation filter.

[0057] In addition, when generating the final inter prediction pixel, the inter prediction unit 205 may apply local illumination compensation (LIC) disclosed in Non-Patent Document 2.

[0058] LIC is a technique for correcting a reference pixel value in a reference block by linear prediction based on decoded pixels adjacent to each of a block to be decoded and a reference block.

[0059] Similar to Non-Patent Document 1 and Non-Patent Document 2, the inter prediction unit 205 may apply LIC when an internal parameter (licFlag, the second internal parameter) for controlling whether to apply LIC indicates that LIC is to be applied.

[0060] Note that the inter prediction unit 205 derives the licFlag by inheritance from control information or the decoded picture buffer 207. Details of the inheritance will be described later.

[0061] In addition, when generating the final inter prediction pixel, the inter prediction unit 205 may apply multi-hypothesis prediction (MHP) disclosed in Non-Patent Document 2.

[0062] MHP is a technique for generating a final inter prediction pixel by weighted average from reference pixel values of at least three or more reference blocks.

[0063] Similar to Non-Patent Document 2, the inter prediction unit 205 may apply MHP when an internal parameter (mhpFlag, the fourth internal parameter) for specifying whether to apply MHP indicates that MHP is to be applied.

[0064] Note that the inter prediction unit 205 derives the mhpFlag by inheritance from control information or the decoded picture buffer 207. Details of the inheritance will be described later.

[0065] The inter prediction unit 205 sends the inter prediction pixels for the block to be decoded to the adder 206, and sends to the decoded picture buffer 207 the motion vector used for generating such inter prediction pixels, an index (reference index) for specifying the reference picture referenced by the motion vector within the reference picture list, and a counter (POC: Picture Order Count) representing the picture output order of the reference picture corresponding to the reference index. Hereinafter, the motion vector, reference index, and POC corresponding to the reference index used for generating these inter prediction pixels are referred to as inter prediction information. Further, the inter prediction unit 205 may send such inter prediction information including at least any one or all of the internal parameters bcwIdx, hpelIfIdx, licFlag, or mhpFlag for changing the above-described method for generating the inter prediction pixels.

[0066] The decoded picture buffer 207 accumulates the inter prediction information for the block to be decoded corresponding to the decoded pixels of the block to be decoded sent from the adder 206.

[0067] The decoded picture buffer 207 may accumulate such inter prediction information in a predetermined size (number of pixels). For example, the decoded picture buffer 207 may accumulate such inter prediction information in units of the minimum size (4×4 pixels) of the block to be decoded as disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0068] By the decoded picture buffer 207 accumulating such inter prediction information in the minimum size of the block to be decoded, the inter prediction accuracy when such inter prediction information is referenced from the block to be decoded that is decoded in the future can be improved.

[0069] The decoded picture buffer 207 accumulates decoded pixels in units of pictures to be decoded. Alternatively, the decoded picture buffer 207 may accumulate decoded pixels in units of slices to be decoded. Hereinafter, a case where decoded pixels are accumulated in units of pictures to be decoded will be described as an example.

[0070] When it is specified that the accumulated decoded pixels in the unit of the picture to be decoded and the inter-prediction information corresponding to the decoded pixels are not referred to as a reference picture and a reference block from a future picture to be decoded or a block to be decoded, the decoded pixels in the unit of the picture to be decoded and the inter-prediction information corresponding to the decoded pixels may be sequentially deleted. According to such a configuration, the amount of information accumulated in the decoded picture buffer 207 can be reduced.

[0071] The decoded picture buffer 207 may derive the number of pictures to be decoded to be accumulated and the reference picture structure between the pictures to be decoded based on the control information decoded by the decoding unit 201.

[0072] As a modification, when there are inter-prediction pixels in a predetermined area outside the picture to be decoded in addition to the picture to be decoded, the decoded picture buffer 207 may additionally accumulate such inter-prediction pixels.

[0073] FIG. 3 is a diagram for explaining reference pixel values referred to when an area outside the picture to be decoded accumulated in the decoded picture buffer 207 disclosed in each of Non-Patent Document 1 and Non-Patent Document 2 is referred to from a future block to be decoded.

[0074] As shown in FIG. 3, in Non-Patent Document 1 (VVC: Versatile Video Coding), when it is referred to from a future block to be decoded within a predetermined range outside the picture to be decoded, a pixel value at the nearest decoded picture boundary is copied to generate a reference pixel value.

[0075] In VVC, as such a predetermined range (padding area), a range obtained by adding 16 pixels to the maximum size (number of pixels) of the width of a block to be decoded is specified.

[0076] In Non-Patent Document 2 (ECM: Enhanced Compression Model), as shown in FIG. 3, between outside the picture to be decoded and the padding area, only a predetermined range is accumulated as an area (inter prediction padding area) where the inter prediction pixel values of the block to be decoded can be referred to from future blocks to be decoded.

[0077] Thereby, even if outside the picture to be decoded is referred to from a future block to be decoded, since there are inter prediction pixel values of the block to be decoded, the inter prediction accuracy of the future block to be decoded is improved.

[0078] FIG. 4 is a conceptual diagram in which the inter prediction padding area described in FIG. 3 is generated.

[0079] As shown in FIG. 4, when an adjacent block of a reference block corresponding to a block to be decoded has inter prediction pixels, even if the block to be decoded is in contact with the picture to be decoded, since inter prediction pixels can be generated outside the picture to be decoded, the decoded picture buffer 207 can accumulate the inter prediction padding area shown in FIG. 3.

[0080] <Method for Deriving Motion Vector in Inter Prediction Unit> Hereinafter, with reference to FIGS. 5 to 17, a method for deriving a motion vector in the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment will be described.

[0081] FIG. 5 is a diagram showing an example of a merge candidate list which is a method for deriving a motion vector in the inter prediction unit 205.

[0082] The merge candidate list is a list that stores candidates for inter prediction information (i.e., merge candidates) constructed by the inter prediction unit 205 to derive motion vectors in the merge (merge mode) disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0083] The inter prediction unit 205 extracts a merge candidate from among a plurality of merge candidates in such a merge candidate list based on an index (merge_idx) for specifying a merge candidate in the merge candidate list sent from the decoding unit 201, and uses it as inter prediction information for generating an inter prediction pixel.

[0084] The upper limit of the number of merge candidates stored in the merge candidate list may be a fixed value or a variable value. In the case of a variable value, for example, control information may be provided in a sequence parameter set or the like as in Non-Patent Document 1 and Non-Patent Document 2, and such a variable value can be set based on such control information.

[0085] The inter prediction unit 205 specifies, from within the merge candidate list, the inter prediction information to be used for generating inter prediction pixels based on one or more merge_idx sent from the decoding unit 201.

[0086] For example, for the above-described GPM, the inter prediction unit 205 may decode one merge_idx for each region obtained by linearly dividing the block to be decoded, and derive inter prediction information.

[0087] As shown in FIG. 5, the inter prediction unit 205 stores inter prediction information (merge candidates) corresponding to L0 and L1 of the merge candidate list.

[0088] Note that, in the example of FIG. 5, the inter prediction unit 205 describes only the motion vector and the reference index as the inter prediction information. However, in addition to these, it may store at least one or more or all of bcwIdx, hpelIfIdx, licFlag, or mhpFlag, which are internal parameters for changing the above-described method for generating inter prediction pixels.

[0089] The inter prediction unit 205 may search for and add inter prediction information, that is, inter prediction information at a position spatially or temporally adjacent or close to the block to be decoded, as a merge candidate, within the merge candidate list.

[0090] As an example, the inter prediction unit 205 may apply a combination of searches for a plurality of types of merge candidates disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0091] FIG. 6 shows the names of a plurality of types of merge candidates in regular merge disclosed in Non-Patent Document 1 and Non-Patent Document 2 and the derivation order of these merge candidates.

[0092] Specifically, in Non-Patent Document 1 (VVC), the inter prediction unit 205 checks the availability of these merge candidates in the order of spatial merge candidates, temporal motion vector prediction (TMVP) candidates, history-based motion vector prediction (HMVP) candidates, and pairwise merge candidates.

[0093] In Non-Patent Document 2 (ECM), in addition to the above-described merge candidates of Non-Patent Document 1, the inter prediction unit 205 checks for the use of non-adjacent (NA) spatial merge candidates between spatial merge candidates and TMVP candidates.

[0094] The outline of each merge candidate is as follows.

[0095] FIG. 7 is a diagram showing search positions of spatial merge candidates and TMVP candidates in a block to be decoded.

[0096] As shown in FIG. 7, in the spatial merge candidates, the inter prediction information is searched at positions of lower left (A0), left (A1), upper right (B0), upper (B1), and upper left (B2) that are adjacent to the block to be decoded (CurBlk), respectively.

[0097] That is, in the spatial merge candidates, the inter prediction information at positions spatially adjacent to the block to be decoded (CurBlk) is used.

[0098] Also, in the TMVP candidates, in a picture different from the block to be decoded (reference picture), the inter prediction information at positions of lower right center (Col) of the block to be decoded (CurBlk) shown in FIG. 7 and lower right (H) outside the block to be decoded (CurBlk) is searched.

[0099] That is, in the TMVP candidates, the inter prediction information at positions temporally adjacent to the block to be decoded is used.

[0100] In the HMVP candidates and non-adjacent spatial merge candidates, the inter prediction information at positions spatially distant (proximate) from the block to be decoded is used.

[0101] In particular, for the HMVP candidates, a table capable of storing FIFO-type inter-prediction information is constructed. Each time new inter-prediction information is searched for, the oldest inter-prediction information is discarded from the table, and such new inter-prediction information is added to the table.

[0102] Similar to the HMVP candidates and non-adjacent spatial merge candidates disclosed in Non-Patent Document 1 and Non-Patent Document 2, the inter-prediction unit 205 of the image decoding apparatus 200 according to the present embodiment can design the position for searching inter-prediction information and the size of the table for HMVP candidates.

[0103] In addition, in the search for merge candidates in the above-described normal merge, the inter-prediction unit 205 may re-search for a new reference block position from the peripheral pixels of the reference block of the inter-prediction information to be searched, and correct the inter-prediction information (motion vector) so that the difference (hereinafter referred to as the template matching cost) between the decoded pixels adjacent to the reference block based on the inter-prediction information to be searched (template of the reference block) and the decoded pixels adjacent to the block to be decoded (template of the block to be decoded) becomes small.

[0104] As an example of this, the inter-prediction unit 205 may apply the template matching merge disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0105] In addition, in the selection of merge candidates from the merge candidate list in the above-described normal merge, the inter-prediction unit 205 derives the L0 inter-prediction information from the control information, and selects, as the L1 merge candidate, the merge candidate for which the difference between the templates of the reference blocks based on the inter-prediction information stored in L1 of the merge candidate list is minimized with respect to the template of the reference block based on the L0 inter-prediction information derived from such control information.

[0106] Based on the L0 inter prediction information derived from such control information, the inter prediction unit 205 can derive the merge candidates of L1 (i.e., the inter prediction information). Therefore, in addition to the control information necessary for deriving the L0 inter prediction information, it does not require decoding of merge_idx to derive the L1 inter prediction information.

[0107] As an example, the inter prediction unit 205 may apply the Adaptive motion vector Prediction-Merge mode disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0108] As a modification example, the inter prediction unit 205 may reverse L0 and L1 in the above example. That is, the inter prediction unit 205 may derive the L1 inter prediction information from control information and derive the L0 inter prediction information from the merge candidate list.

[0109] In addition, the inter prediction unit 205 may construct a merge candidate list for merge for deriving inter prediction information in units of sub-blocks obtained by dividing the block to be decoded, and derive the inter prediction information.

[0110] As an example, the inter prediction unit 205 may construct a merge candidate list for SbTMVP (Sub-block TMVP) or Affine merge disclosed in Non-Patent Document 1 and Non-Patent Document 2, and derive the inter prediction information.

[0111] <1. Basic concept of beaded motion vector prediction (derivation of beaded motion vector)> The methods for deriving the inter prediction information (hereinafter, referred to as motion vectors for simplicity of explanation) by the above-described inter prediction unit are all limited to those stored at positions spatially or temporally adjacent or close to the block to be decoded.

[0112] For further improvement of the symbolization efficiency, the inter prediction unit 205 may derive, as the motion vector of the block to be decoded, a motion vector generated by adding a motion vector recursively searched and derived from the motion vectors stored in the decoded picture buffer 207 to the motion vectors at positions spatially or temporally adjacent or close to the block to be decoded.

[0113] Here, regarding the technique of recursively (in a chain) searching the motion vectors stored in the decoded picture buffer 207 starting from the motion vectors at positions spatially or temporally adjacent or close to the block to be decoded to generate a new motion vector (chained motion vector), this specification refers to it as "Chained Motion Vector Prediction (CMVP)". Hereinafter, the technical details of such chained motion vector prediction will be described in detail.

[0114] Using FIGS. 8 to 17, the method for deriving the chained motion vector in the chained motion vector prediction will be described.

[0115] FIG. 8 is a diagram showing an example of the method for deriving the chained motion vector in the chained motion vector prediction.

[0116] As shown in FIG. 8, the inter prediction unit 205 starts from the motion vector MvL0 k (0) of L0 corresponding to any merge candidate stored in the merge candidate list and the reference index, and recursively (in a chain) searches the motion vectors stored in the decoded picture buffer 207 to generate a new motion vector.

[0117] For example, as shown in (Equation 1) and (Equation 2), the inter prediction unit 205 can derive a new motion vector generated by chained motion vector prediction for the reference block RefBlk of the reference picture RefPicL0 k (m) and the reference picture.

[0118] MvL0 k / m =MvL0k (0) + MvL0 k (1) + MvL0 k (2) + … + MvL0 k (m) (Equation 1) RefPicL0 k / m = RefPicL0 k (m) (Equation 2) That is, the new motion vector MvL0 generated by the beaded string motion vector prediction k / m can be calculated by adding all the motion vectors derived by searching the motion vectors stored in the decoded picture buffer 207 recursively (in a beaded string manner).

[0119] Also, the new reference picture generated by the beaded string motion vector prediction is the reference picture referred to by MvL0 k / m is the reference picture.

[0120] FIG. 9 is a diagram showing an example of a method for deriving a beaded string motion vector in beaded string motion vector prediction.

[0121] As shown in FIG. 9, in the beaded string motion vector prediction, the inter prediction unit 205 may add a block vector (a vector indicating a reference destination within the same picture) to the search target recursively in addition to the search for the motion vector.

[0122] For example, when a block vector (Bv: Block vector) is added, in the derivation formula of the new motion vector generated by the beaded string motion vector prediction for the reference block RefBlk of the reference picture RefPicL0 k (m), Bv is added as shown in (Equation 3).

[0123] MvL0 k / m = MvL0 k (0) + Bv k (0) + MvL0 k (1) + MvL0 k (2) + … + MvL0 k (m) (Equation 3) FIG. 10 is a diagram showing an example of a recursive search of motion vectors including dual prediction (i.e., inter prediction using two different motion vectors for L0 and L1). FIG. 11 is a diagram showing an example of a beaded motion vector derived by a recursive search of motion vectors including dual prediction.

[0124] As shown in FIGS. 10 and 11, in the beaded motion vector prediction, the inter prediction unit 205 uses each search list number (L0 or L1) and each search depth (or each search count) of the motion vectors stored in the decoded picture buffer 207 to newly search and derive one or more (up to two in the examples of FIGS. 10 and 11) motion vectors from one reference motion vector, and adds all the already derived motion vectors to each of the one or more newly derived motion vectors to obtain the beaded motion vector MvL0 k / m is generated.

[0125] FIGS. 12 and 13 are diagrams showing an example of searching for beaded motion vector prediction candidates after searching for multiple types of merge candidates of normal merge disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0126] As shown in FIGS. 12 and 13, the inter prediction unit 205 may search for beaded motion vector prediction candidates after searching for a predetermined merge candidate.

[0127] For example, as shown in FIG. 12, in VVC, the inter prediction unit 205 may search for beaded motion vector prediction candidates after searching for TMVP candidates.

[0128] Since the beaded motion vector prediction searches for new motion vectors using the motion vectors of the reference picture rather than the picture to be decoded stored in the decoded picture buffer 207, it will not be effective unless TMVP is effective in the sequence to be decoded or the group of pictures to be decoded or the picture to be decoded or the slice to be decoded.

[0129] Therefore, searching for the string-of-beads motion vector prediction candidates after searching for the TMVP candidates can be said to be a natural design method as a method for deriving the motion vector.

[0130] As a modification example of FIG. 12, in the case of VVC and ECM, as shown in FIG. 13, the inter prediction unit 205 may search for the string-of-beads motion vector prediction candidates after searching for the HMVP candidates.

[0131] Since the string-of-beads motion vector prediction generates a new motion vector by recursively searching for the motion vector, it must be applied in a state where at least one merge candidate is stored in the merge candidate list.

[0132] As shown in FIG. 13, by searching for the string-of-beads motion vector prediction candidates after searching for the HMVP candidates for which the storage of one or more motion vector candidates can be expected in the merge candidate list, the possibility of generating a new motion vector by the string-of-beads motion vector is increased.

[0133] As another modification example, in the case of ECM, as shown in FIG. 12, the inter prediction unit 205 may search for the string-of-beads motion vector prediction candidates after searching for the spatial merge candidates.

[0134] Although the search position of the spatial merge candidates is shown in FIG. 7, for example, if the search position of the spatial merge candidates is increased, the possibility that one or more motion vector candidates are stored in the merge candidate list is increased, and it becomes easier to generate a new motion vector candidate by the string-of-beads merge candidate.

[0135] FIGS. 14 to 17 are diagrams showing an example of the position (index number) in the merge candidate list where the merge candidate which is the derivation target of the string-of-beads motion vector prediction candidate and the string-of-beads motion vector prediction candidate are added.

[0136] Note that in FIGS. 14 to 17, for the sake of simplifying the description of the technical content, only the motion vectors are described as the inter-prediction information stored in the merge candidate list, but the above-described reference indexes are also stored.

[0137] In addition to the motion vectors and the reference indexes, the inter-prediction unit 205 may store bcwIdx, hpelIfIdx, licFlag, and mhpFlag, which are internal parameters for changing the above-described inter-prediction pixel generation method.

[0138] As shown in FIG. 14, after storing a predetermined merge candidate in the merge candidate list, the inter-prediction unit 205 sequentially applies the chain motion vector prediction from the first merge candidate stored in the merge candidate list, and may sequentially store all of the new merge candidates generated by the chain motion vector prediction for each merge candidate in the merge candidate list.

[0139] Here, the predetermined merge candidate may be any of the plurality of types of merge candidates in the normal merge described with reference to FIGS. 12 and 13, or may be a predetermined number (predetermined number of merge_idx) of merge candidates.

[0140] Such a predetermined number may be set as a fixed value, or may be set as a variable value in the decoding target sequence or the decoding target picture group or the decoding target picture or the decoding target slice unit by using the control information of the decoding target sequence or the decoding target picture group or the decoding target picture or the decoding target slice unit.

[0141] As a modification example of FIG. 14, as shown in FIG. 15, after storing a predetermined merge candidate in the merge candidate list, the inter-prediction unit 205 sequentially applies the chain motion vector prediction from the first merge candidate stored in the merge candidate list, and may sequentially store the new chain motion vectors (i.e., merge candidates) generated at each search depth of the chain motion vector prediction for each merge candidate in the merge candidate list.

[0142] As a modification example of FIGS. 14 and 15, as shown in FIG. 16, after the first merge candidate is stored in the merge candidate list, the inter prediction unit 205 sequentially applies the bead string movement vector prediction from the merge candidate, and all of the new bead string movement vectors (i.e., merge candidates) generated by the bead string movement vector prediction for each merge candidate may be sequentially stored in the merge candidate list.

[0143] As a modification example of FIGS. 14 to 16, as shown in FIG. 17, after the first merge candidate is stored in the merge candidate list, the inter prediction unit 205 sequentially applies the bead string movement vector prediction from the merge candidate, and the new bead string movement vectors (i.e., merge candidates) generated at each search depth of the bead string movement vector prediction for each merge candidate may be sequentially stored in the merge candidate list.

[0144] As a modification example of FIGS. 14 to 17, the inter prediction unit 205 may combine the bead string movement vectors generated using each search depth of the bead string movement vector prediction for each merge candidate list with the bead string movement vectors generated at a search depth different from each such search depth to generate new merge candidates.

[0145] As a modification example of FIGS. 14 to 17, the inter prediction unit 205 may combine the bead string movement vectors generated using each search list number or each search depth of the bead string movement vector prediction for each merge candidate list with the bead string movement vectors generated using a search list number or a search depth different from each such search list number or each search depth to generate new merge candidates.

[0146] <2. Pruning Process of Bead String Movement Vector Prediction Candidates> Using FIG. 18, the pruning process when the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment adds a merge candidate (the above-described bead string movement vector prediction candidate) corresponding to the bead string movement vector derived by the bead string movement vector prediction to the merge candidate list will be described.

[0147] FIG. 18 is a diagram showing an example of a pruning process when the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment adds a merge candidate corresponding to a bead string motion vector derived by bead string motion vector prediction to a merge candidate list.

[0148] The inter prediction unit 205 determines whether to add a merge candidate corresponding to a bead string motion vector derived by bead string motion vector prediction based on a predetermined condition.

[0149] Specifically, as shown in FIG. 18, when the inter prediction unit 205 determines in step S000 that a predetermined condition is satisfied, it determines in step S002 not to add a merge candidate corresponding to a bead string motion vector derived by bead string motion vector prediction.

[0150] On the other hand, when the inter prediction unit 205 determines in step S000 that the predetermined condition is not satisfied, it determines in step S001 to add a merge candidate corresponding to a bead string motion vector derived by bead string motion vector prediction.

[0151] Here, for example, the predetermined condition may be that the reference pictures and motion vectors associated with at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector match.

[0152] When the inter prediction unit 205 determines that the reference pictures and motion vectors associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector match, the merge candidate corresponding to the bead string motion vector is not added to the merge candidate list, that is, it is determined that the merge candidate corresponding to the bead string motion vector is pruned (to be a pruning target), thereby preventing a case where two or more identical motion vectors are stored in the merge candidate list, reducing the amount of code of merge_idx (merge index) sent to the image decoding apparatus 200, and improving the coding efficiency.

[0153] Alternatively, the predetermined condition may be that all of one or more reference pictures and motion vectors associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector match.

[0154] Alternatively, the predetermined condition may be that the reference pictures and motion vectors of merge candidate list number 0 (L0) and merge candidate list number 1 (L1) associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector all match.

[0155] Furthermore, the predetermined condition may be that the reference pictures associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector match, and the error (Euclidean distance) of the reference positions of the motion vectors associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector is less than a predetermined threshold.

[0156] As a result, the pruning process of merge candidates corresponding to the beaded movement vector is strengthened, cases where two or more similar movement vectors are stored in the merge candidate list are more effectively prevented, the amount of code of merge_idx sent to the image decoding apparatus 200 is reduced, and the coding efficiency is improved.

[0157] As a further control, the inter prediction unit 205 may set a predetermined threshold value when there is one motion vector to be compared to be larger than a predetermined threshold value when there are two motion vectors to be compared.

[0158] In this way, by setting the predetermined threshold value when there is one motion vector to be compared to be larger than the predetermined threshold value when there are two motion vectors to be compared, a more accurate pruning process becomes possible.

[0159] Here, the inter prediction unit 205 may set the predetermined threshold value when there is one motion vector to be compared to 1.5 pixels.

[0160] Alternatively, the inter prediction unit 205 may set the predetermined threshold value when there is one motion vector to be compared to 1.0 pixel or 2.0 pixels.

[0161] Also, the inter prediction unit 205 may set the predetermined threshold value when there are two motion vectors to be compared to 1.0 pixel.

[0162] Alternatively, the inter prediction unit 205 may set the predetermined threshold value when there are two motion vectors to be compared to 0.5 pixel or 1.5 pixels.

[0163] Further, the above-described predetermined conditions may also be that internal parameters that change the reference picture, the motion vector, and the generation method of the predetermined inter prediction pixel associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived beaded motion vector match.

[0164] Alternatively, the predetermined condition may be that the internal parameter (hpelIfIdx) that specifies the reference picture, motion vector, and whether or not to apply the switching interpolation filter associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector matches.

[0165] Alternatively, the predetermined condition may be that the internal parameter (licFlag) that controls the reference picture, motion vector, and whether or not to apply local luminance compensation associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector matches.

[0166] Furthermore, the predetermined condition may be that the internal parameter (bcwIdx) that specifies the reference picture, motion vector, and the weight value of weighted bi-prediction in block units associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector matches.

[0167] By including the above-described internal parameters in the predetermined condition, the inter prediction unit 205 can leave room for improving the inter prediction accuracy by generating inter prediction pixels in different ways when the reference pictures and motion vectors to be compared match but the internal parameters are different, so that an improvement in coding efficiency can be expected.

[0168] <3. Control of Reference Source and Reference Destination When Deriving Bead String Motion Vector> With reference to FIGS. 19 to 22, a method for controlling the reference source and reference destination when the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment derives the bead string motion vector in the bead string motion vector prediction will be described.

[0169] First, with reference to FIG. 19, a method for controlling the reference source when deriving the bead string motion vector in the bead string motion vector prediction will be described.

[0170] FIG. 19 is a diagram showing an example of a method for controlling a reference source when deriving a bead string movement vector in the bead string movement vector prediction by the inter prediction unit 205.

[0171] When deriving the bead string movement vector in the bead string movement vector prediction, the inter prediction unit 205 may set the starting point of the first movement vector constituting the bead string movement vector to a part or all of the block to be decoded.

[0172] As an example of setting to a part of the block to be decoded, as shown in FIG. 19, when deriving the bead string movement vector in the bead string movement vector prediction, the inter prediction unit 205 may set the starting point of the first movement vector constituting the bead string movement vector to the center (C) of the block to be decoded (CurBlk).

[0173] Alternatively, as shown in FIG. 19, when deriving the bead string movement vector in the bead string movement vector prediction, the inter prediction unit 205 may, in addition to the center (C) of the block to be decoded (CurBlk), set the starting point of the first movement vector constituting the bead string movement vector to a predetermined position of the block to be decoded (CurBlk), and search whether movement vectors are accumulated at the reference destinations of the movement vectors in a predetermined order.

[0174] Specifically, as shown in FIG. 19, when deriving the bead string movement vector in the bead string movement vector prediction, the inter prediction unit 205 may, in addition to the center (C) of the block to be decoded (CurBlk), set the starting point of the first movement vector constituting the bead string movement vector to the four corners of the block to be decoded (CurBlk), and search whether movement vectors are accumulated at the reference destinations of the movement vectors in the order of the center (C), top left (TL), top right (TR), bottom left (BL), and bottom right (BR) of the block to be decoded as a predetermined order.

[0175] The inter prediction unit 205 may generate a bead string movement vector for each movement vector derived by searching at each point.

[0176] As a modification example, the inter prediction unit 205 may compare the differences between the adjacent decoded pixel values of the reference block and the block to be decoded of each motion vector derived by searching at each point, and limit to at least one or more motion vectors with small such differences to generate a connected motion vector.

[0177] As another modification example, the inter prediction unit 205 may derive a new motion vector for generating a connected motion vector by weighted-averaging each motion vector derived by searching at each point.

[0178] Specifically, when weighted-averaging each motion vector, the inter prediction unit 205 may scale each motion vector based on the distance of the reference picture of each motion vector from the block to be decoded to generate a new motion vector. The scaling destination may be set to the reference picture with the minimum distance from the block to be decoded.

[0179] When deriving a connected motion vector, the inter prediction unit 205 sets the starting point of the first motion vector constituting the connected motion vector not only at the center of the block to be decoded but also at the four corners of the block to be decoded, which increases the possibility of deriving a connected motion vector, and as a result, an improvement in coding efficiency can be expected.

[0180] As further control, in the connected motion vector prediction, the inter prediction unit 205 may control the starting point of the first motion vector constituting the connected motion vector according to the block size or block aspect ratio of the block to be decoded when deriving the connected motion vector.

[0181] For example, in the connected motion vector prediction, when deriving the connected motion vector, the inter prediction unit 205 may set the starting point of the first motion vector constituting the connected motion vector only at the center of the block to be decoded when the block size of the block to be decoded is equal to or less than the first size.

[0182] When the block size of the block to be decoded is less than or equal to the first size, when deriving the beaded movement vector, the first movement vector that constitutes the beaded movement vector is likely to be the same at the center and the four corners of the block to be decoded.

[0183] Therefore, when the block size of the block to be decoded is less than or equal to the first size, when deriving the beaded movement vector, by setting the starting point of the first movement vector that constitutes the beaded movement vector only at the center of the block to be decoded (without adding the four corners of the block to be decoded), the search processing amount of the beaded movement vector can be reduced.

[0184] Alternatively, in the beaded movement vector prediction, when the inter prediction unit 205 derives the beaded movement vector, when the block size of the block to be decoded is larger than the first size, in addition to the center of the block to be decoded, the starting point of the first movement vector that constitutes the beaded movement vector is set at the four corners of the block to be decoded, and in the predetermined order of the center, upper left, upper right, lower left, and lower right of the block to be decoded, it may be searched whether the movement vector is stored at the reference destination of the movement vector.

[0185] When the block size of the block to be decoded is larger than the first size, when deriving the beaded movement vector, the first movement vector that constitutes the beaded movement vector is likely to be different at the center and the four corners of the block to be decoded.

[0186] Therefore, when the block size of the block to be decoded is larger than the first size, when deriving the beaded movement vector, by setting the starting point of the first movement vector that constitutes the beaded movement vector at the four corners of the block to be decoded in addition to the center of the block to be decoded, different beaded movement vectors are generated, and an improvement in coding efficiency can be expected.

[0187] Here, since a case where the block to be decoded is non-square is also assumed, the block size (the number of pixels in the block) of the block to be decoded may be replaced with the width and height of the block to be decoded, and the above control may be performed.

[0188] Specifically, when deriving the bead string motion vector in the bead string motion vector prediction, the motion vector prediction unit 205 sets the starting point of the first motion vector constituting the bead string motion vector to the four corners of the block to be decoded in addition to the center of the block to be decoded when the width or height of the block of the block to be decoded is larger than a predetermined number of pixels. Then, it may be searched whether the motion vectors are accumulated at the reference destinations of the motion vectors in the order of the center, upper left, upper right, lower left, and lower right of the block to be decoded as a predetermined order.

[0189] The motion vector prediction unit 205 may set any one of 8 pixels, 16 pixels, or 32 pixels as such a predetermined number of pixels.

[0190] Alternatively, when deriving the bead string motion vector in the bead string motion vector prediction, the motion vector prediction unit 205 sets the lower left and upper right of the block to be decoded in addition to the center of the block to be decoded when the aspect ratio of the block of the block to be decoded is larger than a predetermined ratio. Then, it may be searched whether the motion vectors are accumulated at the reference destinations of the motion vectors in the order of the center, lower left, and upper right of the block to be decoded as a predetermined order.

[0191] The motion vector prediction unit 205 may set any one of 4, 8, or 16 as such a predetermined ratio.

[0192] Second, an example of a control method for the reference destination when deriving the bead string motion vector in the bead string motion vector prediction will be described with reference to FIGS. 20 and 21.

[0193] FIG. 20 is a diagram showing an example of a method for controlling a reference destination when deriving a rosary connection motion vector in the rosary connection motion vector prediction by the inter prediction unit 205.

[0194] As shown in FIG. 20, in the recursive motion vector search process when the inter prediction unit 205 derives a rosary connection motion vector, when the reference position of each motion vector refers to outside the reference picture, the motion vector may be searched at the accumulation location of the motion vector within the nearest reference picture from such a reference position.

[0195] Since the reference position of the motion vector may refer to outside the reference picture, and there may be a case where the rosary-connected motion vector from such a motion vector is in a different reference picture, a new rosary connection motion vector can be generated.

[0196] Alternatively, as shown in FIG. 20, in the recursive motion vector search process when the inter prediction unit 205 derives a rosary connection motion vector, when the reference position of each motion vector refers to outside the reference picture, the generation of the rosary connection motion vector may be terminated.

[0197] When the reference position of the motion vector refers to outside the reference picture, it is highly likely that the rosary-connected motion vector from such a motion vector also refers to outside the reference picture in a different reference picture. Therefore, by terminating the generation of the rosary connection motion vector, the decoding processing amount can be reduced.

[0198] Alternatively, when the decoding picture buffer 207 stores an inter prediction padding region outside the reference picture, the description of "outside the reference picture" above may be replaced with "outside the inter prediction padding region", and the inter prediction unit 205 may perform similar processing.

[0199] FIG. 21 is a diagram showing an example of a method for controlling a reference source and a reference destination when deriving a chained motion vector in chained motion vector prediction by an inter prediction unit 205 in a case where sequential decoder refresh (GDR: Gradual Decoding Refresh), which is known as an encoding / decoding technique for low-latency video transmission, is applied to a group of pictures to be decoded.

[0200] GDR is a technique for reducing the processing delay of video encoding / decoding at a certain bit rate as compared with video encoding / decoding in a random access configuration including an intra picture widely used in video transmission by updating only a partial area of a screen by encoding / decoding using intra prediction (hereinafter referred to as intra update) and moving such a partial area for each picture to be decoded.

[0201] FIG. 21 illustrates an intra update area by a slice (Intra slice), and shows an example in which such a slice moves from the upper end to the lower end of a picture to be decoded.

[0202] Here, in this specification, three divided areas by GDR in a picture to be decoded are referred to as an area after intra update, an area during intra update, and an area before intra update.

[0203] As shown in FIG. 21, when sequential decoder refresh is applied to a group of pictures to be decoded, the inter prediction unit 205 may limit the recursive search for a motion vector of chained motion vector prediction so as not to exceed the area during intra update.

[0204] For example, as shown in FIG. 21, when sequential decoder refresh is applied to a group of pictures to be decoded and a block to be decoded is included in the area after intra update, the inter prediction unit 205 may limit the area that can be referred to in each search for a recursive motion vector of chained motion vector prediction to only the area after intra update in the reference picture for each search target.

[0205] Alternatively, when the decoder update is sequentially applied to the pictures to be decoded in the inter prediction unit 205 and the block to be decoded is included in the area before intra update, the areas that can be referenced in each search for the recursive motion vectors of the connected motion vector prediction may be limited to only the areas before intra update in the reference pictures for each search target.

[0206] In this way, when the decoder update is sequentially applied to the pictures to be decoded in the inter prediction unit 205, by limiting the search for the recursive motion vectors of the connected motion vector prediction so as not to exceed the area during intra update, a connected motion vector with high prediction accuracy can be generated.

[0207] FIG. 22 is a diagram showing an example of controlling the reference destination when the connected motion vector in the connected motion vector prediction is derived by the inter prediction unit 205 in a case where reference picture resampling (RPR) disclosed in Non-Patent Document 1, in which reference to motion vectors to reference pictures with different resolutions (sampling ratios) from the decoded picture is allowed, is applied to the block to be decoded.

[0208] In RPR, the inter prediction unit 205 calculates the ratios of the width and height of the reference picture to the width and height of the decoded picture, respectively, and scales the pixel position and motion vector length of the reference picture with respect to the decoded picture according to the ratios to generate an inter prediction pixel.

[0209] Therefore, for example, as shown in FIG. 22, when the width and height of the reference picture are 0.5 times the width and height of the decoded picture, the pixel position and motion vector length of the reference picture with respect to the decoded picture are scaled by 0.5 times.

[0210] On the other hand, when generating the connected motion vector, the inter prediction unit 205 needs to connect the motion vectors at the same scale as the decoded picture.

[0211] Therefore, as shown in FIG. 22, when the inter prediction unit 205 refers to motion vectors for reference pictures with a resolution (sampling ratio) different from that of the picture to be decoded, recursively searches for motion vectors from such reference pictures, and generates a chained motion vector, the length of the recursively derived motion vector may be inversely scaled according to the ratios of the width and height of the picture to be decoded and the width and height of each reference picture, and then added to generate a chained motion vector.

[0212] As a modification example, when reference to motion vectors is allowed for reference pictures with a resolution (sampling ratio) different from that of the picture to be decoded for the sequence to be decoded or the group of pictures to be decoded, the inter prediction unit 205 may apply the chained motion vector only to reference pictures with the same sampling ratio as the picture to be decoded.

[0213] In other words, when reference to motion vectors is allowed for reference pictures with a resolution (sampling ratio) different from that of the picture to be decoded for the sequence to be decoded or the group of pictures to be decoded, the inter prediction unit 205 may choose not to apply the chained motion vector to reference pictures with a sampling ratio different from that of the picture to be decoded.

[0214] <4. Inheritance of Inter Prediction Information in Chained Motion Vector Prediction> When deriving merge candidates by chained motion vector prediction, the inter prediction unit 205 may control whether to inherit predetermined internal parameters for changing the method of generating inter prediction pixels.

[0215] (Switching Interpolation Filter) As an example, when deriving merge candidates by beaded movement vector prediction, the inter prediction unit 205 does not inherit the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter associated with each recursively searched movement vector, and always sets the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter corresponding to the merge candidate derived by beaded movement vector prediction so as to indicate that the switching interpolation filter is not applied.

[0216] Alternatively, when deriving merge candidates by beaded movement vector prediction, the inter prediction unit 205 may inherit, as the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter corresponding to the merge candidate derived by beaded movement vector prediction, the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter associated with the first or last movement vector constituting the beaded movement vector among the internal parameters (hpelIfIdx) that specify whether to apply the switching interpolation filter associated with each recursively searched movement vector.

[0217] Alternatively, when deriving merge candidates by beaded movement vector prediction, among the internal parameters (hpelIfIdx) that specify whether to apply the switching interpolation filter associated with each recursively searched movement vector, if at least one of the internal parameters (hpelIfIdx) that specify whether to apply the switching interpolation filter indicates that the switching interpolation filter is applied, it may be determined that the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter corresponding to the merge candidate derived by beaded movement vector prediction indicates that the switching interpolation filter is applied.

[0218] Furthermore, when the rosary movement vector predicted by the inter prediction unit 205 is composed of at least one or more block vectors, the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter corresponding to the merge candidate derived by the rosary movement vector prediction may always be set to indicate that the switching interpolation filter is not applied.

[0219] (Local Luminance Compensation) As an example, when deriving a merge candidate by the rosary movement vector prediction, the inter prediction unit 205 does not inherit the internal parameter (licFlag) that controls whether to apply the local luminance compensation associated with each recursively searched motion vector, and the internal parameter (licFlag) that controls whether to apply the local luminance compensation corresponding to the merge candidate derived by the rosary movement vector prediction may always be set to indicate that the local luminance compensation is not applied.

[0220] Alternatively, when deriving a merge candidate by the rosary movement vector prediction, among the internal parameters (licFlag) that control whether to apply the local luminance compensation associated with each recursively searched motion vector, the internal parameter (licFlag) that controls whether to apply the local luminance compensation associated with the first or last motion vector that constitutes the rosary movement vector may be inherited as the internal parameter (licFlag) that controls whether to apply the local luminance compensation corresponding to the merge candidate derived by the rosary movement vector prediction.

[0221] Alternatively, when deriving a merge candidate by the rosary movement vector prediction, among the internal parameters (licFlag) that control whether to apply the local luminance compensation associated with each recursively searched motion vector, if at least one or more of the internal parameters (licFlag) that control whether to apply the local luminance compensation indicate that the local luminance compensation is applied, the internal parameter (licFlag) that controls whether to apply the local luminance compensation corresponding to the merge candidate derived by the rosary movement vector prediction may be set to indicate that the local luminance compensation is applied.

[0222] Furthermore, when the rosary movement vector generated by the rosary movement vector prediction is composed of at least one or more block vectors, the inter prediction unit 205 may always set an internal parameter (licFlagg) for controlling whether to apply local luminance compensation corresponding to the merge candidate derived by the rosary movement vector prediction so as to indicate that local luminance compensation is not applied.

[0223] (Weighted bi-prediction in block units) As an example, when deriving a merge candidate by rosary movement vector prediction, the inter prediction unit 205 does not inherit an internal parameter (bcwIdx) for specifying the weight value of weighted bi-prediction in block units associated with each recursively searched motion vector, and the internal parameter (bcwIdx) for specifying the weight value of weighted bi-prediction in block units corresponding to the merge candidate derived by rosary movement vector prediction may always be set so as to indicate that the weight value of weighted bi-prediction in block units is a simple average.

[0224] Alternatively, when deriving a merge candidate by rosary movement vector prediction, among the internal parameters (bcwIdx) for specifying the weight values of weighted bi-prediction in block units associated with each recursively searched motion vector, the internal parameter (bcwIdx) for specifying the weight value of weighted bi-prediction in block units associated with the first or last motion vector constituting the rosary movement vector may be inherited as the internal parameter (bcwIdx) for specifying the weight value of weighted bi-prediction in block units corresponding to the merge candidate derived by rosary movement vector prediction.

[0225] As a modification example, when the inter prediction unit 205 derives a merge candidate by beaded movement vector prediction, among the internal parameters (bcwIdx) that specify the weighted bi-prediction weight values for each block unit associated with each recursively searched motion vector, if at least one or more internal parameters (bcwIdx) that specify the weighted bi-prediction weight values for each block unit indicate that the weighted bi-prediction weight value for each block unit is a simple average, the internal parameter (bceIdx) that specifies the weighted bi-prediction weight value for each block unit corresponding to the merge candidate derived by beaded movement vector prediction may also indicate that the weighted bi-prediction weight value for each block unit is a simple average.

[0226] (Multiple hypothesis prediction) As an example, when the inter prediction unit 205 derives a merge candidate by beaded movement vector prediction, without inheriting the internal parameters (mhpFlag, the fourth internal parameter) that control the applicability of multiple hypothesis predictions associated with each recursively searched motion vector, the internal parameter (mhpFlag) that controls the applicability of multiple hypothesis predictions corresponding to the merge candidate derived by beaded movement vector prediction may always be set to indicate that multiple hypothesis predictions are not applied.

[0227] As a modification example, when the inter prediction unit 205 derives a merge candidate by beaded movement vector prediction, among the internal parameters (mhpFlag) that control the applicability of multiple hypothesis predictions associated with each recursively searched motion vector, the internal parameter (mhpFlag) that controls the applicability of multiple hypothesis predictions associated with the first or last motion vector constituting the beaded movement vector may be inherited as the internal parameter (mhpFlag) that controls the applicability of multiple hypothesis predictions corresponding to the merge candidate derived by beaded movement vector prediction.

[0228] As a modification example, when deriving a merge candidate by beaded movement vector prediction, among the internal parameters (mhpFlag) that control whether or not to apply a plurality of hypothesis predictions associated with each movement vector recursively searched, if at least one or more of the internal parameters (mhpFlag) that control whether or not to apply a plurality of hypothesis predictions indicate that the plurality of hypothesis predictions are to be applied, it may be determined that the internal parameter (mhpFlag) that controls whether or not to apply a plurality of hypothesis predictions corresponding to the merge candidate derived by beaded movement vector prediction indicates that the plurality of hypothesis predictions are to be applied.

[0229] As a further modification example, when the beaded movement vector generated by beaded movement vector prediction is composed of at least one or more block vectors, the internal parameter (mhpFlag) that controls whether or not to apply a plurality of hypothesis predictions corresponding to the merge candidate derived by beaded movement vector prediction may always be set to indicate that the plurality of hypothesis predictions are not to be applied.

[0230] As described above, if the internal parameter that changes the method for generating an inter-predicted pixel is not always inherited by beaded movement vector prediction, the amount of code of the inter-prediction information that needs to be stored in the decoded picture buffer 207 is reduced.

[0231] On the other hand, as described above, if the internal parameter that changes the method for generating an inter-predicted pixel is inherited by beaded movement vector prediction, the inter-predicted pixel using the merge candidate generated by beaded movement vector prediction may be of higher accuracy, and an improvement in coding efficiency can be expected.

[0232] Also, when inheriting the above-described internal parameter, when the beaded movement vector is composed of at least one or more block vectors, since there is little need to change the method for generating an inter-predicted pixel, by not inheriting such an internal parameter, that is, by setting not to change the method for generating an inter-predicted pixel, an improvement in coding efficiency can be expected.

[0233] <5. Rearrangement of Merge Candidates Containing Bead String Movement Vector Prediction Candidates and Vector Correction> (Rearrangement of Merge Candidates) The later the bead string movement vector prediction candidate is derived compared to other merge candidates, the more difficult it is to store the bead string movement vector prediction candidate in the merge candidate list.

[0234] As a method for solving this problem, the inter prediction unit 205 calculates the template matching cost of all merge candidates stored in the merge candidate list including the merge candidates derived by the bead string movement vector prediction, and may rearrange the storage positions of each merge candidate in the merge candidate list in ascending order according to the ascending order of the template matching cost.

[0235] As an example, as a technique for rearranging the storage positions of each merge candidate in the merge candidate list based on the template matching cost of each merge candidate, the inter prediction unit 205 may use the Adaptive Reordering Merge Candidate (ARMC) disclosed in Non-Patent Document 2.

[0236] Furthermore, as described above, since the bead string movement vector prediction recursively generates new motion vectors, if other merge candidates are derived after the derivation of the bead string movement vector prediction candidate, there is a possibility that the other merge candidates will not be stored in the merge candidate list.

[0237] As a method for solving this problem, the inter prediction unit 205 calculates the template matching cost of all unpruned merge candidates including the merge candidates derived by the bead string movement vector prediction, and may store them in the merge candidate list in ascending order according to the ascending order of the template matching cost.

[0238] Alternatively, the inter prediction unit 205 may calculate the template matching costs of all the unpruned merge candidates including the merge candidates derived by the bead-chain motion vector prediction, and perform a series of processes until they are stored in ascending order in the merge candidate list in ascending order of the template matching costs for a predetermined number of times. For example, such a predetermined number of times may be two or three times.

[0239] As a result, merge candidates capable of generating inter prediction pixels with high accuracy are arranged in the merge candidate list, and an improvement in coding efficiency can be expected.

[0240] (Vector Correction) The inter prediction unit 205 may correct the motion vector associated with the merge candidate derived by the bead-chain motion vector prediction by a predetermined method.

[0241] As an example, the inter prediction unit 205 may correct the motion vector associated with the merge candidate derived by the bead-chain motion vector prediction by template matching.

[0242] Here, the inter prediction unit 205 may apply a technique similar to the template matching disclosed in Non-Patent Document 2 as such template matching.

[0243] As an example, the inter prediction unit 205 may correct the motion vector associated with the merge candidate derived by the bead-chain motion vector prediction by bilateral matching.

[0244] Here, the inter prediction unit 205 may apply a technique similar to the bilateral matching disclosed in Non-Patent Document 1 and Non-Patent Document 2 as such bilateral matching. For example, the inter prediction unit 205 may apply the decoder-side motion vector refinement (DMVR) disclosed in Non-Patent Document 1.

[0245] As an example, the inter prediction unit 205 may correct the motion vector associated with the merge candidate derived by the beaded motion vector prediction by adding the motion vector difference derived from the control information.

[0246] Here, as control information indicating the motion vector difference, the inter prediction unit 205 may apply a technique similar to the merge motion vector difference (MMVD: Merge with Motion Vector Difference) disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0247] By the inter prediction unit 205 correcting the motion vector associated with the merge candidate derived by the beaded motion vector prediction by a predetermined method, the inter prediction accuracy is improved, and an improvement in coding efficiency can be expected.

[0248] <6. Various restrictions on beaded motion vector prediction (applicability, number of pictures, number of merge candidates)> Using FIG. 23, various restriction methods of the beaded motion vector prediction by the decoding unit 201 and the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment will be described.

[0249] (Applicability of beaded motion vector prediction) First, regarding the applicability (applicable or not) of the beaded motion vector prediction, as described above, by its nature, when TMVP is applicable, the beaded motion vector prediction is applicable. In other words, when TMVP is not applicable, the beaded motion vector prediction is not applicable.

[0250] Therefore, regarding the applicability or non-applicability of the beaded motion vector prediction, it can be controlled according to the control information (syntax) that controls the applicability or non-applicability of the temporal motion vector prediction TMVP.

[0251] That is, the inter prediction unit 205 may control the applicability or non-applicability of the beaded motion vector prediction based on the control information that controls the applicability or non-applicability of the temporal motion vector prediction (TMVP) sent from the decoding unit 201.

[0252] Here, in Non-Patent Document 1, as syntax for controlling the applicability of TMVP, syntax at the sequence unit to be decoded (sps_temporal_mvp_enabled_flag, first control information, first syntax) and syntax at the picture unit to be decoded (ph_temporal_mvp_enabled_flag, second control information, second syntax) are defined.

[0253] That is, regarding the applicability of the bead-chain motion vector prediction, it can be controlled in the same unit as TMVP or a unit lower than TMVP.

[0254] For example, the method for controlling the applicability of the bead-chain motion vector prediction in the inter prediction 205 that controls the applicability of the bead-chain motion vector prediction in units of the block to be decoded is as follows.

[0255] The inter prediction unit 205 may control the applicability of the bead-chain motion vector prediction for the block to be decoded based on the control information that controls the applicability of the temporal motion vector prediction (TMVP) at the sequence unit to be decoded sent from the decoding unit 201.

[0256] More specifically, when the control information that controls the applicability of the temporal motion vector prediction (TMVP) at the sequence unit to be decoded sent from the decoding unit 201 specifies that the temporal motion vector prediction is applicable, the inter prediction unit 205 specifies that the bead-chain motion vector prediction for the block to be decoded is applicable.

[0257] On the other hand, when the above-described control information specifies that the temporal motion vector prediction is not applicable, the inter prediction unit 205 specifies that the bead-chain motion vector prediction for the block to be decoded is not applicable.

[0258] Alternatively, the inter prediction unit 205 may control whether or not to apply the bead-chain motion vector prediction to the block to be decoded based on the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP) for each picture to be decoded sent from the decoding unit 201.

[0259] More specifically, when the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP) for each picture to be decoded sent from the decoding unit 201 specifies that the temporal motion vector prediction can be applied, the inter prediction unit 205 specifies that the bead-chain motion vector prediction can be applied to the block to be decoded.

[0260] On the other hand, when the above control information specifies that the temporal motion vector prediction cannot be applied, the inter prediction unit 205 specifies that the bead-chain motion vector prediction cannot be applied to the block to be decoded.

[0261] The inter prediction unit 205 may control whether or not to apply the bead-chain motion vector prediction in units of the sequence to be decoded or units of the decoding target smaller than the sequence to be decoded based on the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP) for each sequence to be decoded sent from the decoding unit 201.

[0262] Alternatively, the inter prediction unit 205 may control whether or not to apply the bead-chain motion vector prediction in units of the picture to be decoded or units of the decoding target smaller than the picture to be decoded based on the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP) for each picture to be decoded sent from the decoding unit 201.

[0263] Alternatively, before the control of whether or not to apply the bead-chain motion vector prediction for each block to be decoded by the inter prediction unit 205, the decoding unit 201 may control whether or not to apply the bead-chain motion vector prediction based on the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP).

[0264] FIG. 23 defines a syntax (sps_chained_mvp_enabled_flag) that controls whether or not to apply the beaded movement vector prediction for each decoding target sequence unit, and shows an example of a process in which the decoding unit 201 determines whether or not to decode sps_chained_mvp_enabled_flag based on a syntax (sps_temporal_mvp_enabled_flag) that controls whether or not to apply the temporal movement vector prediction (TMVP) for each decoding target sequence unit.

[0265] As shown in FIG. 23, in step S100, when the decoding unit 201 determines that the syntax (sps_temporal_mvp_enabled_flag) that controls whether or not to apply the temporal movement vector prediction (TMVP) for each decoding target sequence unit is 1, in step S101, it decodes a syntax (sps_chained_mvp_enabled_flag, the third syntax) that controls whether or not to apply the beaded movement vector prediction for each decoding target sequence unit.

[0266] On the other hand, when the decoding unit 201 determines in step S100 that the syntax (sps_temporal_mvp_enabled_flag) that controls whether or not to apply the temporal movement vector prediction (TMVP) for each decoding target sequence unit is not 1, in step S102, it does not decode the syntax (sps_chained_mvp_enabled_flag) that controls whether or not to apply the beaded movement vector prediction for each decoding target sequence unit.

[0267] Here, when the value of sps_temporal_mvp_enabled_flag is 1, the decoding unit 201 determines that temporal motion vector prediction (TMVP) can be applied in the sequence unit to be decoded. When the value of sps_temporal_mvp_enabled_flag is 0, the decoding unit 201 determines that temporal motion vector prediction (TMVP) cannot be applied in the sequence unit to be decoded. When sps_temporal_mvp_enabled_flag has not been decoded, the value of sps_temporal_mvp_enabled_flag is presumed to be 0.

[0268] In addition, when the value of sps_chained_mvp_enabled_flag is 1, the decoding unit 201 determines that chained motion vector prediction can be applied in the sequence unit to be decoded. When the value of sps_chained_mvp_enabled_flag is 0, the decoding unit 201 determines that chained motion vector prediction cannot be applied in the sequence unit to be decoded. When sps_chained_mvp_enabled_flag has not been decoded, the value of sps_chained_mvp_enabled_flag is presumed to be 0.

[0269] In this way, by determining whether sps_chained_mvp_enabled_flag has been decoded based on the value of sps_temporal_mvp_enabled_flag, in a case where it is obvious that decoding of sps_chained_mvp_enabled_flag is unnecessary, the decoding of sps_chained_mvp_enabled_flag can be omitted, so that the amount of code can be reduced.

[0270] When the chained motion vector prediction is used in combination with the above-described rearrangement (ARMC: Adaptive Reordering Merge Candidate) by comparing the template matching costs of the merge candidates, a greater improvement in the coding efficiency can be expected.

[0271] Therefore, as a modification example of the control method for enabling or disabling the prediction of the beading motion vector based on the applicability of the above-mentioned TMVP, the inter prediction unit 205 may control the applicability of the beading motion vector prediction based on both the applicability of TMVP and the applicability of ARMC.

[0272] Specifically, when TMVP is applicable and ARMC is also applicable, the inter prediction unit 205 may specify that the beading motion vector prediction is applicable. In other cases, the inter prediction unit 205 may specify that the beading motion vector prediction is not applicable.

[0273] Regarding the applicability of ARMC, similar control information to that of TMVP may be provided, and the determination may be made based on the value of such control information.

[0274] (Number of reference pictures and search depth for beading motion vector prediction) Hereinafter, a method for restricting the number of reference pictures or the search depth that can be referred to during the recursive motion vector search of the beading motion vector prediction will be described.

[0275] First, a method for restricting the number of reference pictures that can be referred to during the recursive motion vector search of the beading motion vector prediction will be described.

[0276] As an example, the inter prediction unit 205 may restrict the recursive motion vector search of the beading motion vector prediction to at least one or more reference pictures included in the reference picture list of the block to be decoded.

[0277] By restricting the recursive motion vector search of the beading motion vector prediction by the inter prediction unit 205 to at least one or more reference pictures included in the reference picture list of the block to be decoded, the decoding processing amount can be reduced compared to the case where such a restriction is not performed (for example, when the inter prediction unit 205 searches all the reference pictures stored in the decoded picture buffer 207).

[0278] Alternatively, the inter prediction unit 205 may limit the search for the recursive motion vector of the beaded motion vector prediction to only the reference picture referred to by the motion vector of the block to be decoded.

[0279] In other words, the inter prediction unit 205 re-searches only the block vectors within the reference picture referred to by the motion vector of the block to be decoded, and generates a beaded motion vector by beaded motion vector prediction.

[0280] Second, a method for limiting the search depth of the recursive motion vector of the beaded motion vector prediction will be described.

[0281] As an example, the inter prediction unit 205 may limit the search depth of the recursive motion vector of the beaded motion vector prediction in the block to be decoded based on the control information.

[0282] Specifically, the inter prediction unit 205 may limit the search depth of the recursive motion vector of the beaded motion vector prediction in the block to be decoded based on the control information (sps_max_num_my_chains) that sets the upper limit of the search depth of the recursive motion vector of the beaded motion vector prediction in the decoding target sequence unit sent from the decoding unit.

[0283] Here, the control information of the decoding target sequence unit may be control information of a picture unit or a slice unit, which is control information lower than the decoding target sequence unit, or may be hierarchical control information combining these.

[0284] Alternatively, the inter prediction unit 205 may set a predetermined fixed value for the search depth of the recursive motion vector of the beaded motion vector prediction. The inter prediction unit 205 may set 1, 2, or 3 as such a predetermined fixed value.

[0285] By limiting the search depth of the chain stitch motion vector prediction in this way, it is possible to reduce the amount of decoding processing required for deriving the chain stitch motion vector as compared with the case where the amount of decoding processing is not limited.

[0286] (Number of merge candidates for chain stitch motion vector prediction) Hereinafter, a method for limiting the number of merge candidates derived by the chain stitch motion vector prediction will be described.

[0287] As an example, the inter prediction unit 205 may limit the number of merge candidates derived by the chain stitch motion vector prediction in the block to be decoded based on the control information.

[0288] Specifically, the inter prediction unit 205 may limit the number of merge candidates derived by the chain stitch motion vector prediction in the block to be decoded based on the control information (sps_max_num_cmvp_cand) that sets the upper limit of the search depth of the number of merge candidates derived by the chain stitch motion vector prediction in the decoding target sequence unit sent from the decoding unit.

[0289] Here, the control information of the decoding target sequence unit may be the control information of the picture unit or the slice unit, which is the control information lower than the decoding target sequence unit, or may be hierarchical control information combining these.

[0290] Alternatively, the inter prediction unit 205 may set a predetermined fixed value for the number of merge candidates derived by the chain stitch motion vector prediction.

[0291] (Restriction on block vector search in chain stitch motion vector prediction) It has been described above that in the recursive motion vector search of the chain stitch motion vector prediction, the search of the block vector may be included. However, block vectors referring within the same picture are easy to find in screen images and game images including characters and computer-generated graphics (CG), but are difficult to find in images taken by a camera.

[0292] Therefore, it is desirable to be able to limit the search for block vectors in the bead string motion vector prediction according to the characteristics of the image.

[0293] As an example, the inter prediction unit 205 may limit the block vector search based on control information that specifies whether or not to search for block vectors in the bead string motion vector prediction for the block to be decoded.

[0294] Specifically, the inter prediction unit 205 may limit whether or not to search for block vectors in the bead string motion vector prediction for the block to be decoded based on control information (sps_cmvp_bv_search_enabled_flag) that specifies whether or not to search for block vectors in the bead string motion vector prediction in the decoding target sequence unit sent from the decoding unit 201.

[0295] Here, the control information in the decoding target sequence unit may be control information in the picture unit or slice unit, which is control information lower than the decoding target sequence unit, or may be hierarchical control information combining these.

[0296] (Restriction on the reference source of the bead string motion vector prediction) In the recursive motion vector search of the bead string motion vector prediction, the method of adding the four corners of the block to be decoded by adding the starting point of the first motion vector to the center of the block to be decoded has been described above.

[0297] However, compared with screen images and game images where pixel values are likely to change rapidly in pixel units, images taken by a camera are less likely to change.

[0298] Therefore, it is desirable to be able to limit the starting point of the first motion vector in the recursive motion vector search of the bead string motion vector prediction according to the characteristics of the image.

[0299] As an example, the inter prediction unit 205 may limit the reference source of the bead string motion vector prediction based on control information for specifying whether to add at least one or more predetermined positions of the decoding target block by adding the starting point of the first motion vector in the bead string motion vector prediction for the decoding target block to the center of the decoding target block. Such predetermined positions may be the four corners within the decoding target block described above.

[0300] Specifically, the inter prediction unit 205 may limit whether to add at least one or more predetermined positions of the decoding target block by adding the starting point of the first motion vector in the bead string motion vector prediction for the decoding target block to the center of the decoding target block based on the sps_cmvp_multi_position_flag.

[0301] Here, the sps_cmvp_multi_position_flag is control information for specifying whether to add at least one or more predetermined positions of the decoding target block in addition to the center of the decoding target block to the starting point of the first motion vector in the bead string motion vector prediction for the decoding target block.

[0302] Here, the control information of the decoding target sequence unit may be control information of a picture unit or a slice unit which is control information lower than the decoding target sequence unit, or may be hierarchical control information combining these.

[0303] <7. Applicable Target of Bead String Motion Vector Prediction> Describe the technical features when applying the bead string motion vector prediction to merges other than the normal merge disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0304] (MMVD, GPM, TM Merge) First, MMVD, GPM, and template matching merge (TM merge) disclosed in Non-Patent Document 1 and Non-Patent Document 2 construct a merge list in the same way as normal merge. However, in Non-Patent Document 2, pruning of merge candidates during the construction of these merge candidate lists is strengthened compared to normal merge, and these merge candidate lists are unlikely to be the same as the merge candidates stored in the merge candidate list of normal merge.

[0305] Therefore, when not using the merge candidate list of normal merge in MMVD, GPM, and TM merge, in addition to applying the bead-chain motion vector prediction to the merge candidates in the merge candidate list of normal merge, if the bead-chain motion vector prediction is applied to the merge candidates in the respective merge candidate lists of MMVD, GPM, and TM merge, an improvement in coding efficiency can be expected.

[0306] Therefore, in addition to applying the bead-chain motion vector prediction to the merge candidates in the merge candidate list of normal merge, the inter prediction unit 205 may apply the bead-chain motion vector prediction to the merge candidates in the merge candidate list of the merge mode motion vector difference.

[0307] In addition, in addition to applying the bead-chain motion vector prediction to the merge candidates in the merge candidate list of normal merge, the inter prediction unit 205 may apply the bead-chain motion vector prediction to the merge candidates in the merge candidate list of the geometric partitioning mode.

[0308] In addition, in addition to applying the bead-chain motion vector prediction to the merge candidates in the merge candidate list of normal merge, the inter prediction unit 205 may apply the bead-chain motion vector prediction to the merge candidates in the merge candidate list of the template matching merge.

[0309] Here, the inter prediction unit 205 may apply pruning processing at the time of storing the merge candidate list, correction of the motion vector, and rearrangement of the merge candidates (motion vectors) in the same manner as normal merge for the bead-chain motion vectors generated for the merge mode motion vector difference, geometric partition mode, and template matching merge.

[0310] (Adaptive Motion Vector Prediction - Merge Mode) In the adaptive motion vector prediction - merge mode (Adaptive Motion Vector Prediction - Merge mode) disclosed in Non - Patent Document 1 and Non - Patent Document 2 described above, since the motion vector of either list 0 (L0) or list 1 (L1) of the reference picture list is derived by adaptive motion vector prediction (AMVP), that is, by control information, if the bead - chain motion vector prediction is applied to the merge candidates in the merge candidate list of the reference picture list number not derived by AMVP, the options for selecting merge candidates that can be selected in the adaptive motion vector prediction - merge mode increase, so the inter prediction accuracy is improved and an improvement in coding efficiency can be expected.

[0311] Therefore, in the adaptive motion vector prediction - merge mode, the inter prediction unit 205 may apply the bead - chain motion vector prediction to the merge candidates in the merge candidate list of the reference picture list number not derived by the adaptive motion vector prediction.

[0312] Furthermore, when the inter prediction unit 205 applies the bead - chain motion vector prediction to the merge candidates in the merge candidate list of the reference picture list number not derived by the adaptive motion vector prediction in the adaptive motion vector prediction - merge mode, it may limit the generated bead - chain motion vector so as not to refer to the reference picture of the motion vector derived by the adaptive motion vector prediction.

[0313] Here, for the chain motion vectors generated for the adaptive motion vector prediction - merge mode, the inter prediction unit 205 may apply pruning processing during storage in the merge candidate list, motion vector correction, and rearrangement of the merge candidates (motion vectors) in the same manner as normal merge.

[0314] (Sub - block merge mode) As disclosed in Non - Patent Document 1 and Non - Patent Document 2 mentioned above, for the Affine merge, in order to derive the motion vectors for each sub - block within the block to be decoded by affine transformation, as shown in FIG. 24, in the case of single prediction (either L0 or L1), two or three control point motion vectors cpMv are derived, and in the case of dual prediction (both L0 and L1), four or six control point motion vectors cpMv are derived.

[0315] Note that all the control point motion vectors of L0 and L1 for single prediction or dual prediction refer to the same reference picture.

[0316] In Affine merge, as disclosed in Non - Patent Document 1 and Non - Patent Document 2, these control point motion vectors are derived from the merge candidates stored in the Affine - merge - specific merge candidate list.

[0317] The inter prediction unit 205 may derive the merge candidates to be stored in the Affine - merge - specific merge candidate list in the same manner as the method for deriving Affine - merge - specific merge candidates disclosed in Non - Patent Document 1 and Non - Patent Document 2.

[0318] For example, when two or three control point motion vectors are accumulated at positions spatially adjacent or close to the block to be decoded in the inter prediction unit 205, they may be added to the Affine - merge - specific merge candidate list of the block to be decoded.

[0319] Alternatively, when a plurality of motion vectors are stored at positions spatially adjacent or close to the block to be decoded, the inter prediction unit 205 may generate a control point motion vector for the block to be decoded based on them and add it to the merge candidate list for affine merge.

[0320] Therefore, in the affine merge mode, the inter prediction unit 205 may add the motion vectors derived by recursively searching for motion vectors with respect to at least two or more control point motion vectors stored in the merge candidate list to each of the first control point motion vectors to generate a chained control point motion vector and store it in the merge candidate list.

[0321] Furthermore, when recursively searching for motion vectors with respect to at least two or more control point motion vectors stored in the merge candidate list in the affine merge mode, the inter prediction unit 205 may limit the reference pictures to be searched for each control point motion vector to the same reference picture.

[0322] Alternatively, when recursively searching for motion vectors with respect to at least two or more control point motion vectors of L0 and L1 stored in the merge candidate list in the affine merge mode, the inter prediction unit 205 may limit the reference pictures to be searched for each control point motion vector of L0 and L1 to the same reference picture.

[0323] Here, for the chained control point motion vector generated for affine merge, the inter prediction unit 205 may apply pruning processing at the time of storing in the merge candidate list, correction of the motion vector, and rearrangement of the merge candidates (motion vectors) in the same manner as normal merge.

[0324] According to the image decoding apparatus according to the present embodiment, since the inter prediction unit 205 adds all the motion vectors derived by recursively searching for the motion vectors at positions spatially or temporally adjacent or close to the block to be decoded in the decoded picture buffer 207 and derives the motion vector generated as the motion vector of the block to be decoded, the options for selecting motion vectors that can be selected in inter prediction increase, the inter prediction accuracy improves, and the coding efficiency is improved.

[0325] The above-described image decoding apparatus 200 may be realized by a program that causes a computer to execute each function (each process).

Industrial Applicability

[0326] Note that according to the present embodiment, for example, since it is possible to realize an improvement in overall service quality in moving image communication, it is possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Build resilient infrastructure, promote sustainable industrialization, and foster innovation."

Explanation of Signs

[0327] 200…Image decoding apparatus 201…Decoder 202…Inverse quantization unit 203…Inverse transform unit 204…Intra prediction unit 205…Inter prediction unit 206…Adder 207…Decoded picture buffer 210…Code input unit 220…Image output unit

Claims

1. An image decoding device, an inter prediction unit that generates inter predicted pixels from control information and decoded pixels stored in a decoded picture buffer; the inter prediction unit performs a daisy chain motion vector prediction in which the inter prediction unit generates a daisy chain motion vector by recursively searching for motion vectors stored in the decoding picture buffer for motion vectors at positions spatially or temporally adjacent or close to the block to be decoded, which is stored in a merge candidate list storing motion vector candidates, and adding all of the derived motion vectors, and derives the daisy chain motion vectors as motion vector candidates for the block to be decoded; The image decoding device, wherein the inter prediction unit determines whether to add a merge candidate corresponding to the daisy chain motion vector derived by the daisy chain motion vector prediction based on a predetermined condition.

2. The inter prediction unit determining not to add a merge candidate corresponding to the chained motion vector derived by the chained motion vector prediction when the predetermined condition is satisfied; The image decoding device according to claim 1 , wherein, when the predetermined condition is not satisfied, it is determined to add a merge candidate corresponding to the daisy chain motion vector derived by the daisy chain motion vector prediction.

3. 2. The image decoding device according to claim 1, wherein the predetermined condition is that the reference pictures and motion vectors associated with at least one merge candidate stored in a merge candidate list and the merge candidate corresponding to the daisy-chain motion vector are identical.

4. The image decoding device according to claim 1, characterized in that the specified condition is that at least one merging candidate stored in a merging candidate list and one or more reference pictures and motion vectors associated with each merging candidate corresponding to the daisy-chain motion vector all match.

5. The image decoding device described in claim 1, characterized in that the specified condition is a condition that the reference pictures and motion vectors of merge candidate list number 0 and merge candidate list number 1 associated with at least one merge candidate stored in a merge candidate list and the merge candidate corresponding to the daisy-chain motion vector all match.

6. 2. The image decoding device according to claim 1, wherein the predetermined condition is that the reference pictures associated with at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the daisy-chain motion vector match, and an error in the reference positions of the motion vectors associated with at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the daisy-chain motion vector is less than a predetermined threshold.

7. 7. The image decoding device according to claim 6, wherein the inter prediction unit sets the predetermined threshold when there is one motion vector to be compared to be greater than the predetermined threshold when there are two motion vectors to be compared.

8. 2. The image decoding device according to claim 1, wherein the predetermined condition is that the reference picture and motion vector associated with at least one merge candidate stored in a merge candidate list and the merge candidate corresponding to the daisy-chain motion vector match an internal parameter that changes the method of generating a predetermined inter-prediction pixel.

9. 2. The image decoding device according to claim 1, wherein the predetermined condition is that a reference picture and a motion vector associated with at least one merge candidate stored in a merge candidate list and a merge candidate corresponding to the daisy-chain motion vector match an internal parameter specifying whether or not to apply a switching interpolation filter.

10. An image decoding method, comprising: generating inter-predicted pixels from the control information and the stored decoded pixels; generating the inter-predicted pixel includes performing a chain motion vector prediction in which a chain motion vector is generated by recursively searching for motion vectors stored in a decoding picture buffer for motion vectors at positions spatially or temporally adjacent or close to the block to be decoded, which is stored in a merge candidate list storing motion vector candidates, and adding all of the derived motion vectors, and deriving the chain motion vectors as candidates for the motion vector of the block to be decoded; determining whether to add a merge candidate corresponding to the daisy chain motion vector derived by the daisy chain motion vector prediction based on a predetermined condition.

11. A program that causes a computer to function as an image decoding device, The image decoding device comprises: an inter prediction unit that generates inter predicted pixels from control information and accumulated decoded pixels; the inter prediction unit performs a daisy chain motion vector prediction in which a daisy chain motion vector is generated by recursively searching for motion vectors stored in a decoding picture buffer for motion vectors at positions spatially or temporally adjacent or close to the block to be decoded, which is stored in a merge candidate list storing motion vector candidates, and adding all of the derived motion vectors, and deriving the daisy chain motion vectors as candidates for the motion vector of the block to be decoded; The program is characterized in that the inter prediction unit determines whether to add a merge candidate corresponding to the daisy chain motion vector derived by the daisy chain motion vector prediction based on a predetermined condition.

Images