Image decoding device, image decoding method, and program

Abstract

To provide an image decoding device, an image decoding method, and a program that improve encoding efficiency.SOLUTION: In an image decoding device 200, an inter-prediction unit 205 performs chained motion vector prediction of recursively adding all motion vectors derived by searching for motion vectors accumulated in a decoding picture buffer, in motion vectors of positions that are spatially or temporally adjacent or proximate to each other, to a decoding target block stored in a merge candidate list that stores candidates for a motion vector, thereby generating a chained motion vector, and deriving the chained motion vector as a candidate for a motion vector of the decoding target block, and in deriving a merge candidate through the chained motion vector prediction, controls the presence or absence of inheritance of a predetermined internal parameter for changing a method for generating an inter-prediction pixel.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image decoding apparatus, an image decoding method, and a program.

Background Art

[0002] Non-Patent Document 1 and Non-Patent Document 2 disclose inter prediction.

[0003] In inter prediction, predicted pixels of a block to be decoded are generated from decoded pixels (reference pixels) in a decoded picture (reference picture) different from the picture to be decoded.

[0004] Also, in inter prediction, using control information, a motion vector of a block to be decoded necessary for generating predicted pixels of the block to be decoded is selected from a plurality of motion vector candidates at positions spatially or temporally adjacent or close to the block to be decoded.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] Since candidates for motion vectors selectable in inter prediction in Non-Patent Document 1 and Non-Patent Document 2 are limited to motion vectors at positions spatially or temporally adjacent or close to the block to be decoded, there is a problem that there is room for improvement in coding efficiency.

[0007] Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide an image decoding apparatus, an image decoding method, and a program with high coding efficiency.

Means for Solving the Problems

[0008] A first feature of the present invention is an image decoding method, which includes: step A of performing variable-length decoding on code information to output a quantization value and control information; step B of inverse quantizing the quantization value to output a transform coefficient; step C of inverse-transforming the transform coefficient to output a prediction residual pixel; step D of generating an intra prediction pixel from the control information and decoded pixels; step E of accumulating the decoded pixels in a decoded picture buffer; step F of generating an inter prediction pixel from the control information and the accumulated decoded pixels; and step G of adding at least one of the intra prediction pixel and the inter prediction pixel to the prediction residual pixel to generate the decoded pixel. In step F, all motion vectors derived by recursively searching for motion vectors at positions spatially or temporally adjacent or close to the motion vector of a decoding target block stored in a merge candidate list storing motion vector candidates are added to generate a chain-like motion vector, and a chain-like motion vector prediction is performed to derive the chain-like motion vector as a candidate for the motion vector of the decoding target block. The gist is to control the presence or absence of inheritance of a predetermined internal parameter for changing the generation method of the inter prediction pixel when deriving a merge candidate by the chain-like motion vector prediction in step F.

[0009] A third feature of the present invention is a program that causes a computer to function as an image decoding device, the image decoding device including a decoding unit that variably decodes code information and outputs a quantization value and control information, an inverse quantization unit that inverse quantizes the quantization value and outputs a transform coefficient, an inverse transform unit that inverse transforms the transform coefficient and outputs a prediction residual pixel, an intra prediction unit that generates an intra prediction pixel from the control information and decoded pixels, a decoded picture buffer that stores the decoded pixels, an inter prediction unit that generates an inter prediction pixel from the control information and the stored decoded pixels, and an adder that adds at least one of the intra prediction pixel and the inter prediction pixel to the prediction residual pixel to generate the decoded pixel. The inter prediction unit recursively searches for motion vectors stored in the decoded picture buffer with respect to motion vectors at positions spatially or temporally adjacent or close to a decoding target block stored in a merge candidate list that stores motion vector candidates, and adds all the motion vectors derived thereby to generate a chained motion vector, and performs chained motion vector prediction to derive the chained motion vector as a candidate for the motion vector of the decoding target block. The gist is that the inter prediction unit controls whether to inherit a predetermined internal parameter that changes the generation method of the inter prediction pixel when deriving a merge candidate by the chained motion vector prediction.

Advantages of the Invention

[0010] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program with high coding efficiency.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components or the like, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0013] <First Embodiment> Hereinafter, an image decoding apparatus 200 according to this embodiment will be described with reference to FIGS. 1 to 24.

[0014] The image decoding apparatus 200 according to the first embodiment of the present invention targets various image signals (hereinafter, images). For example, the image decoding apparatus 200 according to this embodiment targets a YUV (YCbCr) image composed of luminance pixels and color difference pixels, an RGB image composed of RGB pixels, or a monochrome image. Here, the pixels constituting each image have discrete values (pixel values) with a predetermined bit width.

[0015] FIG. 1 is a diagram showing an example of a functional block of the image decoding apparatus 200 according to this embodiment.

[0016] As shown in FIG. 1, the image decoding apparatus 200 includes a code input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transformation unit 203, an intra prediction unit 204, an inter prediction unit 205, an adder 206, a decoded picture buffer 207, and an image output unit 220.

[0017] Hereinafter, in the description of the functions of each unit, the places described as "pixel" may be blocks (units) composed of pixels, tree blocks which are the maximum size of the blocks, slices, tiles, or images (pictures) larger than the tree blocks.

[0018] Although not shown in FIG. 1, the image decoding apparatus 200 may include a loop filter for correcting the pixel values of the decoded pixels between the adder 206 and the decoded picture buffer 207.

[0019] The code input unit 210 is configured to acquire the coded information coded by an image coding apparatus.

[0020] The decoding unit 201 is configured to decode control information and quantization values from the coded information input from the code input unit 210. For example, the decoding unit 201 is configured to output control information and quantization values by performing variable length decoding on such coded information.

[0021] Here, the quantization values are sent to the inverse quantization unit 202, and the control information is sent to the decoded picture buffer 207, the intra prediction unit 204, and the inter prediction unit 205. Such control information (syntax) includes information necessary for controlling the decoded picture buffer 207, the intra prediction unit 204, the inter prediction unit 205, etc., and may include header information such as a sequence parameter set, a picture parameter set, a picture header, and a slice header.

[0022] The inverse quantization unit 202 is configured to inverse quantize the quantization values sent from the decoding unit 201 to obtain decoded transform coefficients. Such transform coefficients are sent to the inverse transformation unit 203.

[0023] The inverse conversion unit 203 is configured to perform inverse conversion on the conversion coefficients sent from the inverse quantization unit 202 to obtain a decoded prediction residual. Such a prediction residual is sent to the adder 206.

[0024] The intra prediction unit 204 is configured to generate an intra prediction pixel based on the decoded pixel and the control information sent from the decoding unit 201. Here, the decoded pixel is obtained via the adder 206 and accumulated in the decoded picture buffer 207. Also, the intra prediction pixel is a prediction pixel for addition with the prediction residual by the adder 206. Note that the intra prediction pixel is sent to the adder 206.

[0025] Here, the prediction performed by the intra prediction unit 204 may be the intra prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2. Alternatively, the prediction performed by the intra prediction unit 204 may be the intra block copy or intra template matching prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0026] The decoded picture buffer 207 accumulates cumulatively the decoded pixels sent from the adder 206 and the inter prediction information corresponding to the decoded pixels sent from the inter prediction unit 205.

[0027] Here, the decoded pixels accumulated in the decoded picture buffer 207 and the inter prediction information corresponding to such decoded pixels are referred to by the inter prediction unit 205. Details of the inter prediction information will be described later.

[0028] The inter prediction unit 205 is configured to generate an inter prediction pixel based on the decoded pixels and inter prediction information obtained by referring to the decoded picture buffer 207 and the control information sent from the decoding unit 201.

[0029] Furthermore, the inter prediction unit 205 sends the generated inter prediction pixels to the adder 206, and sends the inter prediction information used for the generation of such inter prediction pixels to the decoded picture buffer 207 as corresponding to the pixels to be decoded.

[0030] Here, the prediction performed by the inter prediction unit 205 may be the inter prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2. Alternatively, the prediction performed by the inter prediction unit 205 may be the intra block copy disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0031] The adder 206 is configured to calculate the decoded pixels by adding at least one of the prediction residual sent from the inverse conversion unit 203 and the input intra prediction pixels and inter prediction pixels. Such decoded pixels are sent to the image output unit 220, the decoded picture buffer 207, and the intra prediction unit 204.

[0032] The image output unit 220 is configured to output the decoded pixels transmitted from the adder 206.

[0033] Hereinafter, the roles of the inter prediction unit 205 and the decoded picture buffer 207, which are characteristic configurations of the image decoding apparatus 200 according to the present embodiment, will be described by taking as an example the case where the prediction performed by the inter prediction unit 205 is inter prediction.

[0034] <Basic Roles of Inter Prediction Unit 205 and Decoded Picture Buffer 207> The basic role of the inter prediction unit 205 is to derive one or more motion vectors (Mv: Motion vector) for the block to be decoded in order to accurately predict the block to be decoded in the subsequent adder 206, and to generate the decoded block (hereinafter, referred to as the reference block) referred to by the Mv in a decoded picture (hereinafter, referred to as the reference picture) different from the decoded picture to which the block to be decoded belongs. Predicting the pixels (hereinafter, referred to as reference pixels), that is, performing inter prediction.

[0035] FIG. 2 shows a conceptual diagram of inter prediction. Specifically, in FIG. 2, in inter prediction, in order to generate inter prediction pixels of a decoding target block (CurBlk) within a decoding target picture (CurPic), two motion vectors MvL0 k and MvL1 k derived by an inter prediction unit 205 are used to reference two different reference blocks (RerBlkL0 k and RerBlkL1 k ) within two different reference pictures (RerPicL0 k and RerPicL1 k ). An example is shown below.

[0036] Here, L0 and L1 indicate the numbers of lists (reference picture lists) in which reference pictures that can be referenced from the decoding target picture are described, and k corresponds to the value of control information (index) for specifying a motion vector described in a motion vector derivation method described later.

[0037] Similar to Non-Patent Document 1 and Non-Patent Document 2, the image decoding apparatus 200 according to the present embodiment may derive list 0 (L0) and list 1 (L1) as two different reference picture lists for each decoding target picture.

[0038] Here, the decoding picture buffer 207 may derive a reference picture list for each decoding target picture based on control information decoded by the decoding unit 201.

[0039] For example, the control information may include information such as the number of reference pictures to be included in the reference picture list and candidates for reference pictures (picture reference structure) for the decoding target picture.

[0040] Details of the motion vector derivation method in the inter prediction unit 205 will be described later. The outline of the inter prediction pixel generation method in the inter prediction unit 205 is as follows.

[0041] The inter prediction unit 205 may generate inter prediction pixels from a single reference block, or as shown in FIG. 2, may generate inter prediction pixels using two reference blocks, or may generate inter prediction pixels using two or more reference blocks.

[0042] Here, as a method for the inter prediction unit 205 to generate inter prediction pixels using two reference blocks, the following method may be applied.

[0043] As an example, the inter prediction unit 205 may generate inter prediction pixels by simply averaging the pixel values of two reference blocks.

[0044] As another example, the inter prediction unit 205 may generate inter prediction pixels by weighted averaging the pixel values of two reference blocks in block units using a predetermined weight value.

[0045] As a technique of such weighted averaging in block units, the inter prediction unit 205 may apply block-level weighted bi-prediction (BCW) disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0046] Note that in Non-Patent Documents 1 and 2, in BCW, a plurality of weight values can be selected including simple averaging, that is, a weight value of 1:1, and such weight values are specified by the values of internal parameters (bcwIdx, the third internal parameter).

[0047] Here, the inter prediction unit 205 derives bcwIdx by inheritance from control information or the decoded picture buffer 207. Details of the inheritance will be described later.

[0048] As another example, the inter prediction unit 205 may generate inter prediction pixels by dividing the block to be decoded into two parts by a predetermined straight line, and weighted averaging the pixel values of the reference blocks corresponding to the divided regions using a weight value corresponding to the distance from the dividing line.

[0049] The inter prediction unit 205 may apply a Geometric Partitioning Mode (GPM) disclosed in Non-Patent Document 1 and Non-Patent Document 2 as a technique of weighted average for each region obtained by linearly bisecting such a block.

[0050] Here, when the motion vector refers to a fractional pixel accuracy position within the reference block, the inter prediction unit 205 may generate an inter prediction pixel using an interpolation filter for deriving a reference pixel value at such a fractional pixel accuracy position from reference pixel values at integer pixel accuracy positions in the vertical, horizontal, upper, and lower directions of such a fractional pixel position.

[0051] Furthermore, when generating an inter prediction pixel, the inter prediction unit 205 may adaptively apply two interpolation filters having different cut-off frequencies for inter prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0052] Among such two interpolation filters, an interpolation filter with a low cut-off frequency is called a Switchable Interpolation Filter (SIF).

[0053] Similar to Non-Patent Document 1 and Non-Patent Document 2, when the motion vector refers to a half pixel accuracy position and an internal parameter (hpelIfIdx, the first internal parameter) for specifying whether or not to apply the SIF indicates that the SIF is to be applied, the inter prediction unit 205 may apply the SIF.

[0054] Note that the inter prediction unit 205 derives hpelIfIdx by inheritance from control information or the decoded picture buffer 207. Details of the inheritance will be described later.

[0055] On the other hand, when the motion vector refers to a fractional pixel accuracy position, the inter prediction unit 205 may round (clip) the reference destination of the motion vector to the nearest integer pixel accuracy position of such fractional pixel accuracy position, and generate an inter prediction pixel without applying an interpolation filter.

[0056] Also, when generating the final inter prediction pixel, the inter prediction unit 205 may apply local illumination compensation (LIC) disclosed in Non-Patent Document 2.

[0057] LIC is a technique for correcting the reference pixel value in the reference block by linear prediction based on the decoded pixels adjacent to the block to be decoded and the reference block, respectively.

[0058] Similar to Non-Patent Document 1 and Non-Patent Document 2, the inter prediction unit 205 may apply LIC when an internal parameter (licFlag, the second internal parameter) for controlling the application of LIC indicates that LIC is to be applied.

[0059] Note that the inter prediction unit 205 derives the licFlag by control information or inheritance from the decoded picture buffer 207. Details of the inheritance will be described later.

[0060] Also, when generating the final inter prediction pixel, the inter prediction unit 205 may apply multi-hypothesis prediction (MHP) disclosed in Non-Patent Document 2.

[0061] MHP is a technique for generating the final inter prediction pixel by weighted average from the reference pixel values of at least three or more reference blocks.

[0062] Similar to Non-Patent Document 2, the inter prediction unit 205 may apply MHP when an internal parameter (mhpFlag, the fourth internal parameter) for specifying the application of MHP indicates that MHP is to be applied.

[0063] Note that the inter prediction unit 205 derives the mhpFlag based on control information or inheritance from the decoded picture buffer 207. Details of the inheritance will be described later.

[0064] The inter prediction unit 205 sends the inter predicted pixels for the block to be decoded to the adder 206, and sends to the decoded picture buffer 207 the motion vector used for generating such inter predicted pixels, an index (reference index) for specifying the reference picture referred to by the motion vector within the reference picture list, and a counter (POC: Picture Order Count) representing the picture output order of the reference picture corresponding to the reference index. Hereinafter, the motion vector, reference index, and POC corresponding to the reference index used for generating these inter predicted pixels are referred to as inter prediction information. Further, the inter prediction unit 205 may send such inter prediction information including at least any one or all of the internal parameters bcwIdx, hpelIfIdx, licFlag, or mhpFlag for changing the generation method of the above-described inter predicted pixels.

[0065] The decoded picture buffer 207 accumulates the inter prediction information for the block to be decoded corresponding to the decoded pixels of the block to be decoded sent from the adder 206.

[0066] The decoded picture buffer 207 may accumulate such inter prediction information in a predetermined size (number of pixels). For example, the decoded picture buffer 207 may accumulate such inter prediction information in units of the minimum size (4×4 pixels) of the block to be decoded as disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0067] By the decoded picture buffer 207 accumulating such inter prediction information in the minimum size of the block to be decoded, the inter prediction accuracy when such inter prediction information is referred to from the block to be decoded to be decoded in the future can be improved.

[0068] The decoded picture buffer 207 accumulates decoded pixels in units of pictures to be decoded. Alternatively, the decoded picture buffer 207 may accumulate decoded pixels in units of slices to be decoded. Hereinafter, a case where decoded pixels are accumulated in units of pictures to be decoded will be described as an example.

[0069] When it is specified that the accumulated decoded pixels in the unit of the picture to be decoded in the decoded picture buffer 207 and the inter-prediction information corresponding to the decoded pixels are not referred to as a reference picture and a reference block from a future picture to be decoded or a block to be decoded, the decoded pixels in the unit of the picture to be decoded and the inter-prediction information corresponding to the decoded pixels may be sequentially deleted. According to such a configuration, the amount of information accumulated in the decoded picture buffer 207 can be reduced.

[0070] The decoded picture buffer 207 may derive the number of pictures to be decoded to be accumulated and the reference picture structure between the pictures to be decoded based on the control information decoded by the decoding unit 201.

[0071] As a modification, when there are inter-prediction pixels in a predetermined area outside the picture to be decoded in addition to the picture to be decoded, the decoded picture buffer 207 may additionally accumulate such inter-prediction pixels.

[0072] FIG. 3 is a diagram for explaining reference pixel values referred to when an area outside the picture to be decoded accumulated in the decoded picture buffer 207 disclosed in each of Non-Patent Document 1 and Non-Patent Document 2 is referred to from a future block to be decoded.

[0073] As shown in FIG. 3, in Non-Patent Document 1 (VVC: Versatile Video Coding), when referred to from a future block to be decoded within a predetermined range outside the picture to be decoded, a pixel value at the nearest decoded picture boundary is replicated to generate a reference pixel value.

[0074] In VVC, as such a predetermined range (padding area), a range obtained by adding 16 pixels to the maximum size (number of pixels) of the width of a block to be decoded is specified.

[0075] In Non-Patent Document 2 (ECM: Enhanced Compression Model), as shown in FIG. 3, between outside the picture to be decoded and the padding area, only a predetermined range is accumulated as an area (inter prediction padding area) where the inter prediction pixel values of the block to be decoded can be referred to from future blocks to be decoded.

[0076] As a result, even if outside the picture to be decoded is referred to from a future block to be decoded, since there are inter prediction pixel values of the block to be decoded, the inter prediction accuracy of the future block to be decoded is improved.

[0077] FIG. 4 is a conceptual diagram showing the generation of the inter prediction padding area described in FIG. 3.

[0078] As shown in FIG. 4, when an adjacent block of a reference block corresponding to a block to be decoded has inter prediction pixels, even if the block to be decoded is in contact with the picture to be decoded, since inter prediction pixels can be generated outside the picture to be decoded, the decoded picture buffer 207 can accumulate the inter prediction padding area shown in FIG. 3.

[0079] <Method for Deriving Motion Vector in Inter Prediction Unit> Hereinafter, with reference to FIGS. 5 to 17, a method for deriving a motion vector in the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment will be described.

[0080] FIG. 5 is a diagram showing an example of a merge candidate list which is a method for deriving a motion vector in the inter prediction unit 205.

[0081] The merge candidate list is a list storing candidates of inter prediction information (i.e., merge candidates) constructed by the inter prediction unit 205 to derive motion vectors in the merge (merge mode) disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0082] The inter prediction unit 205 extracts a merge candidate from among a plurality of merge candidates in the merge candidate list based on an index (merge_idx) for specifying the merge candidate in the merge candidate list sent from the decoding unit 201, and uses it as inter prediction information for generating an inter prediction pixel.

[0083] The upper limit of the number of merge candidates stored in the merge candidate list may be a fixed value or a variable value. In the case of a variable value, for example, as in Non-Patent Document 1 and Non-Patent Document 2, control information is provided in a sequence parameter set or the like, and such variable value can be set based on such control information.

[0084] The inter prediction unit 205 specifies, from within the merge candidate list, the inter prediction information to be used for generating an inter prediction pixel based on one or more merge_idx sent from the decoding unit 201.

[0085] For example, the inter prediction unit 205 may decode one merge_idx for each region obtained by linearly dividing the block to be decoded with respect to the above-described GPM, and derive inter prediction information.

[0086] As shown in FIG. 5, the inter prediction unit 205 stores inter prediction information (merge candidates) corresponding to L0 and L1 of the merge candidate list.

[0087] Note that, in the example of FIG. 5, the inter prediction unit 205 describes only the motion vector and the reference index as the inter prediction information. However, in addition to these, at least one or more or all of the internal parameters bcwIdx, hpelIfIdx, licFlag, or mhpFlag, which are the internal parameters for changing the above-described method for generating the inter prediction pixels, may be stored.

[0088] The inter prediction unit 205 may search for and add, as the inter prediction information, that is, as a merge candidate, the inter prediction information at a position spatially or temporally adjacent or close to the block to be decoded in the merge candidate list.

[0089] As an example, the inter prediction unit 205 may apply a combination of searches for a plurality of types of merge candidates disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0090] FIG. 6 shows the names of a plurality of types of merge candidates in the regular merge disclosed in Non-Patent Document 1 and Non-Patent Document 2 and the derivation order of these merge candidates.

[0091] Specifically, in Non-Patent Document 1 (VVC), the inter prediction unit 205 checks the availability of these merge candidates in the order of spatial merge candidates, temporal motion vector prediction (TMVP) candidates, history-based motion vector prediction (HMVP) candidates, and pairwise merge candidates.

[0092] In Non-Patent Document 2 (ECM), in addition to the above-described merge candidates in Non-Patent Document 1, the inter prediction unit 205 checks for the use of non-adjacent (NA) spatial merge candidates between the spatial merge candidates and the TMVP candidates.

[0093] The outline of each merge candidate is as follows.

[0094] FIG. 7 is a diagram showing the search positions of the spatial merge candidates and the TMVP candidates in the block to be decoded.

[0095] As shown in FIG. 7, in the spatial merge candidates, the inter prediction information is searched at the positions of the lower left (A0), left (A1), upper right (B0), upper (B1), and upper left (B2) that are adjacent to the block to be decoded (CurBlk), respectively.

[0096] That is, in the spatial merge candidates, the inter prediction information at the positions spatially adjacent to the block to be decoded (CurBlk) is used.

[0097] Also, in the TMVP candidates, in a picture different from the block to be decoded (reference picture), the inter prediction information at the positions of the lower right center (Col) of the block to be decoded (CurBlk) shown in FIG. 7 and the lower right (H) outside the block to be decoded (CurBlk) is searched.

[0098] That is, in the TMVP candidates, the inter prediction information at the positions temporally adjacent to the block to be decoded is used.

[0099] In the HMVP candidates and the non-adjacent spatial merge candidates, the inter prediction information at the positions spatially distant (proximate) from the block to be decoded is used.

[0100] In particular, in the case of the HMVP candidate, a table capable of storing FIFO-type inter-prediction information is constructed. Each time new inter-prediction information is searched for, the oldest inter-prediction information is discarded from the table, and such new inter-prediction information is added to the table.

[0101] Similar to the HMVP candidate and the non-adjacent spatial merge candidate disclosed in Non-Patent Document 1 and Non-Patent Document 2, the inter-prediction unit 205 of the image decoding apparatus 200 according to the present embodiment can design the position for searching inter-prediction information and the size of the table for the HMVP candidate.

[0102] In addition, in the search for merge candidates in the above-described normal merge, the inter-prediction unit 205 may re-search for a new reference block position from the peripheral pixels of the reference block of the inter-prediction information to be searched, based on the decoded pixels adjacent to the reference block based on the inter-prediction information to be searched (template of the reference block) and the decoded pixels adjacent to the block to be decoded (template of the block to be decoded), so that the difference therebetween (hereinafter referred to as the template matching cost) is reduced, and correct the inter-prediction information (motion vector).

[0103] As an example of this, the inter-prediction unit 205 may apply the template matching merge disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0104] In addition, in the selection of merge candidates from the merge candidate list in the above-described normal merge, the inter-prediction unit 205 derives the L0 inter-prediction information from the control information, and selects, as the L1 merge candidate, the merge candidate for which the difference between the template of the reference block based on the L0 inter-prediction information derived from the control information and the template of the reference block based on the inter-prediction information stored in L1 of the merge candidate list is minimized.

[0105] The inter prediction unit 205 can derive the merge candidates of L1 (i.e., the inter prediction information) based on the inter prediction information of L0 derived from such control information. Therefore, in addition to the control information necessary for deriving the inter prediction information of L0, it does not require decoding of merge_idx to derive the inter prediction information of L1.

[0106] As an example, the inter prediction unit 205 may apply the Adaptive motion vector Prediction-Merge mode disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0107] As a modification example, the inter prediction unit 205 may reverse L0 and L1 in the above example. That is, the inter prediction unit 205 may derive the inter prediction information of L1 from control information and derive the inter prediction information of L0 from the merge candidate list.

[0108] In addition, the inter prediction unit 205 may construct a merge candidate list for merge for deriving inter prediction information in units of sub-blocks obtained by dividing the block to be decoded, and derive the inter prediction information.

[0109] As an example, the inter prediction unit 205 may construct a merge candidate list for Sub-block TMVP (SbTMVP) or Affine merge disclosed in Non-Patent Document 1 and Non-Patent Document 2, and derive the inter prediction information.

[0110] <1. Basic concept of beaded motion vector prediction (derivation of beaded motion vector)> The methods for deriving the inter prediction information (hereinafter, referred to as motion vectors for simplicity of explanation) by the above-described inter prediction unit are all limited to those stored at positions spatially or temporally adjacent or close to the block to be decoded.

[0111] For further improvement of the symbolization efficiency, the inter prediction unit 205 may derive, as the motion vector of the block to be decoded, a motion vector generated by adding a motion vector derived by recursively searching for the motion vectors stored in the decoded picture buffer 207 with respect to the motion vectors at positions spatially or temporally adjacent or close to the block to be decoded.

[0112] Here, regarding the technique of recursively (in a chain) searching for the motion vectors stored in the decoded picture buffer 207 starting from the motion vectors at positions spatially or temporally adjacent or close to the block to be decoded to generate a new motion vector (chained motion vector), this specification refers to it as "Chained Motion Vector Prediction (CMVP)". Hereinafter, the technical details of such chained motion vector prediction will be described in detail.

[0113] Using FIGS. 8 to 17, a method for deriving the chained motion vector in the chained motion vector prediction will be described.

[0114] FIG. 8 is a diagram showing an example of a method for deriving the chained motion vector in the chained motion vector prediction.

[0115] As shown in FIG. 8, the inter prediction unit 205 starts from the motion vector MvL0 k (0) of L0 corresponding to any merge candidate stored in the merge candidate list and the reference index, and recursively (in a chain) searches for the motion vectors stored in the decoded picture buffer 207 to generate a new motion vector.

[0116] For example, as shown in (Equation 1) and (Equation 2), the inter prediction unit 205 can derive a new motion vector generated by chained motion vector prediction and the reference picture for the reference block RefBlk of the reference picture RefPicL0 k (m).

[0117] MvL0 k / m =MvL0k (0) + MvL0 k (1) + MvL0 k (2) + … + MvL0 k (m) (Equation 1) RefPicL0 k / m = RefPicL0 k (m) (Equation 2) That is, the new motion vector MvL0 generated by the beaded motion vector prediction k / m can be calculated by adding all the motion vectors derived by searching the motion vectors stored in the decoding picture buffer 207 recursively (beaded).

[0118] Also, the new reference picture generated by the beaded motion vector prediction is the reference picture referred to by MvL0 k / m

[0119] FIG. 9 is a diagram showing an example of a method for deriving a beaded motion vector in the beaded motion vector prediction.

[0120] As shown in FIG. 9, in the beaded motion vector prediction, the inter prediction unit 205 may add a block vector (a vector indicating a reference destination within the same picture) to the search target recursively in addition to the search for the motion vector.

[0121] For example, when a block vector (Bv: Block vector) is added, a new motion vector derivation formula generated by the beaded motion vector prediction for the reference block RefBlk of the reference picture RefPicL0 k (m) has Bv added as shown in (Equation 3).

[0122] MvL0 k / m = MvL0 k (0) + Bv k (0) + MvL0 k (1) + MvL0 k (2) + … + MvL0 k (m) (Equation 3) ​FIG. 10 is a diagram showing an example of a recursive search of motion vectors including dual prediction (i.e., inter prediction using two different motion vectors for L0 and L1). FIG. 11 is a diagram showing an example of a bead-chain motion vector derived by a recursive search of motion vectors including dual prediction.

[0123] As shown in FIGS. 10 and 11, in the bead-chain motion vector prediction, the inter prediction unit 205 uses each search list number (L0 or L1) and each search depth (or each search count) of the motion vectors stored in the decoded picture buffer 207 to newly search for and derive one or more (up to two in the examples of FIGS. 10 and 11) motion vectors from one reference motion vector, and adds all the already derived motion vectors to each of the one or more newly derived motion vectors to obtain the bead-chain motion vector MvL0 k / m to generate.

[0124] FIGS. 12 and 13 are diagrams showing an example of searching for bead-chain motion vector prediction candidates after searching for multiple types of merge candidates of normal merge disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0125] As shown in FIGS. 12 and 13, the inter prediction unit 205 may search for bead-chain motion vector prediction candidates after searching for a predetermined merge candidate.

[0126] For example, as shown in FIG. 12, in VVC, the inter prediction unit 205 may search for bead-chain motion vector prediction candidates after searching for TMVP candidates.

[0127] Due to the nature of the bead-chain motion vector prediction that searches for new motion vectors using the motion vectors of the reference picture rather than the picture to be decoded stored in the decoded picture buffer 207, it will not be effective unless TMVP is effective in the sequence to be decoded or the group of pictures to be decoded or the picture to be decoded or the slice to be decoded.

[0128] Therefore, searching for the chain motion vector prediction candidates after searching for the TMVP candidates can be said to be a natural design method as a method for deriving motion vectors.

[0129] As a modification example of FIG. 12, as shown in FIG. 13, in the VVC and ECM, the inter prediction unit 205 may search for the chain motion vector prediction candidates after searching for the HMVP candidates.

[0130] Since the chain motion vector prediction generates a new motion vector by recursively searching for a motion vector, it must be applied in a state where at least one merge candidate is stored in the merge candidate list.

[0131] As shown in FIG. 13, by searching for the chain motion vector prediction candidates after searching for the HMVP candidates for which the storage of one or more motion vector candidates can be expected in the merge candidate list, the possibility of generating a new motion vector by the chain motion vector is increased.

[0132] As another modification example, as shown in FIG. 12, in the ECM, the inter prediction unit 205 may search for the chain motion vector prediction candidates after searching for the spatial merge candidates.

[0133] Although the search position of the spatial merge candidates is shown in FIG. 7, for example, if the search position of the spatial merge candidates is increased, the possibility of storing one or more motion vector candidates in the merge candidate list increases, and a new motion vector candidate is easily generated by the chain merge candidates.

[0134] FIGS. 14 to 17 are diagrams showing an example of the positions (index numbers) in the merge candidate list where the merge candidates that are the derivation targets of the chain motion vector prediction candidates and the chain motion vector prediction candidates are added.

[0135] Note that in FIGS. 14 to 17, for the sake of simplifying the description of the technical content, only motion vectors are described as the inter-prediction information stored in the merge candidate list, but the above-described reference indexes are also stored.

[0136] In addition to the motion vector and the reference index, the inter-prediction unit 205 may store bcwIdx, hpelIfIdx, licFlag, and mhpFlag, which are internal parameters for changing the above-described inter-prediction pixel generation method.

[0137] As shown in FIG. 14, after storing a predetermined merge candidate in the merge candidate list, the inter-prediction unit 205 sequentially applies a consecutive motion vector prediction from the first merge candidate stored in the merge candidate list, and may sequentially store all of the new merge candidates generated by the consecutive motion vector prediction for each merge candidate in the merge candidate list.

[0138] Here, the predetermined merge candidate may be any of the plurality of types of merge candidates in the normal merge described with reference to FIGS. 12 and 13, or may be a predetermined number (predetermined number of merge_idx) of merge candidates.

[0139] Such a predetermined number may be set as a fixed value, or may be set as a variable value in the decoding target sequence or decoding target picture group or decoding target picture or decoding target slice unit using the control information of the decoding target sequence or decoding target picture group or decoding target picture or decoding target slice unit.

[0140] As a modification of FIG. 14, as shown in FIG. 15, after storing a predetermined merge candidate in the merge candidate list, the inter-prediction unit 205 sequentially applies a consecutive motion vector prediction from the first merge candidate stored in the merge candidate list, and may sequentially store the new consecutive motion vectors (i.e., merge candidates) generated at each search depth of the consecutive motion vector prediction for each merge candidate in the merge candidate list.

[0141] As a modification example of FIGS. 14 and 15, as shown in FIG. 16, after the first merge candidate is stored in the merge candidate list, the inter prediction unit 205 sequentially applies the bead string movement vector prediction from the merge candidate, and all of the new bead string movement vectors (i.e., merge candidates) generated by the bead string movement vector prediction for each merge candidate may be sequentially stored in the merge candidate list.

[0142] As a modification example of FIGS. 14 to 16, as shown in FIG. 17, after the first merge candidate is stored in the merge candidate list, the inter prediction unit 205 sequentially applies the bead string movement vector prediction from the merge candidate, and the new bead string movement vectors (i.e., merge candidates) generated at each search depth of the bead string movement vector prediction for each merge candidate may be sequentially stored in the merge candidate list.

[0143] As a modification example of FIGS. 14 to 17, the inter prediction unit 205 may combine the bead string movement vectors generated using each search depth of the bead string movement vector prediction for each merge candidate list with the bead string movement vectors generated at search depths different from such each search depth to generate new merge candidates.

[0144] As a modification example of FIGS. 14 to 17, the inter prediction unit 205 may combine the bead string movement vectors generated using each search list number or each search depth of the bead string movement vector prediction for each merge candidate list with the bead string movement vectors generated using search list numbers or search depths different from such each search list number or each search depth to generate new merge candidates.

[0145] <2. Pruning Process of Bead String Movement Vector Prediction Candidates> Using FIG. 18, the pruning process when the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment adds a merge candidate (the above-described bead string movement vector prediction candidate) corresponding to the bead string movement vector derived by the bead string movement vector prediction to the merge candidate list will be described.

[0146] FIG. 18 is a diagram showing an example of a pruning process when the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment adds a merge candidate corresponding to a rosary movement vector derived by rosary movement vector prediction to a merge candidate list.

[0147] The inter prediction unit 205 determines whether to add a merge candidate corresponding to the rosary movement vector derived by rosary movement vector prediction based on a predetermined condition.

[0148] Specifically, as shown in FIG. 18, when the inter prediction unit 205 determines in step S000 that a predetermined condition is satisfied, in step S002, it determines not to add a merge candidate corresponding to the rosary movement vector derived by rosary movement vector prediction.

[0149] On the other hand, when the inter prediction unit 205 determines in step S000 that the predetermined condition is not satisfied, in step S001, it determines to add a merge candidate corresponding to the rosary movement vector derived by rosary movement vector prediction.

[0150] Here, for example, the predetermined condition may be that the reference pictures and motion vectors associated with at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived rosary movement vector match.

[0151] When the inter prediction unit 205 determines that the reference pictures and motion vectors associated with each of at least one merge candidate stored in the merge candidate list in this way and the merge candidate corresponding to the derived bead-string motion vector match, the merge candidate corresponding to the bead-string motion vector is not added to the merge candidate list, that is, it is determined that the merge candidate corresponding to the bead-string motion vector is pruned (to be a pruning target), thereby preventing a case where two or more identical motion vectors are stored in the merge candidate list, reducing the amount of code of merge_idx (merge index) sent to the image decoding apparatus 200, and improving the coding efficiency.

[0152] Alternatively, the predetermined condition may be that all of one or more reference pictures and motion vectors associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead-string motion vector match.

[0153] Alternatively, the predetermined condition may be that all of the reference pictures and motion vectors of merge candidate list number 0 (L0) and merge candidate list number 1 (L1) associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead-string motion vector match.

[0154] Furthermore, the predetermined condition may be that the reference pictures associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead-string motion vector match, and the error (Euclidean distance) of the reference positions of the motion vectors associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead-string motion vector is less than a predetermined threshold.

[0155] As a result, the pruning process of merge candidates corresponding to the beaded movement vector is strengthened, cases where two or more similar movement vectors are stored in the merge candidate list are more effectively prevented, the amount of code of merge_idx sent to the image decoding apparatus 200 is reduced, and the coding efficiency is improved.

[0156] As a further control, the inter prediction unit 205 may set a predetermined threshold value when there is one motion vector to be compared to be larger than a predetermined threshold value when there are two motion vectors to be compared.

[0157] In this way, by setting the predetermined threshold value when there is one motion vector to be compared to be larger than the predetermined threshold value when there are two motion vectors to be compared, a more accurate pruning process becomes possible.

[0158] Here, the inter prediction unit 205 may set the predetermined threshold value when there is one motion vector to be compared to 1.5 pixels.

[0159] Alternatively, the inter prediction unit 205 may set the predetermined threshold value when there is one motion vector to be compared to 1.0 pixel or 2.0 pixels.

[0160] Also, the inter prediction unit 205 may set the predetermined threshold value when there are two motion vectors to be compared to 1.0 pixel.

[0161] Alternatively, the inter prediction unit 205 may set the predetermined threshold value when there are two motion vectors to be compared to 0.5 pixel or 1.5 pixels.

[0162] Further, the above-described predetermined conditions may also be that internal parameters that change the reference picture, the motion vector, and the generation method of predetermined inter prediction pixels associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived beaded movement vector match.

[0163] Alternatively, the predetermined condition may be that the internal parameter (hpelIfIdx) that specifies the application presence or absence of the reference picture, motion vector, and switching interpolation filter associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector matches.

[0164] Alternatively, the predetermined condition may be that the internal parameter (licFlag) that controls the application presence or absence of the reference picture, motion vector, and local luminance compensation associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector matches.

[0165] Furthermore, the predetermined condition may be that the internal parameter (bcwIdx) that specifies the weighted value of weighted bi-prediction in block units of the reference picture and motion vector associated with each of at least one merge candidate stored in the merge candidate list and the merge candidate corresponding to the derived bead string motion vector matches.

[0166] By including the above-described internal parameter in the predetermined condition, the inter prediction unit 205 can leave room for improving the inter prediction accuracy by generating inter prediction pixels in different ways even when the reference pictures and motion vectors to be compared match but the internal parameters are different, so that an improvement in coding efficiency can be expected.

[0167] <3. Control of the reference source and reference destination when deriving the bead string motion vector> With reference to FIGS. 19 to 22, a method for controlling the reference source and reference destination when the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment derives the bead string motion vector in the bead string motion vector prediction will be described.

[0168] First, with reference to FIG. 19, a method for controlling the reference source when deriving the bead string motion vector in the bead string motion vector prediction will be described.

[0169] FIG. 19 is a diagram showing an example of a method for controlling a reference source when deriving a bead string movement vector in the bead string movement vector prediction by the inter prediction unit 205.

[0170] When deriving the bead string movement vector in the bead string movement vector prediction, the inter prediction unit 205 may set the starting point of the first movement vector constituting the bead string movement vector to a part or all of the block to be decoded.

[0171] As an example of setting to a part of the block to be decoded, as shown in FIG. 19, when deriving the bead string movement vector in the bead string movement vector prediction, the inter prediction unit 205 may set the starting point of the first movement vector constituting the bead string movement vector to the center (C) of the block to be decoded (CurBlk).

[0172] Alternatively, as shown in FIG. 19, when deriving the bead string movement vector in the bead string movement vector prediction, the inter prediction unit 205 may, in addition to setting the starting point of the first movement vector constituting the bead string movement vector to the center (C) of the block to be decoded (CurBlk), set it to a predetermined position of the block to be decoded (CurBlk), and search whether movement vectors are accumulated at the reference destinations of the movement vectors in a predetermined order.

[0173] Specifically, as shown in FIG. 19, when deriving the bead string movement vector in the bead string movement vector prediction, the inter prediction unit 205 may, in addition to setting the starting point of the first movement vector constituting the bead string movement vector to the center (C) of the block to be decoded (CurBlk), set it to the four corners of the block to be decoded (CurBlk), and search whether movement vectors are accumulated at the reference destinations of the movement vectors in the order of the center (C), top left (TL), top right (TR), bottom left (BL), and bottom right (BR) of the block to be decoded as a predetermined order.

[0174] The inter prediction unit 205 may generate a bead string movement vector for each movement vector derived by searching at each point.

[0175] As a modification example, the inter prediction unit 205 may compare the differences between the adjacent decoded pixel values of the reference block and the block to be decoded of each motion vector derived by searching at each point, and limit to at least one or more motion vectors with small such differences to generate a connected motion vector.

[0176] As another modification example, the inter prediction unit 205 may derive a new motion vector for generating a connected motion vector by weighted-averaging each motion vector derived by searching at each point.

[0177] Specifically, when weighted-averaging each motion vector, the inter prediction unit 205 may scale each motion vector based on the distance of the reference picture of each motion vector from the block to be decoded to generate a new motion vector. The scaling destination may be set to the reference picture with the minimum distance from the block to be decoded.

[0178] When deriving the connected motion vector, the inter prediction unit 205 sets the starting point of the first motion vector constituting the connected motion vector not only at the center of the block to be decoded but also at the four corners of the block to be decoded, which increases the possibility of deriving a connected motion vector, and as a result, an improvement in coding efficiency can be expected.

[0179] As further control, in the connected motion vector prediction, the inter prediction unit 205 may control the starting point of the first motion vector constituting the connected motion vector according to the block size or block aspect ratio of the block to be decoded when deriving the connected motion vector.

[0180] For example, in the connected motion vector prediction, when deriving the connected motion vector, the inter prediction unit 205 may set the starting point of the first motion vector constituting the connected motion vector only at the center of the block to be decoded when the block size of the block to be decoded is equal to or less than the first size.

[0181] When the block size of the block to be decoded is less than or equal to the first size, when deriving the beaded movement vector, the first movement vector constituting the beaded movement vector is likely to be the same at the center and the four corners of the block to be decoded.

[0182] Therefore, when the block size of the block to be decoded is less than or equal to the first size, when deriving the beaded movement vector, by setting the starting point of the first movement vector constituting the beaded movement vector only at the center of the block to be decoded (without adding the four corners of the block to be decoded), the search processing amount of the beaded movement vector can be reduced.

[0183] Alternatively, when predicting the beaded movement vector, the inter prediction unit 205, when deriving the beaded movement vector, sets the starting point of the first movement vector constituting the beaded movement vector at the four corners of the block to be decoded in addition to the center of the block to be decoded when the block size of the block to be decoded is larger than the first size, and searches in the order of the center, upper left, upper right, lower left, lower right of the block to be decoded as a predetermined order to see if a movement vector is stored at the reference destination of the movement vector.

[0184] When the block size of the block to be decoded is larger than the first size, when deriving the beaded movement vector, the first movement vector constituting the beaded movement vector is likely to be different at the center and the four corners of the block to be decoded.

[0185] Therefore, when the block size of the block to be decoded is larger than the first size, when deriving the beaded movement vector, by setting the starting point of the first movement vector constituting the beaded movement vector at the four corners of the block to be decoded in addition to the center of the block to be decoded, different beaded movement vectors are generated, and an improvement in coding efficiency can be expected.

[0186] Here, since a case where the block to be decoded is non-square is also assumed, the block size (the number of pixels in the block) of the block to be decoded may be replaced with the width and height of the block to be decoded, and the above-described control may be performed.

[0187] Specifically, when deriving the bead-chain motion vector in the bead-chain motion vector prediction, the inter prediction unit 205, when the width or height of the block of the block to be decoded is larger than a predetermined number of pixels, sets the starting point of the first motion vector constituting the bead-chain motion vector not only at the center of the block to be decoded but also at the four corners of the block to be decoded, and searches in the predetermined order of the center, upper left, upper right, lower left, and lower right of the block to be decoded to check whether motion vectors are accumulated at the reference destinations of the motion vectors.

[0188] The inter prediction unit 205 may set any one of 8 pixels, 16 pixels, or 32 pixels as such a predetermined number of pixels.

[0189] Alternatively, when deriving the bead-chain motion vector in the bead-chain motion vector prediction, the inter prediction unit 205, when the aspect ratio of the block of the block to be decoded is larger than a predetermined ratio, sets the lower left and upper right of the block to be decoded in addition to the center of the block to be decoded, and searches in the predetermined order of the center, lower left, and upper right of the block to be decoded to check whether motion vectors are accumulated at the reference destinations of the motion vectors.

[0190] The inter prediction unit 205 may set any one of 4, 8, or 16 as such a predetermined ratio.

[0191] Second, an example of a control method for the reference destination when deriving the bead-chain motion vector in the bead-chain motion vector prediction will be described with reference to FIGS. 20 and 21.

[0192] FIG. 20 is a diagram showing an example of a method for controlling a reference destination when deriving a string-of-beads motion vector in the string-of-beads motion vector prediction by the inter prediction unit 205.

[0193] As shown in FIG. 20, when the reference position of each motion vector refers to outside the reference picture in the recursive motion vector search process for deriving the string-of-beads motion vector, the inter prediction unit 205 may search for the motion vector at the accumulation location of the motion vector within the nearest reference picture from such a reference position.

[0194] Since there may be a case where the reference position of the motion vector refers to outside the reference picture and the motion vector strung with beads from such a motion vector is in a different reference picture, a new string-of-beads motion vector can be generated.

[0195] Alternatively, as shown in FIG. 20, when the reference position of each motion vector refers to outside the reference picture in the recursive motion vector search process for deriving the string-of-beads motion vector, the inter prediction unit 205 may terminate the generation of the string-of-beads motion vector.

[0196] When the reference position of the motion vector refers to outside the reference picture, it is highly likely that the motion vector strung with beads from such a motion vector also refers to outside the reference picture in a different reference picture. Therefore, by terminating the generation of the string-of-beads motion vector, the decoding processing amount can be reduced.

[0197] Alternatively, when the decoding picture buffer 207 stores an inter prediction padding area outside the reference picture, the description of outside the reference picture above is replaced with the description of outside the inter prediction padding area, and the inter prediction unit 205 may perform the same processing.

[0198] FIG. 21 is a diagram showing an example of a method for controlling a reference source and a reference destination when deriving a chained motion vector in the chained motion vector prediction by the inter prediction unit 205 in a case where sequential decoder refresh (GDR: Gradual Decoding Refresh), which is known as an encoding / decoding technique for low-latency video transmission, is applied to a group of pictures to be decoded.

[0199] GDR is a technique for reducing the processing delay of video encoding / decoding at a certain bit rate compared to video encoding / decoding in a random access configuration including an intra picture widely used in video transmission by updating (hereinafter referred to as intra update) only a partial area of a screen by encoding / decoding using intra prediction and moving such a partial area for each picture to be decoded.

[0200] FIG. 21 illustrates an intra update area by a slice (Intra slice), and shows an example in which such a slice moves from the upper end to the lower end of a picture to be decoded.

[0201] Here, in this specification, three divided areas by GDR in a picture to be decoded are referred to as an area after intra update, an area during intra update, and an area before intra update.

[0202] As shown in FIG. 21, when sequential decoder refresh is applied to a group of pictures to be decoded, the inter prediction unit 205 may limit the recursive search of the chained motion vector for the chained motion vector prediction so as not to exceed the area during intra update.

[0203] For example, as shown in FIG. 21, when sequential decoder refresh is applied to a group of pictures to be decoded and the block to be decoded is included in the area after intra update, the inter prediction unit 205 may limit the area that can be referenced in each search of the recursive motion vector for the chained motion vector prediction to only the area after intra update in the reference picture for each search target.

[0204] Alternatively, when the decoder update is sequentially applied to the picture group to be decoded and the block to be decoded is included in the area before intra-update, the inter prediction unit 205 may limit the area that can be referred to in each search of the recursive motion vectors of the bead-chain motion vector prediction to only the area before intra-update in the reference picture for each search target.

[0205] In this way, when the decoder update is sequentially applied to the picture group to be decoded, the inter prediction unit 205 restricts the search for the recursive motion vectors of the bead-chain motion vector prediction so as not to exceed the area during intra-update, thereby enabling the generation of a bead-chain motion vector with high prediction accuracy.

[0206] FIG. 22 is a diagram showing an example of controlling the reference destination when the bead-chain motion vector in the bead-chain motion vector prediction is derived by the inter prediction unit 205 in a case where reference picture resampling (RPR: Reference Picture Resampling) disclosed in Non-Patent Document 1, in which reference to a reference picture with a different resolution (sampling ratio) from the decoded picture is allowed, is applied to the block to be decoded.

[0207] In RPR, the inter prediction unit 205 calculates the ratios of the width and height of the reference picture to the width and height of the decoded picture, respectively, and scales the pixel position and motion vector length of the reference picture with respect to the decoded picture according to the ratios to generate an inter prediction pixel.

[0208] Therefore, for example, as shown in FIG. 22, when the width and height of the reference picture are 0.5 times the width and height of the decoded picture, the pixel position and motion vector length of the reference picture with respect to the decoded picture are scaled by 0.5 times.

[0209] On the other hand, when generating the bead-chain motion vector, the inter prediction unit 205 needs to connect the motion vectors at the same scale as the decoded picture.

[0210] Therefore, as shown in FIG. 22, when the inter prediction unit 205 refers to motion vectors for reference pictures with a resolution (sampling ratio) different from that of the picture to be decoded, recursively searches for motion vectors from such reference pictures, and generates a chained motion vector, the length of the recursively derived motion vector may be inversely scaled according to the ratios of the width and height of the picture to be decoded and the width and height of each reference picture, and then added, so as to generate a chained motion vector.

[0211] As a modification example, when motion vector reference is allowed for a reference picture with a resolution (sampling ratio) different from that of the picture to be decoded for the sequence to be decoded or the group of pictures to be decoded, the inter prediction unit 205 may apply the chained motion vector only to a reference picture with the same sampling ratio as the picture to be decoded.

[0212] In other words, when motion vector reference is allowed for a reference picture with a resolution (sampling ratio) different from that of the picture to be decoded for the sequence to be decoded or the group of pictures to be decoded, the inter prediction unit 205 may decide not to apply the chained motion vector to a reference picture with a sampling ratio different from that of the picture to be decoded.

[0213] <4. Inheritance of Inter Prediction Information in Chained Motion Vector Prediction> When deriving a merge candidate by chained motion vector prediction, the inter prediction unit 205 may control whether to inherit a predetermined internal parameter for changing the method of generating inter prediction pixels.

[0214] (Switching Interpolation Filter) As an example, when deriving merge candidates by beaded movement vector prediction, the inter prediction unit 205 does not inherit the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter associated with each recursively searched motion vector, and always sets the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter corresponding to the merge candidate derived by beaded movement vector prediction so as to indicate that the switching interpolation filter is not applied.

[0215] Alternatively, when deriving merge candidates by beaded movement vector prediction, the inter prediction unit 205 may inherit, as the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter corresponding to the merge candidate derived by beaded movement vector prediction, the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter associated with the first or last motion vector constituting the beaded movement vector among the internal parameters (hpelIfIdx) that specify whether to apply the switching interpolation filter associated with each recursively searched motion vector.

[0216] Alternatively, when deriving merge candidates by beaded movement vector prediction, if, among the internal parameters (hpelIfIdx) that specify whether to apply the switching interpolation filter associated with each recursively searched motion vector, at least one internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter indicates that the switching interpolation filter is applied, the internal parameter (hpelIfIdx) that specifies whether to apply the switching interpolation filter corresponding to the merge candidate derived by beaded movement vector prediction may be set to indicate that the switching interpolation filter is applied.

[0217] Furthermore, when the bead string movement vector generated by the bead string movement vector prediction is composed of at least one or more block vectors, the inter prediction unit 205 may always set an internal parameter (hpelIfIdx) that specifies whether to apply a switching interpolation filter corresponding to a merge candidate derived by the bead string movement vector prediction so as to indicate that the switching interpolation filter is not applied.

[0218] (Local Luminance Compensation) As an example, when deriving a merge candidate by the bead string movement vector prediction, the inter prediction unit 205 does not inherit an internal parameter (licFlag) that controls whether to apply local luminance compensation associated with each recursively searched movement vector, and always sets an internal parameter (licFlag) that controls whether to apply local luminance compensation corresponding to the merge candidate derived by the bead string movement vector prediction so as to indicate that local luminance compensation is not applied.

[0219] Alternatively, when deriving a merge candidate by the bead string movement vector prediction, the inter prediction unit 205 may inherit, as the internal parameter (licFlag) that controls whether to apply local luminance compensation corresponding to the merge candidate derived by the bead string movement vector prediction, the internal parameter (licFlag) that controls whether to apply local luminance compensation associated with the first or last movement vector that constitutes the bead string movement vector among the internal parameters (licFlag) that control whether to apply local luminance compensation associated with each recursively searched movement vector.

[0220] Alternatively, when deriving a merge candidate by the bead string movement vector prediction, among the internal parameters (licFlag) that control whether to apply local luminance compensation associated with each recursively searched movement vector, if at least one or more internal parameters (licFlag) that control whether to apply local luminance compensation indicate that local luminance compensation is applied, the internal parameter (licFlag) that controls whether to apply local luminance compensation corresponding to the merge candidate derived by the bead string movement vector prediction may be set to indicate that local luminance compensation is applied.

[0221] Furthermore, when the string-of-beads motion vector predicted by the inter prediction unit 205 is composed of at least one or more block vectors, an internal parameter (licFlagg) that controls whether to apply local luminance compensation corresponding to a merge candidate derived by the string-of-beads motion vector prediction may always be set so as to indicate not to apply local luminance compensation.

[0222] (Weighted bi-prediction in block units) As an example, when deriving a merge candidate by the string-of-beads motion vector prediction, the inter prediction unit 205 does not inherit an internal parameter (bcwIdx) that specifies the weight value of the weighted bi-prediction in block units associated with each recursively searched motion vector, and always sets the internal parameter (bcwIdx) that specifies the weight value of the weighted bi-prediction in block units corresponding to the merge candidate derived by the string-of-beads motion vector prediction so as to indicate that the weight value of the weighted bi-prediction in block units is the simple average.

[0223] Alternatively, when deriving a merge candidate by the string-of-beads motion vector prediction, among the internal parameters (bcwIdx) that specify the weight values of the weighted bi-prediction in block units associated with each recursively searched motion vector, the internal parameter (bcwIdx) that specifies the weight value of the weighted bi-prediction in block units associated with the first or last motion vector constituting the string-of-beads motion vector may be inherited as the internal parameter (bcwIdx) that specifies the weight value of the weighted bi-prediction in block units corresponding to the merge candidate derived by the string-of-beads motion vector prediction.

[0224] As a modification example, when deriving a merge candidate by beaded movement vector prediction, among internal parameters (bcwIdx) that specify weight values of block-based weighted bi-prediction associated with each recursively searched motion vector, if the internal parameter (bcwIdx) that specifies the weight value of block-based weighted bi-prediction for at least one or more blocks indicates that the weight value of block-based weighted bi-prediction is a simple average, the internal parameter (bceIdx) that specifies the weight value of block-based weighted bi-prediction corresponding to the merge candidate derived by beaded movement vector prediction may be set to indicate that the weight value of block-based weighted bi-prediction is a simple average.

[0225] (Multiple hypothesis prediction) As an example, when deriving a merge candidate by beaded movement vector prediction, the inter prediction unit 205 may always set the internal parameter (mhpFlag) that controls whether to apply multiple hypothesis predictions corresponding to the merge candidate derived by beaded movement vector prediction so as to indicate that multiple hypothesis predictions are not applied, without inheriting the internal parameter (mhpFlag, the fourth internal parameter) that controls whether to apply multiple hypothesis predictions associated with each recursively searched motion vector.

[0226] As a modification example, when deriving a merge candidate by beaded movement vector prediction, among the internal parameters (mhpFlag) that control whether to apply multiple hypothesis predictions associated with each recursively searched motion vector, the internal parameter (mhpFlag) that controls whether to apply multiple hypothesis predictions associated with the first or last motion vector that constitutes the beaded movement vector may be set to indicate inheritance as the internal parameter (mhpFlag) that controls whether to apply multiple hypothesis predictions corresponding to the merge candidate derived by beaded movement vector prediction.

[0227] As a modification example, when deriving a merge candidate by beaded movement vector prediction, among the internal parameters (mhpFlag) that control whether to apply multiple hypothesis predictions associated with each recursively searched movement vector in the inter prediction unit 205, if at least one of the internal parameters (mhpFlag) that control whether to apply multiple hypothesis predictions indicates that multiple hypothesis predictions are to be applied, it may be determined that the internal parameter (mhpFlag) that controls whether to apply multiple hypothesis predictions corresponding to the merge candidate derived by beaded movement vector prediction indicates that multiple hypothesis predictions are to be applied.

[0228] As a further modification example, when the beaded movement vector generated by beaded movement vector prediction is composed of at least one block vector, the internal parameter (mhpFlag) that controls whether to apply multiple hypothesis predictions corresponding to the merge candidate derived by beaded movement vector prediction may always be set to indicate that multiple hypothesis predictions are not applied.

[0229] As described above, if the internal parameter that changes the generation method of inter prediction pixels is not always inherited in beaded movement vector prediction, the coded amount of inter prediction information that needs to be stored in the decoded picture buffer 207 is reduced.

[0230] On the other hand, as described above, if the internal parameter that changes the generation method of inter prediction pixels is inherited in beaded movement vector prediction, there is a possibility that the inter prediction pixels using the merge candidate generated by beaded movement vector prediction will be more accurate, and an improvement in coding efficiency can be expected.

[0231] Also, when inheriting the above internal parameter, if the beaded movement vector is composed of at least one block vector, since the need to change the generation method of inter prediction pixels is low, by not inheriting such an internal parameter, that is, setting it so as not to change the generation method of inter prediction pixels, an improvement in coding efficiency can be expected.

[0232] <5. Rearrangement of Merge Candidates Containing Bead Connection Movement Vector Prediction Candidates and Correction of Vectors> (Rearrangement of Merge Candidates) The later the bead connection movement vector prediction candidate is derived compared to other merge candidates, the more difficult it is to store the bead connection movement vector prediction candidate in the merge candidate list.

[0233] As a method for solving this problem, the inter prediction unit 205 calculates the template matching cost of all merge candidates stored in the merge candidate list including the merge candidates derived by bead connection movement vector prediction, and may rearrange the storage positions of each merge candidate in the merge candidate list in ascending order according to the ascending order of the template matching cost.

[0234] As an example, as a technique for rearranging the storage positions of each merge candidate in the merge candidate list based on the template matching cost of each merge candidate, the inter prediction unit 205 may use the adaptive reordering merge candidate (ARMC) disclosed in Non-Patent Document 2.

[0235] Furthermore, as described above, since the bead connection movement vector prediction recursively generates new movement vectors, when other merge candidates are derived after the derivation of the bead connection movement vector prediction candidate, there is a possibility that the other merge candidates may not be stored in the merge candidate list.

[0236] As a method for solving this problem, the inter prediction unit 205 calculates the template matching cost of all unpruned merge candidates including the merge candidates derived by bead connection movement vector prediction, and may store them in the merge candidate list in ascending order according to the ascending order of the template matching cost.

[0237] Alternatively, the inter prediction unit 205 may calculate the template matching costs of all unpruned merge candidates including the merge candidates derived by the bead string movement vector prediction, and perform a series of processes until they are stored in ascending order in the merge candidate list in ascending order of the template matching costs a predetermined number of times. For example, such a predetermined number of times may be two or three times.

[0238] As a result, merge candidates capable of generating inter prediction pixels with high accuracy are arranged in the merge candidate list, and an improvement in coding efficiency can be expected.

[0239] (Correction of Vector) The inter prediction unit 205 may correct the motion vector associated with the merge candidate derived by the bead string movement vector prediction by a predetermined method.

[0240] As an example, the inter prediction unit 205 may correct the motion vector associated with the merge candidate derived by the bead string movement vector prediction by template matching.

[0241] Here, the inter prediction unit 205 may apply a technique similar to the template matching disclosed in Non-Patent Document 2 as such template matching.

[0242] As an example, the inter prediction unit 205 may correct the motion vector associated with the merge candidate derived by the bead string movement vector prediction by bilateral matching.

[0243] Here, the inter prediction unit 205 may apply a technique similar to the bilateral matching disclosed in Non-Patent Document 1 and Non-Patent Document 2 as such bilateral matching. For example, the inter prediction unit 205 may apply the decoder-side motion vector refinement (DMVR) disclosed in Non-Patent Document 1.

[0244] As an example, the inter prediction unit 205 may correct the motion vector associated with the merge candidate derived by the bead string motion vector prediction by adding the motion vector difference derived from the control information.

[0245] Here, as control information indicating the motion vector difference, the inter prediction unit 205 may apply a technique similar to the merge motion vector difference (MMVD: Merge with Motion Vector Difference) disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0246] By the inter prediction unit 205 correcting the motion vector associated with the merge candidate derived by the bead string motion vector prediction by a predetermined method, the inter prediction accuracy is improved, and an improvement in coding efficiency can be expected.

[0247] <6. Various restrictions on bead string motion vector prediction (applicability, number of pictures, number of merge candidates)> Using FIG. 23, various restriction methods for the bead string motion vector prediction by the decoding unit 201 and the inter prediction unit 205 of the image decoding apparatus 200 according to the present embodiment will be described. (Applicability of bead string motion vector prediction) First, regarding the applicability (applicable or not) of the bead string motion vector prediction, as described above, by its nature, when TMVP is applicable, the bead string motion vector prediction is applicable. In other words, when TMVP is not applicable, the bead string motion vector prediction is not applicable.

[0248] Therefore, the applicability of the bead string motion vector prediction can be controlled according to the control information (syntax) that controls the applicability of the temporal motion vector prediction TMVP.

[0249] That is, the inter prediction unit 205 may control the applicability of the bead string motion vector prediction based on the control information that controls the applicability of the temporal motion vector prediction (TMVP) sent from the decoding unit 201.

[0250] Here, in Non-Patent Document 1, as syntax for controlling the applicability of TMVP, syntax at the sequence unit to be decoded (sps_temporal_mvp_enabled_flag, first control information, first syntax) and syntax at the picture unit to be decoded (ph_temporal_mvp_enabled_flag, second control information, second syntax) are defined.

[0251] That is, regarding the applicability of the bead-chain motion vector prediction, it can be controlled in the same unit as TMVP or a unit lower than TMVP.

[0252] For example, the method for controlling the applicability of the bead-chain motion vector prediction in the inter prediction 205 that controls the applicability of the bead-chain motion vector prediction in units of the block to be decoded is as follows.

[0253] The inter prediction unit 205 may control the applicability of the bead-chain motion vector prediction for the block to be decoded based on the control information that controls the applicability of the temporal motion vector prediction (TMVP) at the sequence unit to be decoded sent from the decoding unit 201.

[0254] More specifically, when the control information that controls the applicability of the temporal motion vector prediction (TMVP) at the sequence unit to be decoded sent from the decoding unit 201 specifies that the temporal motion vector prediction is applicable, the inter prediction unit 205 specifies that the bead-chain motion vector prediction for the block to be decoded is applicable.

[0255] On the other hand, when the above control information specifies that the temporal motion vector prediction is not applicable, the inter prediction unit 205 specifies that the bead-chain motion vector prediction for the block to be decoded is not applicable.

[0256] Alternatively, the inter prediction unit 205 may control whether or not to apply the bead-chain motion vector prediction to the block to be decoded based on the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP) for each picture to be decoded sent from the decoding unit 201.

[0257] More specifically, when the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP) for each picture to be decoded sent from the decoding unit 201 specifies that the temporal motion vector prediction can be applied, the inter prediction unit 205 specifies that the bead-chain motion vector prediction can be applied to the block to be decoded.

[0258] On the other hand, when the above control information specifies that the temporal motion vector prediction cannot be applied, the inter prediction unit 205 specifies that the bead-chain motion vector prediction cannot be applied to the block to be decoded.

[0259] The inter prediction unit 205 may control whether or not to apply the bead-chain motion vector prediction in units of the sequence to be decoded or units of the decoding target smaller than the sequence to be decoded based on the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP) for each sequence to be decoded sent from the decoding unit 201.

[0260] Alternatively, the inter prediction unit 205 may control whether or not to apply the bead-chain motion vector prediction in units of the picture to be decoded or units of the decoding target smaller than the picture to be decoded based on the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP) for each picture to be decoded sent from the decoding unit 201.

[0261] Alternatively, before controlling whether or not to apply the bead-chain motion vector prediction for each block to be decoded by the inter prediction unit 205, the decoding unit 201 may control whether or not to apply the bead-chain motion vector prediction based on the control information for controlling whether or not to apply the temporal motion vector prediction (TMVP).

[0262] FIG. 23 shows an example of a process in which a syntax (sps_chained_mvp_enabled_flag) that controls whether or not to apply a beaded movement vector prediction is defined for each decoding target sequence unit, and the decoding unit 201 determines whether or not to decode the sps_chained_mvp_enabled_flag based on a syntax (sps_temporal_mvp_enabled_flag) that controls whether or not to apply a temporal movement vector prediction (TMVP) for each decoding target sequence unit.

[0263] As shown in FIG. 23, in step S100, when the decoding unit 201 determines that the syntax (sps_temporal_mvp_enabled_flag) that controls whether or not to apply a temporal movement vector prediction (TMVP) for each decoding target sequence unit is 1, in step S101, the decoding unit 201 decodes a syntax (sps_chained_mvp_enabled_flag, the third syntax) that controls whether or not to apply a beaded movement vector prediction for each decoding target sequence unit.

[0264] On the other hand, when the decoding unit 201 determines in step S100 that the syntax (sps_temporal_mvp_enabled_flag) that controls whether or not to apply a temporal movement vector prediction (TMVP) for each decoding target sequence unit is not 1, in step S102, the decoding unit 201 does not decode the syntax (sps_chained_mvp_enabled_flag) that controls whether or not to apply a beaded movement vector prediction for each decoding target sequence unit.

[0265] Here, when the value of sps_temporal_mvp_enabled_flag is 1, the decoding unit 201 determines that temporal motion vector prediction (TMVP) can be applied in the decoding target sequence unit. When the value of sps_temporal_mvp_enabled_flag is 0, the decoding unit 201 determines that temporal motion vector prediction (TMVP) cannot be applied in the decoding target sequence unit. When sps_temporal_mvp_enabled_flag is not decoded, the value of sps_temporal_mvp_enabled_flag is presumed to be 0.

[0266] In addition, when the value of sps_chained_mvp_enabled_flag is 1, the decoding unit 201 determines that chained motion vector prediction can be applied in the decoding target sequence unit. When the value of sps_chained_mvp_enabled_flag is 0, the decoding unit 201 determines that chained motion vector prediction cannot be applied in the decoding target sequence unit. When sps_chained_mvp_enabled_flag is not decoded, the value of sps_chained_mvp_enabled_flag is presumed to be 0.

[0267] In this way, by determining whether sps_chained_mvp_enabled_flag is decoded based on the value of sps_temporal_mvp_enabled_flag, in a case where it is obvious that decoding of sps_chained_mvp_enabled_flag is unnecessary, the sps_chained_mvp_enabled_flag does not need to be decoded, so that the amount of code can be reduced.

[0268] When the chained motion vector prediction is used in combination with the above-described rearrangement (ARMC: Adaptive Reordering Merge Candidate) by comparing the template matching costs of the merge candidates, a greater improvement in the coding efficiency can be expected.

[0269] Therefore, as a modified example of the control method for enabling or disabling the prediction of the beaded movement vector based on the enable or disable of the above-described TMVP, the inter prediction unit 205 may control the enable or disable of the prediction of the beaded movement vector based on both the enable or disable of the TMVP and the enable or disable of the ARMC.

[0270] Specifically, when the TMVP is applicable and the ARMC is applicable, the inter prediction unit 205 may specify that the prediction of the beaded movement vector is applicable. Also, in other cases, the inter prediction unit 205 may specify that the prediction of the beaded movement vector is not applicable.

[0271] Regarding the enable or disable of the ARMC, control information similar to that of the TMVP may be provided, and the determination may be made based on the value of such control information.

[0272] (Number of reference pictures and search depth for beaded movement vector prediction) Hereinafter, a method for restricting the number of reference pictures or the search depth that can be referred to during the recursive search of the movement vector for the beaded movement vector prediction will be described.

[0273] First, a method for restricting the number of reference pictures that can be referred to during the recursive search of the movement vector for the beaded movement vector prediction will be described.

[0274] As an example, the inter prediction unit 205 may restrict the recursive search of the movement vector for the beaded movement vector prediction to only at least one or more reference pictures included in the reference picture list of the block to be decoded.

[0275] By restricting the recursive search of the movement vector for the beaded movement vector prediction by the inter prediction unit 205 to only at least one or more reference pictures included in the reference picture list of the block to be decoded, the decoding processing amount can be reduced compared to the case where such a restriction is not performed (for example, when the inter prediction unit 205 searches all the reference pictures stored in the decoded picture buffer 207).

[0276] Alternatively, the inter prediction unit 205 may limit the search for the recursive motion vector of the bead string motion vector prediction to only the reference picture referred to by the motion vector of the block to be decoded.

[0277] In other words, the inter prediction unit 205 re-searches only the block vectors within the reference picture referred to by the motion vector of the block to be decoded, and generates a bead string motion vector by bead string motion vector prediction.

[0278] Second, a method for limiting the search depth of the recursive motion vector of the bead string motion vector prediction will be described.

[0279] As an example, the inter prediction unit 205 may limit the search depth of the recursive motion vector of the bead string motion vector prediction in the block to be decoded based on the control information.

[0280] Specifically, the inter prediction unit 205 may limit the search depth of the recursive motion vector of the bead string motion vector prediction in the block to be decoded based on the control information (sps_max_num_my_chains) that sets the upper limit of the search depth of the recursive motion vector of the bead string motion vector prediction in units of the sequence to be decoded sent from the decoding unit.

[0281] Here, the control information of the sequence unit to be decoded may be control information of a picture unit or a slice unit, which is control information lower than the sequence unit to be decoded, or may be hierarchical control information combining these.

[0282] Alternatively, the inter prediction unit 205 may set a predetermined fixed value for the search depth of the recursive motion vector of the bead string motion vector prediction. The inter prediction unit 205 may set 1, 2, or 3 as such a predetermined fixed value.

[0283] By limiting the search depth of the beaded movement vector prediction in this way, the amount of decoding processing required for deriving the beaded movement vector can be reduced as compared to the case where it is not limited.

[0284] (Number of merge candidates for beaded movement vector prediction) Hereinafter, a method for limiting the number of merge candidates derived by beaded movement vector prediction will be described.

[0285] As an example, the inter prediction unit 205 may limit the number of merge candidates derived by beaded movement vector prediction in the block to be decoded based on the control information.

[0286] Specifically, the inter prediction unit 205 may limit the number of merge candidates derived by beaded movement vector prediction in the block to be decoded based on the control information (sps_max_num_cmvp_cand) that sets the upper limit of the search depth of the number of merge candidates derived by beaded movement vector prediction in the decoding target sequence unit sent from the decoding unit.

[0287] Here, the control information of the decoding target sequence unit may be control information of a picture unit or a slice unit which is control information lower than the decoding target sequence unit, or may be hierarchical control information combining these.

[0288] Alternatively, the inter prediction unit 205 may set a predetermined fixed value for the number of merge candidates derived by beaded movement vector prediction.

[0289] (Restriction on block vector search in beaded movement vector prediction) In the recursive motion vector search of the beaded movement vector prediction, it has been described above that the search of the block vector may be included. However, block vectors referring within the same picture are easy to find in screen images or game images including characters and computer-generated graphics (CG), but are difficult to find in images taken by a camera.

[0290] Therefore, it is desirable to be able to limit the search for block vectors in the beaded movement vector prediction according to the characteristics of the image.

[0291] As an example, the inter prediction unit 205 may limit the block vector search based on control information that specifies whether or not to search for block vectors in the beaded movement vector prediction for the block to be decoded.

[0292] Specifically, the inter prediction unit 205 may limit whether or not to search for block vectors in the beaded movement vector prediction for the block to be decoded based on control information (sps_cmvp_bv_search_enabled_flag) that specifies whether or not to search for block vectors in the beaded movement vector prediction in units of the sequence to be decoded sent from the decoding unit 201.

[0293] Here, the control information in units of the sequence to be decoded may be control information in units of pictures or slices, which are control information lower than the units of the sequence to be decoded, or may be hierarchical control information combining these.

[0294] (Restriction on the reference source of the beaded movement vector prediction) In the recursive search for movement vectors in the beaded movement vector prediction, the method of adding the four corners of the block to be decoded in addition to the center of the block to be decoded as the starting point of the first movement vector has been described above.

[0295] However, compared with screen images and game images in which pixel values are likely to change rapidly in pixel units, images taken with a camera are less likely to change.

[0296] Therefore, it is desirable to be able to limit the starting point of the first movement vector in the recursive search for movement vectors in the beaded movement vector prediction according to the characteristics of the image.

[0297] As an example, the inter prediction unit 205 may limit the reference source of the bead string motion vector prediction based on control information for specifying whether to add at least one or more predetermined positions of the block to be decoded by adding the starting point of the first motion vector in the bead string motion vector prediction for the block to be decoded to the center of the block to be decoded. Such predetermined positions may be the four corners within the block to be decoded described above.

[0298] Specifically, the inter prediction unit 205 may limit whether to add at least one or more predetermined positions of the block to be decoded by adding the starting point of the first motion vector in the bead string motion vector prediction for the block to be decoded to the center of the block to be decoded based on the sps_cmvp_multi_position_flag.

[0299] Here, the sps_cmvp_multi_position_flag is control information for specifying whether to add at least one or more predetermined positions of the block to be decoded in addition to the center of the block to be decoded to the starting point of the first motion vector in the bead string motion vector prediction for the block to be decoded.

[0300] Here, the control information of the decoding target sequence unit may be control information of a picture unit or a slice unit, which is control information lower than the decoding target sequence unit, or may be hierarchical control information combining these.

[0301] <7. Application Target of Bead String Motion Vector Prediction> The technical features in the case of applying the bead string motion vector prediction to merges other than the normal merge disclosed in Non-Patent Document 1 and Non-Patent Document 2 are described.

[0302] (MMVD, GPM, TM Merge) First, MMVD, GPM, and template matching merge (TM merge) disclosed in Non-Patent Document 1 and Non-Patent Document 2 construct a merge list in the same way as normal merge. However, in Non-Patent Document 2, the pruning of merge candidates when constructing these merge candidate lists is enhanced compared to normal merge, and these merge candidate lists are unlikely to be the same as the merge candidates stored in the merge candidate list of normal merge.

[0303] Therefore, when not reusing the merge candidate list of normal merge in MMVD, GPM, and TM merge, in addition to applying the bead-chain motion vector prediction to the merge candidates in the merge candidate list of normal merge, applying the bead-chain motion vector prediction to the merge candidates in each merge candidate list of MMVD, GPM, and TM merge can be expected to improve the coding efficiency.

[0304] Therefore, in addition to applying the bead-chain motion vector prediction to the merge candidates in the merge candidate list of normal merge, the inter prediction unit 205 may apply the bead-chain motion vector prediction to the merge candidates in the merge candidate list of the merge mode motion vector difference.

[0305] In addition, in addition to applying the bead-chain motion vector prediction to the merge candidates in the merge candidate list of normal merge, the inter prediction unit 205 may apply the bead-chain motion vector prediction to the merge candidates in the merge candidate list of the geometric partitioning mode.

[0306] In addition, in addition to applying the bead-chain motion vector prediction to the merge candidates in the merge candidate list of normal merge, the inter prediction unit 205 may apply the bead-chain motion vector prediction to the merge candidates in the merge candidate list of the template matching merge.

[0307] Here, for the beaded motion vectors generated for the merge mode motion vector difference, geometric division mode, and template matching merge, the inter prediction unit 205 may apply, in the same manner as normal merge, the pruning process at the time of storing the merge candidate list, the correction of the motion vector, and the rearrangement of the merge candidates (motion vectors).

[0308] (Adaptive Motion Vector Prediction - Merge Mode) In the adaptive motion vector prediction - merge mode (Adaptive Motion Vector Prediction - Merge mode) disclosed in Non - Patent Document 1 and Non - Patent Document 2 described above, since the motion vector of either list 0 (L0) or list 1 (L1) of the reference picture list is derived by adaptive motion vector prediction (AMVP), that is, by control information, if the beaded motion vector prediction is applied to the merge candidates of the merge candidate list of the reference picture list number not derived by AMVP, the options for selecting merge candidates that can be selected in the adaptive motion vector prediction - merge mode increase, so the inter prediction accuracy is improved and an improvement in coding efficiency can be expected.

[0309] Therefore, in the adaptive motion vector prediction - merge mode, the inter prediction unit 205 may apply the beaded motion vector prediction to the merge candidates of the reference picture list number not derived by the adaptive motion vector prediction in the merge candidate list.

[0310] Furthermore, when applying the beaded motion vector prediction to the merge candidates of the reference picture list number not derived by the adaptive motion vector prediction in the merge candidate list in the adaptive motion vector prediction - merge mode, the inter prediction unit 205 may limit the generated beaded motion vector so as not to refer to the reference picture of the motion vector derived by the adaptive motion vector prediction.

[0311] Here, for the chain-like motion vectors generated for the adaptive motion vector prediction-merge mode, the inter prediction unit 205 may apply pruning processing during storage in the merge candidate list, motion vector correction, and rearrangement of merge candidates (motion vectors) in the same manner as normal merge.

[0312] (Sub-block merge mode) As disclosed in Non-Patent Document 1 and Non-Patent Document 2 described above, in affine merge, in order to derive motion vectors for each sub-block within a block to be decoded by affine transformation, as shown in FIG. 24, in the case of single prediction (either L0 or L1), two or three control point motion vectors cpMv are derived, and in the case of dual prediction (both L0 and L1), four or six control point motion vectors cpMv are derived.

[0313] Note that all of the control point motion vectors of L0 and L1 for single prediction or dual prediction refer to the same reference picture.

[0314] In affine merge, as disclosed in Non-Patent Document 1 and Non-Patent Document 2, these control point motion vectors are derived from the merge candidates stored in the merge candidate list for affine merge.

[0315] The inter prediction unit 205 may derive merge candidates to be stored in the merge candidate list for affine merge in the same manner as the method for deriving merge candidates for affine merge disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0316] For example, when two or three control point motion vectors are accumulated at positions spatially adjacent or close to the block to be decoded in the inter prediction unit 205, they may be added to the merge candidate list for affine merge of the block to be decoded.

[0317] Alternatively, when a plurality of motion vectors are stored at positions spatially adjacent or close to the block to be decoded, the inter prediction unit 205 may generate a control point motion vector of the block to be decoded based on them and add it to the merge candidate list for affine merge.

[0318] Therefore, in the affine merge mode, the inter prediction unit 205 may add the motion vectors derived by recursively searching for motion vectors with respect to at least two or more control point motion vectors stored in the merge candidate list to each of the first control point motion vectors to generate a chained control point motion vector and store it in the merge candidate list.

[0319] Furthermore, in the affine merge mode, when recursively searching for motion vectors with respect to at least two or more control point motion vectors stored in the merge candidate list, the inter prediction unit 205 may limit the reference pictures to be searched for each control point motion vector to the same reference picture.

[0320] Alternatively, in the affine merge mode, when recursively searching for motion vectors with respect to at least two or more control point motion vectors of L0 and L1 stored in the merge candidate list, the inter prediction unit 205 may limit the reference pictures to be searched for each control point motion vector of L0 and L1 to the same reference picture.

[0321] Here, for the chained control point motion vector generated for affine merge, the inter prediction unit 205 may apply pruning processing at the time of storing in the merge candidate list, correction of the motion vector, and rearrangement of the merge candidates (motion vectors) in the same manner as normal merge.

[0322] According to the image decoding apparatus according to the present embodiment, since the inter prediction unit 205 adds all the motion vectors derived by recursively searching for the motion vectors stored in the decoded picture buffer 207 for the motion vectors at positions spatially or temporally adjacent or close to the block to be decoded, and derives the generated motion vector as the motion vector of the block to be decoded, the options for selecting motion vectors that can be selected in inter prediction increase, the inter prediction accuracy improves, and the coding efficiency is improved.

[0323] The above-described image decoding apparatus 200 may be realized by a program that causes a computer to execute each function (each process).

Industrial Applicability

[0324] Note that according to the present embodiment, for example, since it is possible to realize an improvement in overall service quality in moving image communication, it is possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Build resilient infrastructure, promote sustainable industrialization and foster innovation."

Explanation of Signs

[0325] 200…Image decoding apparatus 201…Decoder 202…Inverse quantization unit 203…Inverse transformation unit 204…Intra prediction unit 205…Inter prediction unit 206…Adder 207…Decoded picture buffer 210…Code input unit 220…Image output unit

Claims

1. An image decoding device, an inter prediction unit that generates inter predicted pixels from control information and decoded pixels stored in a decoded picture buffer; the inter prediction unit performs a daisy chain motion vector prediction in which the inter prediction unit generates a daisy chain motion vector by recursively searching for motion vectors stored in the decoding picture buffer for motion vectors at positions spatially or temporally adjacent or close to the block to be decoded, which is stored in a merge candidate list storing motion vector candidates, and adding all of the derived motion vectors, and derives the daisy chain motion vectors as motion vector candidates for the block to be decoded; The image decoding device is characterized in that the inter prediction unit controls whether to inherit a predetermined internal parameter that changes the method of generating the inter prediction pixel when deriving merge candidates using the daisy chain motion vector prediction.

2. The image decoding device described in claim 1, characterized in that, when deriving merge candidates using the daisy chain motion vector prediction, the inter prediction unit does not inherit a first internal parameter that specifies whether or not to apply a switching interpolation filter associated with each motion vector searched recursively, but always sets the first internal parameter to indicate that a switching interpolation filter is not to be applied.

3. The image decoding device described in claim 1, characterized in that, when deriving merge candidates using the daisy chain motion vector prediction, the inter prediction unit inherits, as the first internal parameter corresponding to the merge candidate, a first internal parameter associated with the first motion vector that constitutes the daisy chain motion vector, among first internal parameters that specify whether or not to apply a switching interpolation filter associated with each motion vector recursively searched.

4. 4. The image decoding device according to claim 3, wherein, when deriving the merging candidates, if at least one of first internal parameters that specify whether or not to apply a switch interpolation filter associated with each motion vector recursively searched for indicates that a switch interpolation filter is to be applied, the inter prediction unit determines that a first internal parameter corresponding to the merging candidate indicates that a switch interpolation filter is to be applied.

5. The image decoding device described in claim 1, characterized in that when the daisy chain motion vector derived by the daisy chain motion vector prediction is composed of at least one block vector, the inter prediction unit always sets a first internal parameter that specifies whether or not to apply a switching interpolation filter corresponding to a merge candidate derived by the daisy chain motion vector prediction to indicate that a switching interpolation filter is not to be applied.

6. The image decoding device described in claim 1, characterized in that when deriving merge candidates using the daisy chain motion vector prediction, the inter prediction unit does not inherit a second internal parameter that controls whether or not to apply local luminance compensation associated with each recursively searched motion vector, but always sets the second internal parameter corresponding to the merge candidate to indicate that local luminance compensation is not applied.

7. The image decoding device described in claim 1, characterized in that when deriving a merge candidate using the daisy chain motion vector prediction, the inter prediction unit inherits, as the second internal parameter corresponding to the merge candidate, a second internal parameter associated with the first motion vector that constitutes the daisy chain motion vector, among second internal parameters that control whether or not to apply local luminance compensation associated with each motion vector recursively searched.

8. 8. The image decoding device of claim 7, wherein, when deriving the merging candidates, if at least one of second internal parameters (second internal parameters that control whether to apply local luminance compensation associated with each motion vector recursively searched) indicates that local luminance compensation is to be applied, the inter prediction unit determines that the second internal parameters corresponding to the merging candidates indicate that local luminance compensation is to be applied.

9. The image decoding device described in claim 1, characterized in that when the daisy chain motion vector derived by the daisy chain motion vector prediction is composed of at least one or more block vectors, the inter prediction unit always sets a second internal parameter that controls whether or not to apply local luminance compensation corresponding to the merge candidate derived by the daisy chain motion vector prediction to indicate that local luminance compensation is not applied.

10. The image decoding device described in claim 1, characterized in that, when deriving merge candidates using the daisy chain motion vector prediction, the inter prediction unit does not inherit a third internal parameter that specifies a weight value for block-based weighted bi-prediction associated with each motion vector recursively searched, but always sets the third internal parameter corresponding to the merge candidate to indicate that the weight value for block-based weighted bi-prediction is a simple average.

11. The image decoding device described in claim 1, characterized in that, when deriving merge candidates using the daisy chain motion vector prediction, the inter prediction unit inherits, as the third internal parameter corresponding to the merge candidate, a third internal parameter associated with the first motion vector that constitutes the daisy chain motion vector, from among third internal parameters that specify weight values ​​for block-based weighted bi-prediction associated with each motion vector recursively searched.

12. The image decoding device described in claim 1, characterized in that, when deriving merge candidates using the daisy chain motion vector prediction, the inter prediction unit does not inherit a fourth internal parameter that controls whether or not to apply multiple hypothesis prediction associated with each recursively searched motion vector, but always sets the fourth internal parameter corresponding to the merge candidate to indicate that multiple hypothesis prediction is not applied.

13. The image decoding device described in claim 1, characterized in that when deriving a merge candidate using the daisy chain motion vector prediction, the inter prediction unit inherits, as the fourth internal parameter corresponding to the merge candidate, a fourth internal parameter associated with the first motion vector that constitutes the daisy chain motion vector, among the fourth internal parameters that control whether or not to apply multiple hypothesis prediction associated with each motion vector recursively searched.

14. 14. The image decoding device according to claim 13, wherein, when deriving merge candidates by the daisy-chain motion vector prediction, if at least one of fourth internal parameters that control whether or not to apply multiple hypothesis prediction associated with each motion vector recursively searched indicates that multiple hypothesis prediction is to be applied, the inter prediction unit determines that a fourth internal parameter corresponding to the merge candidate indicates that multiple hypothesis prediction is to be applied.

15. 2. The image decoding device according to claim 1, wherein, when the daisy chain motion vector generated by the daisy chain motion vector prediction is composed of at least one block vector, the inter prediction unit always sets a fourth internal parameter that controls whether or not to apply multiple hypothesis prediction corresponding to the merge candidate derived by the daisy chain motion vector prediction to indicate that multiple hypothesis prediction is not applied.

16. An image decoding method, comprising: generating inter-predicted pixels from the control information and the stored decoded pixels; generating the inter-predicted pixel includes performing a chain motion vector prediction in which a chain motion vector is generated by recursively searching for motion vectors stored in a decoding picture buffer for motion vectors at positions spatially or temporally adjacent or close to the block to be decoded, which is stored in a merge candidate list storing motion vector candidates, and adding all of the derived motion vectors, and deriving the chain motion vectors as candidates for the motion vector of the block to be decoded; When deriving merge candidates using the chained motion vector prediction, the image decoding method further comprises controlling whether or not to inherit a predetermined internal parameter that changes the method of generating the inter-predicted pixels.

17. A program that causes a computer to function as an image decoding device, The image decoding device comprises: an inter prediction unit that generates inter predicted pixels from control information and accumulated decoded pixels; the inter prediction unit performs a daisy chain motion vector prediction in which a daisy chain motion vector is generated by recursively searching for motion vectors stored in a decoding picture buffer for motion vectors at positions spatially or temporally adjacent or close to the block to be decoded, which is stored in a merge candidate list storing motion vector candidates, and adding all of the derived motion vectors, and deriving the daisy chain motion vectors as motion vector candidates for the block to be decoded; The program is characterized in that the inter prediction unit controls whether to inherit a predetermined internal parameter that changes the method of generating the inter prediction pixel when deriving merge candidates using the daisy chain motion vector prediction.

Images