Image decoding apparatus, image decoding method, and program

Abstract

In the image decoding device (200) according to the present application, the inter prediction unit (205) performs chain motion vector prediction, and generates a chain motion vector by adding all motion vectors derived by re-searching new motion vectors in a manner that the reference target is a motion vector stored in the collocated picture in the decoding picture buffer (207) from among the motion vectors stored in the merge candidate list of the stored motion vector candidates, which are limited only to the positions spatially or temporally adjacent or close to the decoding target block, and derives the chain motion vector as a candidate of the motion vector of the decoding target block.

Description

Technical Field

[0001] This invention relates to an image decoding apparatus, an image decoding method, and a program. Background Technology

[0002] Inter-frame prediction is disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0003] Inter-frame prediction generates predicted pixels for the decoded object block from decoded pixels (reference pixels) within a decoded image (reference image) that is different from the decoded object image.

[0004] In addition, inter-frame prediction uses control information to select the motion vector of the decoded object block needed to generate the predicted pixels of the decoded object block from a pool of candidates of motion vectors that are spatially or temporally adjacent to or close to the decoded object block. Existing technical documents Non-patent literature

[0005] Non-patent literature 1: ITU-T H.266 / VVC Non-Patent Literature 2: M. Coban et al., Algorithm Description of Enhanced Compression Model 10 (ECM 10), JVET-AE2025, 2023 Summary of the Invention The problem that the invention aims to solve

[0006] In Non-Patent Literature 1 and Non-Patent Literature 2, the candidates for motion vectors that can be selected in inter-frame prediction are limited to motion vectors that are spatially or temporally adjacent to or close to the block to be decoded, thus leaving room for improvement in coding efficiency.

[0007] Therefore, the present invention was made in view of the above-mentioned problems, and its object is to provide an image decoding device, image decoding method and program with high encoding efficiency. Solution for solving the problem

[0008] The first feature of this invention is an image decoding apparatus, comprising: a decoding unit that performs variable-length decoding on encoded information and outputs quantized values ​​and control information; an inverse quantization unit that performs inverse quantization on the aforementioned quantized values ​​and outputs transform coefficients; an inverse transform unit that performs inverse transform on the aforementioned transform coefficients and outputs predicted residual pixels; an intra-frame prediction unit that generates intra-frame predicted pixels from the aforementioned control information and decoded pixels; a decoded image buffer that stores the aforementioned decoded pixels; an inter-frame prediction unit that generates inter-frame predicted pixels from the aforementioned control information and the aforementioned decoded pixels stored in the aforementioned decoded image buffer; and an adder that performs an addition operation on the aforementioned intra-frame predicted pixels. The aforementioned decoded pixel is generated by adding at least one of the aforementioned inter-frame predicted pixels and the aforementioned predicted residual pixels. The aforementioned inter-frame prediction unit performs chained motion vector prediction. The aforementioned chained motion vector prediction generates a chained motion vector by adding all motion vectors derived from motion vectors that are spatially or temporally adjacent or close to the decoded object block in a merged candidate list of stored motion vector candidates, with the reference target being the juxtaposed image stored in the aforementioned decoded image buffer. The aforementioned chained motion vector is then derived as a candidate motion vector for the aforementioned decoded object block.

[0009] The second feature of this invention is an image decoding method, comprising: step A, performing variable-length decoding on encoded information and outputting quantized values ​​and control information; step B, performing inverse quantization on the aforementioned quantized values ​​and outputting transform coefficients; step C, performing inverse transform on the aforementioned transform coefficients and outputting predicted residual pixels; step D, generating intra-frame predicted pixels from the aforementioned control information and decoded pixels; step E, storing the aforementioned decoded pixels in a decoded image buffer; step F, generating inter-frame predicted pixels from the aforementioned control information and the aforementioned decoded pixels stored in the aforementioned decoded image buffer; and step F, storing the aforementioned intra-frame predicted pixels and the aforementioned... In step G, at least one of the inter-frame predicted pixels is added to the aforementioned predicted residual pixel to generate the aforementioned decoded pixel. In step F, chained motion vector prediction is performed. The chained motion vector prediction generates a chained motion vector by adding all motion vectors derived from motion vectors that are spatially or temporally adjacent or close to the decoded object block in a merged candidate list of stored motion vector candidates, with the reference target being the juxtaposed image stored in the aforementioned decoded image buffer. The chained motion vector is then derived as a candidate motion vector for the aforementioned decoded object block.

[0010] The third feature of this invention is a program that enables a computer to function as an image decoding device, the image decoding device comprising: a decoding unit that performs variable-length decoding on encoded information and outputs quantized values ​​and control information; an inverse quantization unit that performs inverse quantization on the quantized values ​​and outputs transform coefficients; an inverse transform unit that performs inverse transform on the transform coefficients and outputs predicted residual pixels; an intra-frame prediction unit that generates intra-frame predicted pixels from the control information and decoded pixels; a decoded image buffer that stores the decoded pixels; an inter-frame prediction unit that generates inter-frame predicted pixels from the control information and the decoded pixels stored in the decoded image buffer; and an addition operation. The calculator adds at least one of the aforementioned intra-frame predicted pixels and the aforementioned inter-frame predicted pixels to the aforementioned prediction residual pixels to generate the aforementioned decoded pixels. The aforementioned inter-frame prediction unit performs chained motion vector prediction. The aforementioned chained motion vector prediction generates chained motion vectors by adding all motion vectors derived from motion vectors that are spatially or temporally adjacent or close to the decoded object block in a merged candidate list of stored motion vector candidates, with the reference target being the juxtaposed image stored in the aforementioned decoded image buffer. The aforementioned chained motion vectors are then derived as candidates for motion vectors of the aforementioned decoded object block. Invention Effects

[0011] According to the present invention, an image decoding apparatus, an image decoding method, and a program with high encoding efficiency can be provided. Attached Figure Description

[0012] Figure 1 This is a diagram illustrating an example of the functional blocks of an image decoding apparatus 200 according to one embodiment. Figure 2 This is a diagram illustrating the concept of inter-frame prediction. Figure 3 This is a diagram illustrating the reference pixel values ​​referenced when a future decoded object block references a decoded object image stored in the decoded image buffer 207, as disclosed in Non-Patent Document 1 and Non-Patent Document 2. Figure 4 It is generated in Figure 3 The conceptual diagram of the inter-frame prediction padding region is explained in the text. Figure 5 This is a diagram showing an example of a merged candidate list as a method for deriving motion vectors in the inter-frame prediction unit 205. Figure 6 This is a diagram illustrating an example of the names of various merger candidates and the order in which these merger candidates are derived in a regular merge disclosed in Non-Patent Document 1 and Non-Patent Document 2. Figure 7This is a diagram illustrating an example of the search positions for spatial merging candidates and TMVP candidates within a decoded object block. Figure 8 This is a diagram illustrating an example of a method for deriving chain motion vectors in chain motion vector prediction. Figure 9 This is a diagram illustrating an example of a method for deriving chain motion vectors in chain motion vector prediction. Figure 10 This is a diagram illustrating an example of a recursive search that includes bidirectional prediction of motion vectors. Figure 11 This is a diagram illustrating an example of a chain of motion vectors derived through a recursive search that includes bidirectional prediction of motion vectors. Figure 12 This is a diagram illustrating an example, as disclosed in Non-Patent Document 1 and Non-Patent Document 2, of searching for chain motion vector prediction candidates after searching for multiple merger candidates in a conventional merging process. Figure 13 This is a diagram illustrating an example, as disclosed in Non-Patent Document 1 and Non-Patent Document 2, of searching for chain motion vector prediction candidates after searching for multiple merger candidates in a conventional merging process. Figure 14 This is a diagram illustrating an example of adding merged candidates as derived objects of chained motion vector prediction candidates, and the position (index number) within the merged candidate list of chained motion vector prediction candidates. Figure 15 This is a diagram illustrating an example of adding merged candidates as derived objects of chained motion vector prediction candidates, and the position (index number) within the merged candidate list of chained motion vector prediction candidates. Figure 16 This is a diagram illustrating an example of adding merged candidates as derived objects of chained motion vector prediction candidates, and the position (index number) within the merged candidate list of chained motion vector prediction candidates. Figure 17 This is a diagram illustrating an example of adding merged candidates as derived objects of chained motion vector prediction candidates, and the position (index number) within the merged candidate list of chained motion vector prediction candidates. Figure 18 This is a diagram illustrating an example of a pruning process in which the inter-frame prediction unit 205 of an image decoding apparatus 200 according to one embodiment adds merging candidates corresponding to chain motion vectors derived by chain motion vector prediction to the merging candidate list. Figure 19 This is a diagram illustrating an example of a method for controlling the generation of a reference source when generating a chain motion vector in the chain motion vector prediction of the inter-frame prediction unit 205 of an image decoding apparatus 200 according to one embodiment. Figure 20This is a diagram illustrating an example of a method for controlling the generation of a reference target in the chain motion vector prediction of the inter-frame prediction unit 205 of an image decoding apparatus 200 according to one embodiment. Figure 21 This diagram illustrates an example of a method for controlling the reference source and reference target when generating chain motion vectors in the chain motion vector prediction of the inter-frame prediction unit 205 of an image decoding apparatus 200 according to an embodiment, in the case of applying GDR. Figure 22 This is a diagram illustrating an example of a method for controlling the reference source when deriving chain motion vectors in chain motion vector prediction. Figure 23 This is a diagram illustrating an example of how the decoding unit 201 of an image decoding apparatus 200 according to an embodiment determines whether there is decoding based on sps_temporal_mvp_enabled_flag. Figure 24 This is a diagram illustrating an example of a control point motion vector cpMv derived by the inter-frame prediction unit 205 of an image decoding apparatus 200 according to one embodiment. Figure 25 This is a diagram illustrating an example of a motion vector prediction derivation method in the inter-frame prediction unit 205 of an image decoding apparatus 200 according to one embodiment. Detailed Implementation

[0013] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Furthermore, the constituent elements in the following embodiments can be appropriately replaced with existing constituent elements, and various modifications can be made, including combinations with other existing constituent elements. Therefore, the description of the following embodiments does not limit the scope of the invention as described in the claims.

[0014] <First Implementation Method> The following is for reference Figures 1 to 24 The image decoding apparatus 200 according to this embodiment will be described.

[0015] The image decoding apparatus 200 according to the first embodiment of the present invention targets various image signals (hereinafter referred to as images). For example, the image decoding apparatus 200 according to this embodiment targets YUV (YCbCr) images including luminance pixels and chrominance pixels, RGB images including RGB pixels, and monochrome images. Here, the pixels constituting each image have discrete values ​​(pixel values) with a predetermined bit width.

[0016] Figure 1This is a diagram illustrating an example of the functional blocks of the image decoding apparatus 200 according to this embodiment.

[0017] like Figure 1 As shown, the image decoding device 200 includes an encoding input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transform unit 203, an intra-frame prediction unit 204, an inter-frame prediction unit 205, an adder 206, a decoded image buffer 207, and an image output unit 220.

[0018] In addition, the parts described as "pixels" in the functional descriptions of each part below can be blocks (units) composed of pixels, tree blocks that are the largest size of the blocks, strips larger than tree blocks, sheets, or images (pictures).

[0019] In addition, although in Figure 1 Although not shown in the diagram, the image decoding device 200 may have an in-loop filter between the adder 206 and the decoded image buffer 207 to correct the pixel values ​​of the decoded pixels.

[0020] The encoding input unit 210 is configured to acquire encoding information encoded by the image encoding device.

[0021] The decoding unit 201 is configured to decode control information and quantization values ​​from encoded information input from the encoding input unit 210. For example, the decoding unit 201 is configured to output control information and quantization values ​​by performing variable-length decoding on the encoded information.

[0022] Here, the quantization value is sent to the inverse quantization unit 202, and control information is sent to the decoded image buffer 207, the intra-frame prediction unit 204, and the inter-frame prediction unit 205. Furthermore, this control information (syntax) may include information required for controlling the decoded image buffer 207, the intra-frame prediction unit 204, and the inter-frame prediction unit 205, including header information such as sequence parameter sets, image parameter sets, image headers, and stripe headers.

[0023] The inverse quantization unit 202 is configured to inverse quantize the quantized value sent from the decoding unit 201 and use it as the transform coefficients to be decoded. These transform coefficients are then sent to the inverse transform unit 203.

[0024] The inverse transform unit 203 is configured to perform an inverse transform on the transform coefficients sent from the inverse quantization unit 202 and use them as the decoded prediction residual. This prediction residual is then sent to the adder 206.

[0025] The intra-prediction unit 204 is configured to generate intra-predictive pixels based on the decoded pixels and control information sent from the decoding unit 201. Here, the decoded pixels are pixels obtained via the adder 206 and stored in the decoded image buffer 207. Furthermore, the intra-predictive pixels are prediction pixels used to add to the prediction residual in the adder 206. Additionally, the intra-predictive pixels are sent to the adder 206.

[0026] Here, the prediction performed by the intra-prediction unit 204 may be the intra-prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2. Alternatively, the prediction performed by the intra-prediction unit 204 may be the intra-block copying or intra-template matching prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0027] The decoded image buffer 207 cumulatively stores inter-frame prediction information corresponding to the decoded pixels sent from the adder 206 and the decoded pixels sent from the inter-frame prediction unit 205.

[0028] Here, the inter-frame prediction unit 205 references the decoded pixels stored in the decoded image buffer 207 and the inter-frame prediction information corresponding to those decoded pixels. Details regarding the inter-frame prediction information will be described later.

[0029] The inter-frame prediction unit 205 is configured to generate inter-frame prediction pixels based on the decoded pixels obtained from the reference decoded image buffer 207, inter-frame prediction information, and control information sent from the decoding unit 201.

[0030] Furthermore, the inter-frame prediction unit 205 sends the generated inter-frame prediction pixel to the addition unit 206, and sends the inter-frame prediction information used to generate the inter-frame prediction pixel as inter-frame prediction information corresponding to the decoded object pixel to the decoded image buffer 207.

[0031] Here, the prediction performed by the inter-frame prediction unit 205 may be the inter-frame prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2. Alternatively, the prediction performed by the inter-frame prediction unit 205 may be the intra-frame block copying disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0032] The adder 206 is configured to add the prediction residual sent from the inverse transform unit 203 to at least one of the input intra-frame prediction pixels and inter-frame prediction pixels to calculate the decoded pixel. The decoded pixel is sent to the image output unit 220, the decoded image buffer 207, and the intra-frame prediction unit 204.

[0033] The image output unit 220 is configured to output the decoded pixels sent from the adder 206.

[0034] Hereinafter, taking the prediction performed by the inter-frame prediction unit 205 as an example, the function of the inter-frame prediction unit 205 and the decoded image buffer 207, which are characteristic configurations of the image decoding apparatus 200 according to this embodiment, will be explained.

[0035] <Basic functions of inter-frame prediction unit 205 and decoded image buffer 207> The basic function of the inter-frame prediction unit 205 is to derive one or more motion vectors (Mv) for the decoded object block in order to predict the decoded object block with high accuracy in the subsequent adder 206, and to predict the pixels (hereinafter referred to as reference pixels) of the decoded block (hereinafter referred to as reference block) referenced by Mv in a decoded image (hereinafter referred to as reference image) that is different from the decoded object image to which the decoded object block belongs. That is, to perform inter-frame prediction.

[0036] exist Figure 2 The diagram below illustrates the concept of inter-frame prediction. Specifically, Figure 2 The following example illustrates how, in inter-frame prediction, to generate inter-frame predicted pixels for a decoded object block (CurBlk) located within the decoded object image (CurPic), the MvL0 of two motion vectors derived by the inter-frame prediction unit 205 is used. k and MvL1 k To refer to two different reference images (RerPicL0) k and RerPicL1 k Reference block (RerBlkL0) within ) k and RerBlkL1 k ).

[0037] Here, L0 and L1 indicate the numbers of a list of reference images that can be referenced by the decoded object image (reference image list), and k corresponds to the value of the control information (index) used to determine the motion vectors described in the method for deriving motion vectors described later.

[0038] As in Non-Patent Document 1 and Non-Patent Document 2, the image decoding apparatus 200 of this embodiment can export list 0 (L0) and list 1 (L1) for each decoded object image as two different reference image lists.

[0039] Here, the decoded image buffer 207 can export a list of reference images for each decoded object image based on the control information decoded by the decoding unit 201.

[0040] For example, control information may include the number of reference images that should be included in the list of reference images, and the candidate reference images (image reference structure) for the image to be decoded.

[0041] The detailed method for deriving motion vectors in the inter-frame prediction unit 205 will be described later. An overview of the method for generating inter-frame prediction pixels in the inter-frame prediction unit 205 is as follows.

[0042] The inter-frame prediction unit 205 can generate inter-frame prediction pixels from a single reference block, or it can... Figure 2 The diagram shows the generation of inter-frame predicted pixels using two reference blocks, but more than two reference blocks can also be used to generate inter-frame predicted pixels.

[0043] Here, the inter-frame prediction unit 205 can apply the following method as a way to generate inter-frame prediction pixels using two reference blocks.

[0044] As an example, the inter-frame prediction unit 205 can simply average the pixel values ​​of two reference blocks to generate inter-frame prediction pixels.

[0045] As another example, the inter-frame prediction unit 205 can use predetermined weight values ​​to generate inter-frame predicted pixels by weighting the pixel values ​​of two reference blocks on a block-by-block basis.

[0046] The inter-frame prediction unit 205 can be applied to the block-weighted bi-directional prediction (BCW: Bi-prediction with CU-level Weights) disclosed in Non-Patent Document 1 and Non-Patent Document 2 as a block-weighted averaging technique.

[0047] Furthermore, in non-patent documents 1 and 2, in BCW, it is possible to select multiple weight values, including a simple average weight value of 1:1, which is determined by the value of the internal parameter (bcwIdx, the third internal parameter).

[0048] Here, the inter-frame prediction unit 205 derives bcwIdx through control information or inheritance from the decoded image buffer 207. Details of inheritance will be described later.

[0049] As another example, the inter-frame prediction unit 205 can divide the decoded target block into two parts using a predetermined straight line, and use a weighted average of the pixel values ​​of the reference blocks corresponding to each of the two parts to generate inter-frame prediction pixels. The aforementioned weight value corresponds to the distance from the dividing straight line.

[0050] The inter-frame prediction unit 205 can be applied to the geometric partitioning mode (GPM) disclosed in Non-Patent Document 1 and Non-Patent Document 2, as a weighted average technique for each region that divides the block into two parts using a straight line.

[0051] Here, in the case of a fractional pixel precision position within the motion vector reference reference block, the inter-frame prediction unit 205 can use an interpolation filter to generate inter-frame prediction pixels. The aforementioned interpolation filter is used to derive the reference pixel value at the fractional pixel precision position from the reference pixel value at the integer pixel precision position in the up, down, left, and right directions located at the fractional pixel position.

[0052] Furthermore, when generating inter-frame prediction pixels, the inter-frame prediction unit 205 can be adaptively applied to two interpolation filters with different cutoff frequencies for inter-frame prediction disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0053] The interpolation filter with the lower cutoff frequency among the two interpolation filters is called the switchable interpolation filter (SIF).

[0054] Similar to Non-Patent Document 1 and Non-Patent Document 2, when the motion vector references a half-pixel precision position and it is determined whether there is an internal parameter (hpelIfIdx, first internal parameter) indicating the application of SIF, the inter-frame prediction unit 205 can apply SIF.

[0055] Furthermore, the inter-frame prediction unit 205 derives hpelIfIdx through control information or inheritance from the decoded image buffer 207. Details regarding inheritance will be described later.

[0056] On the other hand, when the motion vector reference is at a fractional pixel precision position, the inter-frame prediction unit 205 can round (cut) the reference target of the motion vector to an integer pixel precision position that is closest to that fractional pixel precision position, and generate inter-frame prediction pixels without applying an interpolation filter.

[0057] In addition, the inter-frame prediction unit 205 may apply the local illumination compensation (LIC) disclosed in Non-Patent Document 2 when generating the final inter-frame prediction pixel.

[0058] LIC is a technique that uses linear prediction to correct the reference pixel values ​​in the reference block based on the decoded pixels that are adjacent to the decoded object block and the reference block, respectively.

[0059] Similar to Non-Patent Document 1 and Non-Patent Document 2, when the internal parameter (licFlag, second internal parameter) indicating whether or not LIC is applied, the inter-frame prediction unit 205 can apply LIC.

[0060] Furthermore, the inter-frame prediction unit 205 derives the licFlag through control information or inheritance from the decoded image buffer 207. Details regarding inheritance will be described later.

[0061] In addition, the inter-frame prediction unit 205 may apply the multi-hypothesis prediction (MHP) disclosed in Non-Patent Document 2 when generating the final inter-frame prediction pixels.

[0062] Specifically, the inter-frame prediction unit 205 can use at least one or more chained motion vectors that are ultimately derived and a motion vector that serves as the source (reference source) of the derived chained motion vectors to generate the final inter-frame prediction pixels.

[0063] Here, as a scheme for generating more than three inter-frame prediction pixels, Non-Patent Document 2 discloses Multi-Hypothesis Prediction (MHP).

[0064] Based on the MHP algorithm, the inter-frame prediction unit 205 can use at least one chained motion vector derived above and a motion vector that serves as the source (reference source) of the chained motion vector to generate the final inter-frame predicted pixels. The specific steps are described below.

[0065] First, in the merged candidate list, in addition to one or more chained motion vectors, the inter-frame prediction unit 205 will also store a motion vector that serves as the derived source (reference source) of the chained motion vector as an MHP-oriented motion vector.

[0066] Next, the decoded image buffer 207 stores the MHP motion vector as a third motion vector (motion information).

[0067] Next, in the generation of the final inter-frame predicted pixel via MHP, the inter-frame prediction unit 205 can synthesize (weighted average) and generate the final inter-frame predicted pixel value by using up to two motion vectors corresponding to the merge candidates stored in the merge candidate list and the aforementioned MHP motion vector, as follows.

[0068] P Final =(1-α)P Merge +αP MHP (Formula 0) Here, P Final P Merge P MHP α and α are the final inter-frame predicted pixel, the reference pixel of the motion vector corresponding to the merged candidate, the reference pixel of the MHP motion vector, and the MHP weight coefficient, respectively.

[0069] The MHP weighting coefficient can be a constant. For example, 1 / 4 or -1 / 8, as disclosed in Non-Patent Document 2, can be fixedly assigned to the MHP weighting coefficient. Alternatively, the decoding unit 201 can decode or estimate the control information that determines the value of the MHP weighting coefficient and derive the MHP weighting coefficient.

[0070] MHP is a technique that generates the final inter-frame predicted pixels by weighted averaging of reference pixel values ​​from at least three reference blocks.

[0071] Similar to Non-Patent Document 2, when it is determined whether there is an internal parameter (mhpFlag, fourth internal parameter) indicating the application of MHP, the inter-frame prediction unit 205 can apply MHP.

[0072] Furthermore, the inter-frame prediction unit 205 derives mhpFlag through control information or inheritance from the decoded image buffer 207. Details regarding inheritance will be described later.

[0073] The inter-frame prediction unit 205 sends inter-frame prediction pixels for the decoded target block to the adder 206, and sends to the decoded image buffer 207 a motion vector for generating the inter-frame prediction pixel, an index of the reference image used to determine the motion vector reference within the reference image list (reference index), and a counter (POC: Picture Order Count) indicating the image output order of the reference image corresponding to the reference index. Hereinafter, the motion vector used to generate these inter-frame prediction pixels, the reference index, and the POC corresponding to the reference index will be referred to as inter-frame prediction information. Furthermore, the inter-frame prediction unit 205 may include at least one or all of bcwIdx, hpelIfIdx, licFlag, or mhpFlag in the inter-frame prediction information and send it, where bcwIdx, hpelIfIdx, licFlag, or mhpFlag are internal parameters that modify the aforementioned method for generating the inter-frame prediction pixels.

[0074] In addition to the decoded pixels of the decoded object block sent from the adder 206, the decoded image buffer 207 also stores inter-frame prediction information of the decoded object block corresponding to the decoded pixels.

[0075] The decoding image buffer 207 can store the inter-frame prediction information in a predetermined size (number of pixels). For example, the decoding image buffer 207 can store the inter-frame prediction information in units of the minimum size (4×4 pixels) of the decoding object block as disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0076] The decoded image buffer 207 can improve the inter-frame prediction accuracy when the inter-frame prediction information is referenced by the decoded object block to be decoded in the future by storing the inter-frame prediction information at the minimum size of the decoded object block.

[0077] The decoded image buffer 207 stores decoded pixels in units of the decoded object image. Alternatively, the decoded image buffer 207 may store decoded pixels in units of decoded object stripes. The following explanation will use the case of storing decoded pixels in units of the decoded object image as an example.

[0078] If it is determined that the stored decoded pixels, per unit of the decoded object image, and the inter-frame prediction information corresponding to those decoded pixels are not referenced as reference images or reference blocks by future decoded object images or decoded object blocks, the decoded image buffer 207 can sequentially delete the decoded pixels, per unit of the decoded object image, and the inter-frame prediction information corresponding to those decoded pixels. According to this configuration, the amount of information stored in the decoded object image buffer 207 can be reduced.

[0079] The decoded image buffer 207 can export the number of stored decoded object images and the reference image structure between the decoded object images based on the control information decoded by the decoding unit 201.

[0080] As a variation, if the inter-frame predicted pixel exists in a predetermined area outside the decoded object image in addition to the decoded object image, the decoded image buffer 207 may additionally store the inter-frame predicted pixel.

[0081] Figure 3 This is a diagram illustrating the reference pixel values ​​referenced when a future decoded object block references a decoded object image stored in the decoded image buffer 207, as disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0082] like Figure 3 As shown in Non-Patent Document 1 (VVC: Versatile Video Coding), when a future decoded object block is referenced within a predetermined range outside the decoded object image, the pixel value located at the nearest decoded object image boundary is copied to generate a reference pixel value.

[0083] In VVC, the range in which 16 pixels are added to the maximum size (number of pixels) of the width of the decoded object block is specified as the specified range (fill area).

[0084] In non-patent literature 2 (ECM: Enhanced Compression Model), such as Figure 3As shown, between the outside of the decoded object image and the padding area, only a predetermined range is stored as the area (inter-frame prediction padding area) that can be referenced by future decoded object blocks to the inter-frame predicted pixel values ​​of the decoded object block.

[0085] Therefore, even if the decoded object image is referenced by future decoded object blocks, the inter-frame prediction pixel values ​​of the decoded object blocks still exist, thus improving the inter-frame prediction accuracy of future decoded object blocks.

[0086] Figure 4 It is generated in Figure 3 The conceptual diagram of the inter-frame prediction padding region is explained in the text.

[0087] like Figure 4 As shown, when the adjacent blocks of the reference block corresponding to the decoded object block have inter-frame prediction pixels, even if the decoded object block is adjacent to the decoded object image, inter-frame prediction pixels can be generated outside the decoded object image. Therefore, the decoded image buffer 207 can store... Figure 3 The inter-frame prediction padding region is shown.

[0088] <Methods for deriving motion vectors in the inter-frame prediction unit> The following is for reference Figures 5 to 17 The method for deriving motion vectors in the inter-frame prediction unit 205 of the image decoding apparatus 200 according to this embodiment will be described.

[0089] Figure 5 This is a diagram showing an example of a merged candidate list as a method for deriving motion vectors in the inter-frame prediction unit 205.

[0090] The merge candidate list is a list of candidates (i.e. merge candidates) disclosed in Non-Patent Document 1 and Non-Patent Document 2, which stores inter-frame prediction information constructed by the inter-frame prediction unit 205 for deriving motion vectors in merge (merging mode).

[0091] The inter-frame prediction unit 205 extracts merge candidates from multiple merge candidates located in the merge candidate list based on the index (merge_idx) sent from the decoding unit 201 for determining merge candidates in the merge candidate list, and uses them as inter-frame prediction information for generating inter-frame prediction pixels.

[0092] The upper limit of the number of merging candidates stored in the merging candidate list can be a fixed value or a variable value. In the case of a variable value, for example, as in Non-Patent Document 1 and Non-Patent Document 2, control information can be set in the sequence parameter set, etc., and the variable value can be configured based on the control information.

[0093] The inter-frame prediction unit 205 determines the inter-frame prediction information for generating inter-frame prediction pixels from the merge candidate list based on one or more merge_idx values ​​sent from the decoding unit 201.

[0094] For example, for the aforementioned GPM, the inter-frame prediction unit 205 can derive inter-frame prediction information by decoding merge_idx one by one for each region that divides the decoded object block into two parts using a straight line.

[0095] like Figure 5 As shown, the inter-frame prediction unit 205 stores inter-frame prediction information (merging candidates) corresponding to L0 and L1 of the merging candidate list.

[0096] In addition, Figure 5 In the example, although only motion vectors and reference indices are recorded as inter-frame prediction information, in addition to this, the inter-frame prediction unit 205 may also store at least one or more of bcwIdx, hpelIfIdx, licFlag or mhpFlag, which are internal parameters of the method for changing the generation of the aforementioned inter-frame prediction pixels, as inter-frame prediction information.

[0097] The inter-frame prediction unit 205 can search for inter-frame prediction information, that is, inter-frame prediction information that is spatially or temporally adjacent to or close to the decoded object block, and add it to the merging candidate list as a merging candidate.

[0098] As an example, the inter-frame prediction unit 205 can combine and apply the search of multiple merged candidates disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0099] Figure 6 The names of several merger candidates in regular merges disclosed in Non-Patent Document 1 and Non-Patent Document 2 are shown, along with the order in which these merger candidates are derived.

[0100] Specifically, in Non-Patent Document 1 (VVC), the inter-frame prediction unit 205 confirms whether or not these merging candidates have been used, in the order of Spatialmerge candidates, Temporal Motion Vector Prediction (TMVP) candidates, History-based Motion Vector Prediction (HMVP) candidates, and Pairwise merge candidates.

[0101] In Non-Patent Document 2 (ECM), in addition to the aforementioned merge candidates in Non-Patent Document 1, the inter-frame prediction unit 205 also confirms whether there are non-adjacent (NA) spatial merge candidates among the spatial merge candidates and TMVP candidates.

[0102] The following is a summary of each merger candidate.

[0103] Figure 7 This is a diagram showing the search positions of spatial merging candidates and TMVP candidates in the decoded object block.

[0104] like Figure 7 As shown, in the spatial merging candidate, inter-frame prediction information is searched at the positions of the lower left (A0), left (A1), upper right (B0), upper (B1), and upper left (B2) adjacent to the decoded object block (CurBlk).

[0105] That is, in the spatial merging candidate, inter-frame prediction information from locations spatially adjacent to the decoded object block (CurBlk) is used.

[0106] Additionally, in the TMVP candidates, a search is conducted in images different from the decoded object block (see reference image). Figure 7 Inter-frame prediction information for each position in the lower right center (Col) and the lower right center (H) outside the decoded object block (CurBlk).

[0107] That is, in the TMVP candidate, inter-frame prediction information from positions that are temporally adjacent to the decoded object block is used.

[0108] In HMVP candidates and non-adjacent spatial merging candidates, inter-frame prediction information is used from locations that are spatially far (close) to the decoded object block.

[0109] Specifically, in the HMVP candidate, a table is constructed that can store FIFO-type inter-frame prediction information. Each time new inter-frame prediction information is searched, the oldest inter-frame prediction information is discarded from the table, and the new inter-frame prediction information is added to the table.

[0110] Similar to the HMVP candidates and non-adjacent spatial merging candidates disclosed in Non-Patent Document 1 and Non-Patent Document 2, the inter-frame prediction unit 205 of the image decoding apparatus 200 according to this embodiment can design the location for searching inter-frame prediction information and the size of the table for HMVP candidates.

[0111] Furthermore, the inter-frame prediction unit 205 can re-search for new reference block positions from the surrounding pixels of the reference block of the inter-frame prediction information of the search object, and correct the inter-frame prediction information (motion vector) so that in the search for merging candidates in the aforementioned conventional merging, the difference (hereinafter referred to as template matching cost) between the decoded pixels (template of the reference block) adjacent to the reference block based on the inter-frame prediction information of the search object and the decoded pixels (template of the decoded object block) adjacent to the decoded object block is reduced.

[0112] As an example, the inter-frame prediction unit 205 can be applied to template matching and merging disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0113] Furthermore, in the selection of merge candidates from the merge candidate list in the aforementioned conventional merging, the inter-frame prediction unit 205 can derive inter-frame prediction information of L0 from the control information and select the merge candidate with the smallest difference between the template of the reference block based on the inter-frame prediction information stored in the merge candidate list L1 and the template of the reference block based on the inter-frame prediction information of L0 derived from the control information, as the merge candidate of L1.

[0114] Since the inter-frame prediction unit 205 can derive the merging candidate (i.e., inter-frame prediction information) of L1 based on the inter-frame prediction information of L0 derived from the control information, it is not necessary to decode merge_idx in order to derive the inter-frame prediction information of L1, except for the control information required to derive the inter-frame prediction information of L0.

[0115] As an example, the inter-frame prediction unit 205 can be applied to the adaptive motion vector prediction-merge mode disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0116] As a variation, the inter-frame prediction unit 205 can reverse L0 and L1 in the aforementioned example. That is, the inter-frame prediction unit 205 can derive inter-frame prediction information for L1 using control information and derive inter-frame prediction information for L0 from the merging candidate list.

[0117] In addition, the inter-frame prediction unit 205 can construct a merge candidate list that derives inter-frame prediction information in units of sub-blocks after the decoded object block is divided, so as to derive inter-frame prediction information.

[0118] As an example, the inter-frame prediction unit 205 can construct a merge candidate list for sub-block TMVP (SbTMVP) or affine merge disclosed in Non-Patent Document 1 and Non-Patent Document 2 to derive inter-frame prediction information.

[0119] <1. Basic Concepts of Chain Motion Vector Prediction (Derivation of Chain Motion Vectors)> The aforementioned methods for deriving inter-frame prediction information (hereinafter, for simplicity, referred to as motion vectors) based on the inter-frame prediction unit are all limited to inter-frame prediction information stored in spatial or temporal locations adjacent to or close to the decoded object block.

[0120] To further improve coding efficiency, the inter-frame prediction unit 205 can export a motion vector generated by adding motion vectors derived from recursively searching motion vectors stored in the decoded image buffer 207 to the motion vectors of positions that are spatially or temporally adjacent to or close to the decoded object block, and use this motion vector as the motion vector of the decoded object block.

[0121] Here, the technique of recursively (chain-likely) searching the motion vectors stored in the decoded image buffer 207 and generating new motion vectors (chained motion vectors) starting from motion vectors located spatially or temporally adjacent to or close to the decoded object block is referred to in this specification as "Chained Motion Vector Prediction (CMVP)". The technical content of this chained motion vector prediction is described in detail below.

[0122] use Figures 8 to 17 This section explains the method for deriving chain motion vectors in chain motion vector prediction.

[0123] Figure 8 This is a diagram illustrating an example of a method for deriving chain motion vectors in chain motion vector prediction.

[0124] like Figure 8 As shown, the inter-frame prediction unit 205 uses the motion vector MvL0 of L0 corresponding to any merge candidate stored in the merge candidate list. k Starting from (0) and the reference index, the motion vectors stored in the decoded image buffer 207 are searched recursively (chained) and new motion vectors are generated.

[0125] For example, as shown in Equation 1 and Equation 2, for the reference image RefPicL0 k The inter-frame prediction unit 205 can derive new motion vectors and reference images generated in chained motion vector prediction from the reference block RefBlk (m).

[0126] MvL0 k / m =MvL0 k (0)+MvL0 k (1)+MvL0 k (2)+…+MvL0 k (m) (Equation 1) RefPicL0 k / m =RefPicL0 k (m) (Equation 2) That is, the new motion vector MvL0 generated in the chained motion vector prediction can be calculated by recursively (chain-like) searching the motion vectors stored in the decoded image buffer 207 and summing all the derived motion vectors. k / m .

[0127] Additionally, the new reference image generated in the chain motion vector prediction is MvL0. k / m Reference image used for reference.

[0128] Figure 9 This is a diagram illustrating an example of a method for deriving chain motion vectors in chain motion vector prediction.

[0129] like Figure 9 As shown, in chained motion vector prediction, the inter-frame prediction unit 205 can add block vectors (vectors indicating reference targets within the same image) to the recursive search object in addition to searching for motion vectors.

[0130] For example, when adding a block vector (Bv), for the reference image RefPicL0 k The reference block RefBlk of (m), as shown in Equation 3, adds Bv to the derivation of the new motion vector generated in the chained motion vector prediction.

[0131] MvL0 k / m =MvL0 k (0)+Bv k (0)+MvL0 k (1)+MvL0 k (2)+…+MvL0 k (m) (Equation 3) As and in Figure 8 as well as Figure 9 In the modified example of the chain motion vector derivation method shown in the figure, the inter-frame prediction unit 205 can use a block vector as the starting vector (initial vector) for the search of the chain motion vector.

[0132] Specifically, as shown in Equation 4 below, the first vector on the right (initial vector) can be Bvk(0).

[0133] MvL0k / m=Bvk(0)+MvL0k(1)+MvL0k(2)+…+MvL0k(m) (Equation 4) Compared to image assets captured by a camera, screen image assets and game image assets are more likely to include block vectors within the decoded object image. Therefore, if block vectors can be chosen as the initial vectors for deriving chained motion vectors, chained motion vectors become easier to derive from screen image assets and game image assets, resulting in an expected improvement in encoding performance.

[0134] As an example of modification, the inter-frame prediction unit 105 can use control information in units of sequences to control whether block vectors can be used as the initial vectors for chain motion vector prediction.

[0135] Specifically, the decoding unit 201 may have a decoding flag (sps_cmvp_initial_bv_enabled_flag). The aforementioned flag indicates, on a unit basis, whether the inter-frame prediction unit 205 can use block vectors as initial vectors for chained motion vector prediction. Based on the decoding result of this flag, the inter-frame prediction unit 205 controls whether block vectors can be used as initial vectors for chained motion vector prediction.

[0136] Figure 10 This is a diagram illustrating an example of a recursive search for motion vectors that includes bidirectional prediction (i.e., inter-frame prediction using two motion vectors different from L0 and L1). Figure 11 This is a diagram illustrating an example of a chain of motion vectors derived through a recursive search that includes bidirectional prediction of motion vectors.

[0137] like Figure 10 as well as Figure 11 As shown, in chained motion vector prediction, the inter-frame prediction unit 205 uses the search list numbers (L0 or L1) and search depths (or search counts) of the motion vectors stored in the decoded image buffer 207 to search for one or more new motion vectors from a reference source. Figure 10 as well as Figure 11 In the example, at most two motion vectors are derived, and all the derived motion vectors are added to each of the newly derived motion vectors to generate a chained motion vector MvL0. k / m .

[0138] Figure 12 as well as Figure 13 This is a diagram illustrating an example, as disclosed in Non-Patent Document 1 and Non-Patent Document 2, of searching for chain motion vector prediction candidates after searching for multiple merger candidates in a conventional merging process.

[0139] like Figure 12 as well as Figure 13 As shown, the inter-frame prediction unit 205 can search for chain motion vector prediction candidates after searching for predetermined merging candidates.

[0140] For example, such as Figure 12 As shown, in VVC, the inter-frame prediction unit 205 can search for chained motion vector prediction candidates after searching for TMVP candidates.

[0141] Chained motion vector prediction is not valid if the TMVP is not valid in the decoded object sequence, decoded object image group, decoded object image, or decoded object stripe, in the property of using motion vectors from reference images instead of decoded object images stored in decoded image buffer 207 to search for new motion vectors.

[0142] Therefore, searching for chained motion vector prediction candidates after searching for TMVP candidates can be seen as a natural design approach as a motion vector derivation method.

[0143] As Figure 12 Examples of changes, such as Figure 13 As shown, in VVC and ECM, the inter-frame prediction unit 205 can search for chained motion vector prediction candidates after searching for HMVP candidates.

[0144] In terms of generating new motion vectors through recursive motion vector search, chained motion vector prediction must be applied when at least one merging candidate is stored in the merging candidate list.

[0145] like Figure 13 As shown, by searching for HMVP candidates that can be expected to store more than one motion vector candidate for the merged candidate list, the likelihood of generating new motion vectors through chained motion vectors is increased.

[0146] As other examples of changes, such as Figure 12 As shown, in ECM, the inter-frame prediction unit 205 can search for chain motion vector prediction candidates after merging candidates in the search space.

[0147] Although Figure 7 The search positions for spatial merging candidates are shown in the figure. However, for example, if the search positions for spatial merging candidates are increased, the likelihood of storing more than one motion vector candidate in the merging candidate list increases, and it is easier to generate new motion vector candidates through chained merging of candidates.

[0148] Figures 14 to 17 This is a diagram illustrating an example of the merged candidates as derived objects of chained motion vector prediction candidates, and the position (index number) of the chained motion vector prediction candidates within the merged candidate list.

[0149] In addition, Figures 14 to 17In order to simplify the explanation of the technical content, the inter-frame prediction information stored in the merged candidate list only records motion vectors, but also stores the aforementioned reference index.

[0150] In addition to motion vectors and reference indices, the inter-frame prediction unit 205 may also store bcwIdx, hpIfIdx, licFlag, and mhpFlag, which are internal parameters for changing the aforementioned inter-frame prediction pixel generation method.

[0151] like Figure 14 As shown, the inter-frame prediction unit 205 can, after storing predetermined merging candidates in the merging candidate list, sequentially apply chained motion vector prediction starting from the first merging candidate stored in the merging candidate list, and sequentially store all new merging candidates generated in the chained motion vector prediction for each merging candidate in the merging candidate list.

[0152] Here, the pre-defined merge candidate can be used Figure 12 as well as Figure 13 The description refers to any of the multiple merge candidates in a regular merge, or it can be a predetermined number (the predetermined number of merge_idx) of merge candidates.

[0153] This predetermined number can be configured as a fixed value, or it can be configured as a variable value using control information in units of decoded object sequences, decoded object image groups, decoded object images, or decoded object stripes.

[0154] As Figure 14 Examples of changes, such as Figure 15 As shown, the inter-frame prediction unit 205 can, after storing the predetermined merging candidates in the merging candidate list, sequentially apply chain motion vector prediction starting from the first merging candidate stored in the merging candidate list, and sequentially store the new chain motion vectors (i.e., merging candidates) generated at each search depth for the chain motion vector prediction of each merging candidate in the merging candidate list.

[0155] As Figure 14 as well as Figure 15 Examples of changes, such as Figure 16 As shown, the inter-frame prediction unit 205 can sequentially apply chained motion vector prediction from the initial merging candidate after it is stored in the merging candidate list, and sequentially store all new chained motion vectors (i.e., merging candidates) generated in the chained motion vector prediction for each merging candidate in the merging candidate list.

[0156] As Figures 14 to 16 Examples of changes, such as Figure 17As shown, the inter-frame prediction unit 205 can sequentially apply chained motion vector prediction from the initial merging candidate after it is stored in the merging candidate list, and sequentially store the new chained motion vectors (i.e., merging candidates) generated at each search depth for the chained motion vector prediction of each merging candidate in the merging candidate list.

[0157] As Figures 14 to 17 In a modified example, the inter-frame prediction unit 205 can combine the chain motion vectors generated at each search depth for each chain motion vector prediction of each merge candidate list, and the chain motion vectors generated at search depths different from each search depth, to generate new merge candidates.

[0158] As Figures 14 to 17 In a modified example, the inter-frame prediction unit 205 can combine chain motion vectors generated from each search list number or each search depth predicted for each merge candidate list, and chain motion vectors generated using search list numbers or search depths different from those search list numbers or search depths, to generate new merge candidates.

[0159] <2. Pruning of Candidates for Chain Motion Vector Prediction> use Figure 18 This describes the pruning process performed by the inter-frame prediction unit 205 of the image decoding apparatus 200 in this embodiment when it adds a merging candidate (the aforementioned merging motion vector prediction candidate) corresponding to the merging candidate derived by the merging motion vector prediction to the merging candidate list.

[0160] Figure 18 This is a diagram illustrating an example of a pruning process in which the inter-frame prediction unit 205 of the image decoding apparatus 200 according to this embodiment adds merging candidates corresponding to the chain motion vectors derived by chain motion vector prediction to the merging candidate list.

[0161] The inter-frame prediction unit 205 determines whether to add a merging candidate corresponding to the chain motion vectors derived from the chain motion vector prediction based on predetermined conditions.

[0162] Specifically, such as Figure 18 As shown, when the inter-frame prediction unit 205 determines in step S000 that the predetermined conditions are met, in step S002, it determines that no merge candidate corresponding to the chain motion vector derived by the chain motion vector prediction is added.

[0163] On the other hand, when the inter-frame prediction unit 205 determines in step S000 that the predetermined conditions are not met, in step S001, it determines to add a merge candidate corresponding to the chain motion vector derived by the chain motion vector prediction.

[0164] Here, for example, the predetermined conditions may be consistent with the reference image and motion vector associated with at least one merge candidate stored in the merge candidate list and the respective merge candidates in the merge candidates corresponding to the derived chain motion vectors.

[0165] When the inter-frame prediction unit 205 determines that the reference image and motion vector are consistent with at least one merge candidate stored in the merge candidate list and each merge candidate in the merge candidate corresponding to the derived chain motion vector, it determines that the merge candidate corresponding to the chain motion vector will not be added to the merge candidate list, that is, it removes (as a pruning object) the merge candidate corresponding to the chain motion vector. This prevents the storage of more than two identical motion vectors in the merge candidate list, reduces the amount of encoding in merge_idx (merge index) sent to the image decoding device 200, and improves encoding efficiency.

[0166] Alternatively, the predetermined condition may be that all reference images and motion vectors associated with at least one merge candidate stored in the merge candidate list and each merge candidate in the merge candidates corresponding to the derived chain of motion vectors are identical.

[0167] Alternatively, the predetermined condition may be that the reference image and motion vectors of merge candidate list number 0 (L0) and merge candidate list number 1 (L1) associated with at least one merge candidate in the merge candidate list and each merge candidate in the merge candidate list corresponding to the derived chain motion vector are all identical.

[0168] Furthermore, the predetermined condition may be that the reference image associated with at least one merge candidate in the merge candidate list and each merge candidate in the merge candidate list corresponding to the derived chain motion vector is consistent, and the error (Euclidean distance) of the reference position of the motion vector associated with at least one merge candidate in the merge candidate list and each merge candidate in the merge candidate list corresponding to the derived chain motion vector is less than a predetermined threshold.

[0169] Therefore, the pruning process for merging candidates corresponding to chained motion vectors is strengthened, thereby preventing the storage of more than two identical motion vectors in the merging candidate list, reducing the amount of encoding sent to merge_idx in the image decoding device 200, and improving encoding efficiency.

[0170] As a further control, the inter-frame prediction unit 205 can configure a predetermined threshold for the case where there is only one motion vector being compared to a predetermined threshold that is greater than the predetermined threshold for the case where there are two motion vectors being compared.

[0171] Thus, by configuring the predetermined threshold for the case where there is only one motion vector to be compared to a predetermined threshold for the case where there are two motion vectors to be compared, more precise pruning processing can be performed.

[0172] Here, the inter-frame prediction unit 205 can configure a predetermined threshold of 1.5 pixels when the motion vector being compared is one.

[0173] Alternatively, the inter-frame prediction unit 205 can configure a predetermined threshold of 1.0 pixels or 2.0 pixels when the motion vector being compared is one.

[0174] In addition, the inter-frame prediction unit 205 can configure a predetermined threshold of 1.0 pixels when there are two motion vectors being compared.

[0175] Alternatively, the inter-frame prediction unit 205 can configure a predetermined threshold of 0.5 pixels or 1.5 pixels when there are two motion vectors being compared.

[0176] In addition, the aforementioned predetermined conditions may be consistent with the reference image and motion vector associated with each of the merge candidates stored in the merge candidate list and the merge candidates corresponding to the derived chain motion vectors, as well as the internal parameters of the method for generating the predetermined inter-frame prediction pixels.

[0177] Alternatively, the predetermined conditions may be consistent with the reference image and motion vector associated with at least one merge candidate stored in the merge candidate list and the merge candidates corresponding to the derived chain motion vectors, as well as the internal parameters (hpelIfIdx) for determining whether a switching interpolation filter is applied.

[0178] Alternatively, the predetermined conditions may be consistent with the reference image and motion vector associated with at least one merge candidate stored in the merge candidate list and the merge candidates corresponding to the derived chain of motion vectors, as well as the internal parameter (licFlag) controlling whether local brightness compensation is applied.

[0179] Furthermore, the predetermined conditions may be consistent with the reference image and motion vector associated with at least one merge candidate stored in the merge candidate list and the respective merge candidates in the merge candidates corresponding to the derived chain motion vectors, as well as the internal parameters (bcwIdx) for determining the weight values ​​of the block-based weighted bidirectional prediction.

[0180] By including the aforementioned internal parameters in the predetermined conditions, even when the reference image being compared and the motion vector are the same, the inter-frame prediction unit 205 can generate inter-frame prediction pixels using different methods when the internal parameters are different, leaving room for improving the accuracy of inter-frame prediction, and thus expecting an improvement in coding efficiency.

[0181] <3. Control of the reference source and reference target when deriving chain motion vectors> use Figures 19 to 22 The method for controlling the reference source and the reference target when the inter-frame prediction unit 205 of the image decoding apparatus 200 according to this embodiment derives the chain motion vector in the chain motion vector prediction will be described.

[0182] First, use Figure 19 The method for controlling the reference source when deriving chain motion vectors in chain motion vector prediction is explained.

[0183] Figure 19 This is a diagram illustrating an example of a method for controlling the deriving of a reference source when generating a chain motion vector in the chain motion vector prediction by the inter-frame prediction unit 205.

[0184] When deriving chained motion vectors in chained motion vector prediction, the inter-frame prediction unit 205 can configure the starting point of the first motion vector constituting the chained motion vector as part or all of the decoded object block.

[0185] As an example of configuration as part of a decoded object block, such as Figure 19 As shown, when deriving chained motion vectors in chained motion vector prediction, the inter-frame prediction unit 205 can configure the starting point of the first motion vector constituting the chained motion vector as the center (C) of the decoded object block (CurBlk).

[0186] Or, such as Figure 19 As shown, when deriving chained motion vectors in chained motion vector prediction, in addition to the center (C) of the decoded object block (CurBlk), the inter-frame prediction unit 205 can also configure the starting point of the first motion vector constituting the chained motion vector as a predetermined position of the decoded object block (CurBlk), and search in a predetermined order whether the motion vector is stored in the reference target of the motion vector.

[0187] Specifically, such as Figure 19As shown, when deriving chained motion vectors in chained motion vector prediction, in addition to the center (C) of the decoded object block (CurBlk), the inter-frame prediction unit 205 can also configure the starting point of the first motion vector constituting the chained motion vector as the four corners of the decoded object block (CurBlk), and search whether the motion vector is stored in the reference target of the motion vector in the order of the center (C), upper left (TL), upper right (TR), lower left (BL), and lower right (BR) of the decoded object block as a predetermined order.

[0188] The inter-frame prediction unit 205 can generate chained motion vectors for each motion vector searched and derived at each point.

[0189] As a variation, the inter-frame prediction unit 205 can compare the difference between the reference block of each motion vector searched and derived at each point and the adjacent decoded pixel values ​​of the decoding target block, and generate a chain of motion vectors by limiting at least one motion vector with a smaller difference.

[0190] As another variation, the inter-frame prediction unit 205 can perform a weighted average of the motion vectors searched and derived at each point to derive a new motion vector for generating a chain of motion vectors.

[0191] Specifically, the inter-frame prediction unit 205 can scale each motion vector based on the distance between the reference image and the decoded object block when weighting and averaging each motion vector, thereby generating new motion vectors. The scaling target can be configured as the reference image with the smallest distance to the decoded object block.

[0192] When deriving chained motion vectors, in addition to the center of the decoded object block, the inter-frame prediction unit 205 further configures the starting point of the first motion vector constituting the chained motion vector as one of the four corners of the decoded object block, thereby increasing the likelihood of chained motion vectors being derived. As a result, an improvement in coding efficiency can be expected.

[0193] As a further control, when deriving chained motion vectors in chained motion vector prediction, the inter-frame prediction unit 205 can control the starting point of the first motion vector constituting the chained motion vector according to the block size or block aspect ratio of the decoded object block.

[0194] For example, when deriving chained motion vectors in chained motion vector prediction, if the block size of the decoded object block is less than or equal to a first size, the inter-frame prediction unit 205 can configure the starting point of the first motion vector constituting the chained motion vector as only the center of the decoded object block.

[0195] When the block size of the decoded object block is less than the first size, when deriving the chained motion vectors, there is a high probability that the initial motion vectors constituting the chained motion vectors are the same at the center of the decoded object block and at the four corners of the decoded object block.

[0196] Therefore, when the block size of the decoded object block is less than or equal to the first size, when deriving the chain motion vector, by configuring the starting point of the first motion vector constituting the chain motion vector only as the center of the decoded object block (excluding the four corners of the decoded object block), the amount of search processing for the chain motion vector can be reduced.

[0197] Alternatively, when deriving chained motion vectors in chained motion vector prediction, if the block size of the decoded object block is larger than the first size, in addition to the center of the decoded object block, the inter-frame prediction unit 205 may also configure the starting point of the first motion vector constituting the chained motion vector as the four corners of the decoded object block, and search whether the motion vector is stored in the reference target of the motion vector in a predetermined order of the center, upper left, upper right, lower left and lower right of the decoded object block.

[0198] When the block size of the decoded object block is larger than the first size, when deriving the chained motion vectors, there is a high probability that the initial motion vectors constituting the chained motion vectors will differ at the center of the decoded object block and at the four corners of the decoded object block.

[0199] Therefore, when the block size of the decoded object block is larger than the first size, when deriving chain motion vectors, in addition to the center of the decoded object block, different chain motion vectors are generated by configuring the starting point of the first motion vector constituting the chain motion vector as the four corners of the decoded object block, and the coding efficiency can be expected to be improved.

[0200] Here, we also envision the case where the decoded object block is not square. Therefore, the block size (number of pixels in the block) of the decoded object block can be replaced with the width and height of the decoded object block to implement the aforementioned control.

[0201] Specifically, when deriving chained motion vectors in chained motion vector prediction, if the width or height of the decoded object block is greater than a predetermined number of pixels, in addition to the center of the decoded object block, the inter-frame prediction unit 205 can also configure the starting point of the initial motion vector constituting the chained motion vector as the four corners of the decoded object block, and search whether the motion vector is stored in the reference target of the motion vector in the order of the center, upper left, upper right, lower left and lower right of the decoded object block as a predetermined order.

[0202] The inter-frame prediction unit 205 can be configured with any one of 8 pixels, 16 pixels, or 32 pixels as the predetermined number of pixels.

[0203] Alternatively, when deriving chained motion vectors in chained motion vector prediction, if the aspect ratio of the block to be decoded is greater than a predetermined ratio, in addition to the center of the block to be decoded, the inter-frame prediction unit 205 may also configure the starting point of the first motion vector constituting the chained motion vector as the lower left and upper right of the block to be decoded, and search for whether the motion vector is stored in the reference target of the motion vector in the order of the center, lower left and upper right of the block to be decoded as a predetermined order.

[0204] The inter-frame prediction unit 205 can be configured to use any one of 4, 8 or 16 as the predetermined ratio.

[0205] Secondly, use Figure 20 as well as Figure 21 An example of a control method for a reference target when deriving chain motion vectors in chain motion vector prediction is given.

[0206] Figure 20 This is a diagram illustrating an example of a method for controlling the generation of a reference target when generating a chain motion vector in the chain motion vector prediction by the inter-frame prediction unit 205.

[0207] like Figure 20 As shown, during the recursive motion vector search process when deriving chained motion vectors, if the reference position of each motion vector is not a reference image, the inter-frame prediction unit 205 can search for motion vectors at the storage location of motion vectors in the reference image closest to that reference position.

[0208] Even if the reference position of the motion vector is outside the reference image, new chain motion vectors can still be generated because there are cases where motion vectors linked from this motion vector are located in different reference images.

[0209] Or, such as Figure 20 As shown, during the recursive motion vector search process when deriving chained motion vectors, if the reference position of each motion vector is not referenced by the reference image, the inter-frame prediction unit 205 can terminate the generation of chained motion vectors.

[0210] When the reference position of a motion vector is outside the reference image, since the motion vectors chained from this motion vector are also likely to reference outside the reference image in different reference images, the amount of decoding processing can be reduced by terminating the generation of the chained motion vectors.

[0211] Alternatively, if the decoding image buffer 207 stores an inter-frame prediction padding region outside the reference image, the aforementioned record of "outside the reference image" can be replaced with the record of "outside the inter-frame prediction padding region," and the inter-frame prediction unit 205 can perform the same processing.

[0212] Figure 21This diagram illustrates an example of a method for controlling the reference source and reference target when exporting chain motion vectors predicted by the chain motion vectors of the inter-frame prediction unit 205 in a case where Gradual Decoding Refresh (GDR), a known encoding and decoding technique for low-latency video transmission, is applied to a group of images to be decoded.

[0213] GDR is a technique that reduces the processing latency of video encoding and decoding at a certain bit rate by using intra-frame prediction encoding and decoding to update only a portion of the frame (hereinafter referred to as intra-frame update) and moving that portion of the frame for each decoded object image. This is compared to video encoding and decoding in random access configurations that include intra-frame images, which are widely used in video transmission.

[0214] Figure 21 The intra-slice diagram illustrates the intra-update region, showing an example of the slice moving from the top to the bottom of the decoded object image.

[0215] In this specification, the three regions in the decoded object image divided by GDR are referred to as the region after intra-frame update, the region during intra-frame update, and the region before intra-frame update.

[0216] like Figure 21 As shown, when progressive decoding refresh is applied in the decoded object image group, the inter-frame prediction unit 205 can limit the search for recursive motion vectors predicted by chained motion vectors to no more than the region during the intra-frame update period.

[0217] For example, such as Figure 21 As shown, when progressive decoding refresh is applied in the group of decoded object images and the decoded object block is included in the region after intra-frame update, the inter-frame prediction unit 205 can limit the region that can be referenced in each search of the recursive motion vector predicted by the chained motion vector to the region after intra-frame update in the reference image of each search object.

[0218] Alternatively, when progressive decoding refresh is applied in the group of decoded object images and the decoded object block is included in the region before the intra-frame update, the inter-frame prediction unit 205 can limit the region that can be referenced in each search of the recursive motion vector predicted by the chained motion vector to the region in the reference image of each search object before the intra-frame update.

[0219] Thus, when progressive decoding refresh is applied in the group of images to be decoded, the inter-frame prediction unit 205 can generate chain motion vectors with high prediction accuracy by limiting the search of recursive motion vectors predicted by chain motion vectors to no more than the region during the intra-frame update.

[0220] Figure 22 This diagram illustrates an example of controlling the reference target when deriving chain motion vectors in chain motion vector prediction via the inter-frame prediction unit 205 in a case where reference picture resampling (RPR) disclosed in Non-Patent Document 1 is applied in a decoding object block, allowing reference to motion vectors of a reference picture with a different resolution (sampling ratio) than the decoded picture.

[0221] In RPR, the inter-frame prediction unit 205 calculates the ratio of the width and height of the reference image to the width and height of the decoded image, and scales the pixel position and motion vector length of the reference image relative to the decoded image based on the ratio to generate inter-frame prediction pixels.

[0222] Therefore, for example, such as Figure 22 As shown, when the width and height of the reference image are 0.5 times the width and height of the decoded image, the pixel position and motion vector length of the reference image relative to the decoded image are scaled to 0.5 times.

[0223] On the other hand, when generating chained motion vectors, the inter-frame prediction unit 205 needs to connect the motion vectors at the same scale as the decoded image.

[0224] Therefore, as Figure 22 As shown, when the inter-frame prediction unit 205 generates chained motion vectors by referencing motion vectors from a reference image with a different resolution (sampling ratio) than the decoding target image, it can inversely scale the length of the recursively derived motion vectors according to the ratio of the width and height of the decoding target image to the width and height of each reference image, and then add them together to generate chained motion vectors.

[0225] As a variation, when it is permissible to reference motion vectors for reference images with a different resolution (sampling ratio) than the decoded object images for a sequence of decoded object images or a group of decoded object images, the inter-frame prediction unit 205 may apply chained motion vectors only to reference images with the same sampling ratio as the decoded object images.

[0226] In other words, when it is permissible to reference motion vectors for a reference image with a different resolution (sampling ratio) than the decoded object image for a sequence of decoded object images or a group of decoded object images, the inter-frame prediction unit 205 can be configured not to apply chained motion vectors to the reference image with a different sampling ratio than the decoded object image.

[0227] <4. Inheritance of inter-frame prediction information in chained motion vector prediction> When merging candidates are derived through chained motion vector prediction, the inter-frame prediction unit 205 can control whether or not predetermined internal parameters of the inter-frame prediction pixel generation method are inherited and changed.

[0228] (Switch the interpolation filter) As an example, when merging candidates are derived through chained motion vector prediction, the inter-frame prediction unit 205 can always be configured such that the internal parameter (hpelIfIdx) corresponding to the merging candidates derived through chained motion vector prediction indicates that the switching interpolation filter is not applied, and does not inherit the internal parameter (hpelIfIdx) associated with each recursively searched motion vector that determines whether the switching interpolation filter is applied.

[0229] Alternatively, when merging candidates are derived through chained motion vector prediction, the inter-frame prediction unit 205 may inherit the internal parameters (hpelIfIdx) associated with the first or last motion vector constituting the chained motion vectors from the internal parameters (hpelIfIdx) for determining whether a switching interpolation filter has been applied, which are associated with each recursively searched motion vector, as the internal parameters (hpelIfIdx) for determining whether a switching interpolation filter has been applied corresponding to the merging candidates derived through chained motion vector prediction.

[0230] Alternatively, when merging candidates are derived through chained motion vector prediction, if at least one of the internal parameters (hpelIfIdx) associated with each recursively searched motion vector indicating the application of a switching interpolation filter indicates the application of a switching interpolation filter, the inter-frame prediction unit 205 may be configured such that the internal parameters (hpelIfIdx) corresponding to the merging candidates derived through chained motion vector prediction indicate the application of a switching interpolation filter.

[0231] Furthermore, when the chain motion vectors generated by chain motion vector prediction include at least one or more block vectors, the inter-frame prediction unit 205 may be configured to always indicate that the internal parameter (hpelIfIdx) corresponding to the merge candidate derived by chain motion vector prediction, which determines whether a switching interpolation filter is applied, is not applied.

[0232] (Local brightness compensation) As an example, when merging candidates are derived through chained motion vector prediction, the inter-frame prediction unit 205 can always be configured such that the internal parameter (licFlag) corresponding to the merging candidates derived through chained motion vector prediction, which controls whether or not local brightness compensation is applied, indicates that local brightness compensation is not applied, and does not inherit the internal parameter (licFlag) associated with each recursively searched motion vector, which controls whether or not local brightness compensation is applied.

[0233] Alternatively, when merging candidates are derived through chained motion vector prediction, the inter-frame prediction unit 205 may inherit the internal parameter (licFlag) associated with the first or last motion vector constituting the chained motion vector from the internal parameters (licFlag) for controlling whether or not local brightness compensation is applied, which is associated with each motion vector recursively searched, and use it as the internal parameter (licFlag) for controlling whether or not local brightness compensation is applied corresponding to the merging candidates derived through chained motion vector prediction.

[0234] Alternatively, when merging candidates are derived through chained motion vector prediction, the inter-frame prediction unit 205 may be configured such that, if at least one of the internal parameters (licFlag) controlling whether or not local brightness compensation is applied, associated with each recursively searched motion vector, indicates the application of local brightness compensation, the internal parameter (licFlag) controlling whether or not local brightness compensation is applied corresponding to the merging candidate derived through chained motion vector prediction also indicates the application of local brightness compensation.

[0235] Furthermore, when the chain motion vectors generated by chain motion vector prediction include at least one or more block vectors, the inter-frame prediction unit 205 can always configure the internal parameter (licFlagg) corresponding to the merging candidate derived by chain motion vector prediction, which controls whether local brightness compensation is applied, to indicate that local brightness compensation is not applied.

[0236] As a variation, the inter-frame prediction unit 205 can re-determine the value of the internal parameter (licFlag) controlling whether or not local brightness compensation inherited from the chained motion vector is applied by comparing the sum of absolute differences (SAD) calculated from the adjacent pixel values ​​of the decoded object block and the reference block with the mean-removal sum of absolute differences (MR-SAD).

[0237] Specifically, when MR-SAD is greater than or equal to SAD, the inter-frame prediction unit 205 can configure the value of the internal parameter (licFlag) that controls whether or not local brightness compensation is applied to be determined as a value that does not apply local brightness compensation. On the other hand, when MR-SAD is less than SAD, the inter-frame prediction unit 205 can configure the value of the internal parameter (licFlag) that controls whether or not local brightness compensation is applied to be determined as a value that applies local brightness compensation.

[0238] As a further modification, the inter-frame prediction unit 205 can be reconfigured based on control information (cu_merge_lic_revert_flag) per decoded object block. The control information per decoded object block controls whether to maintain or reverse the value of the internal parameter (licFlag). The internal parameter controls whether to apply local brightness compensation inherited from the chained motion vector.

[0239] (Weighted bidirectional prediction on a block-by-block basis) As an example, when merging candidates are derived through chained motion vector prediction, the inter-frame prediction unit 205 can always be configured such that the internal parameter (bcwIdx) that determines the weight value of the weighted bidirectional prediction on a block-by-block basis, corresponding to the merging candidates derived through chained motion vector prediction, indicates that the weight value of the weighted bidirectional prediction on a block-by-block basis is a simple average value, and does not inherit the internal parameter (bcwIdx) that determines the weight value of the weighted bidirectional prediction on a block-by-block basis associated with each motion vector recursively searched.

[0240] Alternatively, when merging candidates are derived through chained motion vector prediction, the inter-frame prediction unit 205 may inherit the internal parameter (bcwIdx) associated with the weight value of the weighted bidirectional prediction on a block-by-block basis from the internal parameter (bcwIdx) that determines the weight value of the weighted bidirectional prediction on a block-by-block basis, which is associated with the first or last motion vector constituting the chained motion vector, as the internal parameter (bcwIdx) for determining the weight value of the weighted bidirectional prediction on a block-by-block basis corresponding to the merging candidate derived through chained motion vector prediction.

[0241] As a variation, when merging candidates are derived through chained motion vector prediction, if at least one of the internal parameters (bcwIdx) associated with each recursively searched motion vector that determine the weight values ​​of the weighted bidirectional prediction on a block-by-block basis indicates that the weight values ​​of the weighted bidirectional prediction on a block-by-block basis are determined to be a simple average, the inter-frame prediction unit 205 may be configured such that the internal parameter (bceIdx) corresponding to the merging candidates derived through chained motion vector prediction determines the weight values ​​of the weighted bidirectional prediction on a block-by-block basis to be determined to be a simple average.

[0242] (Multiple Hypothesis Prediction) As an example, when merging candidates are derived through chained motion vector prediction, the inter-frame prediction unit 205 can be configured such that the internal parameter (mhpFlag) corresponding to the merging candidates derived through chained motion vector prediction, which controls whether or not multi-hypothesis prediction is applied, indicates that multi-hypothesis prediction is not applied, and does not inherit the internal parameter (mhpFlag, fourth internal parameter) associated with each recursively searched motion vector, which controls whether or not multi-hypothesis prediction is applied.

[0243] As a variation, when merging candidates are derived through chained motion vector prediction, the inter-frame prediction unit 205 may be configured to indicate the internal parameter (mhpFlag) of the control on whether or not multiple hypothesis prediction is applied, which is associated with the first or last motion vector constituting the chained motion vector, as the internal parameter (mhpFlag) of the control on whether or not multiple hypothesis prediction is applied, corresponding to the merging candidates derived through chained motion vector prediction.

[0244] As a variation, when merging candidates are derived through chained motion vector prediction, the inter-frame prediction unit 205 may be configured such that, if at least one or more of the internal parameters (mhpFlag) associated with each recursively searched motion vector indicating whether or not multi-hypothesis prediction is applied indicate the application of multi-hypothesis prediction, the internal parameter (mhpFlag) corresponding to the merging candidates derived through chained motion vector prediction indicates the application of multi-hypothesis prediction.

[0245] As a further variation, when the chain motion vectors generated by chain motion vector prediction include at least one or more block vectors, the inter-frame prediction unit 205 may be configured to always indicate that multi-hypothesis prediction is not applied, corresponding to the merge candidates derived by chain motion vector prediction and controlling whether or not multi-hypothesis prediction is applied.

[0246] As mentioned above, if the internal parameters of the method for generating inter-frame prediction pixels are not inherited in chained motion vector prediction, the amount of inter-frame prediction information that needs to be stored in the decoded image buffer 207 is reduced.

[0247] On the other hand, as mentioned above, if the internal parameters of the method for generating inter-frame predicted pixels are inherited in chained motion vector prediction, the inter-frame predicted pixels generated in chained motion vector prediction can potentially be made more accurate, and an improvement in coding efficiency can be expected.

[0248] Furthermore, since the necessity of changing the generation method of inter-frame predicted pixels is low when the chained motion vector includes at least one or more block vectors in the case of inheriting the aforementioned internal parameters, an improvement in coding efficiency can be expected by configuring it not to inherit the internal parameters, i.e., not to change the generation method of inter-frame predicted pixels.

[0249] <5. Including candidate merging, candidate rearrangement, and vector correction for chained motion vector prediction candidates> (Rearrangement of merged candidates) The later a chain of motion vector prediction candidates is derived compared to other merge candidates, the more difficult it is for the chain of motion vector prediction candidates to be stored in the merge candidate list.

[0250] As a method to solve this problem, the inter-frame prediction unit 205 can calculate the template matching cost of all merge candidates stored in the merge candidate list, and rearrange them in ascending order according to the template matching cost from low to high in the storage location of each merge candidate in the merge candidate list, which includes merge candidates derived by chain motion vector prediction.

[0251] As an example, the inter-frame prediction unit 205 may use the Adaptive Reordering Merge Candidate (ARMC) disclosed in Non-Patent Document 2 as a technique related to the reordering of the storage location of each merge candidate in the merge candidate list based on the template matching cost of each merge candidate.

[0252] Furthermore, as described above, since chained motion vector prediction recursively generates new motion vectors, other merged candidates may not be stored in the merged candidate list when other merged candidates are derived after the chained motion vector prediction candidates are derived.

[0253] As a method to solve this problem, the inter-frame prediction unit 205 can calculate the template matching cost of all unpruned merge candidates, including merge candidates derived from chained motion vector prediction, and store them in the merge candidate list in ascending order according to the template matching cost from low to high.

[0254] Alternatively, the inter-frame prediction unit 205 may perform a series of processes up to a predetermined number of times, including calculating the template matching costs of all unpruned merge candidates derived from chained motion vector predictions, and storing them in an ascending order of template matching costs in a merge candidate list. For example, this predetermined number of times could be two or three.

[0255] Therefore, merging candidates that can generate inter-frame predicted pixels with high accuracy are placed in the merging candidate list, which can be expected to improve coding efficiency.

[0256] (Correction Vector) The inter-frame prediction unit 205 can use a predetermined method to correct the motion vectors associated with the merged candidates derived from chained motion vector prediction.

[0257] As an example, the inter-frame prediction unit 205 can use template matching to correct motion vectors associated with merged candidates derived through chained motion vector prediction.

[0258] Here, the inter-frame prediction unit 205 may apply the same template matching technique as disclosed in Non-Patent Document 2 as the template matching.

[0259] As an example, the inter-frame prediction unit 205 can use bilateral matching to correct motion vectors associated with merged candidates derived through chained motion vector prediction.

[0260] Here, the inter-frame prediction unit 205 may apply the same bilateral matching technique disclosed in Non-Patent Document 1 and Non-Patent Document 2 as the bilateral matching. For example, the inter-frame prediction unit 205 may apply the decoder-side motion vector refinement (DMVR) disclosed in Non-Patent Document 1.

[0261] As an example, the inter-frame prediction unit 205 can differentially add motion vectors derived from control information to correct motion vectors associated with merged candidates derived through chained motion vector prediction.

[0262] Here, the inter-frame prediction unit 205 can apply the same technique as the merge with motion vector difference (MMVD) disclosed in Non-Patent Document 1 and Non-Patent Document 2 as control information indicating motion vector difference.

[0263] By using a predetermined method, the inter-frame prediction unit 205 corrects the motion vectors associated with the merged candidates derived from chained motion vector prediction, thereby improving the accuracy of inter-frame prediction and potentially improving coding efficiency.

[0264] <6. Limitations of chained motion vector prediction (application status / number of images / number of merged candidates)> use Figure 23 Various limiting methods for chain motion vector prediction by the decoding unit 201 and the inter-frame prediction unit 205 of the image decoding apparatus 200 according to this embodiment will be described.

[0265] (Is there an application of chain motion vector prediction?) First, regarding the application of chained motion vector prediction (whether it can be applied), as mentioned above, if TMVP can be applied in its nature, then chained motion vector prediction can be applied. In other words, if TMVP cannot be applied, then chained motion vector prediction cannot be applied.

[0266] Therefore, it is possible to control whether chained motion vector prediction can be applied based on the control information (syntax) of whether time-based motion vector prediction (TMVP) can be applied.

[0267] That is, the inter-frame prediction unit 205 can control whether to apply chained motion vector prediction based on the control information sent from the decoding unit 201 to control whether to apply temporal motion vector prediction (TMVP).

[0268] Here, in Non-Patent Document 1, the syntax based on the sequence of decoded objects (sps_temporal_mvp_enabled_flag, first control information, first syntax) and the syntax based on the image of the decoded object (ph_temporal_mvp_enabled_flag, second control information, second syntax) are defined as the syntax for controlling whether or not TMVP can be applied.

[0269] That is, it is also possible to control whether chain motion vector prediction can be applied using units that are the same as or lower than TMVP.

[0270] For example, the control method for whether chained motion vector prediction can be applied in the inter-frame prediction unit 205, which controls whether chained motion vector prediction can be applied on a unit basis of decoded object blocks, is as follows.

[0271] The inter-frame prediction unit 205 can control whether to apply chained motion vector prediction for the decoded object block based on control information sent from the decoding unit 201 to control whether to apply time motion vector prediction (TMVP) in units of the decoded object sequence.

[0272] More specifically, when the control information sent from the decoding unit 201 determines whether temporal motion vector prediction (TMVP) can be applied in units of the decoded object sequence, the inter-frame prediction unit 205 determines that chained motion vector prediction for the decoded object block can be applied.

[0273] On the other hand, if the aforementioned control information determines that temporal motion vector prediction cannot be applied, the inter-frame prediction unit 205 determines that chained motion vector prediction for the decoded object block cannot be applied.

[0274] Alternatively, the inter-frame prediction unit 205 can control whether to apply chained motion vector prediction for the decoded object block based on control information sent from the decoding unit 201 to control whether to apply time motion vector prediction (TMVP) per decoded object image.

[0275] More specifically, when the control information sent from the decoding unit 201 determines whether temporal motion vector prediction (TMVP) can be applied in units of decoded object images, the inter-frame prediction unit 205 determines that chained motion vector prediction for decoded object blocks can be applied.

[0276] On the other hand, if the aforementioned control information determines that temporal motion vector prediction cannot be applied, the inter-frame prediction unit 205 determines that chained motion vector prediction for the decoded object block cannot be applied.

[0277] The inter-frame prediction unit 205 can control whether to apply chained motion vector prediction based on the decoded object sequence or on decoded objects smaller than the decoded object sequence, based on the control information sent from the decoding unit 201 regarding whether to apply time motion vector prediction (TMVP) on a unit basis.

[0278] Alternatively, the inter-frame prediction unit 205 can control whether to apply chained motion vector prediction based on the control information sent from the decoding unit 201 regarding whether to apply time motion vector prediction (TMVP) on a unit of decoded object images or on a unit of decoded objects smaller than the decoded object images.

[0279] Alternatively, before the inter-frame prediction unit 205 controls whether chained motion vector prediction can be applied on a per-decoded object block basis, the decoding unit 201 can control whether chained motion vector prediction can be applied based on the control information for controlling whether temporal motion vector prediction (TMVP) can be applied.

[0280] Figure 23 This diagram illustrates an example of a process where the decoding unit 201 determines whether or not to decode a sequence of decoded objects based on a syntax (sps_chained_mvp_enabled_flag) that controls whether or not to apply chained motion vector prediction.

[0281] like Figure 23 As shown, when it is determined in step S100 that the syntax (sps_temporal_mvp_enabled_flag) for controlling whether to apply Temporal Motion Vector Prediction (TMVP) on a unit basis for the decoded object sequence is 1, the decoding unit 201 decodes the syntax (sps_chained_mvp_enabled_flag, third syntax) for controlling whether to apply Chained Motion Vector Prediction on a unit basis for the decoded object sequence in step S101.

[0282] On the other hand, when it is determined in step S100 that the syntax (sps_temporal_mvp_enabled_flag) for controlling whether or not to apply Temporal Motion Vector Prediction (TMVP) is applied on a unit basis for the decoded object sequence is not 1, the decoding unit 201 does not decode the syntax (sps_chained_mvp_enabled_flag) for controlling whether or not to apply Chained Motion Vector Prediction on a unit basis for the decoded object sequence in step S102.

[0283] Here, the decoding unit 201 determines that temporal motion vector prediction (TMVP) can be applied to the decoded object sequence when the value of sps_temporal_mvp_enabled_flag is 1, determines that temporal motion vector prediction (TMVP) cannot be applied to the decoded object sequence when the value of sps_temporal_mvp_enabled_flag is 0, and estimates the value of sps_temporal_mvp_enabled_flag as 0 when sps_temporal_mvp_enabled_flag is not decoded.

[0284] Furthermore, when the value of sps_chained_mvp_enabled_flag is 1, the decoding unit 201 determines that chained motion vector prediction can be applied on a unit basis for the decoded object sequence; when the value of sps_chained_mvp_enabled_flag is 0, it determines that chained motion vector prediction cannot be applied on a unit basis for the decoded object sequence; and when sps_chained_mvp_enabled_flag is not decoded, the value of sps_chained_mvp_enabled_flag is estimated to be 0.

[0285] Thus, since the value of sps_temporal_mvp_enabled_flag is used to determine whether sps_chained_mvp_enabled_flag has been decoded, in cases where decoding of sps_chained_mvp_enabled_flag is obviously not required, sps_chained_mvp_enabled_flag can be left undecoded, thereby reducing the amount of encoding.

[0286] When chained motion vector prediction is combined with Adaptive Reordering Merge Candidate (ARMC) by comparing the template matching costs of the aforementioned merged candidates, improvements in coding efficiency can be expected.

[0287] Therefore, as a modified example of the aforementioned control method for whether or not chained motion vector prediction can be applied based on whether or not TMVP can be applied, the inter-frame prediction unit 205 can control whether or not chained motion vector prediction can be applied based on both whether or not TMVP can be applied and whether or not ARMC can be applied.

[0288] Specifically, when both TMVP and ARMC can be applied, the inter-frame prediction unit 205 can determine that chained motion vector prediction can be applied. Otherwise, the inter-frame prediction unit 205 can determine that chained motion vector prediction cannot be applied.

[0289] In addition, the same control information as TMVP can be set to determine whether ARMC can be applied, and the determination can be made based on the value of this control information.

[0290] (Number of reference images for chain motion vector prediction / search depth) The following explains the methods for limiting the number of reference images or the search depth when searching for recursive motion vectors in chained motion vector prediction.

[0291] First, we will explain the method for limiting the number of reference images that can be consulted when searching for recursive motion vectors in chain motion vector prediction.

[0292] As an example, the inter-frame prediction unit 205 may limit the search for recursive motion vectors predicted by chained motion vectors to at least one reference image included in the list of reference images in the decoded object block.

[0293] By limiting the search for recursive motion vectors predicted by the inter-frame prediction unit 205 to at least one reference image included in the reference image list of the decoded object block, the amount of decoding processing can be reduced compared to the case where no such restriction is imposed (e.g., the inter-frame prediction unit 205 searches all reference images stored in the decoded image buffer 207).

[0294] Alternatively, the inter-frame prediction unit 205 may limit the search for recursive motion vectors in chained motion vector prediction to a reference image of the motion vector reference of the decoded object block.

[0295] In other words, the inter-frame prediction unit 205 only re-searches for block vectors within the reference image of the motion vector reference of the decoded object block, and generates chain motion vectors through chain motion vector prediction.

[0296] As a variation, the inter-frame prediction unit 205 may limit the search for recursive motion vectors in chained motion vector prediction to reference pictures that the decoded object block can refer to in Temporal Motion Vector Prediction (TMVP), which are usually called collocated pictures.

[0297] Specifically, the inter-frame prediction unit 205 can generate chained motion vectors by re-searching for new motion vectors and adding all the motion vectors derived from motion vectors that are spatially or temporally adjacent to or close to the decoded object block in a candidate list of candidates stored in the storage motion vectors, with the reference target being the juxtaposed image stored in the aforementioned decoded image buffer.

[0298] Here, when there are two or more juxtaposed images, the inter-frame prediction unit 205 can use all of these juxtaposed images to search for chain motion vectors.

[0299] Alternatively, when there are two or more juxtaposed images, the inter-frame prediction unit 205 can select the juxtaposed image to search for based on the difference in distance (POC: Picture Order Count) with the decoded target block. For example, the inter-frame prediction unit 205 can search only for juxtaposed images with the smaller POC.

[0300] Alternatively, when there are two or more juxtaposed images, the inter-frame prediction unit 205 can select based on the magnitude of the quantization parameters of the juxtaposed images. For example, the inter-frame prediction unit 205 can search only for juxtaposed images with larger quantization parameters.

[0301] Alternatively, when there are two or more juxtaposed images, the inter-frame prediction unit 205 may search only the juxtaposed images in list L0.

[0302] As a modified example (with stronger restrictions), the inter-frame prediction unit 205 can restrict the search area for the recursive motion vectors predicted by the chained motion vectors to a region that includes a predetermined number of rows of a coding tree unit (CTU) that is the same as the pixel position of the decoded object block in the juxtaposed image, commonly referred to as the juxtaposed CTU.

[0303] Here, the predetermined number of rows can be one. Or, the predetermined number of rows can be two. Or it can be four.

[0304] By limiting the search area for recursive motion vectors predicted by chained motion vectors to only one row of the juxtaposed image or juxtaposed CTU, the amount of inter-frame prediction information required for searching chained motion vectors in the decoded image buffer 207 can be reduced.

[0305] Secondly, the method for limiting the search depth of recursive motion vectors in chain-like motion vector prediction is explained.

[0306] As an example, the inter-frame prediction unit 205 can limit the search depth of the recursive motion vectors predicted by the chained motion vectors in the decoded object block based on control information.

[0307] Specifically, the inter-frame prediction unit 205 can limit the search depth of the recursive motion vectors predicted by the chain motion vectors in the decoded object block based on the control information (sps_max_num_my_chains) sent from the decoding unit that configures the upper limit of the search depth of the recursive motion vectors predicted by the chain motion vectors in units of the decoded object sequence.

[0308] Here, control information in units of decoded object sequences can be control information in units of images or strips, which are lower-level control information than control information in units of decoded object sequences, or hierarchical control information that combines these control information.

[0309] Alternatively, the inter-frame prediction unit 205 can configure a predetermined fixed value as the search depth of the recursive motion vector for chained motion vector prediction. The inter-frame prediction unit 205 can configure 1, 2, or 3 as this predetermined fixed value.

[0310] By limiting the search depth for chained motion vector prediction in this way, the amount of decoding processing required to derive the chained motion vectors can be reduced compared to the unrestricted case.

[0311] (The number of merged candidates for chain motion vector prediction) The following explains the method for limiting the number of merged candidates derived from chain motion vector prediction.

[0312] As an example, the inter-frame prediction unit 205 can limit the number of merge candidates derived from chained motion vector prediction in the decoded object block based on control information.

[0313] Specifically, the inter-frame prediction unit 205 can limit the upper limit of the search depth of the merge candidate number derived by chained motion vector prediction in the decoded object block based on the control information (sps_max_num_cmvp_cand) sent from the decoding unit, which configures the upper limit of the search depth of the merge candidate number derived by chained motion vector prediction on a per-decoded object sequence.

[0314] Here, control information in units of decoded object sequences can be control information in units of images or strips, which are lower-level control information than control information in units of decoded object sequences, or hierarchical control information that combines these control information.

[0315] Alternatively, the inter-frame prediction unit 205 can configure a predetermined fixed value as the number of merged candidates derived from chained motion vector prediction.

[0316] (Limitations of block vector search in chained motion vector prediction) The above describes how the search for block vectors can be included in the recursive motion vector search for chained motion vector prediction. However, while block vectors referencing the same image are easily found in screen images including characters, computer-generated graphics (CG), and game images, they are difficult to find in images captured by a camera.

[0317] Therefore, it is desirable to restrict the search for block vectors in chain motion vector prediction based on the features of the image.

[0318] As an example, the inter-frame prediction unit 205 can limit the block vector search based on control information, which determines whether the block vector in the chain motion vector prediction for the decoded target block can be searched.

[0319] Specifically, the inter-frame prediction unit 205 can limit whether or not to search for block vectors in chained motion vector prediction for the decoded object block based on the control information (sps_cmvp_bv_search_enabled_flag) sent from the decoding unit 201 that determines whether or not to search for block vectors in chained motion vector prediction on a per-decoded object sequence.

[0320] Here, control information in units of decoded object sequences can be control information in units of images or strips, which are lower-level control information than control information in units of decoded object sequences, or hierarchical control information that combines these control information.

[0321] (Limitations of the reference source for chain motion vector prediction) The above describes a method for adding the four corners of the decoded object block to the starting point of the initial motion vector in the search of recursive motion vectors for chained motion vector prediction, in addition to the center of the decoded object block.

[0322] However, images captured by a camera are much harder to change than screen images or game images, where pixel values ​​can change rapidly on a pixel-by-pixel basis.

[0323] Therefore, it is desirable to limit the starting point of the initial motion vector in the search for recursive motion vectors in chain motion vector prediction based on the features of the image.

[0324] As an example, the inter-frame prediction unit 205 can limit the reference source for chained motion vector prediction based on control information. This control information determines whether, in addition to the center of the decoded object block, at least one predetermined position of the decoded object block should be added to the starting point of the initial motion vector in the chained motion vector prediction for the decoded object block. This predetermined position can be one of the four corners within the decoded object block.

[0325] Specifically, the inter-frame prediction unit 205 can, based on sps_cmvp_multi_position_flag, limit whether, in addition to the center of the decoded object block, at least one or more predetermined positions of the decoded object block are added to the starting point of the first motion vector in the chained motion vector prediction for the decoded object block.

[0326] Here, sps_cmvp_multi_position_flag is the control information that determines whether, in addition to the center of the decoded object block, at least one predetermined position of the decoded object block is added to the starting point of the first motion vector in the chained motion vector prediction for the decoded object block.

[0327] Here, control information in units of decoded object sequences can be control information in units of images or strips, which are lower-level control information than control information in units of decoded object sequences, or hierarchical control information that combines these control information.

[0328] (Application limitations of chain motion vectors based on quantization parameters) Chained motion vectors are readily applied when the quantization parameter of the reference block corresponding to the vector from the reference source is large. Since large quantization parameters easily generate quantization noise (encoding degradation), chained motion vectors are generated to avoid generating predicted pixels that include this noise.

[0329] Therefore, if the inter-frame prediction unit 205 controls whether or not to apply chain motion vectors based on the quantization parameters of the reference block, chain motion vectors can be applied effectively.

[0330] Specifically, if the quantization parameter of the reference block corresponding to the vector of the reference source that is the object of the chain motion vector generation is less than a predetermined quantization parameter, the inter-frame prediction unit 205 determines that the chain motion vector will not be applied to the decoded object block.

[0331] On the other hand, if the quantization parameter of the reference block corresponding to the vector of the reference source that is the object of the chain motion vector generation is greater than a predetermined quantization parameter, the inter-frame prediction unit 205 determines that the chain motion vector will not be applied to the decoding object block.

[0332] Here, the predetermined quantization parameter can be the quantization parameter of a decoded object block. Alternatively, the predetermined quantization parameter can be a decoded object stripe that includes the decoded object block. Or, the predetermined quantization parameter can be a decoded object image that includes the decoded object block.

[0333] As a variation, the inter-frame prediction unit 205 can determine whether a chain motion vector has been applied by checking whether the difference between the quantization parameter of the reference block and a predetermined quantization parameter is greater than a predetermined threshold. Here, the predetermined threshold can be 6, 12, 18, or 24.

[0334] As an example of modification, the inter-frame prediction unit 205 can control whether or not chain motion vector prediction is applied based on the POC.

[0335] As another variation, the inter-frame prediction unit 205 can control whether or not chained motion vector prediction is applied based on the depth of the image reference structure (the value of Temporal_id).

[0336] Specifically, the inter-frame prediction unit 205 may determine that chain motion vectors should not be applied if the Temporal_id of the reference image, which is set as the reference source for chain motion vectors, is less than a predetermined value. On the other hand, specifically, it may determine that chain motion vector prediction should be applied if the Temporal_id of the reference image, which is set as the reference source for chain motion vectors, is greater than or equal to a predetermined value.

[0337] The greater the depth of the image reference structure (the value of Temporal_id), the larger the quantization parameter offset is generally applied. Therefore, when the depth of the image reference structure (the value of Temporal_id) is above a predetermined value, by applying chained motion vector prediction, inter-frame prediction pixels can be generated from reference images with less coding degradation, thereby improving coding performance.

[0338] Conversely, when the depth of the image reference structure (the value of Temporal_id) is less than a predetermined value, the motion vector search processing time for chained motion vector prediction can be reduced by not applying chained motion vector prediction.

[0339] Here, the predetermined value can be obtained by multiplying the maximum value of Temporal_id by at least one of 0.5, 0.25, or 0.125, and then discarding the decimal part. Alternatively, the predetermined value can be a fixed value. For example, the predetermined value can be at least one of 0, 1, 2, 3, 4, and 5.

[0340] <7. Applications of Chain Motion Vector Prediction> This describes the technical features disclosed in Non-Patent Document 1 and Non-Patent Document 2, which apply chain motion vector prediction to mergers other than conventional mergers. (MMVD, GPM, and TM merged) First, the MMVD, GPM, and template matching merge (TM merge) methods disclosed in Non-Patent Document 1 and Non-Patent Document 2 construct merge lists in the same way as conventional merges. However, in Non-Patent Document 2, the pruning (trimming) of merge candidates when constructing these merge candidate lists is enhanced compared to conventional merges, making it difficult for these merge candidate lists to be identical to the merge candidate lists stored in conventional merges.

[0341] Therefore, in MMVD, GPM, and TM merging, without using the regular merging candidate list, if chained motion vector prediction is applied to the merging candidates of each of the merging candidate lists in MMVD, GPM, and TM merging, in addition to applying chained motion vector prediction to the merging candidates of the regular merging candidate list, then an improvement in coding efficiency can be expected.

[0342] Therefore, in addition to applying chained motion vector prediction to the merge candidate list for regular merging, the inter-frame prediction unit 205 can also apply chained motion vector prediction to the merge candidate list for merging mode motion vector difference.

[0343] In addition to applying chain motion vector prediction to the merge candidates in the merge candidate list for regular merging, the inter-frame prediction unit 205 can also apply chain motion vector prediction to the merge candidates in the merge candidate list for geometric segmentation mode.

[0344] In addition to applying chained motion vector prediction to the merge candidates in the merge candidate list for regular merging, the inter-frame prediction unit 205 can also apply chained motion vector prediction to the merge candidates in the merge candidate list for template matching merging.

[0345] Here, regarding the chain of motion vectors generated by motion vector difference for merging mode, geometric segmentation mode, and template matching merging, the inter-frame prediction unit 205 can apply the same pruning process, motion vector correction, and merging candidate (motion vector) rearrangement as in regular merging when storing the merging candidate list.

[0346] (Adaptive Motion Vector Prediction - Merging Mode) In the Adaptive Motion Vector Prediction-Merge mode disclosed in Non-Patent Documents 1 and 2, since the motion vectors of either List 0 (L0) or List 1 (L1) of the reference image list are derived through Adaptive Motion Vector Prediction (AMVP), i.e. through control information, if chained motion vector prediction is applied to the merge candidate list of the reference image list number of the application object for which chained motion vector prediction was not derived in AMVP, the number of merge candidate options that can be selected in the Adaptive Motion Vector Prediction-Merge mode increases, thereby improving the inter-frame prediction accuracy and the expected improvement in coding efficiency.

[0347] Therefore, in the adaptive motion vector prediction-merging mode, the inter-frame prediction unit 205 can apply chained motion vector prediction to merge candidates that are reference image list numbers not derived in the adaptive motion vector prediction.

[0348] Furthermore, in the adaptive motion vector prediction-merging mode, when chained motion vector prediction is applied to a merge candidate in the merge candidate list that is a reference image list number not derived in the adaptive motion vector prediction, the inter-frame prediction unit 205 may restrict the reference image of the motion vector derived in the adaptive motion vector prediction to not refer to the generated chained motion vector.

[0349] Here, regarding the chained motion vectors generated for the adaptive motion vector prediction-merging mode, the inter-frame prediction unit 205 can, in the same manner as conventional merging, apply the pruning process when storing the merging candidate list, the correction of motion vectors, and the rearrangement of merging candidates (motion vectors).

[0350] (Sub-block merging mode) In the affine merge disclosed in the aforementioned Non-Patent Documents 1 and 2, in order to derive the motion vector of each sub-block within the decoded object block in the affine transformation, such as... Figure 24 As shown, in the case of unidirectional prediction (either L0 or L1), two or three control point motion vectors cpMv are derived, and in the case of bidirectional prediction (both L0 and L1), four or six control point motion vectors cpMv are derived.

[0351] Furthermore, the motion vectors of each control point in L0 and L1 for both unidirectional and bidirectional predictions are referenced to the same reference image.

[0352] In affine merging, as disclosed in Non-Patent Document 1 and Non-Patent Document 2, these control point motion vectors are derived from merge candidates stored in a merge candidate list for affine merging.

[0353] The inter-frame prediction unit 205 can derive the merge candidates stored in the merge candidate list for affine merging in the same manner as the deriving method for merge candidates for affine merging disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0354] For example, if two or three control point motion vectors are stored in locations that are spatially adjacent to or close to the decoded object block, the inter-frame prediction unit 205 can add these motion vectors to the affine merging candidate list of the decoded object block.

[0355] Alternatively, if multiple motion vectors are stored in locations that are spatially adjacent to or close to the decoded object block, the inter-frame prediction unit 205 can generate control point motion vectors for the decoded object block based on these motion vectors and add them to the merge candidate list for affine merging.

[0356] Therefore, in affine merging mode, the inter-frame prediction unit 205 can recursively search for motion vectors for at least two control point motion vectors stored in the merging candidate list and add the derived motion vectors to each of the initial control point motion vectors to generate chained control point motion vectors, and store them in the merging candidate list.

[0357] Furthermore, when recursively searching for motion vectors for at least two control point motion vectors stored in the merge candidate list in affine merging mode, the inter-frame prediction unit 205 can restrict the reference image for the search object for each control point motion vector to the same reference image.

[0358] Alternatively, in affine merging mode, when recursively searching for motion vectors for at least two control point motion vectors stored in the merging candidate list for L0 and L1, the inter-frame prediction unit 205 may restrict the reference image for the search object of each control point motion vector in L0 and L1 to the same reference image.

[0359] Here, regarding the chain of control point motion vectors generated for affine merging, the inter-frame prediction unit 205 can apply the same pruning process, motion vector correction, and rearrangement of merging candidates (motion vectors) as in regular merging.

[0360] (Application of chained motion vector prediction for adaptive motion vector prediction) The inter-frame prediction unit 205 can be applied to chained motion vector prediction for adaptive motion vector prediction disclosed in Non-Patent Document 1. The specific method will be described below.

[0361] Adaptive motion vector prediction stores candidate motion vectors of the decoded object block at spatially or temporally adjacent locations in an adaptive motion vector candidate list (hereinafter referred to as the adaptive motion vector candidate list).

[0362] Here, as Figure 25 As shown, in adaptive motion vector prediction, unlike merging candidate lists, a list for storing motion vector candidates is constructed for each reference image stored in the reference image lists L0 and L1.

[0363] The method for deriving motion vector candidates (MVP: motion vector prediction) from the adaptive motion vector candidate list in the inter-frame prediction unit 205 is as follows.

[0364] First, the decoding unit 201 decodes the indices (ref_idx_l0, ref_idx_l1) of the reference images used to determine the respective candidates for deriving motion vectors from the reference image lists L0 and L1. The inter-frame prediction unit 205 then determines the reference images based on the values ​​of these indices.

[0365] Next, the decoding unit 201 decodes the flags (mvp_l0_flag, mvp_l1_flag) for deriving motion vector candidates from the motion vector candidate list corresponding to the determined reference images of L0 and L1. The inter-frame prediction unit 205 determines the motion vectors based on the values ​​of these flags.

[0366] Finally, the inter-frame prediction unit 205 adds the motion vector difference (MVD) of L0 and L1 respectively transmitted from the coding device to the derived MVP of L0 and L1 respectively, so as to finally derive the motion vectors of L0 and L1.

[0367] Here, in this invention, chained motion vector prediction is applied to generate motion vector candidates stored in the aforementioned adaptive motion vector candidate list.

[0368] Specifically, in adaptive motion vector prediction, as mentioned earlier, for each reference image, motion vectors referencing each reference image are stored. Therefore, a search is conducted for motion vectors that store motion vectors referencing each reference image at locations spatially or temporally adjacent to the decoded object block. Whether a motion vector is stored in the adaptive motion vector candidate list is determined by whether it references the reference image of the search object determined by ref_idx_l0 and ref_idx_l1.

[0369] That is, when the motion vector being searched is referenced to the reference image of the search object determined by ref_idx_l0 and ref_idx_l1, the inter-frame prediction unit 205 determines that it is stored in the adaptive motion vector candidate list; when it does not refer to the aforementioned reference image, it determines that it is not stored in the adaptive motion vector candidate list.

[0370] Here, the inter-frame prediction unit 205 retains motion vectors that were not stored in the adaptive motion vector candidate list during the motion vector search process and applies chained motion vector prediction to those motion vectors.

[0371] Then, as a result of applying chain motion vector prediction, the inter-frame prediction unit 205 determines whether to store the chain motion vector in the adaptive motion vector candidate list if the final derived motion vector (chain motion vector) is referenced to the reference image of the search object, and determines whether to store the chain motion vector in the adaptive motion vector candidate list if it is not referenced to the aforementioned reference image.

[0372] When there are two reference image lists (i.e., L0 and L1), the inter-frame prediction unit 205 can use the reference images in each of the reference image lists of L0 and L1 to derive candidates for motion vectors.

[0373] As described above, the inter-frame prediction unit 205 can apply chained motion vector prediction to the motion vector derived from the adaptive motion vector and derive a new motion vector.

[0374] Here, Non-Patent Document 1 discloses an Adaptive Motion Vector Resolution (AMVR) technique for quantizing the pixel precision of motion vectors as a method to reduce the amount of transmission coding required to construct the motion vector difference that ultimately derives from the adaptive motion vector.

[0375] When AMVR is active, the inter-frame prediction unit 205 can match (round down) the pixel precision of the final motion vector derived to the chained motion vector with the motion vector quantized by AMVR.

[0376] Alternatively, when AMVR is active, the inter-frame prediction unit 205 can match (round down) the pixel precision of the searched motion vector with the pixel precision of the motion vector quantized by AMVR during the recursive motion vector search process when deriving chained motion vectors.

[0377] Non-patent document 1 discloses a flag for controlling whether AMVR is applied on a block-by-block basis, and an index for determining the pixel precision of motion vectors using AMVR.

[0378] When the decoding unit 201 determines from these flags and indexes that AMVR is applied and the pixel precision of the motion vector is quantized, the inter-frame prediction unit 205 can change the pixel precision of the motion vector when applying chained motion vectors to motion vectors derived from adaptive motion vectors to search for new motion vectors to the pixel precision quantized by AMVR.

[0379] Alternatively, as a variation, the inter-frame prediction unit 205 may, during the search process for motion vectors predicted by chained motion vectors, not match the pixel precision of the motion vectors quantized by AMVR, but instead match the pixel precision of the motion vectors finally derived from the chained motion vector prediction with the pixel precision of the motion vectors quantized by AMVR.

[0380] (Application of chain motion vector prediction for bilateral merging mode) The inter-frame prediction unit 205 can be applied to chain motion vector prediction for the bilateral merging mode disclosed in Non-Patent Document 2. The specific method will be described below.

[0381] Similar to the aforementioned conventional merging mode, in the bilateral merging mode (BM), the inter-frame prediction unit 205 constructs a merging candidate list and derives motion vectors.

[0382] However, unlike the conventional merging mode, the bilateral merging mode stores two motion vectors that satisfy the following conditions: the reference targets of the reference images stored in the merging candidate lists of L0 and L1 point to different time directions (one is the past direction and the other is the future direction when viewed from the decoded object image including the decoded object block), and the distance from the decoded object image to each reference image is the same (BM condition). Two motion vectors that do not satisfy the BM condition are pruned and not stored.

[0383] In this invention, the inter-frame prediction unit 205 can maintain motion vectors that do not meet the BM condition, apply chained motion vector prediction to search for new motion vectors, and if a motion vector that meets the BM condition is found, it is stored in the merged candidate list.

[0384] As a variation, the inter-frame prediction unit 205 can construct a separate merge candidate list for storing new motion vectors that do not meet the BM conditions but are generated through chained motion vector prediction, in addition to storing only the merge candidate list that stores motion vectors that meet the BM conditions.

[0385] According to the image decoding apparatus of this embodiment, since the inter-frame prediction unit 205 derives motion vectors as motion vectors for the decoding target block, that is, it generates a motion vector by recursively searching for motion vectors stored in the decoding image buffer 207 for positions that are adjacent or close to the decoding target block in space or time and adding all the derived motion vectors, the options for motion vectors that can be selected in inter-frame prediction are increased, the accuracy of inter-frame prediction is improved, and the coding efficiency is improved.

[0386] The aforementioned image decoding device 200 can be a program that enables the computer to perform various functions (steps) and is implemented. Industrial availability

[0387] Furthermore, according to this embodiment, for example, since it is possible to improve the overall quality of service in dynamic image communication, it is possible to contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs): "Building resilient infrastructure, promoting sustainable industrialization and pursuing the expansion of innovation." Figure Labels

[0388] 200 Image Decoding Device 201 Decoding Department 202 Inverse Quantization Department 203 Inverse Transformation Unit 204 Intra-frame Prediction Unit 205 Inter-frame Prediction Unit 206 Adder 207 Decoding Image Buffer 210 Encoding Input Section 220 Image Output Unit

Claims

1. An image decoding apparatus, comprising: The decoding unit performs variable-length decoding on the encoded information and outputs quantization values ​​and control information. The inverse quantization unit performs inverse quantization on the quantized value and outputs the transformation coefficients; The inverse transform unit performs an inverse transform on the transform coefficients and outputs the predicted residual pixels; The intra-frame prediction unit generates intra-frame predicted pixels from the control information and the decoded pixels; Decode the image buffer to store the decoded pixels; The inter-frame prediction unit generates inter-frame prediction pixels from the control information and the decoded pixels stored in the decoded image buffer; as well as The adder adds at least one of the intra-frame predicted pixels and the inter-frame predicted pixels to the predicted residual pixels to generate the decoded pixels. The inter-frame prediction unit performs chained motion vector prediction, which generates chained motion vectors by re-searching for new motion vectors in a manner that limits the search to motion vectors located spatially or temporally adjacent to or close to the decoded object block in a merged candidate list of stored motion vector candidates, with the reference target being the motion vectors within the juxtaposed image stored in the decoded image buffer, and summing all derived motion vectors. The chained motion vectors are then exported as candidates for motion vectors of the decoded object block.

2. The image decoding apparatus according to claim 1, wherein the inter-frame prediction unit restricts the search area of ​​the recursive motion vector predicted by the chained motion vector to the area of ​​one row of a coding tree unit, the coding tree unit including a reference block having the same pixel position as the decoded object block in the juxtaposed image.

3. An image decoding method, comprising: Step A: Variable-length decoding of encoded information, outputting quantized values ​​and control information; Step B involves inverse quantization of the quantized value and outputting the transformation coefficients. Step C involves performing an inverse transform on the transform coefficients and outputting the predicted residual pixels; Step D: Generating intra-frame predicted pixels from the control information and the decoded pixels; Step E: Storing the decoded pixels in the decoded image buffer; Step F: Generating inter-frame prediction pixels from the control information and the decoded pixels stored in the decoded image buffer; as well as Step G involves adding at least one of the intra-frame predicted pixels and the inter-frame predicted pixels to the predicted residual pixels to generate the decoded pixels. In step F, chain motion vector prediction is performed. The chain motion vector prediction is achieved by re-searching for new motion vectors in a manner that limits the search to motion vectors located in spatially or temporally adjacent to or close to the decoded object block, with the reference target being the motion vectors within the juxtaposed image stored in the decoded image buffer, and summing all the derived motion vectors to generate chain motion vectors. The chain motion vectors are then derived as candidates for motion vectors of the decoded object block.

4. A program that enables a computer to function as an image decoding device. The image decoding device includes: The decoding unit performs variable-length decoding on the encoded information and outputs quantization values ​​and control information. The inverse quantization unit performs inverse quantization on the quantized value and outputs the transformation coefficients; The inverse transform unit performs an inverse transform on the transform coefficients and outputs the predicted residual pixels; The intra-frame prediction unit generates intra-frame predicted pixels from the control information and the decoded pixels; Decode the image buffer to store the decoded pixels; The inter-frame prediction unit generates inter-frame prediction pixels from the control information and the decoded pixels stored in the decoded image buffer; as well as The adder adds at least one of the intra-frame predicted pixels and the inter-frame predicted pixels to the predicted residual pixels to generate the decoded pixels. The inter-frame prediction unit performs chained motion vector prediction, which generates chained motion vectors by re-searching for new motion vectors in a manner that limits the search to motion vectors located spatially or temporally adjacent to or close to the decoded object block in a merged candidate list of stored motion vector candidates, with the reference target being the motion vectors within the juxtaposed image stored in the decoded image buffer, and summing all derived motion vectors. The chained motion vectors are then exported as candidates for motion vectors of the decoded object block.

Images