Image decoding device, image decoding method, and program

Abstract

An image decoding device (200) according to the present invention is provided with: a first intra prediction unit (204) that generates first predicted pixels on the basis of decoded pixels and control information; a second intra prediction unit (205) that, on the basis of the decoded pixels and the control information, generates second prediction pixels from two or more block vectors having integer pixel accuracy or decimal pixel accuracy for the block to be decoded; a storage unit (208) that stores the decoded pixels; an inter prediction unit (206) that generates third prediction pixels on the basis of the stored decoded pixels and the control information; and an adder (207) that adds the prediction residual with the first to third prediction pixels to obtain a decoded pixel.

Description

Technical Field

[0001] This invention relates to an image decoding apparatus, an image decoding method, and a program. Background Technology

[0002] Non-patent documents 1 and 2 disclose Intra Block Copy (IBC) and Intra Template Matcing Prediction (IntraTMP).

[0003] IBC and IntraTMP use the decoded pixel region of the target frame as the reference pixel for the predicted pixel of the target block.

[0004] The difference between IBC and IntraTMP is that IBC explicitly sends signaling for control signals used to decode the block vector representing the coordinates of the reference, while IntraTMP does not send signaling for control signals used to decode the block vector by searching for similar regions using neighboring pixels as templates.

[0005] Although the signaling for block vectors differs, they are all like... Figure 2 As shown, pixels are copied from a reference block indicated by the block vector of the decoded target block and used as predicted pixels.

[0006] Existing technical documents Non-patent literature Non-patent literature 1: ITU-T H.266 / VVC Non-Patent Literature 2: M. Coban et al., “Algorithm description of EnhancedCompression Model 7 (ECM 7)”, JVET-AB2025, 2022. Summary of the Invention

[0007] The problem the invention aims to solve However, in Non-Patent Documents 1 and 2, since the block vector is limited to one, there is room for improvement in terms of coding efficiency. Furthermore, in Non-Patent Documents 1 and 2, since only blocks are copied, there is room for improvement in terms of coding efficiency.

[0008] Therefore, the present invention was made in view of the above-mentioned problems, and its object is to provide an image decoding device, image decoding method and program with high encoding efficiency.

[0009] Solution for solving the problem The first feature of the present invention is an image decoding apparatus, comprising: a decoding unit that decodes control information and quantization values; an inverse quantization unit that inversely quantizes the aforementioned quantization values ​​and uses them as transform coefficients; an inverse transform unit that inversely transforms the aforementioned transform coefficients and uses them as prediction residuals; a first intra-frame prediction unit that generates a first predicted pixel based on the decoded pixels and the aforementioned control information; a second intra-frame prediction unit that generates a second predicted pixel based on the aforementioned decoded pixels and the aforementioned control information from two or more block vectors with integer pixel precision or fractional pixel precision for the decoded target block; a storage unit that stores the aforementioned decoded pixels; an inter-frame prediction unit that generates a third predicted pixel based on the aforementioned stored decoded pixels and the aforementioned control information; and an adder that adds the aforementioned prediction residuals to the aforementioned first to third predicted pixels to obtain the aforementioned decoded pixels.

[0010] The second feature of the present invention is an image decoding method, comprising: a step of decoding control information and quantization values; a step of inverse quantization of the aforementioned quantization values ​​and using them as transform coefficients; a step of inverse transforming the aforementioned transform coefficients and using them as prediction residuals; a step of generating a first predicted pixel based on the decoded pixel and the aforementioned control information; a step of generating a second predicted pixel based on the aforementioned decoded pixel and the aforementioned control information from two or more block vectors with integer pixel precision or fractional pixel precision for the decoded target block; a step of storing the aforementioned decoded pixel; a step of generating a third predicted pixel based on the aforementioned stored decoded pixel and the aforementioned control information; and a step of adding the aforementioned prediction residuals to the aforementioned first to third predicted pixels to obtain the aforementioned decoded pixel.

[0011] The third feature of the present invention is a program that enables a computer to function as an image decoding device, the image decoding device comprising: a decoding unit that decodes control information and quantization values; an inverse quantization unit that inverse quantizes the quantization values ​​and uses them as transform coefficients; an inverse transform unit that inverse transforms the transform coefficients and uses them as prediction residuals; a first intra-frame prediction unit that generates a first predicted pixel based on the decoded pixel and the control information; a second intra-frame prediction unit that generates a second predicted pixel based on the decoded pixel and the control information from two or more block vectors with integer pixel precision or fractional pixel precision for the decoded target block; a storage unit that stores the decoded pixel; an inter-frame prediction unit that generates a third predicted pixel based on the stored decoded pixel and the control information; and an adder that adds the prediction residual to the first to third predicted pixels to obtain the decoded pixel.

[0012] Invention Effects According to the present invention, an image decoding apparatus, an image decoding method, and a program with high encoding efficiency can be provided. Attached Figure Description

[0013] Figure 1 This is a diagram illustrating an example of the functional blocks of an image decoding apparatus 200 according to one embodiment.

[0014] Figure 2 This diagram is used to illustrate IBC and IntraTMP.

[0015] Figure 3 This is a diagram illustrating an example of a functional block of the second intra-frame prediction unit 205 of an image decoding apparatus 200 according to an embodiment.

[0016] Figure 4 This is a diagram illustrating an example of a BV derivation method in the BV derivation unit 205A of the second intra-frame prediction unit 205 of an image decoding apparatus 200 according to one embodiment.

[0017] Figure 5 This is a diagram illustrating an example of a BV derivation method in the BV derivation unit 205A of the second intra-frame prediction unit 205 of an image decoding apparatus 200 according to one embodiment.

[0018] Figure 6 This is a diagram illustrating an example of a BV derivation method in the BV derivation unit 205A of the second intra-frame prediction unit 205 of an image decoding apparatus 200 according to one embodiment.

[0019] Figure 7 This is a diagram illustrating an example of a BV derivation method in the BV derivation unit 205A of the second intra-frame prediction unit 205 of an image decoding apparatus 200 according to one embodiment.

[0020] Figure 8 This is a diagram illustrating an example of a BV derivation method in the BV derivation unit 205A of the second intra-frame prediction unit 205 of an image decoding apparatus 200 according to one embodiment.

[0021] Figure 9 This is a diagram illustrating an example of a BV derivation method via IBC GPM in the BV derivation unit 205A of the second intra-frame prediction unit 205 of an image decoding apparatus 200 according to one embodiment.

[0022] Figure 10 This is a diagram illustrating an example of a method for synthesizing predicted pixels of a decoded target block in a second intra-frame prediction pixel generation unit 205B of an image decoding apparatus 200 according to an embodiment.

[0023] Figure 11This is a diagram illustrating an example of a method for synthesizing predicted pixels of a decoded target block in a second intra-frame prediction pixel generation unit 205B of an image decoding apparatus 200 according to an embodiment.

[0024] Figure 12 This is a diagram illustrating an example of a method for synthesizing predicted pixels of a decoded target block in a second intra-frame prediction pixel generation unit 205B of an image decoding apparatus 200 according to an embodiment.

[0025] Figure 13 This is a diagram illustrating an example of a method for synthesizing predicted pixels of a decoded target block in a second intra-frame prediction pixel generation unit 205B of an image decoding apparatus 200 according to an embodiment.

[0026] Figure 14 This is a diagram illustrating an example of a method for synthesizing predicted pixels of a decoded target block in a second intra-frame prediction pixel generation unit 205B of an image decoding apparatus 200 according to an embodiment.

[0027] Figure 15 This is a flowchart illustrating an example of a decoding control method for predetermined flags associated with IBC on a sequence-by-sequence basis.

[0028] Figure 16 This is a diagram illustrating one example of various symbols associated with IBC.

[0029] Figure 17 This is a flowchart illustrating an example of a decoding method in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment, which controls whether a dual-predictive IBC flag can be applied.

[0030] Figure 18 This is a flowchart illustrating an example of a decoding method in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment, which controls whether a dual-predictive IBC flag can be applied.

[0031] Figure 19 This is a flowchart illustrating an example of a decoding method for pred_mode_ibc_flag, where pred_mode_ibc_flag is a flag in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment, which controls whether to apply IBC in units of decoding target blocks.

[0032] Figure 20 This is a flowchart illustrating an example of a method for selecting the mode of an IBC in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment.

[0033] Figure 21 This is a flowchart illustrating an example of a decoding method in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment, which controls a flag indicating whether dual-predictive IBC merging is valid on a per-decoding-target-block basis.

[0034] Figure 22 This is a flowchart illustrating an example of a method for decoding control information related to IBC merging in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment.

[0035] Figure 23 This is a flowchart illustrating an example of a method for decoding control information related to IBC merging in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment.

[0036] Figure 24 This is a flowchart illustrating an example of a decoding method in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment, for controlling the flag indicating whether IBC BVP / merging is valid on a per-decoding target block basis.

[0037] Figure 25 This is a flowchart illustrating an example of a decoding method for control information related to IBC BVP / merging in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment.

[0038] Figure 26 This is a flowchart illustrating an example of a decoding method for controlling information related to MBVD in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment, which corrects the BV of the IBC.

[0039] Figure 27 This is a flowchart illustrating an example of a method for applying second intra-frame prediction in a block-by-block manner in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment.

[0040] Figure 28 This is a flowchart illustrating the application of a correction BV to a BV in the decoding unit 201 of an image decoding apparatus 200 according to one embodiment.

[0041] Figure 29 This is a flowchart illustrating an example of a decoding method in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment, for controlling the flag indicating whether the dual-predictive IBC BVP is valid on a per-decoding target block basis.

[0042] Figure 30 This is a flowchart illustrating an example of a decoding method for control information related to dual-predictive IBC BVP in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment.

[0043] Figure 31 This is a diagram illustrating an example of a binarization method for the IBC merge index in the decoding unit 201 of an image decoding apparatus 200 according to an embodiment. Detailed Implementation

[0044] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Furthermore, the constituent elements in the following embodiments can be appropriately replaced with existing constituent elements, and various modifications are possible, including combinations with other existing constituent elements. Therefore, the description of the following embodiments does not limit the scope of the invention as described in the claims.

[0045] <First Implementation Method> The following is for reference Figures 1 to 29 The image decoding apparatus 200 according to this embodiment will be described. Figure 1 This is a diagram illustrating an example of the functional blocks of the image decoding apparatus 200 according to this embodiment.

[0046] like Figure 1 As shown, the image decoding device 200 includes a code input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transform unit 203, a first intra-frame prediction unit 204, a second intra-frame prediction unit 205, an inter-frame prediction unit 206, an adder 207, a storage unit 208, and an image output unit 220.

[0047] The code input unit 210 is configured to acquire code information encoded by the image encoding device.

[0048] The decoding unit 201 is configured to decode control information and quantization values ​​from code information input from the code input unit 210. For example, the decoding unit 201 is configured to output control information and quantization values ​​by performing variable-length decoding on the code information.

[0049] Here, the quantization value is sent to the inverse quantization unit 202, and control information is sent to the first intra-frame prediction unit 204, the second intra-frame prediction unit 205, and the inter-frame prediction unit 206. Furthermore, this control information includes information required for controlling the first intra-frame prediction unit 204, the second intra-frame prediction unit 205, and the inter-frame prediction unit 206, and may include header information such as sequence parameter sets, image parameter sets, image headers, or slice headers.

[0050] The inverse quantization unit 202 is configured to inverse quantize the quantized value sent from the decoding unit 201 and use it as a transform coefficient. This transform coefficient is then sent to the inverse transform unit 203.

[0051] The inverse transform unit 203 is configured to perform an inverse transform on the transform coefficients sent from the inverse quantization unit 202 and use them as prediction residuals. These prediction residuals are then sent to the adder 207.

[0052] The first intra-frame prediction unit 204 is configured to generate a first prediction pixel for addition with the prediction residual by the adder 207, based on the decoded pixel obtained via the adder 207 and the control information decoded by the decoding unit 201. The first prediction pixel is sent to the adder 207.

[0053] The inter-frame prediction unit 206 is configured to generate a third prediction pixel based on the decoded pixel obtained from the reference storage unit 208 and the control information decoded by the decoding unit 201, for addition to the prediction residual using the adder 207. This third prediction pixel is then sent to the adder 207.

[0054] The storage unit 208 is configured to cumulatively store decoded pixels sent from the adder 207. These decoded pixels receive a reference from the inter-frame prediction unit 206 via the storage unit 208.

[0055] The adder 207 is configured to add the prediction residual sent from the inverse transform unit 203 to any one of the first to third prediction pixels sent from the first intra-frame prediction unit 204, the second intra-frame prediction unit 205, and the inter-frame prediction unit 206 to obtain a decoded pixel. This decoded pixel is then sent to the image output unit 220, the storage unit 208, and the inter-frame prediction unit 206.

[0056] (Second intra-frame prediction unit 205) Hereinafter, an example of the prediction method performed by the second intra-frame prediction unit 205 will be described.

[0057] <Derivation of block vectors> The function of the second intra-frame prediction unit 205 is to address issues such as... Figure 2 The decoded target block shown derives more than one block vector (hereinafter referred to as BV: Block Vector) and predicts the pixels of the block referenced by the BV (second intra-frame prediction) to predict the decoded target block with high accuracy in the subsequent adder 207.

[0058] Examples of this second intra-frame prediction include intra-block copy (hereinafter referred to as IBC) and intra-template matching prediction (hereinafter referred to as IntraTMP) disclosed in Non-Patent Documents 1 and 2.

[0059] This second intra-frame prediction can be applied to cases where the decoded target block references other regions in the BV, but IBC will be used as an example of the second intra-frame prediction below. Furthermore, in other examples, there are modes such as IntraTMP that reference predicted pixels intra-frame.

[0060] exist Figure 3 The image shows an example of a functional block of the intra-frame prediction unit 205. For example... Figure 3 As shown, the second intra-frame prediction unit 205 includes a block vector derivation unit (hereinafter referred to as BV derivation unit) 205A and a second intra-frame prediction pixel generation unit 205B.

[0061] The BV export unit 205A is configured to export one or more BVs for the decoded target block.

[0062] In this embodiment, the BV derivation unit 205A can derive more than one BV using at least one of the following four methods as BV derivation methods for IBC. Furthermore, the method used to derive the BV will be explained in detail in the signaling (decoding processing of control information) section of the decoding unit 201 described later.

[0063] A. IBC Block Vector Prediction (hereinafter referred to as IBC BVP) B. IBC Merger C. IBC BVP / Merger D. IBC GPM Here, although methods A and B for deriving the above-mentioned BV are disclosed in Non-Patent Documents 1 and 2, the methods disclosed in Non-Patent Documents 1 and 2 are all based on the premise of deriving a BV.

[0064] The objective (effect) of this invention is to improve coding efficiency by increasing the precision of the second intra-frame prediction derived from two or more BVs. Therefore, in the descriptions relating to BV derivation methods A and B below, the differences in the case of deriving two or more BVs will be emphasized. Furthermore, the aforementioned BV derivation methods C and D are novel methods based on the premise of deriving two or more BVs. Additionally, as an example of the second intra-frame prediction (multi-prediction IBC or multi-prediction IBC) derived from these two or more BVs, the IBC derived from two BVs (double-prediction IBC or double-prediction IBC) will be described below.

[0065] (A. IBC BVP) The following uses Figures 4 to 6 An example of the BV export method via IBC BVP in BV export section 205A will be explained.

[0066] Figure 4 An example of BV including block vector prediction (BVP) and block vector difference (BVD) is shown.

[0067] In IBC BVP, the BV derivation unit 205A derives Block Vector Prediction (BVP) and Block Vector Difference (BVD) based on the values ​​of control information related to BVP and BVD decoded by the decoding unit 201, and then derives BV. Here, the control information related to BVP and BVD will be described in detail in the description of the signaling in the decoding unit 201.

[0068] The following is a summary of the methods for exporting BVP in IBC BVP.

[0069] In step S1, the BV output section 205A confirms... Figure 5 Check whether the adjacent blocks of the target block have been decoded. If the adjacent blocks have been decoded, check whether BV is available.

[0070] In step S2, the BV derivation unit 205A takes the BV of the adjacent block having BV as... Figure 6 The BVP candidates shown are registered in the list (hereinafter referred to as the BVP candidate list).

[0071] In step S3, the BV exporting unit 205A selects a BVP from the BVP candidate list based on the value of the control information (IBCBVP flag or IBC BVP index) related to the BVP decoded by the decoding unit 201.

[0072] Here, the confirmation in step S1 regarding whether decoding has been completed and the number (position) of adjacent blocks with BV can be the same as in non-patent document 1, and can be of two types ( Figure 5 The "A" and "L" can also be in five types, similar to non-patent literature 2. Figure 5 (The "A", "L", "AR", "LB" and "AL").

[0073] The order of adjacent blocks can be the same as in Non-Patent Documents 1 and 2, respectively. Here, “A”, “L”, “AR”, “LB” and “AL” are “Above”, “Left”, “AboveRight”, “LeftBottom” and “AboveLeft” in Non-Patent Documents 1 and 2, respectively.

[0074] Here, the BV derivation unit 205A can use BVs of neighboring blocks (referred to as spatial BVPs in Non-Patent Document 2), BVs used in the past (referred to as historical BVPs in Non-Patent Document 2), etc., as BVP candidates.

[0075] In addition, in step S1, if the newly confirmed unavailability BV is the same as the BV that has already been confirmed as available, it can be determined that the newly confirmed unavailability BV is unavailable.

[0076] Furthermore, a predetermined rearrangement method can be used to rearrange the registration order of the BVs (BVP candidates) determined to be available in step S1 in the BVP candidate list. The details of this rearrangement method are described later.

[0077] The following is a summary of the methods for exporting BVDs in IBC BVP.

[0078] In step S1, the BV derivation unit 205A derives the BVD based on control information related to the BVD.

[0079] In step S2, the BV derivation unit 205A rounds the BVD derived in step S1 based on control information related to the pixel accuracy of the BVD.

[0080] Here, the method for deriving the BVD based on the control information related to the BVD can be configured in the same way as in Non-Patent Document 1. Specifically, the decoding unit 201 decodes (or estimates) the values ​​of the control information related to the size (absolute value) of the BVD and the code of the BVD, and the BVD derivation unit 205A derives the BVD from these values.

[0081] Finally, the BV export unit 205A adds the exported BVP to BVD to export BV.

[0082] The above is a summary of the methods for exporting BV in IBC BVP.

[0083] As another example, in IBC BVP, when exporting more than two BVs, the BV exporting unit 205A can export the BVP and BVD required in the export of each BV by the following method.

[0084] In step S1, the BV output section 205A confirms... Figure 5 Check whether the adjacent blocks of the target block have been decoded. If the adjacent blocks have been decoded, check whether BV is available.

[0085] In step S2, the BV derivation unit 205A takes the BV of the adjacent block having BV as... Figure 6 The BVP candidates shown are registered in the list (hereinafter referred to as the BVP candidate list).

[0086] In step S3, the BV exporting unit 205A selects two or more different BVPs from the BVP candidate list based on the values ​​of control information (IBC BVP flags or IBC BVP indexes) related to two or more different BVPs decoded by the decoding unit 201.

[0087] Furthermore, when the BV exporting unit 205A exports BVs corresponding to more than one index for the BVP candidate list, the second and subsequent indexes are preferably relative indexes starting from the index immediately preceding them. Details are described later.

[0088] Here, the confirmation in step S1 regarding whether decoding has been completed and the number (position) of adjacent blocks with BV can be the same as in non-patent document 1, and can be of two types ( Figure 5 The "A" and "L" can also be in five types, similar to non-patent literature 2. Figure 5 (The "A", "L", "AR", "LB" and "AL").

[0089] The order of adjacent blocks can be the same as in Non-Patent Documents 1 and 2, respectively. Here, “A”, “L”, “AR”, “LB” and “AL” are “Above”, “Left”, “AboveRight”, “LeftBottom” and “AboveLeft” in Non-Patent Documents 1 and 2, respectively.

[0090] Here, the BV derivation unit 205A can use BVs of neighboring blocks (referred to as spatial BVPs in Non-Patent Document 2), BVs used in the past (referred to as historical BVPs in Non-Patent Document 2), etc., as BVP candidates.

[0091] Furthermore, in step S1, if the newly confirmed usable BV is the same as the already confirmed usable BV, the BV output unit 205A can determine that the newly confirmed usable BV is unusable.

[0092] Furthermore, the BV exporting unit 205A can use a predetermined rearrangement method to rearrange the registration order of the BVs (BVPs) determined to be available in step S1 in the BVP candidate list. The details of this rearrangement method are described later.

[0093] The following is a summary of the methods for exporting BVDs in IBC BVP.

[0094] In step S1, the BV derivation unit 205A derives the BVD based on control information related to two or more different BVDs.

[0095] In step S2, the BV derivation unit 205A rounds two or more different BVDs derived in step S1 based on control information related to the pixel accuracy of the BVD.

[0096] Here, the method for deriving the BVD based on control information related to the BVD can be configured in the same way as in Non-Patent Document 1. Specifically, the decoding unit 201 decodes or estimates the values ​​of control information related to the size (absolute value) of the BVD and the code of the BVD, and the BVD derivation unit 205A derives the BVD from these values.

[0097] Finally, the BV derivation unit 205A adds the two or more different BVPs derived from BVD to derive two or more different BVs.

[0098] The above is a summary of the methods for deriving BVs in IBC BVPs with two or more BVs (double predictions or multiple predictions).

[0099] (B. IBC merger) The following uses Figures 4 to 7 An example of the BV derivation method by IBC merging in BV derivation section 205A will be explained.

[0100] In IBC merging, unlike IBC BVP, BV export section 205A does not export BVD, but only exports BVP, and exports it as BV.

[0101] The following is a summary of the methods for deriving BV in IBC merging.

[0102] In step S1, the BV output section 205A confirms... Figure 5 Check whether the adjacent blocks of the target block have been decoded. If the adjacent blocks have been decoded, check whether BV is available.

[0103] In step S2, the BV derivation unit 205A takes the BV of the adjacent block having BV as... Figure 6 or Figure 7 The BVP candidates (or merge candidates) shown are registered in a list (hereinafter referred to as the BVP candidate list or merge candidate list).

[0104] In step S3, the BV exporting unit 205A selects a BVP from the BVP candidate list based on the value of the control information (IBC merge index) related to the BVP decoded by the decoding unit 201, and exports the BVP as a BV.

[0105] Here, the confirmation in step S1 regarding whether decoding has been completed and the number (position) of adjacent blocks with BV can be the same as in non-patent document 1, and can be of two types ( Figure 5The "A" and "L" can also be in five types, similar to non-patent literature 2. Figure 5 (The "A", "L", "AR", "LB" and "AL").

[0106] The order of adjacent blocks can be the same as in Non-Patent Documents 1 and 2, respectively. Here, “A”, “L”, “AR”, “LB” and “AL” are “Above”, “Left”, “AboveRight”, “LeftBottom” and “AboveLeft” in Non-Patent Documents 1 and 2, respectively.

[0107] Here, BVP candidates can use BV of neighboring blocks (referred to as spatial merging in Non-Patent Document 2), BV used in the past (referred to as historical merging in Non-Patent Document 2), average BV (referred to as pairwise average merging in Non-Patent Document 2), etc.

[0108] Furthermore, in step S1, if the newly confirmed usable BV is the same as the already confirmed usable BV, the BV output unit 205A can determine that the newly confirmed usable BV is unusable.

[0109] Furthermore, the BV exporting unit 205A can use a predetermined rearrangement method to rearrange the registration order of the BVs (BVPs) determined to be available in step S1 in the BVP candidate list. The details of this rearrangement method are described later.

[0110] The above is a summary of the methods for deriving BV in IBC merging.

[0111] Here, when more than two BVs are exported in IBC merging, the BV exporting unit 205A can export more than two BVs in the following way.

[0112] As an example, such as Figure 6 As shown, the BV export unit 205A can select two or more BVP candidates from the BVP candidate list based on the values ​​of two or more control information (IBC merge index) of the BVP candidates (or merged candidates) in the BVP candidate list that is instructed to be decoded by the decoding unit 201, and export them as BV respectively.

[0113] Furthermore, when the BV exporting unit 205A exports BVs corresponding to more than one index for the BVP candidate list, the second and subsequent indexes are preferably relative indexes starting from the index immediately preceding them. Details are described later.

[0114] Alternatively, the BV derivation unit 205A can derive a BV by consistently selecting one or more high-order bits from the BVP candidate list. In this case, it is desirable for the BV derivation unit 205A to perform a rearrangement of the BVP candidates within the BVP candidate list, as described later. With this configuration, by consistently selecting a subset of two or more BVs, the amount of code required for indexing to derive more than two BVs is eliminated, thus improving coding efficiency.

[0115] As other examples, such as Figure 7 As shown, the BV export unit 205A can be configured with a BVP candidate list corresponding to the number of BVs held in the decoding target block, and then export two or more BVs through an index.

[0116] For example, when multiple BVs are held as BVP candidates in neighboring blocks, the BV derivation unit 205A registers each BV in its corresponding BVP candidate list. Thus, by pre-constructing multiple BVP candidate lists and sharing an index, more than one BV can be derived from a single index. For instance, if the decoded target block holds at most two BVs, the BV derivation unit 205A can prepare two lists (e.g., the same L0 list and L1 list as in normal intra-frame prediction), deriving a BV corresponding to the index from the L0 list and another BV corresponding to the index from the L1 list. If the BV corresponding to the index exists only in one of the L0 or L1 lists, the BV derivation unit 205A can use that one.

[0117] (C. IBC BVP / Merger) The following uses Figures 4 to 6 as well as Figure 8 An example of the BV export method in BV export section 205A, which is a BV export method using IBC BVP / merging, will be explained.

[0118] The IBC BVP / merge method derives the first BV based on the BV derivation method in the IBC BVP described above, and then derives another (or more remaining) BV based on the BV derivation method in the IBC Merge described above, to derive two (or more) BVs. The specific derivation method is as follows.

[0119] First, in the IBC BVP / merging, the BV derivation unit 205A derives the BVP and BVD based on the values ​​of the control information related to each of the BVP and BVD decoded by the decoding unit 201 using the same derivation method as the IBC BVP, and adds these derived BVP and BVD to derive the first BV.

[0120] Secondly, in IBC BVP / merging, the BV exporting unit 205A selects a BVP from the BVP candidate list and exports the BVP as the second BV based on the value of the control information (IBC merge index) related to the BVP decoded by the decoding unit 201 using the same exporting method as IBC BVP.

[0121] As other examples, such as Figure 8 As shown, the BV exporting unit 205A can create a template (i.e., a predetermined neighboring pixel of the reference block of the BV reference destination) based on the first exported BV, and export the BVP candidate (IBC merge candidate) with the highest similarity (minimum template cost) as the second BV of IBC BVP / merging by evaluating the similarity between each BVP candidate (each IBC merge candidate) in the IBC merge BVP candidate list and the template (a predetermined neighboring pixel of the reference block of the reference destination).

[0122] Alternatively, as another example, the BV exporting unit 205A may select a BVP from the BVP candidate list in IBC merging based on the value of the control information (IBC merge index) related to the BVP decoded by the decoding unit 201, and export the BVP as a second BV.

[0123] Here, the BV derivation unit 205A can use the template of the decoded target block, rather than the template of the BV derived from the IBC BVP, as the target for the similarity evaluation of each IBC merging candidate with the template.

[0124] Furthermore, the BV export unit 205A can use methods such as Sum of Squared Error (SSE) and Sum of Absolute Difference (SAD) in the evaluation of template similarity (comparison of template costs).

[0125] Additionally, the predetermined pixels of the template created in the template similarity evaluation can be decoded pixels of a 1-pixel row adjacent to the left or top of each block.

[0126] When there are no decoded pixels adjacent to the left or top at the image boundary or slice boundary, the BV export unit 205A can use the decoded pixels at the available locations to perform template similarity evaluation.

[0127] In addition, when there are two or more BVs for export targets, the BV exporting unit 205A can export BVP candidates in the order of high template similarity (order of low template cost).

[0128] The more detailed steps for deriving BV via IBC BVP / merging are as follows.

[0129] The following is a summary of the BVP export method based on the BV export method via IBC BVP in IBC BVP / merging.

[0130] In step S1, the BV output section 205A confirms... Figure 5 Check whether the adjacent blocks of the target block have been decoded. If the adjacent blocks have been decoded, check whether BV is available.

[0131] In step S2, the BV derivation unit 205A takes the BV of the adjacent block having BV as... Figure 6 The BVP candidates shown are registered in the list (hereinafter referred to as the BVP candidate list).

[0132] In step S3, the BV exporting unit 205A selects a BVP from the BVP candidate list based on the value of the control information (IBCBVP flag or IBC BVP index) related to the BVP decoded by the decoding unit 201.

[0133] Here, the confirmation in step S1 regarding whether decoding has been completed and the number (position) of adjacent blocks with BV can be the same as in non-patent document 1, and can be of two types ( Figure 5 The "A" and "L" can also be in five types, similar to non-patent literature 2. Figure 5 (The "A", "L", "AR", "LB" and "AL").

[0134] The order of adjacent blocks can be the same as in Non-Patent Documents 1 and 2, respectively. Here, “A”, “L”, “AR”, “LB” and “AL” are “Above”, “Left”, “AboveRight”, “LeftBottom” and “AboveLeft” in Non-Patent Documents 1 and 2, respectively.

[0135] Here, the BV derivation unit 205A can use BVs of neighboring blocks (referred to as spatial BVPs in Non-Patent Document 2), BVs used in the past (referred to as historical BVPs in Non-Patent Document 2), etc., as BVP candidates.

[0136] Furthermore, in step S1, if the newly confirmed usable BV is the same as the already confirmed usable BV, the BV output unit 205A can determine that the newly confirmed usable BV is unusable.

[0137] Furthermore, the BV exporting unit 205A can use a predetermined rearrangement method to rearrange the registration order of the BVs (BVPs) determined to be available in step S1 in the BVP candidate list. The details of this rearrangement method are described later.

[0138] The following is a summary of the BVD derivation method based on the BV derivation method via IBC BVP in IBC BVP / merging.

[0139] In step S1, the BV derivation unit 205A derives the BVD based on control information related to the BVD.

[0140] In step S2, the BV derivation unit 205A rounds the BVD derived in step S1 based on control information related to the pixel accuracy of the BVD.

[0141] Here, the method for deriving the BVD based on control information related to the BVD can be configured in the same way as in Non-Patent Document 1. Specifically, the decoding unit 201 decodes or estimates the values ​​of control information related to the size (absolute value) of the BVD and the code of the BVD, and the BVD derivation unit 205A derives the BVD from these values.

[0142] The BVD export unit 205A adds the exported BVP to the BVD to export the first BV.

[0143] The following is a summary of the second BV export method in IBC BVP / merging, which is based on the export method of BV via IBC.

[0144] Furthermore, in IBC BVP / merge, in the BV derivation of IBC BVP and the BV derivation of IBC merge, the BV derivation unit 205A can construct two different BV candidate lists (i.e., one is the BVP candidate list for IBC BVP, and the other is the merge candidate list for IBC BVP / merge).

[0145] As another example, in order to derive a second BV, the BV deriving unit 205A can derive the second BV by evaluating the similarity of the template based on the first BV, as follows.

[0146] In step S1, the BV exporting unit 205A uses the neighboring pixels of the first BV reference block or the decoding target block as a template.

[0147] In step S2, the BV exporting unit 205A evaluates the similarity between the template exported in step S1 and the corresponding pixels in the blocks of each BVP candidate reference in the BVP candidate list.

[0148] In step S3, the BV exporting unit 205A selects the BVP with the highest similarity and exports the BVP as the second BV.

[0149] Here, the BV derivation unit 205A does not include BVP candidates that are consistent with the first BV in the BVP candidate list of step S2 in the similarity evaluation target.

[0150] Alternatively, the BV exporting unit 205A may not register a BVP candidate that is consistent with the first BV in the BVP candidate list in step S2 during the construction phase of the BVP candidate list.

[0151] Alternatively, in step S3, the BV exporting unit 205A may rearrange the registration order of the BVP candidates in the BVP candidate list according to the order of high similarity, select a BVP based on the value of the control information (IBC merge index) related to the BVP decoded by the decoding unit 201, and export the BVP as a BV.

[0152] The above is a summary of the methods for exporting BV in IBC BVP / merging.

[0153] (D. IBC GPM) The following uses Figure 9 An example of the BV export method via IBC GPM in BV export section 205A will be explained.

[0154] In IBC GPM, such as Figure 9 As shown, the BV derivation unit 205A, similar to the Geometric Partitioning Mode (GPM) disclosed in Non-Patent Document 1, utilizes an arbitrary straight line selected from candidates of straight lines (geometric partitioning lines) of different multiple patterns. Figure 9 The solid line in the diagram divides the target block into two parts. Then, for each segmented region, one or more different BVs are derived. The reference destination (pixels of the reference block) of each BV is weighted and averaged according to its distance from the dividing line.

[0155] Here, the BV exporting unit 205A can be combined with the aforementioned IBC to export the BV required for the IBC GPM.

[0156] In addition, the BV derivation unit 205A can determine the candidates for the dividing line in the IBC GPM based on the control information for determining the candidates for the dividing line of the GPM disclosed in Non-Patent Document 1 or 2.

[0157] <Pixel precision> Furthermore, the BV export unit 205A can set the pixel precision of the exported BV in either integer pixel precision or fractional pixel precision.

[0158] IBC was designed as a coding technique for screen images, but when applied to natural images captured by a camera, it can improve prediction accuracy by setting the pixel precision of BV to a fraction of a pixel (i.e., generating prediction pixels with an interpolation filter, similar to regular inter-frame prediction).

[0159] In addition, the BV export unit 205A can adaptively select multiple different pixel precisions.

[0160] That is, the BV export unit 205A can set the pixel precision of BV with multiple different integer pixel precisions or multiple different decimal pixel precisions.

[0161] For example, the BV export unit 205A can change the pixel precision of the BV in units of sequence and / or image and / or slice and / or block.

[0162] That is, the BV export unit 205A can change the pixel precision of BV according to any combination of sequence units, image units, slice units and block units.

[0163] Regarding the selection of pixel precision for block unit BV, Non-Patent Document 1 discloses a technique for applying Adaptive Motion Vector Resolution (AMVR) to IBC.

[0164] Specifically, this technique involves selecting the pixel precision of the BVD from either 1 pixel precision or 4 pixel precision during the BV derivation in the IBC, and finally rounding the BV to the selected pixel precision of the BVD.

[0165] The BV export unit 205A can add the aforementioned fractional pixel precision (e.g., 1 / 4 pixel precision and / or 1 / 2 pixel precision) as an option for this AMVR.

[0166] As another example, the BV export unit 205A can also change the pixel precision of the BV according to the number of BVs in the decoded target block.

[0167] For example, when BV is 1, the BV exporting unit 205A can set the pixel precision of BV to decimal pixel precision, and when BV is two or more, the BV exporting unit 205A can set the pixel precision of BV to integer pixel precision.

[0168] As the number of BVs increases, the intra-frame prediction in the second frame generally becomes more accurate. On the other hand, the processing load required to derive the BVs increases. Therefore, as the number of BVs increases, the pixel accuracy of the BVs becomes coarser, and thus an effect of suppressing the decoding processing load while maintaining the prediction accuracy of the intra-frame prediction in the second frame can be obtained.

[0169] For example, when there are two or more BVs, the BV derivation unit 205A can set the pixel accuracy of the BVs to integer accuracy (limiting the selectable pixel accuracy to only integer accuracy).

[0170] On the other hand, when there is one BV, the BV derivation unit 205A can set the pixel accuracy of the BV to integer pixel accuracy or fractional pixel accuracy (as the selectable pixel accuracy, it can be not limited to integer accuracy).

[0171] <BVP candidate list> As described above, the BV derivation unit 205A can configure a BVP candidate list corresponding to the number of BVs held by the decoding target block.

[0172] Referring again to the examples of the L0 list and the L1 list, it is divided into cases of using only a single list (L0 list) and cases of using multiple lists (for example, when the BV derivation unit 205A derives up to two BVs, it is the L0 list and the L1 list).

[0173] In either case, for the BVP candidates registered in the BVP candidate list, the BV derivation unit 205A can configure them using, for example, Figure 5 the BV of the neighboring block shown, the previously used BV, the average value of the BVs, etc. of the decoding target block.

[0174] Alternatively, the BV derivation unit 205A can configure the BVP candidates registered in the BVP candidate list using the BVs derived in IntraTMP disclosed in Non-Patent Document 2.

[0175] The registration order of the BVP candidates registered in these BVP candidate lists can follow the registration order disclosed in Non-Patent Documents 1 and 2.

[0176] In addition, the BV derivation unit 205A searches for and registers the above BVP candidates until the BVP candidate list is filled with BVP candidates.

[0177] As another example, the BV derivation unit 205A can confirm whether it is possible to use BVP candidates exceeding the maximum number of BVP candidates that can be registered in the BVP candidate list, and select the BVP candidates to be registered in the BVP candidate list from among those candidates.

[0178] For example, the BV derivation unit 205A can register in the BVP candidate list according to the search order of the BVP candidate list, or it can register after rearranging the order of the registerable BVP candidates using the rearrangement method described later.

[0179] In addition, the maximum number of BVP candidates can be set to a fixed value, or it can be adaptively set through the control information of sequence units, image units, or slice units described later.

[0180] Additionally, the BV export unit 205A may share or not share the BVP candidate list for each of IBC BVP, IBC merge, and IBC BVP / merge (different BVP candidate lists may be used for each of IBC BVP, IBC merge, and IBC BVP / merge).

[0181] As an example of the former, for instance, the BV export unit 205A may set the maximum number of BVP candidates in the BVP candidate list that can be registered to export BVP (BV) using IBC merge, IBC BVP / merge with the IBC merge index, to be the same as the maximum number of BVP candidates in the BVP candidate list that can be registered to export BVP using IBC BVP, IBC BVP / merge with the IBC BVP flag or IBC BVP index.

[0182] Thus, by sharing the BVP candidate list, the circuit size of the image decoding device 200 can be reduced.

[0183] As an example of the latter, the BV export unit 205A may set the maximum number of BVP candidates (IBC merge candidates) in the BVP candidate list (merge candidate list) that can be registered to export BVP (merge candidate or BV) using IBC merge, IBC BVP / merge with the IBC merge index, to be greater than the maximum number of BVP candidates in the BVP candidate list that can be registered to export BVP using IBC BVP, IBC BVP / merge with the IBC BVP flag or IBC BVP index.

[0184] For example, as in Non-Patent Document 2, the BV export unit 205A can fix the maximum number of BVP candidates that can be registered in the BVP candidate list to 2, and set the maximum number of merge candidates that can be registered in the merge candidate list to 6.

[0185] Thus, by setting the maximum number of merge candidates that can be registered in the merge candidate list to be greater than the BVD candidate list, the number of decodeable BV modes is increased, and thus coding efficiency can be expected to be improved.

[0186] When BV exporting unit 205A checks the availability of BVs in neighboring blocks, etc., in order to search for BVP candidates (or merge candidates) that can be registered in the BVP candidate list (or merge candidate list), if the newly confirmed unavailable BV is the same as the BVP candidate (or merge candidate) that has been confirmed as available, it can determine (prune) the newly confirmed unavailable BV as unavailable.

[0187] The BV export unit 205A can improve the coding efficiency of BVP-related control information used to select a BVP from the BVP candidate list by ensuring that no more than two completely identical BVP candidates are registered in the BVP candidate list.

[0188] Furthermore, when AMVR or other methods that change the pixel precision of BV are applied to the decoded target block and the pixel precision of the final exported BV is greater than that of AMVR or other methods that are not applied, the BV exporting unit 205A can determine whether to use the pixel precision of BVP candidates extracted from neighboring blocks of the decoded target block, etc., in order to determine whether the BVP candidate list is available, based on the pixel precision rounded to the final exported BV.

[0189] For example, the BV exporting unit 205A can determine whether to use each BV of the BVP candidate list after rounding to 4 pixels, if the BVs (1 pixel precision) of each neighboring block extracted in order to determine whether the BVP candidate list is available are ultimately rounded to 4 pixels precision by AMVR or the like.

[0190] In addition, the BV export unit 205A can use two or more BVP candidate lists (or merged candidate lists) when exporting BV. If the pixel precision of the BVP candidates (or merged candidates) in at least one list is rounded by AMVR or the like, the pixel precision of the BVP candidates (or merged candidates) in other lists is also rounded (to the same pixel precision), and then the BVP candidate list (or merged candidate list) is constructed.

[0191] As another example, the BV export unit 205A can use two or more BVP candidate lists (or merged candidate lists) when exporting BV. If the pixel precision of the BVP candidates (or merged candidates) in at least one list is rounded by AMVR or the like, the pixel precision of the BVP candidates (or merged candidates) in other lists is not rounded, and a BVP candidate list (or merged candidate list) is constructed.

[0192] For example, since AMVR is a technique that reduces the amount of code in BVD by coarsening the pixel precision of BVD required for BV export from IBC, when IBC and AMVR are applied in combination, the pixel precision of BVP and even BV exported by the above-mentioned IBC BVP, IBC BVP / IBC merging BV export methods is naturally rounded to the pixel precision of BVD.

[0193] On the other hand, the pixel precision of the BVP (or merge candidate), or even the BV, derived by the IBC merging BVP / merging method in IBC can be rounded to the pixel precision of BVD, or it can be left unrounded.

[0194] Therefore, for IBC BVP / merging, when AMVR and the like are applied to the decoding target block, the BV derivation unit 205A can determine whether to use the pixel precision of each BVP candidate that can be registered in the IBC BVP / merging BVP candidate list on the basis of rounding, but it can also determine whether to use the pixel precision of the BVP candidate (or merge candidate) that can be registered in the merge candidate list without rounding.

[0195] In addition, the BV export unit 205A can rearrange the registration order of BVP candidates in the BVP candidate list using a predetermined method.

[0196] Specifically, the BV derivation unit 205A uses the neighboring pixels of the decoding target block as templates, evaluates the similarity with the neighboring pixels of the reference block of the BV reference of each BVP candidate (which are the same size as the template of the decoding target block), and rearranges the BVP candidates in the BVP candidate list in order of high similarity.

[0197] Here, the BV derivation unit 205A can use the sum of squared differences (SSE), sum of absolute differences (SAD), etc., to evaluate the similarity of the template.

[0198] Alternatively, the BV derivation unit 205A can use the similarity between the blocks referenced by the BV for rearranging the BVP candidates in the BVP candidate list.

[0199] The BV derivation unit 205A can use the sum of absolute differences of the Adama transform (SATD) as the similarity.

[0200] When using multiple lists such as L0 list and L1 list, BV derivation unit 205A can evaluate the similarity after synthesizing the neighborhood of the block referenced by the two lists.

[0201] Furthermore, the reordering of the registration order of BVs registered in the BVP candidate list can be applied to all available BVP candidates or limited to a subset.

[0202] Since the BV derivation unit 205A can register BVP candidates in the BVP candidate list according to the above-mentioned order of high similarity, thereby deriving high-precision BVPs from the BVP candidate list, as a result, it is expected that the intra-frame prediction will be more accurate and even the coding efficiency will be improved.

[0203] <Correction> The BV exporting unit 205A can correct the reference position of the BVP exported from the above-mentioned IBC BVP, the BVP(BV) exported from the IBC merge, or the BVP(BV) exported from the IBC BVP / merge by a predetermined method.

[0204] As predetermined methods, they can be broadly divided into methods that add BVD as the correction block vector to the derived BVP(BV), and methods that search for new reference positions by using BVP(BV) as the starting point without adding BVD, as described later.

[0205] Regarding the correction method for BV in the former (hereinafter referred to as the BVD addition method), due to the property of adding the new BVD, the BV derivation part 205A can be applied not to IBC BVP, but only to the BVP(BV) corresponding to the IBC merge index of IBC merge or IBC BVP / merge.

[0206] Regarding the BV correction method of the latter (hereinafter referred to as the BV search method), since it is a method that does not add BVD, the BV derivation unit 205A can be applied to any of IBC BVP, IBC merging, or IBC BVP / merging.

[0207] As an example of the BVD addition method, the BV derivation unit 205A can be applied to the merge mode with block vector difference (MBVD) disclosed in Non-Patent Document 2.

[0208] Specifically, when MBVD is applied, the BV derivation unit 205A derives (determines) the value of the correction block vector based on control information.

[0209] In addition, when applying MBVD, the BV derivation unit 205A can set constraints on the possible values ​​(reference positions) of the correction block vector to suppress the amount of code in the correction block vector.

[0210] For example, when applying MBVD, the BV derivation unit 205A can limit the distance and direction of the possible values ​​(reference positions) of the correction block vector to discrete values. For example, when applying MBVD, the BV derivation unit 205A can limit the distance of the possible values ​​(reference positions) of the correction block vector to powers of 2, and / or limit it to the up, down, left, and right directions of the possible values ​​(reference positions) of the correction block vector.

[0211] Furthermore, the decoding unit 201 and the BV export unit 205A can use control information of sequence units and / or image units and / or slice units to control whether candidates for correction block vectors in MBVD include candidates for correction block vectors with fractional pixel precision (or set to only integer pixels).

[0212] Alternatively, the BV export unit 205A can set constraints on the value (reference position) of the correction block vector based on the number of block vectors in the decoded target block.

[0213] For example, when the target block being decoded has two or more block vectors, the BV derivation unit 205A can limit the value (reference position) of the correction block vector to integer pixel precision only.

[0214] As another example, the BV export unit 205A can perform control by determining whether multiple BVs can be exported for IBC, on a unit of sequence and / or image and / or slice.

[0215] As an example of the BVD addition method, the BV derivation unit 205A can be applied to the template matching disclosed in Non-Patent Document 2.

[0216] Similar to the BVP candidate rearrangement method described above, template matching is a method that uses the neighboring pixels of the decoded target block as templates, and starts from the derived BVP(BV), while checking the similarity with the neighboring pixels of the reference block to search for the reference position of the new BVP(BV).

[0217] The BV exporting unit 205A can limit the search range to the same size as that in Non-Patent Document 2 (e.g., a range of 8 pixels in the up, down, left, and right directions with BVP (BV) as the origin).

[0218] In addition, the BV export unit 205A can adaptively limit the pixel precision during the search for the pixel precision selected using AMVR or the like for the above-mentioned IBC (for example, when 1 pixel precision is selected using AMVR or the like, the search pixel precision by template matching can also be matched with 1 pixel precision).

[0219] As an example of other BVD addition methods, when there are two or more derived BVs, the BV derivation unit 205A can apply a BV search technique such as decoder-side motion vector refinement (DMVR) of non-patent document 1, namely decoder-side block vector refinement (DBVR).

[0220] Specifically, the BV exporting unit 205A applies a pre-set correction block vector candidate group to the two or more exported BVPs (BVs) and corrects the BVs (BVPs) by searching for correction block vectors with high similarity among multiple reference blocks.

[0221] Here, the BV derivation unit 205A can use the aforementioned sum of squared differences (SSE), sum of absolute differences (SAD), and sum of absolute differences with Adamas transform (SATD) to calculate the similarity of DBVR. Furthermore, DBVR relies on the search method using two or more BVs, therefore it is not applicable when deriving only one BV.

[0222] <save> The BV exporting unit 205A stores one or more BVs and reference images exported for the decoding target block in the memory of the image decoding device 200 in units of predetermined pixels, so that subsequent decoding target blocks can refer to one or more BVs and reference images exported for the decoding target block.

[0223] The BV export unit 205A stores the BV exported for the decoded target block and the reference image in memory. It can use the BV of the decoded target block and the reference image to apply IBC to another decoded target block that is decoded after the decoded target block. Therefore, it can increase the application rate of IBC and improve the coding efficiency.

[0224] The BV exporting unit 205A can, for example, as in Non-Patent Document 1, use 4×4 pixel sub-blocks, which subdivide the decoding target block into 4×4 pixel units, as a predetermined pixel unit, to store one or more BVs exported for the decoding target block and a reference image in the memory.

[0225] Alternatively, the BV exporting unit 205A can store one or more BVs exported for the decoding target block and a reference image in memory, in units of pixels smaller than 4×4 pixels or in units of pixels larger than 4×4 pixels (powers of 2 or 4).

[0226] The BV export unit 205A can store one or more BVs exported for the decoding target block and a reference image in memory, depending on the number of block vector candidate lists.

[0227] For example, if there is only one exported BV, the BV exporting unit 205A saves the BV and the corresponding reference image in the L0 list used in normal inter-frame prediction.

[0228] In addition, when there are two exported BVs, the BV exporting unit 205A saves the BV and the corresponding reference image one by one in each of the L0 list and L1 list used in normal inter-frame prediction.

[0229] Furthermore, when there are three or more exported BVs, the BV exporting unit 205A adds an Ln list in addition to the L0 list and L1 list used in the usual inter-frame prediction, and saves the BV and the corresponding reference image one by one in each of them.

[0230] When referenced as a BVP candidate in another decoding target block, the BV derivation unit 205A expands the BV stored in memory as described above as a BVP candidate that can be used in the BVP candidate list.

[0231] If there is only one BV stored in memory, the BV derivation unit 205A expands the BV into the BVP candidate as a list number (one BVP candidate).

[0232] On the other hand, if there are two or more BVs stored in the memory, the BV derivation unit 205A expands the BVs as two or more list numbers (two or more BVP candidates).

[0233] In addition, the BV export unit 205A can store one or more BVs exported for the decoding target block and a reference image in another FIFO (First In First Out) type memory (hereinafter referred to as history) for reference when referencing the historical BVPs or historical BV merging of the BVs used in the past.

[0234] Here, if only one BV is exported, the BV exporting unit 205A saves the BV and the corresponding reference image in the history as is. However, if the same BV already exists in the history, the BV exporting unit 205A does not save the BV.

[0235] On the other hand, when there are two or more exported BVs, the BV exporting unit 205A saves the two or more BVs and their corresponding reference images in the history. However, if the same BV already exists in the history, the BV exporting unit 205A does not register the BV, just as it does when only one BV is exported.

[0236] Furthermore, if the BV stored in the aforementioned memory or history is corrected as described above, the BV export unit 205A can save the corrected BV or the BV before correction.

[0237] In this case, the BV derivation unit 205A can achieve the effect of omitting the code representing the BV before and after correction by pre-determining which BV to register after correction.

[0238] Conversely, the BV derivation unit 205A can improve coding efficiency by adaptively determining which of the registered corrected BV and the original BV.

[0239] <Synthesis> The second intra-frame prediction pixel generation unit 205B is configured to generate a second prediction pixel by synthesizing the prediction pixel of the decoding target block using the pixels (reference pixels) of a reference block of one or more BV references.

[0240] When there is one BV, the second intra-frame prediction pixel generation unit 205B can use the reference pixel of the reference block as the prediction pixel of the decoding target block as is, or it can correct the pixel of the reference block.

[0241] For example, the second intra-frame prediction pixel generation unit 205B can perform this correction by using the polynomial of the reference pixel described above.

[0242] Specifically, the second intra-frame prediction pixel generation unit 205B can be configured to generate a correction value P(x,y) of the prediction pixel at coordinates (x,y) within the decoding target block by using the reference pixel Q(x,y) and a polynomial (refer to the following formula) defined by the weight coefficients C1 and C2.

[0243] P(x, y) = C1 × Q(x, y) + C2 Alternatively, the second intra-frame prediction pixel generation unit 205B can define the above polynomial using multiple reference pixels Q1, Q2, ... Qn in a block of BV references for a prediction pixel.

[0244] The following is an example of defining a polynomial in a 3×3 region centered at coordinates (x, y).

[0245] P(x, y)=C1×Q(x-1, y-1)+C2×Q(x, y-1)+C3×Q(x+1, y-1)+C4×Q(x-1, y)+C5× Q(x,y)+C6×Q(x+1,y)+C7×Q(x-1,y+1)+C8×Q(x,y+1)+C9×Q(x+1,y+1)+C10 Figure 10An example with 10 weighting coefficients is shown. For simplicity, Q is used. i，j Let Q(x+i, y+j) be used.

[0246] In the case where multiple reference pixels are used to define the polynomial, the second intra-frame prediction pixel generation unit 205B preferably uses nearby pixels.

[0247] Alternatively, the second intra-frame prediction pixel generation unit 205B can define multiple polynomials based on the distribution of the aforementioned reference pixels.

[0248] The following is an example of a polynomial with two coefficients defined using a threshold Th when the histogram of the reference pixel is bimodal.

[0249] P(x, y)=C1×Q(x, y)+C2 P(x, y)>Th P(x,y)=C3×Q(x,y)+C4 P(x,y)<=Th Here, the second intra-frame prediction pixel generation unit 205B can set the threshold to a value that separates the two peaks. Alternatively, the second intra-frame prediction pixel generation unit 205B can simplify the threshold and use an average value.

[0250] If the distribution of the aforementioned reference pixels is multi-peaked, the second intra-frame prediction pixel generation unit 205B can also apply a polynomial with more coefficients.

[0251] When the target block being decoded has two BVs, the second intra-frame prediction pixel generation unit 205B can be configured to generate a second prediction pixel by weighting the pixels (reference pixels) of the two reference blocks referenced by the BV using a predetermined weight value.

[0252] For example, the second intra-frame prediction pixel generation unit 205B can be configured to generate a second prediction pixel by simply averaging reference pixels of two or more BV references of the target block at a 1:1 ratio.

[0253] That is, the second intra-frame prediction pixel generation unit 205B can either fix the predetermined weight value to 1:1 or adaptively set the predetermined weight value.

[0254] When the predictive pixel generation unit 205B adaptively sets the predetermined weight values ​​mentioned above in the second frame, it is possible to set non-1:1 weight values ​​based on the length of BV and the evaluation of the similarity between the neighboring pixels (templates) of the decoding target block and each reference block.

[0255] Here, by simply averaging the predetermined weight values ​​at a 1:1 ratio, the processing load can be reduced. On the other hand, by adaptively setting the predetermined weight values, the coding efficiency can be improved.

[0256] In addition, it can be assumed that the reference block with the shorter BV length among multiple BVs has a smaller error with the decoded target block. Therefore, by increasing the weight of the reference block with the shorter BV length, the prediction accuracy can be improved when generating prediction pixels based on more than two BVs.

[0257] In addition, in template similarity evaluation, it can be assumed that the reference block with high similarity (low template cost) has a small error with the decoding target block. Therefore, by increasing the weight of the reference block with a short BV, the prediction accuracy can be improved when generating predicted pixels based on more than two BVs.

[0258] As another example, the second intra-frame prediction pixel generation unit 205B can perform a weighted average of the reference pixels of two BV reference reference blocks, as disclosed in Non-Patent Document 1 and Non-Patent Document 2, based on control information (hereinafter referred to as the BCW index) that determines the weight values ​​used in the weighted average.

[0259] The decoding unit 201 can decode the BCW index for double-predicted IBC BVP (or IBC BVP / merge), or it can inherit the BCW index of the merge candidate in the merge candidate list instead of decoding the BCW index for double-predicted IBC merge (or IBC BVP / merge).

[0260] Furthermore, as in Non-Patent Document 2, the second intra-frame prediction pixel generation unit 205B can correct the weight value of the BCW corresponding to the inherited BCW index value by using a similarity evaluation of the templates for the aforementioned decoded target block and reference block.

[0261] Alternatively, the second intra-frame prediction pixel generation unit 205B can be configured to generate the second prediction pixel by using a polynomial defined by pixels of multiple reference destinations.

[0262] Specifically, the second intra-frame prediction pixel generation unit 205B can be configured to generate a correction value P(x,y) of the prediction pixel at coordinates (x,y) within the decoded target block by using a polynomial (refer to the following formula) defined by different multiple reference pixels Q(x,y) and R(x,y) and weight coefficients C1, C2, and C3.

[0263] P(x,y)=C1×Q(x,y)+C2×R(x,y)+C3 Based on the above configuration, simplifying the polynomial can reduce the computational load. Conversely, increasing the polynomial's complexity can improve prediction accuracy.

[0264] The second intra-frame prediction pixel generation unit 205B can be configured to predefine multiple polynomials as the aforementioned polynomials and make them selectable. According to this configuration, appropriate polynomials can be used, thereby improving coding efficiency.

[0265] The second intra-frame prediction pixel generation unit 205B can be configured to derive the coefficients (weighting coefficients) of the above polynomial from the neighboring pixels of the decoded target block and the neighboring pixels of the reference block.

[0266] Specifically, such as Figure 11 As shown, the second intra-frame prediction pixel generation unit 205B derives a weighting coefficient C (reference) that makes the neighboring pixels P' of the decoded target block and the neighboring pixels Q' of the reference block consistent through correction. Figure 11 (X1 in the middle).

[0267] For example, the second intra-frame prediction pixel generation unit 205B is defined by the error function E shown in the following formula, and the weight coefficient C that minimizes E is derived.

[0268] E=Σ(P'(x, y)-(C1×Q'(x-1, y-1)+C2×Q'(x, y-1)+C3×Q'(x+1, y-1)+C4×Q'(x-1, y)+C5 ×Q'(x, y)+C6×Q'(x+1, y+1)+C7×Q'(x-1, y+1)+C8×Q'(x, y+1)+C9×Q'(x+1, y+1)+C10)) 2 The second intra-frame prediction pixel generation unit 205B can use the least squares method, etc., in the derivation of the weight coefficients.

[0269] Alternatively, the second intra-frame prediction pixel generation unit 205B can also use robust estimation methods such as principal component regression and partial least squares regression to derive the weight value, thereby reducing the impact of outliers.

[0270] The second intra-frame prediction pixel generation unit 205B applies the derived weight coefficient C to the neighboring pixels Q of the reference block to calculate the correction value P. Figure 11 X2 in the image is used as the predicted pixel (second predicted pixel) of the target block for decoding. Figure 11 (X3 in the middle).

[0271] According to this configuration, compared to the case where the weight coefficient is kept as control information, by deriving the weight coefficient from neighboring pixels, the weight coefficient does not need to be decoded, thus improving the coding efficiency.

[0272] Furthermore, since applying weighting coefficients to multiple reference pixels is equivalent to adaptively deriving a more accurate fractional precision BV, it is possible to improve coding efficiency by reducing the amount of code for the BV.

[0273] Here, the second intra-frame prediction pixel generation unit 205B can be configured to use neighboring pixels P' and Q' within a certain range in the derivation of the aforementioned weighting coefficients (coefficients of the polynomial).

[0274] For example, such as Figure 12 As shown in (a), the second intra-frame prediction pixel generation unit 205B can be configured to use neighboring pixels within 4 pixel rows of the decoding target block in the derivation of the aforementioned weight coefficients (coefficients of the polynomial).

[0275] Conversely, the second intra-frame prediction pixel generation unit 205B can also limit the neighboring pixels used to derive the aforementioned weighting coefficients.

[0276] For example, the second intra-frame prediction pixel generation unit 205B can be as follows: Figure 12 As shown in (b), weight coefficients can be derived using only the neighboring pixels of the region located above the decoded target block, or as... Figure 12 As shown in (c), only the neighboring pixels of the region located to the left of the decoded target block are used to derive the weight coefficients.

[0277] Furthermore, the second intra-frame prediction pixel generation unit 205B can be configured to select and specify multiple ranges of the aforementioned neighboring pixels. According to this configuration, appropriate weighting coefficients can be derived, thereby improving coding efficiency.

[0278] (Basic Concepts of IBC OBMC) The second intra-frame prediction pixel generation unit 205B can perform correction for the reference pixels in the reference block of the decoding target block by calculating the weighted average of the reference pixels in the reference block of the decoding target block and the neighboring pixels in the reference blocks of the adjacent blocks, rather than by the reference pixels of the decoding target block themselves.

[0279] Thus, as in the Overlapped Block Motion Compensation (OBMC) for inter-frame prediction devices disclosed in Non-Patent Document 2, it is possible to achieve a smooth connection between the prediction blocks generated by IBC and their adjacent neighboring blocks.

[0280] Figure 13 This shows the case where adjacent blocks of the target block have BV.

[0281] like Figure 13 As shown, firstly, the second intra-frame prediction pixel generation unit 205B acquires the neighboring pixel (hereinafter referred to as the neighboring reference neighboring pixel) whose relative position is equivalent to that of the decoding target block at the neighboring reference block of the BV reference of the neighboring block.

[0282] exist Figure 13 In the example, since the adjacent block is located to the left of the decoding target block, the second frame intra-prediction pixel generation unit 205B obtains the adjacent reference neighbor pixel to the right of the adjacent reference block.

[0283] The range of adjacent reference pixels can be set fixedly or variably.

[0284] For example, the second intra-frame prediction pixel generation unit 205B can change the range of adjacent reference neighbor pixels according to the block size and aspect ratio.

[0285] Next, the second intra-frame prediction pixel generation unit 205B calculates the weighted average of the neighboring reference pixels and the reference pixels of the decoded target block. This process will be referred to as "IBC OBMC" below.

[0286] (Application judgment and application processing unit of IBC OBMC) The second intra-frame prediction pixel generation unit 205B can determine whether to apply IBC OBMC by using the decoded target block as a unit, by using the block smaller than the decoded target block (hereinafter referred to as sub-block) as a unit, or by using both the decoded target block unit and the sub-block unit as two levels.

[0287] As an example of determining the sub-block, as disclosed in Non-Patent Document 2, the second intra-frame prediction pixel generation unit 205B can determine whether to apply IBC OBMC for each 4×4 pixel sub-block facing the block boundary between the left and top parts of the decoding target block.

[0288] Thus, by determining whether to apply IBC OBMC on a per-block basis, the increase in the amount of processing required for the determination can be suppressed.

[0289] On the other hand, by determining whether to apply IBC OBMC on a sub-block basis, when IBC is applied to the target block being decoded, IBC OBMC can be applied to the part of the target block where IBC OBMC is effective, thus improving prediction accuracy.

[0290] In addition, the second intra-frame prediction pixel generation unit 205B can apply IBC OBMC in units of blocks smaller than the decoded target block (hereinafter referred to as sub-blocks) based on the determination result of whether IBC OBMC is applied.

[0291] (Application conditions of IBC OBMC) It is possible that the second frame intra-predictive pixel generation unit 205B, on a unit basis, determines whether to apply IBC OBMC if a predetermined condition is met, and determines whether to not apply IBCOBMC if the predetermined condition is not met.

[0292] The second intra-frame prediction pixel generation unit 205B can configure the predetermined condition using at least one of the conditions shown below.

[0293] 1. Able to effectively apply IBC OBMC control information (IBC OBMC flags) at the sequence and / or image and / or slice level. 2. Able to effectively apply IBC OBMC control information (IBC OBMC flags) on a per-decoded target block basis. 3. The area (number of pixels) of the target block to be decoded is 32 pixels or more. *Regarding condition 3, it can be 16 pixels, 64 pixels, or 128 pixels, but not 32 pixels.

[0294] 4. The adjacent blocks adjacent to the target block (or sub-block) have BV or motion vectors. 5. The BV of the decoded target block (or sub-block) is different from the BV or motion vector of the adjacent blocks. 6. The local brightness compensation disclosed in Non-Patent Document 2 is not applied in the decoded target block (it is invalid). 7. The maximum error between the reference pixel of the BV based on the decoded target block (or sub-block) and the pixel value of the neighboring reference neighbor pixel based on the BV of the neighboring block or motion vector is below a predetermined threshold. 8. In the gradient histogram of the reference pixel of the decoded target block, the main gradients are mostly... 9. The BV of the decoded target block (or sub-block) is 1. 10. The BV or motion vector of the adjacent blocks of the target block (or sub-block) is 1. The following describes examples of changes to the aforementioned predetermined conditions.

[0295] Regarding condition 5 above, if the reference frame for the BV reference of the decoded target block (or sub-block) is different from the reference frame for the BV or motion vector reference of the adjacent block, the intra-frame prediction pixel generation unit 205B can determine that the BV of the decoded target block (or sub-block) is different from the BV or motion vector of the adjacent block.

[0296] Regarding conditions 6 and 7 above, the second intra-frame prediction pixel generation unit 205B can use the prediction pixel after applying interpolation filtering to the reference pixel, instead of using the reference pixel itself.

[0297] Furthermore, the second intra-frame prediction pixel generation unit 205B can set different application limitations for the IBC OBMC based on the number of derived BVs for the decoded target block (or sub-block) or adjacent blocks. Specifically, as follows.

[0298] The second intra-frame prediction pixel generation unit 205B can add the condition that the BV of the decoded target block (or sub-block) is 1 to the aforementioned predetermined conditions.

[0299] The second intra-frame prediction pixel generation unit 205B is able to suppress the increase in memory bandwidth by limiting the application of IBC OBMC to only when the BV of the decoded target block (or sub-block) is 1. This memory bandwidth is used to obtain the required adjacent reference neighbor pixels when applying IBC OBMC.

[0300] The second intra-frame prediction pixel generation unit 205B can add the condition that the BV or motion vector of the adjacent block of the decoded target block (or sub-block) is 1 to the above-mentioned predetermined conditions.

[0301] The second intra-frame prediction pixel generation unit 205B is able to suppress the increase in memory bandwidth by limiting the application of IBC OBMC to only when the BV or motion vector of the adjacent block of the decoded target block (or sub-block) is 1. This memory bandwidth is used to acquire the required adjacent reference neighbor pixels when applying IBC OBMC.

[0302] When there are two or more BV or motion vectors in the adjacent blocks of the target block (or sub-block), the second intra-frame prediction pixel generation unit 205B can select one BV or motion vector by comparing the size of the BV or motion vector or template cost of the adjacent block.

[0303] For example, the second intra-frame prediction pixel generation unit 205B can select the neighboring block whose BV or motion vector size is the smallest or whose template cost is the smallest.

[0304] Setting the IBC OBMC application conditions for decoding target block units and / or sub-block units as described above has the following purpose (expected effect).

[0305] For example, in images rendered using CG or screen images, the edges are often sharp in the original image, and block boundaries tend to appear as discontinuous images. Therefore, in such images, it is easier to suppress prediction errors near block boundaries if IBC OBMC is not applied.

[0306] In addition, the second intra-frame predictive pixel generation unit 205B can apply IBC OBMC only for specific slices or specific block sizes.

[0307] Thus, by limiting the application of IBC OBMC based on slice and / or block size, the amount of encoding processing can be reduced.

[0308] For example, IBC applies restrictions by slice, while IntraTMP does not apply restrictions by slice.

[0309] Compared to IBC, BV has a lighter search processing load and lower prediction accuracy. Therefore, by increasing the number of application objects, the coding efficiency can be improved.

[0310] (IBC OBMC weighted average) Alternatively, the second intra-frame prediction pixel generation unit 205B, as the aforementioned neighboring pixel, can be used for the correction employed in the correction of the aforementioned reference pixel.

[0311] The second intra-frame prediction pixel generation unit 205B derives weight coefficients from all or part of the pixels of neighboring blocks and the pixels of neighboring reference blocks.

[0312] That is, such as Figure 14 As shown, the second intra-frame prediction pixel generation unit 205B sets the pixels of adjacent blocks as P' and the pixels of adjacent reference blocks as Q', deriving the weight coefficient C( Figure 14 The value of Y1 is then applied to the neighboring pixel Q of the adjacent reference block to obtain the correction value P. Figure 14 Y2 in the middle), and calculate the weighted average of the predicted pixels of the decoded target block (Y2). Figure 14 Y3 in the middle.

[0313] According to this configuration, by reflecting the neighboring pixels of adjacent reference blocks onto the predicted pixels of the decoded target block, the prediction error can be reduced, thereby improving coding efficiency.

[0314] Furthermore, when the number of BVs in the target block and the reference block are different, the second frame intra-prediction pixel generation unit 205B adjusts the number of BVs in the block with more BVs to the number of BVs in the block with fewer BVs (reducing the number of BVs in the target block and adjacent blocks to the minimum number of BVs).

[0315] Similarly, when the pixel precision of BV is different in the target block and the reference block, the second frame intra-prediction pixel generation unit 205B adjusts the pixel precision of the BV with the finer pixel precision to the pixel precision of the BV with the coarser pixel precision (rounding the pixel precision of each BV in the target block and adjacent blocks to the maximum pixel precision).

[0316] (IBC GPM) When the IBC GPM is applied to the target block for decoding, the second intra-frame prediction pixel generation unit 205B can perform a weighted average of the reference destination pixels (reference pixels) of one or more BVs for each segmented region divided in two by the IBC GPM segmentation line, based on the distance from the IBC GPM segmentation line.

[0317] As a variation, the second intra-frame prediction pixel generation unit 205B can be fixedly or adaptively set to the region where the weighted average is performed, that is, the number of pixels in each segmented region direction from the IBC GPM segmentation line.

[0318] As a fixed setting method, the second intra-frame prediction pixel generation unit 205B can be set to values ​​such as 0 pixels, 1 / 4 pixel, 1 / 2 pixel, 1 pixel, 2 pixels, 4 pixels, and 8 pixels.

[0319] For example, for screen images, by using a small number of pixels, such as 0 pixels or 1 / 4 pixels, as the number of pixels in the weighted average region width of IBC GPM, it is easy to apply the IBC GPM segmentation line, which can maintain the pixel values ​​of the edge boundaries unique to the screen image without excessive smoothing, thus easily reducing prediction error.

[0320] Furthermore, as an adaptive setting method, the second intra-frame prediction pixel generation unit 205B can be applied to the GPM adaptive mixing method disclosed in Non-Patent Document 2.

[0321] For example, for natural images, such as the adaptive blending method of GPM disclosed in Non-Patent Document 2, the prediction error near the segmentation line of IBC GPM can be easily reduced by adaptively setting the number of pixels of the weighted average region width of IBC GPM based on control information and block size.

[0322] <Transformation> For the transform coefficients of the decoded target block using IBC (or IntraTMP), the inverse transform unit 203 can adaptively select a transform basis from multiple different transform bases and apply inverse transform processing.

[0323] Here, as an example of multiple different transformation bases, the inverse transformation unit 203 can use the multiple transformation selection (MTS) disclosed in Non-Patent Document 1.

[0324] In addition, among the MTSs disclosed in Non-Patent Document 1, there are MTSs for intra-frame prediction and MTSs for inter-frame prediction.

[0325] The inverse transform unit 203 can be applied to the MTS for intra-frame prediction disclosed in Non-Patent Document 1, or to the MTS for inter-frame prediction disclosed in Non-Patent Document 1.

[0326] The inverse transform unit 203 can determine and apply the transform basis of the MTS based on the value of the control information, which is used to determine the candidates of the transform basis of the MTS, the candidates of the transform basis of the MTS decoded or estimated by the decoding unit 201.

[0327] As a variation, for the transform coefficients of the decoded target block using IBC (or IntraTMP), the inverse transform unit 203 can apply a two-stage inverse transform process using different or the same transform basis.

[0328] Here, as an example of a two-stage inverse transform process using different transform bases, the inverse transform unit 203 can use the low-frequency non-separable transform (LFNST) disclosed in Non-Patent Document 1.

[0329] The decoding unit 201 can select a transformation basis from multiple different transformation bases and decode the control information used to determine whether to perform an inverse transformation on the transformation coefficients. The inverse transformation unit 203 can specify and apply the transformation basis based on the value of the control information, which is used to determine the candidates of the transformation basis of the LFNST, and the candidates of the transformation basis of the LFNST decoded or estimated by the decoding unit 201.

[0330] The reasons for applying MTS or LFNST to the decoding target block of IBC (or IntraTMP) are as follows.

[0331] In IBC (or IntraTMP), due to the nature of the signal processing described above, the applied blocks often have complex image characteristics such as edges or stripe patterns. Therefore, the transform coefficients derived after transforming the prediction error of the decoded target block using IBC (or IntraTMP) tend to be distributed on high-frequency components, or biased towards horizontal, vertical, or tilted components.

[0332] For such a distribution of transform coefficients, adaptively selecting and applying an effective inverse transform basis from multiple different transform bases, or applying a two-level inverse transform using different or the same transform basis, can be expected to reduce the redundancy between transform coefficients (and even the spatial redundancy of prediction errors). As a result, an improvement in coding efficiency can be expected.

[0333] <Color difference> In the decoding (or encoding) of moving images, in order to reduce the amount of information, the RGB signals that make up the moving image are sometimes converted into luminance signals (Y signals) and color difference signals (UV signals or CbCr signals), and then decoded (or encoded).

[0334] In addition, to further reduce the amount of information, the number of pixels in the color difference signal is sometimes removed (downsampled) at fixed intervals (e.g., halved in the horizontal and / or vertical directions) relative to the number of pixels in the luminance signal.

[0335] Furthermore, Non-Patent Document 1 discloses a technique (single-tree structure) and a technique (dual-tree structure) for making the segmentation structure of the same decoding (or encoding) tree block identical in the image of the luminance signal and the image of the chrominance signal identical.

[0336] In situations like those described above, such as when the number of pixels in the chrominance signal is downsampled relative to the number of pixels in the luminance signal, and the decoding (or encoding) tree block including the decoding target block is a single-tree structure, the BV derivation unit 205A can derive the BV of the decoding target block for the chrominance signal based on the size (horizontal and vertical components) of one or more BVs derived for the decoding target block for the luminance signal and the downsampling ratio of the luminance signal and the chrominance signal.

[0337] More specifically, the BV exporting unit 205A can reduce the size (horizontal component and vertical component) of one or more BVs exported from the decoding target block for the luminance signal based on the downsampling ratio of the luminance signal and the chrominance signal.

[0338] Furthermore, in situations like those described above, such as when the image of the chromatic difference signal is downsampled relative to the image of the luminance signal, and the decoding (or encoding) tree block including the decoding target block is a dual-tree structure, the BV derivation unit 205A may not derive the BV for the decoding target block of the chromatic difference signal.

[0339] <Signaling> The following describes the control information (syntax) for the decoding and holding BV mode (hereinafter referred to as BV mode) of the decoding unit 201.

[0340] The code information input to the decoding unit 201 may include a sequence parameter set (SPS), which summarizes the control information (syntax) of the target sequence unit to be decoded.

[0341] Additionally, the code information may include a Picture Parameter Set (PPS) or a Picture Header (PH), which summarizes the control information for decoding the target picture unit.

[0342] Alternatively, the code information may include a slice header (SH) that summarizes the control information for decoding the target slice unit.

[0343] The following uses Figure 15 as well as Figure 16 The decoding control method for various IBC-related flags in the decoding unit 201, which are in units of sequence, image, or slice, and the definition (meaning) of each flag are explained.

[0344] Figure 15 This diagram illustrates a decoding control method for predetermined flags associated with IBC on a sequence-by-sequence basis. The decoding unit 201 operates as follows.

[0345] like Figure 15 As shown, in step S100, the decoding unit 201 determines whether sps_ibc_enabled_flag (second syntax) is 1. If it is 1, the action proceeds to step S101; if it is not 1, the action proceeds to step S102.

[0346] Here, sps_ibc_enabled_flag is a flag that controls (determines) whether sequence-based IBC can be applied. When sps_ibc_enabled_flag is 1, the decoding unit 201 determines that IBC can be applied, and when sps_ibc_enabled_flag is 0, it determines that IBC cannot be applied.

[0347] The decoding unit 201 decodes the predetermined flag in step S101 and does not decode the predetermined flag in step S102, thus ending the process.

[0348] Here, the decoding unit 201 can decode such as Figure 16 The various symbols related to IBC shown are used as the predetermined symbols. Specifically, as follows.

[0349] Figure 16The sps_biibc_enabled_flag shown is a flag that controls (determines) whether double prediction IBC can be applied on a sequence-by-sequence basis (first syntax). When sps_biibc_enabled_flag is 1, the decoding unit 201 determines that double prediction IBC can be applied, and when sps_biibc_enabled_flag is 0, it determines that double prediction IBC cannot be applied.

[0350] As a modified example of sps_biibc_enabled_flag, the decoding unit 201 can decode sps_multiibc_enabled_flag.

[0351] sps_multiibc_enabled_flag is a flag that controls (determines) whether multi-predictive IBC can be applied on a sequence-by-sequence basis. When sps_multiibc_enabled_flag is 1, the decoding unit 201 determines that multi-predictive IBC can be applied, and when sps_multiibc_enabled_flag is 0, it determines that multi-predictive IBC cannot be applied.

[0352] Figure 16 The sps_ibcmbvd_enabled_flag shown is a flag that controls (determines) whether IBC MBVD can be applied on a sequence-by-sequence basis. When sps_ibcmbvd_enabled_flag is 1, the decoding unit 201 determines that IBC MBVD can be applied, and when sps_ibcmbvd_enabled_flag is 0, it determines that IBC MBVD cannot be applied.

[0353] Figure 16 The sps_six_minus_max_num_ibc_merge_cand.16 shown is a control information that sets the maximum value of the candidate list for the above IBC merge (or IBC BVP) on a sequence-by-sequence basis.

[0354] For example, when the maximum settable value of the candidate list for IBC merging (or IBC BVP) designed by the image decoding device 200 is 6 (corresponding to "six" in sps_six_minus_max_num_ibc_merge_cand), the control information can be used to change the maximum value.

[0355] Specifically, as shown below, the decoding unit 201 can control (change settings) the maximum value of the candidate list of IBC merging (or IBC BVP) of the image decoding device 200 on a sequence-by-sequence basis.

[0356] if(sps_ibc_enabled_flag) MaxNumIbcMergeCand=6-sps_six_minus_max_num_ibc_merge_cand else MaxNumIbcMergeCand=0 Here, MaxNumIbcMergeCand is an internal parameter within the decoding device, representing the maximum value set in units of sequences (or pictures, slices, or decoded target blocks) for deriving a candidate list of block vectors for IBC merging (or IBC BVP).

[0357] Figure 16 The sps_ibctm_enabled_flag shown is a flag that controls (determines) whether the sequence-based TM for IBC (IBC TM) can be applied. When sps_ibctm_enabled_flag is 1, the decoding unit 201 determines that IBC TM can be applied, and when sps_ibctm_enabled_flag is 0, it determines that IBC TM cannot be applied.

[0358] Figure 16 The sps_ibcdbvr_enabled_flag shown is a flag that controls (determines) whether sequence-based DBVR for IBC (IBC DBVR) can be applied. When sps_ibcdbvr_enabled_flag is 1, the decoding unit 201 determines that IBC DBVR can be applied, and when sps_ibcdbvr_enabled_flag is 0, it determines that IBC DBVR cannot be applied.

[0359] Figure 16 The sps_ibcciip_enabled_flag shown is a flag that controls (determines) whether the Combined Intra InterPrediction (CIIP) for IBC disclosed in Non-Patent Document 2 can be applied on a sequence-by-sequence basis. When sps_ibcciip_enabled_flag is 1, the decoding unit 201 determines that IBC CIIP can be applied, and when sps_ibcciip_enabled_flag is 0, it determines that IBC CIIP cannot be applied.

[0360] Figure 16The sps_ibcgpm_enabled_flag shown is a flag that controls (determines) whether IBC GPM can be applied in sequence. When sps_ibcgpm_enabled_flag is 1, the decoding unit 201 determines that IBC GPM can be applied, and when sps_ibcgpm_enabled_flag is 0, it determines that IBC GPM cannot be applied.

[0361] Figure 16 The sps_ibcobmc_enabled_flag shown is a flag that controls (determines) whether IBC OBMC can be applied on a sequence-by-sequence basis. When sps_ibcobmc_enabled_flag is 1, the decoding unit 201 determines that IBC OBMC can be applied, and when sps_ibcobmc_enabled_flag is 0, it determines that IBC OBMC cannot be applied.

[0362] Figure 16 The sps_ibcflm_enabled_flag shown is a flag that controls (determines) whether the weight coefficients defined by the above polynomial for IBC (IBC FLM: IBC Filtered Linear Model) on a sequence-by-sequence basis can be applied. When sps_ibcflm_enabled_flag is 1, the decoding unit 201 determines that IBC FLM can be applied, and when sps_ibcflm_enabled_flag is 0, it determines that IBC FLM cannot be applied.

[0363] Figure 16 The sps_ibcmts_enabled_flag shown is a flag that controls (determines) whether the sequence-based MTS for IBC (IBC MTS) can be applied. When sps_ibcmts_enabled_flag is 1, the decoding unit 201 determines that IBC MTS can be applied, and when sps_ibcmts_enabled_flag is 0, it determines that IBC MTS cannot be applied.

[0364] Figure 16The sps_ibclfnst_enabled_flag shown is a flag that controls (determines) whether LFNST for IBC (IBC LFNST) can be applied on a sequence-by-sequence basis. When sps_ibclfnst_enabled_flag is 1, the decoding unit 201 determines that IBC LFNST can be applied, and when sps_ibclfnst_enabled_flag is 0, it determines that IBC LFNST cannot be applied.

[0365] If the aforementioned flags are not decoded, the decoding unit 201 can determine the value of these flags as 0.

[0366] Here, although Figure 15 as well as Figure 16 The text describes decoding control methods for various flags at the sequence level, but the same control can also be performed at finer units, such as at the image level or at the slice level. Furthermore, control can be performed only at these specific levels or across multiple levels in a multi-level manner.

[0367] For example, it is possible to suppress the increase in code size by setting it only at the high level, or to perform adaptive control by setting it at the low level as well, and then prioritizing the settings at the low level.

[0368] Furthermore, while the examples above described methods for setting correction methods on a sequence, image, or slice basis, these methods can be omitted, and instead, the method can be selected directly on a block basis, as described later. In this case, the aforementioned increase in header information can be avoided.

[0369] The following uses Figure 17 as well as Figure 18 The decoding unit 201 explains whether the decoding method using a flag with two BVs for IBC (Bipredictional IBC) can be applied in slice-based (or image-based or sequence-based) units, and the definition (meaning) of the flag.

[0370] Furthermore, the following describes a decoding method using two BV IBCs (Bipredictional IBC) as an example, but the same decoding method can also be achieved by using more than two BV IBCs (Multipredictional IBC).

[0371] Figure 17 This is a diagram illustrating an example of a decoding method that determines whether the control (determination) in the decoding unit 201 can apply the flag of dual-predictive IBC.

[0372] exist Figure 17 As described below, the decoding unit 201 can determine whether to decode the flags that enable the application of dual predictive IBC, or to estimate them without decoding.

[0373] like Figure 17 As shown, in step S200, the decoding unit 201 determines whether sps_ibc_enabled_flag is 1 and sh_slice_type is I (whether it is an I slice).

[0374] If yes, proceed to step S201; if no, proceed to step S202.

[0375] Here, sh_slice_type is control information (syntax) indicating the type of slice that includes the target block for decoding, on a slice-by-slice basis.

[0376] As for the types of slices, there are I-slices (slices that can apply first intra-frame prediction or second intra-frame prediction), B-slices (slices that can apply first intra-frame prediction, second intra-frame prediction or inter-frame prediction), and P-slices (slices that can apply first intra-frame prediction, second intra-frame prediction or inter-frame prediction. Among them, inter-frame prediction can only apply single prediction).

[0377] In step S201, the decoding unit 201 decodes sh_biibc_enabled_flag and ends the process.

[0378] In step S202, the decoding unit 201 terminates the process without decoding sh_biibc_enabled_flag.

[0379] Here, sh_biibc_enabled_flag (third syntax) is a flag that controls whether bipredictive IBC can be applied on a slice-by-slice basis.

[0380] When sh_biibc_enabled_flag is 1, the decoding unit 201 determines that dual prediction IBC can be applied, and when sh_biibc_enabled_flag is 0, it determines that dual prediction IBC cannot be applied.

[0381] If sh_biibc_enabled_flag is not decoded, the decoding unit 201 can estimate the value of sh_biibc_enabled_flag based on the value of sh_slice_type.

[0382] Specifically, when sh_slice_type is B (in the case of B slice), the decoding unit 201 can estimate sh_biibc_enabled_flag as 1, and when sh_slice_type is P (in the case of P slice), it can estimate sh_biibc_enabled_flag as 0.

[0383] The decoding unit 201 can estimate sh_biibc_enabled_flag according to the following formula.

[0384] sh_biibc_enabled_flag=(sh_slice_type==B?): 1 or 0 Here, as mentioned above, the reason for controlling whether dual-predictive IBC can be applied in terms of decoding or estimation on a slice-by-slice basis is as follows.

[0385] Since I-slices may have image characteristics such as IBC or IntraTMP, where the first intra-frame prediction is more effective than the second intra-frame prediction, controlling whether dual prediction IBC can be applied on a slice-by-slice basis—setting dual prediction IBC to be applicable for slices where the second intra-frame prediction is particularly effective—can improve coding efficiency. On the other hand, setting dual prediction IBC to be ineffective for slices where the second intra-frame prediction is ineffective can reduce the amount of code for the control information required for dual prediction IBC (described later). Therefore, as a result, coding efficiency can be expected to be improved.

[0386] In B-slices, the reason for estimating sh_biibc_enabled_flag as 1 without decoding sh_biibc_enabled_flag is that in B-slices, normal inter-frame prediction (including bi-prediction using two motion vectors) can be applied. Therefore, by making bi-prediction IBC always applicable, we seek to share the design with normal inter-frame prediction. In addition, we can expect to improve coding efficiency.

[0387] In P-slices, the reason for estimating sh_biibc_enabled_flag as 0 without decoding sh_biibc_enabled_flag is that double prediction cannot be applied in P-slices in normal inter-frame prediction. Therefore, similarly, IBC cannot be applied by setting it to double prediction. This seeks to share the design with normal inter-frame prediction and, in addition, can be expected to improve coding efficiency.

[0388] Figure 18 yes Figure 17 Example of changes. Figure 17 Compared to Figure 16 The difference is in step S200A.

[0389] Specifically, such as Figure 18 As shown, in step S200A, the decoding unit 201 determines that sps_ibc_enabled_flag is 1 and sh_slice_type is I or B (I slice or B slice).

[0390] If yes, proceed to step S201; if no, proceed to step S202.

[0391] Here, in step S200A, in addition to sh_slice_type being I (I slice), sh_slice_type being B (B slice) is also added. Thus, even when the slice including the target block being decoded is a B slice, it is possible to control whether double prediction IBC can be applied on a slice-by-slice basis, thereby expecting an improvement in coding efficiency.

[0392] In the following, use Figures 19 to 24 The decoding control method for various flags related to IBC in the decoding unit 201, which are based on the decoding target block (decoding block, prediction block, or transform block), and the definition (meaning) of each flag are explained.

[0393] Figure 19 This is a diagram illustrating an example of the decoding method for pred_mode_ibc_flag, which is a flag in the decoding unit 201 that controls whether IBC is applied in units of decoding target blocks (decoding blocks, prediction blocks, or transform blocks).

[0394] Here, the decoding unit 201 may determine to apply IBC on a per-decoding target block (decoding block, prediction block, or transform block) when pred_mode_ibc_flag is 1, and determine not to apply IBC when pred_mode_ibc_flag is 0.

[0395] If pred_mode_ibc_flag is not decoded, the decoding unit 201 can estimate pred_mode_ibc_flag as 0.

[0396] Alternatively, when pred_mode_ibc_flag is 1, the decoding unit 201 can also determine whether to apply IBC based on whether the target block is a single tree.

[0397] Specifically, the decoding unit 201 may determine that IBC is applied in the case of a single tree, and determine that IBC is not applied in the case of a dual tree.

[0398] As a variation, even in the case of a dual-tree system with a luma block, the decoding unit 201 determines that IBC is being applied.

[0399] In addition, the decoding unit 201 can, in the same manner as Non-Patent Document 1, use CuPredMode, an internal parameter representing the prediction mode of the decoding target block, to control whether the decoding target block (decoding block, prediction block, or transform block) is an IBC.

[0400] Specifically, the decoding unit 201 may determine that the target block (decoding block, prediction block, or transform block) is IBC when CuPredMode is MODE_IBC, otherwise it may determine that the target block (decoding block, prediction block, or transform block) is not IBC.

[0401] The following is about Figure 19 Instructions are given for the actions.

[0402] like Figure 19 As shown, in step S300, the decoding unit 201 determines whether the predetermined conditions are met.

[0403] If yes, the action proceeds to step S301; otherwise, the action proceeds to step S302.

[0404] In step S301, the decoding unit 201 decodes pred_mode_ibc_flag and ends the process.

[0405] In step S302, the decoding unit 201 terminates the process without decoding pred_mode_ibc_flag.

[0406] Here, the predetermined conditions may include at least one of the following conditions.

[0407] 1. The width or height of the target block being decoded is not 128 pixels (the width or height of the target block being decoded is less than 64 pixels). ※Here, the number of pixels used as the threshold for condition 1 can be changed from 64 pixels to a power of 2 such as 128 pixels or less, 256 pixels or less, 32 pixels or less, etc.

[0408] 2. The prediction mode for the target block being decoded is neither inter-frame prediction nor intra-frame prediction.

[0409] 3. Decoding target block is not a color difference block of a dual-tree architecture. In the following, use Figures 20 to 23 For the decoding unit 201 Figure 19The method for selecting the IBC mode (IBC BVP, IBC merge, IBC BVP / merge) after decoding is explained.

[0410] Figure 20 This is a diagram illustrating an example of a method for selecting the IBC mode (IBC BVP, IBC merge, IBC BVP / merge) in the decoding unit 201.

[0411] like Figure 20 As shown, the decoding unit 201 can select a unique mode from the IBC modes (IBC BVP, IBC merge, and IBC BVP / merge mentioned above). Details are as follows.

[0412] like Figure 20 As shown, in step S400, the decoding unit 201 determines whether general_merge_flag is 1.

[0413] If yes, the action proceeds to step S401; if no, the action proceeds to step S402.

[0414] Here, general_merge_flag is a flag that controls (determines) whether to apply inter-frame prediction or IBC merging on a per-decode target block basis.

[0415] When general_merge_flag is 1, it indicates that the application of inter-frame prediction or IBC merging is valid; when general_merge_flag is 0, it indicates that the application of inter-frame prediction or IBC merging is invalid.

[0416] In step S401, the decoding unit 201 determines whether CuPredMode is MODE_IBC.

[0417] If yes, the action proceeds to step S403; if no, the action proceeds to step S404.

[0418] In step S403, the decoding unit 201 transitions to the mode described later (using...) Figures 21 to 23 (Explanation of decoding processing related to IBC merging)

[0419] In step S404, the decoding unit 201 transitions to decoding processing other than IBC merging, which will be described later.

[0420] In step S402, the decoding unit 201 determines whether CuPredMode is MODE_IBC.

[0421] If yes, the action proceeds to step S405; if no, the action proceeds to step S406.

[0422] In step S405, the decoding unit 201 transitions to the mode described later (using... Figure 23 (Explanation) Decoding processing related to IBC BVP.

[0423] In step S406, the decoding unit 201 transitions to decoding processing other than the IBC BVP described later.

[0424] In the following, use Figure 21 The decoding method for the flag in the decoding unit 201 used to control (determine) whether the double prediction IBC merging is valid (i.e., whether to apply the double prediction IBC merging at the level of the decoding target block) is valid is explained.

[0425] Figure 21 This is a diagram illustrating an example of a decoding method in the decoding unit 201 that controls (determines) whether double-predictive IBC combining is effective (i.e., whether double-predictive IBC combining is applied on a per-decoding-target-block basis).

[0426] like Figure 21 As shown, the decoding unit 201 can decode the flag used to control (determine) whether double-predictive IBC merging is valid (i.e., whether double-predictive IBC merging is applied on a per-decoding-target-block basis). Specifically, as follows.

[0427] like Figure 21 As shown, in step S500, the decoding unit 201 determines whether the predetermined condition 2 is satisfied.

[0428] If yes, the action proceeds to step S501; if no, the action proceeds to step S502.

[0429] In step S501, the decoding unit 201 decodes biibc_merge_flag and ends the process.

[0430] In step S502, the decoding unit 201 terminates the process without decoding biibc_merge_flag.

[0431] Here, biibc_merge_flag is a flag that controls (determines) whether to apply bipredictive IBC merging on a per-decode target block basis (fourth syntax).

[0432] When biibc_merge_flag is 1, the decoding unit 201 determines to apply the double-predicted IBC merging to the decoding target block, and when biibc_merge_flag is 0, it determines not to apply the double-predicted IBC merging to the decoding target block.

[0433] If biibc_merge_flag is not decoded, the decoding unit 201 can estimate biibc_merge_flag as 0.

[0434] In addition, predetermined condition 2 may include at least one of the conditions shown below.

[0435] 1. `sps_ibc_enabled_flag` is set to 1. 2. sh_biibc_enabled_flag is 1 (and / or sps_biibc_enabled_flag is 1) 3. general_merge_flag is 1 4. CupredMode is IBC 5. sh_slice_type is I or B As a variation, a condition based on the block size of the target block (the product of the number of pixels in the vertical and horizontal directions) can be added to the predetermined condition 2.

[0436] For example, you can add a condition that the block size of the target block to be decoded is 16 pixels or more or 32 pixels or more (or that the width or height of the target block to be decoded is 8 pixels or more).

[0437] Therefore, since IBC merging, which can suppress double prediction, is applied to relatively small decoding target blocks, a reduction in processing power can be expected.

[0438] On the other hand, the following conditions can be added: the block size of the decoded target block is less than 32 pixels or less than 16 pixels, and the number of pixels is less than the threshold used in the decision condition for whether to apply single-prediction IBC merging.

[0439] Therefore, since IBC merging, which can suppress double prediction, is applied to relatively large decoding target blocks, a reduction in processing power can be expected.

[0440] In the following, use Figure 22 The decoding method for control information related to IBC merging in the decoding unit 201 will be explained.

[0441] In IBC merging, the decoding unit 201 decodes the IBC merge index (fifth syntax) in order to select one or more merge candidates from the above merge candidate list. The IBC merge index is control information used to determine the merge candidates in the merge candidate list.

[0442] Figure 22 This is a diagram illustrating an example of a decoding method for control information related to IBC merging in the decoding unit 201.

[0443] like Figure 22 As shown, the decoding unit 201 can control the decoding of control information related to IBC merging. Specifically, as follows.

[0444] like Figure 22 As shown, in step S600, the decoding unit 201 determines whether biibc_merge_flag is 1.

[0445] If yes, the action proceeds to step S601; if no, the action proceeds to step S602.

[0446] In step S601, the decoding unit 201 determines whether MaxNumIbcMergeCand is greater than 2.

[0447] If the condition is yes (MaxNumIbcMergeCand is greater than 2), the action proceeds to step S603; if the condition is no (MaxNumIbcMergeCand is less than 2), the action proceeds to step S607.

[0448] In step S603, the decoding unit 201 decodes ibc_merge_idx0.

[0449] Here, ibc_merge_idx0 is the first IBC merge index in a double-predictive IBC merge (sixth syntax).

[0450] In step S604, the decoding unit 201 determines whether ibc_merge_idx0 is not MaxNumIbcMergeCand-2 (ibc_merge_idx0!=MaxNumIbcMergeCand-2).

[0451] If yes, the action proceeds to step S605; if no, the action proceeds to step S606.

[0452] In step S605, the decoding unit 201 decodes ibc_merge_idx1 and ends the process.

[0453] In step S606, the decoding unit 201 terminates the process without decoding ibc_merge_idx1.

[0454] Here, ibc_merge_idx1 is the second IBC merge index in a double-predictive IBC merge (seventh syntax).

[0455] Here, when the condition that ibc_merge_idx0 is not MaxNumIbcMergeCand-2 is not met in step S604, that is, when ibc_merge_idx0 is MaxNumIbcMergeCand-2, as described later, ibc_merge_idx1 is explicitly MaxNumIbcMergeCand-1.

[0456] Therefore, in step S606, the decoding unit 201 does not decode ibc_merge_idx1, thereby expecting to reduce the amount of code.

[0457] In step S607, the decoding unit 201 terminates the process without decoding ibc_merge_idx0 and ibc_merge_idx1.

[0458] Here, as mentioned above, MaxNumIbcMergeCand is the maximum number of merge candidates that can be registered in the merge candidate list for IBC merge. If MaxNumIbcMergeCand is less than 2 in step S601, then the IBC merge candidate used in double-predicted IBC merge is definite even if the two merge indices are not decoded.

[0459] Therefore, in step S607, the decoding unit 201 does not decode ibc_merge_idx0 and ibc_merge_idx1, thereby expecting to reduce the amount of code.

[0460] In step S602, the decoding unit 201 determines whether MaxNumIbcMergeCand is greater than 1.

[0461] If yes, the action proceeds to step S608; if no, the action proceeds to step S609.

[0462] In step S608, the decoding unit 201 decodes ibc_merge_idx and ends the process.

[0463] In step S609, the decoding unit 201 terminates the process without decoding ibc_merge_idx.

[0464] Here, if MaxNumIbcMergeCand is not greater than 1 in step S602, that is, MaxNumIbcMergeCand is 1, then the IBC merge candidate used is clear even if the selected merge index is not decoded in the single-prediction IBC merge.

[0465] Therefore, in step S609, the decoding unit 201 does not decode ibc_merge_idx, thereby expecting to reduce the amount of code.

[0466] Furthermore, although the merge indexes (ibc_merge_idx0 and ibc_merge_idx1) for dual-prediction IBC merging and the merge index (ibc_merge_idx) for single-prediction IBC merging have been described above, ibc_merge_idx0 and ibc_merge_idx can be shared in the image decoding device 200.

[0467] That is, the design of context values ​​for decoding (encoding), truncated binarization, and other decoding methods from binary to multi-value (encoding from multi-value) can be shared.

[0468] Figure 23 It shows the use Figure 22 The following is an example of a modification to the decoding method for control information related to IBC merging (constructing a method corresponding to the number of block vectors, such as...). Figure 7 The diagram shows different lists.

[0469] like Figure 23 As shown, the decoding unit 201 can decode control information related to IBC merging. Details are as follows.

[0470] like Figure 23 As shown, in step S600, the decoding unit 201 determines whether biibc_merge_flag is 1.

[0471] If yes, the action proceeds to step S601A; if no, the action proceeds to step S602.

[0472] In step S601A, the decoding unit 201 determines whether MaxNumIbcMergeCand is greater than 1.

[0473] If yes, the action proceeds to step S608A; if no (if MaxNumIbcMergeCand is 1 or less), the action proceeds to step S609A.

[0474] In step S608A, the decoding unit 201 decodes ibc_merge_idx and ends the process.

[0475] In step S609A, the decoding unit 201 terminates the process without decoding ibc_merge_idx.

[0476] Steps S602, S608, and S609 Figure 22 same.

[0477] In the following, use Figure 24 The decoding method for the flag in the decoding unit 201 used to control (determine) whether IBC BVP / merging is valid (i.e., whether to apply IBC BVP / merging on a per-decoding-target-block basis) is explained.

[0478] Figure 24 This is a diagram illustrating an example of a decoding method in the decoding unit 201 for controlling (determining) whether IBC BVP / merging is valid (i.e., whether to apply IBC BVP / merging on a per-decoding-target-block basis) of the flag.

[0479] like Figure 24 As shown, the decoding unit 201 can decode the flags used to control (determine) whether the IBC BVP / merge is valid (i.e., whether to apply IBC BVP / merge on a per-decoding-target-block basis) on a per-block basis. Specifically, as follows.

[0480] like Figure 24 As shown, in step S700, the decoding unit 201 determines whether the predetermined condition 3 is satisfied.

[0481] If yes, the action proceeds to step S701; if no, the action proceeds to step S702.

[0482] In step S701, the decoding unit 201 decodes ibc_bvpmerge_flag and ends the process.

[0483] In step S702, the decoding unit 201 terminates the process without decoding ibc_bvpmerge_flag.

[0484] Here, ibc_bvpmerge_flag is a flag that controls (determines) whether IBC BVP / merge is valid (i.e., whether IBC BVP / merge is applied on a per-decode-target-block basis) (eighth syntax).

[0485] When ibc_bvpmerge_flag is 1, the decoding unit 201 determines that IBC BVP / merge will be applied to the decoding target block, and when ibc_bvpmerge_flag is 0, it determines that IBC BVP / merge will not be applied to the decoding target block.

[0486] If ibc_bvpmerge_flag is not decoded, the decoding unit 201 can estimate ibc_bvpmerge_flag as 0.

[0487] In addition, predetermined condition 3 may include at least one of the conditions shown below.

[0488] 6. `sps_ibc_enabled_flag` is 1 7. general_merge_flag is 1 8. CupredMode is IBC 9. sh_slice_type is I or B As a variation, a condition based on the block size of the target block (the product of the number of pixels in the vertical and horizontal directions) can be added to the predetermined condition 3.

[0489] For example, you can add a condition that the block size of the target block to be decoded is 16 pixels or more or 32 pixels or more (or that the width or height of the target block to be decoded is 8 pixels or more).

[0490] Therefore, since IBC BVP / merging can be suppressed for relatively small decoding target blocks, a reduction in processing volume can be expected.

[0491] On the other hand, the following conditions can be added: the block size of the decoded target block is less than 32 pixels or less than 16 pixels, and the number of pixels is less than the threshold used in the decision condition for whether to apply single-prediction IBC merging.

[0492] Therefore, since IBC BVP / merging can be suppressed for relatively large decoding target blocks, a reduction in processing power can be expected. In the following, using... Figure 25 The decoding method for control information related to IBC BVP / merging in the decoding unit 201 will be explained.

[0493] Figure 25 This is a diagram illustrating an example of a decoding method for control information related to IBC BVP / merging in the decoding unit 201.

[0494] like Figure 25 As shown, the decoding unit 201 can decode control information related to IBC BVP / merging. Details are as follows.

[0495] like Figure 25 As shown, in step S800, the decoding unit 201 determines whether ibc_bvpmerge_flag is 1.

[0496] If yes, this action proceeds to step S801; if no, this action ends the process.

[0497] In step S801, the decoding unit 201 decodes the control information related to BVP and BVD and proceeds to step S802.

[0498] In step S802, the decoding unit 201 determines whether MaxNumIbcMergeCand is greater than 1.

[0499] If yes, the action proceeds to step S803; if no (if MaxNumIbcMergeCand is less than 1), the action proceeds to step S804.

[0500] In step S803, the decoding unit 201 decodes ibc_merge_idx and ends the process.

[0501] In step S804, the decoding unit 201 terminates the process without decoding ibc_merge_idx.

[0502] Here, as mentioned above, MaxNumIbcMergeCand is the maximum number of merge candidates that can be registered in the merge candidate list in IBC merge. If MaxNumIbcMergeCand is less than 1 in step S801, then the IBC merge candidate used is clear even if the merge index is not decoded in IBC BVP / merge.

[0503] Therefore, in step S804, the decoding unit 201 does not decode ibc_merge_idx, thereby expecting to reduce the amount of code.

[0504] In the following, use Figure 26 The decoding method for the control information related to the MBVD of the BV of the correction IBC in the decoding unit 201 will be described.

[0505] Figure 26 This is a diagram illustrating an example of a decoding method for the control information related to the MBVD described above in the BV of the corrected IBC in the decoding unit 201.

[0506] like Figure 26 As shown, the decoding unit 201 can decode the control information related to the MBVD of the BV of the IBC correction. Specifically, as follows.

[0507] like Figure 26 As shown, in step S901, the decoding unit 201 determines whether biibc_merge_flag is 1.

[0508] If yes, the action proceeds to step S902; if no, the action proceeds to step S903.

[0509] In step S902, the decoding unit 201 determines whether ibc_mbvd_flag is 1.

[0510] If yes, the action proceeds to step S904; if no, the action proceeds to step S905.

[0511] Here, ibc_mbvd_flag is a flag used to control whether MBVD is applied on a per-decode target block basis.

[0512] The decoding unit 201 determines that MBVD should be applied when ibc_mbvd_flag is 1, and determines that MBVD should not be applied when ibc_mbvd_flag is 0.

[0513] If ibc_mbvd_flag is not decoded, the decoding unit 201 can estimate ibc_mbvd_flag as 0.

[0514] In step S904, the decoding unit 201 determines whether bimbvd_flag is 1.

[0515] If yes, the action proceeds to step S906; if no, the action proceeds to step S907.

[0516] Here, bimbvd_flag is a flag used to control whether MBVD is applied to the two combination candidates (BV) in IBC merging for double prediction on a per-decode target block basis.

[0517] When bimbvd_flag is 1, the decoding unit 201 determines that MBVD should be applied to the two merge candidates (BV), and when bimbvd_flag is 0, it determines that MBVD should not be applied to the two merge candidates (BV) (that is, MBVD is only applied to one merge candidate (BV) in the IBC merge of the two predictions).

[0518] If bimbvd_flag is not decoded, the decoding unit 201 can estimate bimbvd_flag as 0.

[0519] In step S906, the decoding unit 201 decodes mbvd_merge_cand_idx0 and mbvd_merge_cand_idx1.

[0520] Here, mbvd_merge_cand_idx0 is an index for selecting, in units of the decoding target block, the first combination of merge candidates (BV) to which MBVD is applied from two combinations of merge candidates (BV) in the dual-prediction IBC merge from the merge candidate list.

[0521] In the case where mbvd_merge_cand_idx0 is not decoded, the decoding unit 201 may estimate mbvd_merge_cand_idx0 as 0.

[0522] In addition, mbvd_merge_cand_idx1 is an index for selecting, in units of the decoding target block, the second combination of merge candidates (BV) to which MBVD is applied from two combinations of merge candidates (BV) in the dual-prediction IBC merge from the merge candidate list.

[0523] In the case where mbvd_merge_cand_idx1 is not decoded, the decoding unit 201 may estimate mbvd_merge_cand_idx1 as 0.

[0524] In addition, when the decoding unit 201 decodes mbvd_merge_cand_idx0 and mbvd_merge_cand_idx1, a size relationship may not be given to them in advance, or a size relationship of mbvd_merge_cand_idx0 < mbvd_merge_cand_idx1 may be given.

[0525] In the case where a size relationship is not given, after the decoding unit 201 decodes mbvd_merge_cand_idx0, it excludes the merge candidate corresponding to mbvd_merge_cand_idx0 from the merge candidate list, and then by decoding mbvd_merge_cand_idx1, even if mbvd_merge_cand_idx0 and mbvd_merge_cand_idx1 are the same, different merge candidates can be derived, so an improvement in coding efficiency can be expected.

[0526] is an index for selecting, in units of the decoding target block, the first combination of merge candidates (BV) to which MBVD is applied from two combinations of merge candidates (BV) in the dual-prediction IBC merge from the merge candidate list.

[0527] If mbvd_merge_cand_idx0 is not decoded, the decoding unit 201 can estimate mbvd_merge_cand_idx0 as 0.

[0528] In step S908, the decoding unit 201 decodes mbvd_idx0 and mbvd_idx1 and ends the process.

[0529] Here, mbvd_idx0 and mbvd_idx1 are indices used to derive the correction vectors for the first and second group of merge candidates (BV) in the IBC merge of two predictions, respectively.

[0530] Specifically, as disclosed in Non-Patent Document 2, the values ​​of mbvd_idx0 and mbvd_idx1 correspond to the discrete reference positions that the correction vector can take.

[0531] The discrete reference position is defined based on the direction and distance of the correction vector with the BV of the merged candidate as the origin.

[0532] The directions can be formed by up, down, left, and right (a total of four directions including vertical and horizontal), as in Non-Patent Document 1, or by forming eight directions by adding four tilting directions of 45 degrees, 135 degrees, 225 degrees, and 315 degrees to up, down, left, and right, as in Non-Patent Document 2.

[0533] Furthermore, regarding the distance, when the pixel precision of IBC is limited to integer precision in units of sequences, images, slices, or decoded target blocks (i.e., only integer precision pixels can be selected), the distance can be configured as 1 pixel, 4 pixels, 8 pixels, 12 pixels, 16 pixels, 24 pixels, 32 pixels, 40 pixels, 48 ​​pixels, 56 pixels, 72 pixels, 80 pixels, 88 pixels, 96 pixels, 104 pixels, 112 pixels, 120 pixels, and 128 pixels, as in Non-Patent Document 2.

[0534] As another example, when the pixel precision of IBC is not limited to integer precision in units of sequence, image, slice, or decoded target block (i.e., both fractional pixel precision and integer pixel precision can be selected), in addition to the integer pixel option mentioned above, 1 / 4 pixel or 1 / 8 pixel can also be added.

[0535] In step S909, the decoding unit 201 advances to... Figure 22 In step S601, if this action ends, then this process ends.

[0536] In step S907, the decoding unit 201 decodes mbvd_merge_cand_idx0.

[0537] In step S910, the decoding unit 201 decodes mbvd_idx0.

[0538] In step S911, the decoding unit 201 advances to... Figure 22 In step S601, if this action ends, then this process ends.

[0539] In step S905, the decoding unit 201 advances to... Figure 22 In step S601, if this action ends, then this process ends.

[0540] In step S903, the decoding unit 201 advances to... Figure 22 In step S602, if this action ends, then this process ends.

[0541] The following is for reference Figure 27 A modified example of applying the second-frame intra-prediction method on a block-by-block basis is explained.

[0542] like Figure 27 As shown, in step S1001, the decoding unit 201 determines whether the target block to be decoded is in the mode of maintaining BV.

[0543] If yes, the action proceeds to step S1002; if no, the action ends.

[0544] In step S1002, the decoding unit 201 decodes cu_ibc_flag, which is a control signal indicating the number of BVs used in the prediction within the second frame.

[0545] Here, by setting the maximum number of BVs to 2, the decoding unit 201 sets the number of BVs to 1 and decodes when cu_ibc_flag is 0, and sets the number of BVs to 2 and decodes when cu_ibc_flag is 1.

[0546] In step S1003, the decoding unit 201 determines whether there is one or two BV.

[0547] In other words, in step S1003, the decoding unit 201 determines whether cu_ibc_flag is 0.

[0548] When BV is 1 (cu_ibc_flag==0) or when the BVP candidate list uses the L0 list and the L1 list, in step S1004, the decoding unit 201 decodes cu_ibc_idx0, where cu_ibc_idx0 is the control signal that specifies BV from the BVP candidate list.

[0549] When there is not one BV (cu_ibc_flag!=0) and the BVP candidate list consists only of the L0 list, in steps S1004 and S1005, the decoding unit 201 decodes cu_ibc_idx0 and cu_ibc_idx1, where cu_ibc_idx0 and cu_ibc_idx1 are control signals that specify BV from the BVP candidate list.

[0550] When cu_ibc_idx1 is represented by relative coordinates, the decoding unit 201 decodes cu_ibc_idx0+cu_ibc_idx1 into cu_ibc_idx1 as absolute coordinates.

[0551] Figure 28 A flowchart is shown illustrating the case where the correction BV is applied to BV.

[0552] like Figure 28 As shown, in step S1101, the decoding unit 201 determines whether the target block to be decoded is in the mode of maintaining BV.

[0553] If yes, the action proceeds to step S1102; if no, the action ends.

[0554] In step S1102, the decoding unit 201 decodes cu_ibc_flag, which is a control signal indicating the number of BVs used in the prediction within the second frame.

[0555] Here, by setting the maximum number of BVs to 2, the decoding unit 201 sets the number of BVs to 1 and decodes when cu_ibc_flag is 0, and sets the number of BVs to 2 and decodes when cu_ibc_flag is 1.

[0556] In step S1103, the decoding unit 201 determines whether there is one or two BV.

[0557] In other words, in step S1103, the decoding unit 201 determines whether cu_ibc_flag is 0.

[0558] When BV is 1 (cu_ibc_flag==0) or when the BVP candidate list uses the L0 list and the L1 list, in step S1104, the decoding unit 201 decodes cu_ibc_idx0, where cu_ibc_idx0 is the control signal that specifies BV from the BVP candidate list.

[0559] In step S1105, the decoding unit 201 decodes cu_ibc_dmv0, which is the control information representing the correction BV, and adds cu_ibc_dmv0 to BV.

[0560] When there is not one BV (cu_ibc_flag!=0) and the BVP candidate list consists only of the L0 list, in steps S1106 and S1107, the decoding unit 201 decodes cu_ibc_idx0 and cu_ibc_idx1, where cu_ibc_idx0 and cu_ibc_idx1 are control signals that specify BV from the BVP candidate list.

[0561] When cu_ibc_idx1 is represented by relative coordinates, the decoding unit 201 decodes cu_ibc_idx0+cu_ibc_idx1 into cu_ibc_idx1 as absolute coordinates.

[0562] In steps S1108 and S1109, the decoding unit 201 decodes cu_ibc_dmv0 and cu_ibc_dmv1, which are control information representing the correction BV, and adds cu_ibc_dmv0 and cu_ibc_dmv1 to the corresponding BV respectively.

[0563] In the following, use Figure 29 The decoding method for the flag in the decoding unit 201 used to control (determine) whether the double prediction IBC BVP is valid (i.e., whether to apply the double prediction IBC BVP on the basis of the decoding target block) is valid will be explained.

[0564] Figure 29 This is a flowchart illustrating an example of a decoding method in the decoding unit 201 for controlling (determining) whether the IBC BVP of double prediction is valid (i.e., whether to apply the IBC BVP of double prediction per unit of the decoding target block) in a decoding unit 201.

[0565] like Figure 29 As shown, the decoding unit 201 can decode the flag used to control (determine) whether the double prediction IBC BVP is valid (i.e., whether to apply the double prediction IBC BVP on a per-decoding-target-block basis). Specifically, as follows.

[0566] like Figure 29 As shown, in step S1200, the decoding unit 201 determines whether the predetermined condition 4 is satisfied.

[0567] If yes, the action proceeds to step S1201; if no, the action proceeds to step S1202.

[0568] In step S1201, the decoding unit 201 decodes biibc_bvp_flag and ends the process.

[0569] In step S1202, the decoding unit 201 terminates the process without decoding biibc_bvp_flag.

[0570] Here, biibc_bvp_flag is a flag that controls (determines) whether to apply IBCBVP with double prediction per decoded target block (Ninth syntax).

[0571] When biibc_bvp_flag is 1, the decoding unit 201 determines to apply the double-predicted IBC BVP to the decoding target block, and when biibc_bvp_flag is 0, it determines not to apply the double-predicted IBC BVP to the decoding target block.

[0572] If biibc_bvp_flag is not decoded, the decoding unit 201 can estimate biibc_bvp_flag as 0.

[0573] In addition, predetermined condition 4 may include at least one of the conditions shown below.

[0574] 1. `sps_ibc_enabled_flag` is set to 1. 2. sh_biibc_enabled_flag is 1 (and / or sps_biibc_enabled_flag is 1) 3. general_merge_flag is 0 4. CupredMode is IBC 5. sh_slice_type is I or B As a variation, a condition based on the block size of the target block (the product of the number of pixels in the vertical and horizontal directions) can be added to the predetermined condition 4.

[0575] For example, you can add a condition that the block size of the target block to be decoded is 16 pixels or more or 32 pixels or more (or the width or height of the target block to be decoded is 8 pixels or more).

[0576] Therefore, since IBC merging, which can suppress double prediction, is applied to relatively small decoding target blocks, a reduction in processing power can be expected.

[0577] On the other hand, the following conditions can be added: the block size of the decoded target block is less than 32 pixels or less than 16 pixels, and the number of pixels is less than the threshold used in the decision condition for whether to apply single prediction IBC BVP.

[0578] Therefore, since IBC BVP, which can suppress double prediction, is applied to relatively large decoding target blocks, a reduction in processing power can be expected.

[0579] In the following, use Figure 30 The decoding method for control information related to IBC BVP in the decoding unit 201 will be explained.

[0580] Figure 30 This is a flowchart illustrating an example of a decoding method for control information related to IBC BVP in the decoding unit 201.

[0581] like Figure 30 As shown, the decoding unit 201 can control the decoding of control information related to the IBC BVP. Specifically, as follows.

[0582] like Figure 30 As shown, in step S1300, the decoding unit 201 determines whether biibc_bvp_flag is 1.

[0583] If yes, the action proceeds to step S1301; if no, the action proceeds to step S1302.

[0584] In step S1301, the decoding unit 201 decodes the control information related to the first set of BVP and BVD in the two sets of BVP and BVD in the dual prediction IBC BVP.

[0585] In step S1303, the decoding unit 201 decodes the control information related to the second set of BVP and BVD in the two sets of BVP and BVD in the dual prediction IBC BVP, and ends the process.

[0586] In step S1302, the decoding unit 201 decodes the control information related to a set of BVPs and BVDs in the IBC BVP and ends the process.

[0587] (Binarization of dual-prediction IBC) use Figure 31 The binarization method of the IBC merging index in the decoding unit 201 will be explained.

[0588] Figure 31 This is a diagram illustrating an example of the binarization method for the aforementioned IBC merge index in the decoding unit 201.

[0589] like Figure 31 As shown, the decoding unit 201 can binarize the IBC merge index. Specifically, as follows.

[0590] First, for ibc_merge_idx, the decoding unit 201 can set the value of cMax to MaxNumIbcMerge-1.

[0591] Here, cMax is the maximum value of the multiple values ​​of the target's control information before binarization.

[0592] Secondly, for ibc_merge_idx0, the decoding unit 201 can set the value of cMax to MaxNumIbcMerge-1.

[0593] Third, for ibc_merge_idx1, the decoding unit 201 can change the value of cMax based on whether biibc_merge_flag is 1.

[0594] Specifically, when biibc_merge_flag is not 1, the decoding unit 201 can set the value of cMax to MaxNumIbcMerge-2.

[0595] On the other hand, when biibc_merge_flag is 1, the decoding unit 201 can set the value of cMax to MaxNumIbcMerge-ibc_merge_idx0-2.

[0596] In other words, when using two different indices (ibc_merge_idx0 and ibc_merge_idx1) to derive two BVs, the range of possible values ​​for ibc_merge_idx1 is 0 (minimum) to the value obtained by subtracting the value of idx0 from the maximum value of the BVP candidate list (MaxNumIbcMerge-2 in the example above).

[0597] As described above, by designing a binarization method, the decoding unit 201 can decode the second and subsequent indices into relative indices starting from the immediately preceding index.

[0598] The aforementioned image decoding device 200 can be a program that enables a computer to perform various functions (steps) and is implemented.

[0599] Industrial availability Furthermore, according to this embodiment, for example, since it is possible to improve the overall quality of service in video communication, it is possible to contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs): “Building resilient infrastructure, promoting sustainable industrialization and pursuing the expansion of innovation.”

[0600] Figure Labels 200 Image Decoding Device 201 Decoding Department 202 Inverse Quantization Department 203 Inverse Transformation Unit 204 First Frame Intra-Prediction Unit 205 Second-frame intra-prediction unit 205A Block Vector Derivation Unit (BV Derivation Unit) 205B Second Intra-Frame Predictive Pixel Generation Unit 206 Inter-frame Prediction Unit 207 Adder 208 Storage Unit 210 Code Input Section 220 Image Output Unit

Claims

1. An image decoding device, characterized in that, include: The decoding unit decodes control information and quantization values. The inverse quantization unit performs inverse quantization on the quantized value and uses it as a transformation coefficient; The inverse transform unit performs an inverse transform on the transform coefficients and uses them as prediction residuals; The first intra-frame prediction unit generates a first predicted pixel based on the decoded pixel and the control information; The second intra-frame prediction unit generates a second predicted pixel from two or more block vectors with integer pixel precision or fractional pixel precision for the decoded target block, based on the decoded pixel and the control information. Storage unit, storing the decoded pixels; The inter-frame prediction unit generates a third predicted pixel based on the stored decoded pixels and the control information. as well as An adder adds the prediction residual to the first to third prediction pixels to obtain the decoded pixel.

2. The image decoding device according to claim 1, characterized in that, The second intra-frame prediction unit sets the pixel precision of the block vector to a plurality of different integer pixel precisions.

3. The image decoding apparatus according to claim 1, characterized in that, The second intra-frame prediction unit sets the pixel precision of the block vector to a plurality of different fractional pixel precisions.

4. The image decoding apparatus according to claim 1, characterized in that, The second intra-frame prediction unit modifies the pixel precision of the block vector using any combination of sequence units, image units, slice units, and block units.

5. The image decoding apparatus according to claim 1, characterized in that, The second intra-frame prediction unit changes the pixel precision of the block vector based on the number of block vectors in the decoded target block.

6. The image decoding apparatus according to claim 1, characterized in that, The more the number of block vectors increases, the coarser the pixel precision of the block vectors becomes in the second intra-frame prediction unit.

7. An image decoding method, characterized in that, include: The steps for decoding control information and quantization values; The step of inverse quantization of the quantized value and using it as a transform coefficient; The inverse transformation of the transformation coefficients is then used as a step to predict the residuals. The step of generating the first predicted pixel based on the decoded pixel and the control information; The step of generating a second predicted pixel from two or more block vectors with integer pixel precision or fractional pixel precision for the decoded target block, based on the decoded pixel and the control information; The step of storing the decoded pixels; The step of generating a third predicted pixel based on the stored decoded pixels and the control information; as well as The step of adding the prediction residual to the first to third prediction pixels to obtain the decoded pixel.

8. A program, characterized in that, The program enables the computer to function as an image decoding device. The image decoding device includes: The decoding unit decodes control information and quantization values. The inverse quantization unit performs inverse quantization on the quantized value and uses it as a transformation coefficient; The inverse transform unit performs an inverse transform on the transform coefficients and uses them as prediction residuals; The first intra-frame prediction unit generates a first predicted pixel based on the decoded pixel and the control information; The second intra-frame prediction unit generates a second predicted pixel from two or more block vectors with integer pixel precision or fractional pixel precision for the decoded target block, based on the decoded pixel and the control information. Storage unit, storing the decoded pixels; The inter-frame prediction unit generates a third predicted pixel based on the stored decoded pixels and the control information; and An adder adds the prediction residual to the first to third prediction pixels to obtain the decoded pixel.

Images