Image decoding device, image decoding method, and program

Abstract

An image decoding device 200 according to the present invention comprises a first intra-frame prediction unit 204 that generates a first prediction pixel on the basis of decoded pixels and control information, a second intra-frame prediction unit 205 that constructs a reference list on the basis of the decoded pixels and the control information and generates a second prediction image using the reference list, an accumulation unit 208 that accumulates the decoded pixels, an inter-frame prediction unit 206 that generates a third prediction pixel on the basis of the accumulated decoded pixels and the control information, and an adder 207 that adds a prediction residual and the first to third prediction pixels to obtain a decoded pixel. When performing intra-template matching prediction, the second intra-frame prediction unit 205, in a search for a block vector of a second hierarchical level, registers a block vector that is of a first hierarchical level and has a cost lower than the maximum cost + M (M > 0) in the reference list.

Description

Image decoding device, image decoding method, and program

[0001] The present invention relates to an image decoding device, an image decoding method, and a program.

[0002] Non-Patent Documents 1 and 2 disclose intra template matching prediction (IntraTMP).

[0003] IntraTMP refers to pixels that match in template matching from the decoded pixel area of ​​the frame to be decoded, and sets these as predicted pixels for the block to be decoded.

[0004] Specifically, the image encoding device uses decoded neighboring pixels as a template, searches for coordinates in the same frame that have a low template matching cost, and defines the displacement amount to the coordinates as a block vector (BV).

[0005] The image coding apparatus constructs a BV list (reference list) in ascending order of template matching cost, and codes the index of this BV list.

[0006] The image decoding device also searches in the same way and reconstructs the BV list, thereby decoding the BV from the index of the BV list.

[0007] As shown in FIG. 2, the image decoding apparatus copies pixels from a reference block indicated by the BV of the block to be decoded and uses them as predicted pixels.

[0008] K. Naser, et al., “EE2: Intra Template Matching,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, JVET-V0130, 22nd meeting, Apr. 2021M. Coban, et al., “Algorithm description of Enhanced Compression Model 9 (ECM 9)”, JVET-AD2025, 2023

[0009] In Non-Patent Documents 1 and 2, the sum of absolute differences (SAD) is used as the cost of template matching.

[0010] In addition, in Non-Patent Document 2, a BV search is performed in two layers in order to speed up the search.

[0011] Here, in the first layer BV search, the BV is roughly searched and the coordinate with the smallest SAD is selected. In the second layer BV search, the BV is searched in detail, centering on the coordinate with the smallest SAD in the first layer.

[0012] At this time, in order to avoid falling into a local minimum in the search for BVs in the first layer, the top N BV candidates with smallest SAD are retained.

[0013] Furthermore, in the second layer, when calculating the SAD of the surroundings around each BV candidate, the BV candidates are limited based on the maximum SAD among the BV candidates.

[0014] However, if the top N SADs are all the same value, the BV to be searched in the second layer is lost, and there is a problem that a valid BV cannot be searched.

[0015] Therefore, the present invention has been made in consideration of the above-mentioned problems, and an object of the present invention is to provide an image decoding device, an image decoding method, and a program with high coding efficiency.

[0016] a storage unit that stores the decoded pixels; an inter-frame prediction unit that generates a third predicted pixel based on the stored decoded pixels and the control information; and an adder that adds the prediction residual and the first to third predicted pixels to obtain the decoded pixel, wherein when performing intra template matching prediction, the second intra-frame prediction unit registers a block vector V having a cost smaller than (a maximum cost in a first layer reference list)+M (M>0) in a search for a block vector in a second layer.

[0017] a step of generating a second predicted pixel based on the control information and the reference list; a step of storing the decoded pixels; a step of storing a third predicted pixel based on the stored decoded pixels and the control information; and a step of adding the prediction residual and the first to third predicted pixels to obtain the decoded pixel; and wherein, in step E, when intra template matching prediction is performed, a block vector having a cost smaller than (the maximum cost in the reference list of the first layer)+M (M>0) is registered in a search for a block vector in a second layer.

[0018] a storage unit that stores the decoded pixels; an inter-frame prediction unit that generates a third predicted pixel based on the stored decoded pixels and the control information; and an adder that adds the prediction residual and the first to third predicted pixels to obtain the decoded pixel; and wherein, when performing intra template matching prediction, the second intra-frame prediction unit registers a block vector having a cost smaller than (a maximum cost in a first layer reference list)+M (M>0) in a search for a block vector in a second layer.

[0019] a storage unit that stores the decoded pixels; an inter-frame prediction unit that generates a third predicted pixel based on the stored decoded pixels and the control information; and an adder that adds the prediction residual and the first to third predicted pixels to obtain the decoded pixel; and wherein, when performing intra template matching prediction, if a block vector having a cost smaller than a maximum cost in a reference list of a first layer is not present in a search for a block vector in a second layer, the second intra-frame prediction unit registers an arbitrary block vector in the reference list of a first layer in the reference list of a second layer.

[0020] According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program with high coding efficiency.

[0021] Fig. 1 is a diagram illustrating an example of functional blocks of an image decoding device 200 according to an embodiment. Fig. 2 is a diagram illustrating an example of an IntraTMP. Fig. 3 is a flowchart illustrating an example of a method for setting a correction method on a sequence-by-sequence basis. Fig. 4 is a flowchart illustrating an example of a method for controlling a search for a BV of an IntraTMP on a block-by-block basis. Fig. 5 is a diagram illustrating an example of a template. Fig. 6 is a diagram illustrating an example of a BV list.

[0022] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0023] First Embodiment An image decoding device 200 according to this embodiment will be described below with reference to Fig. 1 to Fig. 6. Fig. 1 is a diagram showing an example of functional blocks of the image decoding device 200 according to this embodiment.

[0024] As shown in FIG. 1, the image decoding device 200 includes a code input unit 210, a decoding unit 201, an inverse quantization unit 202, an inverse transform unit 203, a first intra-frame prediction unit 204, a second intra-frame prediction unit 205, an inter-frame prediction unit 206, an adder 207, a storage unit 208, and an image output unit 220.

[0025] The code input unit 210 is configured to obtain code information coded by the image coding device.

[0026] The decoding unit 201 is configured to decode the control information and the quantized value from the code information input from the code input unit 210. For example, the decoding unit 201 is configured to output the control information and the quantized value by performing variable length decoding on the code information.

[0027] Here, the quantized value is sent to the inverse quantization unit 202, and the control information is sent to the first intra-frame prediction unit 204, the second intra-frame prediction unit 205, and the inter-frame prediction unit 206. Note that this control information includes information necessary for controlling the first intra-frame prediction unit 204, the second intra-frame prediction unit 205, the inter-frame prediction unit 206, etc., and may include header information such as a sequence parameter set, a picture parameter set, a picture header, and a slice header.

[0028] The inverse quantization unit 202 is configured to inversely quantize the quantized values ​​sent from the decoding unit 201 to generate transform coefficients. The transform coefficients are sent to the inverse transform unit 203.

[0029] The inverse transform unit 203 is configured to inversely transform the transform coefficients sent from the inverse quantization unit 202 to generate prediction residuals. The prediction residuals are sent to an adder 207.

[0030] The first intra-frame prediction unit 204 is configured to generate a first predicted pixel to be added to the prediction residual by the adder 207, based on the decoded pixel obtained via the adder 207 and the control information decoded by the decoding unit 201. The first predicted pixel is sent to the adder 207 and the second intra-frame prediction unit 205.

[0031] The inter-frame prediction unit 206 is configured to generate a third predicted pixel to be added to the prediction residual by the adder 207, based on the decoded pixel obtained by referring to the storage unit 208 and the control information decoded by the decoding unit 201. The third predicted pixel is sent to the adder 207.

[0032] The storage unit 208 is configured to cumulatively store the decoded pixels sent from the adder 207. The decoded pixels are referenced by the inter-frame prediction unit 206 via the storage unit 208.

[0033] The adder 207 is configured to obtain decoded pixels by adding the prediction residual sent from the inverse transform unit 203 to any one of the first to third predicted pixels sent from the first intra-frame prediction unit 204, the second intra-frame prediction unit 205, and the inter-frame prediction unit 206. These decoded pixels are sent to the image output unit 220, the storage unit 208, the first intra-frame prediction unit 204, and the second intra-frame prediction unit 205.

[0034] (Second Intra-Frame Prediction Unit 205) Hereinafter, an example of a method for deriving second predicted pixels by the second intra-frame prediction unit 205 will be described.

[0035] The role of the second intra-frame prediction unit 205 is to derive a block vector (hereinafter referred to as BV) for the block to be decoded and predict the pixels of the block referenced by the BV (second intra-frame prediction) in order to compensate the block to be decoded with high precision in the subsequent adder 207.

[0036] An example of the second intra-frame prediction is intra-template matching prediction (hereinafter referred to as IntraTMP) disclosed in Non-Patent Document 2.

[0037] When performing IntraTMP, the second intra-frame prediction unit 205 uses a decoded pixel in the same frame as a template, searches for coordinates in the same frame that match the template, sets the displacement amount to the coordinates as BV, and references the block displaced from the block to be decoded to BV as the predicted pixel of the block to be decoded.

[0038] The template can use neighboring pixels of the block to be decoded and can use any number of lines. Also, as shown in Figure 5, the template can use both neighboring pixels above and to the left of the block to be decoded, or can use only neighboring pixels above or only neighboring pixels to the left.

[0039] Here, when performing IntraTMP, the second intra-frame prediction unit 205 searches for BVs hierarchically in order to reduce the search time.

[0040] The number of layers can be set to any value, but the following describes an example in which the second intra-frame prediction unit 205 searches for BVs in two layers. Also, a case in which SAD (Sum of Absolute Difference) is used as the cost of template matching for BV search will be described, but any cost such as SATD can be used.

[0041] The second intra-frame prediction unit 205 roughly searches for BVs in the first layer and selects BV candidates, and then searches for BVs in detail in the second layer and determines BVs.

[0042] Here, the second intra-frame prediction unit 205 may be configured to change the coarseness and fineness of the BV search based on control information that sets the coarseness and fineness of the BV search when roughly searching for the BV in the first layer.

[0043] For example, in screen content such as a document, coordinates that perfectly match may exist, and even a deviation of one pixel will cause a large change in SAD, so it is desirable to set the BV to be searched for closely in screen content.

[0044] On the other hand, in camera-captured content, the distribution of SAD is relatively gentle, so it is desirable to set the BV to be sparsely searched for in camera-captured content.

[0045] Furthermore, the second intra-frame prediction unit 205 may be configured to change the coarseness or fineness of the BV search depending on the BV search direction without using explicit control information.

[0046] When the same texture is repeated, it is often the case that the coordinates that perfectly match are located directly to the side or directly above, so it is desirable to search for BVs densely in the direction directly to the side or directly above, while it is desirable to search for BVs sparsely in the diagonal direction.

[0047] Alternatively, the second intra-frame prediction unit 205 may be configured to change the coarseness or fineness of the BV search depending on the block size, without using explicit control information.

[0048] The smaller the block size, the easier it is to search for coordinates that perfectly match, so it is desirable to search for BVs more densely as the block size becomes smaller.

[0049] Conversely, the larger the block size, the greater the improvement in coding efficiency when matching coordinates can be found, so the larger the block size, the more densely BVs may be searched.

[0050] In either case, the second intra-frame prediction unit 205 selects the top N BV candidates in the first layer BV search and uses them in the second layer BV search.

[0051] When searching for BVs hierarchically, the second intra-frame prediction unit 205 constructs a BV list for each layer.

[0052] Note that a BV list can be prepared for each type of template. For example, as shown in Fig. 5, different BV lists are prepared for the case where both neighboring pixels above and to the left of the block to be decoded are used as templates, the case where only neighboring pixels above the block to be decoded are used as templates, and the case where only neighboring pixels to the left of the block to be decoded are used as templates. The lengths of the respective BV lists may be the same or different.

[0053] The SAD calculation involves calculating the SAD using a template of only the neighboring pixels above the block to be decoded and the SAD using a template of only the neighboring pixels to the left of the block to be decoded, and then calculating the sum of both SADs, thereby reducing the amount of calculation.

[0054] Alternatively, after calculating the SAD using a template for only the neighboring pixels above the block to be decoded and the SAD using a template for only the neighboring pixels to the left of the block to be decoded, the SAD calculation can be performed by subtracting the amount corresponding to each template from both the SAD using a template for only the neighboring pixels above the block to be decoded and the SAD using a template for only the neighboring pixels to the left of the block to be decoded.This can reduce the amount of calculation.The second intra-frame prediction unit 205 prepares a BV list in which multiple BVs can be registered in each layer, and when calculating the SAD for each coordinate, registers only BVs in the BV list that are smaller than a predetermined SAD.

[0055] If the BV list of the previous layer is completely filled, the predetermined SAD is set to the maximum value of the SAD of the BV list; otherwise, it is set to a preset value.

[0056] The preset value can also be changed depending on the number of pixels and bit depth of the block to be decoded.

[0057] If the BV list is full, add a BV with a SAD smaller than the predetermined SAD and delete the BV with the largest SAD.

[0058] When registering BVs, it is desirable to sort the BV list in ascending order of SAD.

[0059] Furthermore, if a BV list is stored for each template type, a predetermined SAD can be set for each BV list. Alternatively, the minimum value may be selected from the maximum SADs of multiple BV lists and set as a common predetermined SAD.

[0060] In addition, if the second intra-frame prediction unit 205 holds a BV list for each type of template, once BV search at all levels is completed, the second intra-frame prediction unit 205 selects as many BVs as will fit in the BV list in ascending order of SAD, merges the BV lists into one, and then determines the index of the BV list.

[0061] Alternatively, the second intra-frame prediction unit 205 may integrate the BV lists into one for each layer. In this case, if the BVs overlap, the second intra-frame prediction unit 205 keeps the BV with the smallest SAD and deletes the BV with the large SAD from the list.

[0062] If a BV list is maintained for each type of template and the BV list is filled, the second intra-frame prediction unit 205 may select the smallest SAD from each BV list among the number of BVs that can fit in the BV list, and set it as a specified SAD in the next layer, as shown in Figure 6.

[0063] In addition, the second intra-frame prediction unit 205 can interrupt the SAD calculation process when the SAD reaches or exceeds a specified SAD in the BV list, even if the SAD calculation is in progress, thereby allowing unnecessary searches to be terminated early and speeding up the search process.

[0064] Furthermore, when a BV is added when the BV list is fully filled, the second intra-frame predictor 205 preferably updates the predetermined SAD to the maximum SAD of the BV list.

[0065] However, when the BV search is performed across layers, the second intra-frame prediction unit 205 registers a BV having a SAD smaller than the maximum SAD+M (M>0) in the BV list of the layer in which the BV was previously searched in the BV list of the layer in which the BV is currently being searched.

[0066] The second intra-frame prediction unit 205 starts the search for the second layer by searching in detail the neighborhood of a BV candidate in the first layer BV list when the second layer BV list is empty. For example, when M=1, the second intra-frame prediction unit 205 registers in the second layer BV list only BVs with SADs smaller than the maximum SAD+1 in the first layer BV list in the BV search for the second layer.

[0067] That is, even if the SAD in the second layer and the maximum SAD in the BV list in the first layer are all the same value, the problem of not registering any BVs in the BV list can be avoided.

[0068] Furthermore, if the maximum SAD+1 in the BV list of the first layer is exceeded, the search for the coordinates beyond that point is discontinued, thereby achieving the effect of shortening the processing time.

[0069] The BV list can also be used in IntraTMP weighted average prediction (IntraTMP Fusion) of multiple different BVs, IntraTMP (Fractional IntraTMP) which allows BVs with fractional pixel accuracy in addition to integer pixel accuracy, and IntraTMP (Multi-candidate IntraTMP) which selects one BV from multiple BV candidates or any number of BVs up to the maximum of the BV list.

[0070] In another embodiment, the second intra-frame prediction unit 205 may register any BV candidate in the first-layer BV list in the second-layer BV list so that the second-layer list does not become empty.

[0071] In such a case, it is desirable to register the BV candidate with the smallest SAD in the first-level BV list in the second-level BV list.

[0072] If there are multiple BVs in the BV list with the smallest SAD, the second intra-frame predictor 205 selects one based on a criterion other than SAD.

[0073] For example, the length of the BV or the distance from the x-axis or y-axis can be used as such a criterion, and it is desirable to select a BV that is small in both cases.

[0074] Furthermore, if the blocks in multiple BV candidates are completely identical (even if the BVs are different, there is substantial overlap in the BV list), the index will be wasted. Therefore, if the difference between the blocks referenced by two or more BVs registered in the BV list is smaller than a predetermined threshold, the second intra-frame prediction unit 205 excludes all but one of the two or more BVs from the BV list.

[0075] Hereinafter, the control information decoded by the decoding unit 201 when performing the second intra-frame prediction will be described with reference to FIGS.

[0076] The code input to the decoding unit 201 can include a sequence parameter set (SPS) that summarizes control information for each sequence.

[0077] The code may also include a picture parameter set (PPS) or a picture header (PH) that summarizes control information for each picture, and may also include a slice header (SH) that summarizes control information for each slice.

[0078] A method for setting a correction method for each sequence will be described with reference to FIG.

[0079] As shown in FIG. 3, in step S101, the decoding unit 201 determines whether sps_itmp_enabled_flag is 1 in the sequence parameter set.

[0080] sps_itmp_enabled_flag is a syntax that controls the presence or absence of IntraTMP. When sps_itmp_enabled_flag is 1, it indicates that IntraTMP is enabled, and when sps_itmp_enabled_flag is 0, it indicates that IntraTMP is disabled.

[0081] If sps_itmp_enabled_flag is 1, the operation proceeds to step S102, and if sps_itmp_enabled_flag is 0, the operation ends.

[0082] In step S102, the decoding unit 201 decodes sps_itmp_mode.

[0083] sps_itmp_mode is a syntax that controls the IntraTMP method.

[0084] By using sps_itmp_mode, the IntraTMP method can be changed according to the image characteristics for each sequence, which is expected to maximize the coding efficiency.

[0085] For example, for sequences composed of CG, pixel distribution is often composed of the same value, so the search for IntraTMP BV can be set to be dense, and for sequences composed of natural images, pixel distribution is diverse, so the search for IntraTMP BV can be set to be sparse, thereby maximizing coding efficiency.

[0086] When the correction method is set on a picture-by-picture basis, the decoding unit 201 similarly decodes pps_itmp_enabled_flag and pps_itmp_mode in the picture parameter set or picture header.

[0087] By using pps_itmp_mode, the correction method can be changed according to the image characteristics on a picture-by-picture basis, which is expected to maximize the coding efficiency.

[0088] For example, for pictures made up of CG, pixel distribution is often made up of the same values, so the search for the BV of the IntraTMP can be set to be dense, and for pictures made up of natural images, pixel distribution is diverse, so the search for the BV of the IntraTMP can be set to be sparse, thereby maximizing coding efficiency.

[0089] When the correction method is set in units of slices, the decoding unit 201 similarly decodes sh_itmp_enabled_flag and sh_itmp_mode in the slice header.

[0090] By using sh_itmp_mode, the correction method can be changed in accordance with the image characteristics for each slice, which is expected to maximize the coding efficiency.

[0091] For example, for slice areas made up of CG, pixel distribution is often made up of the same values, so the search for the BV of the IntraTMP can be set to be dense, while for slice areas made up of natural images, pixel distribution is diverse, so the search for the BV of the IntraTMP can be set to be sparse, thereby maximizing coding efficiency.

[0092] By setting only the upper layer, it is possible to suppress an increase in the amount of coding, and by setting the lower layer as well and then prioritizing the setting in the lower layer, adaptive control is possible.

[0093] Alternatively, if the above-mentioned correction method is set in advance, the decoding of the correction method itself can be omitted.

[0094] In the above example, the IntraTMP method is set in units of sequences, pictures, or slices, but the method may be directly selected in units of blocks (described later) without setting these. In this case, the increase in header information described above can be avoided.

[0095] A method for controlling the search for BVs of an IntraTMP in units of blocks will be described below with reference to FIG.

[0096] As shown in FIG. 4, in step S201, the decoding unit 201 determines whether any one of sps_itmp_enabled_flag, pps_itmp_enabled_flag, and sh_itmp_enabled_flag is 1 or not.

[0097] If neither is 1, the operation ends, and if either is 1, the operation proceeds to step S202.

[0098] In step S202, the decoding unit 201 decodes cu_itmp_mode, which is an IntraTMP control signal.

[0099] In step S203, the decoding unit 201 decodes cu_itmp_index, which is a control signal representing an index.

[0100] According to this embodiment, by adaptively setting a BV list in decoding IntraTMP BVs, decoding is performed using a relatively small amount of code, thereby improving coding efficiency.

[0101] The above-described image decoding device 200 may be realized as a program that causes a computer to execute each function (each step).

[0102] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation."

[0103] 200... Image decoding device 201... Decoding unit 202... Inverse quantization unit 203... Inverse transformation unit 204... First intra-frame prediction unit 205... Second intra-frame prediction unit 206... Inter-frame prediction unit 207... Adder 208... Storage unit 210... Code input unit 220... Image output unit

Claims

1. An image decoding device comprising: a decoding unit that decodes control information and quantized values; an inverse quantization unit that inversely quantizes the quantized values ​​to obtain transform coefficients; an inverse transform unit that inversely transforms the transform coefficients to obtain prediction residuals; a first intra-frame prediction unit that generates a first predicted pixel based on decoded pixels and the control information; a second intra-frame prediction unit that builds a reference list based on the decoded pixels and the control information and generates a second predicted image using the reference list; a storage unit that accumulates the decoded pixels; an inter-frame prediction unit that generates a third predicted pixel based on the accumulated decoded pixels and the control information; and an adder that adds the prediction residual and the first to third predicted pixels to obtain the decoded pixel, wherein when performing intra template matching prediction, the second intra-frame prediction unit registers a block vector having a cost smaller than the maximum cost in the reference list of the first layer plus M (M>0) in a search for a block vector of the second layer.

2. An image decoding device comprising: a decoding unit that decodes control information and quantized values; an inverse quantization unit that inversely quantizes the quantized values ​​to obtain transform coefficients; an inverse transform unit that inversely transforms the transform coefficients to obtain prediction residuals; a first intra-frame prediction unit that generates a first predicted pixel based on decoded pixels and the control information; a second intra-frame prediction unit that builds a reference list based on the decoded pixels and the control information and generates a second predicted image using the reference list; a storage unit that accumulates the decoded pixels; an inter-frame prediction unit that generates a third predicted pixel based on the accumulated decoded pixels and the control information; and an adder that adds the prediction residual and the first to third predicted pixels to obtain the decoded pixel, wherein when performing intra template matching prediction, if no block vector having a cost smaller than the maximum cost in the reference list of the first tier is present in a search for block vectors in the second tier, the second intra-frame prediction unit registers an arbitrary block vector in the reference list of the first tier in the reference list of the second tier.

3. The image decoding device according to claim 1 or 2, wherein the cost is a sum of absolute differences.

4. The image decoding device according to claim 1, wherein M is 1.

5. The image decoding device described in claim 1, characterized in that the control information is information that sets the coarseness and fineness of block vector search, and the second intra-frame prediction unit changes the coarseness and fineness of the block vector search based on the control information.

6. The image decoding device according to claim 1, characterized in that the second intra-frame prediction section changes the coarseness or fineness of block vector search depending on the block vector search direction.

7. The image decoding device according to claim 6, wherein the second intra-frame prediction section searches for block vectors densely in the horizontal direction or the vertical upward direction.

8. The image decoding device according to claim 6, wherein the second intra-frame predictor searches for block vectors sparsely in a diagonal direction.

9. The image decoding device according to claim 1, characterized in that the second intra-frame prediction section changes the coarseness or fineness of block vector search depending on the block size.

10. The image decoding device according to claim 9, characterized in that the second intra-frame prediction section searches for block vectors more densely as the block size becomes smaller.

11. The image decoding device according to claim 9, characterized in that the second intra-frame prediction section searches for block vectors more densely as the block size becomes larger.

12. The image decoding device described in claim 1, characterized in that the second intra-frame prediction unit excludes all but one of the two or more block vectors from the reference list when the difference between the blocks referenced by two or more block vectors is smaller than a predetermined threshold.

13. An image decoding method comprising: a step A of decoding control information and quantization values; a step B of inversely quantizing the quantization values ​​to obtain transform coefficients; a step C of inversely transforming the transform coefficients to obtain prediction residuals; a step D of generating a first predicted pixel based on decoded pixels and the control information; a step E of constructing a reference list based on the decoded pixels and the control information and generating a second predicted image using the reference list; a step F of accumulating the decoded pixels; a step G of generating a third predicted pixel based on the accumulated decoded pixels and the control information; and a step H of adding the prediction residual and the first to third predicted pixels to obtain the decoded pixel, wherein in step E, when intra template matching prediction is performed, a block vector having a cost smaller than the maximum cost in the reference list of the first layer plus M (M>0) is registered in the search for a block vector in the second layer.

14. A program causing a computer to function as an image decoding device, the image decoding device comprising: a decoding unit that decodes control information and quantized values; an inverse quantization unit that inversely quantizes the quantized values ​​to obtain transform coefficients; an inverse transform unit that inversely transforms the transform coefficients to obtain predicted residuals; a first intra-frame prediction unit that generates a first predicted pixel based on decoded pixels and the control information; a second intra-frame prediction unit that builds a reference list based on the decoded pixels and the control information and generates a second predicted image using the reference list; a storage unit that accumulates the decoded pixels; an inter-frame prediction unit that generates a third predicted pixel based on the accumulated decoded pixels and the control information; and an adder that adds the predicted residuals and the first to third predicted pixels to obtain the decoded pixels, wherein when performing intra template matching prediction, the second intra-frame prediction unit registers a block vector having a cost smaller than the maximum cost in the reference list of the first layer plus M (M>0) in a search for a block vector of a second layer.

Images