Encoding device, encoding method, decoding device, decoding method, program

Abstract

The present invention aims to provide an encoding device, a decoding device, an encoding method, a decoding method, and a program that can efficiently specify information used in predictive processing. [Solution] The encoding device 100 includes a prediction unit that identifies a filter from among a plurality of filters for a predetermined intra-prediction process for a target block and generates a predicted image by a predetermined intra-prediction process using the selected filter, and an encoding unit that encodes a syntax element for identifying the filter used from among the plurality of filters. The plurality of filters include at least a first filter and a second filter, and the sizes of the first filter and the second filter are different.

Description

[Technical Field]

[0001] This invention relates to image encoding and decoding techniques. [Background technology]

[0002] High Efficiency Video Coding (HEVC) is a well-known encoding method for compressed video recording. Versatile Video Coding (VVC), a successor to HEVC, is also known. HEVC employs larger basic blocks than conventional macroblocks (16x16 pixels) to improve encoding efficiency. These larger basic blocks are called Coding Tree Units (CTUs), and their size is up to 64x64 pixels. In VVC, CTUs can take up to 128x128 pixels. CTUs are further divided into subblocks, which serve as units for prediction and transformation.

[0003] In recent years, efforts have been made to internationally standardize a more efficient encoding scheme as a successor to VVC, and standardization is being advanced by JVET (Joint Video Experts Team). Non-patent document 1 is the algorithm description of the reference model (hereinafter referred to as ECM) of the new encoding scheme that has been published to date.

[0004] Furthermore, Non-Patent Document 1 describes Extrapolation filter-based intra prediction (EIP) as a new intra prediction process. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Algorithm description of Enhanced Compression Model 14(ECM 14),(JVET-AI2025)(version1-date 2024-10-21) [Overview of the project] [Problems that the invention aims to solve]

[0006] Therefore, the present invention aims to provide a technology that enables efficient specification of information used in predictive processing. [Means for solving the problem]

[0007] In view of the above problems, the encoding device according to the present invention includes prediction means for a target block, which identifies a filter from a plurality of filters for a predetermined intra-prediction process and generates a predicted image by the predetermined intra-prediction process using the selected filter, and encoding means for encoding a syntax element for identifying the filter used from the plurality of filters, wherein the plurality of filters include at least a first filter and a second filter, and the sizes of the first filter and the second filter are different.

[0008] Furthermore, the decoding device according to the present invention includes decoding means for decoding syntax elements for identifying a filter from a plurality of filters for a predetermined intra-prediction process for a target block, and prediction means for generating a predicted image by the predetermined intra-prediction process using the filter identified according to the syntax elements, wherein the plurality of filters include at least a first filter and a second filter, and the sizes of the first filter and the second filter are different. [Effects of the Invention]

[0009] According to the present invention, it is possible to reduce the amount of information used in predictive processing. [Brief explanation of the drawing]

[0010] [Figure 1] Block diagram showing an example of the functional configuration of the symbolization device. [Figure 2] Block diagram showing an example of the functional configuration of the decoding device. [Figure 3] Diagram showing an example of a filter. [Figure 4] Diagram showing an example of a reconstruction area. [Figure 5] Diagram for explaining the EIP process. [Figure 6] Diagram showing an example of the filter scan. [Figure 7] Diagram for explaining the generation of the EIP merge list. [Figure 8] Diagram showing an example of block division. [Figure 9] Diagram showing an example of a bit stream. [Figure 10] Diagram showing an example of the coding_unit syntax structure. [Figure 11] Flowchart of the encoding process. [[ID=三十四]] [Figure 12] Diagram showing an example of a filter. [Figure 13] Diagram showing an example of the coding_unit syntax structure. [Figure 14] Flowchart of the decoding process. [Figure 15] Diagram showing an example of the hardware configuration of a computer device.

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments will be described with reference to the accompanying drawings. The embodiments described below are examples, and are one of the specific embodiments of the configuration described in the claims. Note that the image to be encoded or decoded in each embodiment may be a still image or a moving image. Further, although the EIP will be described as an example, each embodiment is applicable to prediction processing other than the EIP. In particular, it is applicable to prediction processing that requires specific information, such as information indicating a filter in the EIP.

[0012] (First Embodiment) First, an example of the functional configuration of the encoding device (image encoding device) 100 according to this embodiment will be explained using the block diagram in Figure 1. The control unit 101 controls the overall operation of the encoding device 100. The block division unit 102 acquires an input image (images of each frame constituting a moving image or still images), divides the input image into a plurality of basic blocks, and outputs each of the divided basic blocks. The basic block is a CTU (Coding Tree Unit), and its size is, for example, 32 pixels × 32 pixels, 64 pixels × 64 pixels, or 128 pixels × 128 pixels.

[0013] The prediction unit 103 divides each basic block into multiple subblocks. A subblock is, for example, a coding unit.

[0014] The prediction unit 103 may also divide the subblocks obtained by dividing the basic block to obtain even smaller subblocks. These further divided smaller subblocks may be, for example, transformation blocks or prediction blocks. Alternatively, they may simply be subblocks of the coding unit.

[0015] The prediction unit 103 generates a predicted image by performing prediction processing such as intra-frame prediction and inter-frame prediction for each sub-block, and calculates the difference between the input image and the predicted image as the prediction error. The prediction unit 103 also generates prediction information, which includes information indicating how the basic block was divided and information necessary for prediction processing, such as the prediction processing mode.

[0016] The transformation / quantization unit 104 generates transformation coefficients for each subblock by performing a transformation process on the prediction error for each subblock. It is possible to perform orthogonal transformations using transformation bases such as DCT2, DCT8, and DST7 as the transformation process. Here, the orthogonal transformation may be performed using the same transformation base in both the horizontal and vertical directions of the subblock, or it may be performed using different transformation bases in the horizontal and vertical directions of the subblock. The transformation / quantization unit 104 then generates quantization coefficients for the subblock by quantizing the transformation coefficients of the subblock. The transformation / quantization unit 104 may also quantize the transformation coefficients using a quantization matrix.

[0017] The inverse quantization / inverse transformation unit 105 inversely quantizes the quantization coefficients of each subblock generated by the transformation / quantization unit 104 to generate transformation coefficients, and then performs an inverse transformation process on these transformation coefficients to generate a prediction error.

[0018] The image reconstruction unit 106 generates a predicted image from the encoded and reconstructed image stored in the frame memory 107, based on the prediction information generated by the prediction unit 103. Furthermore, it reconstructs the image from the predicted image and the prediction error generated by the inverse quantization / inverse transform unit 105. The image reconstruction unit 106 then stores the reconstructed image in the frame memory 107. The reconstructed image stored in the frame memory 107 becomes the image data that the prediction unit 103 references when performing prediction processing on the image of the current frame or the next frame.

[0019] The in-loop filter unit 108 performs in-loop filtering, such as deblocking filtering and sample adaptive offsetting, on the images stored in the frame memory 107.

[0020] The encoding unit 109 generates encoded data by entropy encoding the quantization coefficients generated by the transformation / quantization unit 104 and the prediction information generated by the prediction unit. The entropy encoding method is not limited to a specific method; Golomb coding, arithmetic coding, Huffman coding, etc., can be used.

[0021] Here, the prediction processing performed by the prediction unit 103 in this embodiment will be described in more detail. The prediction unit 103 in this embodiment is capable of performing intra-prediction and inter-prediction in the HEVC standard and VVC standard. Furthermore, it is capable of performing a new intra-prediction process (a predetermined intra-prediction process) called EIP (Extrapolation filter-based intra-prediction). Here, with reference to Figure 3, the EIP performed by the prediction unit will be described.

[0022] Figure 3 shows an example of a filter used in EIP. The prediction unit 103 selects one of the filters shown in Figure 3 for the subblock currently being processed (hereinafter referred to as the target block). Figure 3(a) is a 4-pixel × 4-pixel filter, Figure 3(b) is an 8-pixel × 2-pixel filter, and Figure 3(c) is a 2-pixel × 8-pixel filter. The prediction unit 103 selects one of these filters. Furthermore, Figure 4 shows the type of reconstruction region. The method for determining each filter coefficient that constitutes the selected filter is not particularly limited. For example, predetermined initial values ​​may be used, or they may be set individually or generated according to the characteristics of the image. Alternatively, one of the reconstruction regions shown in Figure 4 may be selected for the target block, and the filter coefficients of the selected filter may be generated using the selected reconstruction region. Alternatively, the correlation between the pixel value at the position corresponding to the predicted pixel in the reconstruction region and the other pixel values ​​may be derived using convolution integral, and the filter coefficients may be determined based on that correlation. In this embodiment, as an example, the prediction unit 103 selects one of the reconstruction regions shown in Figure 4 for the target block, and generates the filter coefficients of the selected filter using the selected filter and the selected reconstruction region. Furthermore, aboveSize shown in Figure 4 is a size determined depending on the height of the target block, and similarly, leftSize is a size determined depending on, for example, the width of the target block. For example, aboveSize is a size equal to the height of the target block, or a size obtained by adding a predetermined value to the height of the target block. Similarly, leftSize is a size equal to the width of the target block, or a size obtained by adding a predetermined value to the width of the target block.

[0023] Equation (1) is an example of a calculation formula for generating predicted pixels using EIP.

[0024] Nod (x,y) This is the predicted pixel value at the (x,y) position in the current target block. i is the filter coefficient for position i. iis the reconstructed pixel value or predicted pixel value at position i, and bitdepth is the bit depth of the image to be encoded. Although the correspondence between each pixel position and the value of i is shown in Figure 3 for each filter, the correspondence between each pixel position and the value of i in each filter is not limited to the examples shown in Figure 3 and may be other correspondences.

[0025]

number

[0026] In the example shown in equation (1), for instance, the reconstructed pixel value or predicted pixel value corresponding to i=14, which is the position furthest from the predicted pixel position (bottom right corner), is not used. In this way, one or more pixels in the region containing the pixel furthest from the predicted pixel position may not be used for prediction.

[0027] Furthermore, as shown in equation (2) below, the reconstructed pixel value or predicted pixel value corresponding to i=14, which is the position furthest from the predicted pixel position (bottom right corner), may also be used.

[0028]

number

[0029] The encoding device may decide which formula to use for the predicted image based on the image characteristics and signal it to the bitstream using an index or similar method.

[0030] Here, an example of prediction processing using EIP when the size of the target block is 4 pixels × 4 pixels and a filter with a size of 4 pixels × 4 pixels shown in FIG. 3(a) is selected will be described. As shown in FIG. 5(a), there is an encoded and reconstructed image region 502 around the target block 501. The prediction unit 103 arranges the filter 503 with a size of 4 pixels × 4 pixels so that the position (x, y) of the filter 503 with a size of 4 pixels × 4 pixels becomes the upper left (0, 0) position of the 4 pixels × 4 pixels target block 501. Then, according to Equation (1) or (2), the predicted pixel value pred (x,y) at the position (x,y) is derived. In this case, the encoded pixel values in the image region 502 are used as t i for positions i = 0 to 13. For example, t0 corresponding to the position i = 0 is the value of the encoded pixel adjacent above the pixel at the position (x, y) in FIG. 5(a). After the predicted pixel value pred (x,y) is derived at the position (x, y) shown in FIG. 5(a), the prediction unit 103 shifts the position of the filter 503 by one pixel according to the scanning order indicated by the arrow in FIG. 5(a), and for the position (x, y) shown in FIG. 5(b), the predicted pixel value pred (x,y) is generated according to Equation (1) or (2). At this time, the encoded pixel values are used as t i for positions i = 0, 2 to 13 in the filter 503 of FIG. 5(b), while the predicted pixel value pred (x,y) already derived according to Equation (1) or (2) in the arrangement of the filter 503 in FIG. 5(a) is used as t1 for the position i = 1 of the filter 503 in FIG. 5(b). As described above, the prediction unit 103 sequentially obtains the predicted pixel value pred (x,y) while shifting the selected filter by one pixel according to the scanning order indicated by the arrow in FIG. 5(a), and generates a predicted image of the target block.

[0031] In this scanning method, the first and last dashed arrows indicate a movement of one pixel to the right. Then, at each position indicated by the solid arrow, the image moves diagonally one pixel to the left and down. When the edge of the target block is reached as a result of one or more consecutive diagonal movements indicated by the solid arrows, the image jumps over multiple pixel positions as indicated by the dashed arrows and moves to a position to the right of or directly below the starting position of the previous one or more consecutive diagonal movements. For convenience, this scanning method shown in Figures 5(a) and 5(b) will be referred to as a Down-left diagonal scan.

[0032] Alternatively, a scanning method as shown in Figure 5(c) may be used as the scanning sequence. This is called an up-right diagonal scan, a scanning method used in HEVC for scanning the quantization matrix. In this scanning method, the first and last dashed arrows move one pixel downwards. Then, at each position indicated by the solid arrows, the cursor moves diagonally one pixel to the upper right. As a result of one or more consecutive diagonal movements indicated by the solid arrows, when the cursor reaches the edge of the target block, it jumps over multiple pixel positions and moves to a position directly below or to the right of the starting position of the previous one or more consecutive diagonal movements, as indicated by the dashed arrows.

[0033] As described above, the prediction unit 103 generates a predicted image by applying a filter while moving it. More specifically, the prediction unit 103 generates a predicted image by applying a filter while moving it to an area that includes at least a portion of the target block and the area surrounding the target block.

[0034] The scanning order of the filter in the target block is not limited to the example shown in Figure 5(a), but may also be the scanning order shown in Figures 6(a) and 6(b). Figure 6(a) shows the order in which each row of pixels (hereinafter referred to as "pixel row") of an 8x8 target block is scanned horizontally. In this case, after the scan for predictive pixel generation is performed on the pixel row at the top of the 8x8 target block, the scan for predictive pixel generation may be performed on the pixel row immediately adjacent to that pixel row. That is, after the scan for predictive pixel generation is performed on a certain pixel row, the scan for predictive pixel generation may be performed on the pixel row immediately adjacent to that pixel row. In addition, predictive pixel generation is performed on the leftmost pixel in the pixel row at the top of the 8x8 target block. Subsequently, in parallel with the predictionive pixel generation on the pixel adjacent to the right of that pixel according to the scanning order shown in Figure 6(a), predictive pixel generation may also be performed on the leftmost pixel of the pixel row adjacent to the pixel row below the top pixel row. In subsequent processing, as shown in Figure 6(a), filter scanning for generating predicted pixels may be performed in parallel, shifting by one pixel for each pixel row of the target block. This makes it possible to perform parallel processing for each pixel row for generating predicted pixels.

[0035] Similarly, Figure 6(b) shows the order in which each pixel column (hereinafter referred to as "pixel column") of an 8x8 target block is scanned vertically. At this time, after the scan for predictive pixel generation is performed on the leftmost pixel column of the 8x8 target block, the scan for predictive pixel generation may be performed on the pixel column adjacent to the right of that pixel column. That is, after the scan for predictive pixel generation is performed on a certain pixel column, the scan for predictive pixel generation may be performed on the pixel column adjacent to the right of that pixel column. In addition, predictive pixel generation is performed on the top pixel of the leftmost pixel column of the 8x8 target block. Subsequently, in parallel with the predictionive pixel generation of the pixels adjacent below that pixel according to the scan order shown in Figure 6(b), predictive pixel generation may be performed on the top pixel of the pixel column adjacent to the right of the leftmost pixel row. At this time, in the subsequent processing, as shown in Figure 6(b), filter scans for predictive pixel generation may be performed in parallel, shifting by one pixel for each pixel column of the target block. This approach makes it possible to perform parallel processing for each pixel row for predictive pixel generation.

[0036] As mentioned above, the filter scan in the target block may also be an up-right diagonal scan as shown in Figure 5(c). This scan order is the same as the scan order used for encoding the quantization matrix and quantization coefficients in HEVC and VVC. By standardizing the scan order with the quantization matrix and quantization coefficients, it is possible to reduce the memory used for the scan table.

[0037] Furthermore, in the explanation above, the filter coefficient c used in the EIP prediction process i The explanation described obtaining this using the reconstruction region surrounding the target block, but it is not limited to this. For example, in a particular mode (hereinafter referred to as EIP merge mode), the filter coefficient c used in other blocks other than the target block where the EIP prediction process was performed. iFilter information such as these may be used in the prediction process of the EIP of the target block. In this mode, by using information used in other blocks to determine the information to be used in the prediction process of the target block, that information can be encoded efficiently.

[0038] The EIP merge mode will be explained below with reference to Figure 7. When EIP merging is used, the prediction unit 103 generates an EIP merge list (mergeEIPCandList) for the target block 701 in the image currently being processed (current image). Specifically, as shown in Figure 7(a), the EIP merge list is generated for the target block 701 from spatially adjacent or nearby blocks (blocks containing positions A0 to A1 / blocks containing positions B0 to B2). Alternatively, the EIP merge list (mergeEIPCandList) is generated for the target block 701 from positions in a temporally different image that are spatially corresponding to the target block. Or, the EIP merge list (mergeEIPCandList) is generated for the target block 701 from adjacent / neighboring blocks (blocks containing position C0 / blocks containing position C1) from a spatially corresponding position. Note that block 702 is a block included in a temporally different reference image from the current image containing the target block 701, and is a block at the same position as the target block 701. In other words, block 702 is a block located collocated with the target block 701. As described above, the list described later may include information about blocks surrounding the target block (including adjacent blocks) as candidate information to be used in the target block. Furthermore, the list described later may include information about blocks (including adjacent blocks) that are spatially corresponding to or surrounding the target block in images at different times as candidate information.

[0039] To generate an EIP merge list (mergeEIPCandList), the prediction unit 103 determines whether a subblock containing the location of B1 is available for the target block 701. For example, if the subblock containing the location of B1 is currently included in the image, the prediction unit 103 sets availableFlagB1 to 1 (true), indicating that the subblock containing the location of B1 is available. On the other hand, if the location of B1 is not currently included in the image, the prediction unit 103 sets availableFlagB1 to 0 (false), indicating that the subblock containing the location of B1 is unavailable. Furthermore, the prediction unit 103 determines whether EIP is being used for the subblock containing the location of B1, setting EIPFlag[B1] to 1 (true) if it is being used, and setting EIPFlag[B1] to 0 (false) if it is not being used. Then, as shown in Figure 7(b), if the conditions are met that availableFlagB1 is 1 and EIPFlag[B1] is 1, the prediction unit 103 stores the information of candidate B1 in the EIP merge list (mergeEIPCandList). The information of candidate B1 includes information that identifies the filter used in the EIP processing in the subblock containing the position of B1, and the coefficient c of that filter. i The information is linked.

[0040] The information stored in the list may be, for example, information that identifies a block called B1 (such as an index or the coordinates corresponding to B1).

[0041] Furthermore, the information stored in the list may also be information that identifies the filter used in the EIP processing of that subblock (in this case, B1). For example, the information stored in the list may be an index for identifying the filter.

[0042] Additionally, the information stored in the list is the filter coefficient c of the filter used in the EIP processing of that subblock (B1 in this case). i This could also be information that identifies the filter coefficient c. For example, the specific filter coefficient c. i It may be a value representing the filter coefficient c. iIt may also be an index that specifies a set of values.

[0043] The information stored in the list includes information identifying the filter used in the EIP processing of that subblock (B1 in this case), and the filter coefficient c of that filter. i This could be either information identifying the subject or information identifying the subject. The same applies to the other candidate information described below.

[0044] Similarly, the prediction unit 103 identifies whether the subblock containing the location of A1 is available and sets the result of this identification as availableFlagA1. Furthermore, the prediction unit 103 identifies whether EIP is used for the subblock containing the location of A1 and sets the result of this identification as EIPFlag[A1]. If the conditions that availableFlagA1 is 1 and EIPFlag[A1] is 1 are met, the prediction unit 103 stores the information of candidate A1 (hereinafter referred to simply as candidate A1 as appropriate) in the EIP merge list (mergeEIPCandList). Candidate A1 contains information identifying the filter used in the EIP processing in the subblock containing the location of A1 and the coefficient c of that filter. i The information is linked.

[0045] Next, the prediction unit 103 determines whether the subblock containing the location of B0 is available. Furthermore, it sets the determined result as availableFlagB0. Furthermore, it determines whether EIP is used for the subblock containing the location of B0 and sets the determined result as EIPFlag[B0]. If the conditions availableFlagB0 is 1 and EIPFlag[B0] is 1 are met, the prediction unit 103 stores the information of candidate B0 (hereinafter, as may be referred to simply as candidate B0) in the EIP merge list (mergeEIPCandList). Candidate B0 contains information that identifies the filter used in the EIP processing in the subblock containing the location of B0 and the coefficient c of that filter. i The information is linked.

[0046] Next, the prediction unit 103 determines whether the subblock at the location of A0 is available. Furthermore, it sets the determined result as availableFlagA0. Then, it determines whether EIP is used for the subblock at the location of A0 and sets the determined result as EIPFlag[A0]. If the conditions that availableFlagA0 is 1 and EIPFlag[A0] is 1 are met, the prediction unit 103 stores the information of candidate A0 (hereinafter referred to simply as candidate A0 as appropriate) in the EIP merge list (mergeEIPCandList). Candidate A0 contains information that identifies the filter used in the EIP processing in the subblock containing the location of A0 and the coefficient c of that filter. i The information is linked.

[0047] Next, the prediction unit 103 determines whether the subblock containing the location of B2 is available. Furthermore, it sets the determined result as availableFlagB2. Then, it determines whether EIP is used for the subblock at the location of B2 and sets the determined result as EIPFlag[B2]. If the conditions availableFlagB2 is 1 and EIPFlag[B2] is 1 are met, the prediction unit 103 stores the information of candidate B2 (hereinafter, as may be referred to simply as candidate B2) in the EIP merge list (mergeEIPCandList). Candidate B2 contains information that identifies the filter used in the EIP processing in the subblock containing the location of B2 and the coefficient c of that filter. i The information is linked.

[0048] The prediction unit 103 then determines whether the subblock containing the location of C0 (block 702 in the example of Figure 7(a)) is available. Furthermore, it sets the determined result as availableFlagCol. Furthermore, it determines whether EIP is used for the subblock containing the location of C0 and sets the determined result as EIPFlag[Col]. If the conditions that availableFlagCol is 1 and EIPFlag[Col] is 1 are met, it stores the information of candidate Col (hereinafter, as appropriate, simply referred to as candidate Col). Specifically, the prediction unit 103 stores the information of candidate Col (hereinafter, as appropriate, simply referred to as candidate Col) in the EIP merge list (mergeEIPCandList). Candidate Col is associated with the filter used in the EIP processing in the subblock containing the location of C0 and the information of the coefficients of that filter. If the subblock containing the location of C0 is not available, or if EIP is not used in the subblock containing the location of C0, the following processing may be performed. For the subblock containing the location of C1, the system may further determine whether the subblock is available and whether EIP was used in that subblock. In this case, the prediction unit 103 identifies whether the subblock containing the location of C1 is available. Furthermore, it sets the identified result as availableFlagCol. Furthermore, it identifies whether EIP was used for the subblock containing the location of C1 and sets the identified result as EIPFlag[Col]. If the conditions availableFlagCol is 1 and EIPFlag[Col] is 1 are met, the prediction unit 103 stores candidate Col in the EIP merge list (mergeEIPCandList). In this case, candidate Col contains information identifying the filter used in the EIP processing in the subblock containing the location of C1 and the coefficient c of that filter. i The information is linked.

[0049] The prediction unit 103 may determine whether availableFlagCol is available based on the value of other syntax elements. For example, it may use the value of a flag (temporal_mvp_enabled_flag) that indicates whether temporal motion vector prediction is possible in interpretation. This flag is signaled to a predetermined parameter set (e.g., SPS: Sequence Parameter Set). Other examples of predetermined parameter sets include PPS: Picture Parameter Set, PH: Picture Header, SH: Slice Header, etc. The predetermined parameter set is a syntax element, and in this embodiment, it is signaled to SPS. Temporal motion vector prediction refers to the following process: An interpretation process that generates a predicted image of a target block using the motion vector used in a predetermined interpretation for the target block. The motion vector used is the motion vector used in interpretation for a block located collated with the target block (e.g., block 702 relative to target block 701). If the prediction unit 103 indicates that temporal motion vector prediction is available in interpretation (temporal_mvp_enabled_flag=1), it sets availableFlagCol, shown in Figure 7(b), to 1. On the other hand, if the prediction unit 103 indicates that temporal motion vector prediction is unavailable in interpretation (temporal_mvp_enabled_flag=0), it sets availableFlagCol, shown in Figure 7(b), to 0. In this case, in EIP merge mode, blocks located in a collocated position relative to the target block are unavailable as candidates for the EIP merge list.

[0050] In the above explanation, as shown in Figure 7(b), the prediction unit 103 generates an EIP merge list for the target block 701 in the order B1→A1→B0→A0→B2→Col, but it is not limited to this. For example, the prediction unit 103 may generate an EIP merge list for the target block 701 in the order Col→B1→A1→B0→A0→B2.

[0051] Furthermore, after generating an EIP merge list as shown in Figures 7(a) and 7(b), the prediction unit 103 may add additional candidates according to the current number of candidates and the maximum number of candidates in the EIP merge list. Specifically, the prediction unit 103 compares the current number of candidates in the EIP merge list based on information from other blocks with the maximum number of candidates. Then, according to the comparison result, the prediction unit 103 adds candidates other than those already included in the list. For example, if the current number of candidates in the EIP merge list based on information from other blocks is less than the maximum number of candidates, new candidates (hereinafter referred to as additional candidates) are added to the EIP merge list until the number of candidates in the EIP merge list reaches the maximum number of candidates. For example, if the number of candidates in the generated EIP merge list is 5 and the maximum number of candidates in the EIP merge list is 7, the prediction unit 103 adds two more candidates. The additional candidates added here are, for example, candidates associated with a predetermined filter (for example, the filter in Figure 3(a)) and a predetermined set of filter coefficients Ci for that predetermined filter. The predetermined filter coefficient Ci may be a fixed value where all values ​​are the same, or it may be a predetermined coefficient generated in advance by machine learning. Furthermore, if there are two or more candidates with the same filter among the candidates already stored in the EIP merge list, a new filter coefficient Ci is obtained by averaging the filter coefficients Ci of those two or more candidates.

[0052]

number

[0053] The set of filters is generated, and additional candidates associated with the set may be added to the EIP merge list. Specifically, if the same filter (for example, the filter in Figure 3(a)) is associated with the first candidate n and the second candidate m already stored in the EIP merge list, the prediction unit 103 uses equation (3) to determine the first candidate n

[0054]

number

[0055] and second candidate m

[0056]

number

[0057] and average to create a new

[0058]

number

[0059] Generates.

[0060]

number

[0061] The maximum number of candidates in the EIP merge list, MaxNumEIPCand, is signaled to the bitstream by the syntax element num_eip_merge_cand_minus2. num_eip_merge_cand_minus2 represents the value obtained by subtracting 2 from the maximum number of candidates in the EIP merge list, MaxNumEIPCand, and is signaled to a predetermined set of parameters (SPS, PPS, PH, SH, etc.). Although num_eip_merge_cand_minus2 has been described as a syntax element that represents the value obtained by subtracting 2 from the maximum number of candidates, it is not limited to this and may represent the value obtained by subtracting other integer values ​​(1, 3, 4, 5, etc.) from the maximum number of candidates.

[0062] As described above, the maximum number of candidates in this explanation functions as a mandatory target number. Even if the number of candidates based on information from other blocks falls short of the target number, generating additional candidates to create a list that always contains the same number (target number) of candidates makes the data easier to handle. Furthermore, always including the maximum number of candidates in the list increases the likelihood of selecting more appropriate information.

[0063] The prediction unit 103 generates an EIP merge list and then encodes the syntax element eip_merge_idx, which identifies the candidate to be used for the EIP of the target block from among the candidates in the generated EIP merge list, into a bitstream. eip_merge_idx is binarized by a predetermined binarization process and encoded using bypass coding or arithmetic coding. In this case, the first N bins (N is a positive integer) of the binarized sequence of eip_merge_idx may be encoded using arithmetic coding, and the rest may be encoded using bypass coding. For example, the first bin may be encoded using arithmetic coding, and the rest may be encoded using bypass coding. When arithmetic coding is used for one or more bins of the binarized sequence of eip_merge_idx, the context variables used for arithmetic coding depend on the following: The context variables depend, for example, on whether the subblock adjacent to the target block and the subblock adjacent to the left of the target block are using EIP.

[0064] Next, the encoding of the input image will be explained with reference to Figure 8. The block division unit 102 divides the input image into multiple basic blocks (CTUs) and outputs each of the divided basic blocks. Furthermore, the block division unit 102 divides the basic blocks into multiple subblocks, and further divides the subblocks divided from the basic blocks into multiple subblocks. Figure 8 is a diagram showing the block division method by the block division unit 102.

[0065] Figure 8(a) shows the case where no further division is performed on a certain block 800 (basic block or subblock). Here, block 800 is described as having a size of 16 pixels × 16 pixels. Figure 8(b) shows a quadtree division, which is one method of subblock division. Figure 8(b) shows a division method (division pattern) in which block 800 is divided into four subblocks by dividing it in half horizontally and vertically. In this case, the size of each of the four subblocks is 8 pixels × 8 pixels. Figure 8(c) shows a division method in which block 800 is divided into two vertically elongated subblocks (each subblock having a size of 8 × 16 pixels) by dividing it in half vertically. Figure 8(d) shows a division method in which block 800 is divided into two horizontally elongated subblocks (each subblock having a size of 16 × 8 pixels) by dividing the basic block 700 in half horizontally. Figure 8(e) shows three horizontally elongated subblocks obtained by vertically dividing block 800 (from left to right: a subblock of 4x16 pixels, a subblock of 8x16 pixels, and a subblock of 4x16 pixels) (ternary tree partitioning).

[0066] Figure 8(f) shows three vertically elongated subblocks obtained by horizontally dividing block 800 (a subblock of 16x4 pixels, a subblock of 16x8 pixels, and a subblock of 16x4 pixels from top to bottom) (ternary tree partitioning). In this embodiment, any of the partitioning methods (partitioning patterns) shown in Figures 8(b) to (f) will be used, but it is not limited to Figures 8(b) to (f). Note that, as shown in Figure 8(a), block 800 may be used as a subblock without partitioning it.

[0067] Next, the encoding of syntax elements related to EIP will be explained with reference to Figures 9-10. Figure 9 is an example of the structure of a bitstream generated by the encoding unit 109 of the encoding device 100 in this embodiment. As shown in Figure 9, the bitstream includes an SPS (Sequence Parameter Set) containing parameters applied to the entire sequence. It also includes a PPS (Picture Parameter Set) containing parameters applied to one or more images, and a PH (Picture Header) containing parameters applied to a single image. Furthermore, it includes Picture data containing the encoded data of the image. In addition, it may also include an SH (Slice Header) containing parameters applied to a single slice.

[0068] Figure 10 shows the syntax structure included in the picture data of the bitstream generated by the coding unit 109, and is information that is signaled on a sub-block (coding unit) basis. For example, the coding unit 109 encodes the syntax elements shown in Figure 10 for the sub-block (target block) currently being processed.

[0069] The eip_flag shown in Figure 10 is a flag that indicates whether or not to execute an EIP on the target block. An eip_flag with a value of 1 indicates that an EIP will be executed on the target block, and an eip_flag with a value of 0 indicates that an EIP will not be executed on the target block. Note that eip_flag is encoded into the coding_unit() syntax structure in Figure 10 when a predetermined condition, condition1, is true. condition1 may be a condition such as whether a syntax element indicating that an EIP is available is true or false, as shown below.

[0070] condition1:eip_enabled_flag eip_enabled_flag is a syntax element that indicates that EIP is available and signals to a higher-level syntax structure in the bitstream than the coding_unit() syntax structure. For example, eip_enabled_flag is a flag that signals to SPS, PPS, PH, SH, etc. In this embodiment, eip_enabled_flag is assumed to be a flag that signals to SPS. eip_enabled_flag with a value of 1 indicates that EIP is available, condition1 becomes true, and eip_flag is encoded. On the other hand, eip_enabled_flag with a value of 0 indicates that EIP is unavailable, condition1 becomes false, eip_flag is not encoded, and the value of eip_flag is always 0. Note that if condition1 is false, the value of eip_flag is estimated to be 0 by the decoding device (image decoding device) 200.

[0071] Another variation of condition1 is that condition1 may be a condition that further depends on the block size. In this case, for example, condition1 would be as follows:

[0072] condition1=eip_enabled_flag&&Max(cbWidth,cbHeight)≦32 In this modified example, condition1 is true if eip_enabled_flag is 1 and the larger of cbWidth or cbHeight is 32 or less. Otherwise, condition1 is false. Here, cbWidth is the width of the luminance component of the target block, and cbHeight is the height of the luminance component of the target block.

[0073] As another variation of condition1, condition1 may also be eip_enabled_flag&&Max(cbWidth,cbHeight)≦16. Alternatively, condition1 may also be eip_enabled_flag&&Max(cbWidth,cbHeight)≦64, etc.

[0074] As another variation of condition1, condition1 may be a condition that depends on the color components of the current target block, as follows:

[0075] Condition 1: cIdx=0 cIdx is a variable that indicates the color component, where cIdx=0 corresponds to the luma component (Y component), cIdx=1 corresponds to the Cb chroma component (Cb component), and cIdx=2 corresponds to the Cr chroma component (Cr component). In the example of condition1 above, if cIdx=0, the condition of condition1 is satisfied, and if not (if it is a chroma component), the condition1 is not satisfied. In other words, if the current target block is a luma component block, the condition of condition1 is satisfied, and if not (if it is a chroma component), the condition1 is not satisfied. That is, in this modified example, eip_flag can only be encoded for the luma component.

[0076] Furthermore, as a variation of condition1, condition1 may be a condition that depends on the current position of the target block. If, for example, the filter shown in Figure 3 does not fit within the picture at the target block position, EIP may be disabled. In other words, if, for example, the filter shown in Figure 3 does not fit within the picture, condition1 may be set to forcibly disable EIP.

[0077] If eip_flag is 1 (true), the encoding unit 109 encodes eip_merge_flag into coding_unit(). eip_merge_flag is a flag indicating whether to use the EIP merge mode for the target block. If eip_merge_flag is 1, the EIP merge mode is used for the target block, while if eip_merge_flag is 0, the EIP merge mode is not used for the target block. If eip_flag is 1 (true), the encoding unit 109 obtains the value of MaxNumEIPCand, which is the maximum number in the EIP merge list, and if this value is greater than 1, it further encodes eip_merge_idx into coding_unit(). The encoding unit 109 also encodes num_eip_merge_cand_minus2, a syntax element for identifying the value of MaxNumEIPCand, into a predetermined parameter set. MaxNumEIPCand is identified by adding 2 to num_eip_merge_cand_minus2. In this embodiment, num_eip_merge_cand_minus2 is encoded in SPS. Note that the maximum number of EIP merge list entries and the maximum number of merge candidates for inter-prediction may be the same. In this case, encoding of num_eip_merge_cand_minus2 may be omitted. Alternatively, the difference between the maximum number of EIP merge list entries and the maximum number of merge candidates for inter-prediction may be encoded. This allows encoding with a shorter code size than num_eip_merge_cand_minus2.

[0078] If eip_merge_flag is 0 (false) and condition2 is true, the encoding unit 109 encodes eip_filter_idx and eip_reconst_idx into the coding_unit() syntax structure. eip_filter_idx is a syntax element that identifies the filter to be used for EIP for the target block from among the multiple filters shown in Figure 3. For example, eip_filter_idx=0 indicates the filter in Figure 3(a), eip_filter_idx=1 indicates the filter in Figure 3(b), and eip_filter_idx=2 indicates the filter in Figure 3(c). eip_reconst_idx is a syntax element for identifying the reconstruction region used to generate filter coefficients in EIP for the target block from among the multiple reconstruction regions shown in Figure 4. For example, eip_reconst_idx=0 indicates the reconstructed region in Figure 4(a), eip_reconst_idx=1 indicates the reconstructed region in Figure 4(b), and eip_reconst_idx=2 indicates the reconstructed region in Figure 4(c). Note that condition2 may also be a condition that depends on the size of the target block, for example, as follows.

[0079] condition2:cbwidth==4&&cbheight==4 Furthermore, condition2 is not limited to this; for example, it could be cbwidth==4||cbheight==4, min(cbwidth,cbheight)≦4, etc. Also, condition2 may be a condition that depends on the position of the target block.

[0080] In this embodiment, if eip_flag is 1 and eip_merge_flag is 0 and condition2 is true, the following encoding process is performed. eip_filter_idx and eip_reconst_idx are encoded into codint_unit(). On the other hand, even if eip_flag is 1 and eip_merge_flag is 0, if condition2 is false, eip_filter_idx and eip_reconst_idx are not encoded. Furthermore, the values ​​of eip_filter_idx and eip_reconst_idx are always 0 (i.e., constrained to the value of 0). In other words, if the above condition2 is not met for the target block, the following processing is performed. As the EIP processing for the target block, the filter corresponding to eip_filter_idx=0 and the reconstructed area corresponding to eip_reconst_idx=0 may be used uniformly. In this case, eip_filter_idx and eip_reconst_idx are not encoded and are estimated to be 0 by the decoding device, thus reducing the amount of code in the bitstream. In this embodiment, eip_filter_idx=0 corresponds to the filter in Figure 3(a), and eip_reconst_idx=0 corresponds to the reconstruction region in Figure 4(a). However, the embodiment is not limited to this; eip_filter_idx=0 may also correspond to the filter in Figure 3(b) or Figure 3(c), and eip_reconst_idx=0 may correspond to the reconstruction region in Figure 4(b) or Figure 4(c).

[0081] In this embodiment, eip_filter_idx=0 corresponds to the filter in Figure 3(a). This is because the filter in Figure 3(a) is square. Compared to the rectangles in Figures 3(b) and 3(c), the square filter is easier to use due to factors such as the position of the target block. Also, as an example, square filters tend to be used more frequently.

[0082] In this embodiment, eip_reconst_idx is described as being encoded, but this is not limited to this, and eip_reconst_idx does not need to be encoded. In this case, when the prediction unit 103 selects one of the filters shown in Figure 3, it determines the reconstruction region to be used according to the selected filter. For example, if the prediction unit 103 selects the filter in Figure 3(a) as the filter to be used, it uniformly selects the reconstruction region in Figure 4(a) as the reconstruction region to be used. Similarly, if the prediction unit 103 selects the filter in Figure 3(b) as the filter to be used, it uniformly selects the reconstruction region in Figure 4(b) as the reconstruction region to be used. Similarly, if the prediction unit 103 selects the filter in Figure 3(c) as the filter to be used, it uniformly selects the reconstruction region in Figure 4(c) as the reconstruction region to be used. In this case, there is no need to encode eip_reconst_idx, and the prediction unit 103 encodes eip_filter_idx. That is, both the filter to be used and the reconstruction region can be selected by eip_filter_idx.

[0083] In this embodiment, the shapes of the filter and reconstruction region specified by the syntax elements are assumed to be the same. For example, the filter shape in Figure 3(b) and the reconstruction region shape in Figure 4(b) are the same, and when the filter in Figure 3(b) is used, the reconstruction region in Figure 4(b) is used. In other words, a horizontally elongated rectangular filter corresponds to a horizontally elongated rectangular reconstruction region. When using the square filter in Figure 3(a), the L-shaped reconstruction region in Figure 4(a) is used. In the case of the L-shaped reconstruction region, the shapes are not exactly the same, but the square filter and the L-shaped reconstruction region are associated. In other words, for the square filter, the L-shaped reconstruction region in Figure 4(a) is used. Here, the L-shaped reconstruction region includes a first part located on the left side of the target block and a second part located on the upper side of the target block. It is sufficient to include at least these first and second parts, but the L-shaped reconstruction region may also include the part located in the upper left of the target block, as shown in Figure 4(a). In the example shown in Figure 4(a), the filter consists of a rectangular section comprising a first and a third part, and a second part perpendicular to the rectangular section. When using the vertically elongated rectangular filter shown in Figure 3(c), the vertically elongated rectangular reconstruction area shown in Figure 4(c) is used. In other words, the shape of the filter and the shape of the reconstruction area are the same and are used together. In this embodiment, by specifying the reconstruction area according to the filter to be used, it becomes possible to use a reconstruction area suitable for the filter when the filter scans the reconstruction area and the target block. The reconstruction area may be adjacent to the target block or may be far from the target block. For example, the reconstruction area may be about 1 to 8 pixels away from the target block. Now, the image encoding process by the encoding device 100 will be further explained according to the flowchart in Figure 11. First, in step S1101, the encoding unit 109 encodes the header information (SPS, PPH, PH, SH, etc.) necessary for image encoding. For example, it encodes num_eip_merge_cand_minus2, temporal_mvp_enabled_flag, eip_enabled_flag, etc.In this embodiment, num_eip_merge_cand_minus2, temporal_mvp_enabled_flag, and eip_enabled_flag are encoded in SPS. However, they are not limited to these and may be encoded in other parameter sets such as PPS, PH, SH, etc. Also, when eip_enabled_flag is 0 (false), it is possible to reduce the amount of code by omitting the encoding of num_eip_merge_cand_minus2. Next, in step S1102, the encoding unit 109, and the block division unit 102, divide the input image into a plurality of basic blocks.

[0084] In step S1103, the prediction unit 103 selects one unselected basic block from the multiple basic blocks obtained in step S1102, according to the raster scan order starting from the top left edge of the image. The prediction unit 103 then determines a subblock division method (division pattern) for the selected basic block and divides the selected basic block into multiple subblocks according to the determined subblock division method. At this time, the prediction unit 103 divides the target basic block into multiple subblocks according to the division method shown in Figure 8, and if necessary, further divides the resulting subblocks according to the division method shown in Figure 8. In this way, the prediction unit 103 recursively divides the basic block into multiple subblocks. The prediction unit 103 also determines the prediction processing mode for each subblock. Prediction processing used at this time includes intra prediction, inter prediction, and EIP. The prediction unit 103 then generates a predicted image for each subblock of the currently selected basic block by using the image in the frame memory 107 and making a prediction according to the determined prediction mode, and calculates the difference between the input image and the predicted image as the prediction error. When EIP is used for a certain target block, the prediction unit 103 generates a predicted image for that target block according to the method described above with reference to Figures 3 to 7. The prediction unit 103 also generates prediction information, such as information indicating the subblock division method and the prediction processing mode. When EIP is used, this prediction information includes, for example, multiple syntax elements as shown in Figure 10.

[0085] In step S1104, the transformation / quantization unit 104 generates transformation coefficients for each subblock by performing a transformation process on the prediction error for each subblock. Then, for each subblock, the transformation / quantization unit 104 quantizes the transformation coefficient of the subblock to generate the quantization coefficient of the subblock. In step S1105, the inverse quantization / inverse transformation unit 105 generates transformation coefficients by inverse quantizing the quantization coefficients of each subblock generated in step S306 using the quantization matrix used for the quantization of the subblock. Then, the inverse quantization / inverse transformation unit 105 performs an inverse transformation process on the generated transformation coefficients to generate the prediction error.

[0086] In step S1106, the image reconstruction unit 106 generates a predicted image from the image currently being processed or from a processed image stored in the frame memory 107, based on the predicted information generated by the prediction unit 103. The image reconstruction unit 106 then reconstructs the image from the predicted image and the prediction error generated by the inverse quantization / inverse transform unit 105. The image reconstruction unit 106 then stores the reconstructed image in the frame memory 107. In step S1107, the encoding unit 109 entropy encodes the quantization coefficients generated by the transform / quantization unit 104 and the predicted information generated by the prediction unit 103 to generate encoded data. Furthermore, the encoding unit 109 multiplexes the header information and the generated encoded data to generate a bitstream. In step S1108, the control unit 101 determines whether all basic blocks have been encoded. As a result of this determination, if all basic blocks have been encoded, the process proceeds to step S1109. If there are still basic blocks that have not yet been encoded, the process proceeds to step S1103, where the next basic block to be processed is selected. In step S1109, the in-loop filter unit 108 performs in-loop filtering, such as deblocking filtering and sample adaptive offsetting, on the image stored in the frame memory 107.

[0087] As described above, in this embodiment, it is possible to encode the target block using EIP processing. In the above description, three patterns shown in Figure 3 were described as types of EIP filters, but this is not the only way. Below, a modified example of this embodiment will be described in which other types of filters can be used.

[0088] Figure 12 shows the types of filters used in the modified examples. The filter types are divided into three size groups. Size group 0 includes, as in Figure 3, a 4-pixel × 4-pixel filter shown in Figure 12(a), an 8-pixel × 2-pixel filter shown in Figure 12(b), and a 2-pixel × 8-pixel filter shown in Figure 12(c). Size group 1 includes a 3-pixel × 3-pixel filter shown in Figure 12(d). Furthermore, it includes a 4-pixel × 2-pixel filter shown in Figure 12(e) and a 2-pixel × 4-pixel filter shown in Figure 12(f). Furthermore, it includes an 8-pixel × 1-pixel filter shown in Figure 12(g) and a 1-pixel × 8-pixel filter shown in Figure 12(h). Furthermore, size group 2 includes a 2-pixel × 2-pixel filter shown in Figure 12(i), a 4-pixel × 1-pixel filter shown in Figure 12(j), and a 1-pixel × 4-pixel filter shown in Figure 12(k). Although the correspondence between each pixel position and the value of i is shown in Figure 12 for each filter, the correspondence between each pixel position and the value of i in each filter is not limited to the examples shown in Figure 12; other correspondences may also be possible.

[0089] When using a filter for size group 0, the prediction unit 103 generates predicted pixels for the target block using equation (1) or (2) described above.

[0090] When using the filter shown in Figure 12(d) which belongs to size group 1, the prediction unit 103 generates predicted pixels for the target block using the following equation (4).

[0091]

number

[0092] As shown in equation (4), size group 1 is smaller than a predetermined size (size group 0). Therefore, unlike in equation (1), the prediction unit 103 creates a predicted image using the reconstructed pixel value or predicted pixel value corresponding to i=7, which is the position furthest from the predicted pixel position (lower right corner). In this way, depending on the size of the target block or filter, the prediction unit 103 may decide whether to use one or more pixels in the region that includes the pixel furthest from the predicted pixel position.

[0093] However, even in size group 1, pixels located far from the predicted pixel position may be excluded, as shown in equation (5) below.

[0094]

number

[0095] Depending on the characteristics of the image, the encoding device may decide which formula to use to predict the image and signal it to the bitstream using an index or the like.

[0096] Furthermore, when using the filters shown in Figures 12(e) to (h) that belong to size group 1, the prediction unit 103 generates predicted pixels for the target block using the following equation (6).

[0097]

number

[0098] As shown in equation (6), size group 1 is smaller than a predetermined size (size group 0). Therefore, unlike in equation (1), the prediction unit 103 creates a predicted image using the reconstructed pixel value or predicted pixel value corresponding to i=7, which is the position furthest from the predicted pixel position (lower right corner). In this way, depending on the size of the target block or filter, the prediction unit 103 may decide whether to use one or more pixels in the region that includes the pixel furthest from the predicted pixel position.

[0099] However, even in size group 1, pixels located far from the predicted pixel position may be excluded, as shown in equation (7) below.

[0100]

number

[0101] Depending on the characteristics of the image, the encoding device may decide which formula to use to predict the image and signal it to the bitstream using an index or the like.

[0102] As described above, within the same size group 1, filters with different pixel counts are mixed, as shown in Figure 12(d) and Figures (e) to (h). By grouping these together in this way, the amount of code used when signaling the group can be suppressed.

[0103] Furthermore, when using the filters shown in Figures 12(i) to (k) that belong to size group 2, the prediction unit 103 generates predicted pixels for the target block using the following equation (8).

[0104]

number

[0105] As shown in equation (8), size group 2 is smaller than the predetermined size (size group 0). Therefore, unlike in equation (1), the prediction unit 103 creates the predicted image using the reconstructed pixel value or predicted pixel value corresponding to i=2, which is the position furthest from the predicted pixel position (lower right corner). In this way, depending on the size of the target block or filter, the prediction unit 103 may decide whether to use one or more pixels in the region that includes the pixel furthest from the predicted pixel position.

[0106] However, even in size group 2, pixels located far from the predicted pixel position may be excluded, as shown in equation (9) below.

[0107]

number

[0108] Depending on the characteristics of the image, the encoding device may decide which formula to use to predict the image and signal it to the bitstream using an index or the like.

[0109] As described above, the prediction unit 103 calculates the filter coefficient c for the selected filter. i When generating the filter coefficient c, use one of the reconstruction regions shown in Figure 4. i This will generate [something].

[0110] Here, we will explain the coding_unit() syntax structure in a modified example of Figure 12 that allows the use of filters. Figure 13 is an example of the coding_unit() syntax structure in a modified example of Figure 12 that allows the use of filters. Note that the same parts as in Figure 10 will not be explained. The difference from the coding_unit() syntax structure in Figure 10 is that the coding_unit() syntax structure in Figure 13 newly includes eip_filter_size_idx. eip_filter_size_idx is a syntax element that identifies the size group of filters to be used for the target block.

[0111] [Table 1]

[0112] As shown in Table 1, eip_filter_size_idx=0 indicates that size group 0 in Figure 12 will be used. In this case, the range of possible values ​​for eip_filter_idx is 0 to 2. A value of 0 for eip_filter_idx indicates the filter in Figure 12(a). A value of 1 for eip_filter_idx indicates the filter in Figure 12(b). A value of 2 for eip_filter_idx indicates the filter in Figure 12(c). Similarly, eip_filter_size_idx=1 indicates that size group 1 in Figure 12 will be used. In this case, the range of possible values ​​for eip_filter_idx is 0 to 4. A value of 0 for eip_filter_idx indicates the filter in Figure 12(d). A value of 1 for eip_filter_idx indicates the filter in Figure 12(e). A value of 2 for eip_filter_idx indicates the filter in Figure 12(f). A value of 3 for eip_filter_idx indicates the filter in Figure 12(g). A value of 2 for eip_filter_idx indicates the filter shown in Figure 12(h). Furthermore, eip_filter_size_idx=2 indicates that size group 2 in Figure 12 is used. In this case, the possible values ​​for eip_filter_idx are between 0 and 2. A value of 0 for eip_filter_idx indicates the filter shown in Figure 12(i). A value of 1 for eip_filter_idx indicates the filter shown in Figure 12(j). A value of 2 for eip_filter_idx indicates the filter shown in Figure 12(k).

[0113] In Figure 12, if eip=filter_size=idx, eip_filter=idx, and eip_reconst=idx are not encoded, each syntax element is constrained to a predetermined value. Here, the cases where they are not encoded include cases where condition2 is false, and the value of each syntax element is always 0 (constrained to the value of 0). When eip_flag is 1 and eip_merge_flag is 0 and condition2 is false, each syntax element takes the following value. In other words, in the above case, eip_filter_size_idx, eip_filter_idx, and eip_reconst_idx are always 0. As a result, the filter in Figure 12(a) and the reconstruction region in Figure 4(a) are used.

[0114] By enabling the use of size group 1 and size group 2 as second filters, in addition to size group 0, which has 16 sizes such as 4x4 as the first filter, it is possible to generate predictive images more flexibly and reduce the amount of data.

[0115] (Second embodiment) In this embodiment, a decoding device 200 that decodes a bitstream obtained by encoding an image using the encoding device 100 according to the first embodiment will be described. The differences from the first embodiment will be described below, and unless otherwise specified, it will be the same as the first embodiment. An example of the functional configuration of the decoding device 200 according to this embodiment will be described using the block diagram in Figure 2.

[0116] The control unit 201 controls the overall operation of the decoding device 200. The separation and decoding unit 202 acquires the bitstream generated by the encoding device 100, separates encoded data related to the decoding process and coefficients from the bitstream, and decodes the encoded data present in the bitstream header. For example, the separation and decoding unit 202 decodes num_eip_merge_cand_minus2, temporal_mvp_enabled_flag, and eip_enabled_flag from the SPS.

[0117] The decoding unit 203 decodes the encoded data separated from the bitstream by the separation decoding unit 202 to obtain quantization coefficients and prediction information. The inverse quantization / inverse transform unit 204 operates similarly to the inverse quantization / inverse transform unit 105 of the encoding device 100. The inverse quantization / inverse transform unit 204 obtains transform coefficients by performing inverse quantization on the obtained quantization coefficients, and obtains the prediction error by performing an inverse transform on these transform coefficients.

[0118] The image reconstruction unit 205 performs prediction processing by referring to the image stored in the frame memory 206 based on the prediction information decoded from the bitstream by the decoding unit 203, and generates a predicted image. Then, the image reconstruction unit 205 reconstructs the image using the generated predicted image and the prediction error obtained by the inverse quantization / inverse transform unit 204, and stores the reconstructed image in the frame memory 206.

[0119] The in-loop filter unit 207 performs in-loop filtering, such as deblocking filtering, on the reconstructed image stored in the frame memory 206. The image stored in the frame memory 206 is output as appropriate by the control unit 201. The output destination of the image is not limited to a specific destination; for example, the reconstructed image may be displayed on the display screen of a display device such as a monitor, or the image may be output to a projection device such as a projector.

[0120] Next, the operation of the decoding device 200 having the above configuration (bitstream decoding process) will be described. In this embodiment, the bitstream input to the separation decoding unit 202 is assumed to be a bitstream for each frame of a moving image, but it may also be a bitstream of a still image.

[0121] Here, the prediction process performed by the image reconstruction unit 205 of the decoding device 200 will be described in more detail. The image reconstruction unit 205 is capable of performing the same prediction process as the prediction unit 103.

[0122] For example, when the image reconstruction unit 205 decodes eip_filter_idx and eip_reconst_idx from the coding_unit() syntax structure in Figure 10 for the current target block, it performs the following process. That is, the image reconstruction unit 205 selects one of the filters in Figure 3 as the filter to be used in EIP for the target block, according to the value of eip_filter_idx. Furthermore, the image reconstruction unit 205 selects one of the reconstruction regions shown in Figure 4 for the target block, according to the decoded value of eip_reconst_idx, and generates the filter coefficients of the selected filter using the selected reconstruction region. Then, the image reconstruction unit 205 generates predicted pixels for the target block using one of the formulas described above.

[0123] Here, we will explain an example where the size of the target block is 4 pixels × 4 pixels, and a 4 pixel × 4 pixel filter as shown in Figure 3(a) is selected. In this case, the image reconstruction unit 205 positions the 4 pixel × 4 pixel filter 503 so that its position (x,y) is at the top left (0,0) position of the 4 pixel × 4 pixel target block 501, similar to the process described in the first embodiment. Then, according to one of the formulas described above, the predicted pixel value pred at position (x,y) is calculated. (x,y) The following is derived. At the position (x,y) shown in Figure 5(a), the predicted pixel value pred (x,y) After the formula is derived, the image reconstruction unit 205 shifts the position of the filter 503 by one pixel according to the scanning order shown by the arrow in Figure 5(a), and predicts the pixel value pred for the position (x,y) shown in Figure 5(b) according to one of the formulas described above. (x,y) The image reconstruction unit 205 shifts the selected filter one pixel at a time according to the scanning order indicated by the arrows in Figure 5(a), and generates the predicted pixel value pred (x,y) The data is acquired sequentially, and a predicted image of the target block is generated.

[0124] The scanning order of the filter in the target block is not limited to the example shown in Figure 5(a), but may also be the scanning order shown in Figures 6(a) and 6(b). Figure 6(a) shows the order in which each row of pixels (hereinafter referred to as "pixel row") in an 8x8 target block is scanned horizontally. In this case, after the scan for predictive pixel generation is performed on the pixel row at the top of the 8x8 target block, the scan for predictive pixel generation may be performed on the pixel row immediately adjacent to that pixel row. That is, after the scan for predictive pixel generation is performed on a certain pixel row, the scan for predictive pixel generation may be performed on the pixel row immediately adjacent to that pixel row. In addition, predictive pixel generation is performed on the leftmost pixel in the pixel row at the top of the 8x8 target block. Subsequently, in parallel with the predictionive pixel generation being performed on the pixel adjacent to the right of that pixel according to the scanning order shown in Figure 6(a), predictive pixel generation may also be performed on the leftmost pixel of the pixel row adjacent to the pixel row below the top pixel row. In subsequent processing, as shown in Figure 6(a), filter scanning for generating predicted pixels may be performed in parallel, shifting by one pixel for each pixel row of the target block. This makes it possible to perform parallel processing for each pixel row for generating predicted pixels.

[0125] Similarly, Figure 6(b) shows the order in which each pixel column (hereinafter referred to as "pixel column") of an 8x8 target block is scanned vertically. At this time, after the scan for predictive pixel generation is performed on the leftmost pixel column of the 8x8 target block, the scan for predictive pixel generation may be performed on the pixel column adjacent to the right of that pixel column. That is, after the scan for predictive pixel generation is performed on a certain pixel column, the scan for predictive pixel generation may be performed on the pixel column adjacent to the right of that pixel column. In addition, predictive pixel generation is performed on the top pixel of the leftmost pixel column of the 8x8 target block. Subsequently, in parallel with the predictionive pixel generation of the pixels adjacent below that pixel according to the scan order shown in Figure 6(b), predictive pixel generation may be performed on the top pixel of the pixel column adjacent to the right of the leftmost pixel row. At this time, in the subsequent processing, as shown in Figure 6(b), filter scans for predictive pixel generation may be performed in parallel, shifting by one pixel for each pixel column of the target block. This approach makes it possible to perform parallel processing for each pixel row for predictive pixel generation.

[0126] Furthermore, the scan order of the filters in the target block may be as shown in Figure 5(c). This scan order is the same as the scan order used for encoding the quantization matrix and quantization coefficients of HEVC and VVC, but by standardizing the scan order, it is possible to reduce the memory used for the table during scanning.

[0127] Furthermore, in the explanation above, the filter coefficient c used in the EIP prediction process i The explanation described obtaining this using the reconstruction region surrounding the target block, but it is not limited to this. For example, when using EIP merge mode, the filter coefficient c used in blocks where prediction processing of EIPs other than the target block was performed. i This may be used in the EIP prediction process for the target block.

[0128] When the image reconstruction unit 205 decodes an eip_merge_flag with a value of 1 for a target block, it decides to use EIP merge mode for that target block. In this case, the image reconstruction unit 205 generates an EIP merge list (mergeEIPCandList) in the same way as the prediction unit 103 processes in EIP merge mode. Specifically, it generates an EIP merge list for the target block 701 in the current image from spatially adjacent blocks or nearby blocks (blocks containing positions A0-A1 / blocks containing positions B0-B2). It also generates an EIP merge list for blocks in temporally different images at positions spatially corresponding to the target block 701. Alternatively, an EIP merge list is generated for the target block 701 from adjacent / neighboring blocks (blocks containing the position of C0 / blocks containing the position of C1). At this time, the prediction unit 103 of the encoding device 100 and the image reconstruction unit 205 of the decoding device 200 each generate the same EIP merge list for the target block. Note that the process of generating the EIP merge list by the image reconstruction unit 205 is the same as the process of generating the EIP merge list by the prediction unit 103 described in Embodiment 1, so the explanation is omitted. As mentioned above, the EIP merge list can include information of blocks surrounding the target block (including adjacent blocks) as candidates for information to be used in the target block. The EIP merge list can also include information of blocks (including adjacent blocks) at spatially corresponding positions to the target block or surrounding blocks in images that are different in time, as candidates.

[0129] The image reconstruction unit 205 then adds 2 to num_eip_merge_cand_minus2, which is decoded from the bitstream, to derive MaxNumEIPCand, the maximum number of candidates in the EIP merge list. After generating an EIP merge list as shown in Figures 7(a) and 7(b), the image reconstruction unit 205 may further add candidates according to the current number of candidates and the maximum number of candidates in the EIP merge list. Specifically, the image reconstruction unit 205 compares the current number of candidates in the EIP merge list, based on information from other blocks, with the maximum number of candidates. Then, according to the result of this comparison, the image reconstruction unit 205 adds candidates other than those already included in the list. For example, if the current number of candidates in the EIP merge list is less than the maximum number of candidates, additional candidates may be added to the EIP merge list until the number of candidates in the EIP merge list reaches the maximum number of candidates. For example, if the number of candidates in the generated EIP merge list is 5 and the maximum number of candidates in the EIP merge list is 7, the image reconstruction unit 205 adds 2 more candidates. The additional candidates added here are, for example, candidates associated with a predetermined filter (e.g., the filter in Figure 3(a)) and a set of predetermined filter coefficients Ci for that predetermined filter. The predetermined filter coefficients Ci may be fixed values ​​where all values ​​are the same, or they may be predetermined coefficients generated in advance by machine learning. Furthermore, if there are two or more candidates with the same filter among the candidates already stored in the EIP merge list, a new new filter coefficient is obtained by averaging the filter coefficients Ci of those two or more candidates.

[0130]

number

[0131] The set of filters is generated, and additional candidates associated with the set may be added to the EIP merge list. Specifically, if the same filter (for example, the filter in Figure 3(a)) is associated with the first candidate n and the second candidate m already stored in the EIP merge list, the prediction unit 103 uses equation (7) to determine the first candidate n

[0132]

number

[0133] and second candidate m

[0134]

number

[0135] and average to create a new

[0136]

number

[0137] This generates the following. Equation (10) below is the same as equation (3) above.

[0138]

number

[0139] The decoding unit 203 decodes the eip_merge_idx for the target block from the coding_unit() syntax structure of the bitstream. The image reconstruction unit 205 generates an EIP merge list and then identifies candidates to be used for the EIP of the target block from among the candidates in the generated EIP merge list, according to the value of eip_merge_idx. eip_merge_idx is binarized by a predetermined binarization process and encoded using bypass decoding or arithmetic decoding. In this case, the first N bins (N is a positive integer) of the binarized sequence of eip_merge_idx may be subjected to arithmetic decoding, and the rest to bypass decoding. For example, the first bin may be subjected to arithmetic decoding, and the rest to bypass decoding. When arithmetic decoding is used for one or more bins of the binarized sequence of eip_merge_idx, the context variables used depend on the following. The context variables used in arithmetic decoding depend, for example, on whether the subblocks adjacent to the target block and the subblock adjacent to the left of the target block are using EIP.

[0140] In this embodiment, the decoding unit 203 decodes information from the bitstream indicating how the basic blocks of the current image to be processed are divided. It also identifies the positions and sizes of multiple subblocks within each basic block of the current image according to this information. Furthermore, it decodes the coding_unit() syntax structure contained in the Picture data of the bitstream in Figure 9 to obtain prediction information for the target block.

[0141] The decoding unit 203 decodes eip_flag if condition1 is true. Note that condition1 is the same as described in the first embodiment, so its explanation is omitted. If condition1 is false, the decoding unit 203 estimates that the value of eip_flag for the target block is 0. In other words, the decoding unit 203 decides not to perform EIP on the target block.

[0142] If the decoded eip_flag is 1 (true), the decoder 203 decodes eip_merge_flag from coding_unit(). If the decoded eip_merge_flag is 1, the decoder 203 decides to use the EIP merge mode for the target block. On the other hand, if eip_merge_flag is 0, the decoder 203 decides not to use the EIP merge mode for the target block. If eip_flag is 1 (true), the decoder 203 obtains the value of MaxNumEIPCand, which is the maximum number in the EIP merge list, and if this value is greater than 1, it further decodes eip_merge_idx into coding_unit(). Note that if MaxNumEIPCand is 1, eip_merge_idx is not decoded, and the value of eip_merge_idx is estimated to be 0. The decoder 203 is identified by adding 2 to num_eip_merge_cand_minus2 decoded from SPS.

[0143] If eip_merge_flag is 0 (false) and condition2 is true, the decoding unit 203 decodes eip_filter_idx and eip_reconst_idx from the coding_unit() syntax structure. In the embodiment of Figure 10, eip_filter_idx is a syntax element that identifies the filter used for EIP for the target block from among the multiple filters shown in Figure 3. For example, eip_filter_idx=0 indicates the filter in Figure 3(a), eip_filter_idx=1 indicates the filter in Figure 3(b), and eip_filter_idx=2 indicates the filter in Figure 3(c). eip_reconst_idx is a syntax element for identifying the reconstruction region used to generate filter coefficients in EIP for the target block from among the multiple reconstruction regions shown in Figure 4. For example, eip_reconst_idx=0 indicates the reconstructed region in Figure 4(a), eip_reconst_idx=1 indicates the reconstructed region in Figure 4(b), and eip_reconst_idx=2 indicates the reconstructed region in Figure 4(c). Note that condition2 is the same as described in the first embodiment, so its explanation is omitted.

[0144] In this embodiment, if eip_flag is 1 and eip_merge_flag is 0 and condition2 is true, the following decoding process is performed. The decoding unit 203 decodes eip_filter_idx and eip_reconst_idx from the codint_unit() syntax structure. On the other hand, even if eip_flag is 1 and eip_merge_flag is 0, if condition2 is false, the following decoding process is performed. In this case, eip_filter_idx and eip_reconst_idx are not decoded from the codint_unit() syntax structure. The decoding unit 203 also estimates that the values ​​of eip_filter_idx and eip_reconst_idx are 0 when they are not decoded from the aforementioned syntax structure. In other words, if the above-mentioned condition2 is not satisfied for a target block, the EIP processing for that target block may uniformly use the filter corresponding to eip_filter_idx=0 and the reconstruction area corresponding to eip_reconst_idx=0. In this embodiment, eip_filter_idx=0 corresponds to the filter in Figure 3(a), and eip_reconst_idx=0 corresponds to the reconstructed region in Figure 4(a). However, the embodiment is not limited to this; eip_filter_idx=0 may also correspond to the filter in Figure 3(b) or Figure 3(c), and eip_reconst_idx=0 may correspond to the reconstructed region in Figure 4(b) or Figure 4(c).

[0145] In this embodiment, eip_reconst_idx is described as being included in the syntax structure of Figure 10, but this is not limited to this, and eip_reconst_idx does not have to be encoded at all (i.e., it does not have to be decoded). In this case, if the decoding unit 103 selects one of the filters shown in Figure 3 according to the value of eip_filter_idx, it determines the reconstruction region according to the selected filter. For example, if the decoding unit 203 selects the filter in Figure 3(a) according to eip_filter_idx, it uniformly selects the reconstruction region in Figure 4(a) as the reconstruction region to be used. Similarly, if the decoding unit 203 selects the filter in Figure 3(b) according to eip_filter_idx, it uniformly selects the reconstruction region in Figure 4(b) as the reconstruction region to be used. Similarly, if the decoding unit 203 selects the filter in Figure 3(c) according to eip_filter_idx, it uniformly selects the reconstruction region in Figure 4(c) as the reconstruction region to be used. In such cases, there is no need to decode eip_reconst_idx, and the decoding unit 203 decodes eip_filter_idx instead. That is, eip_filter_idx allows selection of both the filter to be used and the reconstruction region.

[0146] Here, the decoding process of a single frame of bitstream by the decoding device 200 will be further explained according to the flowchart in Figure 14. First, in step S1401, the separation and decoding unit 202 acquires a single frame of bitstream generated by the encoding device 100. The separation and decoding unit 202 then separates encoded data related to the decoding process and coefficients from the bitstream and decodes the encoded data present in the bitstream header. Here, the separation and decoding unit 202 decodes, for example, num_eip_merge_cand_minus2, temporal_mvp_enabled_flag, eip_enabled_flag, etc. from the SPS.

[0147] Next, in step S1402, the decoding unit 203 decodes information from the bitstream indicating how the basic block of the image currently being processed is divided. Furthermore, according to this information, it identifies the positions and sizes of the multiple subblocks obtained by the division in the basic block. Note that the division into multiple subblocks performed recursively in the basic block is the same as described with reference to Figure 8, so the explanation is omitted.

[0148] From here on, steps 1403 to 1405 are processes performed for each subblock included in the basic block currently being processed. In step S1403, the inverse quantization / inverse transform unit 204 obtains transform coefficients by performing inverse quantization on the quantization coefficients of the subblock to be decoded, and obtains a prediction error by performing an inverse transform on these transform coefficients. In step S1404, the image reconstruction unit 205 generates a predicted image by performing a prediction process that references the image stored in the frame memory 206 based on the prediction information decoded for the subblock to be decoded by the decoding unit 203. Prediction processes used at this time include intra-prediction, inter-prediction, and EIP. When EIP is used for the subblock to be decoded, the image reconstruction unit 205 generates a predicted image for that target block according to the method described above with reference to Figures 3 to 7. The image reconstruction unit 205 then decodes the image of the subblock to be decoded using the generated predicted image and the prediction error obtained by the inverse quantization / inverse transform unit 204, and stores the generated subblock image in the frame memory 206.

[0149] In step S1406, the control unit 201 determines whether or not the decoding of all basic blocks contained in the bitstream has been completed. If the result of this determination is that the decoding of all basic blocks contained in the bitstream has been completed, the process proceeds to step S1407. On the other hand, if there are still basic blocks in the bitstream that have not yet been decoded, the process proceeds to step S402 for those basic blocks that have not yet been decoded, and the process is repeated.

[0150] In step S1407, the in-loop filter unit 207 performs in-loop filtering, such as deblocking filtering, on the reconstructed image stored in the frame memory 206.

[0151] As described above, in this embodiment, it is possible to decode an image using EIP processing on the block to be decoded. In the above description, three patterns of EIP filters shown in Figure 3 were described, but this is not limited to these. As with the description in the first embodiment, it is possible to use the filter shown in Figure 12 as a modified example of this embodiment. The following describes the case in which the filter shown in Figure 12 is used in this embodiment.

[0152] In this modified example, the decoding unit 203 can decode eip_filter_size_idx from the coding_unit() syntax structure shown in Figure 13, which is the coding_unit() syntax structure for the subblock currently to be decoded. Specifically, the decoding unit 203 performs the following processing when the value of eip_flag decoded from the coding_unit() syntax structure in Figure 13 is 1 and the value of the decoded eip_merge_flag is 0 for the subblock to be decoded. In this case, according to condition 2, eip_filter_size_idx, eip_filter_idx, and eip_reconst_idx are decoded from the coding_unit() syntax structure. Then, as shown in Table 1 in the first embodiment, the decoding unit 203 identifies the size group to be used according to the value of eip_filter_size_idx. Furthermore, from the multiple filters belonging to that size group, the filter to be used is identified from the value of the decoded eip_filter_idx.

[0153] When using filters from size group 0, the image reconstruction unit 205 generates predicted pixels for the target block using equation (1) or (2). When using the filter shown in Figure 12(d) belonging to size group 1, the image reconstruction unit 205 generates predicted pixels for the target block using equation (4) or (5). When using the filters shown in Figures 12(e) to (h) belonging to size group 1, the image reconstruction unit 205 generates predicted pixels for the target block using equation (6) or (7). When using the filters shown in Figures 12(i) to (k) belonging to size group 2, the image reconstruction unit 205 generates predicted pixels for the target block using equation (8) or (9).

[0154] The image reconstruction unit 205 calculates the filter coefficient c for the selected filter. i When generating the reconstructed region, one of the reconstruction regions shown in Figure 4 is identified according to the decoded value of eip_reconst_idx, and the filter coefficient c of the selected filter is applied to the subblock to be decoded using the identified reconstruction region. i This will generate [something].

[0155] Note that eip_filter_size_idx=0 indicates that size group 0 in Figure 12 will be used. In this case, the range of possible values ​​for eip_filter_idx is 0 to 2. A value of 0 for eip_filter_idx indicates the filter in Figure 12(a). A value of 1 for eip_filter_idx indicates the filter in Figure 12(b). A value of 2 for eip_filter_idx indicates the filter in Figure 12(c). Similarly, eip_filter_size_idx=1 indicates that size group 1 in Figure 12 will be used. In this case, the range of possible values ​​for eip_filter_idx is 0 to 4. A value of 0 for eip_filter_idx indicates the filter in Figure 12(d). A value of 1 for eip_filter_idx indicates the filter in Figure 12(e). A value of 2 for eip_filter_idx indicates the filter in Figure 12(f). A value of 3 for eip_filter_idx indicates the filter in Figure 12(g). A value of 2 for eip_filter_idx indicates the filter shown in Figure 12(h). Furthermore, eip_filter_size_idx=2 indicates that size group 2 in Figure 12 is used. In this case, the possible values ​​for eip_filter_idx are between 0 and 2. A value of 0 for eip_filter_idx indicates the filter shown in Figure 12(i). A value of 1 for eip_filter_idx indicates the filter shown in Figure 12(j). A value of 2 for eip_filter_idx indicates the filter shown in Figure 12(k).

[0156] In Figure 12, if eip_filter_size_idx, eip_filter_idx, and eip_reconst_idx are not decoded, the decoding unit 203 estimates that the value of each syntax element is 0. The cases in which the aforementioned syntax elements are not decoded include cases where condition2 is false. In particular, a case where eip_flag is 1 and eip_merge_flag is 0 and condition2 is false is included. In this case, eip_filter_size_idx, eip_filter_idx, and eip_reconst_idx will always be 0. As a result, the image reconstruction unit 205 will use the filter in Figure 12(a) and the reconstruction region in Figure 4(a) for the subblock to be decoded.

[0157] (Third embodiment) Each of the functional units shown in Figures 1 and 2 may be implemented entirely in hardware, but some may be implemented in software (computer programs). In that case, a computer device having frame memory 107 and frame memory 206 as memory devices and capable of executing the computer program can be applied to the above-mentioned encoding device 100 and decoding device 200. An example of the hardware configuration of a computer device applicable to the above-mentioned encoding device 100 and decoding device 200 will be explained using the block diagram in Figure 15.

[0158] The CPU 1501 executes various processes using computer programs and data stored in the RAM 1502 and ROM 503. In this way, the CPU 501 controls the operation of the entire computer system and also executes or controls the processes performed by the encoding device 100 and decoding device 200.

[0159] RAM 1502 has an area for storing computer programs and data loaded from ROM 1503 and external storage device 1506, as well as data received from the outside via I / F (interface) 1507 (for example, the video and still image data mentioned above). Furthermore, RAM 1502 has a work area used by CPU 1501 when executing various processes. In this way, RAM 1502 can provide various areas as appropriate. ROM 1503 stores computer device configuration data and startup programs, etc.

[0160] The control unit 1504 is a user interface such as a keyboard, mouse, or touch panel, and the user can input various instructions to the CPU 1501 by operating it.

[0161] The display unit 1505 is composed of an LCD screen or a touch panel screen, and can display the processing results of the CPU 1501 as images or text. For example, the playback image decoded by the decoding device 100 may be displayed on this display unit 1505. The display unit 1505 may also be a projection device such as a projector that projects images or text.

[0162] The external storage device 1506 is a large-capacity information storage device such as a hard disk drive. The external storage device 1506 stores the OS (operating system), computer programs and data that cause the CPU 1501 to execute or control the various processes described above performed by the encoding device 100 and the decoding device 200. The computer programs stored in the external storage device 1506 include computer programs that cause the CPU 1501 to realize the functions of each functional unit other than the frame memory 107 and frame memory 206.

[0163] Computer programs and data stored in the external storage device 1506 are loaded into the RAM 1502 as appropriate according to the control of the CPU 1501 and become subject to processing by the CPU 1501. The frame memory 107 and frame memory 206 mentioned above can be implemented using memory devices such as RAM 1502, ROM 1503, and external storage device 1506.

[0164] I / F1507 functions as an interface for data communication with external devices. For example, moving images and still images can be acquired from an external server device or imaging device via I / F1507 and stored in RAM1502 or external storage device 1506.

[0165] The CPU 1501, RAM 1502, ROM 1503, operation unit 1504, display unit 1505, external storage device 1506, and I / F 1507 are all connected to the bus 1508. Note that the configuration shown in Figure 5 is merely one example of a computer device hardware configuration applicable to the above-mentioned encoding device 100 and decoding device 200, and various modifications and variations are possible.

[0166] (Other embodiments) In other embodiments, a program that implements one or more of the functions of the embodiments described above is supplied to the system or device via a network or storage medium. Furthermore, it can also be implemented by a process in which one or more processors in the computer of the system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0167] The following disclosures of embodiments include the following sections:

[0168] (Section 1) A prediction means that identifies a filter from a plurality of filters for a predetermined intra-prediction process for a target block, and generates a predicted image by the predetermined intra-prediction process using the selected filter, The system includes encoding means for encoding syntax elements to identify the filter used among the aforementioned plurality of filters, The plurality of filters include at least a first filter and a second filter, wherein the size of the first filter and the size of the second filter are different. An encoding device characterized by the following features.

[0169] (Section 2) The predetermined intra-prediction process is a prediction process that generates the predicted image by applying the filter while moving it. The encoding device according to item 1, characterized in that

[0170] (Section 3) The predetermined intra-prediction process is a prediction process that generates the predicted image by applying the filter while moving it over a region that includes at least a portion of the target block and the region surrounding the target block. The encoding device according to item 1 or item 2, characterized in that it is a device that provides an encoding device according to item 1 or item 2.

[0171] (Section 4) The encoding means encodes syntax elements for determining the size of the filter used. The encoding device according to any one of claims 1 to 3, characterized in that it is an encoding device.

[0172] (Section 5) The value of the syntax element used to identify the filter used is constrained by the syntax element used to identify the size of the filter used. The encoding device according to item 4, characterized in that

[0173] (Section 6) The size of the first filter is 16. The encoding device according to any one of claims 1 to 5, characterized in that

[0174] (Section 7) The size of the second filter is 9. The encoding device according to item 6, characterized in that

[0175] (Section 8) If the syntax element for identifying the filter used is not encoded, the size of the filter used will be 16. The encoding device according to any one of claims 1 to 7, characterized in that

[0176] (Section 9) If the syntax element for identifying the filter used is not encoded, then the filter used is square. The encoding device according to item 8, characterized in that

[0177] (Section 10) A decoding means for decoding syntax elements for identifying a filter from a plurality of filters for predetermined intra-prediction processing for a target block, Prediction means for generating a predicted image by a predetermined intra-prediction process using a filter identified according to the syntax elements, The plurality of filters include at least a first filter and a second filter, wherein the size of the first filter and the size of the second filter are different. A decoding device characterized by the following features.

[0178] (Section 11) The predetermined intra-prediction process is a prediction process that generates the predicted image by applying the filter while moving it. The decoding device according to item 10, characterized in that

[0179] (Section 12) The predetermined intra-prediction process is a prediction process that generates the predicted image by applying the filter while moving it over a region that includes at least a portion of the target block and the region surrounding the target block. A decoding device according to claim 10 or claim 11, characterized in that it is a decoding device according to claim 10 or claim 11.

[0180] (Section 13) The decoding means decodes syntax elements for determining the size of the filter used in the predetermined intra prediction process. A decoding device according to any one of claims 10 to 12, characterized in that it is a decoding device according to any one of claims 10 to 12.

[0181] (Section 14) The value of the syntax element for identifying the filter used in the predetermined intra-prediction process is constrained by the syntax element for identifying the size of the filter used in the predetermined intra-prediction process. The decoding device according to item 13, characterized in that

[0182] (Section 15) The size of the first filter is 16. The decoding device according to item 14, characterized in that

[0183] (Section 16) The size of the second filter is 9. The encoding device according to item 15, characterized in that

[0184] (Section 17) If the syntax element for identifying the filter used in the predetermined intra-prediction process is not decoded, the size of the filter used in the predetermined intra-prediction process becomes 16. The encoding device according to any one of claims 10 to 16, characterized in that

[0185] (Section 18) If the syntax element for identifying the filter used in the predetermined intra-prediction process is not decoded, the filter used in the predetermined intra-prediction process is square. The encoding device according to item 17, characterized in that

[0186] (Section 19) A prediction step involves identifying a filter from a plurality of filters for a predetermined intra-prediction process for a target block, and generating a predicted image by the predetermined intra-prediction process using the selected filter. The process includes an encoding step for encoding a syntax element to identify the filter used from among the aforementioned plurality of filters, The plurality of filters include at least a first filter and a second filter, wherein the size of the first filter and the size of the second filter are different. An encoding method characterized by the following.

[0187] (Section 20) A decoding step involves decoding syntax elements for identifying a filter from a set of filters for a predetermined intra-prediction process for the target block, A prediction step of generating a predicted image by a predetermined intra-prediction process using a filter identified according to the syntax elements, The plurality of filters include at least a first filter and a second filter, wherein the size of the first filter and the size of the second filter are different. A decoding method characterized by the following:

[0188] (Section 21) A program characterized by causing a computer to function as each of the means of the encoding device described in item 1.

[0189] (Section 22) A program characterized by causing a computer to function as one of the means of the decoding device described in item 10. [Explanation of Symbols]

[0190] 102 Block division section 103 Prediction Section 104 Conversion / Quantization Section 105 Inverse Quantization / Inverse Transformation Section 106 Image Reconstruction Unit 107 Frame Memory 108 In-loop filter section 109 Symbolization Department

Claims

1. A prediction means that identifies a filter from a plurality of filters for a predetermined intra-prediction process for a target block, and generates a predicted image by the predetermined intra-prediction process using the selected filter, The system includes encoding means for encoding syntax elements to identify the filter used among the aforementioned plurality of filters, The plurality of filters include at least a first filter and a second filter, wherein the size of the first filter and the size of the second filter are different. An encoding device characterized by the following features.

2. The predetermined intra-prediction process is a prediction process that generates the predicted image by applying the filter while moving it. The encoding device according to feature 1.

3. The predetermined intra-prediction process is a prediction process that generates the predicted image by applying the filter while moving it over a region that includes at least a portion of the target block and the region surrounding the target block. The encoding device according to feature 1.

4. The encoding means encodes syntax elements for determining the size of the filter used. The encoding device according to feature 1.

5. The value of the syntax element used to identify the filter used is constrained by the syntax element used to identify the size of the filter used. The encoding device according to feature 4.

6. The size of the first filter is 16. The encoding device according to feature 1.

7. The size of the second filter is 9. The encoding device according to feature 6.

8. If the syntax element for identifying the filter used is not encoded, the size of the filter used will be 16. The encoding device according to feature 1.

9. If the syntax element for identifying the filter used is not encoded, then the filter used is square. The encoding device according to feature 8.

10. A decoding means for decoding syntax elements for identifying a filter from a plurality of filters for predetermined intra-prediction processing for a target block, Prediction means for generating a predicted image by a predetermined intra-prediction process using a filter identified according to the syntax elements, The plurality of filters include at least a first filter and a second filter, wherein the size of the first filter and the size of the second filter are different. A decoding device characterized by the following features.

11. The predetermined intra-prediction process is a prediction process that generates the predicted image by applying the filter while moving it. The decoding device according to claim 10.

12. The predetermined intra-prediction process is a prediction process that generates the predicted image by applying the filter while moving it over a region that includes at least a portion of the target block and the region surrounding the target block. The decoding device according to claim 10.

13. The decoding means decodes syntax elements for determining the size of the filter used in the predetermined intra prediction process. The decoding device according to claim 10.

14. The value of the syntax element for identifying the filter used in the predetermined intra-prediction process is constrained by the syntax element for identifying the size of the filter used in the predetermined intra-prediction process. The decoding device according to feature 13.

15. The size of the first filter is 16. The decoding device according to feature 14.

16. The size of the second filter is 9. The encoding device according to feature 15.

17. If the syntax element for identifying the filter used in the predetermined intra-prediction process is not decoded, the size of the filter used in the predetermined intra-prediction process becomes 16. The encoding device according to feature 10.

18. If the syntax element for identifying the filter used in the predetermined intra-prediction process is not decoded, the filter used in the predetermined intra-prediction process is square. The encoding device according to feature 17.

19. A prediction step involves identifying a filter from a plurality of filters for a predetermined intra-prediction process for a target block, and generating a predicted image by the predetermined intra-prediction process using the selected filter. The process includes an encoding step for encoding a syntax element to identify the filter used from among the aforementioned plurality of filters, The plurality of filters include at least a first filter and a second filter, wherein the size of the first filter and the size of the second filter are different. An encoding method characterized by the following.

20. A decoding step involves decoding syntax elements for identifying a filter from a set of filters for a predetermined intra-prediction process for the target block, A prediction step of generating a predicted image by a predetermined intra-prediction process using a filter identified according to the syntax elements, The plurality of filters include at least a first filter and a second filter, wherein the size of the first filter and the size of the second filter are different. A decoding method characterized by the following:

21. A program characterized by causing a computer to function as each of the means of the encoding device described in claim 1.

22. A program characterized by causing a computer to function as each of the means of the decoding device described in claim 10.

Images