Encoding device, encoding method, decoding device, decoding method, and program

Abstract

An encoding device according to the present invention is characterized by comprising: a prediction means that performs a prescribed intra-prediction process using a filter on a target block and that generates a predicted image by applying the filter while moving the filter; and an encoding means that encodes a syntax element for specifying, among a plurality of filters, a filter to be used in the prescribed intra-prediction process on the target block, wherein when the encoding means does not encode the syntax element, the value of the syntax element is constrained to a prescribed value corresponding to one of the plurality of filters. The present invention can be applied to Extrapolation filter-based intra prediction (EIP).

Description

Encoding device, encoding method, decoding device, decoding method, program

[0001] This invention relates to image encoding and decoding techniques.

[0002] As an encoding method for compressed recording of moving images, the HEVC (High Efficiency Video Coding) encoding method (hereinafter referred to as HEVC) is known. In addition, the VVC (Versatile Video Coding) encoding method (hereinafter referred to as VVC), which is the successor to HEVC, is also known. In HEVC, in order to improve encoding efficiency, a larger basic block size than the conventional macroblock (16 pixels x 16 pixels) is adopted. This larger basic block is called a CTU (Coding Tree Unit), and its size is up to 64 pixels x 64 pixels. In VVC, the size of the CTU can be up to 128 pixels x 128 pixels. The CTU is further divided into subblocks that serve as units for prediction and transformation.

[0003] In recent years, efforts have been made to internationally standardize an even more efficient encoding scheme as a successor to VVC, and standardization is being advanced by JVET (Joint Video Experts Team). Non-patent document 1 is the algorithm description of the reference model (hereinafter referred to as ECM) of the new encoding scheme that has been published to date.

[0004] Furthermore, Non-Patent Document 1 describes Extraction filter-based intra prediction (EIP) as a new intra prediction process.

[0005] Algorithm description of Enhanced Compression Model 14 (ECM 14), (JVET-AI2025) (version1-date 2024-10-21)

[0006] Therefore, the present invention aims to provide a technology that enables efficient specification of information used in predictive processing.

[0007] In view of the above problems, the encoding device of the present invention comprises prediction means for performing a predetermined intra-prediction process using a filter on a target block and generating a predicted image by applying the filter while moving it, and encoding means for encoding a syntax element for identifying a filter used in the predetermined intra-prediction process for the target block from among a plurality of filters, wherein if the encoding means does not encode the syntax element, the value of the syntax element is constrained to a predetermined value corresponding to one of the plurality of filters.

[0008] Furthermore, the decoding device of the present invention includes a decoding means for decoding a syntax element for identifying a filter to be used in a predetermined intra-prediction process for a target block from among a plurality of filters for a predetermined intra-prediction process; a prediction means for identifying one of the plurality of filters according to the syntax element, executing the predetermined intra-prediction process using the identified filter, and generating a predicted image by applying the filter while moving it; and, if the decoding means does not decode the syntax element, the value of the syntax element is estimated to be a predetermined value corresponding to one of the plurality of filters.

[0009] According to the present invention, it is possible to reduce the amount of information used in predictive processing.

[0010] Block diagram showing an example of the functional configuration of an encoding device. Block diagram showing an example of the functional configuration of a decoding device. Diagram showing an example of a filter. Diagram showing an example of a filter. Diagram showing an example of a filter. Diagram showing an example of a reconstruction area. Diagram showing an example of a reconstruction area. Diagram showing an example of a reconstruction area. Diagram explaining the processing of EIP. Diagram showing an example of filter scanning. Diagram showing an example of filter scanning. Diagram explaining the generation of an EIP merge list. Diagram explaining the generation of an EIP merge list. Diagram explaining the generation of an EIP merge list. Diagram showing an example of block partitioning. Diagram showing an example of block partitioning. Diagram showing an example of block partitioning. Diagram showing an example of block partitioning. Diagram showing an example of block partitioning. Diagram showing an example of bitstream. Diagram showing an example of the coding_unit syntax structure. Flowchart of the encoding process. Diagram showing an example of a filter. Diagram showing an example of the coding_unit syntax structure. Flowchart of the decoding process. Diagram showing an example of the hardware configuration of a computer device.

[0011] The embodiments will be described below with reference to the attached drawings. The embodiments described below are examples and represent one specific embodiment of the configuration described in the claims.

[0012] The images encoded or decoded in each embodiment may be still images or videos. Furthermore, while EIP is used as an example, each embodiment is applicable to prediction processes other than EIP. In particular, it is applicable to prediction processes that require specific information, such as the information indicating filters in EIP.

[0013] (First Embodiment) First, an example of the functional configuration of the encoding device (image encoding device) 100 according to this embodiment will be explained using the block diagram in Figure 1. The control unit 101 controls the overall operation of the encoding device 100. The block division unit 102 acquires an input image (images of each frame constituting a moving image or still images), divides the input image into a plurality of basic blocks, and outputs each of the divided basic blocks. The basic block is a CTU (Coding Tree Unit), and its size is, for example, 32 pixels × 32 pixels, 64 pixels × 64 pixels, or 128 pixels × 128 pixels.

[0014] The prediction unit 103 divides each basic block into multiple subblocks. A subblock is, for example, a coding unit.

[0015] The prediction unit 103 may also divide the subblocks obtained by dividing the basic block to obtain even smaller subblocks. These further divided smaller subblocks may be, for example, a transformation block or a prediction block. Alternatively, they may simply be subblocks of a coding unit.

[0016] The prediction unit 103 generates a predicted image by performing prediction processing such as intra-prediction (intra-frame prediction) and inter-prediction (inter-frame prediction) for each sub-block, and calculates the difference between the input image and the predicted image as the prediction error. The prediction unit 103 also generates prediction information, which includes information indicating how the basic block was divided and information necessary for prediction processing, such as the prediction processing mode.

[0017] The transformation / quantization unit 104 generates transformation coefficients for each subblock by performing a transformation process on the prediction error for each subblock. It is possible to perform orthogonal transformations using transformation bases such as DCT2, DCT8, and DST7 as the transformation process. Here, the orthogonal transformation may be performed using the same transformation base in both the horizontal and vertical directions of the subblock, or it may be performed using different transformation bases in the horizontal and vertical directions of the subblock. The transformation / quantization unit 104 then generates quantization coefficients for the subblock by quantizing the transformation coefficients of the subblock. The transformation / quantization unit 104 may also quantize the transformation coefficients using a quantization matrix.

[0018] The inverse quantization / inverse transformation unit 105 inversely quantizes the quantization coefficients of each subblock generated by the transformation / quantization unit 104 to generate transformation coefficients, and then performs an inverse transformation process on these transformation coefficients to generate a prediction error.

[0019] The image reconstruction unit 106 generates a predicted image from the encoded and reconstructed images stored in the frame memory 107, based on the prediction information generated by the prediction unit 103. Furthermore, it reconstructs the image from the predicted image and the prediction error generated by the inverse quantization / inverse transform unit 105. The image reconstruction unit 106 then stores the reconstructed image in the frame memory 107. The reconstructed image stored in the frame memory 107 becomes the image data that the prediction unit 103 references when performing prediction processing on the image of the current frame or the next frame.

[0020] The in-loop filter unit 108 performs in-loop filtering, such as deblocking filtering and sample adaptive offsetting, on the images stored in the frame memory 107.

[0021] The encoding unit 109 generates encoded data by entropy encoding the quantization coefficients generated by the transformation / quantization unit 104 and the prediction information generated by the prediction unit. The entropy encoding method is not limited to a specific method; Golomb coding, arithmetic coding, Huffman coding, etc., can be used.

[0022] Here, the prediction processing performed by the prediction unit 103 in this embodiment will be described in more detail. The prediction unit 103 in this embodiment is capable of performing intra-prediction and inter-prediction in HEVC and VVC standards. Furthermore, it is capable of performing a new intra-prediction process (a predetermined intra-prediction process) called EIP (Extrapolation filter-based intra-prediction). Here, with reference to Figures 3A to 3C, the EIP performed by the prediction unit will be described.

[0023] Figures 3A to 3C show examples of filters used in EIP. The prediction unit 103 selects one of the filters from Figures 3A to 3C for the subblock currently to be processed (hereinafter referred to as the target block). Figure 3A is a 4-pixel × 4-pixel size filter, Figure 3B is an 8-pixel × 2-pixel size filter, and Figure 3C is a 2-pixel × 8-pixel size filter. The prediction unit 103 selects one of these filters. Furthermore, Figures 4A to 4C show the types of reconstruction regions. The method for determining each filter coefficient that constitutes the selected filter is not particularly limited. For example, predetermined initial values ​​may be used, or they may be set individually or generated according to the characteristics of the image. Alternatively, one of the reconstruction regions shown in Figures 4A to 4C may be selected for the target block, and the filter coefficients of the selected filter may be generated using the selected reconstruction region. Alternatively, the correlation between the pixel value at the position corresponding to the predicted pixel in the reconstruction region and the other pixel values ​​may be derived using convolution integral, and the filter coefficients may be determined based on that correlation. In this embodiment, as an example, the prediction unit 103 selects one of the reconstruction regions shown in Figures 4A to 4C for the target block, and generates the filter coefficients of the selected filter using the selected filter and the selected reconstruction region. Furthermore, aboveSize shown in Figures 4A to 4C is a size determined depending on the height of the target block, and similarly, leftSize is a size determined depending on, for example, the width of the target block. For example, aboveSize is a size equal to the height of the target block, or a size obtained by adding a predetermined value to the height of the target block. Similarly, leftSize is a size equal to the width of the target block, or a size obtained by adding a predetermined value to the width of the target block.

[0024] Equation (1) is an example of a calculation formula for generating predicted pixels using EIP.

[0025] pred （ｘ，ｙ） This is the predicted pixel value at the (x, y) position in the current target block. ｉ is the filter coefficient for position i. ｉis the reconstructed pixel value or predicted pixel value at position i, and bit depth is the bit depth of the image to be coded. Although the correspondence between each pixel position and the value of i is shown for each filter in FIGS. 3A to 3C, the correspondence between each pixel position and the value of i in each filter is not limited to the examples shown in FIGS. 3A to 3C and may be other correspondences.

[0026]

[0027] In the example shown by Expression (1), as an example, the reconstructed pixel value or predicted pixel value corresponding to i = 14, which is the position farthest from the predicted pixel position (lower right end), is not used. Thus, one or more pixels in the region including the pixel at the position farthest from the predicted pixel position may not be used for prediction.

[0028] Also, as shown in the following Expression (2), the reconstructed pixel value or predicted pixel value corresponding to i = 14, which is the position farthest from the predicted pixel position (lower right end), may also be used.

[0029]

[0030] Which expression to use for predicting the image may be determined on the encoder side according to the characteristics of the image, and signaled in the bit stream with an index or the like.

[0031] Here, an example of prediction processing using EIP when the size of the target block is 4 pixels × 4 pixels and the filter of 4 pixels × 4 pixels shown in FIG. 3A is selected will be described. As shown in FIG. 5(a), there is a reconstructed image region 502 that has been coded around the target block 501. The prediction unit 103 arranges the filter 503 of 4 pixels × 4 pixels so that the position (x, y) of the filter 503 of 4 pixels × 4 pixels becomes the upper left (0, 0) position of the target block 501 of 4 pixels × 4 pixels. Then, according to Expression (1) or (2), the predicted pixel value pred （ｘ，ｙ） at the position (x, y) is derived. In this case, the coded pixel values in the image region 502 are used as t ｉ for positions i = 0 to 13. For example, t ０becomes the value of the encoded pixel adjacent to the pixel at the position (x, y) in FIG. 5(a). At the position (x, y) shown in FIG. 5(a), the predicted pixel value pred （ｘ，ｙ） After being derived, the prediction unit 103 shifts the position of the filter 503 by one pixel according to the scanning order indicated by the arrow in FIG. 5(a), and for the position (x, y) shown in FIG. 5(b), the predicted pixel value pred （ｘ，ｙ） is generated according to Equation (1) or (2). At this time, the encoded pixel value is used as t ｉ at positions i = 0, 2 to 13 in the filter 503 of FIG. 5(b), while the predicted pixel value pred （ｘ，ｙ） already derived according to Equation (1) or (2) in the arrangement of the filter 503 in FIG. 5(a) is used as t １ at position i = 1 of the filter 503 in FIG. 5(b). As described above, the prediction unit 103 sequentially obtains the predicted pixel value pred （ｘ，ｙ） [[ID=?]]by shifting the selected filter one pixel at a time according to the scanning order indicated by the arrow in FIG. 5(a), and generates the predicted image of the target block.

[0032] In this scanning method, at the first and last dashed arrows, it moves one pixel to the right.

[0033] And at each position indicated by each solid arrow, it moves diagonally one pixel at a time to the lower left. And as a result of one or more consecutive diagonal movements indicated by the solid arrow, when reaching the end of the target block, it jumps over a plurality of pixel positions and moves to the position immediately to the right or directly below the start position of the previous one or more consecutive diagonal movements, as indicated by the dashed arrow. The scanning method shown in FIGS. 5(a) and 5(b) is conveniently referred to as Down-left diagonal scan.

[0034] There seems to be an error in the line numbering in the original text where the line number jumps from 12 to 13 without an intervening line. I've translated it as best as possible with the assumption that the content is correct as presented. If there are any corrections or clarifications needed regarding the text, please let me know.Alternatively, a scanning method as shown in Figure 5(c) may be used as the scanning sequence. These are called Up-right diagonal scans, which were used in HEVC for scanning the quantization matrix. In this scanning method, the first and last dashed arrows move one pixel downwards. Then, at each position indicated by the solid arrows, the cursor moves diagonally one pixel to the upper right. As a result of one or more consecutive diagonal movements indicated by the solid arrows, when the cursor reaches the edge of the target block, it jumps over multiple pixel positions as indicated by the dashed arrows and moves to a position directly below or to the right of the starting position of the previous one or more consecutive diagonal movements.

[0035] As described above, the prediction unit 103 generates a predicted image by applying a filter while moving it. More specifically, the prediction unit 103 generates a predicted image by applying a filter while moving it to an area that includes at least a portion of the target block and the area surrounding the target block.

[0036] Note that the scanning order of the filter in the target block is not limited to the example shown in Figure 5(a), but may be the scanning order shown in Figures 6A and 6B. Figure 6A shows the order in which each row of pixels (hereinafter referred to as "pixel row") of an 8x8 target block is scanned horizontally. In this case, after the scan for predictive pixel generation is performed on the pixel row at the top of the 8x8 target block, the scan for predictive pixel generation may be performed on the pixel row immediately adjacent to that pixel row. That is, after the scan for predictive pixel generation is performed on a certain pixel row, the scan for predictive pixel generation may be performed on the pixel row immediately adjacent to that pixel row. In addition, predictive pixel generation is performed on the leftmost pixel in the pixel row at the top of the 8x8 target block. Subsequently, in parallel with the predictionive pixel generation being performed on the pixel adjacent to the right of that pixel according to the scanning order shown in Figure 6A, the predictionive pixel generation may be performed on the leftmost pixel of the pixel row adjacent to the pixel row below the top pixel row. In subsequent processing, as shown in Figure 6A, filter scanning for generating predicted pixels may be performed in parallel, shifting by one pixel for each pixel row of the target block. This makes it possible to perform parallel processing for each pixel row for generating predicted pixels.

[0037] Similarly, Figure 6B shows the order in which each pixel column (hereinafter referred to as "pixel column") of an 8x8 target block is scanned vertically. At this time, after the scan for predictive pixel generation is performed on the leftmost pixel column of the 8x8 target block, the scan for predictive pixel generation may be performed on the pixel column adjacent to the right of that pixel column. That is, after the scan for predictive pixel generation is performed on a certain pixel column, the scan for predictive pixel generation may be performed on the pixel column adjacent to the right of that pixel column. In addition, predictive pixel generation is performed on the top pixel of the leftmost pixel column of the 8x8 target block. Subsequently, in parallel with the predictionive pixel generation of the pixels adjacent below that pixel according to the scan order shown in Figure 6B, predictive pixel generation may be performed on the top pixel of the pixel column adjacent to the right of the leftmost pixel row. At this time, in the subsequent processing, as shown in Figure 6B, filter scans for predictive pixel generation may be performed in parallel, shifting by one pixel for each pixel column of the target block. In this way, it is possible to perform parallel processing for predictive pixel generation for each pixel row.

[0038] As mentioned above, the filter scan in the target block may be an Up-right diagonal scan as shown in Figure 5(c). This scan order is the same as the scan order used for encoding the quantization matrix and quantization coefficients of HEVC and VVC. By standardizing the scan order with the quantization matrix and quantization coefficients, it is possible to reduce the memory used for the scan table.

[0039] Furthermore, in the above explanation, the filter coefficient c used in the EIP prediction process is described. ｉ The explanation described how to obtain this using the reconstruction area surrounding the target block, but it is not limited to this. For example, in a particular mode (hereinafter referred to as EIP merge mode), the filter coefficient c used in other blocks other than the target block where the EIP prediction process was performed. ｉFilter information such as these may be used in the prediction process of the EIP of the target block. In this mode, by using information used in other blocks to determine the information to be used in the prediction process of the target block, that information can be encoded efficiently.

[0040] The EIP merge mode will be explained below with reference to Figures 7A to 7C. When EIP merging is used, the prediction unit 103 generates an EIP merge list (mergeEIPCanandList) for the target block 701 in the image currently being processed (current image). Specifically, as shown in Figure 7A, the EIP merge list is generated from spatially adjacent or nearby blocks (blocks containing positions A0 to A1 / blocks containing positions B0 to B2) to the target block 701. Alternatively, the EIP merge list (mergeEIPCanandList) is generated from positions in temporally different images that are spatially corresponding to the target block 701. Or, the EIP merge list (mergeEIPCanandList) is generated from blocks adjacent / nearby (blocks containing position C0 / blocks containing position C1) to a spatially corresponding position to the target block 701. Block 702 is a block included in a reference image that is temporally different from the current image containing the target block 701, and is located at the same position as the target block 701. In other words, block 702 is a block that is collated with the target block 701. As described above, the list described later may include information about blocks surrounding the target block (including adjacent blocks) as candidates for information to be used in the target block. Furthermore, the list described later may include information about blocks (including adjacent blocks) that are spatially corresponding to or surrounding the target block in temporally different images as candidates.

[0041] To generate the EIP merge list (mergeEIPCanandList), the prediction unit 103 determines whether a subblock containing the location of B1 is available for the target block 701. For example, if the subblock containing the location of B1 is currently included in the image, the prediction unit 103 determines that the subblock containing the location of B1 is available and sets availableFlagB1 to 1 (true). On the other hand, if the location of B1 is not currently included in the image, the prediction unit 103 determines that the subblock containing the location of B1 is unavailable and sets availableFlagB1 to 0 (false). Furthermore, the prediction unit 103 determines whether EIP is being used for the subblock containing the location of B1, and sets EIPFlag[B1] to 1 (true) if it is being used, and sets EIPFlag[B1] to 0 (false) if it is not being used. Then, as shown in Figure 7B, if the conditions are met that availableFlagB1 is 1 and EIPFlag[B1] is 1, the prediction unit 103 stores the information of candidate B1 in the EIP merge list (mergeEIPCandList). The information of candidate B1 includes information that identifies the filter used in the EIP processing in the subblock containing the position of B1, and the coefficient c of that filter. ｉ The information is linked.

[0042] The information stored in the list may be, for example, information that identifies a block called B1 (such as an index or the coordinates corresponding to B1).

[0043] Furthermore, the information stored in the list may also be information that identifies the filter used in the EIP processing of that subblock (in this case, B1). For example, the information stored in the list may be an index for identifying the filter.

[0044] Furthermore, the information stored in the list is the filter coefficient c of the filter used in the EIP processing of that subblock (B1 in this case). ｉ This could be information that identifies the filter coefficient c. For example, the specific coefficient c of the filter. ｉ It may be a value representing the filter coefficient c. ｉ It may also be an index that specifies a set of values.

[0045] Furthermore, the information stored in the list includes information identifying the filter used in the EIP processing of that subblock (in this case, B1), and the filter coefficient c of that filter. ｉ This could be either information identifying the subject or information identifying the subject. The same applies to the other candidate information described below.

[0046] Similarly, the prediction unit 103 identifies whether the subblock containing the location of A1 is available and sets the result of this identification as availableFlagA1. Furthermore, the prediction unit 103 identifies whether EIP is used for the subblock containing the location of A1 and sets the result of this identification as EIPFlag[A1]. If the conditions are met that availableFlagA1 is 1 and EIPFlag[A1] is 1, the prediction unit 103 stores the information of candidate A1 (hereinafter referred to simply as candidate A1 as appropriate) in the EIP merge list (mergeEIPCandList). Candidate A1 contains information identifying the filter used in the EIP processing in the subblock containing the location of A1 and the coefficient c of the said filter. ｉ The information is linked.

[0047] Next, the prediction unit 103 determines whether the subblock containing the location of B0 is available. Furthermore, it sets the determined result as availableFlagB0. Furthermore, it determines whether EIP is used for the subblock containing the location of B0 and sets the determined result as EIPFlag[B0]. If the conditions are met that availableFlagB0 is 1 and EIPFlag[B0] is 1, the prediction unit 103 stores the information of candidate B0 (hereinafter, as appropriate, simply referred to as candidate B0) in the EIP merge list (mergeEIPCandList). Candidate B0 contains information that identifies the filter used in the EIP processing in the subblock containing the location of B0 and the coefficient c of that filter. ｉ The information is linked.

[0048] Next, the prediction unit 103 determines whether the subblock at position A0 is available.

[0049] Furthermore, the identified result is set as availableFlagA0. Next, it is determined whether EIP is used in the subblock at the location of A0, and the identified result is set as EIPFlag[A0]. If the conditions are met (availableFlagA0 is 1 and EIPFlag[A0] is 1), the prediction unit 103 stores the information of candidate A0 (hereinafter, as may be simply referred to as candidate A0) in the EIP merge list (mergeEIPCandList). Candidate A0 contains information identifying the filter used in the EIP processing in the subblock containing the location of A0, and the coefficient c of that filter. ｉ The information is linked.

[0050] Next, the prediction unit 103 determines whether the subblock containing the location of B2 is available. Furthermore, it sets the determined result as availableFlagB2. It then determines whether EIP is used for the subblock at the location of B2 and sets the determined result as EIPFlag[B2]. If the conditions are met that availableFlagB2 is 1 and EIPFlag[B2] is 1, the prediction unit 103 stores the information of candidate B2 (hereinafter, as appropriate, simply referred to as candidate B2) in the EIP merge list (mergeEIPCandList). Candidate B2 contains information that identifies the filter used in the EIP processing in the subblock containing the location of B2 and the coefficient c of that filter. ｉ The information is linked.

[0051] The prediction unit 103 then determines whether the subblock containing the location of C0 (block 702 in the example of Figure 7A) is available. Furthermore, it sets the determined result as availableFlagCol. It then determines whether EIP is used for the subblock containing the location of C0 and sets the determined result as EIPFlag[Col]. If the conditions are met that availableFlagCol is 1 and EIPFlag[Col] is 1, it stores the candidate Col information (hereinafter referred to as candidate Col as appropriate). Specifically, the prediction unit 103 stores the candidate Col information (hereinafter referred to as candidate Col as appropriate) in the EIP merge list (mergeEIPCandList). The candidate Col is linked to the filter used in the EIP processing in the subblock containing the location of C0 and the coefficient information of that filter. Furthermore, if the subblock containing the location of C0 is unavailable, or if EIP is not used in the subblock containing the location of C0, the following processing may be performed. For the subblock containing the location of C1, it may be further determined whether the subblock is available and whether EIP is used in the subblock. In this case, the prediction unit 103 determines whether the subblock containing the location of C1 is available. Furthermore, it sets the result of this determination as availableFlagCol. Furthermore, it determines whether EIP is used in the subblock containing the location of C1 and sets the result of this determination as EIPFlag[Col]. If the conditions are met that availableFlagCol is 1 and EIPFlag[Col] is 1, the prediction unit 103 stores the candidate Col in the EIP merge list (mergeEIPCandList). The candidate Col at this time is information that identifies the filter used in the EIP processing in the subblock containing the position of C1, and the coefficient c of that filter. ｉ The information is linked.

[0052] The prediction unit 103 may determine whether availableFlagCol is usable based on the values ​​of other syntax elements. For example, it may use the value of a flag (temporal_mvp_enabled_flag) that indicates whether temporal motion vector prediction is possible in interpretation. This flag is signaled to a predetermined parameter set (e.g., SPS: Sequence Parameter Set). Other examples of predetermined parameter sets include PPS: Picture Parameter Set, PH: Picture Header, SH: Slice Header, etc. The predetermined parameter set is a syntax element, and in this embodiment, it is signaled to SPS. Temporal motion vector prediction refers to the following process: an interpretation process that generates a predicted image of a target block using the motion vector used in a predetermined interpretation for the target block. The motion vector used is the motion vector used in interpretation for a block located collated with the target block (for example, block 702 relative to target block 701). The prediction unit 103 sets availableFlagCol, shown in Figure 7B, to 1 if the flag indicates that temporal motion vector prediction is available in interpretation (temporal_mvp_enabled_flag = 1). On the other hand, the prediction unit 103 sets availableFlagCol, shown in Figure 7B, to 0 if the flag indicates that temporal motion vector prediction is unavailable in interpretation (temporal_mvp_enabled_flag = 0). In this case, in EIP merge mode, blocks located collated with the target block are unavailable as candidates for the EIP merge list.

[0053] In the above description, as shown in Figure 7B, the prediction unit 103 generates an EIP merge list for the target block 701 in the order B1→A1→B0→A0→B2→Col, but it is not limited to this. For example, the prediction unit 103 may generate an EIP merge list for the target block 701 in the order Col→B1→A1→B0→A0→B2.

[0054] Furthermore, after generating an EIP merge list as shown in Figures 7A and 7B, the prediction unit 103 may add additional candidates according to the current number of candidates and the maximum number of candidates in the EIP merge list. Specifically, the prediction unit 103 compares the current number of candidates in the EIP merge list, based on information from other blocks, with the maximum number of candidates. Then, according to the comparison result, the prediction unit 103 adds candidates other than those already included in the list.

[0055] For example, if the current number of candidates in the EIP merge list based on information from other blocks is less than the maximum number of candidates, new candidates (hereinafter referred to as "additional candidates") are added to the EIP merge list until the number of candidates in the EIP merge list reaches the maximum number of candidates. For example, if the number of candidates in the generated EIP merge list is 5 and the maximum number of candidates in the EIP merge list is 7, the prediction unit 103 adds two more candidates. The additional candidates added here are, for example, candidates associated with a predetermined filter (for example, the filter in Figure 3A) and a set of predetermined filter coefficients Ci for that predetermined filter. The predetermined filter coefficients Ci may be fixed values ​​where all values ​​are the same, or they may be predetermined coefficients generated in advance by machine learning. Also, if there are two or more candidates with the same filter among the candidates already stored in the EIP merge list, a new filter coefficient Ci is obtained by averaging the filter coefficients Ci of those two or more candidates.

[0056]

[0057] The set of filters is generated, and additional candidates associated with the set may be added to the EIP merge list. Specifically, if the same filter (for example, the filter in Figure 3A) is associated with the first candidate n and the second candidate m already stored in the EIP merge list, the prediction unit 103 uses equation (3) to determine the first candidate n

[0058]

[0059] and second candidate m

[0060]

[0061] and averaging to create a new

[0062]

[0063] Generates.

[0064]

[0065] The maximum number of candidates in the EIP merge list, MaxNumEIPCand, is signaled to the bitstream by the syntax element num_eip_merge_cand_minus2. num_eip_merge_cand_minus2 represents the value obtained by subtracting 2 from the maximum number of candidates in the EIP merge list, MaxNumEIPCand, and is signaled to a predetermined parameter set (SPS, PPS, PH, SH, etc.). Although num_eip_merge_cand_minus2 has been described as a syntax element representing the maximum number of candidates minus 2, it is not limited to this and may represent the maximum number of candidates minus other integer values ​​(1, 3, 4, 5, etc.).

[0066] As described above, the maximum number of candidates in this explanation functions as a mandatory target number. Even if the number of candidates based on information from other blocks falls short of the target number, generating additional candidates to create a list that always contains the same number (target number) of candidates makes the data easier to handle. Furthermore, always including the maximum number of candidates in the list increases the likelihood of selecting more appropriate information.

[0067] The prediction unit 103 generates an EIP merge list and then encodes the syntax element eip_merge_idx, which identifies the candidate to be used for the EIP of the target block from among the candidates in the generated EIP merge list, into a bitstream. eip_merge_idx is binarized by a predetermined binarization process and encoded using bypass coding or arithmetic coding. In this case, the first N bins (N is a positive integer) of the binarized sequence of eip_merge_idx may be encoded using arithmetic coding, and the rest may be encoded using bypass coding. For example, the first bin may be encoded using arithmetic coding, and the rest may be encoded using bypass coding. When arithmetic coding is used for one or more bins of the binarized sequence of eip_merge_idx, the context variables used for arithmetic coding depend on the following: The context variables depend, for example, on whether the subblock adjacent to the target block and the subblock adjacent to the left of the target block are using EIP.

[0068] Next, the encoding of the input image will be explained with reference to Figures 8A to 8F. The block division unit 102 divides the input image into multiple basic blocks (CTUs) and outputs each of the divided basic blocks. Furthermore, the block division unit 102 divides the basic blocks into multiple subblocks, and further divides the subblocks divided from the basic blocks into multiple subblocks. Figures 8A to 8F are diagrams showing the block division method by the block division unit 102.

[0069] Figure 8A shows the case where no further division is performed on a certain block 800 (basic block or subblock). Here, block 800 is described as having a size of 16 pixels × 16 pixels. Figure 8B shows a quadtree division, which is one method of subblock division. Figure 8B shows a division method (division pattern) in which block 800 is divided into four subblocks by dividing it in half horizontally and vertically. In this case, the size of each of the four subblocks is 8 pixels × 8 pixels. Figure 8C shows a division method in which block 800 is divided into two vertically elongated subblocks (each subblock having a size of 8 × 16 pixels) by dividing it in half vertically. Figure 8D shows a division method in which block 800 is divided into two horizontally elongated subblocks (each subblock having a size of 16 × 8 pixels) by dividing the basic block 700 in half horizontally. Figure 8E shows three horizontally elongated subblocks obtained by vertically dividing block 800 (from left to right: a subblock of 4x16 pixels, a subblock of 8x16 pixels, and a subblock of 4x16 pixels) (ternary tree partitioning).

[0070] Figure 8F shows three vertically elongated subblocks obtained by horizontally dividing block 800 (a subblock of 16 x 4 pixels, a subblock of 16 x 8 pixels, and a subblock of 16 x 4 pixels from top to bottom) (ternary tree partitioning). In this embodiment, any of the partitioning methods (partitioning patterns) shown in Figures 8B to 8F will be used, but it is not limited to Figures 8B to 8F. Note that, as shown in Figure 8A, block 800 may be used as a subblock without partitioning it.

[0071] Next, the encoding of syntax elements related to EIP will be explained with reference to Figures 9 to 10. Figure 9 is an example of the structure of a bitstream generated by the encoding unit 109 of the encoding device 100 in this embodiment. As shown in Figure 9, the bitstream includes an SPS (Sequence Parameter Set) containing parameters applied to the entire sequence. It also includes a PPS (Picture Parameter Set) containing parameters applied to one or more images, and a PH (Picture Header) containing parameters applied to one image. Furthermore, it includes Picture data containing the encoded data of the image. In addition, it may also include an SH (Slice Header) containing parameters applied to one slice.

[0072] Figure 10 shows the syntax structure included in the Picture data of the bitstream generated by the encoding unit 109, and is information that is signaled on a subblock (coding unit) basis. For example, the encoding unit 109 encodes the syntax elements shown in Figure 10 for the subblock (target block) currently being processed.

[0073] The eip_flag shown in Figure 10 is a flag indicating whether or not to execute EIP on the target block. An eip_flag value of 1 indicates that EIP will be executed on the target block, while an eip_flag value of 0 indicates that EIP will not be executed on the target block. Note that eip_flag is encoded into the coding_unit() syntax structure shown in Figure 10 when a predetermined condition, condition1, is true. condition1 may be, for example, a condition indicating whether a syntax element indicating that EIP is available is true or false, as shown below: condition1: eip_enabled_flag

[0074] The eip_enabled_flag is a syntax element that indicates that EIP is available and signals higher-level syntax structures in the bitstream than the coding_unit() syntax structure. For example, eip_enabled_flag is a flag that signals SPS, PPS, PH, SH, etc.

[0075] In this embodiment, eip_enabled_flag is a flag signaled to the SPS. A value of 1 for eip_enabled_flag indicates that EIP is available, condition1 becomes true, and eip_flag is encoded. On the other hand, a value of 0 for eip_enabled_flag indicates that EIP is unavailable, condition1 becomes false, eip_flag is not encoded, and the value of eip_flag is always 0. If condition1 is false, the value of eip_flag is estimated to be 0 by the decoding device (image decoding device) 200.

[0076] As another variation of condition1, condition1 may be a condition that further depends on the block size. In this case, for example, condition1 would be as follows: condition1 = eip_enabled_flag && Max(cbWidth, cbHeight) ≤ 32

[0077] In this modified example, condition1 is true if eip_enabled_flag is 1 and the larger of cbWidth and cbHeight is 32 or less. Otherwise, condition1 is false. Here, cbWidth is the width of the luminance component of the target block, and cbHeight is the height of the luminance component of the target block.

[0078] As another variation of condition1, condition1 may also be eip_enabled_flag&&Max(cbWidth, cbHeight)≦16.

[0079] Additionally, condition1 may be eip_enabled_flag&&Max(cbWidth, cbHeight)≦64, etc.

[0080] As another variation of condition1, condition1 may be a condition that depends on the color component of the current target block, as follows: condition1: cIdx = 0

[0081] cIdx is a variable that indicates the color component, where cIdx=0 corresponds to the luma component (Y component), cIdx=1 corresponds to the Cb chroma component (Cb component), and cIdx=2 corresponds to the Cr chroma component (Cr component). In the example of condition1 described above, if cIdx=0, the condition of condition1 is satisfied, and if not (if it is a chroma component), condition1 is not satisfied. In other words, if the target block currently under consideration is a luma component block, the condition of condition1 is satisfied, and if not (if it is a chroma component), condition1 is not satisfied. That is, in this modified example, eip_flag can only be encoded for the luma component.

[0082] Furthermore, as a variation of condition1, condition1 may be a condition that depends on the current position of the target block. If, for example, the filter shown in Figures 3A to 3C does not fit within the picture at the target block position, EIP may be disabled. In other words, if the filter shown in Figures 3A to 3C does not fit within the picture, condition1 may be set to forcibly disable EIP.

[0083] If eip_flag is 1 (true), the encoding unit 109 encodes eip_merge_flag into coding_unit(). eip_merge_flag is a flag indicating whether to use the EIP merge mode for the target block. If eip_merge_flag is 1, the EIP merge mode is used for the target block, while if eip_merge_flag is 0, the EIP merge mode is not used for the target block. If eip_flag is 1 (true), the encoding unit 109 obtains the value of MaxNumEIPCand, which is the maximum number in the EIP merge list, and if this value is greater than 1, it further encodes eip_merge_idx into coding_unit(). The encoding unit 109 encodes num_eip_merge_cand_minus2, which is a syntax element for identifying the value of MaxNumEIPCand, into a predetermined parameter set. MaxNumEIPCand is identified by adding 2 to num_eip_merge_cand_minus2. In this embodiment, num_eip_merge_cand_minus2 is encoded into SPS. The maximum number of EIP merge lists and the maximum number of merge candidates for inter prediction may be the same. In this case, encoding of num_eip_merge_cand_minus2 may be omitted. Alternatively, the difference between the maximum number of EIP merge lists and the maximum number of merge candidates for inter prediction may be encoded. By doing so, it becomes possible to encode with a shorter code size than num_eip_merge_cand_minus2.

[0084] If eip_merge_flag is 0 (false) and condition2 is true, the encoding unit 109 encodes eip_filter_idx and eip_reconst_idx into the coding_unit() syntax structure. eip_filter_idx is a syntax element that identifies the filter used for EIP for the target block from among the multiple filters shown in Figures 3A to 3C. For example, eip_filter_idx=0 indicates the filter in Figure 3A, eip_filter_idx=1 indicates the filter in Figure 3B, and eip_filter_idx=2 indicates the filter in Figure 3C. eip_reconst_idx is a syntax element for identifying the reconstruction region used to generate filter coefficients in EIP for the target block from among the multiple reconstruction regions shown in Figures 4A to 4C. For example, eip_reconst_idx=0 indicates the reconstruction region in Figure 4A, eip_reconst_idx=1 indicates the reconstruction region in Figure 4B, and eip_reconst_idx=2 indicates the reconstruction region in Figure 4C. Note that condition2 may also be a condition that depends on the size of the target block, for example, as follows: condition2: cbwidth==4&&cbheight==4

[0085] Furthermore, condition2 is not limited to this; for example, cbwidth == 4 || cbheight == 4, min(cbwidth, cbheight) ≤ 4, etc. Also, condition2 may be a condition that depends on the position of the target block.

[0086] In this embodiment, if eip_flag is 1 and eip_merge_flag is 0 and condition2 is true, the following encoding process is performed: eip_filter_idx and eip_reconst_idx are encoded into codent_unit(). On the other hand, even if eip_flag is 1 and eip_merge_flag is 0, if condition2 is false, eip_filter_idx and eip_reconst_idx are not encoded. Furthermore, the values ​​of eip_filter_idx and eip_reconst_idx are always 0 (i.e., constrained to the value of 0). In other words, if the above condition2 is not met for the target block, the following process is performed. As part of the EIP processing for the target block, a filter corresponding to eip_filter_idx=0 and a reconstruction area corresponding to eip_reconst_idx=0 may be uniformly used. In this case, eip_filter_idx and eip_reconst_idx are not encoded and are estimated to be 0 by the decoding device, thus reducing the amount of code in the bitstream. In this embodiment, eip_filter_idx=0 corresponds to the filter in Figure 3A, and eip_reconst_idx=0 corresponds to the reconstruction area in Figure 4A. However, it is not limited to this, and eip_filter_idx=0 may correspond to the filter in Figure 3B or Figure 3C, and eip_reconst_idx=0 may correspond to the reconstruction area in Figure 4B or Figure 4C.

[0087] In this embodiment, eip_filter_idx=0 corresponds to the filter in Figure 3A. This is because the filter in Figure 3A is square. Compared to the rectangles in Figures 3B and 3C, the square filter is easier to use due to factors such as the position of the target block. Also, as an example, square filters tend to be used more frequently.

[0088] In this embodiment, eip_reconst_idx is described as being encoded, but this is not limited to this, and eip_reconst_idx does not need to be encoded. In this case, when the prediction unit 103 selects one of the filters shown in Figures 3A to 3C, it determines the reconstruction region to be used according to the selected filter. For example, if the prediction unit 103 selects the filter in Figure 3A as the filter to be used, it uniformly selects the reconstruction region in Figure 4A as the reconstruction region to be used. Similarly, if the prediction unit 103 selects the filter in Figure 3B as the filter to be used, it uniformly selects the reconstruction region in Figure 4B as the reconstruction region to be used. Similarly, if the prediction unit 103 selects the filter in Figure 3C as the filter to be used, it uniformly selects the reconstruction region in Figure 4C as the reconstruction region to be used. In this case, there is no need to encode eip_reconst_idx, and the prediction unit 103 encodes eip_filter_idx. In other words, eip_filter_idx allows you to select both the filter to use and the reconstruction region.

[0089] In this embodiment, the shapes of the filter and reconstruction region specified by the syntax elements are assumed to be the same. For example, the filter shape in Figure 3B and the reconstruction region shape in Figure 4B are the same, and when the filter in Figure 3B is used, the reconstruction region in Figure 4B is used. In other words, a horizontally elongated rectangular filter corresponds to a horizontally elongated rectangular reconstruction region. When the square filter in Figure 3A is used, the L-shaped reconstruction region in Figure 4A is used. In the case of the L-shaped reconstruction region, the shapes are not exactly the same, but the square filter and the L-shaped reconstruction region are associated. In other words, in the case of a square filter, the L-shaped reconstruction region in Figure 4A is used. Here, the L-shaped reconstruction region includes a first part located on the left side of the target block and a second part located on the upper side of the target block. It is sufficient to include at least this first part and second part, but the L-shaped reconstruction region may also include a part located in the upper left of the target block, as shown in Figure 4A. In the example shown in Figure 4A, it consists of a rectangular part made up of a first part and a third part, and a second part orthogonal to that rectangular part. When using the vertically elongated rectangular filter in Figure 3C, use the vertically elongated rectangular reconstruction region in Figure 4C.

[0090] In other words, the shape of the filter and the shape of the reconstruction region are the same and are used together. In this embodiment, by specifying the reconstruction region according to the filter to be used, it becomes possible to use a reconstruction region suitable for the filter when the filter scans the reconstruction region and the target block. The reconstruction region may be adjacent to the target block or may be far from the target block. For example, the reconstruction region may be about 1 to 8 pixels away from the target block. Now, the image encoding process by the encoding device 100 will be further explained according to the flowchart in Figure 11. First, in step S1101, the encoding unit 109 encodes the header information (SPS, PPH, PH, SH, etc.) necessary for encoding the image. For example, it encodes num_eip_merge_cand_minus2, temporary_mvp_enabled_flag, eip_enabled_flag, etc. In this embodiment, num_eip_merge_cand_minus2, temporary_mvp_enabled_flag, and eip_enabled_flag are encoded in SPS. However, they are not limited to this and may be encoded in other parameter sets such as PPS, PH, SH, etc. Also, when eip_enabled_flag is 0 (false), it is possible to reduce the amount of encoding by omitting the encoding of num_eip_merge_cand_minus2. Next, in step S1102, the encoding unit 109, the block division unit 102 divides the input image into a plurality of basic blocks.

[0091] In step S1103, the prediction unit 103 selects one unselected basic block from the multiple basic blocks obtained in step S1102, according to the raster scan order starting from the top left edge of the image. The prediction unit 103 then determines a subblock division method (division pattern) for the selected basic block and divides the selected basic block into multiple subblocks according to the determined subblock division method. At this time, the prediction unit 103 divides the target basic block into multiple subblocks according to the division methods shown in Figures 8A to 8F, and if necessary, further divides the resulting subblocks according to the division methods shown in Figures 8A to 8F. In this way, the prediction unit 103 recursively divides the basic block into multiple subblocks. The prediction unit 103 also determines the prediction processing mode for each subblock.

[0092] Prediction processes used in this process include intra-prediction, inter-prediction, and EIP.

[0093] The prediction unit 103 then generates a predicted image for each subblock of the currently selected basic block by using the image in the frame memory 107 and making a prediction according to the determined prediction mode, and calculates the difference between the input image and the predicted image as the prediction error. When EIP is used for a certain target block, the prediction unit 103 generates a predicted image for that target block according to the method described above with reference to Figures 3A to 3C, 4A to 4C, 5, 6A, 6B, and 7A to 7C. The prediction unit 103 also generates prediction information, such as information indicating the subblock division method and the prediction processing mode. When EIP is used, this prediction information includes, for example, multiple syntax elements as shown in Figure 10.

[0094] In step S1104, the transformation / quantization unit 104 generates transformation coefficients for each subblock by performing a transformation process on the prediction error for each subblock. Then, for each subblock, the transformation / quantization unit 104 quantizes the transformation coefficient of the subblock to generate the quantization coefficient of the subblock. In step S1105, the inverse quantization / inverse transformation unit 105 generates transformation coefficients by inverse quantizing the quantization coefficient of each subblock generated in step S306 using the quantization matrix used for the quantization of the subblock. Then, the inverse quantization / inverse transformation unit 105 performs an inverse transformation process on the generated transformation coefficients to generate the prediction error.

[0095] In step S1106, the image reconstruction unit 106 generates a predicted image from the image currently being processed or from a processed image stored in the frame memory 107, based on the predicted information generated by the prediction unit 103. The image reconstruction unit 106 then reconstructs the image from the predicted image and the prediction error generated by the inverse quantization / inverse transform unit 105. The image reconstruction unit 106 then stores the reconstructed image in the frame memory 107. In step S1107, the encoding unit 109 entropy encodes the quantization coefficients generated by the transform / quantization unit 104 and the predicted information generated by the prediction unit 103 to generate encoded data.

[0096] Furthermore, the encoding unit 109 multiplexes the header information and the generated encoded data to generate a bitstream. In step S1108, the control unit 101 determines whether all basic blocks have been encoded. If all basic blocks have been encoded, the process proceeds to step S1109. If there are still basic blocks that have not been encoded, the process proceeds to step S1103, where the next basic block to be processed is selected. In step S1109, the in-loop filter unit 108 performs in-loop filtering, such as deblocking filtering and sample adaptive offsetting, on the image stored in the frame memory 107.

[0097] As described above, in this embodiment, it is possible to encode the target block using EIP processing. In the above description, three patterns of EIP filter types were described as shown in Figures 3A to 3C, but this is not limited to these. Below, a modified example of this embodiment will be described in which other filter types can be used.

[0098] Figure 12 shows the types of filters used in the modified examples. The types of filters are divided into three size groups. Size group 0 includes, as in Figures 3A to 3C, a 4-pixel x 4-pixel filter shown in Figure 12(a), an 8-pixel x 2-pixel filter shown in Figure 12(b), and a 2-pixel x 8-pixel filter shown in Figure 12(c). Size group 1 includes a 3-pixel x 3-pixel filter shown in Figure 12(d). Furthermore, it includes a 4-pixel x 2-pixel filter shown in Figure 12(e) and a 2-pixel x 4-pixel filter shown in Figure 12(f). Furthermore, it includes an 8-pixel x 1-pixel filter shown in Figure 12(g) and a 1-pixel x 8-pixel filter shown in Figure 12(h). Furthermore, size group 2 includes a 2-pixel x 2-pixel filter shown in Figure 12(i), a 4-pixel x 1-pixel filter shown in Figure 12(j), and a 1-pixel x 4-pixel filter shown in Figure 12(k). Although the correspondence between each pixel position and the value of i is shown in Figure 12 for each filter, the correspondence between each pixel position and the value of i in each filter is not limited to the examples shown in Figure 12; other correspondences may also be possible.

[0099] When using a filter for size group 0, the prediction unit 103 generates predicted pixels for the target block using the above-described formula (1) or (2).

[0100] When using the filter shown in Figure 12(d) which belongs to size group 1, the prediction unit 103 generates predicted pixels for the target block using the following equation (4).

[0101]

[0102] As shown in equation (4), size group 1 is smaller than a predetermined size (size group 0). Therefore, unlike in equation (1), the prediction unit 103 creates a predicted image using the reconstructed pixel value or predicted pixel value corresponding to i=7, which is the position furthest from the predicted pixel position (lower right corner). In this way, depending on the size of the target block or filter, the prediction unit 103 may decide whether to use one or more pixels in the region that includes the pixel furthest from the predicted pixel position.

[0103] However, even in size group 1, pixels located far from the predicted pixel position may be excluded, as shown in equation (5) below.

[0104]

[0105] Depending on the characteristics of the image, the encoding device may decide which formula to use to predict the image and signal it to the bitstream using an index or similar method.

[0106] Furthermore, when using the filters shown in Figures 12(e) to (h) that belong to size group 1, the prediction unit 103 generates predicted pixels for the target block using the following equation (6).

[0107]

[0108] As shown in equation (6), size group 1 is smaller than a predetermined size (size group 0). Therefore, unlike in equation (1), the prediction unit 103 creates a predicted image using the reconstructed pixel value or predicted pixel value corresponding to i=7, which is the position furthest from the predicted pixel position (lower right corner). In this way, depending on the size of the target block or filter, the prediction unit 103 may decide whether to use one or more pixels in the region that includes the pixel furthest from the predicted pixel position.

[0109] However, even in size group 1, pixels located far from the predicted pixel position may be excluded, as shown in equation (7) below.

[0110]

[0111] Depending on the characteristics of the image, the encoding device may decide which formula to use to predict the image and signal it to the bitstream using an index or similar method.

[0112] As described above, within the same size group 1, filters with different pixel counts are mixed, as shown in Figure 12(d) and Figures (e) to (h). By grouping these together in this way, the amount of code used when signaling the group can be suppressed.

[0113] Furthermore, when using the filters shown in Figures 12(i) to (k) that belong to size group 2, the prediction unit 103 generates predicted pixels for the target block using the following equation (8).

[0114]

[0115] As shown in equation (8), size group 2 is smaller than the predetermined size (size group 0). Therefore, unlike in equation (1), the prediction unit 103 creates the predicted image using the reconstructed pixel value or predicted pixel value corresponding to i=2, which is the position furthest from the predicted pixel position (lower right corner). In this way, depending on the size of the target block or filter, the prediction unit 103 may decide whether to use one or more pixels in the region that includes the pixel furthest from the predicted pixel position.

[0116] However, even in size group 2, pixels located far from the predicted pixel position may not be used, as shown in equation (9) below.

[0117]

[0118] Depending on the characteristics of the image, the encoding device may decide which formula to use to predict the image and signal it to the bitstream using an index or similar method.

[0119] As described above, the prediction unit 103 calculates the filter coefficient c for the selected filter. ｉ When generating the filter coefficient c, use one of the reconstruction regions shown in Figures 4A to 4C. ｉ This will generate [something].

[0120] Here, we will explain the coding_unit() syntax structure in a modified example that allows the use of the filter shown in Figure 12. Figure 13 is an example of the coding_unit() syntax structure in a modified example that allows the use of the filter shown in Figure 12. Note that the same parts as in Figure 10 will not be explained. The difference from the coding_unit() syntax structure in Figure 10 is that the coding_unit() syntax structure in Figure 13 newly includes eip_filter_size_idx. eip_filter_size_idx is a syntax element that identifies the size group of filters to be used for the target block.

[0121]

[0122] As shown in Table 1, eip_filter_size_idx = 0 indicates that size group 0 in Figure 12 is used. In this case, the range of possible values ​​for eip_filter_idx is 0 or greater and 2 or less. A value of 0 for eip_filter_idx represents the filter in Figure 12(a). A value of 1 for eip_filter_idx represents the filter in Figure 12(b). A value of 2 for eip_filter_idx represents the filter in Figure 12(c). Similarly, eip_filter_size_idx = 1 indicates that size group 1 in Figure 12 is used. In this case, the range of possible values ​​for eip_filter_idx is 0 or greater and 4 or less. A value of 0 for eip_filter_idx represents the filter in Figure 12(d). A value of 1 for eip_filter_idx indicates the filter in Figure 12(e). A value of 2 for eip_filter_idx indicates the filter in Figure 12(f). A value of 3 for eip_filter_idx indicates the filter in Figure 12(g). A value of 2 for eip_filter_idx indicates the filter in Figure 12(h). Furthermore, eip_filter_size_idx = 2 indicates that size group 2 in Figure 12 is used. In this case, the range of possible values ​​for eip_filter_idx is 0 or greater and 2 or less. A value of 0 for eip_filter_idx indicates the filter in Figure 12(i). A value of 1 for eip_filter_idx indicates the filter in Figure 12(j). The value of eip_filter_idx for 2 represents the filter shown in Figure 12(k).

[0123] In Figure 12, if eip=filter_size=idx, eip_filter=idx, and eip_reconst=idx are not encoded, each syntax element is constrained to a predetermined value. Here, the cases where they are not encoded include cases where condition2 is false, and the value of each syntax element is always 0 (constrained to the value of 0). When eip_flag is 1 and eip_merge_flag is 0 and condition2 is false, each syntax element takes the following value. In other words, in the above case, eip_filter_size_idx, eip_filter_idx, and eip_reconst_idx are each always 0. As a result, the filter in Figure 12(a) and the reconstruction region in Figure 4A are used.

[0124] By enabling the use of size groups 1 and 2 as second filters, in addition to size group 0, which has 16 sizes such as 4x4 as the first filter, it is possible to generate predictive images more flexibly and reduce the amount of data.

[0125] (Second Embodiment) In this embodiment, a decoding device 200 that decodes a bitstream obtained by encoding an image using the encoding device 100 according to the first embodiment will be described. The differences from the first embodiment will be described below, and unless otherwise specified, it will be the same as the first embodiment. An example of the functional configuration of the decoding device 200 according to this embodiment will be described using the block diagram in Figure 2.

[0126] The control unit 201 controls the overall operation of the decoding device 200. The separation and decoding unit 202 acquires the bitstream generated by the encoding device 100, separates encoded data related to the decoding process and coefficients from the bitstream, and decodes the encoded data present in the bitstream header. For example, the separation and decoding unit 202 decodes num_eip_merge_cand_minus2, temporary_mvp_enabled_flag, and eip_enabled_flag from the SPS.

[0127] The decoding unit 203 obtains quantization coefficients and prediction information by decoding the encoded data separated from the bitstream by the separation decoding unit 202. The inverse quantization / inverse transformation unit 204 operates in the same way as the inverse quantization / inverse transformation unit 105 of the encoding device 100.

[0128] The inverse quantization / inverse transformation unit 204 obtains transformation coefficients by performing inverse quantization on the acquired quantization coefficients, and obtains the prediction error by performing an inverse transformation on these transformation coefficients.

[0129] The image reconstruction unit 205 performs prediction processing by referring to the image stored in the frame memory 206 based on the prediction information decoded from the bitstream by the decoding unit 203, and generates a predicted image. Then, the image reconstruction unit 205 reconstructs the image using the generated predicted image and the prediction error obtained by the inverse quantization / inverse transform unit 204, and stores the reconstructed image in the frame memory 206.

[0130] The in-loop filter unit 207 performs in-loop filtering, such as deblocking filtering, on the reconstructed image stored in the frame memory 206. The image stored in the frame memory 206 is output as appropriate by the control unit 201. The output destination of the image is not limited to a specific destination; for example, the reconstructed image may be displayed on the display screen of a display device such as a monitor, or the image may be output to a projection device such as a projector.

[0131] Next, the operation of the decoding device 200 having the above configuration (bitstream decoding process) will be described. In this embodiment, the bitstream input to the separation decoding unit 202 is assumed to be a bitstream for each frame of a moving image, but it may also be a bitstream of a still image.

[0132] Here, the prediction process performed by the image reconstruction unit 205 of the decoding device 200 will be described in more detail. The image reconstruction unit 205 is capable of performing the same prediction process as the prediction unit 103.

[0133] For example, when the image reconstruction unit 205 decodes eip_filter_idx and eip_reconst_idx from the coding_unit() syntax structure in Figure 10 for the current target block, it performs the following processing. That is, the image reconstruction unit 205 selects one of the filters in Figures 3A to 3C as the filter to be used in EIP for the target block, according to the value of eip_filter_idx. Furthermore, the image reconstruction unit 205 selects one of the reconstruction regions shown in Figures 4A to 4C for the target block, according to the decoded value of eip_reconst_idx, and generates the filter coefficients of the selected filter using the selected reconstruction region. Then, the image reconstruction unit 205 generates predicted pixels for the target block using one of the formulas described above.

[0134] Here, we will explain an example where the size of the target block is 4 pixels × 4 pixels, and a 4 pixel × 4 pixel filter as shown in Figure 3A is selected. In this case, the image reconstruction unit 205 positions the 4 pixel × 4 pixel filter 503 so that its position (x, y) is at the top left (0, 0) of the 4 pixel × 4 pixel target block 501, similar to the process described in the first embodiment. Then, according to one of the formulas described above, the predicted pixel value at position (x, y) is pre-determined. （ｘ，ｙ） The following is derived. At the position (x, y) shown in Figure 5(a), the predicted pixel value pred （ｘ，ｙ） After the formula is derived, the image reconstruction unit 205 shifts the position of the filter 503 by one pixel according to the scanning order shown by the arrow in Figure 5(a), and predicts the pixel value pred for the position (x, y) shown in Figure 5(b) according to one of the formulas described above. （ｘ，ｙ） This generates the following. As described above, the image reconstruction unit 205 shifts the selected filter one pixel at a time according to the scanning order shown by the arrows in Figure 5(a), and generates the predicted pixel values. （ｘ，ｙ） The data is acquired sequentially, and a predicted image of the target block is generated.

[0135] Note that the scanning order of the filter in the target block is not limited to the example shown in Figure 5(a), but may be the scanning order shown in Figures 6A and 6B. Figure 6A shows the order in which each row of pixels (hereinafter referred to as "pixel row") of an 8x8 target block is scanned horizontally. In this case, after the scan for predictive pixel generation is performed on the pixel row at the top of the 8x8 target block, the scan for predictive pixel generation may be performed on the pixel row immediately adjacent to that pixel row. That is, after the scan for predictive pixel generation is performed on a certain pixel row, the scan for predictive pixel generation may be performed on the pixel row immediately adjacent to that pixel row. In addition, predictive pixel generation is performed on the leftmost pixel in the pixel row at the top of the 8x8 target block. Subsequently, in parallel with the predictionive pixel generation being performed on the pixel adjacent to the right of that pixel according to the scanning order shown in Figure 6A, the predictionive pixel generation may be performed on the leftmost pixel of the pixel row adjacent to the pixel row below the top pixel row. In subsequent processing, as shown in Figure 6A, filter scanning for generating predicted pixels may be performed in parallel, shifting by one pixel for each pixel row of the target block. This makes it possible to perform parallel processing for each pixel row for generating predicted pixels.

[0136] Similarly, Figure 6B shows the order in which each pixel column (hereinafter referred to as "pixel column") of an 8x8 target block is scanned vertically. At this time, after the scan for predictive pixel generation is performed on the leftmost pixel column of the 8x8 target block, the scan for predictive pixel generation may be performed on the pixel column adjacent to the right of that pixel column. That is, after the scan for predictive pixel generation is performed on a certain pixel column, the scan for predictive pixel generation may be performed on the pixel column adjacent to the right of that pixel column. In addition, predictive pixel generation is performed on the top pixel of the leftmost pixel column of the 8x8 target block. Subsequently, in parallel with the predictionive pixel generation of the pixels adjacent below that pixel according to the scan order shown in Figure 6B, predictive pixel generation may be performed on the top pixel of the pixel column adjacent to the right of the leftmost pixel row. At this time, in the subsequent processing, as shown in Figure 6B, filter scans for predictive pixel generation may be performed in parallel, shifting by one pixel for each pixel column of the target block. In this way, it is possible to perform parallel processing for predictive pixel generation for each pixel row.

[0137] Furthermore, the scan order of the filters in the target block may be as shown in Figure 5(c). This scan order is the same as the scan order used for encoding the quantization matrix and quantization coefficients of HEVC and VVC, but by standardizing the scan order, it is possible to reduce the memory used for the table used for scanning.

[0138] Furthermore, in the above explanation, the filter coefficient c used in the EIP prediction process is described. ｉ The explanation described obtaining this using the reconstruction area surrounding the target block, but it is not limited to this. For example, when using EIP merge mode, the filter coefficient c used in blocks where prediction processing of EIPs other than the target block was performed. ｉ This may be used in the EIP prediction process for the target block.

[0139] When the image reconstruction unit 205 decodes an eip_merge_flag with a value of 1 for the target block, it decides to use EIP merge mode for that target block.

[0140] In this case, the image reconstruction unit 205 generates an EIP merge list (mergeEIPCandList) in the same manner as the prediction unit 103 in EIP merge mode. Specifically, it generates an EIP merge list from spatially adjacent or nearby blocks (blocks containing positions A0 to A1 / blocks containing positions B0 to B2) for the target block 701 in the current image. It also generates an EIP merge list for blocks in images at different times that are spatially corresponding to the target block 701.

[0141] Alternatively, an EIP merge list is generated for the target block 701 from adjacent / nearby blocks (blocks containing the position of C0 / blocks containing the position of C1). At this time, the prediction unit 103 of the encoding device 100 and the image reconstruction unit 205 of the decoding device 200 each generate the same EIP merge list for the target block. Note that the process of generating the EIP merge list by the image reconstruction unit 205 is the same as the process of generating the EIP merge list by the prediction unit 103 described in Embodiment 1, so the explanation is omitted. As mentioned above, the EIP merge list can include information of blocks surrounding the target block (including adjacent blocks) as candidates for information to be used in the target block. The EIP merge list can also include information of blocks (including adjacent blocks) at spatially corresponding positions to the target block or surrounding blocks in images that are different in time, as candidates.

[0142] The image reconstruction unit 205 then adds 2 to num_eip_merge_cand_minus2, which is decoded from the bitstream, to derive the maximum number of candidates in the EIP merge list, MaxNumEIPCand. After generating an EIP merge list as shown in Figures 7A and 7B, the image reconstruction unit 205 may further add candidates according to the current number of candidates and the maximum number of candidates in the EIP merge list. Specifically, the image reconstruction unit 205 compares the current number of candidates in the EIP merge list, based on information from other blocks, with the maximum number of candidates. Then, according to the comparison result, the image reconstruction unit 205 adds candidates other than those already included in the list. For example, if the current number of candidates in the EIP merge list is less than the maximum number of candidates, additional candidates may be added to the EIP merge list until the number of candidates in the EIP merge list reaches the maximum number of candidates. For example, if the number of candidates in the generated EIP merge list is 5 and the maximum number of candidates in the EIP merge list is 7, the image reconstruction unit 205 adds two more candidates. The additional candidates added here are, for example, candidates associated with a predetermined filter (for example, the filter in Figure 3A) and a set of predetermined filter coefficients Ci for that predetermined filter. The predetermined filter coefficients Ci may be fixed values ​​where all values ​​are the same, or they may be predetermined coefficients generated in advance by machine learning. Also, if there are two or more candidates with the same filter among the candidates already stored in the EIP merge list, a new filter coefficient Ci is obtained by averaging the filter coefficients Ci of those two or more candidates.

[0143]

[0144] The set of filters is generated, and additional candidates associated with the set may be added to the EIP merge list. Specifically, if the same filter (for example, the filter in Figure 3A) is associated with the first candidate n and the second candidate m already stored in the EIP merge list, the prediction unit 103 uses equation (7) to determine the first candidate n

[0145]

[0146] and second candidate m

[0147]

[0148] and averaging to create a new

[0149]

[0150] This generates the following. Equation (10) below is the same as equation (3) above.

[0151]

[0152] The decoding unit 203 decodes eip_merge_idx for the target block from the coding_unit() syntax structure of the bitstream. The image reconstruction unit 205 generates an EIP merge list and then identifies candidates to be used for the EIP of the target block from among the candidates in the generated EIP merge list, according to the value of eip_merge_idx. eip_merge_idx is binarized by a predetermined binarization process and encoded using bypass decoding or arithmetic decoding. In this case, arithmetic decoding may be used for the first N bins (N is a positive integer) of the binarized sequence of eip_merge_idx, and the rest may be bypass decoding. For example, arithmetic decoding may be used for the first bin and bypass decoding for the rest. When arithmetic decoding is used for one or more bins of the binarized sequence of eip_merge_idx, the context variables used depend on the following. The context variables used in arithmetic decoding depend, for example, on whether the subblocks adjacent to the target block and the subblock adjacent to the left of the target block are using EIP.

[0153] In this embodiment, the decoding unit 203 decodes information from the bitstream indicating how the basic blocks of the current image to be processed are divided. It also identifies the positions and sizes of multiple subblocks within each basic block of the current image according to this information. Furthermore, it decodes the coding_unit() syntax structure contained in the Picture data of the bitstream in Figure 9 to obtain prediction information for the target block.

[0154] The decoding unit 203 decodes eip_flag if condition1 is true. Note that condition1 is the same as described in the first embodiment, so its explanation is omitted. If condition1 is false, the decoding unit 203 estimates that the value of eip_flag for the target block is 0. In other words, the decoding unit 203 decides not to perform EIP on the target block.

[0155] If the decoded eip_flag is 1 (true), the decoder 203 decodes eip_merge_flag from coding_unit(). If the decoded eip_merge_flag is 1, the decoder 203 decides to use the EIP merge mode for the target block. On the other hand, if eip_merge_flag is 0, the decoder 203 decides not to use the EIP merge mode for the target block. If eip_flag is 1 (true), the decoder 203 obtains the value of MaxNumEIPCand, which is the maximum number in the EIP merge list, and if this value is greater than 1, it further decodes eip_merge_idx into coding_unit(). If MaxNumEIPCand is 1, eip_merge_idx is not decoded, and the value of eip_merge_idx is assumed to be 0. The decoding unit 203 is identified by adding 2 to num_eip_merge_cand_minus2 decoded from SPS.

[0156] If eip_merge_flag is 0 (false) and condition2 is true, the decoding unit 203 decodes eip_filter_idx and eip_reconst_idx from the coding_unit() syntax structure. In the embodiment of Figure 10, eip_filter_idx is a syntax element that identifies the filter to be used for the EIP for the target block from among the multiple filters shown in Figures 3A to 3C. For example, eip_filter_idx=0 indicates the filter in Figure 3A, eip_filter_idx=1 indicates the filter in Figure 3B, and eip_filter_idx=2 indicates the filter in Figure 3C. eip_reconst_idx is a syntax element used to identify the reconstruction region used to generate filter coefficients in the EIP for the target block from among the multiple reconstruction regions shown in Figures 4A to 4C. For example, eip_reconst_idx=0 indicates the reconstruction region in Figure 4A, eip_reconst_idx=1 indicates the reconstruction region in Figure 4B, and eip_reconst_idx=2 indicates the reconstruction region in Figure 4C. Note that condition2 is the same as described in the first embodiment, so its explanation is omitted.

[0157] In this embodiment, if eip_flag is 1 and eip_merge_flag is 0 and condition2 is true, the following decoding process is performed. The decoding unit 203 decodes eip_filter_idx and eip_reconst_idx from the codent_unit() syntax structure. On the other hand, even if eip_flag is 1 and eip_merge_flag is 0, if condition2 is false, the following decoding process is performed. In this case, eip_filter_idx and eip_reconst_idx are not decoded from the codent_unit() syntax structure. Furthermore, the decoding unit 203 estimates that the values ​​of eip_filter_idx and eip_reconst_idx are both 0 if the syntax structure described above cannot be decoded.

[0158] In other words, if the target block does not satisfy the conditions of condition 2 described above, the EIP processing for that target block may uniformly use a filter corresponding to eip_filter_idx=0 and a reconstruction area corresponding to eip_reconst_idx=0. In this embodiment, eip_filter_idx=0 corresponds to the filter in Figure 3A, and eip_reconst_idx=0 corresponds to the reconstruction area in Figure 4A. However, it is not limited to this, and eip_filter_idx=0 may correspond to the filter in Figure 3B or Figure 3C, and eip_reconst_idx=0 may correspond to the reconstruction area in Figure 4B or Figure 4C.

[0159] In this embodiment, eip_reconst_idx is described as being included in the syntax structure of Figure 10, but this is not limited to this, and eip_reconst_idx does not have to be encoded at all (i.e., it does not have to be decoded). In this case, the decoding unit 103 selects one of the filters shown in Figures 3A to 3C according to the value of eip_filter_idx, and then determines the reconstruction region according to the selected filter. For example, if the decoding unit 203 selects the filter in Figure 3A according to eip_filter_idx, it uniformly selects the reconstruction region in Figure 4A as the reconstruction region to be used. Similarly, if the decoding unit 203 selects the filter in Figure 3B according to eip_filter_idx, it uniformly selects the reconstruction region in Figure 4B as the reconstruction region to be used. Similarly, if the decoding unit 203 selects the filter in Figure 3C according to eip_filter_idx, it uniformly selects the reconstruction region in Figure 4C as the reconstruction region to be used. In such cases, there is no need to decrypt eip_reconst_idx, and the decryption unit 203 decrypts eip_filter_idx instead.

[0160] In other words, eip_filter_idx allows you to select both the filter to use and the reconstruction region.

[0161] Here, the decoding process of a single frame of bitstream by the decoding device 200 will be further explained according to the flowchart in Figure 14. First, in step S1401, the separation and decoding unit 202 acquires a single frame of bitstream generated by the encoding device 100. The separation and decoding unit 202 then separates encoded data related to the decoding process and coefficients from the bitstream and decodes the encoded data present in the bitstream header. Here, the separation and decoding unit 202 decodes, for example, num_eip_merge_cand_minus2, temporary_mvp_enabled_flag, eip_enabled_flag, etc. from the SPS.

[0162] Next, in step S1402, the decoding unit 203 decodes information from the bitstream indicating how the basic block of the image currently being processed is divided. Furthermore, according to this information, it identifies the positions and sizes of the multiple subblocks obtained by the division in the basic block. Note that the division into multiple subblocks performed recursively in the basic block is the same as described with reference to Figures 8A to 8F, so the explanation is omitted.

[0163] From here on, steps 1403 to 1405 are processes performed on each of the subblocks included in the basic block currently being processed. In step S1403, the inverse quantization / inverse transformation unit 204 obtains transformation coefficients by performing inverse quantization on the quantization coefficients of the subblock to be decoded, and obtains the prediction error by performing an inverse transformation on these transformation coefficients. In step S1404, the image reconstruction unit 205 generates a predicted image by performing a prediction process that refers to the image stored in the frame memory 206 based on the prediction information decoded for the subblock to be decoded by the decoding unit 203. Prediction processes used at this time include intra-prediction, inter-prediction, and EIP. When EIP is used for the subblock to be decoded, the image reconstruction unit 205 generates a predicted image for the target block in accordance with the method described above with reference to Figures 3A to 3C, 4A to 4C, 5, 6A, 6B, and 7A to 7C. The image reconstruction unit 205 then uses the generated predicted image and the prediction error obtained by the inverse quantization / inverse transform unit 204 to decode the image of the subblock to be decoded, and stores the generated image of the subblock in the frame memory 206.

[0164] In step S1406, the control unit 201 determines whether or not the decoding of all basic blocks contained in the bitstream has been completed. If the result of this determination is that the decoding of all basic blocks contained in the bitstream has been completed, the process proceeds to step S1407. On the other hand, if there are still basic blocks in the bitstream that have not yet been decoded, the process proceeds to step S402 for those basic blocks that have not yet been decoded, and the process is repeated.

[0165] In step S1407, the in-loop filter unit 207 performs in-loop filtering, such as deblocking filtering, on the reconstructed image stored in the frame memory 206.

[0166] As described above, in this embodiment, it is possible to decode an image using EIP processing on the block to be decoded. In the above description, three patterns of EIP filters shown in Figures 3A to 3C were described, but this is not limited to these. As with the description in the first embodiment, it is possible to use the filter shown in Figure 12 as a modification of this embodiment. The following describes the case in which the filter shown in Figure 12 is used in this embodiment.

[0167] In this modified example, the decoding unit 203 can decode eip_filter_size_idx from the coding_unit() syntax structure shown in Figure 13, which is the coding_unit() syntax structure for the subblock currently to be decoded. Specifically, the decoding unit 203 performs the following processing when the value of eip_flag decoded from the coding_unit() syntax structure in Figure 13 is 1 and the value of the decoded eip_merge_flag is 0 for the subblock to be decoded. In this case, according to condition 2, eip_filter_size_idx, eip_filter_idx, and eip_reconst_idx are decoded from the coding_unit() syntax structure. Then, as shown in Table 1 in the first embodiment, the decoding unit 203 identifies the size group to be used according to the value of eip_filter_size_idx. Furthermore, from a plurality of filters belonging to that size group, it identifies the filter to be used from the decoded value of eip_filter_idx.

[0168] When using filters from size group 0, the image reconstruction unit 205 generates predicted pixels for the target block using equation (1) or (2). When using the filter shown in Figure 12(d) belonging to size group 1, the image reconstruction unit 205 generates predicted pixels for the target block using equation (4) or (5). When using the filters shown in Figures 12(e) to (h) belonging to size group 1, the image reconstruction unit 205 generates predicted pixels for the target block using equation (6) or (7). When using the filters shown in Figures 12(i) to (k) belonging to size group 2, the image reconstruction unit 205 generates predicted pixels for the target block using equation (8) or (9).

[0169] The image reconstruction unit 205 calculates the filter coefficient c for the selected filter. ｉ When generating the reconstructed region, one of the reconstruction regions shown in Figures 4A to 4C is identified according to the decoded value of eip_reconst_idx, and the filter coefficient c of the selected filter is applied to the subblock to be decoded using the identified reconstruction region. ｉ This will generate [something].

[0170] Note that eip_filter_size_idx = 0 indicates that size group 0 in Figure 12 will be used. In this case, the range of possible values ​​for eip_filter_idx is 0 or greater and 2 or less. A value of 0 for eip_filter_idx indicates the filter in Figure 12(a). A value of 1 for eip_filter_idx indicates the filter in Figure 12(b). A value of 2 for eip_filter_idx indicates the filter in Figure 12(c). Similarly, eip_filter_size_idx = 1 indicates that size group 1 in Figure 12 will be used.

[0171] In this case, the range of possible values ​​for eip_filter_idx is 0 to 4. A value of 0 for eip_filter_idx represents the filter in Figure 12(d). A value of 1 for eip_filter_idx represents the filter in Figure 12(e). A value of 2 for eip_filter_idx represents the filter in Figure 12(f). A value of 3 for eip_filter_idx represents the filter in Figure 12(g). A value of 2 for eip_filter_idx represents the filter in Figure 12(h). Furthermore, eip_filter_size_idx = 2 indicates that size group 2 in Figure 12 is used. In this case, the range of possible values ​​for eip_filter_idx is 0 to 2. A value of 0 for eip_filter_idx represents the filter shown in Figure 12(i). A value of 1 for eip_filter_idx represents the filter shown in Figure 12(j). A value of 2 for eip_filter_idx represents the filter shown in Figure 12(k).

[0172] In Figure 12, if eip_filter_size_idx, eip_filter_idx, and eip_reconst_idx are not decoded, the decoding unit 203 estimates that the value of each syntax element is 0. The cases in which the syntax elements are not decoded include cases where condition2 is false. In particular, a case where eip_flag is 1 and eip_merge_flag is 0 and condition2 is false is an example. In this case, eip_filter_size_idx, eip_filter_idx, and eip_reconst_idx will always be 0. As a result, the image reconstruction unit 205 will use the filter in Figure 12(a) and the reconstruction region in Figure 4A for the subblock to be decoded.

[0173] (Third Embodiment) All of the functional units shown in Figures 1 and 2 may be implemented in hardware, but some may be implemented in software (computer program). In that case, a computer device having frame memory 107 and frame memory 206 as memory devices and capable of executing the computer program can be applied to the above-mentioned encoding device 100 and decoding device 200. An example of the hardware configuration of a computer device applicable to the above-mentioned encoding device 100 and decoding device 200 will be explained using the block diagram in Figure 15.

[0174] The CPU 1501 executes various processes using computer programs and data stored in the RAM 1502 and ROM 503. In this way, the CPU 501 controls the operation of the entire computer system and also executes or controls the processes performed by the encoding device 100 and decoding device 200.

[0175] The RAM 1502 has an area for storing computer programs and data loaded from the ROM 1503 or external storage device 1506, and data received from the outside via the I / F (interface) 1507 (for example, the video and still image data mentioned above).

[0176] Furthermore, the RAM 1502 has a work area used by the CPU 1501 when executing various processes. In this way, the RAM 1502 can provide various areas as appropriate.

[0177] ROM1503 stores configuration data and startup programs for the computer device.

[0178] The operation unit 1504 is a user interface such as a keyboard, mouse, or touch panel, and the user can input various instructions to the CPU 1501 by operating it.

[0179] The display unit 1505 is composed of an LCD screen or a touch panel screen, and can display the processing results of the CPU 1501 as images or text. For example, the playback image decoded by the decoding device 100 may be displayed on this display unit 1505.

[0180] The display unit 1505 may also be a projection device such as a projector that projects images or text.

[0181] The external storage device 1506 is a large-capacity information storage device such as a hard disk drive. The external storage device 1506 stores the OS (operating system), computer programs and data that cause the CPU 1501 to execute or control the various processes described above performed by the encoding device 100 and the decoding device 200. The computer programs stored in the external storage device 1506 include computer programs that cause the CPU 1501 to realize the functions of each functional unit other than the frame memory 107 and frame memory 206.

[0182] Computer programs and data stored in the external storage device 1506 are loaded into the RAM 1502 as appropriate according to the control of the CPU 1501 and become subject to processing by the CPU 1501. The frame memory 107 and frame memory 206 mentioned above can be implemented using memory devices such as RAM 1502, ROM 1503, and external storage device 1506.

[0183] I / F 1507 functions as an interface for data communication with external devices. For example, moving images and still images can be acquired from an external server device or imaging device via I / F 1507 and stored in RAM 1502 or external storage device 1506.

[0184] The CPU 1501, RAM 1502, ROM 1503, operation unit 1504, display unit 1505, external storage device 1506, and I / F 1507 are all connected to the bus 1508. Note that the configuration shown in Figure 5 is merely one example of a hardware configuration for a computer device applicable to the above-mentioned encoding device 100 and decoding device 200, and various modifications and variations are possible.

[0185] (Other Embodiments) In other embodiments, a program that implements one or more of the functions of the above-described embodiments is supplied to a system or device via a network or storage medium. Furthermore, it can also be implemented by a process in which one or more processors in the computer of the system or device read and execute the program. It can also be implemented by a circuit (e.g., ASIC) that implements one or more functions.

[0186] (Item 1) An encoding device comprising: prediction means for performing a predetermined intra prediction process using a filter on a target block and generating a predicted image by applying the filter while moving it; and encoding means for encoding a syntax element for identifying a filter used in the predetermined intra prediction process for the target block from among a plurality of filters, wherein if the encoding means does not encode the syntax element, the value of the syntax element is constrained to a predetermined value corresponding to one of the plurality of filters.

[0187] (Item 2) The encoding device according to Item 1, characterized in that the predetermined value is 0.

[0188] (Item 3) The encoding device according to Item 1 or 2, characterized in that the filter corresponding to the predetermined value is square.

[0189] (Item 4) The decoding device according to Item 3, characterized in that the square is 8 x 8 in size.

[0190] (Clause 5) The encoding device according to any one of Clauses 1 to 4, characterized in that the plurality of filters include a square and a rectangle.

[0191] (Item 6) The encoding apparatus according to any one of items 1 to 5, characterized in that the predetermined intra prediction process is an extraction filter-based intra prediction (EIP).

[0192] (Item 7) The encoding device according to any one of items 1 to 6, characterized in that the encoding means encodes a syntax element having information on whether or not the predetermined intra prediction process is performed.

[0193] (Clause 8) The encoding apparatus according to any one of Clauses 1 to 7, characterized in that, if the encoding means does not encode the syntax elements, the prediction means generates the predicted image using a filter corresponding to the predetermined value.

[0194] (Clause 9) The encoding device according to any one of Clauses 1 to 8, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the filter to a region including at least a portion of the target block and the region surrounding the target block while moving the filter.

[0195] (Item 10) A decoding device comprising: a decoding means for decoding a syntax element for identifying a filter to be used in a predetermined intra prediction process for a target block from among a plurality of filters for a predetermined intra prediction process; a prediction means for identifying one of the plurality of filters according to the syntax element, executing the predetermined intra prediction process using the identified filter, and generating a predicted image by applying the filter while moving it; and, if the decoding means does not decode the syntax element, the value of the syntax element is estimated to be a predetermined value corresponding to one of the plurality of filters.

[0196] (Item 11) The decoding device according to item 10, characterized in that the predetermined value is 0.

[0197] (Item 12) The decoding device according to item 10 or 11, characterized in that the filter corresponding to the predetermined value is square.

[0198] (Item 13) The decoding device according to Item 12, characterized in that the square is 8 x 8 in size.

[0199] (Clause 14) The decoding device according to any one of Clauses 10 to 13, characterized in that the plurality of filters include a square and a rectangle.

[0200] (Clause 15) The decoding device according to any one of Clauses 10 to 14, characterized in that the predetermined intra prediction process is an extraction filter-based intra prediction (EIP).

[0201] (Clause 16) The decoding device according to any one of Clauses 10 to 15, characterized in that the decoding means decodes a syntax element having information on whether or not the predetermined intra prediction process is performed.

[0202] (Clause 17) The decoding apparatus according to any one of Clauses 10 to 16, wherein if the decoding means does not decode the syntax element, the prediction means generates the predicted image using a filter corresponding to the predetermined value.

[0203] (Clause 18) The encoding device according to any one of Clauses 10 to 17, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the filter to a region including at least a portion of the target block and the region surrounding the target block while moving the filter.

[0204] (Item 19) An encoding method comprising: a prediction step of performing a predetermined intra prediction process using a filter on a target block and applying the filter while moving it to generate a predicted image; and an encoding step of encoding a syntax element for identifying a filter used in the predetermined intra prediction process for the target block from among a plurality of filters, wherein if the syntax element is not encoded by the encoding step, the value of the syntax element is constrained to a predetermined value corresponding to one of the plurality of filters.

[0205] (Item 20) A decoding method characterized in that: a decoding step of decoding a syntax element for identifying a filter to be used in a predetermined intra prediction process for a target block from among a plurality of filters for a predetermined intra prediction process; a prediction step of identifying one of the plurality of filters according to the syntax element, executing the predetermined intra prediction process using the identified filter, and generating a predicted image by applying the filter while moving it; and if the decoding step does not decode the syntax element, the value of the syntax element is estimated to be a predetermined value corresponding to one of the plurality of filters.

[0206] (Item 21) A program characterized in that it causes a computer to function as each of the means of the encoding device described in Item 1.

[0207] (Item 22) A program characterized in that it causes a computer to function as one of the means of the decoding device described in Item 10.

[0208] The present invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are attached to make the scope of the invention public.

[0209] This application claims priority based on Japanese Patent Application No. 2025-005970, filed on 16 January 2025, and all of its contents are incorporated herein by reference.

Claims

1. An encoding device comprising: prediction means for performing a predetermined intra-prediction process using a filter on a target block and generating a predicted image by applying the filter while moving it; and encoding means for encoding a syntax element for identifying a filter used in the predetermined intra-prediction process for the target block from among a plurality of filters, wherein if the encoding means does not encode the syntax element, the value of the syntax element is constrained to a predetermined value corresponding to one of the plurality of filters.

2. The encoding device according to claim 1, characterized in that the predetermined value is 0.

3. The encoding device according to claim 1, characterized in that the filter corresponding to the predetermined value is square.

4. The encoding device according to claim 3, characterized in that the square has an 8x8 size.

5. The encoding device according to claim 1, characterized in that the plurality of filters include a square and a rectangle.

6. The encoding device according to claim 1, characterized in that the predetermined intra prediction process is an extraction filter-based intra prediction (EIP).

7. The encoding device according to claim 1, characterized in that the encoding means encodes a syntax element having information on whether or not the predetermined intra prediction process is performed.

8. The encoding apparatus according to claim 1, characterized in that, if the encoding means does not encode the syntax elements, the prediction means generates the predicted image using a filter corresponding to the predetermined value.

9. The encoding device according to claim 1, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the filter while moving it over a region including at least a portion of the target block and the region surrounding the target block.

10. A decoding device comprising: a decoding means for decoding a syntax element for identifying a filter to be used in a predetermined intra-prediction process for a target block from among a plurality of filters for a predetermined intra-prediction process; a prediction means for identifying one of the plurality of filters according to the syntax element, executing the predetermined intra-prediction process using the identified filter, and generating a predicted image by applying the filter while moving it; and, if the decoding means does not decode the syntax element, the value of the syntax element is estimated to be a predetermined value corresponding to one of the plurality of filters.

11. The decoding device according to claim 10, characterized in that the predetermined value is 0.

12. The decoding device according to claim 10, characterized in that the filter corresponding to the predetermined value is square.

13. The decoding device according to claim 12, characterized in that the square is 8 x 8 in size.

14. The decoding device according to claim 10, characterized in that the plurality of filters include squares and rectangles.

15. The decoding device according to claim 10, characterized in that the predetermined intra prediction process is an extraction filter-based intra prediction (EIP).

16. The decoding device according to claim 10, characterized in that the decoding means decodes a syntax element having information on whether or not the predetermined intra prediction process is performed.

17. The decoding apparatus according to claim 10, characterized in that, if the decoding means does not decode the syntax element, the prediction means generates the predicted image using a filter corresponding to the predetermined value.

18. The decoding device according to claim 10, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the filter while moving it over a region including at least a portion of the target block and the region surrounding the target block.

19. An encoding method comprising: a prediction step of performing a predetermined intra-prediction process using a filter on a target block and generating a predicted image by applying the filter while moving it; and an encoding step of encoding a syntax element for identifying a filter to be used in the predetermined intra-prediction process for the target block from among a plurality of filters, wherein if the syntax element is not encoded by the encoding step, the value of the syntax element is constrained to a predetermined value corresponding to one of the plurality of filters.

20. A decoding method comprising: a decoding step of decoding a syntax element for identifying a filter to be used in a predetermined intra prediction process for a target block from among a plurality of filters for a predetermined intra prediction process; a prediction step of identifying one of the plurality of filters according to the syntax element, executing the predetermined intra prediction process using the identified filter, and generating a predicted image by applying the filter while moving it; and, if the decoding step does not decode the syntax element, the value of the syntax element is estimated to be a predetermined value corresponding to one of the plurality of filters.

21. A program characterized by causing a computer to function as each of the means of the encoding device described in claim 1.

22. A program characterized by causing a computer to function as each of the means of the decoding device described in claim 10.

Images