Encoding apparatus, encoding method, decoding apparatus, decoding method, and program

Abstract

This encoding apparatus is characterized in that: a prediction means specifies a filter to be used from among a plurality of filters for predetermined intra prediction processing, the plurality of filters including a filter that uses at least one of a reconstructed pixel value or a predicted pixel value corresponding to a pixel position included in a region to the right of the position of a predicted pixel value to be derived in the predetermined intra prediction processing, and a reconstructed pixel value or a predicted pixel value corresponding to a pixel position included in a region below the position of the predicted pixel value to be derived; and the prediction means specifies a reconstructed region corresponding to the specified filter from among a plurality of reconstructed regions, derives a filter coefficient of the specified filter from the specified filter and the specified reconstructed region, and generates a prediction image by predetermined intra prediction using the specified filter and the filter coefficient.

Description

Encoding device, encoding method, decoding device, decoding method, program

[0001] This invention relates to image encoding and decoding techniques.

[0002] As an encoding method for compressed recording of moving images, the HEVC (High Efficiency Video Coding) encoding method (hereinafter referred to as HEVC) is known. In addition, the VVC (Versatile Video Coding) encoding method (hereinafter referred to as VVC), which is the successor to HEVC, is also known. In HEVC, in order to improve encoding efficiency, a larger basic block size than the conventional macroblock (16 pixels x 16 pixels) is adopted. This larger basic block is called a CTU (Coding Tree Unit), and its size is up to 64 pixels x 64 pixels. In VVC, the size of the CTU can be up to 128 pixels x 128 pixels. The CTU is further divided into subblocks that serve as units for prediction and transformation.

[0003] In recent years, efforts have been made to internationally standardize an even more efficient encoding scheme as a successor to VVC, and standardization is being advanced by JVET (Joint Video Experts Team). Non-patent document 1 is the algorithm description of the reference model (hereinafter referred to as ECM) of the new encoding scheme that has been published to date.

[0004] Furthermore, Non-Patent Document 1 describes Extraction filter-based intra prediction (EIP) as a new intra prediction process.

[0005] Algorithm description of Enhanced Compression Model 14 (ECM 14), (JVET-AI2025) (version1- date 2024-10-21)

[0006] Therefore, the present invention aims to provide a technology that enables efficient specification of information used in predictive processing.

[0007] In view of the above problems, the encoding device according to the present invention comprises: prediction means for deriving a plurality of predicted pixel values ​​included in a predicted image by applying a filter to a target block while moving the filter over it; and encoding means for encoding the target block using the predicted image. The prediction means identifies a filter to be used from a plurality of filters for a predetermined intra-prediction process, and generates a predicted image by the predetermined intra-prediction process using the identified filter. The plurality of filters include a filter that uses at least one of a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in a region to the right of the position of the predicted pixel value to be derived in the predetermined intra-prediction process, and a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in a region below the position of the predicted pixel value to be derived. The prediction means identifies a reconstruction region corresponding to the identified filter from a plurality of reconstruction regions, derives the filter coefficient of the identified filter from the identified filter and the identified reconstruction region, and generates the predicted image by the predetermined intra-prediction using the identified filter and the filter coefficient.

[0008] Furthermore, the decoding device according to the present invention comprises a prediction means for deriving a plurality of predicted pixel values ​​included in a predicted image by applying a filter to a target block while moving the filter, and a decoding means for decoding the target block using the predicted image, wherein the prediction means identifies a filter to be used from a plurality of filters for a predetermined intra-prediction process, generates a predicted image by the predetermined intra-prediction process using the identified filter, the plurality of filters includes a filter that uses at least one of a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in a region to the right of the position of the predicted pixel value to be derived in the predetermined intra-prediction process, and a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in a region below the position of the predicted pixel value to be derived, the prediction means identifies a reconstruction region corresponding to the identified filter from a plurality of reconstruction regions, derives the filter coefficient of the identified filter from the identified filter and the identified reconstruction region, and generates the predicted image by the predetermined intra-prediction using the identified filter and the filter coefficient.

[0009] According to the present invention, it is possible to reduce the amount of information used in predictive processing.

[0010] Block diagram showing an example of the functional configuration of an encoding device. Block diagram showing an example of the functional configuration of a decoding device. Diagram showing an example of a filter. Diagram showing an example of a filter. Diagram showing an example of a filter. Diagram showing an example of a filter. Diagram showing an example of a filter. Diagram showing an example of a filter. Diagram showing an example of a reconstructed area. Diagram showing an example of a reconstructed area. Diagram showing an example of a reconstructed area. Diagram explaining the processing of the EIP. Diagram showing an example of filter scanning. Diagram showing an example of filter scanning. Diagram explaining the generation of an EIP merge list. Diagram showing an example of block partitioning. Diagram showing an example of block partitioning. Diagram showing an example of block partitioning. Diagram showing an example of block partitioning. Diagram showing an example of block partitioning. Diagram showing an example of block partitioning. Diagram showing an example of bitstream. Diagram showing an example of the coding_unit syntax structure. Flowchart of the encoding process. Diagram showing an example of a filter. Diagram showing an example of the coding_unit syntax structure. Diagram showing an example of the coding_unit syntax structure. A diagram showing an example of the coding_unit syntax structure. A flowchart of the decoding process. A flowchart of the decoding process. A diagram showing an example of the hardware configuration of a computer device. A diagram showing an example of a filter. A diagram showing an example of a filter. A diagram showing an example of a filter. A diagram showing an example of a filter. A diagram showing an example of a filter.

[0011] The embodiments will be described below with reference to the attached drawings. The embodiments described below are examples and represent one specific embodiment of the configuration described in the claims.

[0012] The images encoded or decoded in each embodiment may be still images or videos. Furthermore, while EIP will be used as an example, each embodiment is applicable to prediction processes other than EIP. In particular, it is applicable to prediction processes that require specific information, such as the information indicating filters in EIP. Also, in the following description, the term "output" will be used interchangeably with "derive" or "determine" as appropriate.

[0013] (First Embodiment) First, an example of the functional configuration of the encoding device (image encoding device) 100 according to this embodiment will be explained using the block diagram in Figure 1. The control unit 101 controls the overall operation of the encoding device 100. The block division unit 102 acquires an input image (images of each frame constituting a moving image or still images), divides the input image into a plurality of basic blocks, and outputs each of the divided basic blocks. The basic block is a CTU (Coding Tree Unit), and its size is, for example, 32 pixels × 32 pixels, 64 pixels × 64 pixels, or 128 pixels × 128 pixels.

[0014] The prediction unit 103 divides each basic block into multiple subblocks. A subblock is, for example, a coding unit.

[0015] The prediction unit 103 may also divide the subblocks obtained by dividing the basic block to obtain even smaller subblocks. These further divided smaller subblocks may be, for example, a transformation block or a prediction block. Alternatively, they may simply be subblocks of a coding unit.

[0016] The prediction unit 103 generates a predicted image by performing prediction processing such as intra-prediction (intra-frame prediction) and inter-prediction (inter-frame prediction) for each sub-block, and calculates the difference between the input image and the predicted image as the prediction error. The prediction unit 103 also generates prediction information, which includes information indicating how the basic block was divided and information necessary for prediction processing, such as the prediction processing mode.

[0017] The transform / quantization unit 104 generates transform coefficients for each subblock by performing a transform process on the prediction error for each subblock. Note that as the transform process, it is possible to perform orthogonal transform using transform bases such as DCT2, DCT8, and DST7. Here, the orthogonal transform may be performed using the same transform basis in each of the horizontal direction and the vertical direction of the subblock, or the orthogonal transform may be performed using different transform bases in the horizontal direction and the vertical direction of the subblock. Then, the transform / quantization unit 104 generates quantized coefficients of the subblock by quantizing the transform coefficients of the subblock. Note that the transform / quantization unit 104 may quantize the transform coefficients using a quantization matrix.

[0018] The inverse quantization / inverse transform unit 105 inversely quantizes the quantized coefficients of each subblock generated by the transform / quantization unit 104 to generate transform coefficients, and performs an inverse transform process on the transform coefficients to generate a prediction error.

[0019] The image reconstruction unit 106 generates a predicted image from an encoded image stored in the frame memory 107 that has been reconstructed by the image reconstruction unit 106, based on the prediction information generated by the prediction unit 103. Further, an image is reconstructed from the predicted image and the prediction error generated by the inverse quantization / inverse transform unit 105. Then, the image reconstruction unit 106 stores the reconstructed image in the frame memory 107. The reconstructed image stored in the frame memory 107 serves as image data that is referenced when the prediction unit 103 performs prediction processing on the image of the current frame or the next frame.

[0020] The in-loop filter unit 108 performs in-loop filter processing such as a deblocking filter or sample adaptive offset on the image stored in the frame memory 107.

[0021] The encoding unit 109 generates encoded data by entropy encoding the quantized coefficients generated by the transform / quantization unit 104 and the prediction information generated by the prediction unit. The method of entropy encoding is not limited to a specific method, and Golomb coding, arithmetic coding, Huffman coding, or the like can be used.

[0022] Here, the prediction processing executed by the prediction unit 103 in the present embodiment will be described in further detail. The prediction unit 103 in the present embodiment is capable of executing intra prediction and inter prediction according to the HEVC standard and the VVC standard. Furthermore, the prediction unit 103 is capable of executing EIP (Extrapolation filter-based intra prediction), which is a new intra prediction process (predetermined intra prediction process). Here, with reference to FIGS. 3A to 3H, EIP executed by the prediction unit will be described.

[0023] FIGS. 3A to 3H are examples of filters used in EIP. The prediction unit 103 selects one filter from among FIGS. 3A to 3H for a current sub-block to be processed (hereinafter referred to as a target block). FIG. 3A is a filter with a size of 4 pixels × 4 pixels, FIG. 3B is a filter with a size of 8 pixels × 2 pixels, and FIG. 3C is a filter with a size of 2 pixels × 8 pixels. The filters in FIG. 3D and FIG. 3F are, for example, filters with a size of 6 pixels × 6 pixels, having 6 pixels in the width (w) direction and 6 pixels in the height (h) direction. Here, the filters in FIG. 3D and FIG. 3F differ in the number of pixel values used. The filter in FIG. 3E is, for example, a filter with a size of 5 pixels × 5 pixels, having 5 pixels in the width (w) direction and 5 pixels in the height (h) direction. FIG. 3G is a filter with a size of 8 pixels × 3 pixels, having 8 pixels in the width (w) direction and 3 pixels in the height (h) direction. FIG. 3H is a filter with a size of 3 pixels × 8 pixels, having 3 pixels in the width (w) direction and 8 pixels in the height (h) direction. Furthermore, the filter shapes from FIG. 3D to FIG. 3F are shapes that include reconstructed pixel values (predicted pixel values) corresponding to pixel positions included in a region to the right and a region below the position of a predicted pixel value to be derived in intra prediction processing. The filter shape in FIG. 3G is a shape that includes a reconstructed pixel value (predicted pixel value) corresponding to a pixel position included in a region below the position of a predicted pixel value to be derived in intra prediction processing.

[0024] The filter shape in Figure 3H is a shape that includes reconstructed pixel values ​​(predicted pixel values) corresponding to pixel positions in the region to the right of the position of the predicted pixel value to be derived in the intra-prediction process. The prediction unit 103 selects one of these filters. Furthermore, Figures 4A to 4C show the types of reconstruction regions. The method for determining each filter coefficient that constitutes the selected filter is not particularly limited. For example, predetermined initial values ​​may be used, or they may be set individually or generated according to the characteristics of the image. Alternatively, the system may be configured to select one of the reconstruction regions shown in Figures 4A to 4C for the target block and generate the filter coefficients of the selected filter using the selected reconstruction region. Alternatively, the correlation between the pixel value at the position corresponding to the predicted pixel in the reconstruction region and other pixel values ​​may be derived using convolution integral, and the filter coefficients may be determined based on that correlation. In this embodiment, as an example, the prediction unit 103 is configured to select one of the reconstruction regions shown in Figures 4A to 4C for the target block and generate the filter coefficients of the selected filter using the selected filter and the selected reconstruction region. Furthermore, the aboveSize shown in Figures 4A to 4C is a size determined by the height of the target block, and similarly, the leftSize is a size determined by, for example, the width of the target block. For example, aboveSize is a size equal to the height of the target block, or a size obtained by adding a predetermined value to the height of the target block. Similarly, leftSize is a size equal to the width of the target block, or a size obtained by adding a predetermined value to the width of the target block.

[0025] Equation (1) is an example of a calculation formula for generating predicted pixels using EIP. （ｘ，ｙ） This is the predicted pixel value at the (x, y) position in the current target block. ｉ is the filter coefficient for position i. ｉis the reconstructed pixel value or predicted pixel value at position i, and bitdepth is the bit depth of the image to be encoded. Although the correspondence between each pixel position and the value of i is shown in Figures 3A to 3H for each filter, the correspondence between each pixel position and the value of i in each filter is not limited to the examples shown in Figures 3A to 3H, and other correspondences may also be observed.

[0026]

[0027] In the example shown in equation (1), for example, the reconstructed pixel value or predicted pixel value corresponding to i = 14, which is the position furthest from the predicted pixel position (bottom right corner), is not used. In this way, one or more pixels in the region containing the pixel furthest from the predicted pixel position may not be used for prediction.

[0028] Furthermore, as shown in equation (2) below, the reconstructed pixel value or predicted pixel value corresponding to i = 14, which is the position furthest from the predicted pixel position (lower right corner), may also be used.

[0029]

[0030] The encoding device may decide which formula to use for the predicted image based on the image characteristics and signal it to the bitstream using an index or similar method.

[0031] Here, we will describe an example of prediction processing using EIP when the size of the target block is 4 pixels × 4 pixels and a 4 pixel × 4 pixel filter as shown in Figure 3A is selected. As shown in Figure 5(a), there is an encoded and reconstructed image region 502 around the target block 501. The prediction unit 103 positions the 4 pixel × 4 pixel filter 503 such that its position (x, y) is at the top left (0, 0) of the 4 pixel × 4 pixel target block 501. Then, according to equation (1) or (2), the predicted pixel value at position (x, y) is pred （ｘ，ｙ） We derive the following. In this case, t from position i = 0 to 13 ｉ The encoded pixel values ​​in the image region 502 are used as follows. For example, t corresponding to position i=0 ０represents the value of the encoded pixel adjacent above the pixel at position (x, y) in Fig. 5(a). For the position (x, y) shown in Fig. 5(a), the predicted pixel value pred （ｘ，ｙ） is derived, and then the prediction unit 103 shifts the position of the filter 503 by one pixel in accordance with the scanning order indicated by arrows in Fig. 5(a), and obtains the predicted pixel value pred for the position (x, y) shown in Fig. 5(b) in accordance with formula (1) or (2) （ｘ，ｙ） . At this time, for t at positions i=0 and 2 to 13 in the filter 503 of Fig. 5(b) ｉ encoded pixel values are used, while the predicted pixel value pred that has already been derived in accordance with formula (1) or (2) in the arrangement of the filter 503 of Fig. 5(a) （ｘ，ｙ） is used as t at position i=1 of the filter 503 in Fig. 5(b) ｉ will be used. As described above, the prediction unit 103 shifts the selected filter by one pixel in accordance with the scanning order indicated by arrows in Fig. 5(a), and obtains the predicted pixel value pred （ｘ，ｙ） sequentially, and generates a prediction image of the target block 501.

[0032] In this scanning method, the first dashed arrow and the last dashed arrow move one pixel to the right.

[0033] Then, at each position indicated by each solid arrow, the scan moves diagonally one pixel at a time toward the lower left. When the end of the target block 501 is reached as a result of one or more consecutive diagonal movements indicated by solid arrows, the scan jumps over a plurality of pixel positions and moves to a position immediately to the right of or directly below the start position of the immediately preceding one or more consecutive diagonal movements, as indicated by a dashed arrow. For convenience, the scanning method shown in Figs. 5(a) and 5(b) is referred to as Down-left diagonal scan.

[0034] Alternatively, a scanning method as shown in Figure 5(c) may be used as the scanning sequence. These are called Up-right diagonal scans, which were used in HEVC for scanning the quantization matrix. In this scanning method, the first and last dashed arrows move one pixel downwards. Then, at each position indicated by the solid arrows, the cursor moves diagonally one pixel to the upper right. As a result of one or more consecutive diagonal movements indicated by the solid arrows, when the cursor reaches the edge of the target block 501, it jumps over multiple pixel positions as indicated by the dashed arrows and moves to a position directly below or to the right of the starting position of the previous one or more consecutive diagonal movements.

[0035] As described above, the prediction unit 103 generates a predicted image by applying a filter while moving it. More specifically, the prediction unit 103 generates a predicted image by applying a filter while moving it to an area that includes at least a portion of the target block 501 and the area surrounding the target block 501.

[0036] The scanning order of the filter in the target block 501 is not limited to the example shown in Figure 5(a), but may also be the scanning order shown in Figures 6A and 6B. Figure 6A shows the order in which each row of pixels (hereinafter referred to as "pixel row") of the 8x8 target block 501 is scanned horizontally. In this case, after the scan for predictive pixel generation is performed on the uppermost pixel row of the 8x8 target block 501, the scan for predictive pixel generation may be performed on the adjacent pixel row directly below that pixel row. That is, after the scan for predictive pixel generation is performed on a certain pixel row, the scan for predictive pixel generation may be performed on the adjacent pixel row directly below that pixel row. Furthermore, predictive pixel generation is performed on the leftmost pixel in the uppermost pixel row of the 8x8 target block 501. Subsequently, in parallel with the predictionive pixel generation being performed on the pixel adjacent to the right of that pixel according to the scanning order shown in Figure 6A, predictive pixel generation may also be performed on the leftmost pixel of the adjacent pixel row below the uppermost pixel row. In subsequent processing, as shown in Figure 6A, filter scanning for generating predicted pixels may be performed in parallel, shifting by one pixel for each pixel row of the target block 501. This makes it possible to perform parallel processing for each pixel row for generating predicted pixels.

[0037] Similarly, Figure 6B shows the order in which each pixel column (hereinafter referred to as "pixel column") of the 8x8 target block 501 is scanned vertically. In this case, after the scan for predictive pixel generation is performed on the leftmost pixel column of the 8x8 target block 501, the scan for predictive pixel generation may be performed on the pixel column adjacent to the right of that pixel column.

[0038] In other words, after a scan for predictive pixel generation is performed on a certain pixel row, a scan for predictive pixel generation may be performed on the pixel row adjacent to the right of that pixel row. Alternatively, predictive pixel generation may be performed on the uppermost pixel in the leftmost pixel row of the 8x8 target block 501. Subsequently, in parallel with the generation of predictive pixels on the pixels adjacent below that pixel according to the scan order shown in Figure 6B, predictive pixel generation may also be performed on the uppermost pixel in the pixel row adjacent to the right of the leftmost pixel row. In this case, in the subsequent processing, as shown in Figure 6B, filter scans for predictive pixel generation may be performed in parallel, shifting by one pixel for each pixel row of the target block 501. This makes it possible to perform parallel processing for predictive pixel generation for each pixel row.

[0039] As mentioned above, the filter scan in the target block 501 may be an Up-right diagonal scan as shown in Figure 5(c). This scan order is the same as the scan order used for encoding the quantization matrix and quantization coefficients of HEVC and VVC. By standardizing the scan order with the quantization matrix and quantization coefficients, it is possible to reduce the memory used for the scan table.

[0040] Here, using Figures 5(d) to 5(f), the prediction process performed by the prediction unit 103 when other filter shapes are used will be explained. The following explanation may be performed in addition to the above explanation, or in place of the above explanation.

[0041] Furthermore, this explanation uses one of the filters shown in Figure 3D as an example. However, the following explanation is applicable to filter shapes that include pixel positions located in the region to the right or below the pixel position where the predicted pixel value is output (derived). In other words, it is applicable to all filters shown in Figures 3D to 3H. To put it another way, it is applicable to filters that use at least one reconstructed pixel value or predicted pixel value corresponding to a pixel position located in the region to the right of the position (output) where the predicted pixel value is derived. It is also applicable to filters that use at least one reconstructed pixel value or predicted pixel value corresponding to a pixel position located in the region below the position where the predicted pixel value is derived.

[0042] When using the filter shown in Figure 3D, there may be cases where reconstructed pixel values ​​or predicted pixel values ​​do not exist depending on the pixel position to which the filter is applied.

[0043] As shown in Figure 5(d), reconstructed pixel values ​​or predicted pixel values ​​may exist at all pixel locations covered by the filter 504. In such cases, the application location of the filter 504 (for example, the location where a predicted pixel value is output) will be referred to as the first pixel location. This first pixel location is a pixel location within the target block 501 that is not adjacent to the bottom or right edge.

[0044] When applying the filter 504 at such a first pixel position to derive a predicted pixel value, the prediction unit 103 may use the respective pixel values ​​corresponding to i = 0 to 13, as described above. For example, as shown in Figure 5(d), if the first pixel position indicated by (x, y) where the predicted pixel value is output is not touching the lower or right edge of the target block 501, the next reconstructed pixel value or predicted pixel value can be used. That is, the reconstructed pixel value or predicted pixel value of the pixel positions i = 0, 3, 4, 8 (second pixel positions) can be used.

[0045] However, depending on the position where the filter is applied, the reconstructed pixel value or predicted pixel value may not yet have been derived and therefore cannot be used. The position where the filter 504 is applied in such a case where either the reconstructed pixel value or predicted pixel value cannot be used (for example, the position where the predicted pixel value is output) will be referred to as the third pixel position. When the filter 504 is applied at such a third pixel position to derive the predicted pixel value, the prediction unit 103 may, as described above, use a predetermined value instead of using the pixel value of the unusable pixel position (referred to as the fourth pixel position).

[0046] For example, in Figure 5(e), processing is not complete at the pixel positions i=0 and i=4 (the fourth pixel position), which are located to the right of the position indicated by (x,y) where the predicted pixel value is output (the third pixel position). Therefore, the reconstructed pixel values ​​and predicted pixel values ​​for these pixel positions cannot be used. In other words, the reconstructed pixel values ​​and predicted pixel values ​​for these positions have not been derived and cannot be used.

[0047] Therefore, the prediction unit 103 uses a predetermined value instead of the reconstructed pixel value or predicted pixel value at the pixel position i=0. This predetermined value is, for example, the reconstructed pixel value or predicted pixel value corresponding to the pixel position i=5. Alternatively, this predetermined value may be the reconstructed pixel value or predicted pixel value corresponding to the pixel position above the pixel position i=9.

[0048] Furthermore, the prediction unit 103 uses a predetermined value instead of the reconstructed pixel value or predicted pixel value at the pixel position i=4. This predetermined value is, for example, the reconstructed pixel value or predicted pixel value corresponding to the pixel position i=9. Alternatively, this predetermined value may be the reconstructed pixel value or predicted pixel value corresponding to a pixel position above the pixel position i=9.

[0049] Thus, the predetermined value used instead may be a reconstructed pixel value or a predicted pixel value corresponding to a pixel position included in the region to the left of the pixel position i=0,4.

[0050] Furthermore, instead of the reconstructed pixel value or predicted pixel value at the pixel positions i=0 or i=4, predetermined values ​​such as those shown below may be used.

[0051] For example, zero may be used as the predetermined value. Alternatively, the predicted pixel value from a pixel position earlier in the derivation order than an unusable pixel position may be used. In particular, the most recently derived predicted pixel value may be used. For example, in Figure 5(e), the predicted pixel value derived at pixel positions i=0 to i=3 may be used instead of the reconstructed pixel value or predicted pixel value at pixel position i=4. Alternatively, the average of all multiple predicted pixel values ​​already derived at pixel positions i=0 to i=3, or the average of some of the predicted pixel values, may be used.

[0052] Alternatively, instead of the reconstructed pixel value or predicted pixel value at the pixel positions i=0 or i=4, a value based on the bit depth of the image containing the target block 501 may be used. For example, the same value as the reconstructed pixel value or predicted pixel value at position 14 (i=14) described in equation (1) above may be used. Alternatively, instead of the reconstructed pixel value or predicted pixel value at the pixel positions i=0 or i=4, other values ​​based on the bit depth of the image containing the target block 501 may be used. For example, the midpoint of the pixel value range obtained by 1 << (BitDepth-1) can be used.

[0053] Furthermore, in Figure 5(f), processing is not complete at the pixel positions i=3 and i=8 (the fourth pixel position), which are located in the region below the position indicated by (x,y) where the predicted pixel value is output (the third pixel position). Therefore, the reconstructed pixel values ​​and predicted pixel values ​​for these pixel positions cannot be used.

[0054] In this case, the prediction unit 103 uses a predetermined value instead of the reconstructed pixel value or predicted pixel value at the pixel position i=3. This predetermined value is, for example, the reconstructed pixel value or predicted pixel value corresponding to the pixel position i=7. Alternatively, this predetermined value may be the reconstructed pixel value or predicted pixel value corresponding to the pixel position to the left of the pixel position i=12.

[0055] Furthermore, the prediction unit 103 uses a predetermined value instead of the reconstructed pixel value or predicted pixel value at the pixel position i=8. This predetermined value is, for example, the reconstructed pixel value or predicted pixel value corresponding to the pixel position i=12. Alternatively, this predetermined value may be the reconstructed pixel value or predicted pixel value corresponding to the pixel position to the left of the pixel position i=12.

[0056] Thus, the predetermined value used instead may be a reconstructed pixel value or a predicted pixel value corresponding to a pixel position included in the region above the pixel position i = 3, 8.

[0057] Furthermore, instead of the reconstructed pixel value or predicted pixel value at the pixel positions i=3 or i=8, predetermined values ​​may be used as described above.

[0058] For example, zero may be used as the predetermined value. Alternatively, the predicted pixel value from a pixel position earlier in the derivation order than an unusable pixel position may be used. In particular, the most recently derived predicted pixel value may be used. For example, in Figure 5(f), instead of the reconstructed pixel value or predicted pixel value at the pixel position i=3, one of the predicted pixel values ​​already derived at the pixel positions i=0 to i=2 may be used. Alternatively, the average of all of the multiple predicted pixel values ​​already derived at the pixel positions i=0 to i=2, or the average of some of the predicted pixel values, may be used.

[0059] In Figure 5(f), instead of the reconstructed pixel value or predicted pixel value at the pixel position i=8, one of the predicted pixel values ​​already derived at the pixel positions i=0 to i=7 may be used. Alternatively, the average of all of the multiple predicted pixel values ​​already derived at the pixel positions i=0 to i=7, or the average of some of the predicted pixel values, may be used.

[0060] Alternatively, instead of the reconstructed pixel value or predicted pixel value at pixel positions i=3 or i=8, a value based on the bit depth of the image containing the target block 501 may be used. For example, the same value as the reconstructed pixel value or predicted pixel value at position 14 (i=14) described in equation (1) above may be used. Alternatively, instead of the reconstructed pixel value or predicted pixel value at pixel positions i=3 or i=8, other values ​​based on the bit depth of the image containing the target block 501 may be used. For example, the midpoint of the pixel value range obtained by 1 << (BitDepth-1) can be used.

[0061] Next, Figure 5(g) will be explained. In Figure 5(g), processing is not complete at the pixel positions i=0 and i=4 (the fourth pixel position), which are located in the region to the right of the position indicated by (x,y) where the predicted pixel value is output. Therefore, the reconstructed pixel values ​​and predicted pixel values ​​for these pixel positions cannot be used. Similarly, processing is not complete at the pixel positions i=3 and i=8 (the fourth pixel position), which are located in the region below the position indicated by (x,y) where the predicted pixel value is output. Therefore, the reconstructed pixel values ​​and predicted pixel values ​​for these pixel positions cannot be used.

[0062] Therefore, the prediction unit 103 uses a predetermined value instead of the reconstructed pixel value or predicted pixel value at each pixel position i = 0, 3, 4, and 8.

[0063] For the predetermined values, the predetermined values ​​explained in Figures 5(e) and 5(f) can be used.

[0064] As described above, by adaptively changing the values ​​used depending on where the filter is applied, the prediction process using the filter can be performed appropriately.

[0065] Here, we will explain further examples of the order in which reconstructed pixel values ​​and predicted pixel values ​​are used when deriving predicted pixel values ​​using equations (1) and (2), using Figures 16A to 16E. In Figures 3A to 3H and Figures 16A to 16E, the order is indicated by the number shown with "i=". In Figures 16A to 16E, the order of reconstructed pixel values ​​or predicted pixel values ​​corresponding to pixel positions that satisfy the following conditions 1 and 2 is set later than the order in cases where conditions 1 and 2 are not satisfied.

[0066] Here, condition 1 is that it corresponds to a pixel position located to the right of the position (output) from which the predicted pixel value is derived.

[0067] Condition 2 is that the pixel position corresponds to a region located below the position from which the predicted pixel value is derived.

[0068] As an example, let's explain using the leftmost diagram in Figure 16(c). The pixel positions i=18 and i=16 are located in the region to the right of the location (output) from which the predicted pixel value is derived. The pixel positions i=17 and i=19 are located in the region below the location (output) from which the predicted pixel value is derived. These pixel values ​​are used in a later order than the other pixel positions.

[0069] By doing this, pixel values ​​that are more likely to be usable can be prioritized for use.

[0070] Next, we will describe a specific mode in the EIP prediction process. In the above description, the filter coefficient c used in the EIP prediction process... ｉ The explanation described how to obtain this using the reconstruction area surrounding the target block, but it is not limited to this. For example, in a particular mode (hereinafter referred to as EIP merge mode), the filter coefficient c used in other blocks other than the target block where the EIP prediction process was performed. ｉ Filter information such as these may be used in the prediction process of the EIP of the target block. In this mode, by using information used in other blocks to determine the information to be used in the prediction process of the target block, that information can be encoded efficiently.

[0071] The EIP merge mode will be explained below with reference to Figure 7. When EIP merging is used, the prediction unit 103 generates an EIP merge list (mergeEIPCandList) for the target block 701 in the image currently being processed (current image). Specifically, as shown in Figure 7(a), the EIP merge list is generated from spatially adjacent or nearby blocks (blocks containing positions A0 to A1 / blocks containing positions B0 to B2) to the target block 701. Alternatively, the EIP merge list (mergeEIPCandList) is generated from positions in temporally different images that are spatially corresponding to the target block 701. Or, the EIP merge list (mergeEIPCandList) is generated from blocks adjacent / nearby (blocks containing position C0 / blocks containing position C1) to a spatially corresponding position to the target block 701. Block 702 is a block included in a reference image that is temporally different from the current image containing the target block 701, and is located at the same position as the target block 701. In other words, block 702 is a block that is collated with the target block 701. As described above, the list described later may include information about blocks surrounding the target block (including adjacent blocks) as candidates for information to be used in the target block. Furthermore, the list described later may include information about blocks (including adjacent blocks) that are spatially corresponding to or surrounding the target block in temporally different images as candidates.

[0072] To generate the EIP merge list (mergeEIPCanandList), the prediction unit 103 determines whether a subblock containing the location of B1 is available for the target block 701. For example, if the subblock containing the location of B1 is currently included in the image, the prediction unit 103 determines that the subblock containing the location of B1 is available and sets availableFlagB1 to 1 (true). On the other hand, if the location of B1 is not currently included in the image, the prediction unit 103 determines that the subblock containing the location of B1 is unavailable and sets availableFlagB1 to 0 (false). Furthermore, the prediction unit 103 determines whether EIP is being used for the subblock containing the location of B1, and sets EIPFlag[B1] to 1 (true) if it is being used, and sets EIPFlag[B1] to 0 (false) if it is not being used. Then, as shown in Figure 7(b), if the conditions are met that availableFlagB1 is 1 and EIPFlag[B1] is 1, the prediction unit 103 stores the information of candidate B1 in the EIP merge list (mergeEIPCandList). The information of candidate B1 includes information that identifies the filter used in the EIP processing in the subblock containing the position of B1, and the coefficient c of that filter. ｉ The information is linked.

[0073] The information stored in the list may be, for example, information that identifies a block called B1 (such as an index or the coordinates corresponding to B1).

[0074] Furthermore, the information stored in the list may also be information that identifies the filter used in the EIP processing of that subblock (in this case, B1). For example, the information stored in the list may be an index for identifying the filter.

[0075] Additionally, the information stored in the list is the filter coefficient c of the filter used in the EIP processing of that subblock (in this case, B1). ｉ This could be information that identifies the filter coefficient c. For example, the specific coefficient c of the filter. ｉ It may be a value representing the filter coefficient c. ｉ It may also be an index that specifies a set of values.

[0076] Furthermore, the information stored in the list includes information identifying the filter used in the EIP processing of that subblock (in this case, B1), and the filter coefficient c of that filter. ｉ This could be either information identifying the subject or information identifying the subject. The same applies to the other candidate information described below.

[0077] Similarly, the prediction unit 103 identifies whether the subblock containing the location of A1 is available and sets the result of this identification as availableFlagA1. Furthermore, the prediction unit 103 identifies whether EIP is used for the subblock containing the location of A1 and sets the result of this identification as EIPFlag[A1]. If the conditions are met that availableFlagA1 is 1 and EIPFlag[A1] is 1, the prediction unit 103 stores the information of candidate A1 (hereinafter referred to simply as candidate A1 as appropriate) in the EIP merge list (mergeEIPCandList). Candidate A1 contains information identifying the filter used in the EIP processing in the subblock containing the location of A1 and the coefficient c of the said filter. ｉ The information is linked.

[0078] Next, the prediction unit 103 determines whether the subblock containing the location of B0 is available. Furthermore, it sets the determined result as availableFlagB0. Furthermore, it determines whether EIP is used for the subblock containing the location of B0 and sets the determined result as EIPFlag[B0]. If the conditions are met that availableFlagB0 is 1 and EIPFlag[B0] is 1, the prediction unit 103 stores the information of candidate B0 (hereinafter, as appropriate, simply referred to as candidate B0) in the EIP merge list (mergeEIPCandList). Candidate B0 contains information that identifies the filter used in the EIP processing in the subblock containing the location of B0 and the coefficient c of that filter. ｉ The information is linked.

[0079] Next, the prediction unit 103 determines whether the subblock at position A0 is available.

[0080] Furthermore, the identified result is set as availableFlagA0. Next, it is determined whether EIP is used in the subblock at the location of A0, and the identified result is set as EIPFlag[A0]. If the conditions are met (availableFlagA0 is 1 and EIPFlag[A0] is 1), the prediction unit 103 stores the information of candidate A0 (hereinafter, as may be simply referred to as candidate A0) in the EIP merge list (mergeEIPCandList). Candidate A0 contains information identifying the filter used in the EIP processing in the subblock containing the location of A0, and the coefficient c of that filter. ｉ The information is linked.

[0081] Next, the prediction unit 103 determines whether the subblock containing the location of B2 is available. Furthermore, it sets the determined result as availableFlagB2. It then determines whether EIP is used for the subblock at the location of B2 and sets the determined result as EIPFlag[B2]. If the conditions are met that availableFlagB2 is 1 and EIPFlag[B2] is 1, the prediction unit 103 stores the information of candidate B2 (hereinafter, as appropriate, simply referred to as candidate B2) in the EIP merge list (mergeEIPCandList). Candidate B2 contains information that identifies the filter used in the EIP processing in the subblock containing the location of B2 and the coefficient c of that filter. ｉ The information is linked.

[0082] The prediction unit 103 then determines whether the subblock containing the location of C0 (block 702 in the example of Figure 7(a)) is available. Furthermore, it sets the determined result as availableFlagCol. It then determines whether EIP is used for the subblock containing the location of C0 and sets the determined result as EIPFlag[Col]. If the conditions are met that availableFlagCol is 1 and EIPFlag[Col] is 1, it stores the candidate Col information (hereinafter referred to as candidate Col as appropriate). Specifically, the prediction unit 103 stores the candidate Col information (hereinafter referred to as candidate Col as appropriate) in the EIP merge list (mergeEIPCandList). The candidate Col is linked to the information of the filter used in the EIP processing in the subblock containing the location of C0 and the coefficients of that filter. Furthermore, if the subblock containing the location of C0 is unavailable, or if EIP is not used in the subblock containing the location of C0, the following processing may be performed. For the subblock containing the location of C1, it may be further determined whether the subblock is available and whether EIP is used in the subblock. In this case, the prediction unit 103 determines whether the subblock containing the location of C1 is available. Furthermore, it sets the result of this determination as availableFlagCol. Furthermore, it determines whether EIP is used in the subblock containing the location of C1 and sets the result of this determination as EIPFlag[Col]. If the conditions are met that availableFlagCol is 1 and EIPFlag[Col] is 1, the prediction unit 103 stores the candidate Col in the EIP merge list (mergeEIPCandList). The candidate Col at this time is information that identifies the filter used in the EIP processing in the subblock containing the position of C1, and the coefficient c of that filter. ｉ The information is linked.

[0083] The prediction unit 103 may determine whether availableFlagCol is usable based on the values ​​of other syntax elements. For example, it may use the value of a flag (temporal_mvp_enabled_flag) that indicates whether temporal motion vector prediction is possible in interpretation. This flag is signaled to a predetermined parameter set (e.g., SPS: Sequence Parameter Set). Other examples of predetermined parameter sets include PPS: Picture Parameter Set, PH: Picture Header, SH: Slice Header, etc. The predetermined parameter set is a syntax element, and in this embodiment, it is signaled to SPS. Temporal motion vector prediction refers to the following process: an interpretation process that generates a predicted image of a target block using the motion vector used in a predetermined interpretation for the target block. The motion vector used is the motion vector used in interpretation for a block located collated with the target block (for example, block 702 relative to target block 701). The prediction unit 103 sets availableFlagCol, shown in Figure 7(b), to 1 if the flag indicates that temporal motion vector prediction is available in interpretation (temporal_mvp_enabled_flag = 1). On the other hand, the prediction unit 103 sets availableFlagCol, shown in Figure 7(b), to 0 if the flag indicates that temporal motion vector prediction is unavailable in interpretation (temporal_mvp_enabled_flag = 0). In this case, in EIP merge mode, blocks located collated with the target block are unavailable as candidates for the EIP merge list.

[0084] In the above description, as shown in Figure 7(b), the prediction unit 103 generates an EIP merge list for the target block 701 in the order B1→A1→B0→A0→B2→Col, but it is not limited to this. For example, the prediction unit 103 may generate an EIP merge list for the target block 701 in the order Col→B1→A1→B0→A0→B2.

[0085] Furthermore, after generating an EIP merge list as shown in Figures 7(a) and 7(b), the prediction unit 103 may add additional candidates according to the current number of candidates and the maximum number of candidates in the EIP merge list. Specifically, if the current number of candidates in the EIP merge list based on information from other blocks is less than the maximum number of candidates, new candidates (hereinafter referred to as additional candidates) are added to the EIP merge list until the number of candidates in the EIP merge list reaches the maximum number of candidates.

[0086] For example, if the number of candidates in the generated EIP merge list is 5 and the maximum number of candidates in the EIP merge list is 7, the prediction unit 103 adds two more candidates. The additional candidates added here are, for example, candidates associated with a predetermined filter (for example, the filter in Figure 3A) and a set of predetermined filter coefficients Ci for that predetermined filter. The predetermined filter coefficients Ci may be fixed values ​​where all values ​​are the same, or they may be predetermined coefficients generated in advance by machine learning.

[0087] Furthermore, if there are two or more candidates with the same filter among the candidates already stored in the EIP merge list, a new filter coefficient C is obtained by averaging the filter coefficients Ci of those two or more candidates. ｉ ｎｅｗ The set of filters is generated, and additional candidates associated with the set may be added to the EIP merge list. Specifically, if the same filter (for example, the filter in Figure 3A) is associated with the first candidate n and the second candidate m already stored in the EIP merge list, the prediction unit 103 uses equation (3) to calculate the filter coefficient C of the first candidate n. ｉ ｎ and the filter coefficient C of the second candidate m ｉｍ Averaging these together creates a new coefficient C ｉ ｎｅｗ Generates.

[0088]

[0089] The maximum number of candidates in the EIP merge list, MaxNumEIPCand, is signaled to the bitstream by the syntax element num_eip_merge_cand_minus2. num_eip_merge_cand_minus2 represents the value obtained by subtracting 2 from the maximum number of candidates in the EIP merge list, MaxNumEIPCand, and is signaled to a predetermined parameter set (SPS, PPS, PH, SH, etc.). Although num_eip_merge_cand_minus2 has been described as a syntax element representing the maximum number of candidates minus 2, it is not limited to this and may represent the maximum number of candidates minus other integer values ​​(1, 3, 4, 5, etc.).

[0090] As described above, the maximum number of candidates in this explanation functions as a mandatory target number. Even if the number of candidates based on information from other blocks falls short of the target number, generating additional candidates to create a list that always contains the same number (target number) of candidates makes the data easier to handle. Furthermore, always including the maximum number of candidates in the list increases the likelihood of selecting more appropriate information.

[0091] The prediction unit 103 generates an EIP merge list and then encodes the syntax element eip_merge_idx, which identifies the candidate to be used for the EIP of the target block from among the candidates in the generated EIP merge list, into a bitstream. eip_merge_idx is binarized by a predetermined binarization process and encoded using bypass coding or arithmetic coding. In this case, the first N bins (N is a positive integer) of the binarized sequence of eip_merge_idx may be encoded using arithmetic coding, and the rest may be encoded using bypass coding. For example, the first bin may be encoded using arithmetic coding, and the rest may be encoded using bypass coding. When arithmetic coding is used for one or more bins of the binarized sequence of eip_merge_idx, the context variables used for arithmetic coding depend on the following: The context variables depend, for example, on whether the subblock adjacent to the target block and the subblock adjacent to the left of the target block are using EIP.

[0092] Next, the encoding of the input image will be explained with reference to Figures 8A to 8F. The block division unit 102 divides the input image into multiple basic blocks (CTUs) and outputs each of the divided basic blocks. Furthermore, the block division unit 102 divides the basic blocks into multiple subblocks, and further divides the subblocks divided from the basic blocks into multiple subblocks. Figures 8A to 8F are diagrams showing the block division method by the block division unit 102.

[0093] Figure 8A shows the case where no further division is performed on a certain block 800 (basic block or subblock). Here, block 800 is described as having a size of 16 pixels × 16 pixels. Figure 8B shows a quadtree division, which is one method of subblock division. Figure 8B shows a division method (division pattern) in which block 800 is divided into four subblocks by dividing it in half horizontally and vertically. In this case, the size of each of the four subblocks is 8 pixels × 8 pixels. Figure 8C shows a division method in which block 800 is divided into two vertically elongated subblocks (each subblock having a size of 8 × 16 pixels) by dividing it in half vertically. Figure 8D shows a division method in which block 800 is divided into two horizontally elongated subblocks (each subblock having a size of 16 × 8 pixels) by dividing the basic block 700 in half horizontally. Figure 8E shows three horizontally elongated subblocks obtained by vertically dividing block 800 (from left to right: a subblock of 4x16 pixels, a subblock of 8x16 pixels, and a subblock of 4x16 pixels) (ternary tree partitioning).

[0094] Figure 8F shows three vertically elongated subblocks obtained by horizontally dividing block 800 (a subblock of 16 x 4 pixels, a subblock of 16 x 8 pixels, and a subblock of 16 x 4 pixels from top to bottom) (ternary tree partitioning). In this embodiment, any of the partitioning methods (partitioning patterns) shown in Figures 8B to 8F will be used, but it is not limited to Figures 8B to 8F. Note that, as shown in Figure 8A, block 800 may be used as a subblock without partitioning it.

[0095] Next, the encoding of syntax elements related to EIP will be explained with reference to Figures 9 to 10. Figure 9 is an example of the structure of a bitstream generated by the encoding unit 109 of the encoding device 100 in this embodiment. As shown in Figure 9, the bitstream includes an SPS (Sequence Parameter Set) containing parameters applied to the entire sequence. It also includes a PPS (Picture Parameter Set) containing parameters applied to one or more images, and a PH (Picture Header) containing parameters applied to one image. Furthermore, it includes Picture data containing the encoded data of the image. In addition, it may also include an SH (Slice Header) containing parameters applied to one slice.

[0096] Figure 10 shows the syntax structure included in the Picture data of the bitstream generated by the encoding unit 109, and is information that is signaled on a subblock (coding unit) basis. For example, the encoding unit 109 encodes the syntax elements shown in Figure 10 for the subblock (target block) currently being processed.

[0097] The eip_flag shown in Figure 10 is a flag indicating whether or not to execute EIP on the target block. An eip_flag value of 1 indicates that EIP will be executed on the target block, while an eip_flag value of 0 indicates that EIP will not be executed on the target block. Note that eip_flag is encoded into the coding_unit() syntax structure shown in Figure 10 when a predetermined condition, condition1, is true. condition1 may be, for example, a condition indicating whether a syntax element indicating that EIP is available is true or false, as shown below: condition1: eip_enabled_flag

[0098] The eip_enabled_flag is a syntax element that indicates that EIP is available and signals higher-level syntax structures in the bitstream than the coding_unit() syntax structure. For example, eip_enabled_flag is a flag that signals SPS, PPS, PH, SH, etc.

[0099] In this embodiment, eip_enabled_flag is a flag signaled to the SPS. A value of 1 for eip_enabled_flag indicates that EIP is available, condition1 becomes true, and eip_flag is encoded. On the other hand, a value of 0 for eip_enabled_flag indicates that EIP is unavailable, condition1 becomes false, eip_flag is not encoded, and the value of eip_flag is always 0. If condition1 is false, the value of eip_flag is estimated to be 0 by the decoding device (image decoding device) 200.

[0100] As another variation of condition1, condition1 may be a condition that further depends on the block size. In this case, for example, condition1 would be as follows: condition1 = eip_enabled_flag && Max(cbWidth, cbHeight) ≤ 32

[0101] In this modified example, condition1 is true if eip_enabled_flag is 1 and the larger of cbWidth and cbHeight is 32 or less. Otherwise, condition1 is false. Here, cbWidth is the width of the luminance component of the target block, and cbHeight is the height of the luminance component of the target block.

[0102] As another variation of condition1, condition1 may also be eip_enabled_flag&&Max(cbWidth, cbHeight)≦16.

[0103] Additionally, condition1 may be eip_enabled_flag&&Max(cbWidth, cbHeight)≦64, etc.

[0104] As another variation of condition1, condition1 may be a condition that depends on the color component of the current target block, as follows: condition1: cIdx = 0

[0105] cIdx is a variable that indicates the color component, where cIdx=0 corresponds to the luma component (Y component), cIdx=1 corresponds to the Cb chroma component (Cb component), and cIdx=2 corresponds to the Cr chroma component (Cr component). In the example of condition1 described above, if cIdx=0, the condition of condition1 is satisfied, and if not (if it is a chroma component), condition1 is not satisfied. In other words, if the target block currently under consideration is a luma component block, the condition of condition1 is satisfied, and if not (if it is a chroma component), condition1 is not satisfied. That is, in this modified example, eip_flag can only be encoded for the luma component.

[0106] Furthermore, as a variation of condition1, condition1 may be a condition that depends on the current position of the target block. If, for example, the filter shown in Figures 3A to 3H does not fit within the picture at the target block position, condition1 may be set so that EIP cannot be used.

[0107] If eip_flag is 1 (true), the encoding unit 109 encodes eip_merge_flag into coding_unit(). eip_merge_flag is a flag indicating whether to use the EIP merge mode for the target block. If eip_merge_flag is 1, the EIP merge mode is used for the target block, while if eip_merge_flag is 0, the EIP merge mode is not used for the target block. If eip_flag is 1 (true), the encoding unit 109 obtains the value of MaxNumEIPCand, which is the maximum number in the EIP merge list, and if this value is greater than 1, it further encodes eip_merge_idx into coding_unit(). The encoding unit 109 encodes num_eip_merge_cand_minus2, which is a syntax element for identifying the value of MaxNumEIPCand, into a predetermined parameter set. MaxNumEIPCand is identified by adding 2 to num_eip_merge_cand_minus2. In this embodiment, num_eip_merge_cand_minus2 is encoded into SPS. The maximum number of EIP merge lists and the maximum number of merge candidates for inter prediction may be the same. In this case, encoding of num_eip_merge_cand_minus2 may be omitted. Alternatively, the difference between the maximum number of EIP merge lists and the maximum number of merge candidates for inter prediction may be encoded. By doing so, it becomes possible to encode with a shorter code size than num_eip_merge_cand_minus2.

[0108] If eip_merge_flag is 0 (false) and condition2 is true, the encoding unit 109 encodes eip_filter_idx and eip_reconst_idx into the coding_unit() syntax structure. eip_filter_idx is a syntax element that identifies the filter used for EIP for the target block from among the multiple filters shown in Figures 3A to 3H. For example, eip_filter_idx=0 indicates the filter in Figure 3A, eip_filter_idx=1 indicates the filter in Figure 3B, and eip_filter_idx=2 indicates the filter in Figure 3C. eip_reconst_idx is a syntax element for identifying the reconstruction region used to generate filter coefficients in EIP for the target block from among the multiple reconstruction regions shown in Figures 4A to 4C. For example, eip_reconst_idx=0 indicates the reconstruction region in Figure 4A, eip_reconst_idx=1 indicates the reconstruction region in Figure 4B, and eip_reconst_idx=2 indicates the reconstruction region in Figure 4C. Note that condition2 may also be a condition that depends on the size of the target block, for example, as follows: condition2: cbwidth==4&&cbheight==4

[0109] Furthermore, condition2 is not limited to this; for example, cbwidth == 4 || cbheight == 4, min(cbwidth, cbheight) ≤ 4, etc. Also, condition2 may be a condition that depends on the position of the target block.

[0110] In this embodiment, if eip_flag is 1 and eip_merge_flag is 0 and condition2 is true, the following encoding process is performed: eip_filter_idx and eip_reconst_idx are encoded into codent_unit(). On the other hand, even if eip_flag is 1 and eip_merge_flag is 0, if condition2 is false, eip_filter_idx and eip_reconst_idx are not encoded. Furthermore, the values ​​of eip_filter_idx and eip_reconst_idx are always 0 (i.e., constrained to the value of 0). In other words, if the above condition2 is not met for the target block, the following process is performed. As part of the EIP processing for the target block, a filter corresponding to eip_filter_idx=0 and a reconstruction area corresponding to eip_reconst_idx=0 may be uniformly used. In this case, eip_filter_idx and eip_reconst_idx are not encoded and are estimated to be 0 by the decoding device, thus reducing the amount of code in the bitstream. In this embodiment, eip_filter_idx=0 corresponds to the filter in Figure 3A, and eip_reconst_idx=0 corresponds to the reconstruction area in Figure 4A. However, it is not limited to this, and eip_filter_idx=0 may correspond to the filter in Figure 3B or Figure 3C, and eip_reconst_idx=0 may correspond to the reconstruction area in Figure 4B or Figure 4C.

[0111] In this embodiment, eip_filter_idx=0 corresponds to the filter in Figure 3A. This is because the filter in Figure 3A is square. Compared to the rectangles in Figures 3B and 3C, the square filter is easier to use due to factors such as the position of the target block. Also, as an example, square filters tend to be used more frequently.

[0112] In this embodiment, eip_reconst_idx is described as being encoded, but this is not limited to this, and eip_reconst_idx does not need to be encoded. In this case, when the prediction unit 103 selects one of the filters shown in Figures 3A to 3H, it determines the reconstruction region to be used according to the selected filter. For example, if the prediction unit 103 selects the filter in Figure 3A as the filter to be used, it uniformly selects the reconstruction region in Figure 4A as the reconstruction region to be used. Similarly, if the prediction unit 103 selects the filter in Figure 3B as the filter to be used, it uniformly selects the reconstruction region in Figure 4B as the reconstruction region to be used. Similarly, if the prediction unit 103 selects the filter in Figure 3C as the filter to be used, it uniformly selects the reconstruction region in Figure 4C as the reconstruction region to be used. Similarly, if the prediction unit 103 selects the filters in Figures 3D to 3F as the filters to be used, it uniformly selects the reconstruction region in Figure 4A as the reconstruction region to be used. Similarly, if the prediction unit 103 selects the filter shown in Figure 3G as the filter to be used, it uniformly selects the reconstruction region shown in Figure 4B as the reconstruction region to be used. Similarly, if the prediction unit 103 selects the filter shown in Figure 3H as the filter to be used, it uniformly selects the reconstruction region shown in Figure 4C as the reconstruction region to be used. In this case, there is no need to encode eip_reconst_idx, and the prediction unit 103 encodes eip_filter_idx. That is, both the filter to be used and the reconstruction region can be selected by eip_filter_idx.

[0113] In this embodiment, the shapes of the filters and reconstruction regions specified by the syntax elements correspond to each other. For example, the filter shapes in Figures 3B and 3G correspond to the shapes of the reconstruction regions in Figure 4B, and when the filter in Figure 3B or 3G is used, the reconstruction region in Figure 4B is used. In other words, a horizontally elongated rectangular filter corresponds to a horizontally elongated rectangular reconstruction region. When using the square filter or a modified square filter in Figures 3A or 3D to 3F, the L-shaped reconstruction region in Figure 4A is used. Here, the L-shape in Figure 4A refers to a shape consisting of a first part located on the left side of the target block that is the target of intra-prediction processing, and a second part located on the upper side of the target block. When using the vertically elongated rectangular filter in Figure 3C or 3G, the vertically elongated rectangular reconstruction region in Figure 4C is used. In other words, the filter shape and the reconstruction region shape correspond to each other, and the selected filter and the reconstruction region corresponding to the filter are used for a predetermined intra-prediction process. In this embodiment, by specifying the reconstruction region according to the filter used, it becomes possible to use a reconstruction region suitable for the filter when the filter scans the reconstruction region and the target block. Furthermore, in this embodiment, multiple filter shapes may use the same reconstruction region. That is, the reconstruction region in Figure 4A may be used when using the filter in Figure 3A, and when using the filters in Figures 3D to 3F. Similarly, the reconstruction region in Figure 4B may be used when the filter shape in Figure 3G is used. Similarly, the reconstruction region in Figure 4C may be used when the filter shape in Figure 3H is used.

[0114] Here, the image encoding process by the encoding device 100 will be further explained according to the flowchart in Figure 11. First, in step S1101, the encoding unit 109 encodes the header information (SPS, PPH, PH, SH, etc.) necessary for image encoding. For example, it encodes num_eip_merge_cand_minus2, temporary_mvp_enabled_flag, eip_enabled_flag, etc. In this embodiment, num_eip_merge_cand_minus2, temporary_mvp_enabled_flag, and eip_enabled_flag are encoded into SPS. However, they are not limited to this and may be encoded into other parameter sets such as PPS, PH, SH, etc. Furthermore, when eip_enabled_flag is 0 (false), it is also possible to reduce the amount of code by omitting the encoding of num_eip_merge_cand_minus2. Next, in step S1102, the encoding unit 109 divides the input image into a plurality of basic blocks.

[0115] In step S1103, the prediction unit 103 selects one unselected basic block from the multiple basic blocks obtained in step S1102, according to the raster scan order starting from the top left edge of the image. The prediction unit 103 then determines a subblock division method (division pattern) for the selected basic block and divides the selected basic block into multiple subblocks according to the determined subblock division method. At this time, the prediction unit 103 divides the target basic block into multiple subblocks according to the division methods shown in Figures 8A to 8F, and if necessary, further divides the resulting subblocks according to the division methods shown in Figures 8A to 8F. In this way, the prediction unit 103 recursively divides the basic block into multiple subblocks. The prediction unit 103 also determines the prediction processing mode for each subblock.

[0116] Prediction processes used in this process include intra-prediction, inter-prediction, and EIP.

[0117] The prediction unit 103 then generates a predicted image for each subblock of the currently selected basic block by using the image in the frame memory 107 and making a prediction according to the determined prediction mode, and calculates the difference between the input image and the predicted image as the prediction error. When EIP is used for a certain target block, the prediction unit 103 generates a predicted image for that target block according to the method described above with reference to Figures 3A to 3H, 4A to 4C, 5, 6A, 6B, and 7. The prediction unit 103 also generates prediction information, such as information indicating the subblock division method and the prediction processing mode. When EIP is used, this prediction information includes, for example, a plurality of syntax elements as shown in Figure 10.

[0118] In step S1104, the transformation / quantization unit 104 generates transformation coefficients for each subblock by performing a transformation process on the prediction error for each subblock. Then, for each subblock, the transformation / quantization unit 104 quantizes the transformation coefficient of the subblock to generate the quantization coefficient of the subblock. In step S1105, the inverse quantization / inverse transformation unit 105 generates transformation coefficients by inverse quantizing the quantization coefficient of each subblock generated in step S306 using the quantization matrix used for the quantization of the subblock. Then, the inverse quantization / inverse transformation unit 105 performs an inverse transformation process on the generated transformation coefficients to generate the prediction error.

[0119] In step S1106, the image reconstruction unit 106 generates a predicted image from the image currently being processed or from a processed image stored in the frame memory 107, based on the predicted information generated by the prediction unit 103. The image reconstruction unit 106 then reconstructs the image from the predicted image and the prediction error generated by the inverse quantization / inverse transform unit 105. The image reconstruction unit 106 then stores the reconstructed image in the frame memory 107. In step S1107, the encoding unit 109 entropy encodes the quantization coefficients generated by the transform / quantization unit 104 and the predicted information generated by the prediction unit 103 to generate encoded data.

[0120] Furthermore, the encoding unit 109 multiplexes the header information and the generated encoded data to generate a bitstream. In step S1108, the control unit 101 determines whether all basic blocks have been encoded. If all basic blocks have been encoded, the process proceeds to step S1109. If there are still basic blocks that have not been encoded, the process proceeds to step S1103, where the next basic block to be processed is selected. In step S1109, the in-loop filter unit 108 performs in-loop filtering, such as deblocking filtering and sample adaptive offsetting, on the image stored in the frame memory 107.

[0121] As described above, in this embodiment, it is possible to encode the target block using EIP processing. While Figures 3A to 3H describe multiple patterns as types of EIP filters, the description is not limited to these. Below, we will describe a modified version of this embodiment in which other filter types can be used.

[0122] Figure 12 shows the types of filters used in the modified examples. The types of filters are divided into three size groups. Size group 0 includes the 4-pixel x 4-pixel filter shown in Figure 12(a), the 8-pixel x 2-pixel filter shown in Figure 12(b), and the 2-pixel x 8-pixel filter shown in Figure 12(c), similar to Figures 3A to 3C. Size group 1 includes the 3-pixel x 3-pixel filter shown in Figure 12(d). Furthermore, it includes the 4-pixel x 2-pixel filter shown in Figure 12(e) and the 2-pixel x 4-pixel filter shown in Figure 12(f). Furthermore, it includes the 8-pixel x 1-pixel filter shown in Figure 12(g) and the 1-pixel x 8-pixel filter shown in Figure 12(h). Furthermore, size group 2 includes the 2-pixel x 2-pixel filter shown in Figure 12(i), the 4-pixel x 1-pixel filter shown in Figure 12(j), and the 1-pixel x 4-pixel filter shown in Figure 12(k). Although the correspondence between each pixel position and the value of i is shown in Figure 12 for each filter, the correspondence between each pixel position and the value of i in each filter is not limited to the examples shown in Figure 12; other correspondences may also be possible.

[0123] When using a filter for size group 0, the prediction unit 103 generates predicted pixels for the target block using the above-described formula (1) or (2).

[0124] When using the filter shown in Figure 12(d) which belongs to size group 1, the prediction unit 103 generates predicted pixels for the target block using the following equation (4).

[0125]

[0126] As shown in equation (4), size group 1 is smaller than a predetermined size (size group 0). Therefore, unlike in equation (1), the prediction unit 103 creates a predicted image using the reconstructed pixel value or predicted pixel value corresponding to i=7, which is the position furthest from the predicted pixel position (lower right corner). In this way, depending on the size of the target block or filter, the prediction unit 103 may decide whether to use one or more pixels in the region that includes the pixel furthest from the predicted pixel position.

[0127] However, even in size group 1, pixels located far from the predicted pixel position may be excluded, as shown in equation (5) below.

[0128]

[0129] Depending on the characteristics of the image, the encoding device may decide which formula to use to predict the image and signal it to the bitstream using an index or similar method.

[0130] Furthermore, when using the filters shown in Figures 12(e) to (h) that belong to size group 1, the prediction unit 103 generates predicted pixels for the target block using the following equation (6).

[0131]

[0132] As shown in equation (6), size group 1 is smaller than a predetermined size (size group 0). Therefore, unlike in equation (1), the prediction unit 103 creates a predicted image using the reconstructed pixel value or predicted pixel value corresponding to i=7, which is the position furthest from the predicted pixel position (lower right corner). In this way, depending on the size of the target block or filter, the prediction unit 103 may decide whether to use one or more pixels in the region that includes the pixel furthest from the predicted pixel position.

[0133] However, even in size group 1, pixels located far from the predicted pixel position may be excluded, as shown in equation (7) below.

[0134]

[0135] Depending on the characteristics of the image, the encoding device may decide which formula to use to predict the image and signal it to the bitstream using an index or similar method.

[0136] As described above, within the same size group 1, filters with different pixel counts are mixed, as shown in Figure 12(d) and Figures 12(e) to (h). By grouping these together in this way, the amount of code used when signaling the group can be suppressed.

[0137] Furthermore, when using the filters shown in Figures 12(i) to (k) that belong to size group 2, the prediction unit 103 generates predicted pixels for the target block using the following equation (8).

[0138]

[0139] As shown in equation (8), size group 2 is smaller than the predetermined size (size group 0). Therefore, unlike in equation (1), the prediction unit 103 creates the predicted image using the reconstructed pixel value or predicted pixel value corresponding to i=2, which is the position furthest from the predicted pixel position (lower right corner). In this way, depending on the size of the target block or filter, the prediction unit 103 may decide whether to use one or more pixels in the region that includes the pixel furthest from the predicted pixel position.

[0140] However, even in size group 2, pixels located far from the predicted pixel position may not be used, as shown in equation (9) below.

[0141]

[0142] Depending on the characteristics of the image, the encoding device may decide which formula to use to predict the image and signal it to the bitstream using an index or similar method.

[0143] As described above, the prediction unit 103 calculates the filter coefficient c for the selected filter. ｉ When generating the filter coefficient c, use one of the reconstruction regions shown in Figures 4A to 4C. ｉ This will generate [something].

[0144] Here, we will explain the coding_unit() syntax structure in a modified example that allows the use of the filter shown in Figure 12. Figures 13A to 13C are examples of the coding_unit() syntax structure in a modified example that allows the use of the filter shown in Figure 12. Note that the same parts as in Figure 10 will not be explained. As a difference from the coding_unit() syntax structure in Figure 10, the coding_unit() syntax structure in Figures 13A to 13C newly includes eip_filter_size_idx. eip_filter_size_idx is a syntax element that identifies the size group of filters to be used for the target block.

[0145]

[0146] As shown in Table 1, eip_filter_size_idx = 0 indicates that size group 0 in Figure 12 is used. In this case, the range of possible values ​​for eip_filter_idx is 0 or greater and 2 or less. A value of 0 for eip_filter_idx represents the filter in Figure 12(a). A value of 1 for eip_filter_idx represents the filter in Figure 12(b). A value of 2 for eip_filter_idx represents the filter in Figure 12(c). Similarly, eip_filter_size_idx = 1 indicates that size group 1 in Figure 12 is used. In this case, the range of possible values ​​for eip_filter_idx is 0 or greater and 4 or less. A value of 0 for eip_filter_idx represents the filter in Figure 12(d). A value of 1 for eip_filter_idx indicates the filter in Figure 12(e). A value of 2 for eip_filter_idx indicates the filter in Figure 12(f). A value of 3 for eip_filter_idx indicates the filter in Figure 12(g). A value of 2 for eip_filter_idx indicates the filter in Figure 12(h). Furthermore, eip_filter_size_idx = 2 indicates that size group 2 in Figure 12 is used. In this case, the range of possible values ​​for eip_filter_idx is 0 or greater and 2 or less. A value of 0 for eip_filter_idx indicates the filter in Figure 12(i). A value of 1 for eip_filter_idx indicates the filter in Figure 12(j). The value of eip_filter_idx for 2 represents the filter shown in Figure 12(k).

[0147] In Figure 12, if eip_filter_size_idx, eip_filter_idx, and eip_reconst_idx are not encoded, each syntax element is constrained to a predetermined value. Here, the aforementioned cases of not being encoded include cases where condition2 is false, and the value of each syntax element is always 0 (constrained to the value of 0). When eip_flag is 1 and eip_merge_flag is 0 and condition2 is false, each syntax element takes the following value. In other words, in the above case, eip_filter_size_idx, eip_filter_idx, and eip_reconst_idx are each always 0. As a result, the filter in Figure 12(a) and the reconstruction region in Figure 4A are used.

[0148] By allowing the use of size groups 1 and 2 as second filters, in addition to size group 0, which has 16 sizes such as 4x4, as the first filter, it is possible to generate predictive images more flexibly and reduce the amount of data. Furthermore, by setting eip_filter_size_idx to a value from 0 to 4, multiple filters shown in Figure 3D may be added as size group 3. Also, multiple filters shown in Figure 3E may be added as size group 4. Also, multiple filters shown in Figure 3F may be added as size group 5.

[0149] (Second Embodiment) In this embodiment, a decoding device 200 that decodes a bitstream obtained by encoding an image using the encoding device 100 according to the first embodiment will be described. The differences from the first embodiment will be described below, and unless otherwise specified, it will be the same as the first embodiment. An example of the functional configuration of the decoding device 200 according to this embodiment will be described using the block diagram in Figure 2.

[0150] The control unit 201 controls the overall operation of the decoding device 200. The separation and decoding unit 202 acquires the bitstream generated by the encoding device 100, separates encoded data related to the decoding process and coefficients from the bitstream, and decodes the encoded data present in the bitstream header. For example, the separation and decoding unit 202 decodes num_eip_merge_cand_minus2, temporary_mvp_enabled_flag, and eip_enabled_flag from the SPS.

[0151] The decoding unit 203 obtains quantization coefficients and prediction information by decoding the encoded data separated from the bitstream by the separation decoding unit 202. The inverse quantization / inverse transformation unit 204 operates in the same way as the inverse quantization / inverse transformation unit 105 of the encoding device 100.

[0152] The inverse quantization / inverse transformation unit 204 obtains transformation coefficients by performing inverse quantization on the acquired quantization coefficients, and obtains the prediction error by performing an inverse transformation on these transformation coefficients.

[0153] The image reconstruction unit 205 performs prediction processing by referring to the image stored in the frame memory 206 based on the prediction information decoded from the bitstream by the decoding unit 203, and generates a predicted image. Then, the image reconstruction unit 205 reconstructs the image using the generated predicted image and the prediction error obtained by the inverse quantization / inverse transform unit 204, and stores the reconstructed image in the frame memory 206.

[0154] The in-loop filter unit 207 performs in-loop filtering, such as deblocking filtering, on the reconstructed image stored in the frame memory 206. The image stored in the frame memory 206 is output as appropriate by the control unit 201. The output destination of the image is not limited to a specific destination; for example, the reconstructed image may be displayed on the display screen of a display device such as a monitor, or the image may be output to a projection device such as a projector.

[0155] Next, the operation of the decoding device 200 having the above configuration (bitstream decoding process) will be described. In this embodiment, the bitstream input to the separation decoding unit 202 is assumed to be a bitstream for each frame of a moving image, but it may also be a bitstream of a still image.

[0156] Here, the prediction process performed by the image reconstruction unit 205 of the decoding device 200 will be described in more detail. The image reconstruction unit 205 is capable of performing the same prediction process as the prediction unit 103.

[0157] For example, when the image reconstruction unit 205 decodes eip_filter_idx and eip_reconst_idx from the coding_unit() syntax structure in Figure 10 for the current target block, it performs the following processing. That is, the image reconstruction unit 205 selects one of the filters in Figures 3A to 3H as the filter to be used in EIP for the target block, according to the value of eip_filter_idx. Furthermore, the image reconstruction unit 205 selects one of the reconstruction regions shown in Figures 4A to 4C for the target block, according to the decoded value of eip_reconst_idx, and generates the filter coefficients of the selected filter using the selected reconstruction region. Then, the image reconstruction unit 205 generates predicted pixels for the target block using one of the formulas described above.

[0158] Here, we will explain an example where the size of the target block is 4 pixels × 4 pixels, and a 4 pixel × 4 pixel filter as shown in Figure 3A is selected. In this case, the image reconstruction unit 205 positions the 4 pixel × 4 pixel filter 503 so that its position (x, y) is at the top left (0, 0) of the 4 pixel × 4 pixel target block 501, similar to the process described in the first embodiment. Then, according to one of the formulas described above, the predicted pixel value at position (x, y) is pre-determined. （ｘ，ｙ） The following is derived. At the position (x, y) shown in Figure 5(a), the predicted pixel value pred （ｘ，ｙ）After the formula is derived, the image reconstruction unit 205 shifts the position of the filter 503 by one pixel according to the scanning order shown by the arrow in Figure 5(a), and predicts the pixel value pred for the position (x, y) shown in Figure 5(b) according to one of the formulas described above. （ｘ，ｙ） This generates the following. As described above, the image reconstruction unit 205 shifts the selected filter one pixel at a time according to the scanning order shown by the arrows in Figure 5(a), and generates the predicted pixel values. （ｘ，ｙ） The data is acquired sequentially, and a predicted image of the target block is generated.

[0159] Note that the scanning order of the filter in the target block is not limited to the example shown in Figure 5(a), but may be the scanning order shown in Figures 6A and 6B. Figure 6A shows the order in which each row of pixels (hereinafter referred to as "pixel row") of an 8x8 target block is scanned horizontally. In this case, after the scan for predictive pixel generation is performed on the pixel row at the top of the 8x8 target block, the scan for predictive pixel generation may be performed on the pixel row immediately adjacent to that pixel row. That is, after the scan for predictive pixel generation is performed on a certain pixel row, the scan for predictive pixel generation may be performed on the pixel row immediately adjacent to that pixel row. In addition, predictive pixel generation is performed on the leftmost pixel in the pixel row at the top of the 8x8 target block. Subsequently, in parallel with the predictionive pixel generation being performed on the pixel adjacent to the right of that pixel according to the scanning order shown in Figure 6A, the predictionive pixel generation may be performed on the leftmost pixel of the pixel row adjacent to the pixel row below the top pixel row. In subsequent processing, as shown in Figure 6A, filter scanning for generating predicted pixels may be performed in parallel, shifting by one pixel for each pixel row of the target block. This makes it possible to perform parallel processing for each pixel row for generating predicted pixels.

[0160] Similarly, Figure 6B shows the order in which each pixel column (hereinafter referred to as "pixel column") of an 8x8 target block is scanned vertically. At this time, after the scan for predictive pixel generation is performed on the leftmost pixel column of the 8x8 target block, the scan for predictive pixel generation may be performed on the pixel column adjacent to the right of that pixel column. That is, after the scan for predictive pixel generation is performed on a certain pixel column, the scan for predictive pixel generation may be performed on the pixel column adjacent to the right of that pixel column. In addition, predictive pixel generation is performed on the top pixel of the leftmost pixel column of the 8x8 target block. Subsequently, in parallel with the predictionive pixel generation of the pixels adjacent below that pixel according to the scan order shown in Figure 6B, predictive pixel generation may be performed on the top pixel of the pixel column adjacent to the right of the leftmost pixel row. At this time, in the subsequent processing, as shown in Figure 6B, filter scans for predictive pixel generation may be performed in parallel, shifting by one pixel for each pixel column of the target block. In this way, it is possible to perform parallel processing for predictive pixel generation for each pixel row.

[0161] Furthermore, the scan order of the filters in the target block may be as shown in Figure 5(c). This scan order is the same as the scan order used for encoding the quantization matrix and quantization coefficients of HEVC and VVC, but by standardizing the scan order, it is possible to reduce the memory used for the table used for scanning.

[0162] Furthermore, in the above explanation, the filter coefficient c used in the EIP prediction process is described. ｉ The explanation described obtaining this using the reconstruction area surrounding the target block, but it is not limited to this. For example, when using EIP merge mode, the filter coefficient c used in blocks where prediction processing of EIPs other than the target block was performed. ｉ This may be used in the EIP prediction process for the target block.

[0163] If the image reconstruction unit 205 decodes an eip_merge_flag with a value of 1 for the target block, it decides to use EIP merge mode for that target block.

[0164] In this case, the image reconstruction unit 205 generates an EIP merge list (mergeEIPCandList) in the same manner as the prediction unit 103 in EIP merge mode. Specifically, it generates an EIP merge list from spatially adjacent or nearby blocks (blocks containing positions A0 to A1 / blocks containing positions B0 to B2) for the target block 701 in the current image. It also generates an EIP merge list for blocks in images at different times that are spatially corresponding to the target block 701.

[0165] Alternatively, an EIP merge list is generated for the target block 701 from adjacent / nearby blocks (blocks containing the position of C0 / blocks containing the position of C1). At this time, the prediction unit 103 of the encoding device 100 and the image reconstruction unit 205 of the decoding device 200 each generate the same EIP merge list for the target block. Note that the process of generating the EIP merge list by the image reconstruction unit 205 is the same as the process of generating the EIP merge list by the prediction unit 103 described in Embodiment 1, so the explanation is omitted. As mentioned above, the EIP merge list can include information of blocks surrounding the target block (including adjacent blocks) as candidates for information to be used in the target block. The EIP merge list can also include information of blocks (including adjacent blocks) at spatially corresponding positions to the target block or surrounding blocks in images that are different in time, as candidates.

[0166] The image reconstruction unit 205 adds 2 to num_eip_merge_cand_minus2, which is decoded from the bitstream, to derive the maximum number of candidates in the EIP merge list, MaxNumEIPCand. After generating an EIP merge list as shown in Figures 7(a) and 7(b), the image reconstruction unit 205 may further add candidates according to the current number of candidates and the maximum number of candidates in the EIP merge list. Specifically, if the current number of candidates in the EIP merge list is less than the maximum number of candidates, additional candidates may be added to the EIP merge list until the number of candidates in the EIP merge list reaches the maximum number of candidates. For example, if the number of candidates in the generated EIP merge list is 5 and the maximum number of candidates in the EIP merge list is 7, the image reconstruction unit 205 adds two more candidates. The additional candidates added here are, for example, candidates associated with a predetermined filter (for example, the filter in Figure 3A) and a predetermined set of filter coefficients Ci for that predetermined filter. The predetermined filter coefficient Ci may be a fixed value where all values ​​are the same, or it may be a predetermined coefficient generated in advance by machine learning. Furthermore, if there are two or more candidates with the same filter among those already stored in the EIP merge list, a new filter coefficient C may be obtained by averaging the filter coefficients Ci of those two or more candidates. ｉ ｎｅｗ The set of filters is generated, and additional candidates associated with the set may be added to the EIP merge list. Specifically, if the same filter (for example, the filter in Figure 3A) is associated with the first candidate n and the second candidate m already stored in the EIP merge list, the prediction unit 103 uses equation (7) to calculate the filter coefficient C of the first candidate n. ｉ ｎ and the filter coefficient C of the second candidate m ｉ ｍ Averaging these together creates a new coefficient C ｉ ｎｅｗ This generates the following. Equation (10) below is the same as equation (3) above.

[0167]

[0168] The decoding unit 203 decodes eip_merge_idx for the target block from the coding_unit() syntax structure of the bitstream. The image reconstruction unit 205 generates an EIP merge list and then identifies candidates to be used for the EIP of the target block from among the candidates in the generated EIP merge list, according to the value of eip_merge_idx. eip_merge_idx is binarized by a predetermined binarization process and encoded using bypass decoding or arithmetic decoding. In this case, arithmetic decoding may be used for the first N bins (N is a positive integer) of the binarized sequence of eip_merge_idx, and the rest may be bypass decoding. For example, arithmetic decoding may be used for the first bin and bypass decoding for the rest. When arithmetic decoding is used for one or more bins of the binarized sequence of eip_merge_idx, the context variables used depend on the following. The context variables used in arithmetic decoding depend, for example, on whether the subblocks adjacent to the target block and the subblock adjacent to the left of the target block are using EIP.

[0169] In this embodiment, the decoding unit 203 decodes information from the bitstream indicating how the basic blocks of the current image to be processed are divided. It also identifies the positions and sizes of multiple subblocks within each basic block of the current image according to this information. Furthermore, it decodes the coding_unit() syntax structure contained in the Picture data of the bitstream in Figure 9 to obtain prediction information for the target block.

[0170] The decoding unit 203 decodes eip_flag if condition1 is true. Note that condition1 is the same as described in the first embodiment, so its explanation is omitted. If condition1 is false, the decoding unit 203 estimates that the value of eip_flag for the target block is 0. In other words, the decoding unit 203 decides not to perform EIP on the target block.

[0171] If the decoded eip_flag is 1 (true), the decoder 203 decodes eip_merge_flag from coding_unit(). If the decoded eip_merge_flag is 1, the decoder 203 decides to use the EIP merge mode for the target block. On the other hand, if eip_merge_flag is 0, the decoder 203 decides not to use the EIP merge mode for the target block. If eip_flag is 1 (true), the decoder 203 obtains the value of MaxNumEIPCand, which is the maximum number in the EIP merge list, and if this value is greater than 1, it further decodes eip_merge_idx into coding_unit(). If MaxNumEIPCand is 1, eip_merge_idx is not decoded, and the value of eip_merge_idx is assumed to be 0. The decoding unit 203 is identified by adding 2 to num_eip_merge_cand_minus2 decoded from SPS.

[0172] If eip_merge_flag is 0 (false) and condition2 is true, the decoding unit 203 decodes eip_filter_idx and eip_reconst_idx from the coding_unit() syntax structure. In the embodiment of Figure 10, eip_filter_idx is a syntax element that identifies the filter to be used for the EIP for the target block from among the multiple filters shown in Figures 3A to 3H. For example, eip_filter_idx=0 indicates the filter in Figure 3A, eip_filter_idx=1 indicates the filter in Figure 3B, and eip_filter_idx=2 indicates the filter in Figure 3C. eip_reconst_idx is a syntax element used to identify the reconstruction region used to generate filter coefficients in the EIP for the target block from among the multiple reconstruction regions shown in Figures 4A to 4C. For example, eip_reconst_idx=0 indicates the reconstruction region in Figure 4A, eip_reconst_idx=1 indicates the reconstruction region in Figure 4B, and eip_reconst_idx=2 indicates the reconstruction region in Figure 4C. Note that condition2 is the same as described in the first embodiment, so its explanation is omitted.

[0173] In this embodiment, if eip_flag is 1 and eip_merge_flag is 0 and condition2 is true, the following decoding process is performed. The decoding unit 203 decodes eip_filter_idx and eip_reconst_idx from the codent_unit() syntax structure. On the other hand, even if eip_flag is 1 and eip_merge_flag is 0, if condition2 is false, the following decoding process is performed. In this case, eip_filter_idx and eip_reconst_idx are not decoded from the codent_unit() syntax structure. Furthermore, the decoding unit 203 estimates that the values ​​of eip_filter_idx and eip_reconst_idx are both 0 if the syntax structure described above cannot be decoded.

[0174] In other words, if the target block does not satisfy the conditions of condition 2 described above, the EIP processing for that target block may uniformly use a filter corresponding to eip_filter_idx=0 and a reconstruction area corresponding to eip_reconst_idx=0. In this embodiment, eip_filter_idx=0 corresponds to the filter in Figure 3A, and eip_reconst_idx=0 corresponds to the reconstruction area in Figure 4A. However, it is not limited to this, and eip_filter_idx=0 may correspond to the filter in Figure 3B or Figure 3C, and eip_reconst_idx=0 may correspond to the reconstruction area in Figure 4B or Figure 4C.

[0175] In this embodiment, eip_reconst_idx is described as being included in the syntax structure of Figure 10, but this is not limited to this, and eip_reconst_idx does not have to be encoded at all (i.e., it does not have to be decoded). In this case, the decoding unit 103 selects one of the filters shown in Figures 3A to 3H according to the value of eip_filter_idx, and then determines the reconstruction region according to the selected filter. For example, if the decoding unit 203 selects the filter in Figure 3A according to eip_filter_idx, it uniformly selects the reconstruction region in Figure 4A as the reconstruction region to be used. Similarly, if the decoding unit 203 selects the filter in Figure 3B according to eip_filter_idx, it uniformly selects the reconstruction region in Figure 4B as the reconstruction region to be used. Similarly, if the decoding unit 203 selects the filter in Figure 3C according to eip_filter_idx, it uniformly selects the reconstruction region in Figure 4C as the reconstruction region to be used. In such cases, there is no need to decrypt eip_reconst_idx, and the decryption unit 203 decrypts eip_filter_idx instead.

[0176] In other words, eip_filter_idx allows you to select both the filter to use and the reconstruction region.

[0177] Here, the decoding process of a single frame of bitstream by the decoding device 200 will be further explained according to the flowchart in Figure 14A. First, in step S1401, the separation and decoding unit 202 acquires a single frame of bitstream generated by the encoding device 100. The separation and decoding unit 202 then separates encoded data related to the decoding process and coefficients from the bitstream and decodes the encoded data present in the bitstream header. Here, the separation and decoding unit 202 decodes, for example, num_eip_merge_cand_minus2, temporary_mvp_enabled_flag, eip_enabled_flag, etc. from the SPS.

[0178] Next, in step S1402, the decoding unit 203 decodes information from the bitstream indicating how the basic block of the image currently being processed is divided. Furthermore, according to this information, it identifies the positions and sizes of the multiple subblocks obtained by the division in the basic block. Note that the division into multiple subblocks performed recursively in the basic block is the same as described with reference to Figures 8A to 8F, so the explanation is omitted.

[0179] From here on, steps 1403 to 1405 are processes performed for each subblock included in the basic block currently being processed. In step S1403, the inverse quantization / inverse transformation unit 204 obtains transformation coefficients by performing inverse quantization on the quantization coefficients of the subblock to be decoded, and obtains the prediction error by performing an inverse transformation on these transformation coefficients. In step S1404, the image reconstruction unit 205 generates a predicted image by performing a prediction process that refers to the image stored in the frame memory 206 based on the prediction information decoded for the subblock to be decoded by the decoding unit 203. Prediction processes used at this time include intra-prediction, inter-prediction, and EIP. When EIP is used for the subblock to be decoded, the image reconstruction unit 205 generates a predicted image for the target block in accordance with the method described above with reference to Figures 3A to 3H, 4A to 4C, 5, 6A, 6B, and 7. The image reconstruction unit 205 then uses the generated predicted image and the prediction error obtained by the inverse quantization / inverse transform unit 204 to decode the image of the subblock to be decoded, and stores the generated image of the subblock in the frame memory 206.

[0180] In step S1406, the control unit 201 determines whether or not the decoding of all basic blocks contained in the bitstream has been completed. If the result of this determination is that the decoding of all basic blocks contained in the bitstream has been completed, the process proceeds to step S1407. On the other hand, if there are still basic blocks in the bitstream that have not yet been decoded, the process proceeds to step S402 for those basic blocks that have not yet been decoded, and the process is repeated.

[0181] In step S1407, the in-loop filter unit 207 performs in-loop filtering, such as deblocking filtering, on the reconstructed image stored in the frame memory 206.

[0182] As described above, in this embodiment, it is possible to decode an image using EIP processing on the block to be decoded. In the above description, three patterns of EIP filters shown in Figures 3A to 3H were described, but this is not limited to these. As with the description in the first embodiment, it is possible to use the filter shown in Figure 12 as a modification of this embodiment. The following describes the case in which the filter shown in Figure 12 is used in this embodiment.

[0183] In this modified example, the decoding unit 203 can decode eip_filter_size_idx from the coding_unit() syntax structure shown in Figures 13A to 13C, which is the coding_unit() syntax structure for the subblock currently to be decoded. Specifically, the decoding unit 203 performs the following processing when the value of eip_flag decoded from the coding_unit() syntax structure shown in Figures 13A to 13C for the subblock to be decoded is 1 and the value of the decoded eip_merge_flag is 0. In this case, according to condition 2, eip_filter_size_idx, eip_filter_idx, and eip_reconst_idx are decoded from the coding_unit() syntax structure. Then, as shown in Table 1 in the first embodiment, the decoding unit 203 identifies the size group to be used according to the value of eip_filter_size_idx. Furthermore, from a plurality of filters belonging to that size group, it identifies the filter to be used from the decoded value of eip_filter_idx.

[0184] When using filters from size group 0, the image reconstruction unit 205 generates predicted pixels for the target block using equation (1) or (2). When using the filter shown in Figure 12(d) belonging to size group 1, the image reconstruction unit 205 generates predicted pixels for the target block using equation (4) or (5). When using the filters shown in Figures 12(e) to (h) belonging to size group 1, the image reconstruction unit 205 generates predicted pixels for the target block using equation (6) or (7). When using the filters shown in Figures 12(i) to (k) belonging to size group 2, the image reconstruction unit 205 generates predicted pixels for the target block using equation (8) or (9).

[0185] The image reconstruction unit 205 calculates the filter coefficient c for the selected filter. ｉWhen generating the reconstructed region, one of the reconstruction regions shown in Figures 4A to 4C is identified according to the decoded value of eip_reconst_idx, and the filter coefficient c of the selected filter is applied to the subblock to be decoded using the identified reconstruction region. ｉ This will generate [something].

[0186] Note that eip_filter_size_idx = 0 indicates that size group 0 in Figure 12 will be used. In this case, the range of possible values ​​for eip_filter_idx is 0 or greater and 2 or less. A value of 0 for eip_filter_idx indicates the filter in Figure 12(a). A value of 1 for eip_filter_idx indicates the filter in Figure 12(b). A value of 2 for eip_filter_idx indicates the filter in Figure 12(c). Similarly, eip_filter_size_idx = 1 indicates that size group 1 in Figure 12 will be used.

[0187] In this case, the range of possible values ​​for eip_filter_idx is 0 to 4. A value of 0 for eip_filter_idx represents the filter in Figure 12(d). A value of 1 for eip_filter_idx represents the filter in Figure 12(e). A value of 2 for eip_filter_idx represents the filter in Figure 12(f). A value of 3 for eip_filter_idx represents the filter in Figure 12(g). A value of 2 for eip_filter_idx represents the filter in Figure 12(h). Furthermore, eip_filter_size_idx = 2 indicates that size group 2 in Figure 12 is used. In this case, the range of possible values ​​for eip_filter_idx is 0 to 2. A value of 0 for eip_filter_idx represents the filter shown in Figure 12(i). A value of 1 for eip_filter_idx represents the filter shown in Figure 12(j). A value of 2 for eip_filter_idx represents the filter shown in Figure 12(k).

[0188] In Figure 12, if eip_filter_size_idx, eip_filter_idx, and eip_reconst_idx are not decoded, the decoding unit 203 estimates that the value of each syntax element is 0. The cases in which the syntax elements are not decoded include cases where condition2 is false. In particular, a case where eip_flag is 1 and eip_merge_flag is 0 and condition2 is false is an example. In this case, eip_filter_size_idx, eip_filter_idx, and eip_reconst_idx will always be 0. As a result, the image reconstruction unit 205 will use the filter in Figure 12(a) and the reconstruction region in Figure 4A for the subblock to be decoded.

[0189] While the above-described embodiment may be used, the following embodiment may also be used.

[0190] A Picture is divided into basic blocks called CTUs, and each CTU is further divided into blocks called Coding Units (CUs). A CU is a unit for efficiently processing video frames by dividing them into smaller parts. Each CU is further divided into even smaller units called Transform Units (TUs).

[0191] This section describes a configuration in which multiple filters used in EIP can be used for a single coding unit. Intra-predictive processing that can use multiple filters for a single coding unit is called multi-model, and intra-predictive processing that uses only one filter for a single coding unit is called non-multi-model.

[0192] This explanation will primarily use Figures 3A to 3H, and Figures 13B, 13C, 14B, and 14C. Figures 3A, 3B, and 3C show examples of filter shapes belonging to Group 1 (first filter). Group 1 includes a filter in which there are no blocks used for prediction in the region to the right of the lowest right predicted pixel position within the filter, and in the region below it. Figures 3D, 3E, and 3F show examples of filter shapes belonging to Group 2 (second filter). Group 2 includes a filter in which there are pixels in the region to the right of or below the lowest right predicted pixel position within the filter. The filters corresponding to Group 1 and Group 2 are each assigned an idx corresponding to eip_filter_idx.

[0193] Figure 14B is a flowchart showing whether or not to use the filter corresponding to Group 2 in the EIP prediction process.

[0194] In S1401b, the multi_model_enabled_flag is checked. The multi_model_enabled_flag is a flag indicating whether or not the use of multi-model is permitted. If the value of multi_model_enabled_flag is 1, the process proceeds to S1402b. If the value of multi_model_enabled_flag is 0, the process proceeds to S1403b.

[0195] In S1402b, the use of a filter corresponding to Group2 is permitted. According to Figure 13B, the signaling of additional_eip_filter_idx is performed. In other words, multiple filters can be applied to a single coding unit.

[0196] In S1403b, the use of the filter corresponding to Group1 is permitted, but the use of the filter corresponding to Group2 is not permitted. Only one filter can be applied to one coding unit.

[0197] On the other hand, another form will be explained using Figures 13C and 14C.

[0198] In S1401c of Figure 14C, the multi_model_enabled_flag is checked. The multi_model_enabled_flag is a flag indicating whether or not the use of multi-model is permitted. If the value of multi_model_enabled_flag is 1, the process proceeds to S1402c. If the value of multi_model_enabled_flag is 0, the process proceeds to S1405c.

[0199] The multi_model_enable_flag flag allows multiple filters to be applied to a single coding unit.

[0200] Proceeding to S1405c, one coding unit is permitted to use a filter corresponding to Group1, and one filter is used.

[0201] In S1402c, the additional_filter_enabled_flag is checked. If the value of additional_filter_enabled_flag is 1, the process proceeds to S1403c. If the value of additional_filter_enabled_flag is 0, the process proceeds to S1404c.

[0202] In S1403c, the use of the filter corresponding to Group2 is permitted. According to Figure 13C, the signaling of additional_eip_filter_idx is performed. In other words, multiple filters can be applied to a single coding unit, and one of these filters is the filter corresponding to Group2.

[0203] In S1404c, the use of filters corresponding to Group2 is not permitted, while the use of multiple filters corresponding to Group1 is permitted. EIP is executed using multiple filters for a single coding unit.

[0204] When multiple filters are used for a single coding unit, the configuration may determine which filter is used based on the position of the predicted pixel. For example, if the predicted pixel position is included in the lowest pixel row of the CTU, it is undesirable to perform EIP using a filter that predicts pixels in the region below the predicted pixel position. Therefore, it is preferable to use a configuration in which the prediction processing is performed using the filters included in Group 1.

[0205] Furthermore, if the predicted pixel position is included in the rightmost pixel column of the CTU, it is undesirable to perform EIP using a filter that uses pixels in the region to the right of the predicted pixel position for prediction. Therefore, it is preferable to configure the system to perform prediction processing using a filter included in Group 1.

[0206] If the process proceeds to S1403c, it is possible to not only add an IDX corresponding to the filter of Group 2 as a candidate, but also to decide on a single IDX corresponding to Group 1. For example, the filter corresponding to Group 1 may be a square filter. In this case, the filter size is preferably 4x4, but not limited to this. It may also be 8x8.

[0207] Furthermore, the multi_model_enabled_flag shown in Figures 13A to 13C and Figures 14A to 14C may not be a CU-level syntax structure, but rather may be incorporated into the syntax structure of the picture header or sequence parameter set.

[0208] Furthermore, when using multi-model, it is also possible to use different filters depending on the values ​​of the surrounding pixel positions of the predicted pixel. For example, one could take the average of the values ​​of the pixels to the left and above the predicted pixel position and use different filters depending on whether or not this value exceeds a predetermined threshold.

[0209] (Third Embodiment) All of the functional units shown in Figures 1 and 2 may be implemented in hardware, but some may be implemented in software (computer program). In that case, a computer device having frame memory 107 and frame memory 206 as memory devices and capable of executing the computer program can be applied to the above-mentioned encoding device 100 and decoding device 200. An example of the hardware configuration of a computer device applicable to the above-mentioned encoding device 100 and decoding device 200 will be explained using the block diagram in Figure 15.

[0210] The CPU 1501 executes various processes using computer programs and data stored in the RAM 1502 and ROM 503. In this way, the CPU 501 controls the operation of the entire computer system and also executes or controls the processes performed by the encoding device 100 and decoding device 200.

[0211] The RAM 1502 has an area for storing computer programs and data loaded from the ROM 1503 or external storage device 1506, and data received from the outside via the I / F (interface) 1507 (for example, the video and still image data mentioned above).

[0212] Furthermore, the RAM 1502 has a work area used by the CPU 1501 when executing various processes. In this way, the RAM 1502 can provide various areas as appropriate.

[0213] ROM1503 stores configuration data and startup programs for the computer device.

[0214] The operation unit 1504 is a user interface such as a keyboard, mouse, or touch panel, and the user can input various instructions to the CPU 1501 by operating it.

[0215] The display unit 1505 is composed of an LCD screen or a touch panel screen, and can display the processing results of the CPU 1501 as images or text. For example, the playback image decoded by the decoding device 100 may be displayed on this display unit 1505.

[0216] The display unit 1505 may also be a projection device such as a projector that projects images or text.

[0217] The external storage device 1506 is a large-capacity information storage device such as a hard disk drive. The external storage device 1506 stores the OS (operating system), computer programs and data that cause the CPU 1501 to execute or control the various processes described above performed by the encoding device 100 and the decoding device 200. The computer programs stored in the external storage device 1506 include computer programs that cause the CPU 1501 to realize the functions of each functional unit other than the frame memory 107 and frame memory 206.

[0218] Computer programs and data stored in the external storage device 1506 are loaded into the RAM 1502 as appropriate according to the control of the CPU 1501 and become subject to processing by the CPU 1501. The frame memory 107 and frame memory 206 mentioned above can be implemented using memory devices such as RAM 1502, ROM 1503, and external storage device 1506.

[0219] I / F 1507 functions as an interface for data communication with external devices. For example, moving images and still images can be acquired from an external server device or imaging device via I / F 1507 and stored in RAM 1502 or external storage device 1506.

[0220] The CPU 1501, RAM 1502, ROM 1503, operation unit 1504, display unit 1505, external storage device 1506, and I / F 1507 are all connected to the bus 1508. Note that the configuration shown in Figure 5 is merely one example of a hardware configuration for a computer device applicable to the above-mentioned encoding device 100 and decoding device 200, and various modifications and variations are possible.

[0221] (Other Embodiments) In other embodiments, a program that implements one or more of the functions of the above-described embodiments is supplied to a system or device via a network or storage medium. Furthermore, it can also be implemented by a process in which one or more processors in the computer of the system or device read and execute the program. It can also be implemented by a circuit (e.g., ASIC) that implements one or more functions.

[0222] (Configuration 1) An encoding device comprising: prediction means for deriving a plurality of predicted pixel values ​​included in a predicted image by applying a filter to a target block while moving the filter over it; and encoding means for encoding the target block using the predicted image, wherein the prediction means identifies a filter to be used from a plurality of filters for a predetermined intra-prediction process, and generates a predicted image by the predetermined intra-prediction process using the identified filter, the plurality of filters include a filter that uses at least one of a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in a region to the right of the position of the predicted pixel value to be derived in the predetermined intra-prediction process, and a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in a region below the position of the predicted pixel value to be derived, the prediction means identifies a reconstruction region corresponding to the identified filter from a plurality of reconstruction regions, derives the filter coefficient of the identified filter from the identified filter and the identified reconstruction region, and generates the predicted image by the predetermined intra-prediction using the identified filter and the filter coefficient.

[0223] (Configuration 2) The encoding device according to Configuration 1, characterized in that the encoding means encodes syntax elements for identifying the reconstruction region.

[0224] (Configuration 3) The encoding device according to Configuration 1, characterized in that the reconstruction region is identified by a syntax element for identifying the filter.

[0225] (Configuration 4) The encoding device according to Configuration 1, characterized in that the shape of the reconstruction region and the shape of the filter are associated.

[0226] (Configuration 5) The encoding device according to Configuration 1, wherein the filter used is a filter that uses a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in the region to the right of the position of the predicted pixel value to be derived in the predetermined intra-prediction processing, and a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in the region below the position of the predicted pixel value to be derived, wherein the identified reconstructed region has a shape consisting of a first part located to the left of the target block to be subjected to the intra-prediction processing and a second part located above the target block.

[0227] (Configuration 6) The encoding device according to Configuration 1, characterized in that, when the identified filter is an 8-pixel × 3-pixel size filter, the identified filter has a shape that includes pixel positions in the region below the predicted pixel value to be derived, and the identified reconstruction region has the shape of a vertically elongated rectangle.

[0228] (Configuration 7) The encoding device according to Configuration 1, characterized in that, when the identified filter is a 3-pixel × 8-pixel size filter, the identified filter has a shape that includes pixel positions in the region to the right of the position of the predicted pixel value to be derived, and the identified reconstruction region has the shape of a horizontally elongated rectangle.

[0229] (Configuration 8) The encoding device according to Configuration 1, characterized in that the size of the reconstruction region is determined depending on the size of the target block which is subject to the predetermined intra prediction processing.

[0230] (Configuration 9) The encoding device according to Configuration 8, characterized in that the height of the reconstruction region is determined depending on the height of the target block.

[0231] (Configuration 10) The encoding device according to Configuration 8, characterized in that the width of the reconstruction region is determined depending on the width of the target block.

[0232] (Configuration 11) The encoding device according to Configuration 1, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the identified filter while moving it.

[0233] (Configuration 12) The encoding device according to Configuration 1, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the identified filter while moving it to an area that includes at least a part of the area in the block to be subjected to the predetermined intra prediction process and the area surrounding the target block.

[0234] (Configuration 13) The encoding device according to Configuration 1, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the identified filter while moving it to an area that includes at least a part of the area in the block to be subjected to the predetermined intra prediction process and the area surrounding the block to be subjected.

[0235] (Configuration 14) A decoding device comprising: prediction means for deriving a plurality of predicted pixel values ​​included in a predicted image by applying a filter to a target block while moving the filter over it; and decoding means for decoding the target block using the predicted image, wherein the prediction means identifies a filter to be used from a plurality of filters for a predetermined intra-prediction process, and generates a predicted image by the predetermined intra-prediction process using the identified filter, the plurality of filters include a filter that uses at least one of a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in a region to the right of the position of the predicted pixel value to be derived in the predetermined intra-prediction process, and a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in a region below the position of the predicted pixel value to be derived, the prediction means identifies a reconstruction region corresponding to the identified filter from a plurality of reconstruction regions, derives a filter coefficient of the identified filter from the identified filter and the identified reconstruction region, and generates the predicted image by the predetermined intra-prediction using the identified filter and the filter coefficient.

[0236] (Configuration 15) The decoding device according to Configuration 14, characterized in that the decoding means encodes syntax elements for identifying the reconstruction region.

[0237] (Configuration 16) The decoding device according to Configuration 14, characterized in that the reconstruction region is identified by a syntax element for identifying the filter.

[0238] (Configuration 17) The decoding device according to Configuration 14, characterized in that the shape of the reconstruction region and the shape of the filter are associated.

[0239] (Configuration 18) The decoding device according to Configuration 14, characterized in that, when the identified filter is a 5-pixel x 5-pixel or 6-pixel x 6-pixel filter, the identified filter has a shape that includes pixel positions in the region to the right of the position of the predicted pixel value to be derived and pixel positions in the region below the predicted pixel value to be derived, and the identified reconstruction region has a shape that consists of a first part located to the left of the target block that is the subject of the intra prediction processing and a second part located above the target block.

[0240] (Configuration 19) The decoding device according to Configuration 14, characterized in that, when the identified filter is an 8-pixel × 3-pixel size filter, the identified filter has a shape that includes pixel positions in the region below the predicted pixel value to be derived, and the identified reconstruction region has the shape of a vertically elongated rectangle.

[0241] (Configuration 20) The decoding device according to Configuration 14, characterized in that, when the identified filter is a 3-pixel × 8-pixel size filter, the identified filter has a shape that includes pixel positions in the region to the right of the position of the predicted pixel value to be derived, and the identified reconstruction region has the shape of a horizontally elongated rectangle.

[0242] (Configuration 21) The decoding device according to Configuration 14, characterized in that the size of the reconstruction region is determined depending on the size of the target block which is subject to the predetermined intra prediction processing.

[0243] (Configuration 22) The decoding device according to Configuration 21, characterized in that the size of the reconstruction region is determined depending on the size of the target block that is subject to the predetermined intra prediction processing.

[0244] (Configuration 23) The decoding device according to Configuration 21, characterized in that the height of the reconstruction region is determined depending on the height of the target block.

[0245] (Configuration 24) The decoding device according to Configuration 21, characterized in that the width of the reconstruction region is determined depending on the width of the target block.

[0246] (Configuration 25) The decoding device according to Configuration 14, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the identified filter while moving it.

[0247] (Configuration 26) The decoding device according to Configuration 14, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the identified filter while moving it to an area that includes at least a part of the area in the block to be subjected to the predetermined intra prediction process and the area surrounding the target block.

[0248] (Configuration 27) The decoding device according to Configuration 14, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the identified filter while moving it to an area that includes at least a part of the area in the block to be subjected to the predetermined intra prediction process and the area surrounding the block to be subjected.

[0249] (Method 1) An encoding method comprising: a prediction step of deriving a plurality of predicted pixel values ​​included in a predicted image by applying a filter to a target block while moving it over the block; and an encoding step of encoding the target block using the predicted image, wherein in the prediction step, one filter is identified from a plurality of filters for a predetermined intra prediction process, and a predicted image is generated by the predetermined intra prediction process using the identified filter, the plurality of filters include a filter shape that includes pixel positions included in the region to the right of the position of the predicted pixel value to be derived in the predetermined intra prediction process and pixel positions included in the region below the position of the predicted pixel value to be derived, one of a plurality of reconstruction regions for the target block is identified according to the identified filter from the plurality of filters, filter coefficients of the identified filter are generated from the identified filter and the identified reconstruction region, and the predicted image is generated by the predetermined intra prediction using the identified filter and the filter coefficients.

[0250] (Method 2) A decoding method comprising: a prediction step of deriving a plurality of predicted pixel values ​​included in a predicted image by applying a filter to a target block while moving it over the block; and a decoding step of decoding the target block using the predicted image, wherein in the prediction step, one filter is identified from a plurality of filters for a predetermined intra-prediction process, and a predicted image is generated by the predetermined intra-prediction process using the identified filter, the plurality of filters include a filter shape that includes pixel positions included in the region to the right of the position of the predicted pixel value to be derived in the predetermined intra-prediction process and pixel positions included in the region below the position of the predicted pixel value to be derived, one of a plurality of reconstruction regions for the target block is identified according to the identified filter, filter coefficients of the identified filter are generated from the identified filter and the identified reconstruction region, and the predicted image is generated by the predetermined intra-prediction using the identified filter and the filter coefficients.

[0251] (Program 1) A program characterized by causing a computer to function as each means of the encoding device described in Configuration 1.

[0252] (Program 2) A program characterized by causing a computer to function as each of the means of the decoding device described in configuration 14.

[0253] The present invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are attached to make the scope of the invention public.

[0254] This application claims priority based on Japanese Patent Application No. 2025-053782, filed on 27 March 2025, and all of its contents are incorporated herein by reference.

[0255] 102 Block division unit 103 Prediction unit 104 Transformation / quantization unit 105 Inverse quantization / inverse transform unit 106 Image reconstruction unit 107 Frame memory 108 In-loop filter unit 109 Encoding unit

Claims

1. An encoding device comprising: prediction means for deriving a plurality of predicted pixel values ​​included in a predicted image by applying a filter to a target block while moving the filter over it; and encoding means for encoding the target block using the predicted image, wherein the prediction means identifies a filter to be used from a plurality of filters for a predetermined intra-prediction process, and generates a predicted image by the predetermined intra-prediction process using the identified filter, the plurality of filters include a filter that uses at least one of a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in a region to the right of the position of the predicted pixel value to be derived in the predetermined intra-prediction process, and a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in a region below the position of the predicted pixel value to be derived, the prediction means identifies a reconstruction region corresponding to the identified filter from a plurality of reconstruction regions, derives a filter coefficient of the identified filter from the identified filter and the identified reconstruction region, and generates the predicted image by the predetermined intra-prediction using the identified filter and the filter coefficient.

2. The encoding apparatus according to claim 1, characterized in that the encoding means encodes syntax elements for identifying the reconstruction region.

3. The encoding device according to claim 1, characterized in that the reconstruction region is identified by a syntax element for identifying the filter.

4. The encoding device according to claim 1, characterized in that the shape of the reconstruction region and the shape of the filter are associated.

5. The encoding device according to claim 1, wherein the filter used is a filter that uses a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in the region to the right of the position of the predicted pixel value to be derived in the predetermined intra-prediction processing, and a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in the region below the position of the predicted pixel value to be derived, the identified reconstructed region has a shape consisting of a first part located to the left of the target block to be subjected to the intra-prediction processing and a second part located above the target block.

6. The encoding device according to claim 1, characterized in that, when the identified filter is an 8-pixel × 3-pixel size filter, the identified filter has a shape that includes pixel positions in the region below the predicted pixel value to be derived, and the identified reconstruction region has a vertically elongated rectangular shape.

7. The encoding device according to claim 1, characterized in that, when the identified filter is a 3x8 pixel size filter, the identified filter has a shape that includes pixel positions in the region to the right of the position of the predicted pixel value to be derived, and the identified reconstruction region has a horizontally elongated rectangular shape.

8. The encoding device according to claim 1, characterized in that the size of the reconstruction region is determined depending on the size of the target block that is subject to the predetermined intra prediction processing.

9. The encoding device according to claim 8, characterized in that the height of the reconstruction region is determined depending on the height of the target block.

10. The encoding device according to claim 8, characterized in that the width of the reconstruction region is determined depending on the width of the target block.

11. The encoding device according to claim 1, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the identified filter while moving it.

12. The encoding device according to claim 1, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the identified filter while moving it to an area that includes at least a portion of the area in the block to be subjected to the predetermined intra prediction process and the area surrounding the target block.

13. The encoding device according to claim 1, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the identified filter while moving it over an area that includes at least a portion of the area in the block to be subjected to the predetermined intra prediction process and the area surrounding the block to be subjected.

14. A decoding device comprising: prediction means for deriving a plurality of predicted pixel values ​​included in a predicted image by applying a filter to a target block while moving the filter over it; and decoding means for decoding the target block using the predicted image, wherein the prediction means identifies a filter to be used from a plurality of filters for a predetermined intra-prediction process, and generates a predicted image by the predetermined intra-prediction process using the identified filter, the plurality of filters include a filter that uses at least one of a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in a region to the right of the position of the predicted pixel value to be derived in the predetermined intra-prediction process, and a reconstructed pixel value or predicted pixel value corresponding to a pixel position included in a region below the position of the predicted pixel value to be derived, the prediction means identifies a reconstruction region corresponding to the identified filter from a plurality of reconstruction regions, derives a filter coefficient of the identified filter from the identified filter and the identified reconstruction region, and generates the predicted image by the predetermined intra-prediction using the identified filter and the filter coefficient.

15. The decoding device according to claim 14, characterized in that the decoding means encodes syntax elements for identifying the reconstruction region.

16. The decoding device according to claim 14, characterized in that the reconstruction region is identified by a syntax element for identifying the filter.

17. The decoding device according to claim 14, characterized in that the shape of the reconstruction region and the shape of the filter are associated.

18. The decoding device according to claim 14, wherein, when the identified filter is a 5-pixel × 5-pixel or 6-pixel × 6-pixel filter, the identified filter has a shape that includes pixel positions in the region to the right of the position of the predicted pixel value to be derived and pixel positions in the region below the predicted pixel value to be derived, and the identified reconstruction region has a shape that consists of a first part located to the left of the target block subject to the intra prediction processing and a second part located above the target block.

19. The decoding device according to claim 14, characterized in that, when the identified filter is an 8-pixel × 3-pixel size filter, the identified filter has a shape that includes pixel positions in the region below the predicted pixel value to be derived, and the identified reconstruction region has a vertically elongated rectangular shape.

20. The decoding device according to claim 14, characterized in that, when the identified filter is a 3-pixel × 8-pixel size filter, the identified filter has a shape that includes pixel positions in the region to the right of the position of the predicted pixel value to be derived, and the identified reconstruction region has a horizontally elongated rectangular shape.

21. The decoding device according to claim 14, characterized in that the size of the reconstruction region is determined depending on the size of the target block that is subject to the predetermined intra prediction processing.

22. The decoding device according to claim 21, characterized in that the size of the reconstruction region is determined depending on the size of the target block that is subject to the predetermined intra prediction processing.

23. The decoding device according to claim 21, characterized in that the height of the reconstruction region is determined depending on the height of the target block.

24. The decoding device according to claim 21, characterized in that the width of the reconstruction region is determined depending on the width of the target block.

25. The decoding device according to claim 14, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the identified filter while moving it.

26. The decoding device according to claim 14, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the identified filter while moving it to a region that includes at least a portion of the region in the block to be subjected to the predetermined intra prediction process and the region surrounding the target block.

27. The decoding device according to claim 14, characterized in that the predetermined intra prediction process is a prediction process that generates the predicted image by applying the identified filter while moving it over an area that includes at least a portion of the area in the block to be subjected to the predetermined intra prediction process and the area surrounding the block to be subjected.

28. An encoding method comprising: a prediction step of deriving a plurality of predicted pixel values ​​included in a predicted image by applying a filter to a target block while moving it over it; and an encoding step of encoding the target block using the predicted image, wherein in the prediction step, one filter is identified from a plurality of filters for a predetermined intra-prediction process, and a predicted image is generated by the predetermined intra-prediction process using the identified filter, the plurality of filters include a filter shape that includes pixel positions included in the region to the right of the position of the predicted pixel value to be derived in the predetermined intra-prediction process and pixel positions included in the region below the position of the predicted pixel value to be derived, one of a plurality of reconstruction regions for the target block is identified according to the identified filter, filter coefficients of the identified filter are generated from the identified filter and the identified reconstruction region, and the predicted image is generated by the predetermined intra-prediction using the identified filter and the filter coefficients.

29. A decoding method comprising: a prediction step of deriving a plurality of predicted pixel values ​​included in a predicted image by applying a filter to a target block while moving it over it; and a decoding step of decoding the target block using the predicted image, wherein in the prediction step, one filter is identified from a plurality of filters for a predetermined intra-prediction process, and a predicted image is generated by the predetermined intra-prediction process using the identified filter, the plurality of filters include a filter shape that includes pixel positions included in the region to the right of the position of the predicted pixel value to be derived in the predetermined intra-prediction process and pixel positions included in the region below the position of the predicted pixel value to be derived, one of a plurality of reconstruction regions for the target block is identified according to the identified filter, filter coefficients of the identified filter are generated from the identified filter and the identified reconstruction region, and the predicted image is generated by the predetermined intra-prediction using the identified filter and the filter coefficients.

30. A program characterized by causing a computer to function as each means of the encoding device described in claim 1.

31. A program characterized by causing a computer to function as each of the means of the decoding device described in claim 14.

Images