Intra prediction device, encoding device, decoding device, and program

Abstract

This intra prediction device, which performs intra prediction in units of blocks obtained by dividing an image, comprises: a region specification means for specifying a decoded region around a target block for intra prediction as a prediction trial region, and specifying a decoded region adjacent to the prediction trial region as a reference region; a cost calculation means for calculating a cost using a cost function by performing prediction trial for predicting the prediction trial region from the reference region for each intra prediction mode; and a prediction mode determination means for determining at least one intra prediction mode to be used for intra prediction according to the cost. The prediction trial region includes a plurality of sub-regions classified according to a relative positional relationship with the target block. The cost calculation means performs weighting for each sub-region when calculating the cost of each intra prediction mode.

Description

Intra prediction device, encoding device, decoding device, and program

[0001] The present disclosure relates to an intra prediction device, an encoding device, a decoding device, and a program.

[0002] In video coding methods such as High Efficiency Video Codec (HEVC) and Versatile Video Coding (VVC), a coding device generates a prediction block, which is a predicted image for a coding block (CU: Coding Unit) obtained by dividing an original image into blocks, and then transforms, quantizes, and entropy-codes the difference between the coding block of the original image and the prediction block before transmitting it.

[0003] As a method for generating a predicted image, inter-prediction that utilizes correlation between frames and intra-prediction that utilizes correlation within a frame are available. Intra-prediction methods include planar prediction and DC prediction, as well as directional prediction, which is a linear prediction, and HEVC allows selection from 33 directions, while VVC allows selection from 65 directions.

[0004] In intra prediction, a technique has been researched in which a peripheral area of ​​a block to be coded is used as a trial prediction area (also referred to as a "template") and reference pixels for the trial prediction area to derive an intra prediction mode, and this intra prediction mode is then used for prediction (see, for example, Non-Patent Document 1). This technique utilizes the fact that the cost calculations for intra prediction in the trial prediction area in both the coding device and the decoding device are consistent. Such a technique can, for example, reduce the signaling overhead of intra prediction modes and derive additional directional predictions that subdivide conventional prediction directions to create new prediction directions.

[0005] JVET-C0061 “Decoder-side intra mode derivation”

[0006] An intra prediction device according to a first aspect performs intra prediction on a block-by-block basis by dividing an image, and includes: a region identification unit that identifies a decoded region surrounding a current block for the intra prediction as a trial prediction region and identifies a decoded region adjacent to the trial prediction region as a reference region; a cost calculation unit that calculates a cost using a cost function by performing a trial prediction of the trial prediction region from the reference region for each intra prediction mode; and a prediction mode determination unit that determines at least one intra prediction mode to be used for the intra prediction based on the cost. The trial prediction region includes a plurality of subregions classified according to their relative positional relationship with the current block. The cost calculation unit weights each subregion when calculating the cost.

[0007] A coding device according to a second aspect includes the intra prediction device according to the first aspect.

[0008] A decoding device according to a third aspect includes the intra prediction device according to the first aspect.

[0009] A program according to a fourth aspect causes a computer to function as the intra prediction device according to the first aspect.

[0010] FIG. 1 is a diagram showing a configuration of an encoding device according to an embodiment. FIG. 2 is a diagram explaining an intra prediction mode according to an embodiment. FIG. 3 is a diagram explaining TIMD according to an embodiment. FIG. 4 is a diagram showing a configuration of an intra prediction unit according to an embodiment. FIG. 5 is a diagram explaining operation of the intra prediction unit according to an embodiment. FIG. 6 is a diagram showing a configuration of a decoding device according to an embodiment. FIG. 7 is a diagram showing a configuration of an intra prediction unit according to an embodiment. FIG. 8 is a diagram showing an example of operation of the intra prediction unit according to an embodiment. FIG. 9 is a diagram explaining a modified example of TIMD according to an embodiment.

[0011] In the intra-prediction mode derivation technique using a template, a prediction mode that has the best result in the cost calculation of intra-prediction in the trial prediction region on the encoding device side and the decoding device side, or a prediction mode whose cost calculation result exceeds a threshold even if the derivation is in progress, is selected. However, such a technique has room for improvement in terms of improving the prediction accuracy of intra-prediction.

[0012] Therefore, the present disclosure provides an intra prediction device, an encoding device, a decoding device, and a program that can improve the prediction accuracy in intra prediction.

[0013] An encoding device and a decoding device including an intra prediction device according to an embodiment will be described with reference to the drawings. The encoding device and the decoding device respectively encode and decode video (i.e., moving images) represented by MPEG. In the following description of the drawings, identical or similar parts are denoted by identical or similar reference numerals.

[0014] (1) Encoding Device The encoding device according to this embodiment will be described with reference to FIGS.

[0015] (1.1) Configuration of the Encoding Device First, the configuration of the encoding device 1 according to this embodiment will be described. Fig. 1 is a diagram showing the configuration of the encoding device 1 according to this embodiment.

[0016] The encoding device 1 is a device that encodes an input image to generate a bitstream and outputs the bitstream. The encoding device 1 includes a block division unit 100, a subtraction unit 110, a transform / quantization unit 120, an entropy encoding unit 130, an inverse quantization / inverse transform unit 140, a synthesis unit 150, a memory 160, and a prediction unit 170.

[0017] The block division unit 100 divides an original image, which is an input image in units of frames (or pictures) constituting a moving image, into a plurality of image blocks, and outputs the image blocks obtained by division to the subtraction unit 110. The size of the image blocks is, for example, 32 x 32 pixels, 16 x 16 pixels, 8 x 8 pixels, or 4 x 4 pixels. The shape of the image blocks is not limited to squares and may be rectangular (non-square). An image block is a unit for encoding by the encoding device 1 and a unit for decoding by the decoding device. Such an image block is also called a coding block (CU).

[0018] The input image is composed of a luminance signal (Y) and color difference signals (Cb, Cr), and each pixel in the input image is composed of a luminance component (Y) and a color difference component (Cb, Cr). The encoding device 1 supports three color difference formats, for example, 4:4:4, 4:2:2, and 4:2:0. The block division unit 100 performs block division on the luminance signal to output a luminance block, and performs block division on the color difference signal to output a color difference block. The shape of the block division may be the same for the luminance signal and the color difference signal, or the shape of the block division may be independently controllable for the luminance signal and the color difference signal.

[0019] The subtraction unit 110 calculates a prediction residual that represents the difference (error) between the coded block output by the block division unit 100 and a predicted block obtained by predicting the coded block by the prediction unit 170. Specifically, the subtraction unit 110 calculates the prediction residual by subtracting each pixel value of the predicted block from each pixel value of the block, and outputs the calculated prediction residual to the transformation / quantization unit 120.

[0020] The transform / quantization unit 120 performs transform processing and quantization processing on a block-by-block basis. The transform / quantization unit 120 includes a transform unit 121 and a quantization unit 122.

[0021] The transform unit 121 performs a transform process on the prediction residuals output by the subtraction unit 110 to calculate transform coefficients, and outputs the calculated transform coefficients to the quantization unit 122. Examples of transform include a discrete cosine transform (DCT), a discrete sine transform (DST), and a Karhunen-Loeve transform (KLT). The transform process includes a transform skip in which no transform process is performed. The transform skip also includes a transform in which only a horizontal transform process is applied, or a transform in which only a vertical transform process is applied. The transform unit 121 may also perform a secondary transform process in which a further transform process is applied to the transform coefficients obtained by the transform process. The secondary transform process may be applied to only a partial region of the transform coefficients.

[0022] The quantization unit 122 quantizes the transform coefficients output by the transform unit 121 using a quantization parameter and a quantization matrix, and outputs the quantized transform coefficients to the entropy coding unit 130 and the inverse quantization and inverse transform unit 140. The quantization parameter is a parameter that is commonly applied to each transform coefficient in a block and determines the coarseness of quantization. The quantization matrix is ​​a matrix whose elements are quantization values ​​used when quantizing each transform coefficient.

[0023] The entropy coding unit 130 performs entropy coding on the quantized transform coefficients output by the quantization unit 122, compresses the data, generates a bitstream, and outputs the bitstream to the outside of the coding device 1. For entropy coding, Huffman coding, CABAC (Context-based Adaptive Binary Arithmetic Coding), etc. may be used. Note that the entropy coding unit 130 receives input of prediction-related information (flags and indexes) from the prediction unit 170, and also encodes the input information and outputs a bitstream.

[0024] The inverse quantization and inverse transform unit 140 performs inverse quantization processing and inverse transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 140 includes an inverse quantization unit 141 and an inverse transform unit 142.

[0025] The inverse quantization unit 141 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122. Specifically, the inverse quantization unit 141 reconstructs the transform coefficients by inverse quantizing the quantized transform coefficients output by the quantization unit 122 using a quantization parameter and a quantization matrix, and outputs the reconstructed transform coefficients to the inverse transform unit 142.

[0026] The inverse transform unit 142 performs inverse transform processing corresponding to the transform processing performed by the transform unit 121. For example, if the transform unit 121 performs discrete cosine transform, the inverse transform unit 142 performs inverse discrete cosine transform. The inverse transform unit 142 performs inverse transform processing on the transform coefficients output by the inverse quantization unit 141 to reconstruct prediction residuals, and outputs the reconstructed prediction residuals to the synthesis unit 150.

[0027] The synthesis unit 150 synthesizes the reconstructed prediction residual output by the inverse transform unit 142 by adding it pixel by pixel to the predicted block output by the prediction unit 170. The synthesis unit 150 adds each pixel value of the reconstructed prediction residual to each pixel value of the predicted block to decode (reconstruct) the block, and outputs the reconstructed block to the memory 160. Hereinafter, the reconstructed block is also referred to as a decoded block.

[0028] The memory 160 stores the reconstructed blocks output by the synthesis unit 150 and accumulates the reconstructed blocks as decoded images on a frame-by-frame basis. The memory 160 outputs the stored reconstructed blocks or decoded images to the prediction unit 170. Note that a loop filter may be provided between the synthesis unit 150 and the memory 160.

[0029] The prediction unit 170 performs prediction in units of blocks and includes an inter prediction unit 171, an intra prediction unit 172, and a switching unit 173.

[0030] The inter prediction unit 171 uses a decoded image stored in the memory 160 as a reference image to calculate a motion vector by a technique such as block matching, predicts a coding block to generate an inter prediction block, and outputs the generated inter prediction block to the switching unit 173. Here, the inter prediction unit 171 selects an optimal inter prediction method from inter prediction using multiple reference images (typically, bi-prediction) and inter prediction using one reference image (unidirectional prediction), and performs inter prediction using the selected inter prediction method. The inter prediction unit 171 outputs information related to the inter prediction (motion vectors, etc.) to the entropy coding unit 130.

[0031] The intra prediction unit 172 generates an intra prediction block by referring to decoded pixels surrounding a block in the decoded image stored in the memory 160, and outputs the generated intra prediction block to the switching unit 173. Generally, the intra prediction unit 172 selects an intra prediction mode to be applied to a predicted coding block of intra prediction from a plurality of intra prediction modes, and predicts the coding block of intra prediction using the selected intra prediction mode. The intra prediction unit 172 outputs information related to the selected intra prediction mode to the entropy coding unit 130.

[0032] The switching unit 173 switches between the inter-predicted block output by the inter-prediction unit 171 and the intra-predicted block output by the intra-prediction unit 172 , and outputs either the predicted block to the subtraction unit 110 and the synthesis unit 150 .

[0033] 2 is a diagram illustrating intra prediction modes according to this embodiment. The intra prediction unit 172 performs intra prediction on a coding block. In the example shown in the figure, the candidates for the intra prediction mode of a luminance block are planar prediction, DC prediction, and 65 directional predictions, for a total of 67 intra prediction modes.

[0034] Prediction mode mode "0" is planar prediction, prediction mode mode "1" is DC prediction, and prediction mode modes "2" to "66" are directional prediction. In directional prediction, the direction of the arrow indicates the prediction direction (reference direction), the starting point of the arrow indicates the position of the pixel to be predicted, and the end point of the arrow indicates the position of the reference pixel used for predicting this pixel to be predicted (also referred to as "reference pixel position"). A total of 65 modes of directional prediction are prepared, and the selectable prediction direction is determined by the shape (aspect ratio) of the block. Note that in this embodiment, it is assumed that there are 65 directional prediction directions, but there may be more than 65 directional prediction directions.

[0035] As prediction directions parallel to a diagonal line passing through the upper right vertex and the lower left vertex of a block, there are mode "2," which is a prediction mode that references the lower left direction, and mode "66," which is a prediction mode that references the upper right direction. Mode numbers are assigned clockwise from mode "2" to mode "66" at predetermined angles. Mode "34" is a prediction mode that references the upper left direction. Specifically, when the horizontal direction is 0°, the prediction direction of mode "2" is −45°, the prediction direction of mode "18" is 0°, the prediction direction of mode "34" is 45°, the prediction direction of mode "50" is 90°, and the prediction direction of mode "66" is 135°. Mode "18" is also referred to as the horizontal direction, and mode "50" is also referred to as vertical prediction.

[0036] Here, directional predictions less than mode "34", i.e., modes "2" to "33", are directional predictions that refer to the left side of the coding block, and the prediction direction is toward the left of the coding block. On the other hand, directional predictions greater than mode "34", i.e., modes "35" to "66", are directional predictions that refer to the upper side of the coding block, and the prediction direction is toward the upper side of the coding block.

[0037] (1.2) Overview of Template-Based Intra Mode Derivation Next, an overview of template-based intra mode derivation according to this embodiment will be described. Template-based intra mode derivation is also called TIMD (Templated-based Intra Mode Derivation). FIG. 3 is a diagram for explaining TIMD according to this embodiment. The intra prediction unit 172 according to this embodiment performs intra prediction compatible with TIMD.

[0038] In TIMD, a peripheral region of a target block for intra prediction is used as a trial prediction region (template), and the encoding side and decoding side perform trial prediction on the trial prediction region using a common algorithm, thereby deriving an intra prediction mode common to both the encoding side and the decoding side. This reduces the signaling overhead of intra prediction modes and enables the deriving of additional directional prediction, which subdivides existing prediction directions into new prediction directions. For blocks to which TIMD is applied, the entropy encoding unit 130 may signal a flag indicating the application of TIMD to the decoding side. TIMD may be applied only to luminance signal blocks and not necessarily to chrominance signal blocks.

[0039] First, the intra prediction unit 172 identifies an adjacent decoded area adjacent to the current block as a prediction trial area (template), and also identifies a decoded area outside the prediction trial area as a reference area. The reference area is a set of adjacent reference pixels (reference pixel line) on the left and upper sides of the prediction trial area. The width L of the prediction trial area may be variably set according to the size of the current block (i.e., the width M × height N of the current block). For example, if the size of the current block is 8 or less, the width L of the prediction trial area may be set to 2, and if not, the width L of the prediction trial area may be set to 4.

[0040] Second, the intra prediction unit 172 calculates a cost using a cost function by performing a prediction trial in which a prediction trial region is predicted from a reference region for each intra prediction mode. The cost function may be, for example, SAD (Sum of Absolute Differences) or SATD (Sum of Absolute Transformed Differences). The intra prediction unit 172 predicts the prediction trial region from the reference region using a candidate intra prediction mode, and calculates the SAD or SATD between the prediction result (each predicted pixel) and the prediction trial region (each decoded pixel) as the cost. Here, the number of conventional directional predictions is 65 (65 directions), but the number of directional predictions in which prediction trials are performed may be 129 (129 directions). In other words, additional directional predictions can be applied in which conventional prediction directions are subdivided to create new prediction directions.

[0041] Third, the intra prediction unit 172 determines at least one intra prediction mode to be used for intra prediction according to the cost calculated for each intra prediction mode. For example, the intra prediction unit 172 calculates the cost for all candidate intra prediction modes and determines the intra prediction mode with the smallest cost as the final intra prediction mode for the current block. However, the intra prediction unit 172 does not necessarily need to calculate the cost for all candidate intra prediction modes. The intra prediction unit 172 may calculate the cost for each candidate intra prediction mode, and if the calculated cost satisfies a predetermined threshold condition, may determine the intra prediction mode that satisfies the threshold condition as the final intra prediction mode for the current block.

[0042] Fourth, the intra prediction unit 172 uses adjacent decoded areas adjacent to the target block as reference areas (reference pixels) to predict each pixel of the target block using the final intra prediction mode determined based on the cost, and generates a predicted block (intra prediction block).

[0043] Note that the number of final intra prediction modes determined based on the calculated costs is not limited to one, and two or more intra prediction modes selected in ascending order of cost may be determined. For example, the intra prediction unit 172 may determine two final intra prediction modes based on the calculated costs, generate two prediction blocks by intra prediction using these two intra prediction modes, and output the final prediction block by weighted averaging (weighted combining) the two prediction blocks according to the costs of each intra prediction mode.

[0044] (1.3) Configuration of Intra Prediction Unit Next, the configuration of the intra prediction unit 172 according to this embodiment will be described. Fig. 4 is a diagram showing the configuration of the intra prediction unit 172 according to this embodiment. Fig. 5 is a diagram for explaining the operation of the intra prediction unit 172 according to this embodiment.

[0045] As shown in FIG. 4 , the intra prediction unit 172 includes a region identification unit 1721 , a cost calculation unit 1722 , a prediction mode determination unit 1723 , and a prediction block generation unit 1724 .

[0046] The region identification unit 1721 identifies an adjacent decoded region adjacent to a current block for intra prediction as a prediction trial region, and also identifies a decoded region outside the prediction trial region as a reference region. The prediction trial region includes a plurality of sub-regions classified according to their relative positional relationship with the current block. In this embodiment, as shown in Figure 5, the plurality of sub-regions include a left sub-region A, which is an adjacent decoded region on the left side of the current block, and an upper sub-region B, which is an adjacent decoded region above the current block.

[0047] The cost calculation unit 1722 calculates a cost (SAD or SATD) using a cost function by performing a prediction trial for each intra prediction mode, predicting a trial prediction region from a reference region. In this embodiment, the cost calculation unit 1722 weights each sub-region when calculating the cost of each intra prediction mode. That is, the cost calculation unit 1722 takes into account the importance of each sub-region determined according to the trial intra prediction mode, and performs weighting to emphasize the cost of sub-regions with high importance. This makes it possible to improve the prediction accuracy of intra prediction.

[0048] The cost calculation unit 1722 sets a weight for each of the multiple sub-regions according to a reference region position, which is the position of a reference region referenced in an intra prediction mode used for a prediction trial, and the positions of each of the multiple sub-regions. Specifically, the cost calculation unit 1722 sets a weight so that it is larger for a second sub-region, which is a second distance from the reference region position that is longer than the first distance, than for a first sub-region, which is a first distance from the reference region position.

[0049] 5A is a diagram illustrating cost calculation when the intra prediction mode to be tried is mode 2. In mode 2, the prediction direction is −45° when the horizontal direction is 0°. Mode 2 is directional prediction that references the left reference region out of the left and upper reference regions. In mode 2, the reference region position, which is the position of the reference region to be referenced, is on the left side, and this left reference region and left sub-region A are adjacent to each other, but the upper sub-region B is separated from the left reference region.

[0050] In this case, because the left reference region and left subregion A are adjacent to each other, when the left subregion A is predicted using mode 2, the prediction result for the left subregion A will have high prediction accuracy. On the other hand, when the upper subregion B is predicted using mode 2, the prediction result for the upper subregion B will have lower prediction accuracy than the left subregion A. Therefore, when calculating the cost, the cost calculation unit 1722 sets a weight that places more importance on the upper subregion B than on the left subregion A.

[0051] In mode 2, first, the cost calculation unit 1722 calculates the SAD (or SATD) between the prediction result of the left subregion A and the left subregion A as cost A, and calculates the SAD (or SATD) between the prediction result of the upper subregion B and the upper subregion B as cost B. Second, the cost calculation unit 1722 weights the cost A with a first weight and weights the cost B with a second weight that is greater than the first weight. Third, the cost calculation unit 1722 calculates the sum (or average) of the weighted costs A and B as the cost of mode 2.

[0052] As a result, the cost calculation unit 1722 can regard the upper sub-region B, which has low prediction accuracy, as a sub-region with high importance and calculate a weighted cost that prioritizes the upper sub-region B, thereby calculating a more appropriate cost for mode 2. Note that although mode 2 has been described as an example in FIG. 5( a), similar weighted cost calculation can be applied to all directional predictions that refer to the left side (for example, modes 2 to 33, and prediction modes intermediate between these).

[0053] 5B is a diagram illustrating cost calculation when the intra prediction mode to be tried is mode 66. In mode 66, the prediction direction is 135° when the horizontal direction is 0°. Mode 66 is directional prediction that references the upper reference region out of the left and upper reference regions. In mode 66, the reference region position, which is the position of the reference region to be referenced, is on the upper side, and this upper reference region and upper sub-region B are adjacent to each other, but the left sub-region A is separated from the upper reference region.

[0054] In this case, because the upper reference region and upper subregion B are adjacent to each other, when the upper subregion B is predicted using mode 66, the prediction result for the upper subregion B will have high prediction accuracy. On the other hand, when the left subregion A is predicted using mode 66, the prediction result for the left subregion A will have lower prediction accuracy than the upper subregion B. Therefore, when calculating the cost, the cost calculation unit 1722 sets a weight that places more importance on the left subregion A than on the upper subregion B.

[0055] In the case of mode 66, first, the cost calculation unit 1722 calculates the SAD (or SATD) between the prediction result of the left subregion A and the left subregion A as cost A, and calculates the SAD (or SATD) between the prediction result of the upper subregion B and the upper subregion B as cost B. Second, the cost calculation unit 1722 weights the cost B with a first weight and weights the cost A with a second weight that is greater than the first weight. Then, the cost calculation unit 1722 calculates the sum (or average) of the weighted costs A and B as the cost of mode 66.

[0056] As a result, the cost calculation unit 1722 can regard the left sub-region A, which has low prediction accuracy, as a sub-region with high importance and calculate a weighted cost that prioritizes the left sub-region A, thereby calculating a more appropriate cost for mode 66. Note that although mode 66 has been described as an example in FIG. 5B , similar weighted cost calculation can be applied to all directional predictions that refer to the upper side (for example, modes 35 to 66, and prediction modes intermediate between these).

[0057] In this embodiment, the cost calculation unit 1722 includes a weight setting unit 1722a, a trial prediction unit 1722b, and a weight calculation unit 1722c. For each trial intra prediction mode, the weight setting unit 1722a sets a weight for each sub-region according to the intra prediction mode. The trial prediction unit 1722b performs trial prediction for each sub-region using the intra prediction mode. The weight calculation unit 1722c calculates a cost for each sub-region and applies a weight according to the results of the trial prediction, thereby deriving a weighted cost for the intra prediction mode.

[0058] For example, the weighting calculation unit 1722c may calculate the weighted cost of each intra prediction mode using the formula "weighted cost = cost A × (1 - α) + cost B × α," where cost A is the cost calculated for the left sub-region A, cost B is the cost calculated for the upper sub-region B, and α is a variable (weight) that takes a value between 0 and 1.

[0059] The weight setting unit 1722a sets, for example, 0.8 as the variable α for all directional predictions that refer only to the left reference region. In this case, "weighted cost = cost A × 0.2 + cost B × 0.8", allowing cost calculations that prioritize the upper sub-region B. On the other hand, the weight setting unit 1722a sets, for example, 0.2 as the variable α for all directional predictions that refer only to the upper reference region. In this case, "weighted cost = cost A × 0.8 + cost B × 0.2", allowing cost calculations that prioritize the left sub-region A. Note that the weight setting unit 1722a may set 0.5 as the variable α for all directional predictions that refer to both the left and upper reference regions, as well as for planar predictions and DC predictions. In this case, "weighted cost = cost A × 0.5 + cost B × 0.5".

[0060] The prediction mode determination unit 1723 determines at least one intra prediction mode to be used for intra prediction based on the weighted cost calculated for each intra prediction mode by the cost calculation unit 1722. For example, the prediction mode determination unit 1723 calculates the cost for all candidate intra prediction modes and determines the intra prediction mode with the smallest cost as the final intra prediction mode for the current block. However, the cost calculation unit 1722 does not necessarily need to calculate the cost for all candidate intra prediction modes. While the cost calculation unit 1722 calculates the cost for each candidate intra prediction mode, if the calculated cost satisfies a predetermined threshold condition, the prediction mode determination unit 1723 may determine the intra prediction mode that satisfies the threshold condition as the final intra prediction mode for the current block. The prediction mode determination unit 1723 may determine two or more intra prediction modes selected in ascending order of cost as the final intra prediction mode for the current block.

[0061] The prediction block generation unit 1724 predicts the current block by intra prediction using at least one intra prediction mode determined by the prediction mode determination unit 1723, and generates a prediction block. When the prediction mode determination unit 1723 determines one intra prediction mode, the prediction block generation unit 1724 generates an intra prediction block by the one intra prediction mode using decoded pixels surrounding the current block as reference pixels, and outputs the generated intra prediction block. When the prediction mode determination unit 1723 determines multiple intra prediction modes, the prediction block generation unit 1724 may generate multiple prediction blocks by intra prediction using each of the multiple intra prediction modes, and output a prediction block obtained by weighted averaging (weighted combining) the multiple prediction blocks according to the cost of each intra prediction mode.

[0062] (2) Decoding Device Next, the decoding device according to this embodiment will be described with reference to FIGS.

[0063] 6 is a diagram showing the configuration of a decoding device 2 according to this embodiment. The decoding device 2 is a device that derives and outputs a decoded image from an input bitstream. The decoding device 2 includes an entropy decoding unit 200, an inverse quantization and inverse transform unit 210, a synthesis unit 220, a memory 230, and a prediction unit 240.

[0064] The entropy decoding unit 200 decodes the bitstream generated by the encoding device 1 and outputs the quantized transform coefficients to the inverse quantization and inverse transform unit 210. The entropy decoding unit 200 also obtains information related to prediction (intra prediction and inter prediction) and outputs the obtained information to the prediction unit 240. In this embodiment, the entropy decoding unit 200 may obtain a flag indicating that TIMD is applied and output the flag to the prediction unit 240.

[0065] The inverse quantization and inverse transform unit 210 performs inverse quantization and inverse transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 210 includes an inverse quantization unit 211 and an inverse transform unit 212.

[0066] The inverse quantization unit 211 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 of the encoding device 1. The inverse quantization unit 211 reconstructs the transform coefficients of the coding block by inverse quantizing the quantized transform coefficients output by the entropy decoding unit 200 using the quantization parameter and the quantization matrix, and outputs the reconstructed transform coefficients to the inverse transform unit 212.

[0067] The inverse transform unit 212 performs inverse transform processing corresponding to the transform processing performed by the transform unit 121 of the encoding device 1. The inverse transform unit 212 performs inverse transform processing on the transform coefficients output by the inverse quantization unit 211 to reconstruct prediction residuals, and outputs the reconstructed prediction residuals to the synthesis unit 220. The inverse transform processing includes transform skip, in which the inverse transform processing is not performed. Furthermore, the inverse transform unit 212 may perform inverse secondary transform processing in which a further inverse transform processing is applied to the signal obtained by the inverse transform processing.

[0068] The synthesis unit 220 synthesizes the prediction residual output by the inverse transform unit 212 and the prediction block output by the prediction unit 240 by adding them together pixel by pixel, decodes (reconstructs) the original block, and outputs the reconstructed block to the memory 230.

[0069] The memory 230 stores the reconstructed blocks output by the synthesis unit 220 and accumulates the reconstructed blocks as decoded images on a frame-by-frame basis. The memory 230 outputs the reconstructed blocks or the decoded images to the prediction unit 240. The memory 230 also outputs the decoded images on a frame-by-frame basis to the outside of the decoding device 2. Note that a loop filter may be provided between the synthesis unit 220 and the memory 230.

[0070] The prediction unit 240 performs prediction in units of blocks and includes an inter prediction unit 241, an intra prediction unit 242, and a switching unit 243.

[0071] The inter prediction unit 241 predicts a coded block by inter prediction using a decoded image stored in the memory 230 as a reference image. The inter prediction unit 241 generates an inter prediction block by performing inter prediction in accordance with the motion vector information output by the entropy decoding unit 200, and outputs the generated inter prediction block to the switching unit 243.

[0072] The intra prediction unit 242 generates an intra prediction block by referring to decoded pixels surrounding a block to be predicted (encoded block) among the decoded images stored in the memory 230, and outputs the generated intra prediction block to the switching unit 243. The intra prediction unit 242 according to this embodiment performs intra prediction compatible with the above-mentioned TIMD.

[0073] The switching unit 243 switches between the inter-predicted block output by the inter-prediction unit 241 and the intra-predicted block output by the intra-prediction unit 242 , and outputs either the predicted block to the synthesis unit 220 .

[0074] Fig. 7 is a diagram showing the configuration of the intra prediction unit 242 according to this embodiment. As shown in Fig. 7, the intra prediction unit 242 has a region identification unit 2421, a cost calculation unit 2422, a prediction mode determination unit 2423, and a prediction block generation unit 2424. Here, the region identification unit 2421, the cost calculation unit 2422, the prediction mode determination unit 2423, and the prediction block generation unit 2424 perform the same processing as the region identification unit 1721, the cost calculation unit 1722, the prediction mode determination unit 1723, and the prediction block generation unit 1724 on the encoding side, respectively.

[0075] (3) Operational Example of Intra Prediction Next, an operational example of intra prediction according to this embodiment will be described. Fig. 8 is a diagram showing an operational example of the decoding-side intra prediction unit 242 according to this embodiment. Here, the operation of the decoding-side intra prediction unit 242 will be described as an example, but the encoding-side intra prediction unit 172 also performs the same operation as the decoding-side intra prediction unit 242.

[0076] In step S1, the region identification unit 2421 identifies an adjacent decoded region adjacent to a current block for intra prediction as a trial prediction region, and identifies a decoded region outside the trial prediction region as a reference region. As shown in Figure 5, the trial prediction region includes a left sub-region A that is an adjacent decoded region on the left side of the current block, and an upper sub-region B that is an adjacent decoded region above the current block.

[0077] In step S2, the cost calculation unit 2422 calculates a cost (SAD or SATD) using a cost function by performing a trial prediction for each intra prediction mode, predicting a trial prediction region from a reference region. The cost calculation unit 2422 weights each subregion when calculating the cost of each intra prediction mode. That is, the cost calculation unit 2422 considers the importance of each subregion determined according to the trial intra prediction mode and performs weighting to emphasize the cost of subregions with high importance. As described above, the cost calculation unit 2422 may calculate the weighted cost of each intra prediction mode using the formula "weighted cost = cost A × (1 - α) + cost B × α." Here, cost A is the cost calculated for the left subregion A, cost B is the cost calculated for the upper subregion B, and α is a variable (weight) ranging from 0 to 1.

[0078] For example, when the horizontal direction is set to 0°, the cost calculation unit 2422 performs the following weighted cost calculation for each directional prediction.

[0079] 1) Intra prediction modes with a prediction direction ranging from −45° to 0° (prediction modes 2 to 18): These intra prediction modes are directional predictions that refer only to the left reference region. Therefore, the cost calculation unit 2422 sets the variable α to a value greater than 0.5. For example, the variable α is set to 0.8. In this case, the "weighted cost = cost A × 0.2 + cost B × 0.8" is obtained, and the cost calculation unit 2422 performs cost calculation with a greater weight on the upper sub-region B.

[0080] 2) Intra prediction modes with a prediction direction ranging from 0° to 90° (prediction modes 18 to 50): These intra prediction modes are directional predictions that refer to both the left reference region and the upper reference region. Therefore, the cost calculation unit 2422 sets the variable α to 0.5. In this case, "weighted cost = cost A × 0.5 + cost B × 0.5", and the cost calculation unit 2422 performs cost calculations in which equal weights are assigned to the left sub-region A and the upper sub-region B.

[0081] 3) Intra prediction modes with a prediction direction in the range of 90° to 135° (prediction modes 50 to 66): These intra prediction modes are directional predictions that refer only to the upper reference region. Therefore, the cost calculation unit 2422 sets the variable α to a value smaller than 0.5. For example, the variable α is set to 0.2. In this case, the "weighted cost = cost A × 0.8 + cost B × 0.2" is obtained, and the cost calculation unit 2422 performs cost calculation with a greater weight on the left sub-region A.

[0082] In step S3, the prediction mode determination unit 2423 determines at least one intra prediction mode to be used for intra prediction, based on the weighted cost calculated for each intra prediction mode by the cost calculation unit 2422. For example, the prediction mode determination unit 2423 determines the intra prediction mode with the smallest cost (weighted cost) as the final intra prediction mode for the current block.

[0083] In step S4, the prediction block generation unit 2424 predicts the current block by intra prediction using at least one intra prediction mode determined by the prediction mode determination unit 2423, and generates a prediction block.

[0084] (4) Summary of the Embodiment Each intra prediction unit 172, 242 according to this embodiment constitutes an intra prediction device that performs intra prediction on a block-by-block basis obtained by dividing an image. Each intra prediction device includes a region identification unit 1721, 2421 that identifies a decoded region adjacent to a current block for intra prediction as a trial prediction region and identifies a decoded region outside the trial prediction region as a reference region; a cost calculation unit 1722, 2422 that calculates a cost using a cost function by performing a trial prediction of the trial prediction region from the reference region for each intra prediction mode; and a prediction mode determination unit 1723, 2423 that determines at least one intra prediction mode to be used for intra prediction based on the cost. The trial prediction region includes multiple subregions classified according to their relative position with respect to the current block. The cost calculation unit 1722, 2422 weights each subregion when calculating the cost of each intra prediction mode. This enables improved prediction accuracy in intra prediction when template-based intra mode derivation is performed.

[0085] (5) Other Embodiments In the above-described embodiment, an example has been described in which the region identification unit 1721 (2421) identifies an adjacent decoded region adjacent to the current block for intra prediction as the trial prediction region and identifies a decoded region outside the trial prediction region as the reference region. However, this is not limiting. FIG. 9 is a diagram for explaining a modified example of TIMD according to an embodiment. In the modified example shown in FIG. 9, the positional relationship between the trial prediction region and the reference region is reversed from that in FIG. 3. That is, the region identification unit 1721 (2421) identifies an adjacent decoded region adjacent to the current block for intra prediction as the reference region and identifies a decoded region of the reference region as the trial prediction region. Therefore, the region identification unit 1721 (2421) may identify a decoded region around the current block for intra prediction as the trial prediction region and identify a decoded region adjacent to the trial prediction region as the reference region. In the modified example shown in Figure 9, the prediction mode determination unit 1723 (2423) may determine the intra prediction mode (directional prediction) selected based on the cost as the final intra prediction mode by rotating it by 180 degrees.

[0086] In the above embodiment, an example has been described in which the trial prediction region includes a plurality of sub-regions classified according to their relative positional relationships with the target block, including a left sub-region A, which is an adjacent decoded region on the left side of the target block, and an upper sub-region B, which is an adjacent decoded region on the upper side of the target block. That is, an example has been described in which the trial prediction region has two sub-regions.

[0087] However, the number of sub-regions in the trial prediction region is not limited to two, and the number of sub-regions in the trial prediction region may be three or more. For example, the left sub-region A may be divided into two, top and bottom, to define an upper left sub-region A1 and a lower left sub-region A2. Furthermore, the upper sub-region B may be divided into two, top and bottom, to define an upper left sub-region B1 and an upper right sub-region B2. As a result, the trial prediction region has four sub-regions (A1, A2, B1, B2). The encoding-side cost calculation unit 1722 and the decoding-side cost calculation unit 2422 each calculate a cost for each of these four sub-regions, and may calculate a weighted cost for each intra prediction mode by weighting the costs of the four sub-regions for each intra prediction mode to be tried.

[0088] In the above embodiment, an example has been described in which the weight (variable) set for a sub-region can be set to 0. In other words, the cost calculation unit may set 0 as the weight for some of the sub-regions among the plurality of sub-regions, thereby calculating the cost only for the sub-regions that are different from the some of the sub-regions.

[0089] A program may be provided that causes a computer to execute each process performed by the image processing device (encoding device 1, decoding device 2). The program may be recorded on a computer-readable medium. The computer-readable medium can be used to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM. Circuits that execute each process performed by the image processing device (encoding device 1, decoding device 2) may be integrated, and the image processing device may be configured as a semiconductor integrated circuit (chip set, SoC).

[0090] The functions performed by the image processing device (encoding device 1, decoding device 2) may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (Central Processing Units), conventional circuits, and / or combinations thereof, programmed to perform the described functions. A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. A processor may also be a programmed processor that executes a program stored in memory. In this specification, circuitry, unit, or means refers to hardware that is programmed to perform the described functions or hardware that executes them. The hardware may be any hardware disclosed herein or any hardware known to be programmed or capable of performing the described functions. If the hardware is a processor, the circuitry, means, or unit is a combination of the hardware and software used to configure the hardware and / or processor.

[0091] As used in this disclosure, the terms "based on" and "depending on / in response to" do not mean "based only on" or "depending only on," unless expressly stated otherwise. The term "based on" means both "based only on" and "based at least in part on." Similarly, the term "depending on" means both "depending only on" and "depending at least in part on." The terms "include," "comprise," and variations thereof do not mean including only the listed items, but may mean including only the listed items or may include additional items in addition to the listed items. Additionally, the term "or," as used in this disclosure, is not intended to mean an exclusive or. Furthermore, any reference to elements using designations such as "first," "second," etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used herein as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed therein or that the first element must precede the second element in some way. In this disclosure, where articles are added by translation, such as a, an, and the in English, these articles shall include the plural unless the context clearly indicates otherwise.

[0092] The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made within the scope that does not deviate from the gist of the invention.

[0093] This application claims priority from Japanese Patent Application No. 2023-107062 (filed June 29, 2023), the entire contents of which are incorporated herein by reference.

[0094] (6) Supplementary Notes The following are additional notes regarding the features of the above-described embodiment.

[0095] (Supplementary Note 1) An intra prediction device (172, 242) that performs intra prediction on a block-by-block basis obtained by dividing an image, comprising: an area identification means (1721, 2421) that identifies a decoded area surrounding a target block for the intra prediction as a prediction trial area, and identifies a decoded area adjacent to the prediction trial area as a reference area; a cost calculation means (1722, 2422) that calculates a cost using a cost function by performing a prediction trial to predict the prediction trial area from the reference area for each intra prediction mode; and a prediction mode determination means (1723, 2423) that determines at least one intra prediction mode to be used for the intra prediction according to the cost, wherein the prediction trial area includes a plurality of sub-areas classified according to their relative positional relationship with the target block, and the cost calculation means weights each of the sub-areas when calculating the cost for each intra prediction mode.

[0096] (Supplementary Note 2) The intra prediction device according to Supplementary Note 1, characterized in that the cost calculation means sets a weight for each of the plurality of sub-regions depending on a reference region position, which is the position of the reference region referenced in the intra prediction mode used for the prediction trial, and the position of each of the plurality of sub-regions.

[0097] (Supplementary Note 3) The intra prediction device according to Supplementary Note 2, wherein the cost calculation means sets a weight for a second sub-region whose distance from the reference region position is a second distance longer than the first distance, to be larger than a weight for a first sub-region whose distance from the reference region position is a first distance.

[0098] (Supplementary Note 4) The intra prediction device according to any one of Supplementary Notes 1 to 3, wherein the cost calculation means includes: a weight setting means (1722a, 2422a) that sets a weight for each of the sub-regions according to the intra prediction mode, for each intra prediction mode used in the prediction trial; a prediction trial means (1722b, 2422b) that performs a prediction trial for each of the sub-regions using the intra prediction mode; and a weight calculation means (1722c, 2422c) that calculates the cost for each of the sub-regions and applies the weight according to a result of the prediction trial, thereby deriving a weighted cost for the intra prediction mode, and the prediction mode determination means determines at least one intra prediction mode to be used for the intra prediction according to the weighted cost of each intra prediction mode.

[0099] (Supplementary Note 5) The intra prediction device according to any one of Supplementary Notes 1 to 4, characterized in that the plurality of sub-regions include a left sub-region (A) that is a decoded region on the left side of the current block, and an upper sub-region (B) that is a decoded region on the upper side of the current block.

[0100] (Supplementary Note 6) A coding device (1) comprising the intra prediction device according to any one of Supplementary Notes 1 to 5.

[0101] (Supplementary Note 7) A decoding device (2) comprising the intra prediction device according to any one of Supplementary Notes 1 to 5.

[0102] (Supplementary Note 8) A program causing a computer to function as the intra prediction device according to any one of Supplementary Notes 1 to 5.

[0103] 1: Encoding device 2: Decoding device 100: Block division unit 110: Subtraction unit 120: Transformation and quantization unit 121: Transformation unit 122: Quantization unit 130: Entropy encoding unit 140: Inverse quantization and inverse transform unit 141: Inverse quantization unit 142: Inverse transform unit 150: Combining unit 160: Memory 170: Prediction unit 171: Inter prediction unit 172: Intra prediction unit 173: Switching unit 200: Entropy decoding unit 210: Inverse quantization and inverse transform unit 211: Inverse quantization unit 212: Inverse transform unit 220: Combining unit 230: Memory 240: Prediction unit 241: Inter prediction unit 242: Intra prediction unit 243: Switching unit 1721: Area identification unit 1722: Cost calculation unit 1722a: Weight setting unit 1722b: Trial prediction unit 1722c: Weighted calculation unit 1723: Prediction mode determination unit 1724: Predicted block generation unit 2421: Area identification unit 2422: Cost calculation unit 2422a: Weight setting unit 2422b: Trial prediction unit 2422c: Weighted calculation unit 2423: Prediction mode determination unit 2424: Predicted block generation unit

Claims

1. An intra prediction device that performs intra prediction on a block-by-block basis obtained by dividing an image, comprising: an area identification means that identifies a decoded area surrounding a target block for the intra prediction as a prediction trial area, and identifies a decoded area adjacent to the prediction trial area as a reference area; a cost calculation means that calculates a cost using a cost function by performing a prediction trial to predict the prediction trial area from the reference area for each intra prediction mode; and a prediction mode determination means that determines at least one intra prediction mode to be used for the intra prediction in accordance with the cost, wherein the prediction trial area includes a plurality of sub-areas classified in accordance with their relative positional relationship with the target block, and the cost calculation means weights each sub-area when calculating the cost for each intra prediction mode.

2. The intra prediction device of claim 1, wherein the cost calculation means sets weights for each of the plurality of sub-regions depending on a reference region position, which is the position of the reference region referenced in the intra prediction mode used for the prediction trial, and the positions of each of the plurality of sub-regions.

3. The intra prediction device according to claim 2, wherein the cost calculation means sets a weight for a second sub-region whose distance from the reference region position is a second distance longer than the first distance, greater than a weight for a first sub-region whose distance from the reference region position is a first distance.

4. An intra prediction device as claimed in any one of claims 1 to 3, wherein the cost calculation means includes: a weight setting means for setting a weight for each sub-region in accordance with the intra prediction mode, for each intra prediction mode used in the prediction trial; a prediction trial means for performing a prediction trial for each sub-region using the intra prediction mode; and a weighting calculation means for calculating the cost for each sub-region and applying the weight in accordance with a result of the prediction trial, to derive a weighted cost for the intra prediction mode, and the prediction mode determination means determines at least one intra prediction mode to be used for the intra prediction in accordance with the weighted cost of each intra prediction mode.

5. An intra prediction device as claimed in any one of claims 1 to 3, wherein the multiple sub-regions include a left sub-region which is a decoded region to the left of the current block, and an upper sub-region which is a decoded region to the upper side of the current block.

6. The intra prediction device according to claim 1, wherein the cost calculation means calculates the cost only for subregions different from the portion of the plurality of subregions by setting zero as a weight for the portion of the plurality of subregions.

7. An encoding device comprising the intra prediction device according to claim 1.

8. A decoding device comprising the intra prediction device according to claim 1.

9. A program for causing a computer to function as the intra prediction device according to claim 1.

Images