Video decoding device, video encoding device, and video encoding method

Abstract

To solve a problem in which a conventional TIMD prediction mode derivation based on the difference between a template prediction image and a template image for each conventional intra prediction mode requires a large amount of calculation for deriving the template image and calculating the cost.SOLUTION: In TIMD prediction that selects an intra-prediction mode for a current block on the basis of a template predicted image and a cost derived from the template image, a video decoding device according to an embodiment of the invention, divides predetermined intra-prediction mode candidates into a plurality of groups on the basis of the prediction direction, generates a template predicted image for each intra-prediction mode candidate selected from each group, evaluates the cost, and derives and selects costs for modes other than the selected candidates in the group to which the intra-prediction mode candidate with the lowest cost belongs.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] An embodiment of the present invention relates to a video decoding device and a video encoding device. [Background technology]

[0002] In order to efficiently transmit or record moving images, a moving image encoding device is used that generates encoded data by encoding the moving images, and a moving image decoding device is used that generates a decoded image by decoding the encoded data.

[0003] Specific examples of video coding methods include H.264 / AVC, High-Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC).

[0004] In such a video coding method, images (pictures) constituting a video are managed in a hierarchical structure consisting of slices obtained by dividing images, coding tree units (CTUs) obtained by dividing slices, coding units (sometimes called coding units: CUs) obtained by dividing coding tree units, and transform units (TUs) obtained by dividing coding units, and are coded / decoded for each CU.

[0005] In such a video coding method, a predicted image is usually generated based on a locally decoded image obtained by encoding / decoding an input image, and a prediction error (sometimes called a "difference image" or "residual image") obtained by subtracting the predicted image from the input image (original image) is coded. Methods for generating a predicted image include inter-prediction and intra-prediction.

[0006] Furthermore, a recent example of a video encoding and decoding technique is Non-Patent Document 1. Non-Patent Document 1 discloses template-based intra mode derivation (TIMD) prediction in which a decoder generates a predicted image by deriving an intra direction prediction mode number using pixels in a neighboring region. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] K. Cao, N. Hu, V. Seregin, M. Karczewicz, Y. Wang, K. Zhang, L. Zhang, “EE2-related: Fusion for template-based intra mode derivation”, JVET-W0123, July 2021 Summary of the Invention [Problem to be solved by the invention]

[0008] In the template-based intra mode derivation as described in Non-Patent Document 1, a template prediction image is generated from an adjacent image (template image) of the target block for an intra prediction mode candidate using an image of a template reference region near the target block. Then, an intra prediction mode candidate that reduces the cost of the template image and the template prediction image is selected as the intra prediction mode of the target block. However, there is a problem in that the amount of calculation is very large because it is necessary to derive a template prediction image and calculate the cost for a plurality of intra prediction mode candidates.

[0009] The present invention aims to reduce the complexity of template-based intra mode derivation. [Means for solving the problem]

[0010] The video decoding device of this embodiment is characterized in that, in TIMD prediction, a template prediction image for a template image and a predetermined intra-prediction mode candidate is generated using images of a template region and a template reference region, and an intra-prediction mode for a target block is selected based on the template prediction image and a cost derived from the template image, the predetermined intra-prediction mode candidates are divided into a plurality of groups based on the prediction direction, a template prediction image is generated for each intra-prediction mode candidate selected from each group, and the cost is evaluated, and in the group to which the intra-prediction mode candidate with the lowest cost belongs, costs are also derived and selected for candidates other than the selected intra-prediction mode candidate. Effect of the Invention

[0011] According to one aspect of the present invention, the complexity of template-based intra mode derivation can be reduced. [Brief description of the drawings]

[0012] [Figure 1] 1 is a schematic diagram showing a configuration of an image transmission system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing a hierarchical structure of data in an encoded stream. [Diagram 3] FIG. 13 is a schematic diagram showing types of intra-prediction modes (mode numbers). [Figure 4] FIG. 1 is a schematic diagram showing a configuration of a video decoding device. [Diagram 5] FIG. 13 is a diagram illustrating a configuration of an intra-prediction image generating unit. [Figure 6] 11 is a diagram showing the relationship between a target block, a template region, and a template reference region. [Figure 7] FIG. 2 is a diagram illustrating details of a TIMD prediction unit. [Figure 8] 11 is a diagram showing the relationship between a target block, a template region, and a template reference region. [Figure 9] FIG. 1 is a diagram showing an example of a group of TIMDs. [Figure 10]FIG. 1 is a block diagram showing a configuration of a video encoding device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] (First embodiment) Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0014] FIG. 1 is a schematic diagram showing the configuration of an image transmission system 1 according to this embodiment.

[0015] The image transmission system 1 is a system that transmits an encoded stream obtained by encoding an image to be encoded, decodes the transmitted encoded stream, and displays an image. The image transmission system 1 includes a video encoding device (image encoding device) 11, a network 21, a video decoding device (image decoding device) 31, and a video display device (image display device) 41.

[0016] An image T is input to the video encoding device 11 .

[0017] The network 21 transmits the encoded stream Te generated by the video encoding device 11 to the video decoding device 31. The network 21 is the Internet, a wide area network (WAN), a local area network (LAN), or a combination of these. The network 21 is not necessarily limited to a bidirectional communication network, and may be a unidirectional communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. The network 21 may also be replaced by a storage medium on which the encoded stream Te is recorded, such as a DVD (Digital Versatile Disc: registered trademark) or a BD (Blu-ray Disc: registered trademark).

[0018] The video decoding device 31 decodes each of the coded streams Te transmitted by the network 21, and generates one or more decoded images Td.

[0019] The moving image display device 41 displays all or part of one or more decoded images Td generated by the moving image decoding device 31. The moving image display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. Examples of the form of the display include stationary, mobile, and HMD. Further, when the moving image decoding device 31 has high processing capabilities, an image with high image quality is displayed, and when it has only lower processing capabilities, an image that does not require high processing capabilities and display capabilities is displayed.

[0020] <Operator> The operators used in this specification are described below.

[0021] >> represents a right bit shift, << represents a left bit shift, & represents a bitwise AND, | represents a bitwise OR, ^ represents a bitwise XOR, |= represents an OR assignment operator,! represents a logical negation (NOT), && represents a logical product (AND), and || represents a logical sum (OR).

[0022] x?y:z is a ternary operator that takes y when x is true (non-zero) and z when x is false (0).

[0023] Clip3(a, b, c) is a function that clips c to a value between a and b (inclusive). If c < a, it returns a; if c > b, it returns b; otherwise, it returns c (where a <= b).

[0024] Clip1Y(c) is an operator where a = 0 and b = (1 << BitDepthY) - 1 are set in Clip3(a, b, c). BitDepthY is the bit depth of luminance.

[0025] abs(a) is a function that returns the absolute value of a.

[0026] Int(a) is a function that returns the integer value of a.

[0027] Floor(a) is a function that returns the largest integer less than or equal to a.

[0028] Log2(a) is a function that returns the base 2 logarithm.

[0029] Ceil(a) is a function that returns the smallest integer greater than or equal to a.

[0030] a / d represents the division of a by d (truncated to an integer).

[0031] Min(a,b) is a function that returns the smaller of a and b.

[0032] <Structure of the coding stream Te> Before describing in detail the video encoding device 11 and the video decoding device 31 according to this embodiment, the data structure of the encoded stream Te generated by the video encoding device 11 and decoded by the video decoding device 31 will be described.

[0033] Fig. 2 is a diagram showing a hierarchical structure of data in an encoded stream Te. The encoded stream Te illustratively includes a sequence and a plurality of pictures constituting the sequence. Fig. 2 shows diagrams showing an encoded video sequence that defines a sequence SEQ, an encoded picture that defines a picture PICT, an encoded slice that defines a slice S, encoded slice data that defines slice data, an encoding tree unit included in the encoded slice data, and an encoding unit included in the encoding tree unit.

[0034] (Coded Video Sequence) The coded video sequence defines a set of data that the video decoding device 31 refers to in order to decode the sequence SEQ to be processed. As shown in the coded video sequence of Fig. 2, the sequence SEQ includes a video parameter set VPS (Video Parameter Set), a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and supplemental enhancement information SEI (Supplemental Enhancement Information).

[0035] The video parameter set VPS specifies a set of coding parameters common to multiple videos composed of multiple layers, as well as a set of coding parameters related to multiple layers and each individual layer included in the video.

[0036] The sequence parameter set SPS specifies a set of coding parameters that the video decoding device 31 refers to in order to decode the target sequence. For example, the width and height of a picture are specified. Note that there may be multiple SPSs. In that case, one of the multiple SPSs is selected from the PPS.

[0037] The picture parameter set PPS specifies a set of coding parameters that the video decoding device 31 refers to in order to decode each picture in the target sequence. For example, the picture parameter set PPS includes a reference value of the quantization width (pic_init_qp_minus26) used in decoding the picture and a flag (weighted_pred_flag) indicating the application of weighted prediction. Note that there may be multiple PPSs. In that case, one of the multiple PPSs is selected for each picture in the target sequence.

[0038] (Encoded Picture) The coded picture defines a set of data to be referenced by the video decoding device 31 in order to decode the picture PICT to be processed. As shown in the coded picture of FIG. 2, the picture PICT includes slices 0 to NS-1 (NS is the total number of slices included in the picture PICT).

[0039] In the following description, when there is no need to distinguish between slices 0 to NS-1, the subscripts of the symbols may be omitted. The same applies to other data that are included in the coded stream Te and that are to be described below and that are to be given subscripts.

[0040] (Coded Slice) An encoded slice defines a set of data to be referenced by the video decoding device 31 in order to decode a slice S to be processed. As shown in the encoded slice of Fig. 2, a slice includes a slice header and slice data.

[0041] The slice header includes a group of coding parameters to be referred to by the video decoding device 31 in order to determine a decoding method for the current slice. Slice type designation information (slice_type) that designates the slice type is an example of a coding parameter included in the slice header.

[0042] Slice types that can be specified by the slice type specification information include (1) an I slice that uses only intra prediction when encoding, (2) a P slice that uses unidirectional prediction or intra prediction when encoding, and (3) a B slice that uses unidirectional prediction, bidirectional prediction, or intra prediction when encoding. Note that inter prediction is not limited to uni-prediction or bi-prediction, and a predicted image may be generated using more reference pictures. Hereinafter, when referring to P or B slice, it refers to a slice including a block that can use inter prediction.

[0043] In addition, the slice header may include a reference to the picture parameter set PPS (pic_parameter_set_id).

[0044] (Encoded slice data) The coded slice data specifies a set of data to be referenced by the video decoding device 31 in order to decode the slice data to be processed. The slice data includes a CTU, as shown in the coded slice header in Fig. 2. A CTU is a block of a fixed size (e.g., 64x64) that constitutes a slice, and is also called a Largest Coding Unit (LCU).

[0045] (coding tree unit) 2 specifies a set of data that the video decoding device 31 refers to in order to decode the CTU to be processed. The CTU is divided into coding units CU, which are basic units of the encoding process, by recursive quad tree division (QT (Quad Tree) division), binary tree division (BT (Binary Tree) division), or ternary tree division (TT (Ternary Tree) division). BT division and TT division are collectively called multi tree division (MT (Multi Tree) division). A node of a tree structure obtained by recursive quad tree division is called a coding node. Intermediate nodes of the quad tree, binary tree, and ternary tree are coding nodes, and the CTU itself is specified as the top coding node.

[0046] (Encoding Unit) As shown in the coding unit of Fig. 2, a set of data to be referenced by the video decoding device 31 in order to decode the coding unit to be processed is defined. Specifically, the CU is composed of a CU header CUH, prediction parameters, transformation parameters, quantization transformation coefficients, etc. The CU header defines a prediction mode, etc.

[0047] The prediction process may be performed in units of CUs, or in units of sub-CUs obtained by further dividing a CU. When the size of a CU and a sub-CU are equal, there is one sub-CU in the CU. When the size of a CU is larger than that of a sub-CU, the CU is divided into sub-CUs. For example, when a CU is 8x8 and a sub-CU is 4x4, the CU is divided into 2 parts horizontally and 2 parts vertically, into 4 sub-CUs.

[0048] There are two types of prediction (prediction modes): intra prediction and inter prediction. Intra prediction is a prediction within the same picture, while inter prediction refers to a prediction process performed between different pictures (for example, between display times or between layer images).

[0049] The transform and quantization processes are performed in units of CUs, but the quantized transform coefficients may be entropy coded in units of sub-blocks such as 4x4.

[0050] (Prediction parameters) The predicted image is derived from prediction parameters associated with the block, which include intra-prediction and inter-prediction parameters.

[0051] The prediction parameters of intra prediction will be described below. The intra prediction parameters are composed of a luminance prediction mode IntraPredModeY and a color difference prediction mode IntraPredModeC. FIG. 3 is a schematic diagram showing types of intra prediction modes (mode numbers). As shown in the figure, there are, for example, 67 types (0 to 66) of intra prediction modes. For example, they are planar prediction (0), DC prediction (1), and angular prediction (2 to 66). In addition, linear model (LM) prediction such as cross component linear model (CCLM) prediction and multi-mode linear model (MMLM) prediction may be used. Furthermore, an LM mode may be added for color difference.

[0052] (Configuration of a video decoding device) The configuration of a video decoding device 31 (FIG. 4) according to this embodiment will be described.

[0053] The video decoding device 31 includes an entropy decoding unit 301, a parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generating unit (prediction image generating device) 308, an inverse quantization and inverse transform unit 311, an adder 312, and a prediction parameter derivation unit 320. Note that, in accordance with the video encoding device 11 described below, the video decoding device 31 may also be configured not to include the loop filter 305.

[0054] The parameter decoding unit 302 further includes a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit), and the CU decoding unit 3022 further includes a TU decoding unit 3024. These may be collectively referred to as a decoding module. The header decoding unit 3020 decodes parameter set information such as VPS, SPS, PPS, and APS, and slice header (slice information) from the encoded data. The CT information decoding unit 3021 decodes the CT from the encoded data. The CU decoding unit 3022 decodes the CU from the encoded data. The TU decoding unit 3024 decodes QP update information (quantization correction value) and quantization prediction error (residual_coding) from the encoded data when a prediction error is included in the TU.

[0055] In cases other than the skip mode (skip_mode==0), the TU decoding unit 3024 decodes the QP update information and the quantized prediction error from the encoded data. More specifically, in cases of skip_mode==0, the TU decoding unit 3024 decodes a flag cu_cbp indicating whether or not the current block includes a quantized prediction error, and decodes the quantized prediction error when cu_cbp is 1. When cu_cbp does not exist in the encoded data, it derives 0.

[0056] The predicted image generating unit 308 includes an inter predicted image generating unit 309 and an intra predicted image generating unit 310 .

[0057] The prediction parameter derivation unit 320 is configured to include an inter prediction parameter derivation unit 303 and an intra prediction parameter derivation unit 304 .

[0058] In addition, although an example in which CTU and CU are used as processing units will be described below, the present invention is not limited to this example, and processing may be performed in sub-CU units. Alternatively, CTU and CU may be read as blocks, and sub-CU as sub-blocks, and processing may be performed in block or sub-block units.

[0059] The entropy decoding unit 301 performs entropy decoding on the externally inputted coded stream Te to decode individual codes (syntax elements). Entropy coding includes a method called CABAC (Context Adaptive Binary Arithmetic Coding) that performs variable-length coding on syntax elements using a context (probability model) adaptively selected according to the type of syntax element and surrounding circumstances, and a method that performs variable-length coding on syntax elements using a predetermined table or formula. The entropy decoding unit 301 initializes all CABAC states at the beginning of a segment (tile, CTU row, slice). The entropy decoding unit 301 converts the syntax elements into a binary string (Bin String) and decodes each bit of the Bin String. When a context is used, a context index ctxInc is derived for each bit of the syntax element, the bit is decoded using the context, and the CABAC state of the used context is updated. Bits without context are decoded with equal probability (EP, bypass) and ctxInc derivation and CABAC state are omitted.

[0060] The entropy decoding unit 301 outputs the decoded code to the parameter decoding unit 302. Control of which code to decode is performed based on an instruction from the parameter decoding unit 302.

[0061] (Configuration of the intra prediction parameter derivation unit 304) The intra prediction parameter derivation unit 304 decodes intra prediction parameters, for example, an intra prediction mode IntraPredMode, with reference to prediction parameters stored in the prediction parameter memory 307, based on the code input from the entropy decoding unit 301. The intra prediction parameter derivation unit 304 outputs the decoded intra prediction parameters to the predicted image generation unit 308, and also stores them in the prediction parameter memory 307. The intra prediction parameter derivation unit 304 may derive different intra prediction modes for luminance and chrominance.

[0062] The loop filter 305 is a filter provided in the encoding loop, which removes block distortion and ringing distortion to improve image quality. The loop filter 305 applies a filter such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the decoded image of the CU generated by the adder 312.

[0063] The reference picture memory 306 stores the decoded image of the CU generated by the adder 312 in a location that is determined in advance for each current picture and current CU.

[0064] The prediction parameter memory 307 stores prediction parameters at a predetermined position for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores the parameters decoded by the parameter decoding unit 302 and the prediction mode predMode separated by the entropy decoding unit 301.

[0065] The prediction image generating unit 308 receives a prediction mode predMode, prediction parameters, and the like. The prediction image generating unit 308 also reads a reference picture from the reference picture memory 306. The prediction image generating unit 308 generates a prediction image of a block or sub-block using the prediction parameters and the read reference picture (reference picture block) in the prediction mode indicated by the prediction mode predMode. Here, the reference picture block is a set of pixels on the reference picture (usually rectangular, so called a block), and is an area to be referenced for generating a prediction image.

[0066] (Intra-prediction image generation unit 310) When the prediction mode predMode indicates an intra prediction mode, the intra prediction image generation unit 310 performs intra prediction using the intra prediction parameters input from the intra prediction parameter derivation unit 304 and reference pixels read from the reference picture memory 306 .

[0067] Specifically, the intra-prediction image generation unit 310 reads adjacent blocks in a predetermined range from a target block on a target picture from the reference picture memory 306. The predetermined range refers to adjacent blocks to the left, upper left, upper, and upper right of the target block, and the area to be referenced differs depending on the intra-prediction mode.

[0068] The intra-prediction image generation unit 310 generates a prediction image of the current block by referring to the read decoded pixel values ​​and the prediction mode indicated by IntraPredMode. The intra-prediction image generation unit 310 outputs the generated prediction image of the block to the addition unit 312.

[0069] Generation of a predicted image based on an intra prediction mode will be described below. In planar prediction, DC prediction, and angular prediction, a decoded surrounding area adjacent (close) to the prediction target block is set as a reference area R. Then, a predicted image is generated by extrapolating pixels in the reference area R in a specific direction. For example, the reference area R may be set as an L-shaped area including the left and top of the prediction target block (or further, the top left, top right, and bottom left).

[0070] (Details of predicted image generation unit) Next, the configuration of the intra-prediction image generation unit 310 will be described in detail with reference to Fig. 5. The intra-prediction image generation unit 310 includes a reference sample filter unit 3103 (second reference image setting unit), a prediction unit 3104, and a prediction image correction unit 3105 (prediction image correction unit, filter switching unit, weighting coefficient changing unit).

[0071] The prediction unit 3104 generates a prediction image (pre-corrected prediction image) of the block to be predicted based on each reference pixel (reference image) in the reference region R, a filtered reference image generated by applying a reference pixel filter (first filter), and the intra prediction mode, and outputs the prediction image to the prediction image correction unit 3105. The prediction image correction unit 3105 corrects the prediction image according to the intra prediction mode, and generates and outputs a prediction image (corrected prediction image).

[0072] Below, each unit included in the intra-predicted image generation unit 310 will be described.

[0073] (Reference sample filter unit 3103) The reference sample filter unit 3103 derives reference samples recSamples[x][y] at each position (x, y) on the reference region R by referring to the reference image. Furthermore, the reference sample filter unit 3103 applies a reference pixel filter (first filter) to the reference samples recSamples[x][y] according to the intra prediction mode to update the reference samples recSamples[x][y] at each position (x, y) on the reference region R (derives filtered reference images recSamples[x][y]). Specifically, a low-pass filter is applied to the position (x, y) and the reference images therearound to derive a filtered reference image. Note that it is not necessarily necessary to apply a low-pass filter to all intra prediction modes, and a low-pass filter may be applied to some intra prediction modes. Note that the filter applied to the reference image on the reference region R in the reference sample filter unit 3103 is referred to as the "reference pixel filter (first filter)", whereas the filter that corrects the predicted image in the predicted image correction unit 3105 described below is referred to as the "position-dependent filter (second filter)".

[0074] (Configuration of prediction unit 3104) The prediction unit 3104 generates a prediction image of a prediction target block based on an intra prediction mode, a reference image, and a filtered reference pixel value, and outputs the prediction image to the prediction image correction unit 3105. The prediction unit 3104 includes a planar prediction unit 31041, a DC prediction unit 31042, an angular prediction unit 31043, an LM prediction unit 31044, a matrix-based intra prediction unit 31045, and a template-based intra mode derivation (TIMD) prediction unit 31047. The prediction unit 3104 selects a specific prediction unit according to the intra prediction mode, and inputs the reference image and the filtered reference image. The relationship between the intra prediction mode and the corresponding prediction unit is as follows. ·Planar prediction ···Planar prediction section 31041 ·DC prediction ···DC prediction section 31042 ·Angular prediction ···Angular prediction part 31043 ·LM prediction ···LM prediction section 31044 · Matrix intra prediction · MIP section 31045 TIMD prediction: TIMD prediction unit 3104

[0075] (Planar forecast) The planar prediction unit 31041 generates a predicted image by linearly adding the reference samples recSamples[x][y] according to the distance between the prediction target pixel position and the reference pixel position, and outputs the predicted image to the predicted image correction unit 3105.

[0076] (DC forecast) The DC prediction unit 31042 derives a DC predicted value equivalent to the average value of the reference samples recSamples[x][y], and outputs a predicted image predSamples[x][y] whose pixel values ​​are the DC predicted values.

[0077] (Angular prediction) The angular prediction unit 31043 generates predicted images predSamples[x][y] using reference samples recSamples[x][y] in the prediction direction (reference direction) indicated by the intra prediction mode, and outputs the predicted images to the predicted image correction unit 3105. If IntraPredMode >= DIR, perform the following processing.

[0078] ref[x] = recSamples[-1-refIdx+x][-1-refIdx] (x=0..cbWidth+refIdx+1) Furthermore, do the following for x = 0..cbWidth-1, y = 0..cbHeight-1.

[0079] iIdx = (((y+1+refIdx) * intraPredAngle) >> 5) + refIdx iFact = ((y+1+refIdx) * intraPredAngle) & 31 predSamples[x][y] = Clip1(((Σ(fT[i]*ref[x+iIdx+i])) + 32) >> 6) Otherwise (IntraPredMode < DIR), the following process is performed.

[0080] ref[x] = recSamples[-1-refIdx][-1-refIdx+x] (x=0..cbHeight+refIdx+1) Furthermore, do the following for x = 0..cbWidth-1, y = 0..cbHeight-1.

[0081] iIdx = (((x+1+refIdx) * intraPredAngle) >> 5) + refIdx iFact = ((x+1+refIdx) * intraPredAngle) & 31 predSamples[x][y] = Clip1(((Σ(fT[i]*ref[y+iIdx+i])) + 32) >> 6) DIR and refIdx are predetermined constants, for example, DIR=34, 66, etc. refIdx=0, 1, 2, etc. In the case of angular prediction, refIdx may be set by decoding the syntax of the encoded data. In addition, in the case of generating a predicted image for TIMD prediction described later, refIdx may be fixed at refIdx=0. In addition, when used to generate a template predicted image, refIdx=2 or 4 may be used.

[0082] (LM forecast) The LM prediction unit 31044 predicts pixel values ​​of chrominance based on pixel values ​​of luminance. Specifically, this is a method of generating a predicted image of a chrominance image (Cb, Cr) using a linear model based on a decoded luminance image. One type of LM prediction is CCLM (Cross-Component Linear Model) prediction. CCLM prediction is a prediction method that uses a linear model to predict chrominance from luminance for one block.

[0083] (Matrix intra prediction) The MIP unit 31045 generates a predicted image predSamples[x][y] by performing a product-sum operation on the reference samples recSamples[x][y] derived from the adjacent region and a weighting matrix, and outputs the generated image to the predicted image correction unit 3105.

[0084] (Configuration of predicted image correction unit 3105) The predicted image correction unit 3105 corrects the predicted image output from the prediction unit 3104 according to the intra prediction mode. Specifically, the predicted image correction unit 3105 derives a position-dependent weighting coefficient for each pixel of the predicted image according to the reference region R and the position of the target predicted pixel. Then, the predicted image correction unit 3105 derives a corrected predicted image (corrected predicted image) Pred[][] by performing weighted addition (weighted averaging) of the reference samples recSamples[][] and the predicted images predSamples[x][y]. Note that in some intra prediction modes, the predicted image correction unit 3105 may set the predicted image predSamples[x][y] to Pred without correcting it.

[0085] (Method of deriving MPM) The intra prediction parameter derivation unit 304 derives the Most Probable Mode (MPM) list candModeList as follows. Let the upper left coordinate of the target block be (x0, y0), the block width be cbWidth, and the block height be cbHeight.

[0086] The intra prediction parameter derivation unit 304 may derive as follows, with the intraPredMode of the block A adjacent to the left of the target block being intraPredModeA and the intraPredMode of the block B adjacent to the top of the target block being intraPredModeB. Block A (hereinafter referred to as A) is a block including the coordinate (x0 - 1, y0 + cbHeight - 1). Block B (hereinafter referred to as B) is a block including the coordinate (x0 + cbWidth - 1, y0 - 1).

[0087] candModePredModeA = (A is not available || A is other than MODE_INTRA)? INTRA_PLANAR : intraPredModeA candModePredModeB = (B is not available || B is other than MODE_INTRA || the position of B is above the upper CTU boundary of the target block)? INTRA_PLANAR : intraPredModeB Note that above the CTU boundary can be determined by whether y0 - 1 < ((y0 >> CtbLog2SizeY) << CtbLog2SizeY) is true. Here, CtbLog2SizeY = Log2(CTU size). The CTU size may be derived as CtbLog2SizeY = sps_log2_ctu_size_minus5 + 5 by decoding the syntax element sps_log2_ctu_size_minus5 in the encoded data. candModeList[] may be derived from candModePredModeA and candModePredModeB as follows.

[0088] candModeList[0] = candIntraPredModeA candModeList[1] = candIntraPredModeB minAB = Min(candIntraPredModeA, candIntraPredModeB) candModeList[2] = 2 + ((minAB + 61) % 64) candModeList[3] = 2 + ((minAB - 1) % 64) candModeList[4] = 2 + ((minAB + 60) % 64

[0089] (TIMD forecast) TIMD prediction is a prediction method that generates a predicted image using an intra prediction mode that is not explicitly signaled. The TIMD prediction unit 31047 derives an intra prediction mode suitable for predicting an image (template image tempSamples) of a template region RT that is an adjacent region of the target block. Specifically, a predicted image tpredSamples (template predicted image) of the template region RT for multiple intra prediction mode candidates is generated using an image of a reference region (template reference region) RTRS near the template region. Then, the intra prediction mode candidate used to derive tpredSamples that minimizes the cost (e.g., the sum of absolute differences) between tempSamples and tpredSamples is selected as the intra prediction mode of the TIMD of the target block. The TIMD prediction unit 31047 generates a predicted image predSamples using the intra prediction mode of this TIMD.

[0090] Example 1 The parameter decoding unit 302 decodes a flag timd_flag indicating whether or not TIMD is used for each block from the encoded data. The block may be a CU, a TU, a subblock, or the like. When timd_flag of the target block is 0, the parameter decoding unit 302 decodes syntax elements related to the intra prediction mode (intra_luma_mpm_flag, intra_luma_mpm_idx, intra_luma_mpm_reminder, and the like). When timd_flag is 1, the parameter decoding unit 302 may not need to decode syntax elements related to the intra prediction mode from the encoded data. Specifically, when timd_flag of the target block is 0, the parameter decoding unit 302 decodes intra_luma_mpm_flag, and when intra_luma_mpm_flag is 0, the parameter decoding unit 302 further decodes intra_luma_mpm_reminder. When intra_luma_mpm_flag is 1, intra_luma_mpm_idx is decoded. intra_luma_mpm_flag is a flag indicating whether or not to derive an intra prediction mode using the intra prediction candidate list candModeList[]. intra_luma_mpm_idx is an index specifying an intra prediction candidate when candModeList[] is used. intra_luma_mpm_reminder is an index selecting an intra prediction candidate not included in candModeList[].

[0091] (Select Reference Area) Figure 6 shows the template region RT and template reference region (template reference sample region) RTRS referenced in TIMD prediction. The template region is the region corresponding to the template image. The template reference region RTRS is the region referenced when generating a template predicted image, which is a predicted image of the template image.

[0092] When timd_flag is 1, the TIMD prediction unit 31047 generates a template prediction image for an intra prediction mode candidate using an image of the template reference region RTRS around the current block, and selects a suitable intra prediction mode for the current block.

[0093] 7 shows the configuration of the TIMD prediction unit 31047 in this embodiment. The TIMD prediction unit 31047 is composed of a reference sample derivation unit 4701, a template derivation unit 4702, an intra prediction unit 4703, an intra prediction mode candidate derivation unit 4711, a template predicted image generation unit 4712, a template cost derivation unit 4713, and an intra prediction mode selection unit 4714. Note that the intra prediction mode candidate derivation unit 4711, the template predicted image generation unit 4712, and the template cost derivation unit 4713 may be collectively referred to as a template intra prediction mode derivation device 4710.

[0094] When timd_flag is 1, the TIMD prediction unit 31047 derives a template predicted image tpredSamples using an intra prediction mode that accurately predicts a template image tempSamples generated from an image of the template region RT. Specifically, the TIMD prediction unit 31047 executes the following procedure.

[0095] (STEP 1) A predicted image tpredSamples is derived using a predetermined prediction mode (intra prediction candidate) and a template reference region RTRS. The predetermined prediction mode may include an angular prediction mode, a planar prediction mode, a DC prediction mode, and an intra prediction mode derived by a decoder-side intra mode derivation (DIMD, Decoder-side Intra Mode Derivation) included in the MPM.

[0096] (STEP 1-1) A template predicted image tpredSamples is derived for at least a part of intra prediction modes included in predetermined prediction modes.

[0097] (STEP1-2) A cost value that indicates the magnitude of the difference between each derived tpredSamples and tempSamples is derived.

[0098] (STEP 2) An intra prediction mode corresponding to tpredSamples determined to have a small cost value, that is, high prediction accuracy, among the derived tpredSamples is selected as a prediction mode for the TIMD.

[0099] (STEP 3) A predicted image predSamples is generated using the intra prediction mode selected in (STEP 2).

[0100] The processes (STEP 1) to (STEP 4) in each part of the TIMD prediction unit 31047 shown in FIG. 7 will be described in more detail below.

[0101] (STEP 1) Predict the image of the template region The intra prediction mode candidate derivation unit 4711 first derives a list of intra prediction mode candidates timdModeList[] from the intra prediction modes of blocks adjacent to the current block. For example, the MPM list candModeList[] may be used as timdModeList.

[0102] timdModeList[i] = candModeList[i] (i=0..numCand-1) numCand is the number of elements in candModeList.

[0103] The template derivation unit 4702 derives a template image tempSamples of the current block. As shown in Fig. 6, the template image may be derived from a template region RT adjacent to the current block. Here, the template region RT is an L-shaped set of decoded pixels recSamples with a width of one pixel.

[0104] tempSamples[i][j] = recSamples[x0+i][y0+j] (i=0..cbWidth-1, j=-tH..-1, and i=-tW..-1, j=0..cbHeight-1) The area on recSamples used to derive tempSamples is called the template area RT and is expressed as a set of coordinates (i, j). tW and tH are the width and height of the template image. That is, RT={{i=0..cbWidth-1, j=-tH..-1}, {i=-tW..-1, j=0..cbHeight-1}}. In Figure 6, (tW,tH)=(1,1). Also, instead of using tempSamples, the array of decoded images recSamples corresponding to the template area may be used as the template image. In this case, the area where tempSamples is located is called the template area. The relationship between the target block, template area RT, and template reference area is also shown in Figure 8.

[0105] The reference sample derivation unit 4701 derives a reference sample refUnit from the template reference region RTRS. Note that the operation of the reference sample derivation unit 4701 may be performed by the reference sample filter unit 3103.

[0106] refUnit[x][y] = recSamples[x0+x][y0+y] Here, the range is x=-tW-1, y=-1-tH..refH-1, and x=-tW..refW-1, y=-1-tH. tW and tH are the width and height of the template region, and in Fig. 6, tW=1, tH=1. refW=cbWidth*2, refH=cbHeight*2, but this is not limiting, and refW=cbWidth, refH=cbHeight, or refW=cbWidth*4, refH=cbHeight*4 may also be used.

[0107] The reference sample derivation unit 4701 may filter the reference sample refUnit[x][y] to derive a reference sample p[x][y].

[0108] The template predicted image generation unit 4712 generates a predicted image (template predicted image tpredSamples) of the intra prediction mode IntraPredMode from the template reference region RTRS. The operation of deriving a predicted image by the template predicted image generation unit 4712 may be performed by the prediction unit 3104. For example, the planar prediction unit 31041, the DC prediction unit 31042, and the angular prediction unit 31043 may perform both the derivation of a template predicted image and the derivation of a predicted image of a current block.

[0109] <Step search configuration example> The TIMD prediction unit 31047 divides predetermined intra prediction mode candidates into a plurality of groups, generates a template prediction image for each intra prediction mode candidate selected from each group, evaluates the cost, and derives and selects the cost of candidates other than the selected intra prediction mode candidate in the group to which the intra prediction mode candidate with the lowest cost belongs. For example, the TIMD prediction unit 31047 divides intra prediction modes into numGr groups (numGr>1) and tests only the representative mode of each group (derives a template prediction image, calculates the cost, and compares it). For intra prediction modes included in the best group whose representative mode has the smallest cost, the intra prediction mode may be tested in more detail. With this configuration, the search is divided into numGr searches of coarse granularity intra prediction modes and M-1 searches of fine granularity intra prediction modes. For example, the predicted image derivation and cost derivation performed numCand (=numGr*M) times of candidates can be reduced to the sum of the number of groups and the number of candidates in the group (numGr+M-1 times), and the complexity can be significantly reduced. Here, numGr is the number of groups, and M is the number of intra-modes contained in the group.

[0110] FIG. 9(a) shows an example of dividing intra prediction modes into groups. Here, intra prediction modes included in timdModeList, which is a candidate list used in TIMD prediction, are classified into nine groups, Gr[0] to Gr[8]. The grouping of intra prediction modes may be determined using a certain threshold value gTH[] and the magnitude relationship between the intra prediction modes. When the intra prediction modes are 0 to 66, the threshold value for dividing the groups may be, for example, {gTH[0], gTH[1], gTH[2], gTH[3], gTH[4], gTH[5], gTH[6], gTH[7]}={1, 10, 18, 26, 34, 42, 50, 58}. In this case, Gr[1] to Gr[8] include intra angular prediction modes 2-10, 11-18, 19-26, 27-34, 35-42, 43-50, 51-58, and 59-66, respectively. Note that, since the inclinations of the prediction directions are similar, the last group of the angular prediction mode (Gr[8]) may be merged with the first group of the angular prediction mode (Gr[1]). If the number of intra prediction modes is numMode and the number of groups is numGr, FIG. 9(a) shows an example where numMode=67 and numGr=9.

[0111] The intra-prediction mode selection unit 4714 may derive the intra-prediction mode of the TIMD using a list GrList[m][n] (m = 0 .. numGr-1, n = 0 .. numCand-1) that stores the intra-prediction mode candidates included in each predetermined group and a list numGrList[m] (m = 0 .. numGr-1) that stores the number of intra-prediction mode candidates included in each group.

[0112] The intra prediction mode selection unit 4714 may divide all angular prediction modes into groups. Alternatively, the intra prediction mode selection unit 4714 may divide only candidates included in the intra prediction mode timdModeList derived from the intra prediction modes of blocks adjacent to the target block into groups. An example of the latter is shown below.

[0113] The intra prediction mode selection unit 4714 may classify the candidates included in timdModeList (timdModeList[k], k = 0 .. numCand-1) according to size, and derive a list (group list) GrList[m][n] indicating the candidates in each group and the number of groups numGrList[m] (m = 0 .. numGr-1). For example, the processing may be as follows. gTH[m] (m = 0 .. numGr-2) is a threshold value for dividing the groups.

[0114] First, the intra-prediction mode selection unit 4714 initializes numGrList and nGrList.

[0115] for (m=0; m <numGr; m++) numGrList[m] = 0 for (m=0; m <numGr; m++) for (n=0; n <numCand; n++) GrList[m][n] = 0 Next, the intra prediction mode selection unit 4714 may determine to which of groups Gr[0] to Gr[8] in FIG. 9A the timdModeList[k] belongs, for each k (k=0 .. numCand-1).

[0116] for (k=0; k <numCand; k++) { m=0 do { if (timdModeList[k]<=gTH[m]) { GrList[m][numGrList[m]] = timdModeList[k] numGrList[m]++ break } else m++ } while (m <numGr-1) if (m==numGr-1) { / / last Group GrList[m][numGrList[m]] = timdModeList[k] numGrList[m]++ } } According to the above configuration, the smallest representative mode and the best group to which it belongs are identified in the first search. Since the intra prediction modes in the group have continuous directions, the second search searches all intra prediction modes in the best group, which is an effect of performing a search without omission. In addition, even if a limited set of candidates, such as intra prediction modes included in timdModeList, is divided into groups and a representative mode is identified, all intra prediction modes in the identified group are searched in the second search. Therefore, it is possible to search candidates in a specific direction without omission. Furthermore, in the above configuration, the groups are configured as exclusive groups without overlap, which allows for efficient search and reduces complexity.

[0117] Note that modes that are equal to or close to the threshold value may be included in both groups on either side of the threshold value. By overlapping some intra prediction modes in this way, it is possible to achieve a stable search while reducing complexity. An appropriate number of overlaps is 1 to 2.

[0118] The intra prediction mode selection unit 4714 sets one representative mode repMode[m] (m = 0..numGr-1) for a group that includes one or more intra prediction mode candidates in timdModeList. For example, the representative mode may be the mode with the highest position in timdModeList or the first mode among the intra prediction modes of each group. In other words, one representative mode is set from the modes stored in GrList[m][0].

[0119] for (m=0; m <numGr; m++) if (numGrList[m] != 0) repMode[m] = GrList[m][0] else repMode[m] = NO_REP NO_REP is a constant indicating that there is no representative mode, and a value that does not overlap with other intra-prediction mode numbers is used.

[0120] The representative mode of each group is not limited to the above, and may be, for example, the intra prediction mode having the smallest mode number in each group.

[0121] repMode[m] = argmin(GrList[m][n]) (n=0..numCand-1) Alternatively, the representative mode of each group may be the maximum intra-prediction mode within each group.

[0122] repMode[m] = argmax(GrList[m][n]) (n=0..numCand-1) Alternatively, the representative mode of each group may be the intra-prediction mode having the median value within each group.

[0123] Alternatively, if the group includes a specific direction such as horizontal, vertical, 45 degrees, or 135 degrees, then that direction is set as the representative mode; if not, then the representative mode may be set using another method, for example, any of the methods described above.

[0124] (STEP1-1) Predict the image of the template region Specifically, for each intra prediction mode (tIntraPredMode=repMode[m]) included in repMode[m], a one-dimensional reference pixel array ref[] is generated from the reference sample refUnit[x][y] or the filtered reference sample p[x][y]. Then, tIntraPredMode and ref[] are transmitted to the intra prediction unit 4703. The intra prediction unit 4703 generates a template predicted image tpredSamples[][] corresponding to tIntraPredMode, and outputs it to the template predicted image generation unit 4712.

[0125] The template prediction image generation unit 4712 generates ref[] as follows, for example, in (Equation TIMD-ANGULAR-REF) below. The following generates tpredSamples[][] from p[][], but refUnit[][] may be used instead of p[][].

[0126] (Equation TIMD-ANGULAR-REF) When tIntraPredMode >= DIR, it is derived as follows.

[0127] ref[x] = p[x][-1 - tH] (x = 0..cbWidth + refIdx + 1) In other cases (tIntraPredMode < DIR), it is derived as follows.

[0128] ref[x] = p[-1 - tW][x] (x = 0..cbHeight + refIdx + 1) DIR is a predetermined constant, for example, DIR = 34, 66, etc. Here, refIdx is the distance from the target block. The template reference area that uses the image of the template area for prediction may use refIdx > 0 (for example, refIdx = 2, 4, etc.). refIdx may be changed according to the block size.

[0129] refIdx = (cbWidth >= 8) && (cbHeight >= 8)? 4 : 2 The template prediction image generation unit 4712 (intra prediction unit 4703) generates a template prediction image tpredSamples[][] corresponding to tIntraPredMode as follows, for example, in (Equation TIMD-ANGULAR-PRED) below. When tIntraPredMode >= DIR, (Equation TIMD-ANGULAR-PRED) iIdx = (((y + 1 + refIdx) * intraPredAngle) >> 5) + refIdx iFact = ((y + 1 + refIdx) * intraPredAngle) & 31 tpredSamples[x][y] = Clip1((Σ(filt[i] * ref[x+iIdx+i]) + 32) >> 6) Otherwise (tIntraPredMode <DIR)、 iIdx = (((x + 1 + refIdx) * intraPredAngle) >> 5) + refIdx iFact = ((x + 1 + refIdx) * intraPredAngle) & 31 tpredSamples[x][y] = Clip1((Σ(filt[i] * ref[y+iIdx+i]) + 32) >> 6) x=0..cbWidth-1,y=0..tH-1 and x=0..tW-1,y=0..cbHeight-1.

[0130] Here, Σ is the sum from i=0..MTAP-1, and filt is an interpolation filter coefficient for the template predicted image. The number of taps MTAP of the interpolation filter used to derive the template predicted image is, for example, 4. intraPredAngle is an angle parameter of each intra prediction mode. filt may be derived from iFast as follows: filtG[phase][j] is a coefficient of the interpolation filter for generating the template predicted image.

[0131] filt[j] = filtG[iFact][j] The template derivation unit 4702 derives the template image tempSamples of the target block using the above method.

[0132] The template predicted image generation unit 4712 generates a predicted image (template predicted image tpredSamplesR) of the representative mode repMode[m] from the template reference region RTRS by the above method.

[0133] for (m=0; m <numGr; m++) if (numGrList[m] != 0) { tIntraPredMode = repMode[m

[0134] Generate tpredSamplesR[x][y] using the formula TIMD-ANGULAR-PRED } (STEP1-2) Cost derivation The template cost derivation unit 4713 derives a cost tempGrCost[m] of the intra prediction mode candidate from the difference between the template predicted image tpredSamplesR of the representative mode candidate repMode[m] and the template image tempSamples. The cost may be the Sum of Absolute Difference (SAD).

[0135] for (m=0; m <numGr; m++) if (numGrList[m] != 0) tempGrCost[m] = Σabs(tpredSamplesR[i][j] - tempSamples[i][j]) else tempGrCost[m] = MAX_COST (i, j) is, for example, (i=0..cbWidth-1, j=0..tH-1), (i=0..tW-1, j=0..cbHeight-1). tW and tH are the width and height of the template image. MAX_COST is a value that indicates that the cost value and its group are invalid, and is a value that will never be smaller than any other cost value. For example, the maximum value of the double type may be used. The cost may be the Sum of Absolute Transformed Difference (SATD) instead of the SAD, or may be a weighted sum of a plurality of costs.

[0136] (STEP 2) Group and mode selection and weight derivation The intra prediction mode selection unit 4714 derives m that minimizes the cost from tempGrCost[m] (m=0 .. numGr-1) derived in (STEP 1-2), and sets the value as mm.

[0137] mm = argmin(tempGrCost[m]) (m=0..numGr-1) (STEP1-1) and (STEP1-2) are executed for all intra prediction modes in group mm. If the intra prediction mode has a cost already derived, (STEP1-1) and (STEP1-2) can be omitted. Select the smallest value costMode1 and the second smallest value costMode2 in group mm, and the corresponding intra prediction mode candidates kMode1 and kMode2.

[0138] kMode1 = argmin(tempCost[k]) (k=0..numGrList[mm]-1) costMode1 = tempCost[kMode1] kMode2 = argmin(tempCost[k]) (k=0..kMode1-1, kMode+1..numGrList[mm] -1) costMode2 = tempCost[kMode2] For example, the intra prediction modes corresponding to the minimum and second smallest costs may be the following pseudocode: for (k = 0; k < numGrList[mm]; k++) { if (tempCost[k] < costMode1) { costMode2 = costMode1 kMode2 = kMode1 costMode1 = tempCost[k] kMode1 = k } } Here, the prediction directions represented by kMode1 and kMode2 may be adjusted to more accurate direction representation. For example, when kMode1=15, an angular prediction image is derived for the intermediate direction between mode 14 and mode 15 (mode kMode1a) and the intermediate direction between mode 15 and mode 16 (mode kMode1b), and the cost is derived. The two derived costs are compared with costMode1, and the mode that gives the smallest cost is newly designated as kMode1, and costMode1 is updated to the minimum value. A similar adjustment may be made for kMode2.

[0139] For the two derived modes kMode1 and kMode2, weights weight1 and weight2 to be used in generating a predicted image are derived according to costs costMode1 and codeMode2. For example, they are derived as follows.

[0140] CostMode1 and costMode2 are compared with a threshold ThCost, and if costMode2 is smaller than ThCost*costMode1 (first condition), weights weight1 and weight2 to be used in generating a predicted image are derived.

[0141] if (costMode2 < ThCost*costMode1) { weight1 = (1<<8) * costMode2 / (costMode1+costMode2) weight2 = (1<<8) - weight1 } Alternatively, the group mm may be further recursively divided into smaller groups. For example, as shown in FIG. 9(b), the intra prediction modes are divided into three groups Gr[0], Gr[1], and Gr[2], and the threshold is set to {gTH[0], gTH[1]}={1, 34}. Assume that the smallest representative mode repMode[m] and the best group to which it belongs are identified as m=1 and Gr[1] in the first search. Gr[1] includes 33 intra prediction modes from 2 to 34, and the number of searches in the second search becomes too large. In such a case, as shown in FIG. 9(c), the smallest representative mode repMode[m] (m=3, 4) and the best group to which it belongs are identified for Gr[3] and Gr[4], which are further divided into Gr[1]. The second search is performed on this best group. If the number of intra prediction modes in Gr[3] or Gr[4] is large, further division may be repeated. By limiting the number of modes to be searched in a stepwise manner in this way, highly accurate prediction modes can be derived sequentially with a small amount of processing.

[0142] For Gr[0] that is not an Angular prediction, costs may always be derived for all modes to update kMode1 and kMode2. Furthermore, costs may always be derived for certain modes other than Gr[0], such as the horizontal direction and vertical direction.

[0143] (STEP 3) Deriving TIMD predicted image The intra prediction unit 4703 derives an intra predicted image predSamples[][] corresponding to IntraPredMode. Then, it outputs it as an intra predicted image of the current block. Note that these processes may be derived by the prediction unit 3104. The predicted image predSamples[x][y] (x=0..cbWidth-1, y=0..cbHeight-1) corresponding to the intra prediction mode candidate intraPredMode is generated as follows.

[0144] if (intraPredMode >= DIR) (expression INTRA-ANGULAR-PRED) iIdx = (((y + 1) * intraPredAngle) >> 5) iFact = ((y + 1) * intraPredAngle) & 31 predSamples[x][y] = Clip1((Σ(fT[i] * ref[x+iIdx+i]) + 32) >> 6) else / * intraPredMode < DIR* / iIdx = (((x + 1) * intraPredAngle) >> 5) iFact = ((x + 1) * intraPredAngle) & 31 predSamples[x][y] = Clip1((Σ(fT[i] * ref[y+iIdx+i]) + 32) >> 6) Here, ref[] is as explained in the template predicted image generation unit.

[0145] Here, Σ is the sum from i=0..NTAP-1, and fT is the interpolation filter coefficient for the intra-prediction image. The number of taps NTAP of the interpolation filter used to derive the intra-prediction image is, for example, 6, and is set to NTAP>=MTAP. MTAP is the number of taps of the interpolation filter filt for deriving the template prediction image tpredSamples.

[0146] fT may be derived from iFast as follows: fG[phase][j] is the coefficient of the interpolation filter for generating the intra-predicted image. fT[j] = fG[iFact][j] If the first condition of (STEP2) is satisfied, intra-predicted images predSamples1[][] and predSamples2[][] corresponding to IntraPredMode=kMode1, kMode2 are generated using (INTRA-ANGULAR-PRED). Then, the predicted image predSamples[][] of the current block is generated as a weighted sum of predSamples1[][] and predSamples2[][].

[0147] predSamples[x][y] = (weight1*predSamples1[x][y] + weight2*predSamples2[x][y]) >> 8 Otherwise, if the first condition of (STEP 2) is not satisfied, predSamples[][] is generated using the intra-predicted image predSamples1[][] corresponding to kMode1. predSamples1[][] is generated by (the formula INTRA-ANGULAR-PRED).

[0148] predSamples[x][y] = predSamples1[x][y] According to the above configuration, the template predicted image and cost calculation is performed only for intra prediction modes belonging to a specific group, which has the effect of reducing the amount of processing required for deriving a template predicted image.

[0149] (Another Example 1 of Example 1) As another example of the first embodiment, another group division method will be described. In this method, the groups are divided so that the number of elements in each group including the Angular prediction mode is approximately equal, and the total number of modes to be processed, i.e., the amount of processing, does not vary significantly depending on the group selected. In other words, intra prediction mode candidates in which the difference in the number of elements in groups including the Angular prediction mode is one or less are used.

[0150] The intra prediction mode selection unit 4714 may further separate modes into non-directional modes (Planar or DC) and directional modes, and perform group division using a candidate timdAngModeList for the directional modes. The non-directional modes may include a plurality of modes such as Planar, PlanarHor, and PlanarVer. Specifically, for each k, it is determined which group of Gr[0] to Gr[m] (m=numGr-1) timdModeList[k] (k=0..numCand-1) belongs to. If timdModeList[k] is Planar or DC, it is Gr[0]. Other than these, that is, a list in which only the angular prediction modes included in timdModeList are sorted in order of mode number is set as timdAngModeList[k] (k=0..numAngCand-1). numAngCand is the number of elements in timdAngModeList.

[0151] The intra prediction mode selection unit 4714 divides timdAngModeList into (numGr-2) groups (Gr[1] to Gr[numGr-1]) so that the number of elements in each group is approximately numAngCand / (numGr-2). An example of processing is shown below. Here, numAngCand=23, numGr=5.

[0152] for (m=0; m <numGr; m++) numGrList[m] = 0 m=0 th=0 GrList[0][numGrList[0]++] = 0 (=DC) GrList[0][numGrList[0]++] = 1 (=Planar) for (k=0; k <numAngCand; k++) { if (k >= th) { m++ th=numAngCand*m / (numGr-1) } GrList[m][numGrList[m]] = timdAngModeList[k] numGrList[m]++ } The subsequent processing can be the same as in embodiment 1. When group mm is divided again, the same method as the division described here may be used.

[0153] According to the above configuration, the difference in the number of elements between any of the groups Gr[1] to Gr[numGr-1] divided from the timdModeList and other groups is at most 1. In other words, no matter which group is selected as the group mm, the number of elements is almost the same, which has the effect of stably reducing the amount of processing.

[0154] In the above, an example of intra prediction modes ranging from 0 to numCand-1 (for example, numCand=67) has been described, but the intra prediction modes are not limited to this. For example, numCand=35 may be used. Alternatively, the intra prediction modes may take values ​​from -14 to 80, such as in the case of wide angle. In the case of wide angle, a new group may be assigned to intra prediction modes ranging from -14 to -1 or 67 to 80. Then, a representative mode or the best group may be identified.

[0155] The video decoding device of this embodiment includes a template derivation unit that generates a template image using an image of a template region adjacent to a target block, a reference sample derivation unit that generates a reference sample using an image of a template reference region near the target block, a template prediction image generation unit that generates a template prediction image for a predetermined intra prediction mode candidate using the reference sample, a template cost derivation unit that derives a cost from the template prediction image and the template image, and an intra prediction mode selection unit that selects the intra prediction mode of the target block based on the cost.

[0156] According to the above configuration, in TIMD prediction in which a template predicted image (intra-predicted image) is repeatedly derived for a current block using a plurality of intra-prediction candidates, an effect of reducing the amount of processing for deriving a predicted image of a template image is achieved.

[0157] The adder 312 generates a decoded image of the block by adding, for each pixel, the predicted image of the block input from the predicted image generation unit 308 and the prediction error input from the inverse quantization and inverse transform unit 311. The adder 312 stores the decoded image of the block in the reference picture memory 306, and also outputs it to the loop filter 305.

[0158] (Configuration of a video encoding device) Next, the configuration of the video encoding device 11 according to this embodiment will be described. Fig. 10 is a block diagram showing the configuration of the video encoding device 11 according to this embodiment. The video encoding device 11 includes a prediction image generating unit 101, a subtraction unit 102, a transformation and quantization unit 103, an inverse quantization and inverse transformation unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit, frame memory) 108, a reference picture memory (reference image storage unit, frame memory) 109, an encoding parameter determining unit 110, a parameter encoding unit 111, a prediction parameter derivation unit 120, and an entropy encoding unit 104.

[0159] The predicted image generation unit 101 generates a predicted image for each CU, which is an area obtained by dividing each picture of the image T. The predicted image generation unit 101 includes an intra predicted image generation unit 122 and an inter predicted image generation unit 123. The predicted image generation unit 101 operates in the same manner as the predicted image generation unit 308 already described, and therefore a description thereof will be omitted.

[0160] The subtraction unit 102 generates a prediction error by subtracting the pixel values ​​of the predicted image of the block input from the predicted image generation unit 101 from the pixel values ​​of the image T. The subtraction unit 102 outputs the prediction error to the transformation and quantization unit 103.

[0161] The transform / quantization unit 103 calculates transform coefficients by frequency transforming the prediction errors input from the subtraction unit 102, and derives quantized transform coefficients by quantizing the prediction errors. The transform / quantization unit 103 outputs the quantized transform coefficients to the parameter coding unit 111 and the inverse quantization / inverse transform unit 105.

[0162] The inverse quantization and inverse transform unit 105 is the same as the inverse quantization and inverse transform unit 311 (FIG. 4) in the video decoding device 31, and a description thereof will be omitted. The calculated prediction error is output to the addition unit .

[0163] The parameter coding unit 111 includes a header coding unit 1110, a CT information coding unit 1111, and a CU coding unit 1112 (prediction mode coding unit). The CU coding unit 1112 further includes a TU coding unit 1114. The following describes an outline of the operation of each module.

[0164] The header encoding unit 1110 performs encoding processing of parameters such as header information, division information, prediction information, and quantized transform coefficients.

[0165] The CT information encoding unit 1111 encodes the QT, MT (BT, TT) division information and the like.

[0166] The CU encoding unit 1112 encodes the CU information, prediction information, division information, and so on.

[0167] When a prediction error is included in a TU, the TU encoding unit 1114 encodes the QP update information and the quantized prediction error.

[0168] The entropy coding unit 104 receives the quantized transform coefficients and the coding parameters from the parameter coding unit 111. The entropy coding unit 104 entropy codes these to generate and output a coded stream Te.

[0169] The prediction parameter derivation unit 120 is a means including an inter-prediction parameter encoding unit 112 and an intra-prediction parameter encoding unit 113, and derives intra-prediction parameters and intra-prediction parameters from the parameters input from the encoding parameter determination unit 110. The derived intra-prediction parameters and intra-prediction parameters are output to the parameter encoding unit 111.

[0170] (Configuration of the intra-prediction parameter encoding unit 113) The intra prediction parameter encoding unit 113 encodes the IntraPredMode and the like input from the encoding parameter determination unit 110. The intra prediction parameter encoding unit 113 includes a part of the same configuration as the configuration in which the intra prediction parameter derivation unit 304 derives intra prediction parameters.

[0171] The adder 106 generates a decoded image by adding, for each pixel, the pixel value of the predicted block input from the predicted image generation unit 101 and the prediction error input from the inverse quantization and inverse transform unit 105. The adder 106 stores the generated decoded image in a reference picture memory 109.

[0172] The loop filter 107 performs deblocking filtering, SAO, and ALF on the decoded image generated by the adder 106. Note that the loop filter 107 does not necessarily have to include the above three types of filters, and may be configured, for example, as only a deblocking filter.

[0173] The prediction parameter memory 108 stores the prediction parameters generated by the coding parameter determination unit 110 in a predetermined location for each current picture and CU.

[0174] The reference picture memory 109 stores the decoded image generated by the loop filter 107 at a predetermined position for each current picture and CU.

[0175] The coding parameter determination unit 110 selects one set from among a plurality of sets of coding parameters. The coding parameters are the above-mentioned QT, BT or TT division information, prediction parameters, or parameters to be coded that are generated in relation to these. The predicted image generation unit 101 generates a predicted image using these coding parameters.

[0176] The coding parameter determination unit 110 calculates an RD cost value indicating the amount of information and the coding error for each of the multiple sets. The coding parameter determination unit 110 selects the set of coding parameters that minimizes the calculated cost value. As a result, the entropy coding unit 104 outputs the selected set of coding parameters as a coded stream Te. The coding parameter determination unit 110 stores the determined coding parameters in the predicted parameter memory 108.

[0177] The video decoding device of this embodiment includes a template derivation unit that generates a template image using an image of a template region adjacent to a target block, a reference sample derivation unit that generates a reference sample using an image of a template reference region near the target block, a template prediction image generation unit that generates a template prediction image for a predetermined intra prediction mode candidate using the reference sample, a template cost derivation unit that derives a cost from the template prediction image and the template image, an intra prediction mode selection unit that selects the intra prediction mode of the target block based on the cost, and a parameter encoding unit that encodes a template-based intra mode derivation flag, and the reference sample derivation unit derives the reference sample by referring to the template-based intra mode derivation flag.

[0178] In addition, a part of the video encoding device 11 and the video decoding device 31 in the above-mentioned embodiment, for example, the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the predicted image generating unit 308, the inverse quantization and inverse transform unit 311, the addition unit 312, the predicted image generating unit 101, the subtraction unit 102, the transform and quantization unit 103, the entropy encoding unit 104, the inverse quantization and inverse transform unit 105, the loop filter 107, the encoding parameter determination unit 110, and the parameter encoding unit 111 may be realized by a computer. In this case, a program for realizing this control function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read and executed by a computer system. In addition, the "computer system" referred to here is a computer system built into either the video encoding device 11 or the video decoding device 31, and includes hardware such as an OS and peripheral devices. In addition, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. Furthermore, the term "computer-readable recording medium" may also include devices that dynamically hold a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and devices that hold a program for a certain period of time, such as volatile memory inside a computer system that serves as a server or client in such cases. Furthermore, the above-mentioned program may be one that realizes part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0179] In addition, a part or the whole of the video encoding device 11 and the video decoding device 31 in the above-mentioned embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the video encoding device 11 and the video decoding device 31 may be individually processed, or a part or the whole may be integrated and processed. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the progress of semiconductor technology, an integrated circuit based on that technology may be used.

[0180] Although one embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc. are possible within the scope that does not deviate from the gist of the present invention. [Industrial Applicability]

[0181] The embodiments of the present invention can be suitably applied to a video decoding device that decodes coded data in which image data is coded, and a video coding device that generates coded data in which image data is coded, and can also be suitably applied to the data structure of coded data that is generated by a video coding device and referenced by the video decoding device. [Explanation of symbols]

[0182] 31 Image Decoding Device 301 Entropy Decoding Unit 302 Parameter Decoding Unit 308 Prediction Image Generation Unit 310 Intra-prediction image generation unit 31047 TIMD Forecasting Department 4701 Reference sample derivation part 4702 Template Derivation Unit 4703 Intra prediction unit 4711 Intra prediction mode candidate derivation unit, 4712 Template Prediction Image Generation Unit 4713 Template Cost Derivation Unit 4714 Intra prediction mode selection unit 311 Inverse quantization and inverse transformation unit 312 Addition section 11 Image encoding device 101 Prediction image generation unit 102 Subtraction section 103 Transformation and Quantization Section 104 Entropy coding unit 105 Inverse quantization and inverse transformation unit 107 Loop Filter 110 Encoding parameter determination unit 111 Parameter Encoding Unit 1110 Header encoding part 1111 CT information encoder 1112 CU encoding unit (prediction mode encoding unit) 1114 TU encoding section

Claims

1. a template predicted image generation unit that generates a template predicted image for a predetermined intra prediction mode candidate using an image of a template reference region near a target block; a template cost derivation unit that derives a cost from the template predicted image and the template image; an intra prediction mode selection unit that selects an intra prediction mode for the current block based on the cost, The video decoding device is characterized in that, as a first stage of processing, the intra-prediction mode selection unit generates a template predicted image for intra-prediction mode candidates of a specified intra-prediction mode, evaluates the cost, and derives the intra-prediction mode candidate with the lowest cost, and as a second stage of processing, evaluates the cost of modes other than the intra-prediction mode candidate.

2. The video decoding device according to claim 1, characterized in that the intra-prediction mode selection unit generates a template predicted image for each intra-prediction mode candidate selected from each intra-prediction mode group consisting of a set of intra-prediction modes, evaluates the cost, and for the group to which the intra-prediction mode candidate with the lowest cost belongs, derives and selects costs for modes other than the selected intra-prediction mode candidate.

3. The video decoding device of claim 1, characterized in that the intra-prediction mode selection unit divides predetermined intra-direction prediction mode candidates into multiple groups so that the difference in the number of elements between the groups is at most 1, generates a template predicted image for each intra-prediction mode candidate selected from each group, evaluates the cost, and, for the group to which the intra-prediction mode candidate with the lowest cost belongs, derives and selects costs for modes other than the selected intra-prediction mode candidate.

4. a template predicted image generation unit that generates a template predicted image for a predetermined intra prediction mode candidate using an image of a template reference region near a target block; a template cost derivation unit that derives a cost from the template predicted image and the template image; an intra prediction mode selection unit that selects an intra prediction mode for the current block based on the cost, In a first stage of processing, the intra-prediction mode selection unit generates a template predicted image for intra-prediction mode candidates for a predetermined intra-prediction mode, evaluates the costs, and derives the intra-prediction mode candidate with the lowest cost, and in a second stage of processing, evaluates the costs of modes other than the intra-prediction mode candidate.

5. A step of generating a template predicted image for a predetermined intra prediction mode candidate using an image of a template reference region near a target block; deriving a cost from the template predicted image and the template image; selecting an intra prediction mode for the current block based on the cost; In the first stage of processing, a template prediction image is generated for each intra-prediction mode candidate of a predetermined intra-prediction mode, costs are evaluated, and the intra-prediction mode candidate with the lowest cost is derived; A video encoding method, characterized in that, in a second stage of processing, costs for modes other than the intra-prediction mode candidates are evaluated.

Images