Video decoding apparatus, video coding apparatus

Abstract

This invention aims to enhance precision by adding the IBC / ITMP modes in TIMD method; and reducing complexity by using a lookup table method to replace division operations in TIMD and DIMD method.

Description

VIDEO DECODING APPARATUS, VIDEO CODING APPARATUS

[0001] The embodiments of the present invention relate to, a video decoding apparatus, a video coding apparatus.

[0002] A video coding apparatus which generates coded data by coding a video, and a video decoding apparatus which generates decoded images by decoding the coded data are used for efficient transmission or recording of videos.

[0003] For example, specific video coding schemes include H.264 / AVC, High‐Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC) schemes, and the like.

[0004] In such a video coding scheme, images (pictures) constituting a video are managed in a hierarchical structure including slices obtained by splitting an image, coding tree units (CTUs) obtained by splitting a slice, units of coding (coding units; which is referred to as CUs) obtained by splitting a coding tree unit, and transform units (TUs) obtained by splitting a coding unit, and are coded / decoded for each CU.

[0005] In such a video coding scheme, usually, a prediction image is generated based on a local decoded image that is obtained by coding / decoding an input image (a source image), and prediction error components (which may be referred to also as “difference images” or “residual images”) obtained by subtracting the prediction image from the input image are coded. Generation methods of prediction images include an inter‐picture prediction (an inter‐prediction) and an intra‐picture prediction (intra prediction).

[0006] In recent video coding and decoding techniques, NPL1 introduced an enhancement TIMD method. In this method, TIMD derives two intra prediciton modes, timdFirstMode and timdSecondMode, as well as correspinding weight, timdFirstWeight and timdSecondWeight, based on the SATD cost. If timdFirstMode and timdSecondMode contain DC or Planar modes, the weighted blend of the prediction images of timdFirstMode and timdSecondMode is used as the prediction result for the current block. If neither DC nor Planar modes are included, the one with the smaller SATD cost between DC or Planar modes is set as timdNonAngMode, and timdNonAngWeight is calculated based on the SATD cost. In this case, the prediction result for the current block is the weighted blend of timdFirstMode, timdSecondMode, and timdNonAngMode.

[0007] NPL 1: P.Andrivan, M.Blestel, “EE2‐1.20: TIMD fusion with non‐angular predictor”, JVET-AG0092, JVET 33th Meeting, Online, 17-26 January 2024

[0008] The current TIMD method includes only angular modes, DC mode, and Planar mode as candidate modes. In the current ECM software, there are intra prediction modes using BV (Block Vector) for intra prediction, called IBC / ITMP (Intra Block Copy / Intra Template Maching Prediction) modes. However, the TIMD method does not consider the IBC / ITMP modes. Additionally, there are a large number of floating‐point and integer division operations involved in the process of calculating weights in TIMD, which can impose a significant burden on both software and hardware implementations. Furthermore, the DIMD method and TIMD method use the same weighting derivation method, leading to a substantial amount of division operations as well.

[0009] The aim of this invention is to enhance prediction accuracy by incorporating IBC / ITMP modes into the candidate list of TIMD method, and reducing complexity through the replacement of division operations with a lookup table method.

[0010] According to an aspect of the present invention, the quality of the codecs can be improved without adding additional calculations.

[0011] FIG. 1 is a schematic diagram illustrating a configuration of an image transmission system according to the present embodiment.FIG. 2 is a diagram showing the hierarchical structure of the coded stream data.FIG. 3 is a schematic diagram showing the type of intra‐prediction mode (mode number).FIG. 4 is a schematic diagram of the video decoding apparatus.FIG. 5 shows the structure of the intra prediction image generation unit.FIG. 6 is a diagram showing the details of the timd prediction Unit.FIG. 7 is a diagram showing the position of template area and reference area used in timd.FIG. 8 shows position of the blocks adjacent to current block.FIG. 9 shows position of the blocks non‐adjacent to current block.FIG. 10 is a block diagram showing the structure of a video coding apparatus.FIG. 11 is a diagram showing the above and left template areas.FIG. 12 is a diagram showing the reference area and filter used in Dimd.FIG. 13 is a diagram showing the structrue of dim prediction unit 31046.

[0012] First Embodiment Hereinafter, embodiments of the present disclosure is described with reference to the drawings.

[0013] FIG. 1 is a schematic diagram illustrating a configuration of an image transmission system 1 according to the present embodiment.

[0014] The image transmission system 1 is a system in which a coding stream obtained by coding a coding target image is transmitted, the transmitted coding stream is decoded, and an image is displayed. The image transmission system 1 includes a video coding apparatus (image coding apparatus) 11, a network 21, a video decoding apparatus (image decoding apparatus) 31, and a video display apparatus (image display apparatus) 41.

[0015] An image T is input to the video coding apparatus 11.

[0016] The network 21 transmits a coding stream Te generated by the video coding apparatus 11 to the video decoding apparatus 31. The network 21 is the Internet, a Wide Area Network (WAN), a Local Area Network (LAN), or a combination thereof. The network 21 is not necessarily limited to a bidirectional communication network, and may be a unidirectional communication network configured to transmit broadcast waves of digital terrestrial television broadcasting, satellite broadcasting or the like. Furthermore, the network 21 may be substituted by a storage medium in which the coding stream Te is recorded, such as a Digital Versatile Disc (DVD: trademark) or a Blu‐ray Disc (BD: trademark).

[0017] The video decoding apparatus 31 decodes each of the coding streams Te transmitted from the network 21 and generates one or multiple decoded images Td which are decoded.

[0018] The video display apparatus 41 displays all or part of the one or multiple decoded images Td generated by the video decoding apparatus 31. For example, the video display apparatus 41 includes a display device such as a liquid crystal display and an organic Electro‐Luminescence (EL) display. Forms of the display include a stationary type, a mobile type, an HMD type, and the like. In addition, in a case that the video decoding apparatus 31 has a high processing capability, an image having high image quality is displayed, and in a case that the apparatus only has a lower processing capability, an image which does not require high processing capability and display capability is displayed.

[0019] Operator Operators and notations used in the present specification is described below.

[0020] >> is an arithmetic right bit shift, << is an arithmetic left bit shift, & is a bitwise AND, | is a bitwise OR, ^ is a bitwise XOR, |= is an OR assignment operator, and || indicates a logical sum.

[0021] x ? y : z is a ternary operator to take y in a case that x is true (other than 0) and take z in a case that x is false (0).

[0022] Clip3(x, y, z) is a function to clip z in a value equal to or greater than x and less than or equal to y, and a function to return x in a case that z is less than x (z < x), return y in a case that z is greater than y (z > y), and return z in other cases.

[0023] abs(a) is a function that returns the absolute value of a.

[0024] Int(a) is a function that returns the integer value of a.

[0025] floor(a) is a function that returns the maximum integer equal to or less than a.

[0026] ceil(a) is a function that returns the minimum integer equal to or greater than a.

[0027] a / d represents division of a by d (round down decimal places).

[0028] x = y..z represents x takes on integer values starting from y to z, inclusive, with x, y, and z being integer numbers and z being greater than or equal to y.

[0029] Structure of Coding Stream Te Prior to the detailed description of the video coding apparatus 11 and the video decoding apparatus 31 according to the present embodiment, a data structure of the coding stream Te generated by the video coding apparatus 11 and decoded by the video decoding apparatus 31 is described.

[0030] FIG. 2 is a diagram illustrating a hierarchical structure of data of the coding stream Te. The coding stream Te includes a sequence and multiple pictures constituting the sequence illustratively. (a) to (f) of FIG. 2 are diagrams illustrating a coded video sequence defining a sequence SEQ, a coded picture prescribing a picture PICT, a coding slice prescribing a slice S, a coding slice data prescribing slice data, a coding tree unit included in the coding slice data, and a coding unit (CU) included in each coding tree unit, respectively.

[0031] Coded Video Sequence In the coded video sequence (CVS, coding stream), a set of data referred to by the video decoding apparatus 31 to decode the coded sequence sequences to be processed is defined. As illustrated in FIG. 2, the CVS includes a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a picture (PICT), and Supplemental Enhancement Information (SEI).

[0032] In the video parameter set VPS, in a video including multiple layers, a set of coding parameters common to multiple videos and a set of coding parameters associated with the multiple layers and an individual layer included in the video are defined.

[0033] In the sequence parameter set SPS, a set of coding parameters referred to by the video decoding apparatus 31 to decode a target sequence is defined. For example, a width and a height of a picture are defined. Note that multiple SPSs may exist. In that case, any of multiple SPSs is selected from the PPS.

[0034] In the picture parameter set PPS, a set of coding parameters referred to by the video decoding apparatus 31 to decode each picture in a target sequence is defined. For example, a reference value (pic_init_qp_minus26) of a quantization step size used for decoding of a picture and a flag (weighted_pred_flag) indicatingan application of a weighted prediction are included. Note that multiple PPSs may exist. In that case, any of multiple PPSs is selected from each picture in a target sequence.

[0035] Coded Picture In the coded picture, a set of data referred to by the video decoding apparatus 31 to decode the picture PICT to be processed is defined. As illustrated in FIG. 2, the picture PICT includes a slice 0 to a slice NS‐1 (NS is the total number of slices included in the picture PICT).

[0036] Note that in a case that it is not necessary to distinguish each of the slice 0 to the slice NS‐1 below, subscripts of reference signs may be omitted. In addition, the same applies to other data with subscripts included in the coding stream Te which is described below.

[0037] Coding Slice In the coding slice, a set of data referred to by the video decoding apparatus 31 to decode the slice S to be processed is defined. As illustrated in FIG. 2, the slice includes a slice header and a slice data.

[0038] The slice header includes a coding parameter group referred to by the video decoding apparatus 31 to determine a decoding method for a target slice. Slice type specification information (slice_type) indicating a slice type is one example of a coding parameter included in the slice header.

[0039] Examples of slice types that may be specified by the slice type specification information include (1) I slice using only an intra prediction in coding, (2) P slice using a unidirectional prediction or an intra prediction in coding, and (3) B slice using a unidirectional prediction, a bidirectional prediction, or an intra prediction in coding, and the like. Note that the inter prediction is not limited to a uni‐prediction and a bi‐prediction, and the prediction image may be generated by using a larger number of reference pictures. Hereinafter, in a case that a slice is referred to as the P or B slice, the slice indicates a slice that includes a block in which the inter prediction may be used.

[0040] Note that, the slice header may include a reference to the picture parameter set PPS (pic_parameter_set_id).

[0041] Coding Slice Data In the coding slice data, a set of data referred to by the video decoding apparatus 31 to decode the slice data to be processed is defined. The slice data include CTUs as illustrated in FIG. 2. The CTU is a block of a fixed size (for example, 64 x 64) constituting a slice.

[0042] Coding Tree Unit In FIG. 2, a set of data referred to by the video decoding apparatus 31 to decode the CTU to be processed is defined. The CTU is split into coding units CUs, each of which is a basic unit of coding processing, by a recursive Quad Tree split (QT split), Binary Tree split (BT split), or Ternary Tree split (TT split). The BT split and the TT split are collectively referred to as a Multi Tree split (MT split). Nodes of a tree structure obtained by recursive quad tree splits are referred to as Coding Nodes. Intermediate nodes of a quad tree, a binary tree, and a ternary tree are coding nodes, and the CTU itself is also defined as the highest coding node.

[0043] Coding Unit As illustrated in FIG. 2, a set of data referred to by the video decoding apparatus 31 to decode the coding unit to be processed is defined. Specifically, the CU includes a CU header CUH, a prediction parameter, a transform parameter, a quantization transform coefficient, and the like. In the CU header, a prediction mode and the like are defined.

[0044] There are cases that the prediction processing is performed in units of CU or performed in units of sub‐CU obtained by further splitting the CU. In a case that the sizes of the CU and the sub‐CU are equal to each other, the number of sub‐CUs in the CU is one. In a case that the CU is larger in size than the sub‐CU, the CU is split into sub‐CUs. For example, in a case that the CU has a size of 8 x 8, and the sub‐CU has a size of 4 x 4, the CU is split into four sub‐CUs which include two horizontal splits and two vertical splits.

[0045] There are two types of predictions (prediction modes), which are an intra prediction and an inter prediction. The intra prediction refers to a prediction in an identical picture, and the inter prediction refers to prediction processing performed between different pictures (for example, between pictures of different display times).

[0046] Transform and quantization processing is performed in units of CU, but the quantization transform coefficient may be subjected to entropy coding in units of subblock such as 4 x 4.

[0047] Prediction parameter A prediction image is derived by a prediction parameter accompanying a block. The prediction parameter includes prediction parameters of the intra prediction and the inter prediction.

[0048] The prediction parameter of the intra prediction is described below. The intra prediction parameter includes a luma intra prediction mode IntraPredModeY and a chroma intra prediction mode IntraPredModeC. FIG. 3 is a schematic diagram indicating types (mode numbers) of the intra prediction mode. As illustrated in the diagram, for example, there are 67 types (0 to 66) of intra prediction modes. Additionally there are 28 types (‐14 to ‐1 and 67 to 80) of intra prediction modes depend on the aspect ratio of CU. For example, a planar prediction (0), a DC prediction (1), and Angular predictions (2 to 66) are present. Furthermore, for chroma, CCLM (Cross Component Linear Model) prediction mode (81 to 83),MMLM (Multi Mode Linear Model) prediction mode, and LM (Linear Model) prediction mode may be added.

[0049] Configuration of video decoding apparatus A configuration of the video decoding apparatus 31 (FIG. 4) according to the present embodiment is described.

[0050] The video decoding apparatus 31 includes an entropy decoding unit 301, a parameter decoding unit (prediction image decoding apparatus) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit 308, an inverse quantization and inverse transform processing unit 311, an addition unit 312, and a prediction parameter derivation unit 320. Note that a configuration in which the loop filter 305 is not included in the video decoding apparatus 31 is also used in accordance with the video coding apparatus 11 described later.

[0051] The parameter decoding unit 302 further includes a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit), and the CU decoding unit 3022 further includes a TU decoding unit 3024. These may be collectively referred to as a decoding module. The header decoding unit 3020 decodes, from coded data, parameter set information such as the VPS, the SPS, and the PPS, and a slice header (slice information). The CT information decoding unit 3021 decodes a CT from coded data. The CU decoding unit 3022 decodes a CU from coded data. In a case that a TU includes a prediction error, the TU decoding unit 3024 decodes QP update information (quantization correction value) and a quantization prediction error (residual_coding) from coded data.

[0052] Furthermore, an example in which a CTU and a CU are used as units of processing is described below, but the processing is not limited to this example, and processing in units of sub‐CU may be performed. Alternatively, by replacing the CTU and the CU by a block and replacing the sub‐CU by a subblock, and processing in units of blocks or subblocks may be performed.

[0053] The entropy decoding unit 301 performs entropy decoding on the coding stream Te input from the outside and separates and decodes individual codes (syntax elements). The separated codes include prediction information to generate a prediction image, a prediction error to generate a difference image, and the like. Entropy coding has a variable length coding method for syntax elements according to the context (probability model) adaptively selected according to the type of syntax elements and the surrounding conditions, and a variable length coding method for syntax elements using a predetermined table or formula.

[0054] The parameter decoding unit 302 notifies the entropy decoding unit 301 of which syntax elements need be decoded. The entropy decoding unit 301 outputs the syntax element to the prediction parameter derivation unit 320.

[0055] Configuration of Prediction Parameter Derivation Unit 320 The prediction parameter derivation unit 320 may derive the prediction parameters based on the output of the parameter decoding unit 302 and the prediction parameters which saved in the prediction parameter memory 307. The derived prediction parameters is output into the prediction image generation unit 308 and also is saved in the prediction parameter memory 307. The prediction parameter derivation unit may derive different prediction mode for the Luma and Chroma prediction.

[0056] The loop filter 305 is a filter provided in the coding loop, and is a filter that removes block distortion and ringing distortion and improves image quality. The loop filter 305 applies a filter such as a deblocking filter, a Sample Adaptive Offset (SAO), and an Adaptive Loop Filter (ALF) on a decoded image of a CU generated by the addition unit 312.

[0057] The reference picture memory 306 stores the decoded image of the CU generated by the addition unit 312 in a predetermined position for each target picture and target CU.

[0058] The prediction parameter memory 307 stores prediction parameters in a predetermined position for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores a parameter derived by the prediction parameter derivation unit 320, a prediction mode predMode separated by the entropy decoding unit 301, and the like.

[0059] The prediction image generation unit 308 receives input of the prediction parameter derived by the prediction parameter deviation unit 320, and the like. In addition, the prediction image generation unit 308 reads a reference picture from the reference picture memory 306. The prediction image generation unit 308 generates a prediction image of a block or a subblock by using the prediction parameter and the read reference picture (reference picture block) in the prediction mode indicated by the prediction mode predMode. Here, the reference picture block refers to a set of pixels (referred to as a block because they are normally rectangular) on a reference picture and is a region that is referred to to generate a prediction image.

[0060] Prediction Image Generation Unit 308 In a case that the prediction mode predMode indicates an intra prediction mode, the intra prediction image generation unit 310 performs an intra prediction by using an intra prediction parameter (luma intra prediction mode IntraPredModeY and / or chroma intra prediction mode IntraPredModeC) input from the prediction parameter derivation unit 320 and reference pixels read from the reference picture memory 306. In a case that the prediction mode predMode indicates an inter prediction mode, the inter prediction image generation unit performs an inter prediction by using an inter prediction parameter input from the prediction parameter derivation unit 320 and reference pixels read from the reference picture memory 306.

[0061] Specifically, the prediction image generation unit 308 reads, from the reference picture memory 306, a neighboring block in a predetermined range from a target block on a target picture. The predetermined range is neighboring blocks on the left, the top left, the top, and the top right of the target block, and the region referred to is different depending on the intra prediction mode.

[0062] The prediction image generation unit 308 generates a prediction image of the target block with reference to the read decoded pixel values and the prediction mode indicated by predMode, IntraPredModeY and / or IntraPredModeC. The prediction image generation unit 308 outputs the generated prediction image of the block to the addition unit 312.

[0063] The generation of the prediction image based on the intra prediction mode is described below. In the Planar prediction, the DC prediction, and the Angular prediction, a decoded peripheral region adjacent to (proximate to) the prediction target block is configured as a reference region R. Then, the pixels on the reference region R are extrapolated in a specific direction to generate the prediction image. For example, the reference region R may be configured as an L‐shaped region including the left and top (or further, top left, top right, bottom left) of the prediction target block.

[0064] Intra prediction image generation unit 310 A configuration of the intra prediction image generation unit 310 is described using FIG. 5. The intra prediction image generation unit 310 includes a reference sample filter unit 3103 (second reference image configuration unit), an intra prediction unit 3104), and a prediction image corrector 3105 (prediction image corrector, filter switching unit, weight coefficient changing unit).

[0065] Based on each reference pixel (unfiltered reference image) on the reference region R, a filtered reference image generated by applying a reference pixel filter (first filter), and the intra prediction mode, the intra prediction unit 3104 generates a prediction image of the target block, and outputs the generated image to the prediction image corrector 3105. The prediction image corrector 3105 corrects the prediction image in accordance with the intra prediction mode, and outputs a corrected prediction image.

[0066] Hereinafter, the units included in the intra prediction image generation unit 310 is described.

[0067] Reference sample filter unit 3103 The reference sample filter unit 3103 applies the reference pixel filter (first filter) to the unfiltered reference image to derive a filtered reference image s[x][y] at each position (x, y) on the reference region R, in accordance with the intra prediction mode. Specifically, a low pass filter is applied to the unfiltered reference image at each position (x, y) and its surroundings, and a filtered reference image is derived. Note that the low pass filter need not necessarily be applied in all the intra prediction modes, and the low pass filter may be applied in some intra prediction modes. Note that the filter applied to an unfiltered reference image on a reference region R in the reference sample filter unit 3103 is referred to as the “reference pixel filter (first filter)”, whereas a filter that corrects the prediction image in the prediction image corrector 3105 described below is referred to as a “boundary filter (second filter)”.

[0068] Configuration of intra prediction unit 3104 The intra prediction unit 3104 generates, based on the intra prediction mode, the unfiltered reference image, and the filtered reference pixel value, a prediction image (prediction pixel value, uncorrected prediction image) of the prediction target block, and outputs a generated image to the prediction image corrector 3105. The intra prediction unit 3104 includes a Planar prediction unit 31041, a DC prediction unit 31042, an Angular prediction unit 31043, an LM prediction unit 31044, an MIP prediction unit (Matrix‐based Intra Prediction) 31045, a DIMD (Decoder side Intra Mode Derivation) prediction unit 31046, and a newMPMList generation unit 31047 in the inside thereof. Also the intra prediction unit 3104 may include a TIMD (Template based Intra Mode Derivation) prediction unit 31048, shown in FIG. 5. The intra prediction unit 3104 selects a specific predictor in accordance with the intra prediction mode, and inputs an unfiltered reference image and a filtered reference image thereto. The relationship between the intra prediction mode and the corresponding predictor is as follows.

[0069] ‐ Planar prediction ... Planar prediction unit 31041 ‐ DC prediction ... DC prediction unit 31042 ‐ Angular prediction ... Angular prediction unit 31043 ‐ LM prediction ... LM prediction unit 31044 ‐ MIP prediction ... MIP prediction unit 31045 ‐ DIMD prediction ... DIMD prediction unit 31046 ‐ TIMD prediction ... TIMD prediction unit 31047 Planar prediction The Planar prediction unit 31041 generates a prediction image q[x][y] by linearly adding multiple filtered reference images s[x][y] in accordance with the distance between the prediction pixel position and the reference pixel position, and outputs the generated image to the prediction image corrector 3105.

[0070] DC prediction The DC prediction unit 31042 derives a DC prediction value corresponding to the average value of the filtered reference image s[x][y], and outputs a prediction image q[x][y], which takes the DC prediction value as a pixel value.

[0071] Angular prediction The Angular prediction unit 31043 generates a prediction image q[x][y] using the filtered reference image s[x][y] in a prediction direction (reference direction) indicated by the intra prediction mode, and outputs the generated image to the prediction image corrector 3105.

[0072] LM prediction The LM prediction unit 31044 predicts the pixel value of the chroma based on the pixel value of luma. More specifically, a linear model is used to generate a prediction chroma image (Cb, Cr) based on the decoded luma image. As an example of LM prediction, there is a CCLM (cross component linear model prediction) prediction. CCLM prediction is a prediction method using a linear model to predict chroma from luma to same block.

[0073] MIP prediction The MIP prediction unit 31045 generates a prediction image q[x][y] by the product sum operation on the reference sample s[x][y] and the weight matrix derived from the neighboring region, and outputs the prediction image q[x][y] to the prediction image corrector 3105.

[0074] (Division approximation unit) In the TIMD and DIMD, numerous division operations are used in location dependent weight and other parts. In this embodiment, the complexity can be reduced by removing these division operations with a lookup table method. This lookup table method is executed by the division approximation unit in the DIMD prediction unit 31046 or the TIMD prediction unit 31047. For the division operation Gx / Gy, the result can be approximated using the following methods as a function of divIntN(N is 16 or 32, i.e. divInt16 or divInt32). The division approximation unit derives a logarithmic value of the denominator (a first shift value x) value Gy and derive a norm value by left shifting the denominator by a positive interger constant K, right shifting the value by the logarithmic value (or the first shift value x), and “bit‐wise and” with a constant value of (1<<K)‐1 (e.g. 15, 31, …), and derive a variable v with a table loop up using a 1<<K (e.g. 16, 32) entry table by indexing by the norm value followed by “bit‐wise or” or adding a constant 1<<(K‐1), 8, 16, the output the variable v and a shift value of the sum of the second shift value x and a constant value of K ‐1 and 1 in the case the norm value is not 0.

[0075] The division approximation unit derives two variables, the v value and the shift value with the following function named getDivValue16 (Gx, Gy, baseShift) where approximate result is derived using a table of length 16: x = Floor( Log2( Gy ) ) norm_Gy = ( (Gy<< 4 >> x ) & 15 ) v = DivSigTable[ normDiff ] | 8 x += (norm_Gy != 0) shift = x + 3 return (v<<baseShift), (shift+baseShift) where DivSigTable

[0016] = { 0, 7, 6, 5 ,5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0 }. baseShift opeartion may be omitted if baseShift is equal to 0. The sae applies in getDivValue32.

[0076] Alternatively, getDivValue16 (Gx, Gy, baseShift) may apply the following.

[0077] x = Floor( Log2( Gy ) ) norm_Gy = ( (Gy<< 4 >> x ) & 15 ) v = DivSigTable16[ normDiff ] | 8 x += (norm_Gy != 0) y = Floor( Log2( Gx ) ) + 1 add = 1<<y>>1 a = (( (Gx * v) << baseShift) + add) >> y shift = x + 3 - y + baseShift return a, shift where DivSigTable16

[0016] = { 0, 7, 6, 5 ,5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0 } The above part of “y = Floor( Log2( Gx ) ) + 1, add = 1<<y>>1, a = (( (Gx * v) << baseShift) + add) >> y” may be derive as y = Floor( Log2( Gx ) ) + 1 - baseShift add = 1<<y>>1 a = ( (Gx * v) + add) >> y or y = Floor( Log2( Gx ) ) + 1 if (y >= baseShift) { y = y - baseShift} else {baseShift = baseShift - y, y = 0} add = 1<<y>>1 a = ( (Gx * v) + add) >> y

[0078] This base shift value baseShift substraction in the logarithmic value of the numerator for right shifting of the multiplication of numerator and the value v reduce the dymamic range of the multiplication to avoid Gx * y << baseShift and reduce the right shifting of y. The same applies at the getDivValue32.The division approximation unit derives a logarithmic value of the denominator value Gy and derive a norm value by left shifting the denominator by a positive interger constant K (=4 , 5, 6..), right shifting the value by the logarithmic value, and “bit‐wise and” with a constant value of (1<<K)‐1, and derive a variable v with a table loop up using a 1<<K entry table by indexing by the norm value followed by “bit‐wise or” or adding a constant 1<<(K‐1), derives the second shift value y by the logarithmic value of numerator, the output is a variable a derived by right shifting the product of the numerator value Gx and the variable v with the second shift value y and the output is a shift value of the sum of the second shift value x and a constant value of K - 1 substracted by the second shift value y. The second shift value may be substracded by the base shift vaue baseShift.

[0079] The division approximation unit derives two variables, the result and the shift value with the following function named getDivValue32(Gx, Gy, baseShift) where approximate result is derived using a table of length 32: x = Floor( Log2( Gy ) ) norm_Gy = ( (Gy<< 5 >> x ) & 31 ) v = DivSigTable32[ normDiff ] | 16 x += (norm_Gy != 0) shift = x + 4 offset = 1<<shift>>1 return (v<<baseShift), (shift+baseShift)where DivSigTable32

[0032] = { 0, 15, 14, 13, 12, 12, 11, 10, 10, 9, 8, 8, 7, 7, 6, 6, 5, 5, 4, 4, 4, 3, 3, 3, 2, 2, 2, 1, 1, 1, 0, 0 }.

[0080] Alternatively, getDivValue32 (Gx, Gy, baseShift) may apply the following.

[0081] x = Floor( Log2( Gy ) ) norm_Gy = ( (Gy<< 5 >> x ) & 31 ) v = DivSigTable32[ normDiff ] | 16 x += (norm_Gy != 0) y = Floor( Log2( Gx ) ) + 1 add = 1<<y>>1 a = (( (Gx * v) << baseShift) + add) >> y shift = x + 4 - y+ baseShift return a, shift where DivSigTable32

[0032] = { 0, 15, 14, 13, 12, 12, 11, 10, 10, 9, 8, 8, 7, 7, 6, 6, 5, 5, 4, 4, 4, 3, 3, 3, 2, 2, 2, 1, 1, 1, 0, 0 }. The baseShift substraction in getDivValu16 may apply too.

[0082] The division approximation unit derives the result of (Gx*sumWeight) / Gy with the following function named divInt16Sum (Gx, Gy, sumWeight) where approximate result is derived using a table of length 32: x = Floor( Log2( Gy ) ) norm_Gy = ( (Gy<< 4 >> x ) & 15 ) v = DivSigTable[ normDiff ] | 8 x += (norm_Gy != 0) shift = x + 3 offset = 1<<shift>>1 result = (Gx* v* sumWeight + offset) >> shift where DivSigTable

[0016] = { 0, 7, 6, 5 ,5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0 }. sumWeight may be ommitedd as divInt16. The same apply in divInt32. divIntN() is divIntNSum (Gx, Gy, 1).

[0083] Alternatively, divInt16Sum (Gx, Gy, sumWeight) may apply the following.

[0084] x = Floor( Log2( Gy ) ) norm_Gy = ( (Gy<< 4 >> x ) & 15 ) v = DivSigTable16[ normDiff ] | 8 x += (norm_Gy != 0) y = Floor( Log2( Gx ) ) + 1 add = 1<<y>>1 a = (Gx * v + add) >> y shift = x + 3 ‐ y offset = 1<<shift>>1 result = (Gx*a* sumWeight + offset) >> shift where DivSigTable16

[0016] = { 0, 7, 6, 5 ,5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0 }

[0085] The division approximation unit derives the result of Gx* sumWeight / Gy with the following function named divInt32(Gx, Gy) where approximate result is derived using a table of length 32: x = Floor( Log2( Gy ) ) norm_Gy = ( (Gy<< 5 >> x ) & 31 ) v = DivSigTable32[ normDiff ] | 16 x += (norm_Gy != 0) shift = x + 4 offset = 1<<shift>>1 result = (Gx* v* sumWeight + offset) >> shift where DivSigTable32

[0032] = { 0, 15, 14, 13, 12, 12, 11, 10, 10, 9, 8, 8, 7, 7, 6, 6, 5, 5, 4, 4, 4, 3, 3, 3, 2, 2, 2, 1, 1, 1, 0, 0 }.

[0086] divInt32 (Gx, Gy) may apply the following.

[0087] x = Floor( Log2( Gy ) ) norm_Gy = ( (Gy<< 5 >> x ) & 31 ) v = DivSigTable32[ normDiff ] | 16 x += (norm_Gy != 0) y = Floor( Log2( Gx ) ) + 1 add = 1<<y>>1 a = (Gx * v* sumWeight + add) >> y shift = x + 4 ‐ y offset = 1<<shift>>1 result = (Gx*a + offset) >> shift where DivSigTable32

[0032] = { 0, 15, 14, 13, 12, 12, 11, 10, 10, 9, 8, 8, 7, 7, 6, 6, 5, 5, 4, 4, 4, 3, 3, 3, 2, 2, 2, 1, 1, 1, 0, 0 }

[0088] DIMD prediction The DIMD prediction unit 31046 employs a prediction method that generates prediction images using decoder‐side derived (not signaled) intra prediction modes. During the encoding and decoding process, the DIMD method utilizes neighborhood information to derive multiple suitable intra prediction modes for the target block, along with the corresponding weights for each mode. Subsequently, the DIMD prediction unit 31046 utilizes these intra prediction modes to generate prediction images.

[0089] TIMD prediction The TIMD method is an intra prediction technique used to generate prediction images using derived intra prediction modes. The TIMD prediction unit 31047 is derives suitable intra prediction modes for the target block by assuming that the direction and appropriate intra prediction mode of the template area (TA) neighboring the target block should be similar to the direction and appropriate intra prediction mode of the target block. Specifically, the unit derives one or more intra prediction modes for the template area (TA) by trying an intra prediction using images from the reference region (Reference Area of Template Area, RATA) near the template area (TA) to generate prediction images (tpredSamples). If the cost (such as the absolute difference sum, SAD) between tpredSamples and the reconstructed pixels (tempSamples) of the TA region is small, the intra prediction mode candidates are selected as timd intra prediction modes and added to intra prediction mode candidate list of the template area (TA). The TIMD prediction unit 31047. The TIMD prediction unit 31047 employs these timd intra prediction modes to generate prediction image for the current block.

[0090] Configuration of prediction image corrector 3105 The prediction image corrector 3105 corrects the prediction image output from the intra prediction unit 3104 in accordance with the intra prediction mode. Specifically, the prediction image corrector 3105 derives, by performing weighted addition (weighted‐averaging) on the unfiltered reference image and the prediction image for each pixel of the prediction image, in accordance with the distance between the reference region R and the target prediction pixel, the prediction image (corrected prediction image) Pred in which the prediction image is modified. Note that in some intra prediction modes (for example, Planar prediction, DC prediction, or the like), the prediction image corrector 3105 may not correct the prediction image, and the output of the intra prediction unit 3104 may be used as the prediction image.

[0091] Application examples The parameter decoding unit 302 decodes signaled data to determine the TIMD flag (timd_flag) for each block. A block can be a Coding Unit (CU), Transform Unit (TU), sub‐block, etc. If the timd_flag for the target block is 0, then the parameter decoding unit 302 decodes syntax elements related to intra prediction modes. However, if timd_flag is 1, the parameter decoding unit 302 may skip decoding syntax elements related to intra prediction modes from the encoded data. When timd_flag is 1, the prediction image of the current block is derived using the TIMD method, which may calculate timdFirstMode, timdSecondMode, timdNonAngMode, timdWeight1, timdWeight2, and timdWeight3 based on the surrounding samples of the current block. The TIMD prediction image is then derived based on these parameters.

[0092] TIMD prediction unit 31047 FIG. 6 shows the structure of the TIMD prediction unit 31047 in this embodiment.

[0093] The TIMD prediction unit 31047 comprises the Reference Sample Derivation Unit 4701, Template Derivation Unit 4702, Weights Derivation Unit 4703, Intra Prediction Unit 4704, and Prediction Mode Derivation Device 4710. The Prediction Mode Derivation Device 4710 consists of the Candidate Derivation Unit 4711 and the Intra Prediction Mode Derivation Unit 4717. The Intra Prediction Mode Derivation Unit 4717 is composed of the Template Prediction Image Generation Unit 4713, the Template Cost Derivation Unit 4714, and the Comparison Unit 4715. TIMD prediction unit 31047 executes the following steps: (STEP1) Generate tpredSamples for the template area using the specified intra prediction mode and the reference area RATA.

[0094] (STEP1‐1) Generate tpredSamples for all candidate modes.

[0095] (STEP1‐2) Calculate the cost values corresponding to the difference between each generated tpredSamples (prediction image of the template area) and tempSamples (reconstructed image of the template area).

[0096] (STEP2) Choose the candidate modes with the minimum cost, i.e., the intra prediction mode corresponding to the tpredSamples deemed to have the highest prediction accuracy, as the prediction mode for TIMD.

[0097] (STEP3) Generate the prediction image using the selected intra prediction mode.

[0098] The processing of (STEP1)~(STEP3) is described in detail below: (Reference Area and Template Area) FIG.7 illustrates the template area (TA) and the reference area (RATA) used in TIMD method. The template area (TA) corresponds to the region adjacent to the current block, while the reference area (RATA) is used as a reference when generating the template prediction image (tpredSamples). The tpredSamples will be compared to the reconstructed image (tempSamples) of the template area to determine the cost. The internal prediction mode and weights of TIMD are determined based on the cost, and the prediction result is obtained accordingly.

[0099] Template Derivation Unit 4702 The Template Derivation Unit 4702 exports the template image tempSamples. FIG.7 shows the spatial relationship between the template area TA and the current block. The template area TA consists of a L‐shaped region with already decoded pixels, represented by recSamples. The template area can be subdivided into five parts: the above part, left part, above‐left part, bottom‐left part, and above‐right part. Given that the coordinates of the above‐left corner pixel of the current block are denoted as (x0, y0), the template area can be defined as follows: (Above template area) tempSamples[i][j] = recSamples[x0+i][y0+j] (i=0..cbWidth‐1, j=‐tH..‐1) (Left template area) tempSamples[i][j] = recSamples[x0+i][y0+j] (i=‐tW..‐1, j=0..cbHeight‐1) (Above‐right template area) tempSamples[i][j] = recSamples[x0+i][y0+j] (i=cbWidth..2cbWidth‐1, j=‐tH..‐1) (Bottom‐left template area) tempSamples[i][j] = recSamples[x0+i][y0+j] (i=‐tW..‐1, j=cbHeight..2cbHeight‐1) (Above‐left template area) tempSamples[i][j] = recSamples[x0+i][y0+j] (i=‐1..‐tW, j=‐1..‐tH)

[0100] The region on recSamples used for deriving tempSamples is represented by the set of coordinates (i, j). Here, tW and tH denote the width and height of the template area, respectively. tW and tH may be positive integers greater than 0, such as 1, 2, 3, 4, etc.

[0101] The Reference Sample Derivation Unit 4701 The Reference Sample Derivation Unit 4701 extracts reference samples refUnit for the reference area RATA. This operation may alternatively be performed by the reference sample filtering unit 3103.

[0102] refUnit[x][y] = recSamples[x0+x][y0+y] Where x=‐rW‐1, y=‐1‐rH..refH‐1, x=‐rW..refW‐1, y=‐1‐rH. Here, rW and rH represent the width and height of the RATA region. In FIG. 7, tW=1, tH=1. refW=2*cbWidth, refH=2*cbHeight. The Reference Sample derivation Unit 4701 may apply filtering to the reference samples refUnit[x][y] to derive reference samples p[x][y].

[0103] The specific implementation may be as follows: Three patterns are defined for the template area usage: (Pattern 1): Utilize only the above and left template areas.

[0104] (Pattern 2): Utilize only the above, left and above‐left template areas.

[0105] (Pattern 3): Utilize the above, left, above‐left, bottom‐left, and above‐right template areas.

[0106] (Method 1) All target blocks employ the template area of Pattern 1, or utilize the template area of Pattern 2, or utilize the template area of Pattern 3 for prediction.

[0107] (Method 2) Determine the template area usage based on the size of the current block. Set a threshold thresh, when cbWidth*cbHeight < thresh, then use the template area of Pattern 1 for the current block, when cbWidth*cbHeight >= thresh, then use the template area of Pattern 2 for the current block. Or, when cbWidth*cbHeight < thresh, use the template area of Pattern 2 for the current block, when cbWidth*cbHeight >= thresh, use the template area of Pattern 3 for the current block. Here, the thresh may be an integer in the range of 0 to1024, like 256, 512,and so on.

[0108] (Method 3) Select the template area based on the length and width of the current block. Set two thresholds thresh1 and thresh2 (0<=thresh1,thresh2<=256). When cbWidth < thresh1 and cbHeight < thresh2, utilize the template area of Pattern 1. When cbWidth >= thresh1 and cbHeight >= thresh2, utilize the template area of Pattern 3. When cbWidth < thresh1 and cbHeight >= thresh2, utilize the above, left, above‐left, and bottom‐left template areas. When cbWidth >= thresh1 and cbHeight < thresh2, utilize the above, left, above‐left, and above‐right template areas.

[0109] Here, the thresh1 and thresh2 may be set to same integer in the range from 4 to 64, like 4, 8, 16,and so on. Also thresh1 and thresh2 may be set to different integer the range from 4 to 64.

[0110] Prediction Mode Derivation Device 4710 The Prediction Mode Derivation Device 4710 consists of the Candidate Derivation Unit 4711 and the Intra Prediction Mode Derivation Unit 4717.

[0111] Candidate Derivation Unit 4711 The Candidate Derivation Unit 4711 initially derives candidate lists candList1, candList2, and candList3 from the adjacent and non‐adjacent blocks to the target block. candList1 may include angular prediction modes, planar prediction mode, and DC prediction mode from MPM list, and intra prediction modes derived by the DIMD method. candList2 may include extended angular prediction modes, with indices numbered [‐1,‐14] and [67,80] as shown in FIG. 3. candList3 may include Intra Block Copy (IBC) and Intra Template Matching Prediction (ITMP) modes.

[0112] When deriving candList3, both adjacent and non‐adjacent blocks of the current block are used. If a block is predicted using the IBC or ITMP method, the block vector (BV) used for its prediction is added to candList3. As shown in FIG.8, adjacent blocks of the current block include A0, A1, B0, B1, B2, etc., located at the bottom‐left, left, above‐right, above, and above‐left of the current block, respectively. As illustrated in FIG.9, there are a total of 18 non‐adjacent blocks of the current block. Assuming the coordinates of the pixel in top‐left corner of the current block are (x, y), and the width and height of the current block are cbWidth and cbHeight, respectively, then the non‐adjacent blocks include blocks containing the following pixels: (x‐cbWidth‐1, y+2cbHeight‐1) (x‐2cbWidth‐1, y+3cbHeight‐1) (x‐3cbWidth‐1, y+4cbHeight‐1) (x‐4cbWidth‐1, y+5cbHeight‐1) (x‐cbWidth‐1, y‐cbHeight / 2) (x‐2cbWidth‐1, y‐cbHeight / 2) (x‐3cbWidth‐1, y‐cbHeight / 2) (x‐cbWidth‐1, y‐cbHeight‐1) (x‐2cbWidth‐1, y‐2cbHeight‐1) (x‐3cbWidth‐1, y‐3cbHeight‐1) (x‐4cbWidth‐1, y‐4cbHeight‐1) (x+cbWidth / 2, y‐cbHeight‐1) (x+cbWidth / 2, y‐2cbHeight‐1) (x+cbWidth / 2, y‐3cbHeight‐1) (x+2cbWidth‐1, y‐cbHeight‐1) (x+3cbWidth‐1, y‐2cbHeight‐1) (x+4cbWidth‐1, y‐3cbHeight‐1) The exploration order of these blocks is not specified.

[0113] Intra Prediction Mode Derivation Unit 4717 Intra Prediction Mode Derivation Unit 4717 is comprised of the Template Prediction Image Generation Unit 4713, the Template Cost Derivation Unit 4714, and the Comparison Unit 4715. The input to the Intra Prediction Mode Derivation Unit 4717 consists of candList1[], candList2[], and candList3[] from the outputs of the Candidate Derivation Unit 4711. For the inputs candList1[], candList2[], and candList3[], the Intra Prediction Mode Derivation Unit 4717 merges candList1[] and candList2[] into a candidate prediction mode list named timdAngCandList[], and renames candList3[] as timdNonAngCandList[]. Within the Intra Prediction Mode Derivation Unit 4717, there is a loop structure that iterates over each candidate prediction mode present in timdAngCandList[], timdAddiAngCandList[], and timdNonAngCandList[]. The timdAddiAngCandList[] are obtained during the loop process based on the elements in timdAngCandList[]. The exploration sequence starts with exploring the elements in timdAngCandList[], followed by exploring the elements in timdAddiAngCandList[], and finally, exploring the elements in timdNonAngCandList[]. For each candidate prediction mode, the unit generates the prediction image for the template area, calculates the cost based on the differences between the prediction image and the reconstructed image in the template area, and compares the costs of prediction images to determine the best (the least cost) intra prediction mode as timdFirstMode, the second best (the second least cost) intra prediction mode as timdSecondMode, the best (the least cost) intra prediction mode in non‐angular intra prediction modes (e.g. DC or planar or IBC / ITMP) as timdNonAngMode and stores corresponding costs as timdCost1 (the cost of timdFirstMode), timdCost2 (the cost of timdSecondMode), and timdCost3 (the cost of timdNonAngMode) respectively. These specific operations are carried out within the Template Prediction Image Generation Unit 4713, the Template Cost Derivation Unit 4714, and the Comparison Unit 4715.

[0114] Template Prediction Image Generation Unit 4713.

[0115] The Template Prediction Image Generation Unit 4713 is responsible for generating the prediction image (tpredSamples) for the template area based on the pixels in the reference area and the specified prediction modes. The input to this unit includes the reference area pixels refUnit[][] output by the Reference Sample Derivation Unit 4701, as well as timdAngCandList[] and timdNonAngCandList[]. The Template Prediction Image Generation Unit 4713 inputs refUnit[][] and each candidate from timdAngCandList[] and timdNonAngCandList[] into the Intra prediction unit 4704. The Intra prediction unit 4704 calculates tpredSamples based on the inputs and returns it to the Template Prediction Image Generation Unit 4713.

[0116] Intra prediction unit 4704 The Intra prediction unit 4704 is responsible for generating the intra prediction image within the specified region based on the input reference pixels and the specified intra prediction mode. In the TIMD method, it generates the prediction image tpredSamples for the template area.

[0117] Template Cost Derivation Unit 4714 The Template Cost Derivation Unit 4714 calculates the difference between tpredSamples and tempSamples (reconstructed image of the template area). This difference can be represented using SATD cost or SAD cost, or other metrics. The Template Cost Derivation Unit 4714 sets the current intra prediction mode from the TIMD candidate intra prediction modes, corresponding to the current tpredSamples as curMode and sets the calculated cost (difference value) as curModeCost. curMode and curModeCost are then output to the Comparison Unit 4715.

[0118] Comparison Unit 4715 The Comparison Unit 4715 updates timdFirstMode, timdSecondMode, timdNonAngMode, timdCost1, timdCost2, and timdCost3 based on the input curMode and curModeCost. In the initial stage (i.e., the first time loop in the Intra prediction mode derivation unit 4717, exploring the first element in timdAngCandList[]), timdFirstMode, timdSecondMode and timdNonAngMode are set to planar mode, and timdCost1, timdCost2 and timdCost3 are set to a predefined maximum value, e.g. MAX_UINT64, where MAX_UINT64 represents the maximum value that an unsigned 64‐bit integer can hold. Then, for each loop, curMode and curModeCost are processed as follows: For each candidate mode belonging to timdAngCandList[], the following operations are performed with the obtained curMode and curModeCost:

[0119] (Embodiment1) <Fusion of two modes> In one embodiment, the Intra prediction unit 4704 may use / fuse two timd intra prediction mode to generate the prediction of the target block. The timd intra prediction mode may be the best one (timdFirstMode) and the second best (timdSecondMode). Using the intra prediction unit 4704 or intra prediction image generation unit 310, generate the prediction images predImage1 and predImage2 for the target block area corresponding to the intra prediction modes timdFirstMode, timdSecondMode.

[0120] predImage[x][y] = (predImage1[x][y] * weight1 + predImage2[x][y] * weight2 + offset) >> shiftVal where shiftVal is a positive integer, such as 5, 6, 7, etc.

[0121] offset = 1<<(shiftVal‐1) or 1<<shiftVal>>1 sum = timdCost1 + timdCost2 baseShift is a positive value, such as 4, 5, 6, 7, etc weight1, shiftVal = getDivValueN(timdCost2, sum, baseShift) weigh2 = (1<<shiftVal) - weight1 N = 16 or 32 (constant weight for two modes fusion) In addition to determining the weights of each prediction mode based on the ratio of costs, weights may also be set to constant values. The prediction image predImage is then derived as follows: predImage[x][y] = (predImage1[x][y] * timdWeight1 + predImage2[x][y] * timdWeight2 + offset) >> shiftVal where sumWeight = 1 <<shiftVal and offset = 1 >> (shiftVal‐1) or 1<<shiftVal>>1.

[0122] timdWeight1 = W1 and timdWeight2 = W2.

[0123] W1 and W2 are positive integer constants that satisfy W1 + W2 = sumWeight.

[0124] W1, W2, and shiftVal may be set to, for example 1, 1, 1 or 3, 1, 2 or or 5, 3, 3 or 11, 5, 4 or 25, 7, 5 or other combinations.

[0125] <Fusion of three modes> In another embodiment, the Intra prediction unit 4704 may use / fuse three timd intra prediction mode to generate the prediction of the target block. The timd intra prediction mode may be the best one (timdFirstMode), the second best (timdSecondMode) and the best non‐angular mode (timdNonAngMode). Using the intra prediction unit 4704 or intra prediction image generation unit 310, generate the prediction images predImage1, predImage2, and predImage3 for the target block area corresponding to the intra prediction modes timdFirstMode, timdSecondMode, and timdNonAngMode.

[0126] predImage[x][y] = (predImage1[x][y] * weight1 + predImage2[x][y] * weight2 + predImage3[x][y] * weight3 + offset) >> shiftVal where sum = timdCost1 + timdCost2 + timdCost3 baseShift is a positive value, such as 4, 5, 6, 7, etc weight1, shiftVal = getDivValueN(timdCost2 + timdCost3, 2*sum, baseShift) weight2, shiftVal = getDivValueN(timdCost1 + timdCost3, 2*sum, baseShift) offset = (1<<shiftVal)>>1 or 1<<shiftVal>>1 In another embodiment, the Intra prediction unit 4704 may use / fuse two timd intra prediction mode to generate the prediction of the target block by using pre‐defined constant weights.

[0127] (constant weight for three modes fusion) In addition to determining the weights of each prediction mode based on the ratio of costs, weights may also be set to constant values. The prediction image predImage is then derived as follows.

[0128] predImage[x][y] = (predImage1[x][y] * timdWeight1 + predImage2[x][y] * timdWeight2 + predImage3[x][y] * timdWeight3 + offset) >> shiftVal where offset = 1<<(shiftVal‐1) or 1<<shiftVal>>1 timdWeight1 = W1, timdWeight2 = W2, and timdWeight3 = W3.

[0129] W1, W2, and W3 are positive integer constants that satisfy W1 + W2 + W3 = 1<<shiftVal. W1, W2, W3, shiftVal may be set to, for example, 2, 1, 1, 2 or 4, 3, 1, 3 or 3, 3, 2, 3 or 8, 5, 3, 4 or 7, 5, 4, 4 or 20, 7, 5, 5 or other combinations.

[0130] (Embodiment2) In another embodiment, more advanced methods are described below. After all candidate modes in timdAngCandList have been explored, the following operations are performed. Generate a list timdAddiAngCandList[] with a maximum length of 4 based on the current timdFirstMode and timdSecondMode. The candidate modes in this list are numbered as timdFirstMode‐1, timdFirstMode+1, timdSecondMode‐1, and timdSecondMode+1, where timdFirstMode‐1, timdFirstMode+1, timdSecondMode‐1, and timdSecondMode+1 must belong to the interval [2, 130]. Then, for each candidate mode belonging to timdAddiAngCandList, the following operations are performed with the obtained curMode and curModeCost: Finally, for each candidate mode belonging to timdNonAngCandList, the following operations are performed with the obtained curMode and curModeCost:

[0131] After exploring all candidate lists, the Intra Prediction Mode Derivation Unit 4717 outputs the exploration results, including timdFirstMode, timdSecondMode, timdNonAngMode, timdCost1, timdCost2, and timdCost3, to the Weights Derivation Unit 4703.

[0132] Weights Derivation Unit 4703 Weights Derivation Unit 4703 calculates timdWeight1, timdWeight2, and timdWeight3 based on timdCost1, timdCost2, and timdCost3. The calculation method is as follows:

[0133] where timdWeight1,timdWeight2 may be derived by the division approximation unit as follows.

[0134] sum = timdCost1 + timdCost2 + timdCost3 baseShift = 5, 6, 7, etc timdWeight1 = divIntN(timdCost2 + timdCost3, 2*sum, baseShift) timdWeight2 = divIntN(timdCost1 + timdCost3, 2*sum, baseShift) Where N may be 16 or 32.

[0135] These weights are then output from the Weights Derivation Unit 4703.

[0136] (Location Dependent Weight) The location dependent weight method determines the weights by assessing the location dependency of the intra prediction modes derived from TIMD / DIMD on the template areas. Taking TIMD as an example, as shown in FIG.11, the template area is divided into two parts: TA_A and TA_L. The prediction images tpredSamples_A and tpredSamples_L for TA_A and TA_L are obtained using the modes timdFirstMode, timdSecondMode, and timdNonAngMode derived from TIMD, and the cost (differences) between these prediction images and the reconstructed images recSamples_A and recSamples_L for TA_A and TA_L are calculated. timdCost1_A, timdCost1_L, timdCost2_A, timdCost2_L, timdCost3_A, and timdCost3_L is set equal to timdCost1 for TA_A, timdCost1 for TA_L, timdCost2 for TA_A, timdCost2 for TA_L, timdCost3 for TA_A, timdCost3 for TA_L respectively.

[0137] Then, the location dependency parameter, locDep1, locDep2, locDep3 for each derived prediction mode is derived. locDep1, locDep2 and locDep3 represents the position dependency indicator for timdFirstMode (Mode=0), timdSecondMode (Mode=1) and timdNonAngMode (Mode=2) respectively, where locDepMode == 0 indicates that the corresponding mode is independent of location, locDepMode == 1 indicates that the mode depends on the above side, TA_A, and locDepMode == 2 indicates that the mode depends on the left side, TA_L. The corresponding mode is timdFirstMode, timdSecondMode and timdNonAngMode for Mode = 0, 1, 2 respectively.

[0138] locDepMode = avalableLeftAbove ? LocLeftAbove : (avalableLeft ? LocLeft : LocAbove) Here LocLeft, LocAbove, LocLeftAbove are labels or variables, which may be 2, 1, 0 or other unique values. avalableLeftAbove is a flag that is set to 1 if the TA_L and TA_A are available, otherwise it's set to 0. avalableLeft is a flag that is set to 1 if the TA_L are available, otherwise the flag is set to 0.

[0139] The location dependency indicator may be derived using timd cost per pixel when avalableLeftAbove is true such that if the cost of left side (timdPixCostModeL) is much lower than the cost of above side (timdPixCostModeA), the indicator is set equal to LocLeft while if the cost of above side is much lower than the cost of left side, the indicator is set equal to LocAbove otherwise the indictor is set equal to LocLeftAbove (For example) locDepMode= (!avalableLeftAbove) ? (avalableLeft ? 2 : 1) : ( (timdCostMode_A >> log2A) < (timdCostMode_L >> (log2L + 1) ) ? 1 : ( (timdCostMode_L >> log2L) < (timdCostMode_A >> (log2A + 1) ) ? 2 : 0 where log2A=log2(cbWidth*tH), log2L=log2(cbHeight*tW).

[0140] or locDepMode = avalableLeftAbove ? timdPixCostModeL < (timdPixCostModeA>>reduceShift) ? 2 timdPixCostModeA < (timdPixCostModeL>>reduceShift) ? 1 : 0 (avalableLeft ? 2 : 1).

[0141] where timdPixCostModeL = timdCostMode>>log2L. timdPixCostModeA = timdCostMode>>log2A, reduceShift = 1, Mode = 1, 2, 3 The calculation of the location dependency parameters locDep2 and locDep3 for timdSecondMode and timdNonAngMode is the same as locDep1 for timdFirstMode.

[0142] locDep1 = (!avalableLeftAbove) ? (avalableLeft ? 2 : 1) : (timdCost1_A >> log2A < (timdCost1_L >> (log2L + 1))) ? 1 : (timdCost1_L >> log2L < (timdCost1_A >> (log2A + 1))) ? 2 : 0 locDep2 = (!avalableLeftAbove) ? (avalableLeft ? 2 : 1) : (timdCost2_A >> log2A < (timdCost2_L >> (log2L + 1))) ? 1 : (timdCost2_L >> log2L < (timdCost2_A >> (log2A + 1))) ? 2 : 0 locDep3 = (!avalableLeftAbove) ? (avalableLeft ? 2 : 1) : (timdCost3_A >> log2A < (timdCost3_L >> (log2L + 1))) ? 1 : (timdCost3_L >> log2L < (timdCost3_A >> (log2A + 1))) ? 2 : 0 It is noted that the derivation of locDep1, locDep2, and locDep3 along with the derivation of timdFirstMode, timdSecondMode, and timdNonAngMode may be performed in the Prediction Mode Derivation Device 4710.

[0143] (Generate the prediction image with location‐dependent weight) The TIMD method uses timdFirstMode, timdSecondMode, timdNonAngMode, timdWeight1, timdWeight2, timdWeight3, locDep1, locDep2, and locDep3 to generate the prediction image predImage. predImage may be the prediction image of the target block generated by Intra prediction unit 4704 or Intra prediction image generation unit 310. The specific calculation steps are as follows: Step 1: Generate the prediction images predImage1, predImage2, and predImage3 corresponding to timdFirstMode, timdSecondMode, and timdNonAngMode, respectively.

[0144] Step 2: Generate the prediction images predImageNon, predImageVer, and predImageHor based on the dependencies.

[0145] Step 3: Generate the final prediction image predImage.

[0146] (Step 1) Using the intra prediction unit 4704 or intra prediction image generation unit 310, generate the prediction images predImage1, predImage2, and predImage3 for the target block area corresponding to the intra prediction modes timdFirstMode, timdSecondMode, and timdNonAngMode, respectively, exported by TIMD.

[0147] (Step2) (Step2‐1) Derive integer weight variables, weightVer, weightHor, and weightNon as follow: timdModes[3] = {timdFirstMode, timdSecondMode, timdNonAngMode} timdWeights[3] = {timdWeight1, timdWeight2, timdWeight3} timdLocDep[3] = {locDep1, locDep2, locDep3} weightVer = 0, weightHor = 0, weightNon = 0

[0148] In summary, if the timdCost of each location is follows, L A LA timdFirstMode 10 10 10 timdSecondMode 20 50 40 timdNonAngular 290 100 110 timdWeight and locDep is derived as follows L A LA timdWeight1 31 / 64 15 / 32 15 / 32 (locDep1 = Loc LeftAbove since 10 == 10) timdWeight2 30 / 64 11 / 32 12 / 32 (locDep2 = LocLeft since 20 << 50) timdWeight3 3 / 64 6 / 32 5 / 32 (locDep3 = LocAbove since 290 >> 100) weightNon = 15 / 32 weightHor = 30 / 64 weightVer = 6 / 32 weights for Non = {15 / 32, 0, 0} weights for Left = {0, 30 / 64, 0} weights for Above = {0, 0, 6 / 32} In summary, if the timdCost of each location is follows, L A LA timdFirstMode 10 10 10 timdSecondMode 20 50 40 timdNonAngular 130 100 110 timdWeight and locDep is derived as follows L A LA timdWeight1 15 / 32 15 / 32 15 / 32 (locDep1 = LocLeftAbove since 10 == 10) timdWeight2 14 / 32 11 / 32 12 / 32 (locDep2 = LocLeft since 20 << 50) timdWeight3 3 / 32 6 / 32 5 / 32 (locDep3 = LocLeftAbove since 130 ~= 100) weightNon = 15 / 32+5 / 32=20 / 32 weightHor = 11 / 32 weightVer = 0 weights for Non = {15 / 32, 0, 5 / 32} weights for Left = {0, 11 / 32, 0} weights for Above = {0, 0, 0} (Step2‐2) Derive predImageNon, predImageVer, and predImageHor as follow: predImageNon is generated by weighted average of prediction images generated by timd derived prediction modes for the local dependency parameter is equal to 0 (timdLocDep[i] == 0).

[0149]

[0150] predImageVer and predImageHor are obtained similarly to predImageNon. They are generated by weighted fusion of prediction images generated by all timd‐exported prediction modes when the local dependency parameters are equal to 1 or 2. The method is as follows:

[0151] (Step3) To calculate the fusion weights for each pixel in the prediction image predImage based on the distance from the predicted pixels to TA_A and TA_L, and then perform weighted fusion to obtain the target pixel value in order to obtain predImage, the following data is required. Let the fusion weights for pixels in predImage be weightHorP, weightVerP, and weightNonP. The range of weights is denoted by range, which is an integer ranging from 0 to weightSum. weightHorP and weightVerP are integers ranging from 0 to clipHor and 0 to clipVer, respectively, where clipHor = weightHor + (weightHor / (weightHor + weightVer)) * weightNon, clipVer = weightVer + (weightVer / (weightHor + weightVer)) * weightNon. weightNonP, an integer ranging from 0 to weightSum. weightSum is the total weight, which may be an integer such as 64 or 128.

[0152]

[0153] Or those may be derived by the division approximation unit as follows respectively.

[0154] predImageNon[x][y] = (predImage1[x][y] * weightsN1 + predImage2[x][y] * weightsN2 + predImage3[x][y] * weightsN3 + offset) >> shiftVal weightN1, shiftVal = getDivValueN(weights[0], weightNon, baseShift) weightN2, shiftVal = getDivValueN(weights[1], weightNon, baseShift) weightN3, shiftVal = getDivValueN(weights[2], weightNon, baseShift) offset = 1<<(shiftVal‐1) or 1<<shiftVal>>1 predImageVer[x][y] = (predImage1[x][y] * weightsV1 + predImage2[x][y] * weightsV2 + predImage3[x][y] * weightsV3 + offset) >> shiftVal weightV1, shiftVal = getDivValueN(weights[0], weightVer, baseShift) weightV2, shiftVal = getDivValueN(weights[1], weightVer, baseShift) weightV3, shiftVal = getDivValueN(weights[2], weightVer, baseShift) offset = 1<<(shiftVal‐1) or 1<<shiftVal>>1 predImageHor[x][y] = (predImage1[x][y] * weightsH1 + predImage2[x][y] * weightsH2 + predImage3[x][y] * weightsH3 + offset) >> shiftVal weightH1, shiftVal = getDivValueN(weights[0], weightHor, baseShift) weightH2, shiftVal = getDivValueN(weights[1], weightHor, baseShift) weightH3, shiftVal = getDivValueN(weights[2], weightHor, baseShift) offset = 1<<(shiftVal‐1) or 1<<shiftVal>>1 weightVerP = weightVer + divIntN(((cbHeight‐1‐2*y)*range), (cbHeight‐1)) weightHorP = weightHor + divIntN(((cbWidth‐1‐2*x)*range), (cbWidth‐1)) predImage[x][y] = divIntN((predImageHor[x][y] * weightHorP + predImageVer[x][y] * weightVerP + predImageNon[x][y] * weightNonP), weightSum) where N may be 16 or 32.

[0155] (DIMD prediction) The parameter decoding unit 302 decodes signaled data to determine the DIMD flag (dimd_flag) for each block. A block may be a Coding Unit (CU), Transform Unit (TU), sub‐block, etc. If the dimd_flag for the target block is 0, then the parameter decoding unit 302 decodes syntax elements related to intra prediction modes. However, if dimd_flag is 1, the parameter decoding unit 302 may skip decoding syntax elements related to intra prediction modes from the encoded data. When timd_flag is 1, the prediction image of the current block is derived using the DIMD method, which may calculate dimdMode1, dimdMode2, dimdMode3, dimdMode4, dimdMode5, dimdMode6, dimdWeight1, dimdWeight2, dimdWeight3, dimdWeight4, dimdWeight5, and dimdWeight6 based on the surrounding samples of the current block. The DIMD prediction image is then derived based on these parameters. FIG.13 shows the structrue of DIMD prediction unit 31046.

[0156] (DIMD reference area) The reference sample derivation unit 310460 derives reference samples from neighboring samples of the target block. FIG.12(a) shows the spatial relationship between the reference samples and the current block. The reference samples have two parts RDL and RDT. Set the coordinates of above‐left corner pixel in current block is (x, y). The area of RDL is the pixel with coordinate of (x + i, y + j), x=‐3..‐1, y=‐3..cbHeight. The area of RDT is the pixel with coordinate of (x + i, y + j), x=‐3..cbWidth, y=‐3..‐1. The RDTL is the combination od RDL and RDT.

[0157] The Gradient derivation unit 310461 use 3x3 filter for gradient derivation. FIG. 12(b) show the 3x3 filter. The angular mode derivation unit 310461 derives Dx and Dy from the each point P of the RDTL of the target block, derives modeVal and conducts histogram counting operation. The target point P is the pixel with coordinates of (x + i, y + j), i=‐2, j=‐2..cbHeight‐1 or i=‐2..cbWidht‐1, j=‐2.

[0158] (Histogram generation) The gradient derivation unit 310461 derives pixel gradient Dx and Dy using the pixel values P[x][y] for a given position (x, y) in the reference samples.

[0159] Dx = P[x‐1][y+1] + 2*P[x][y+1] + P[x+1][y+1] ‐ P[x‐1][y‐1] ‐ 2*P[x][y‐1] ‐ P[x+1][y‐1] Dy = P[x‐1][y‐1] + 2*P[x‐1][y] + P[x‐1][y+1] ‐ P[x+1][y‐1] ‐ 2*P[x+1][y] ‐ P[x+1][y+1] The gradient derivation unit 310461 derives signx, signy, xgty and quadrant as follows.

[0160] absx = abs(Dx) absy = abs(Dy) signx = Dx < 0 ? 1 : 0 signy = Dy < 0 ? 1 : 0 xgty = absx > absy ? 1 : 0 quadrant = xgty ? ( (signx^signy) ? 1 : 0 ) : ( (signx^signy) ? 2 : 3)

[0161] Here, unequal sign (>, <) may be replaced by (>=, <=). The angular information may be derived from the signx, signy, and xgty. ^ is an XOR calculation. The quadrant is represented by the value from 0 to 3, {Ra, Rb, Rc, Rd} = {0, 1, 2, 3}. The value of the quadrant is not limited to the above.

[0162] The angular mode derivation unit 310462 derives iRatio.

[0163] iRatio = R_UNIT * absy / absx R_UNIT is the exponential power of 2 (1<<shiftR), e.g. R_UNIT = 65536 when shiftR = 16. The division may be replaced with “multiplied by its reciprocal”. The reciprocal is derived by a LUT.

[0164] e.g. LUT[k] = R_UNIT / k, iRatio = absy * LUT[absx] The angular mode derivation unit 310462 converts the derived pixel gradient to an angular prediction mode, modeVal by searching corresponding angular mode corresponding to the iRatio.

[0165] mode_delta = 16; for( int i = 1; i < 17; i++ ){ The predImage may be obtained as follows:

[0166] (Intra prediction mode derivation) Angular mode derivation unit derives angular mode from HistMode. dimdMode1, dimdMode2, dimdMode3, dimdMode4, and dimdMode5 are derived from HistMode, representing the angular modes with the highest, second‐highest, third‐highest, fourth‐highest, and fifth‐highest gradient sums in HistMode, respectively.

[0167] dimdMode1 = modeVal (HistMode[modeVal] have the hightst value in HistMode) dimdMode2 = modeVal (HistMode[modeVal] have the second‐hightst value in HistMode) dimdMode3 = modeVal (HistMode[modeVal] have the third‐hightst value in HistMode) dimdMode4 = modeVal (HistMode[modeVal] have the fourth‐hightst value in HistMode) dimdMode5 = modeVal (HistMode[modeVal] have the fifth‐hightst value in HistMode) The corresponding gradient sums are stored in gSum1, gSum2, gSum3, gSum4, and gSum5. The corresponding weights, dimdWeight1, dimdWeight2, dimdWeight3, dimdWeight4, and dimdWeight5, are exported as follows: sumWeight is a positive integer that is a power of 2, such as 32, 64, 128, etc.

[0168] sum = gSum1 + gSum2 + gSum3 + gSum4 + gSum5 dimdweight1 = gSum1 / sum * (sumWeight- KK) dimdweight2 = gSum2 / sum * (sumWeight- KK) dimdweight3 = gSum3 / sum * (sumWeight- KK) dimdweight4 = gSum4 / sum * (sumWeight- KK) dimdweight5 = sumWeight - KK ‐ dimdweight1 - dimdweight2 - dimdweight3 ‐ dimdweight4. KK may be 16 or a positive interger value.

[0169] The above calculation may also be performed using the division approximation unit.

[0170] dimdweight1 = divIntNSum(gSum1, sum, sumWeight - KK) dimdweight2 = divIntNSum(gSum2, sum, sumWeight - KK) dimdweight3 = divIntNSum(gSum3, sum, sumWeight - KK) dimdweight4 = divIntNSum(gSum4, sum, sumWeight - KK) dimdweight5 = divIntNSum(gSum5, sum, sumWeight - KK) dimdMode6 and dimdWeight6 are set as constants, which are the planar mode and KK, respectively.

[0171] (Generate the prediction image with location‐dependent weight) Prediction Image Generation Unit 310463 generates the prediction image predImage for the current block based on the internal prediction modes and weights derived by DIMD. The generation of the prediction image involves the following steps: Step 1: Generate the dependency flags dimdLocDep1, dimdLocDep2, dimdLocDep3, dimdLocDep4, dimdLocDep5, and dimdLocDep6 for dimdMode1, dimdMode2, dimdMode3, dimdMode4, dimdMode5, and dimdMode6, respectively.

[0172] Step 2: Generate the prediction images predImage1, predImage2, predImage3, predImage4, predImage5, and predImage6 corresponding to dimdMode1, dimdMode2, dimdMode3, dimdMode4, dimdMode5, and dimdMode6, respectively.

[0173] Step 3: Generate the prediction images predImageNon, predImageVer, and predImageHor based on the dependencies.

[0174] Step 4: Generate the final prediction image predImage.

[0175] (Step1) Create HistMode_T and HistMode_L using the same method as HistMode, with the difference being that the reference regions for HistMode_T and HistMode_L are RDT and RDL, respectively. The dependency of each mode may be derived as follows: dimdLocDep1 = HistMode_L[dimdMode1] < HistMode[dimdMode1] / 3 ? 1 : HistMode_T[dimdMode1] < HistMode[dimdMode1] / 3 ? 2 : 0 dimdLocDep2 = HistMode_L[dimdMode2] < HistMode[dimdMode2] / 3 ? 1 : HistMode_T[dimdMode2] < HistMode[dimdMode2] / 3 ? 2 : 0 dimdLocDep3 = HistMode_L[dimdMode3] < HistMode[dimdMode3] / 3 ? 1 : HistMode_T[dimdMode3] < HistMode[dimdMode3] / 3 ? 2 : 0 dimdLocDep4 = HistMode_L[dimdMode1] < HistMode[dimdMode4] / 3 ? 1 : HistMode_T[dimdMode4] < HistMode[dimdMode4] / 3 ? 2 : 0 dimdLocDep5 = HistMode_L[dimdMode5] < HistMode[dimdMode5] / 3 ? 1 : HistMode_T[dimdMode5] < HistMode[dimdMode5] / 3 ? 2 : 0 dimdLocDep6 = 0 Or dimdLocDep1 = 3*HistMode_L[dimdMode1] < HistMode[dimdMode1] ? 1 : 3*HistMode_T[dimdMode1] < HistMode[dimdMode1] ? 2 : 0 dimdLocDep2 = 3*HistMode_L[dimdMode2] < HistMode[dimdMode2] ? 1 : HistMode_T[dimdMode2] < 3*HistMode[dimdMode2] ? 2 : 0 dimdLocDep3 = 3*HistMode_L[dimdMode3] < HistMode[dimdMode3] ? 1 : 3*HistMode_T[dimdMode3] < HistMode[dimdMode3] ? 2 : 0 dimdLocDep4 = 3*HistMode_L[dimdMode1] < HistMode[dimdMode4] ? 1 : 3*HistMode_T[dimdMode4] < HistMode[dimdMode4] ? 2 : 0 dimdLocDep5 = 3*HistMode_L[dimdMode5] < HistMode[dimdMode5] ? 1 : 3*HistMode_T[dimdMode5] < HistMode[dimdMode5] ? 2 : 0 dimdLocDep6 = 0 (Step2) Using the intra prediction image generation unit 310, generate the prediction images predImage1, predImage2, predImage3, predImage4, predImage5, and predImage6 for the target block area corresponding to the intra prediction modes dimdMode1, dimdMode2, dimdMode3, dimdMode1, dimdMode2, and dimdMode3 respectively, exported by DIMD.

[0176] (Step3) (Step3‐1) Derive integer variables, weightVer, weightHor, and weightNon as follow: dimdModes[6] = {dimdMode1, timdMode2, timdMode3, dimdMode4, timdMode5, timdMode6} dimdWeights[6] = {dimdWeight1, dimdWeight2, dimdWeight3, dimdWeight4, dimdWeight5, dimdWeight6} dimdLocDep[6] = {dimdLocDep1, dimdLocDep2, dimdLocDep3, dimdLocDep4, dimdLocDep5, dimdLocDep6} weightVer = 0, weightHor = 0, weightNon = 0 for (i = 0; i < 6; i++)

[0177] (Step3‐2) Derive predImageNon, predImageVer, and predImage. predImageNon is generated by weighted average of prediction images generated by dimd derived prediction modes when the local dependency parameter is equal to 0 (dimdLocDep[i] == 0). The method is as follows:

[0178] predImageVer and predImageHor are obtained similarly to predImageNon. They are generated by weighted fusion of prediction images generated by all dimd‐exported prediction modes when the local dependency parameters are equal to 1 or 2. The method is as follows:

[0179] (Step4) To calculate the fusion weights for each pixel in the prediction image predImage based on the distance from the predicted pixels to RDT and RDL, and then perform weighted fusion to obtain the target pixel value in order to obtain predImage, the following data is required. Let the fusion weights for pixels in predImage be weightHorP, weightVerP, and weightNonP. The range of weights is denoted by range, which is an integer ranging from 0 to weightSum. weightHorP and weightVerP are integers ranging from 0 to clipHor and 0 to clipVer, respectively, where clipHor = weightHor + (weightHor / (weightHor + weightVer)) * weightNon, clipVer = weightVer + (weightVer / (weightHor + weightVer)) * weightNon. weightNonP, an integer ranging from 0 to weightSum. weightSum is the total weight, which may be an integer such as 64 or 128.

[0180]

[0181] Where weightVerP = weightVer + ((cbHeight‐1‐2*y)*range) / (cbHeight‐1) weightHorP = weightHor + ((cbWidth‐1‐2*x)*range) / (cbWidth‐1) predImage[x][y] = (predImageHor[x][y] * weightHorP + predImageVer[x][y] * weightVerP + predImageNon[x][y] * weightNonP) / weightSum may be derived by the division approximation unit as follows respectively.

[0182] weightVerP = weightVer + divIntN(((cbHeight‐1‐2*y)*range), (cbHeight‐1)) weightHorP = weightHor + divIntN(((cbWidth‐1‐2*x)*range), (cbWidth‐1)) predImage[x][y] = divIntN((predImageHor[x][y] * weightHorP + predImageVer[x][y] * weightVerP + predImageNon[x][y] * weightNonP), weightSum) where N may be 16 or 32.

[0183] The inverse quantization and inverse transform processing unit 311 performs inverse quantization on a quantization transform coefficient input from the prediction parameter derivation unit 320 to calculate a transform coefficient. This quantization transform coefficient is a coefficient obtained by performing a frequency transform such as a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), or the like on prediction errors to quantize in coding processing. The inverse quantization and inverse transform processing unit 311 performs an inverse frequency transform such as an inverse DCT, an inverse DST, or the like on the calculated transform coefficient to calculate a prediction error. The inverse quantization and inverse transform processing unit 311 outputs the prediction error to the addition unit 312.

[0184] The addition unit 312 adds the prediction image of the block input from the intra prediction image generation unit 310 and the prediction error input from the inverse quantization and inverse transform processing unit 311 for each pixel and generates a decoded image of the block. The addition unit 312 stores the decoded image of the block in the reference picture memory 306 and outputs the image to the loop filter 305.

[0185] Configuration of video coding apparatus Next, a configuration of the video coding apparatus 11 according to the present embodiment is described. FIG. 7 is a block diagram illustrating a conuration of the video coding apparatus 11 according to the present embodiment. The video coding apparatus 11 is configured to include a prediction image generation unit 101, a subtraction unit 102, a transform and quantization processing unit 103, an inverse quantization and inverse transform processing unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (a prediction parameter storage unit, a frame memory) 108, a reference picture memory (a reference image storage unit, a frame memory) 109, a coding parameter determination unit 110, a parameter coding unit 111, prediction parameter derivation unit 120, and an entropy coding unit 104.

[0186] The prediction image generation unit 101 generates a prediction image for each CU that is a region obtained by splitting each picture of the image T. The operation of the prediction image generation unit 101 is the same as that of the intra prediction image generation unit 310 already described, and thus descriptions thereof is omitted.

[0187] The subtraction unit 102 subtracts a pixel value of the prediction image of the block input from the prediction image generation unit 101 from a pixel value of the image T to generate a prediction error. The subtraction unit 102 outputs the prediction error to the transform and quantization processing unit 103.

[0188] The transform and quantization processing unit 103 calculates a transform coefficient by performing a frequency transform on the prediction error input from the subtraction unit 102, and derives a quantization transform coefficient by quantization. The transform and quantization proceessing unit 103 outputs the quantization transform coefficient to the entropy coding unit 104 and the inverse quantization and inverse transform processing unit 105.

[0189] The inverse quantization and inverse transform processing unit 105 is the same as the inverse quantization and inverse transform processing unit 311 (FIG. 4) in the video decoding apparatus 31, and descriptions thereof are omitted. The calculated prediction error is output to the addition unit 106.

[0190] To the entropy coding unit 104, the quantization transform coefficient is input from the transform and quantization processing unit 103, and coding parameters are input from the parameter coding unit 111. The entropy coding unit 104 performs entropy coding on split information, the prediction parameters, the quantization transform coefficient, and the like to generate and output the coding stream Te.

[0191] The parameter coding unit 111 instructs the entropy coding unit 104 to encode the prediction parameters and quantization coefficients, derived from the prediction parameter derivation unit 120.

[0192] The prediction parameter derivation unit 120 derives the syntax element from the parameters inputted from the coding parameter determination unit 110. Some parts of the prediction parameter derivation unit 120 have the same structure as the prediction parameter derivation unit 320.

[0193] The addition unit 106 adds a pixel value of the prediction image of the block input from the prediction image generation unit 101 and the prediction error input from the inverse quantization and inverse transform processing unit 105 to each other for each pixel, and generates a decoded image. The addition unit 106 stores the generated decoded image in the reference picture memory 109.

[0194] The loop filter 107 applies a deblocking filter, an SAO, and an ALF to the decoded image generated by the addition unit 106. Note that the loop filter 107 need not necessarily include the above‐described three types of filters, and may have a configuration of only the deblocking filter, for example.

[0195] The prediction parameter memory 108 stores the prediction parameters generated by the prediction parameter derivation unit 120 for each target picture and CU at a predetermined position. It may stores the transform coefficients created by the transform and quantization processing unit 103.

[0196] The reference picture memory 109 stores the decoded image generated by the loop filter 107 for each target picture and CU at a predetermined position.

[0197] The coding parameter determination unit 110 selects one set among multiple sets of coding parameters. A coding parameter refers to the above‐mentioned QT, BT, or TT split information, the prediction parameter, or a parameter to be coded, the parameter being generated in association therewith. The prediction image generation unit 101 generates the prediction image by using these coding parameters.

[0198] The coding parameter determination unit 110 calculates, for each of the multiple sets, an RD cost value indicating the magnitude of an amount of information and a coding error. The RD cost value is, for example, the sum of a code amount and the value obtained by multiplying a coefficient λ by a square error. The coding parameter determination unit 110 selects a set of coding parameters of which cost value calculated is a minimum value. With this configuration, the entropy coding unit 104 outputs the selected set of coding parameters as the coding stream Te. The coding parameter determination unit 110 outputs the determined coding parameters in the parameter coding unit 111, the prediction parameter derivation unit 120, the prediction image generation unit 101.

[0199] Note that, some of the video coding apparatus 11 and the video decoding apparatus 31 in the above‐described embodiment, for example, the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the intra prediction image generation unit 310, the inverse quantization and inverse transform processing unit 311, the addition unit 312, the prediction parameter derivation unit 320, the prediction image generation unit 101, the subtraction unit 102, the transform and quantization processing unit 103, the entropy coding unit 104, the inverse quantization and inverse transform processing unit 105, the loop filter 107, the coding parameter determination unit 110, and the parameter coding unit 111, the prediction parameter derivation unit 120, may be realized by a computer. In that case, this configuration may be realized by recording a program for realizing such control functions on a computer‐readable recording medium and causing a computer system to read the program recorded on the recording medium for execution. Note that the “computer system” mentioned here refers to a computer system built into either the video coding apparatus 11 or the video decoding apparatus 31 and is assumed to include an OS and hardware components such as a peripheral apparatus. Furthermore, a “computer‐readable recording medium” refers to a portable medium such as a flexible disk, a magneto‐optical disk, a ROM, a CD‐ROM, and the like, and a storage device such as a hard disk built into the computer system. Moreover, the “computer‐readable recording medium” may include a medium that dynamically stores a program for a short period of time, such as a communication line in a case that the program is transmitted over a network such as the Internet or over a communication line such as a telephone line, and may also include a medium that stores the program for a fixed period of time, such as a volatile memory included in the computer system functioning as a server or a client in such a case. Furthermore, the above‐described program may be one for realizing some of the above‐described functions, and also may be one capable of realizing the above‐described functions in combination with a program already recorded in a computer system.

[0200] Furthermore, a part or all of the video coding apparatus 11 and the video decoding apparatus 31 in the embodiment described above may be realized as an integrated circuit such as a Large Scale Integration (LSI). Each function block of the video coding apparatus 11 and the video decoding apparatus 31 may be individually realized as processors, or part or all may be integrated into processors. The circuit integration technique is not limited to LSI, and the integrated circuits for the functional blocks may be realized as dedicated circuits or a multi‐purpose processor. In a case that with advances in semiconductor technology, a circuit integration technology with which an LSI is replaced appears, an integrated circuit based on the technology may be used.

[0201] The embodiment of the present disclosure has been described in detail above referring to the drawings, but the specific configuration is not limited to the above embodiments and various amendments may be made to a design that fall within the scope that does not depart from the gist of the present disclosure.

[0202] The embodiment of the present invention may be applied to a video decoding device that decodes encoded data of image data, and a video encoding device that generates encoded data from image data. In addition, the data structure of the encoded data is generated by the video encoding device and referenced by the video decoding device.

[0203] Reference Signs List 31 Image decoding apparatus 301 Entropy decoding unit 302 Parameter decoding unit 310 Prediction image generation unit 3104 Intra prediction unit 31046 DIMD prediction unit 310460 Reference sample derivation unit 310461 Gradient derivation unit 310462 Angular mode derivation unit 310463 Prediction image generation unit 31047 TIMD prediction unit 4701 Reference sample derivation unit 4702 Template derivation unit 4710 Prediction Mode Derivation Device 4711 Candidate derivation unit 4717 Intra prediction mode derivation unit 4713 Template prediction image generation unit 4714 Template cost derivation unit 4715 Comparition unit 4703 Weights Derivation unit 4704 Intra prediction unit 311 Inverse quantization and inverse transform processing unit 312 Addition unit 11 Image coding apparatus 101 Prediction image generation unit 102 Subtraction unit 103 Transform and quantization processing unit 104 Entropy coding unit 105 Inverse quantization and inverse transform processing unit 107 Loop filter 110 Coding parameter determination unit 111 Parameter coding unit <Cross Reference> This patent application claims priority under on JP Patent Application No. 2024‐062522 filed on April 9, 2024, the entire contents of which are hereby incorporated by reference.

Claims

1. A video decoding apparatus for generating a prediction image, the video decoding apparatus comprising a TIMD prediction unit and a DIMD prediction unit that utilizes the weighted fusion results of single‐frame intra prediction modes as the prediction output.

2. A video decoding apparatus of claim 1, wherein the video decoding apparatus further comprising a TIMD prediction unit configured to derive intra prediction modes, which include angular modes, DC mode, planar mode, and IBC / ITMP modes.

3. A video decoding apparatus of claim 1, wherein the video decoding apparatus further comprising a TIMD prediction unit configured to derive prediction mode using different template size, template size is determined by the block size.

4. A video decoding apparatus of claim 1, wherein the video decoding apparatus further comprising a common division processing unit, which is used to replace all the division operations in the TIMD and DIMD parts by deriving an approximate result with a lookup table method.

5. An video coding apparatus comprising a TIMD prediction unit and a TIMD prediction unit configured to derive a plural intra predicition modes and derive prediction image by weighted fusing these intra prediction modes.

Images