Video decoding apparatus, video coding apparatus

Abstract

This invention aims to improve accuracy by enhancing the existing EIP prediction method. It introduces new methods for determining the size of the reference area, reordering the curModelList, utilizing multi-models, and adding bias terms in the function model, thereby improving the predictive results of the EIP method.

Description

VIDEO DECODING APPARATUS, VIDEO CODING APPARATUS

[0001] The embodiments of the present invention relate to, a video decoding apparatus, a video coding apparatus.

[0002] A video coding apparatus which generates coded data by coding a video, and a video decoding apparatus which generates decoded images by decoding the coded data are used for efficient transmission or recording of videos.

[0003] For example, specific video coding schemes include H.264 / AVC, High-Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC) schemes, and the like.

[0004] In such a video coding scheme, images (pictures) constituting a video are managed in a hierarchical structure including slices obtained by splitting an image, coding tree units (CTUs) obtained by splitting a slice, units of coding (coding units; which is referred to as CUs) obtained by splitting a coding tree unit, and transform units (TUs) obtained by splitting a coding unit, and are coded / decoded for each CU.

[0005] In such a video coding scheme, usually, a prediction image is generated based on a local decoded image that is obtained by coding / decoding an input image (a source image), and prediction error components (which may be referred to also as “difference images” or “residual images”) obtained by subtracting the prediction image from the input image are coded. Generation methods of prediction images include an inter-picture prediction (an inter-prediction) and an intra-picture prediction (intra prediction).

[0006] In recent video coding and decoding techniques, NPL1 introduced a method for coding and decoding the current block. This method utilizes an extrapolation filter to establish a functional relationship between the target pixel value and the 15 surrounding pixel values. This relationship forms a function model that can predict the target pixel value from the 15 inputs. The method defines three filters and three reference templates, resulting in nine possible modes through their combinations. During execution, the method establishes two lists, curModelList and mergeModelList. The cu.eip flag identifies whether the current block employs the EIP method, while the cu.eipMerge flag indicates which list is used. The idx is then used to specify the position of the model within the list.

[0007] Luhang Xu, Yue Yu, Haoping Yu, Jonathan Gan, Dong Wang, “EE2-1.14: An extrapolation filter-based intra prediction mode, ” JVET-AG0058, 33rd Meeting, by teleconference, 17-26 January 2024.

[0008] The EIP method provides three types of filters: square filter, horizontal filter, and vertical filter, used to generate prediction models. From these three models, it is observed that the EIP method cannot generate models for predicting diagonal data. Additionally, for each target block, the EIP method uses only one prediction model, without considering that complex data in a block cannot be represented by a single model. Therefore, in this invention, diagonal filters and the EIP multi-model mode have been added. In EIP multi-model mode two models are used to predict the target block.

[0009] The aim of this invention is to improve prediction accuracy by adding new filters and eip multi-model mode.

[0010] In one example, a video decoding apparatus for generating a prediction image is provided. The apparatus includes an EIP prediction unit configured to generate intra prediction images using a function model generated based on samples in a reference area.

[0011] In one example, video coding apparatus is provided. The apparatus includes an EIP prediction unit configured to predict a target block using a function model generated based on samples in a reference area.

[0012] According to an aspect of the present invention, the quality of the codecs is improved without adding additional calculations.

[0013] FIG. 1 is a schematic diagram illustrating a configuration of an image transmission system according to the present embodiment.FIG. 2 is a diagram showing the hierarchical structure of the coded stream data.FIG. 3 is a schematic diagram showing the type of intra-prediction mode (mode number)FIG. 4 is a schematic diagram of the video decoding apparatus.FIG. 5 shows the structure of the intra prediction image generation unit.FIG. 6 is a diagram showing the details of the EIP Prediction Unit.FIG. 7 is a diagram showing 3 type of reference areas used in EIP method.FIG. 8 is a diagram showing 3 type of filters used in EIP method.FIG. 9 is a diagram showing the defined elements of curModelList for each size of curBlock.FIG. 10 is a diagram showing the reference and template area for sorting.FIG. 11 is a block diagram showing the structure of a video coding apparatus.

[0014] First Embodiment Hereinafter, embodiments of the present disclosure is described with reference to the drawings.

[0015] FIG. 1 is a schematic diagram illustrating a configuration of an image transmission system 1 according to the present embodiment.

[0016] The image transmission system 1 is a system in which a coding stream obtained by coding a coding target image is transmitted, the transmitted coding stream is decoded, and an image is displayed. The image transmission system 1 includes a video coding apparatus (image coding apparatus) 11, a network 21, a video decoding apparatus (image decoding apparatus) 31, and a video display apparatus (image display apparatus) 41.

[0017] An image T is input to the video coding apparatus 11.

[0018] The network 21 transmits a coding stream Te generated by the video coding apparatus 11 to the video decoding apparatus 31. The network 21 is the Internet, a Wide Area Network (WAN), a Local Area Network (LAN), or a combination thereof. The network 21 is not necessarily limited to a bidirectional communication network, and may be a unidirectional communication network configured to transmit broadcast waves of digital terrestrial television broadcasting, satellite broadcasting or the like. Furthermore, the network 21 may be substituted by a storage medium in which the coding stream Te is recorded, such as a Digital Versatile Disc (DVD: trademark) or a Blu-ray Disc (BD: trademark).

[0019] The video decoding apparatus 31 decodes each of the coding streams Te transmitted from the network 21 and generates one or multiple decoded images Td which are decoded.

[0020] The video display apparatus 41 displays all or part of the one or multiple decoded images Td generated by the video decoding apparatus 31. For example, the video display apparatus 41 includes a display device such as a liquid crystal display and an organic Electro-Luminescence (EL) display. Forms of the display include a stationary type, a mobile type, an HMD type, and the like. In addition, in a case that the video decoding apparatus 31 has a high processing capability, an image having high image quality is displayed, and in a case that the apparatus only has a lower processing capability, an image which does not require high processing capability and display capability is displayed.

[0021] Operator Operators and notations used in the present specification is described below.

[0022] >> is an arithmetic right bit shift, << is an arithmetic left bit shift, & is a bitwise AND, | is a bitwise OR, ^ is a bitwise XOR, |= is an OR assignment operator, and || indicates a logical sum.

[0023] x ? y : z is a ternary operator to take y in a case that x is true (other than 0) and take z in a case that x is false (0).

[0024] Clip3(x, y, z) is a function to clip z in a value equal to or greater than x and less than or equal to y, and a function to return x in a case that z is less than x (z < x), return y in a case that z is greater than y (z > y), and return z in other cases.

[0025] abs(a) is a function that returns the absolute value of a.

[0026] Int(a) is a function that returns the integer value of a.

[0027] floor(a) is a function that returns the maximum integer equal to or less than a.

[0028] ceil(a) is a function that returns the minimum integer equal to or greater than a.

[0029] a / d represents division of a by d (round down decimal places).

[0030] x = y..z represents x takes on integer values starting from y to z, inclusive, with x, y, and z being integer numbers and z being greater than or equal to y.

[0031] Structure of Coding Stream Te Prior to the detailed description of the video coding apparatus 11 and the video decoding apparatus 31 according to the present embodiment, a data structure of the coding stream Te generated by the video coding apparatus 11 and decoded by the video decoding apparatus 31 is described.

[0032] FIG. 2 is a diagram illustrating a hierarchical structure of data of the coding stream Te. The coding stream Te includes a sequence and multiple pictures constituting the sequence illustratively. (a) to (f) of FIG. 2 are diagrams illustrating a coded video sequence defining a sequence SEQ, a coded picture prescribing a picture PICT, a coding slice prescribing a slice S, a coding slice data prescribing slice data, a coding tree unit included in the coding slice data, and a coding unit (CU) included in each coding tree unit, respectively.

[0033] Coded Video Sequence In the coded video sequence (CVS, coding stream), a set of data referred to by the video decoding apparatus 31 to decode the coded sequence sequences to be processed is defined. As illustrated in FIG. 2, the CVS includes a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a picture (PICT), and Supplemental Enhancement Information (SEI).

[0034] In the video parameter set VPS, in a video including multiple layers, a set of coding parameters common to multiple videos and a set of coding parameters associated with the multiple layers and an individual layer included in the video are defined.

[0035] In the sequence parameter set SPS, a set of coding parameters referred to by the video decoding apparatus 31 to decode a target sequence is defined. For example, a width and a height of a picture are defined. Note that multiple SPSs may exist. In that case, any of multiple SPSs is selected from the PPS.

[0036] In the picture parameter set PPS, a set of coding parameters referred to by the video decoding apparatus 31 to decode each picture in a target sequence is defined. For example, a reference value (pic_init_qp_minus26) of a quantization step size used for decoding of a picture and a flag (weighted_pred_flag) indicatingan application of a weighted prediction are included. Note that multiple PPSs may exist. In that case, any of multiple PPSs is selected from each picture in a target sequence.

[0037] Coded Picture In the coded picture, a set of data referred to by the video decoding apparatus 31 to decode the picture PICT to be processed is defined. As illustrated in FIG. 2, the picture PICT includes a slice 0 to a slice NS-1 (NS is the total number of slices included in the picture PICT).

[0038] Note that in a case that it is not necessary to distinguish each of the slice 0 to the slice NS-1 below, subscripts of reference signs may be omitted. In addition, the same applies to other data with subscripts included in the coding stream Te which is described below.

[0039] Coding Slice In the coding slice, a set of data referred to by the video decoding apparatus 31 to decode the slice S to be processed is defined. As illustrated in FIG. 2, the slice includes a slice header and a slice data.

[0040] The slice header includes a coding parameter group referred to by the video decoding apparatus 31 to determine a decoding method for a target slice. Slice type specification information (slice_type) indicating a slice type is one example of a coding parameter included in the slice header.

[0041] Examples of slice types that may be specified by the slice type specification information include (1) I slice using only an intra prediction in coding, (2) P slice using a unidirectional prediction or an intra prediction in coding, and (3) B slice using a unidirectional prediction, a bidirectional prediction, or an intra prediction in coding, and the like. Note that the inter prediction is not limited to a uni-prediction and a bi-prediction, and the prediction image may be generated by using a larger number of reference pictures. Hereinafter, in a case that a slice is referred to as the P or B slice, the slice indicates a slice that includes a block in which the inter prediction may be used.

[0042] Note that, the slice header may include a reference to the picture parameter set PPS (pic_parameter_set_id).

[0043] Coding Slice Data In the coding slice data, a set of data referred to by the video decoding apparatus 31 to decode the slice data to be processed is defined. The slice data include CTUs as illustrated in FIG. 2. The CTU is a block of a fixed size (for example, 64 x 64) constituting a slice.

[0044] Coding Tree Unit In FIG. 2, a set of data referred to by the video decoding apparatus 31 to decode the CTU to be processed is defined. The CTU is split into coding units CUs, each of which is a basic unit of coding processing, by a recursive Quad Tree split (QT split), Binary Tree split (BT split), or Ternary Tree split (TT split). The BT split and the TT split are collectively referred to as a Multi Tree split (MT split). Nodes of a tree structure obtained by recursive quad tree splits are referred to as Coding Nodes. Intermediate nodes of a quad tree, a binary tree, and a ternary tree are coding nodes, and the CTU itself is also defined as the highest coding node.

[0045] Coding Unit As illustrated in FIG. 2, a set of data referred to by the video decoding apparatus 31 to decode the coding unit to be processed is defined. Specifically, the CU includes a CU header CUH, a prediction parameter, a transform parameter, a quantization transform coefficient, and the like. In the CU header, a prediction mode and the like are defined.

[0046] There are cases that the prediction processing is performed in units of CU or performed in units of sub-CU obtained by further splitting the CU. In a case that the sizes of the CU and the sub-CU are equal to each other, the number of sub-CUs in the CU is one. In a case that the CU is larger in size than the sub-CU, the CU is split into sub-CUs. For example, in a case that the CU has a size of 8 x 8, and the sub-CU has a size of 4 x 4, the CU is split into four sub-CUs which include two horizontal splits and two vertical splits.

[0047] There are two types of predictions (prediction modes), which are an intra prediction and an inter prediction. The intra prediction refers to a prediction in an identical picture, and the inter prediction refers to prediction processing performed between different pictures (for example, between pictures of different display times).

[0048] Transform and quantization processing is performed in units of CU, but the quantization transform coefficient may be subjected to entropy coding in units of subblock such as 4 x 4.

[0049] Prediction parameter A prediction image is derived by a prediction parameter accompanying a block. The prediction parameter includes prediction parameters of the intra prediction and the inter prediction.

[0050] The prediction parameter of the intra prediction is described below. The intra prediction parameter includes a luma intra prediction mode IntraPredModeY and a chroma intra prediction mode IntraPredModeC. FIG. 3 is a schematic diagram indicating types (mode numbers) of the intra prediction mode. As illustrated in the diagram, for example, there are 67 types (0 to 66) of intra prediction modes. Additionally there are 28 types (-14 to -1 and 67 to 80) of intra prediction modes depend on the aspect ratio of CU. For example, a planar prediction (0), a DC prediction (1), and Angular predictions (2 to 66) are present. Furthermore, for chroma, CCLM (Cross Component Linear Model) prediction mode (81 to 83),MMLM (Multi Mode Linear Model) prediction mode, and LM (Linear Model) prediction modemay be added.

[0051] Configuration of video decoding apparatus A configuration of the video decoding apparatus 31 (FIG. 4) according to the present embodiment is described.

[0052] The video decoding apparatus 31 includes an entropy decoding unit 301, a parameter decoding unit (prediction image decoding apparatus) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit 308, an inverse quantization and inverse transform processing unit 311, an addition unit 312, and a prediction parameter derivation unit 320. Note that a configuration in which the loop filter 305 is not included in the video decoding apparatus 31 is also used in accordance with the video coding apparatus 11 described later.

[0053] The parameter decoding unit 302 further includes a header decoding unit 3020, a CT information decoding unit 3021, and a CU decoding unit 3022 (prediction mode decoding unit), and the CU decoding unit 3022 further includes a TU decoding unit 3024. These may be collectively referred to as a decoding module. The header decoding unit 3020 decodes, from coded data, parameter set information such as the VPS, the SPS, and the PPS, and a slice header (slice information). The CT information decoding unit 3021 decodes a CT from coded data. The CU decoding unit 3022 decodes a CU from coded data. In a case that a TU includes a prediction error, the TU decoding unit 3024 decodes QP update information (quantization correction value) and a quantization prediction error (residual_coding) from coded data.

[0054] Furthermore, an example in which a CTU and a CU are used as units of processing is described below, but the processing is not limited to this example, and processing in units of sub-CU may be performed. Alternatively, by replacing the CTU and the CU by a block and replacing the sub-CU by a subblock, and processing in units of blocks or subblocks may be performed.

[0055] The entropy decoding unit 301 performs entropy decoding on the coding stream Te input from the outside and separates and decodes individual codes (syntax elements). The separated codes include prediction information to generate a prediction image, a prediction error to generate a difference image, and the like. Entropy coding has a variable length coding method for syntax elements according to the context (probability model) adaptively selected according to the type of syntax elements and the surrounding conditions, and a variable length coding method for syntax elements using a predetermined table or formula.

[0056] The parameter decoding unit 302 notifies the entropy decoding unit 301 of which syntax elements need be decoded. The entropy decoding unit 301 outputs the syntax element to the prediction parameter derivation unit 320.

[0057] Configuration of Prediction Parameter Derivation Unit 320 The prediction parameter derivation unit 320 may derive the prediction parameters based on the output of the paremater decoding unit 302 and the prediction parematers which saved in the prediction parameter memory 307. The derived prediction parameters is output into the prediction image generation unit 308 and also is saved in the prediction parameter memory 307. The prediction parameter derivetion unit may derive different prediction mode for the Luma and Chroma prediction.

[0058] The loop filter 305 is a filter provided in the coding loop, and is a filter that removes block distortion and ringing distortion and improves image quality. The loop filter 305 applies a filter such as a deblocking filter, a Sample Adaptive Offset (SAO), and an Adaptive Loop Filter (ALF) on a decoded image of a CU generated by the addition unit 312.

[0059] The reference picture memory 306 stores the decoded image of the CU generated by the addition unit 312 in a predetermined position for each target picture and target CU.

[0060] The prediction parameter memory 307 stores prediction parameters in a predetermined position for each CTU or CU to be decoded. Specifically, the prediction parameter memory 307 stores a parameter derived by the prediction parameter derivation unit 320, a prediction mode predMode separated by the entropy decoding unit 301, and the like.

[0061] The prediction image generation unit 308 receives input of the prediction parameter derived by the prediction parameter deviation unit 320, and the like. In addition, the prediction image generation unit 308 reads a reference picture from the reference picture memory 306. The prediction image generation unit 308 generates a prediction image of a block or a subblock by using the prediction parameter and the read reference picture (reference picture block) in the prediction mode indicated by the prediction mode predMode. Here, the reference picture block refers to a set of pixels (referred to as a block because they are normally rectangular) on a reference picture and is a region that is referred to to generate a prediction image.

[0062] Prediction Image Generation Unit 308 In a case that the prediction mode predMode indicates an intra prediction mode, the intra prediction image generation unit 310 performs an intra prediction by using an intra prediction parameter (luma intra prediction mode IntraPredModeY and / or chroma intra prediction mode IntraPredModeC) input from the prediction parameter derivation unit 320 and reference pixels read from the reference picture memory 306. In a case that the prediction mode predMode indicates an inter prediction mode, the inter prediction image generation unit performs an inter prediction by using an inter prediction parameter input from the prediction parameter derivation unit 320 and reference pixels read from the reference picture memory 306.

[0063] Specifically, the prediction image generation unit 308 reads, from the reference picture memory 306, a neighbouring block in a predetermined range from a target block on a target picture. The predetermined range is neighbouring blocks on the left, the above left, the above, and the above right of the target block, and the region referred to is different depending on the intra prediction mode.

[0064] The prediction image generation unit 308 generates a prediction image of the target block with reference to the read decoded pixel values and the prediction mode indicated by predMode, IntraPredModeY and / or IntraPredModeC. The prediction image generation unit 308 outputs the generated prediction image of the block to the addition unit 312.

[0065] The generation of the prediction image based on the intra prediction mode is described below. In the Planar prediction, the DC prediction, and the Angular prediction, a decoded peripheral region adjacent to (proximate to) the prediction target block is configured as a reference region R. Then, the pixels on the reference region R are extrapolated in a specific direction to generate the prediction image. For example, the reference region R may be configured as an L-shaped region including the left and above (or further, above left, above right, bottom left) of the prediction target block.

[0066] Intra prediction image generation unit 310 A configuration of the intra prediction image generation unit 310 is described using FIG. 5. The intra prediction image generation unit 310 includes a reference sample filter unit 3103 (second reference image configuration unit), an intra prediction unit 3104), and a prediction image corrector 3105 (prediction image corrector, filter switching unit, weight coefficient changing unit).

[0067] Based on each reference pixel (unfiltered reference image) on the reference region R, a filtered reference image generated by applying a reference pixel filter (first filter), and the intra prediction mode, the intra prediction unit 3104 generates a prediction image of the target block, and outputs the generated image to the prediction image corrector 3105. The prediction image corrector 3105 corrects the prediction image in accordance with the intra prediction mode, and outputs a corrected prediction image.

[0068] Hereinafter, the units included in the intra prediction image generation unit 310 is described.

[0069] Reference sample filter unit 3103 The reference sample filter unit 3103 applies the reference pixel filter (first filter) to the unfiltered reference image to derive a filtered reference image s[x][y] at each position (x, y) on the reference region R, in accordance with the intra prediction mode. Specifically, a low pass filter is applied to the unfiltered reference image at each position (x, y) and its surroundings, and a filtered reference image is derived. Note that the low pass filter need not necessarily be applied in all the intra prediction modes, and the low pass filter may be applied in some intra prediction modes. Note that the filter applied to an unfiltered reference image on a reference region R in the reference sample filter unit 3103 is referred to as the “reference pixel filter (first filter)”, whereas a filter that corrects the prediction image in the prediction image corrector 3105 described below is referred to as a “boundary filter (second filter)”.

[0070] Configuration of intra prediction unit 3104 The intra prediction unit 3104 generates, based on the intra prediction mode, the unfiltered reference image, and the filtered reference pixel value, a prediction image (prediction pixel value, uncorrected prediction image) of the prediction target block, and outputs a generated image to the prediction image corrector 3105. The intra prediction unit 3104 includes a Planar prediction unit 31041, a DC prediction unit 31042, an Angular prediction unit 31043, an LM prediction unit 31044, an MIP prediction unit (Matrix-based Intra Prediction) 31045, a DIMD (Decoder side Intra Mode Derivation) prediction unit 31046, and a newMPMList generation unit 31047 in the inside thereof. Also the intra prediction unit 3104 may include a TIMD (Template based Intra Mode Derivation) prediction unit 31048, shown in FIG. 5. The intra prediction unit 3104 selects a specific predictor in accordance with the intra prediction mode, and inputs an unfiltered reference image and a filtered reference image thereto. The relationship between the intra prediction mode and the corresponding predictor is as follows.

[0071] - Planar prediction ... Planar prediction unit 31041 - DC prediction ... DC prediction unit 31042 - Angular prediction ... Angular prediction unit 31043 - LM prediction ... LM prediction unit 31044 - MIP prediction ... MIP prediction unit 31045 - DIMD prediction ... DIMD prediction unit 31046 - EIP prediction ... EIP prediction unit 31047 Planar prediction The Planar prediction unit 31041 generates a prediction image q[x][y] by linearly adding multiple filtered reference images s[x][y] in accordance with the distance between the prediction pixel position and the reference pixel position, and outputs the generated image to the prediction image corrector 3105.

[0072] DC prediction The DC prediction unit 31042 derives a DC prediction value corresponding to the average value of the filtered reference image s[x][y], and outputs a prediction image q[x][y], which takes the DC prediction value as a pixel value.

[0073] Angular prediction The Angular prediction unit 31043 generates a prediction image q[x][y] using the filtered reference image s[x][y] in a prediction direction (reference direction) indicated by the intra prediction mode, and outputs the generated image to the prediction image corrector 3105.

[0074] LM prediction The LM prediction unit 31044 predicts the pixel value of the chroma based on the pixel value of luma. More specifically, a linear model generates a prediction chroma image (Cb, Cr) based on the decoded luma image. As an example of LM prediction, there is a CCLM (cross component linear model prediction) prediction. CCLM prediction is a prediction method using a linear model to predict chroma from luma to same block.

[0075] MIP prediction The MIP prediction unit 31045 generates a prediction image q[x][y] by the product sum operation on the reference sample s[x][y] and the weight matrix derived from the neighboring region, and outputs the prediction image q[x][y] to the prediction image corrector 3105.

[0076] DIMD prediction The DIMD method selects five intra prediction angular modes from the gradient histogram obtained using a 3x3 filter in the neighboring reference area of the current block. The final prediction result is then obtained by weighted calculation using these five angular modes and the prediction result of the planar mode.

[0077] EIP prediction (Extrapolation intra prediction, Extrapolation filter-based Intra Prediction mode) The EIP prediction unit 31047 uses an extrapolation filter for predicting current block. The EIP method may take the 15 pixels in the top-left corner (4x4 squire pixels minus bottom-right pixel) as input of the extrapolation filter and derives 1 pixel in the bottom-right corner as output of the extrapolation filter. It uses a function model (an EIP parameter) to predict the target pixel of the bottom-right corner based on the reference pixels located around the target pixel, e.g. in the top-left corner. Additionally, the EIP method uses Merge mode, inheriting the EIP paramters from adjacent and non-adjacent blocks that use the EIP method, and applies them to the current block. The EIP method uses two mode lists, namely curModelList and mergeModelList. FIG.6 illustrates the details of the EIP method.

[0078] Configuration of prediction image corrector 3105 The prediction image corrector 3105 corrects the prediction image output from the intra prediction unit 3104 in accordance with the intra prediction mode. Specifically, the prediction image corrector 3105 derives, by performing weighted addition (weighted-averaging) on the unfiltered reference image and the prediction image for each pixel of the prediction image, in accordance with the distance between the reference region R and the target prediction pixel, the prediction image (corrected prediction image) Pred in which the prediction image is modified. Note that in some intra prediction modes (for example, Planar prediction, DC prediction, or the like), the prediction image corrector 3105 may not correct the prediction image, and the output of the intra prediction unit 3104 may be used as the prediction image.

[0079] Application examples EIP generation unit 31047 FIG. 6 illustrates the structure of the EIP Prediction Unit 31047, comprising a Reference Sample Derivation Unit 4701 and a Prediction Mode Derivation Unit 4710. The Prediction Mode Derivation Unit 4710 is composed of the mergeModelList Derivation Unit 4702 and the curModelList Derivation Unit 4703. The mergeModelList Derivation Unit 4702 includes the mergeModelList Derivation Unit 47021 and the mergeModelList Reorder Unit 47022. The curModelList Derivation Unit 4703 consists of the Candidates Derivation Unit 47031, Model Derivation Unit 47032, and curModelList Reorder Unit 47033.

[0080] Reference Sample Derivation Unit 4701 The Reference Sample Derivation Unit 4701 determines the reference area used for the EIP model derivation in Model Derivation Unit 47032. The EIP Prediction Unit 31047 may use three extrapolation filters: Square filter, Horizontal filter, and Vertical filter and the input pixels used by the extrapolation filter may depend on the extrapolation filters shape.FIG.8 illustrates these three filters where the gray pixels (InpW x InpH pixels excluding / minus bottom-right pixel) in each filter represents the input to the model, and the white pixel represents the output of the model. (InpW, InpH) of SquareFilter may be (4, 4), (3, 3) where InpW = InpH.. (InpW, InpH) of Vertical filter may be (2, 8), (3, 5), (2, 5) or (2, 4) where InpW < InpH. (InpW, InpH) of Horizontal filter may be (8, 2), (5, 3), (5, 2), (4, 2) where InpW > InpH.

[0081] To derive the EIP parameter (filterCoefficient[], a set of integer values), the EIP Model Derivation Unit uses a different template area and a corresponding referene area. The EIP Model Derivation Unit derives an appropriate EIP paramter to predict target pixels of the template area using referene pixels. There are three reference area (refArea) patterns, as shown in FIG.7: above_left (A_L), above (A), and left (L). Each reference area comprises two parts: Reference of Input (ROI) and Reference and Template (RTEMP). ROI is only used for input / reference of the EIP parameter derivation. RTEMP is used both for input / reference of EIP parameter derivation and for template / output. When the filters require the InpW and InpH pixels around output pixel as the input, ROI has InpW-1 and InpH-1 more lines than RTEMP. InpW and InpH is an integer constant depending on filterShape. The lines TempA and TempL of RTEMP above and left the current block may be dynamically determined based on the height and width of the current block.

[0082] The value of TempL and TempA may be consistent with the smaller of curWidth and curHeight, where curWidth and curHeight represent the width and height of the current block, respectively. Different template size values may be used for left neighbour side of the current block and above neighbour side of the current block, denoted as TempL and TempA. Combining with FIG.7, the range of the top reference area is: (x, y), where the range of x is [-TempL-InpW-1, 2*curWidth-1] and the range of y is [-TempA-InpH-1, -1]; the range of the left reference area is: (x, y), where the range of x is [-TempL-InpW-1, -1] and the range of y is [-TempA-InpH-1, 2*curHeight]; the range of the top-left reference area includes both the top and left reference areas.

[0083] In this embodiment, template size (TempA and TempL) may be set using the current block size as the following methods: (Example 1: Same fixed value for the top and left sides.) The template size is set to a fixed value, denoted as n, where n > 0. TempA = TempL = n.

[0084] (Example 2: Same block size dependent value for the above and left sides.) The template size is set as a block size dependent value such as being set to N times the smaller of the width and height of the block (where N is an integer greater than or equal to 1). TempL = N*curWidth, TempA = N*curHeight. N can be rational numbers and their denominator may be 2, 4, 8, … power of 2 values. TempL = (N*curWidth)>>shiftN, TempL = (N*curHeight)>> shiftN. shiftN may be 2, 4, 8 etc. Alternatively, The template size is set to M times the larger of the width and height of the block (where M is an integer greater than or equal to 1). TempL = curWidth / M, TempA = curHeight / M. division may be replaced with shift operation such as TempL = curWidth>>shiftM, TempA = curHeight>>shiftM. shiftM may be 2, 4, 8 etc.

[0085] (Example 3: Different block size dependent values for the above and left sides.) The value above the current block (TempA) is set as N times the width of the current block (where N is an integer greater than or equal to 1), and the value to the left (TempL) is set as M times the height of the current block (where M is an integer greater than or equal to 1).

[0086] TempL = N*curWidth, TempA = M*curHeight. N and M can be rational numbers and their denominator may be 2, 4, 8, … power of 2 values. TempL = (N*curWidth)>>shiftN, TempA = (M*curHeight)>> shiftN. shiftN may be 2, 4, 8 etc. Alternative; TempA and TempL may be set individually using curWidth and curHeight with conditions.

[0087] TempL = 4 (if curWidth < 8) else TempL = 8 (if curWidth < 16) else TempL = 16 (otherwise) TempA = 4 (if curHeight < 8) else TempA = 8 (if curHeight < 16) else TempA = 16 (otherwise) where TempL and TempA are determined with one or more conditions which compares the current block width and current block height with the pre-determined value.

[0088] TempL and TempA may be determined using look up table in which index of the table is determined using log2 of the current block width (=log2(curWidth)) and current block height (=log2(curHeight)) where the table has the values in the ascending order with the maximum value is equal to or less than the specific value.

[0089] TempL = TempTableL[log2(curWidth)], TempTableL[] may be {1, 2, 4, 8, 8, 8, 8, 8, 8}.

[0090] TempA = TempTableA[log2(curWidth)] , TempTableA[] may be {1, 2, 4, 8, 8, 8, 8, 8, 8}.

[0091] In other embodiment. InpW and InpH may be determined using the current block size as well.

[0092] e.g.

[0093] For Square filter, InpW = InpH = 3 if the current block is small blocks InpW = InpH = 4 for otherwise / non-small blocks.

[0094] For Vertical filter, InpW = 2, InpH = 4 if the current block is small blocks InpW = 2, InpH = 8 for otherwise / non-small blocks.

[0095] For Horizontal filter, InpW = 4, InpH = 2 if the current block is small blocks InpW = 8, InpH = 2 for otherwise / non-small blocks.

[0096] InpW and InpH may be determined with one or more conditions which compares the current block width and current block height with the pre-determined value.

[0097] InpW and InpH may be determined using look up table in which index of the table is determined using log2 of the current block width (=log2(curWidth)) and current block height (=log2(curHeight)) where the table has the values in the ascending order with the maximum value is equal to or less than the specific value.

[0098] InpW = InpTableW[log2(curWidth)], InpTableW[] may be {4, 4, 4, 8, 8, 8, 8, 8, 8}.

[0099] InpH = InpTableH[log2(curWidth)] , InpTableH[] may be {4, 4, 4, 8, 8, 8, 8, 8, 8}.

[0100] The curModelList Derivation Unit 4703 The curModelList Derivation Unit 4703 generates a list of EIP paramters calculated based on the reference area of the current block. It includes Candidates Derivation Unit 47031, Model Derivation Unit 47032, and curModelList Reorder Unit 47033. A function model comprises filterShape and the filterCoefficient[]. The filterShape represents the input pixels for the extrapolation filter being employed (in this embodiment it can be S, V, H as describe above), and filterCoefficient[] includes NCoef filter coefficients that will be multiplied with each input pixel to predict a target pixel. NCoef is set equal to NRef = (InpH x InpW) - 1 if bias term is not used. If bias team is used, NCoef is set equal to NRef + 1. NRef is the number of reference / input pixels to predict the target pixel. For 4x4 Squqre, 2x8 Vertical and 8x2 Horizontal flter, NRef is 15. For 3x3 Squqre, 2x5 Vertical and 5x2 Horizontal flter, NRef is 8, 9 and 9 respectively. For 2x4 Vertical and 4x2 Horizontal flter, NRef is 7. For 3x5 Vertical and 5x3 Horizontal flter, NRef is 14.

[0101] Candidates Derivation Unit 47031 The EIP method generates multiple candidate models for a target block, with each candidate model differing either in filter shape (filterShape), reference area (), or both. The number of candidate models produced for the current block, as well as the filters and reference areas used by each candidate model, are determined based on the size of the current block. The specific definition is illustrated in FIG.9. As shown in the table in FIG.9, the first row represents the current block's width (w), which takes values of (4, 8, 16, 32). The first column represents the current block's height (h), with possible values of (4, 8, 16, 32). Each cell in the table corresponds to a specific block size, for example, the cell in the second row and second column represents a target block of size 4x4. Each cell contains information (setting) about the number of candidate of models to be used for the target block size, along with the filter and reference area for each model. The selectable candiate filter shape (fiterShape) is represented as (S, V, H), indicating (Square filter, Vertical filter, Horizontal filter), as shown in FIG.8. The selectable candidate set of reference area (refArea) is represented as (A_L, A, L), indicating (above_left, above, left) area, as shown in FIG.7. The Candidates Derivation Unit 47031 generates a candidate list named curCandiList as the candidate models for the target block based on the table shown in FIG.9 andthe size / shape of the target block. In the case of 4x4 block, the curCandiList is (A_L, S), (A_L, V) and (A_L, H).

[0102] Model Derivation Unit 40732 The Model Derivation Unit 40732 derives a set offilter coefficients (EIP parameter for each model (filterShape, refArea) based on the elements in curCandiList. The Model Derivation Unit 40732 derives curModelList by storing the derived EIP paramter (filterCoefficient[]) and filterShape in the curModelList. In the case of 4x4 block, the curCandiList is (filterCoeff0, S), (filterCoeff1, V), (filterCoeff2, H). filterCoeff0, filterCoeff1 and filterCoeff2 is derived filter coefficients using reference area and filterShape of (A_L, S), (A_L, V) and (A_L, H) in the curCandiList, respectively.

[0103] Assuming there exists the following functional relationship between the input and output pixels in the currently used filter: output = (a0*x0 + a1*x1 + a2*x2 + a3*x3 + a4*x4 + a5*x5 + a6*x6 + a7*x7 + a8*x8 + a9*x9 + a10*x10 + a11*x11 + a12*x12 + a13*x13 + a14*x14 + offset) >> shift (xi is set of values of the input pixels, output is value of the output pixel). The EIP parameters or filterCoefficient[]= [a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, a12, a13, a14] is derived based on multiple [input, output] samples in the current reference where positions of output is of pixels of the template area specified by the refArea and input is of input pixels for the corresponding output pixel specified by the filterShape.

[0104] Based on multiple reference samples, the coefficient matrix A for the system of equations is obtained. A is a 2D matrix with dimensions NxN, where N is the number of the current reference samples of the number of the filterCoefficient[], e.g. 15. Let matrix P be a 1D column matrix with a length equal to M (according to the given relationship equation, P is the transpose of [a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, a12, a13, a14]). Let matrix T be a 1D column matrix with a length equal to N, where each element in T corresponds to the target pixel.

[0105] The relationship is then given by: A・P=T Now, pre-multiply both sides by the transpose of A (At): At・A・P = At・T Since At・A results in a symmetric matrix S[][] and At・T resuls in a vector V[], it can undergo an LDL decomposition. Let L be a lower triangular matrix, and D be a diagonal matrix, with Lt being the transpose of L.

[0106] L・D・Lt・P = At・T Given that A and T are known, L and D are calculated using the LDL decomposition or Cholesky decomposition formula. Therefore, matrix P, representing the coefficients of the given function relationship, is obtained through simple computation.

[0107] The Model Derivation Unit 40732 stores pixels in the reference area in separate arrays, refX[][] and refY[]. refX[][] is a two-dimensional array with a size of NCoef x NTemp, where NCoef is the number of the reference pixels for a target pixel or the number of filter coefficien, filterCoefficient[], e.g. NCoef = 15. NTemp is the number of currently available reference pixels or template pixels within RTEMP (e.g. NTemp = TempA*TempH + TempA*curWidth*2 + TempH*curHeight*2). In RefX[][] is stored the transpose of the coefficient matrix A, denoted as At. It is noted xi correspinding to refX[i][].

[0108] Independend of the filterShape (type of filter), the Model Derivation Unit 40732 sets refY[]as follows: refY[] = recSamples[x][y] Depending on the filterShape (type of filter), the Model Derivation Unit 40732 sets refX[][]as follows: Square filter (filterShape = S): refX[0][] = recSamples[x-1][y] refX[1][] = recSamples[x-2][y] refX[2][] = recSamples[x-3][y] refX[3][] = recSamples[x][y-1] refX[4][] = recSamples[x-1][y-1] refX[5][] = recSamples[x-2][y-1] refX[6][] = recSamples[x-3][y-1] refX[7][] = recSamples[x][y-2] refX[8][] = recSamples[x-1][y-2] refX[9][] = recSamples[x-2][y-2] refX

[0010] [] = recSamples[x-3][y-2] refX

[0011] [] = recSamples[x][y-3] refX

[0012] [] = recSamples[x-1][y-3] refX

[0013] [] = recSamples[x-2][y-3] refX

[0014] [] = recSamples[x-3][y-3] refX

[0015] [] = 1 if bias term is used.

[0109] Horizontal filter (filterShape = H): refX[0][] = recSamples[x-1][y] refX[1][] = recSamples[x][y-1] refX[2][] = recSamples[x-1][y-1] refX[3][] = recSamples[x][y-2] refX[4][] = recSamples[x-1][y-2] refX[5][] = recSamples[x][y-3] refX[6][] = recSamples[x-1][y-3] refX[7][] = recSamples[x][y-4] refX[8][] = recSamples[x-1][y-4] refX[9][] = recSamples[x][y-5] refX

[0010] [] = recSamples[x-1][y-5] refX

[0011] [] = recSamples[x][y-6] refX

[0012] [] = recSamples[x-1][y-6] refX

[0013] [] = recSamples[x][y-7] refX

[0014] [] = recSamples[x-1][y-7] refX

[0015] [] = 1 if bias term is used.

[0110] Vertical filter (filterShape = V): refX[0][] = recSamples[x][y-1] refX[1][] = recSamples[x-1][y] refX[2][] = recSamples[x-1][y-1] refX[3][] = recSamples[x-2][y] refX[4][] = recSamples[x-2][y-1] refX[5][] = recSamples[x-3][y] refX[6][] = recSamples[x-3][y-1] refX[7][] = recSamples[x-4][y] refX[8][] = recSamples[x-4][y-1] refX[9][] = recSamples[x-5][y] refX

[0010] [] = recSamples[x-5][y-1] refX

[0011] [] = recSamples[x-6][y] refX

[0012] [] = recSamples[x-6][y-1] refX

[0013] [] = recSamples[x-7][y] refX

[0014] [] = recSamples[x-7][y-1] Here, (x, y) represents the coordinates of the current target pixel in the reconstructed image, and recSamples is the reconstructed image.

[0111] refX

[0015] [] = 1 if bias term is used.

[0112] The Model Derivation Unit 40732 derives XX[i][j] and XY[i] as follows: XX[i][j] and XY[i] is initizalied to 0 and updated as follows: XX[i][j] += refX[i][]*ref[j][], i = 0..NCoef-1, j=0..NCoef-1 XY[i] += refX[i]*refY[i], i = 0..NCoef-1 In this embodiment, the bias item is omitted or added as either a fixed or dynamic value.

[0113] (Example 1: Omitting the bias item) Omitting the bias item, in a model with 15 coefficients, each multiplied by the input pixel values.

[0114] output = (a0*x0 + a1*x1 + a2*x2 + a3*x3 + a4*x4 + a5*x5 + a6*x6 + a7*x7 + a8*x8 + a9*x9 + a10*x10 + a11*x11 + a12*x12 + a13*x13 + a14*x14 + offset) >> shift. offset = (1<<shift)>>1 or 0. shift is a constant integer value, which may be 6 to 15.

[0115] (Example 2: Use a fixed bias item) Use a fixed bias item, in a model with a total of 16 coefficients. The first 15 coefficients are the same as in Example 1, and the 16th is the bias term denoted as bias. x15 = refX

[0015] [] = 1 is used. It is noted that the order of these 16 items is interchangeable.

[0116] output = (a0*x0 + a1*x1 + a2*x2 + a3*x3 + a4*x4 + a5*x5 + a6*x6 + a7*x7 + a8*x8 + a9*x9 + a10*x10 + a11*x11 + a12*x12 + a13*x13 + a14*x14 + bias (=a15 * x15) + offset) >> shift EIP multi-Model In this embodiment, the EIP multi-Model mode is derived and used, in which one model (single model) or two models (multi model) are used for predicting the current block. In the multil model, The Model Derivation Unit 40732 derives two set ofEIP paramters based on the input and output parts of the filter.

[0117] output0 = (a[0][0]*x[0][0] + a[0][1]*x[0][1] + a[0][2]*x[0][2] + a[0][3]*x[0][3] + a[0][4]*x[0][4] + a[0][5]*x[0][5] + a[0][6]*x[0][6] + a[0][7]*x[0][7] + a[0][8]*x[0][8] + a[0][9]*x[0][9] + a[0]

[0010] *x[0]

[0010] + a[0]

[0011] *x[0]

[0011] + a[0]

[0012] *x[0]

[0012] + a[0]

[0013] *x[0]

[0013] + a[0]

[0014] *x[0]

[0014] + offset) >> shift (for model0) output1 = (a[1][0]*x[1][0] + a[1][1]*x[1][1] + a[1][2]*x[1][2] + a[1][3]*x[1][3] + a[1][4]*x[1][4] + a[1][5]*x[1][5] + a[1][6]*x[1][6] + a[1][7]*x[1][7] + a[1][8]*x[1][8] + a[1][9]*x[1][9] + a[1]

[0010] *x[1]

[0010] + a[1]

[0011] *x[1]

[0011] + a[1]

[0012] *x[1]

[0012] + a[1]

[0013] *x[1]

[0013] + a[1]

[0014] *x[1]

[0014] + offset) >> shift (for model1) Note: a[i][j] correspinds to aj (=filterCoefficient[j]) for model i. x[i][j] corresponds to xj (=refX[j][]) of reference pixels in which the target pixel is classified to class i.

[0118] The samples in the reference are classified into two classes, denoted as Class 0 and Class 1. Samples in Class 0 are used to derive the EIP parameters of Model 0, and those in Class 1 are used to derives the EIP parameters of Model 1.

[0119] Set a threshold thres as the average pixel value in the current reference. Regarding classification methods, there are the following implementation options: (Method1) For each sample, if the average pixel value of the filter input is less than a thres, the sample is classified into Class 0; otherwise, it is classified into Class 1. The average avg may be derived as ΣrefX[i] / 15. avg may be derived as (refX[0] +ΣrefX[i] + 8) >> 4. i=0..15.

[0120] (Method2) Compute the average value aver2 of the pixels to neibouring pixels of the output / target pixel. The neibouring pixels may be the left and above of the output / target pixel pixels in the filter. If aver2 is less than thres, the sample is classified into Class 0; otherwise, it is classified into Class 1. The average value aver2 may be derived as (recSamples[x-1][y] + recSamples[x][y-1]) >>1.

[0121] (Method3) The neibouring pixels may be the left, above, and above-left of the output pixels in the filter. The neibouring pixels may be the left and above of the output / target pixel pixels in the filter. The average value aver3 may be derived as (recSamples[x-1][y] + recSamples[x][y-1] + recSamples[x-1][y-1]) / 3. If aver3 is less than a thres, the sample is classified into Class 0; otherwise, it is classified into Class 1. aver3 may be derived as (1*recSamples[x-1][y] + 2*recSamples[x][y-1] + 1*recSamples[x-1][y-1] + 2) >>2. aver3 may be derived as (3*recSamples[x-1][y] + 3*recSamples[x][y-1] + 2*recSamples[x-1][y-1] + 4) >>3.

[0122] For the configuration of single-model or multi-model usage in blocks, the following settings is applied: (Setting1) Single-model is applied in all blocks.

[0123] (Setting2) Multi-model is applied in all blocks.

[0124] (Setting3) Single-model is applied in small blocks, while multi-model is applied in large blocks. The definition of small blocks is curBlockWidth * curBlockHeight < N, and large blocks is defined as curBlockWidth * curBlockHeight >= N. The parameter N is set to values such as 16, 64, or 128. Here, curBlockWidth and curBlockHeight represent the width and height of the current block, respectively.

[0125] The curModelList Reorder Unit 47033 The curModelList Reorder Unit 47033 sorts the models in its input curModelList based on the SAD cost, with smaller SAD costs coming first and larger ones coming later. As illustrated in FIG.10, the curModelList Reorder Unit 47033 utilizes pixels from the Reference Of Input (ROI) and RTemp to predict pixels in the template area of RTEMP. It then compares the predicted pixels with the reconstructed pixels to obtain the SAD cost. sad = Σ (x, y) abs(output value for the targe pixel[x][y] derived using the input values correspondint to the filterShape - reconSamples[x][y]), where Σ(x, y) = posision of the template of RTEMP. Subsequently, it reorders the curModelList in ascending order based on the SAD cost.

[0126] The mergeModelList Derivation Unit 4702 The mergeModelList Derivation Unit 4702 generates the mergeModelList. It consists of two components, the mergeModelList Derivation Unit 47021, and the mergeModelList Reorder Unit 47022.

[0127] The mergeModelList Derivation Unit 47021 The mergeModelList Derivation Unit 47021 generates the mergeModelList, which consists of four parts: spatial adjacent candidates (A0, A1, B0, B1, B2), spatial non-adjacent candidates (N0, N1, N2, …), temporal candidates (Tmp), and history candidates (Hist0, Hist1, …). The constructed EIP merge list may include up to 5 to 12 candidates, e.g. {A1, B1, B0, A0, B2, N0, N1,,,Ni, Tmp, Hist0, Hist1, …Histj}.

[0128] The mergeModelList Derivation Unit 47021 The mergeModelList Derivation Unit 47021, similar to the role of The curModelList Reorder Unit 47033, reorders the models in the mergeModelList in ascending order.

[0129] The inverse quantization and inverse transform processing unit 311 performs inverse quantization on a quantization transform coefficient input from the prediction parameter derivation unit 320 to calculate a transform coefficient. This quantization transform coefficient is a coefficient obtained by performing a frequency transform such as a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), or the like on prediction errors to quantize in coding processing. The inverse quantization and inverse transform processing unit 311 performs an inverse frequency transform such as an inverse DCT, an inverse DST, or the like on the calculated transform coefficient to calculate a prediction error. The inverse quantization and inverse transform processing unit 311 outputs the prediction error to the addition unit 312.

[0130] The addition unit 312 adds the prediction image of the block input from the intra prediction image generation unit 310 and the prediction error input from the inverse quantization and inverse transform processing unit 311 for each pixel and generates a decoded image of the block. The addition unit 312 stores the decoded image of the block in the reference picture memory 306 and outputs the image to the loop filter 305.

[0131] Configuration of video coding apparatus Next, a configuration of the video coding apparatus 11 according to the present embodiment is described. FIG.11 is a block diagram illustrating a conuration of the video coding apparatus 11 according to the present embodiment. The video coding apparatus 11 is configured to include a prediction image generation unit 101, a subtraction unit 102, a transform and quantization processing unit 103, an inverse quantization and inverse transform processing unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (a prediction parameter storage unit, a frame memory) 108, a reference picture memory (a reference image storage unit, a frame memory) 109, a coding parameter determination unit 110, a parameter coding unit 111, prediction parameter derivation unit 120, and an entropy coding unit 104.

[0132] The prediction image generation unit 101 generates a prediction image for each CU that is a region obtained by splitting each picture of the image T. The operation of the prediction image generation unit 101 is the same as that of the intra prediction image generation unit 310 already described, and thus descriptions thereof is omitted.

[0133] The subtraction unit 102 subtracts a pixel value of the prediction image of the block input from the prediction image generation unit 101 from a pixel value of the image T to generate a prediction error. The subtraction unit 102 outputs the prediction error to the transform and quantization processing unit 103.

[0134] The transform and quantization processing unit 103 calculates a transform coefficient by performing a frequency transform on the prediction error input from the subtraction unit 102, and derives a quantization transform coefficient by quantization. The transform and quantization proceessing unit 103 outputs the quantization transform coefficient to the entropy coding unit 104 and the inverse quantization and inverse transform processing unit 105.

[0135] The inverse quantization and inverse transform processing unit 105 is the same as the inverse quantization and inverse transform processing unit 311 (FIG. 4) in the video decoding apparatus 31, and descriptions thereof are omitted. The calculated prediction error is output to the addition unit 106.

[0136] To the entropy coding unit 104, the quantization transform coefficient is input from the transform and quantization processing unit 103, and coding parameters are input from the parameter coding unit 111. The entropy coding unit 104 performs entropy coding on split information, the prediction parameters, the quantization transform coefficient, and the like to generate and output the coding stream Te.

[0137] The parameter coding unit 111 instructs the entropy coding unit 104 to encode the prediction parameters and quantization coefficients, derived from the prediction parameter derivation unit 120.

[0138] The prediction parameter derivation unit 120 derives the syntax element from the parameters inputted from the coding parameter determination unit 110. Some parts of the prediction parameter derivation unit 120 have the same structure as the prediction parameter derivation unit 320.

[0139] The addition unit 106 adds a pixel value of the prediction image of the block input from the prediction image generation unit 101 and the prediction error input from the inverse quantization and inverse transform processing unit 105 to each other for each pixel, and generates a decoded image. The addition unit 106 stores the generated decoded image in the reference picture memory 109.

[0140] The loop filter 107 applies a deblocking filter, an SAO, and an ALF to the decoded image generated by the addition unit 106. Note that the loop filter 107 need not necessarily include the above-described three types of filters, and may have a configuration of only the deblocking filter, for example.

[0141] The prediction parameter memory 108 stores the prediction parameters generated by the prediction parameter derivation unit 120 for each target picture and CU at a predetermined position. It may stores the transform coefficients created by the transform and quantization processing unit 103.

[0142] The reference picture memory 109 stores the decoded image generated by the loop filter 107 for each target picture and CU at a predetermined position.

[0143] The coding parameter determination unit 110 selects one set among multiple sets of coding parameters. A coding parameter refers to the above-mentioned QT, BT, or TT split information, the prediction parameter, or a parameter to be coded, the parameter being generated in association therewith. The prediction image generation unit 101 generates the prediction image by using these coding parameters.

[0144] The coding parameter determination unit 110 calculates, for each of the multiple sets, an RD cost value indicating the magnitude of an amount of information and a coding error. The RD cost value is, for example, the sum of a code amount and the value obtained by multiplying a coefficient λ by a square error. The coding parameter determination unit 110 selects a set of coding parameters of which cost value calculated is a minimum value. With this configuration, the entropy coding unit 104 outputs the selected set of coding parameters as the coding stream Te. The coding parameter determination unit 110 outputs the determined coding parameters in the parameter coding unit 111, the prediction parameter derivation unit 120, the prediction image generation unit 101.

[0145] Note that, some of the video coding apparatus 11 and the video decoding apparatus 31 in the above-described embodiment, for example, the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the intra prediction image generation unit 310, the inverse quantization and inverse transform processing unit 311, the addition unit 312, the prediction parameter derivation unit 320, the prediction image generation unit 101, the subtraction unit 102, the transform and quantization processing unit 103, the entropy coding unit 104, the inverse quantization and inverse transform processing unit 105, the loop filter 107, the coding parameter determination unit 110, and the parameter coding unit 111, the prediction parameter derivation unit 120, may be realized by a computer. In that case, this configuration may be realized by recording a program for realizing such control functions on a computer-readable recording medium and causing a computer system to read the program recorded on the recording medium for execution. Note that the “computer system” mentioned here refers to a computer system built into either the video coding apparatus 11 or the video decoding apparatus 31 and is assumed to include an OS and hardware components such as a peripheral apparatus. Furthermore, a “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, and the like, and a storage device such as a hard disk built into the computer system. Moreover, the “computer-readable recording medium” may include a medium that dynamically stores a program for a short period of time, such as a communication line in a case that the program is transmitted over a network such as the Internet or over a communication line such as a telephone line, and may also include a medium that stores the program for a fixed period of time, such as a volatile memory included in the computer system functioning as a server or a client in such a case. Furthermore, the above-described program may be one for realizing some of the above-described functions, and also may be one capable of realizing the above-described functions in combination with a program already recorded in a computer system.

[0146] Furthermore, a part or all of the video coding apparatus 11 and the video decoding apparatus 31 in the embodiment described above may be realized as an integrated circuit such as a Large Scale Integration (LSI). Each function block of the video coding apparatus 11 and the video decoding apparatus 31 may be individually realized as processors, or part or all may be integrated into processors. The circuit integration technique is not limited to LSI, and the integrated circuits for the functional blocks may be realized as dedicated circuits or a multi-purpose processor. In a case that with advances in semiconductor technology, a circuit integration technology with which an LSI is replaced appears, an integrated circuit based on the technology may be used.

[0147] The embodiment of the present disclosure has been described in detail above referring to the drawings, but the specific configuration is not limited to the above embodiments and various amendments may be made to a design that fall within the scope that does not depart from the gist of the present disclosure.

[0148] The embodiment of the present invention may be applied to a video decoding device that decodes encoded data of image data, and a video encoding device that generates encoded data from image data. In addition, the data structure of the encoded data is generated by the video encoding device and referenced by the video decoding device.

[0149] In one example, a video decoding apparatus for generating a prediction imag is provided. The appratus includes an EIP prediction unit configured to generate intra prediction images using a function model generated based on samples in a reference area.

[0150] In one example, a video coding apparatus is provided. The appratus includes an EIP prediction unit configured to predict the target block using the function model.

[0151] <Cross Reference> This patent application claims priority on JP Patent Application No. 2024-043054 filed on March 19, 2024, the entire contents of which are hereby incorporated by reference.

[0152] Reference Signs List 31 Image decoding apparatus 301 Entropy decoding unit 302 Parameter decoding unit 311 Inverse quantization and inverse transform processing unit 312 Addition unit 310 Prediction image generation unit 3104 Intra prediction unit 31047 EIP prediction unit 4701 Reference sample derivation unit 4710 Prediction Mode Derivation Unit 4702 mergeModelList Derivation Unit 47021 mergeModelList Derivation Unit 47022 mergeModelList Reorder Unit 4703 curModelList Derivation Unit 47031 Candidates Derivation Unit 47032 Model Derivation Unit 47033 curModelList Reorder Unit 11 Image coding apparatus 101 Prediction image generation unit 102 Subtraction unit 103 Transform and quantization processing unit 104 Entropy coding unit 105 Inverse quantization and inverse transform processing unit 107 Loop filter 110 Coding parameter determination unit 111 Parameter coding unit

Claims

1. A video decoding apparatus for generating a prediction image, the video decoding apparatus comprising an EIP prediction unit configured to generate intra prediction images using a function model generated based on samples in a reference area.

2. The video decoding apparatus according to claim 1, wherein the EIP prediction unit designed to determine a number of lines in the reference area, denoted as a tempA and a TempLwherein the tempA and the TempL are set as a fixed value, a dynamic value, or different values for above and left parts of the reference area, depending on width and height of a target block.

3. The video decoding apparatus of claim 1, wherein the EIP prediction unit further configured to derive a second function model, with or without a bias item, where an input of the bias item is set as a fixed value or a dynamic value.

4. The video decoding apparatus of claim 1 further comprising a second EIP prediction unit with a reordering curModelList method configured to increase a likelihood of selecting certain elements first.

5. A video coding apparatus comprising an EIP prediction unit configured to predict a target block using a function model generated based on samples in a reference area.

Images