Apparatus, method and computer program for video encoding and decoding

Abstract

9. A method for predicting at least one color channel sample of an image block unit, the method comprising: receiving an image block unit of a frame, the image block unit including color channel samples including at least one chrominance channel and one luminance channel (900); reconstructing the luminance channel samples of the image block unit (902); determining a reference area for predicting at least one color channel sample of the image block unit, the reference area including one or more of reference samples in a neighboring block of a current color channel / frame, in a neighboring block of a reference color channel / frame, and / or inside a collocated block of a reference color channel / frame (904); determining filter coefficients of a filter for prediction based on the reference samples and a shape of the filter (906); and predicting at least one color channel sample of the image block by performing a convolution between the luminance channel samples and the filter coefficients (908).

Description

[Technical field]

[0001] The present invention relates to an apparatus, a method and a computer program for video encoding and decoding. [Background technology]

[0002] In video coding, video and image samples are typically coded using a color representation such as YUV or YCbCr, which consists of one luminance (luma) and two chrominance (chroma) channels. In these cases, the luminance channel represents much of the illumination of the scene and is typically coded at a particular resolution, while the chrominance channels typically represent the differences between particular color components and are often coded at a second resolution, which is lower than the luminance signal. The aim of this type of difference representation is to decorrelate the color components so that the data can be compressed more efficiently.

[0003] In the Versatile Video Coding (VVC / H.266) standard, a cross-component linear model (CCLM) is used as a linear model to predict samples in chroma channels (e.g., Cb and Cr). The model parameters are derived based on reconstructed samples in the neighborhood of a chroma block, co-located neighboring samples in a luma block, and reconstructed samples inside the co-located luma block.

[0004] The purpose of the model is to determine the correlation of samples between two or more channels. However, the first-order linear model of the CCLM method is not suitable for modeling spatial correlations between luma and chroma samples (such as textures or surfaces), resulting in suboptimal performance. Summary of the Invention [Problem to be solved by the invention]

[0005] Now, to at least alleviate the above problems, an enhanced method for achieving better correlation between the luma and chroma channels is introduced herein.

[0006] The scope of protection sought for various embodiments of the invention is set forth in the independent claims. The embodiments and features described herein that are not within the scope of the independent claims, if any, should be interpreted as example implementations useful for understanding various embodiments of the invention. [Means for solving the problem]

[0007] A method according to a first aspect includes the steps of receiving an image block unit of a frame, the image block unit including color channel samples including at least one chrominance channel and one luminance channel; reconstructing the luminance channel samples of the image block unit; determining a reference area for predicting the at least one color channel sample of the image block unit, the reference area including one or more of the reference samples in neighboring blocks of a current color channel / frame, in neighboring blocks of a collocated block in a reference color channel / frame, and / or inside a collocated block in a reference color channel / frame; determining filter coefficients of a filter for prediction based on the reference samples and a shape of the filter; and predicting the at least one color channel sample of the image block unit by performing a convolution between the luminance channel samples and the filter coefficients.

[0008] An apparatus according to a second aspect includes: means for receiving an image block unit of a frame, the image block unit including color channel samples including at least one chrominance channel and one luminance channel; means for reconstructing the luminance channel samples of the image block unit; means for determining a reference area for predicting the at least one color channel sample of the image block unit, the reference area including one or more of reference samples in a neighboring block in a current color channel / frame, in a neighborhood of a collocated block in a reference color channel / frame, and / or inside a collocated block in a reference color channel / frame; means for determining filter coefficients of a filter for prediction based on the reference samples and a shape of the filter; and means for predicting the at least one color channel sample of the image block unit by performing a convolution between the luminance channel samples and the filter coefficients.

[0009] According to one embodiment, the device comprises means for downsampling the samples of the luminance channel to correspond to a sample size of the chrominance samples before determining the filter coefficients.

[0010] According to one embodiment, the means for determining the filter coefficients includes means for obtaining statistics from reference samples for linear regression based on the shape of the filter, and means for obtaining the filter coefficients by applying a linear regression method to the obtained statistics.

[0011] According to one embodiment, the statistics include an autocorrelation matrix and a cross-correlation vector.

[0012] According to one embodiment, the shape of the filter is a one-dimensional (1D) vector of filter coefficients or a two-dimensional (2D) array of filter coefficients.

[0013] According to one embodiment, the device comprises means for applying auxiliary parameters from outside the shape of the filter to the convolution.

[0014] According to one embodiment, the apparatus includes means for inputting to a filter a luma sample collocated with an output chroma sample, at least one luma sample above the collocated sample, at least one luma sample below the collocated sample, at least one luma sample to the left of the collocated sample, at least one luma sample to the right of the collocated sample, and a bias term.

[0015] According to one embodiment, the apparatus includes means for inputting to the filter values ​​determined by applying a non-linear function to the values ​​of the collocated samples.

[0016] According to one embodiment, the apparatus includes means for storing filter coefficients in a memory and means for receiving an indication from an encoder to reuse at least a subset of previously stored filter coefficients to predict samples of at least one color channel of a current image block unit.

[0017] According to one embodiment, the apparatus includes means for determining samples outside a reference area, means for obtaining statistics from the samples outside the reference area for linear regression based on a shape of the filter, means for obtaining filter coefficients by applying a linear regression method to the obtained statistics, and means for storing the filter coefficients in a memory.

[0018] According to one embodiment, the apparatus includes means for replacing any missing downsampled luminance samples by zero value samples.

[0019] According to one embodiment, the apparatus includes means for determining samples that do not belong to any object or texture inside the image block unit, and means for removing the samples from the reference area.

[0020] Thus, as described above, an apparatus and a computer readable storage medium having code stored thereon is configured to carry out one or more of the above described methods and related embodiments.

[0021] For a better understanding of the invention, reference is now made, by way of example, to the accompanying drawings, in which: [Brief description of the drawings]

[0022] [Figure 1] 1 is a schematic diagram illustrating an electronic device utilizing an embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic diagram illustrating user equipment suitable for utilizing an embodiment of the present invention; [Diagram 3] FIG. 1 is a schematic diagram illustrating electronic devices utilizing an embodiment of the present invention connected using wireless and wired network connections. [Figure 4a] 1 is a schematic diagram of an encoder and a decoder suitable for implementing embodiments of the present invention; [Figure 4b] 1 is a schematic diagram of an encoder and a decoder suitable for implementing embodiments of the present invention; [Diagram 5] FIG. 1 illustrates sample locations used to derive parameters of a cross-component linear model. [Figure 6a] FIG. 1 shows an example of classification of luma samples into two classes in the sample domain. [Figure 6b] FIG. 1 shows an example of classification of luma samples into two classes in the spatial domain. [Figure 7] FIG. 2 shows an example of four reference lines in the neighborhood of a prediction block. [Figure 8] FIG. 2 illustrates a matrix weighting intra prediction process. [Figure 9] 4 is a flow diagram illustrating a method for predicting samples of at least one color channel according to one embodiment of the present invention. [Figure 10]FIG. 2 illustrates an example of a collocated reference sample area consisting of reconstructed luma and chroma samples defined for both luma and chroma, according to one embodiment of the invention. [Figure 11] 5A-5C are diagrams illustrating various examples of filter kernel dimensions according to one embodiment of the present invention. [Figure 12] 1 is a schematic diagram illustrating an exemplary multimedia communication system in which various embodiments may be implemented; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] The following describes in more detail suitable apparatus and possible mechanisms for prediction of chroma samples. In this regard, reference is first made to Figures 1 and 2, which shows a block diagram of a video encoding system according to an exemplary embodiment as a schematic block diagram of an exemplary apparatus or electronic device 50 that may incorporate a codec according to one embodiment of the present invention. Figure 2 shows the layout of an apparatus according to an exemplary embodiment. The elements of Figures 1 and 2 are now described.

[0024] The electronic device 50 may be, for example, a mobile terminal or user equipment of a wireless communication system, however, it will be appreciated that embodiments of the invention may be implemented in any electronic device or apparatus that may require the encoding and / or decoding of video images.

[0025] The device 50 may include a housing 30 that houses and protects the device. The device 50 may further include a display 32 in the form of a liquid crystal display. In other embodiments of the invention, the display may be any suitable display technology suitable for displaying images or video. The device 50 may further include a keypad 34. In other embodiments of the invention, any suitable data or user interface mechanism may be utilized. For example, the user interface may be implemented as a virtual keyboard or data entry system as part of a touch-sensitive display.

[0026] The device may include a microphone 36 or any suitable audio input, which may be a digital or analog signal input. The device 50 may further include an audio output device, which in embodiments of the invention may be any one of an earphone 38, a speaker, or an analog or digital audio output connection. The device 50 may also include a battery (or in other embodiments of the invention, the device may be powered by any suitable mobile energy device, such as a solar cell, a fuel cell, or a clockwork generator). The device may further include a camera, which may record or capture images and / or videos. The device 50 may further include an infrared port for short-range line-of-sight communication with other devices. In other embodiments, the device 50 may further include any suitable short-range communication solution, such as, for example, a Bluetooth wireless connection or a USB / Firewire wired connection.

[0027] The device 50 may include a controller 56, processor or processor circuitry that controls the device 50. The controller 56 may be connected to a memory 58, which may store data, both in the form of image and audio data, and / or may store instructions for the controller 56 to carry out in embodiments of the present invention. The controller 56 may further be connected to a codec circuit 54 suitable for performing encoding and decoding of audio and / or video data or for assisting in the encoding and decoding performed by the controller.

[0028] The device 50 may further include a card reader 48 and a smart card 46, for example a UICC and UICC reader suitable for providing user information and credentials for authentication and authorization of the user on the network.

[0029] The device 50 may include a radio interface circuit 52 coupled to the controller and suitable for generating wireless communication signals for communication with, for example, a cellular communication network, a wireless communication system, or a wireless local area network. The device 50 may further include an antenna 44 coupled to the radio interface circuit 52 for transmitting radio frequency signals generated by the radio interface circuit 52 to other devices and for receiving radio frequency signals from other devices.

[0030] The device 50 may include a camera capable of recording or detecting individual frames that are then passed to a codec 54 or controller for processing. The device may receive video image data from another device for processing before transmission and / or storage. The device 50 may also receive images for encoding / decoding either wirelessly or via a wired connection. The structural elements of the device 50 described above represent examples of means for performing the corresponding functions.

[0031] With reference to Figure 3, an example of a system in which an embodiment of the present invention may be utilized is shown. System 10 includes a number of communication devices capable of communicating over one or more networks. System 10 may include any combination of wired or wireless networks, including, but not limited to, wireless cellular telephone networks (such as GSM, UMTS, CDMA networks), wireless local area networks (WLANs) as defined by any of the IEEE 802.x standards, Bluetooth personal area networks, Ethernet local area networks, token ring local area networks, wide area networks, and the Internet.

[0032] System 10 may include both wired and wireless communication devices and / or apparatus 50 suitable for implementing embodiments of the present invention.

[0033] For example, the system depicted in Figure 3 shows a representation of a mobile telephone network 11 and the Internet 28. Connectivity to the Internet 28 can include, but is not limited to, long-range wireless connections, short-range wireless connections, and various wired connections, including, but not limited to, telephone lines, cable lines, electrical wires, and similar communication paths.

[0034] Exemplary communication devices shown in system 10 may include, but are not limited to, electronic device or device 50, a combination personal digital assistant (PDA) and mobile phone 14, a PDA 16, an integrated messaging device (IMD) 18, a desktop computer 20, and a notebook computer 22. Device 50 may be stationary or mobile when carried by an individual on the move. Device 50 may also be located in a mode of transportation, including, but not limited to, a car, truck, taxi, bus, train, boat, airplane, bicycle, motorcycle, or any similar suitable mode of transportation.

[0035] Embodiments may also be implemented in set-top boxes, i.e. digital TV receivers which may or may not have display or wireless capabilities, in tablets or (laptop) personal computers (PCs) with hardware or software or a combination of encoder / decoder implementations in various operating systems, and also in chipsets, processors, DSPs and / or embedded systems providing hardware / software based encoding.

[0036] Some or other devices may send and receive calls and messages and may communicate with a service provider via a wireless connection 25 to a base station 24. The base station 24 may be connected to a network server 26 that enables communication between the mobile telephone network 11 and the Internet 28. The system may include additional communication devices and different types of communication devices.

[0037] The communication devices may communicate using a variety of transmission technologies, including, but not limited to, Code Division Multiple Access (CDMA), Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Transmission Control Protocol-Internet Protocol (TCP-IP), Short Messaging Service (SMS), Multimedia Messaging Service (MMS), Email, Instant Messaging Service (IMS), Bluetooth, IEEE 802.11, and any similar wireless communication technologies. The communication devices involved in implementing various embodiments of the present invention may communicate using a variety of mediums, including, but not limited to, radio, infrared, laser, cable connections, and any suitable connection.

[0038] In telecommunications and data networks, a channel can refer to either a physical channel or a logical channel. A physical channel can refer to a physical transmission medium such as a wire, while a logical channel can refer to a logical connection through a multiplexer medium that can carry several logical channels. A channel can be used to transmit an information signal, e.g., a bit stream, from one or more senders (or transmitters) to one or more receivers.

[0039] The MPEG-2 Transport Stream (TS), specified in ISO / IEC 13818-1 or equivalently in ITU-T Recommendation H,222.0, is a format for carrying audio, video, and other media as well as program or other metadata in a multiplexed stream. Packet Identifiers (PIDs) are used within a TS to identify elementary streams (also known as packetized elementary streams). Thus, logical channels within an MPEG-2 TS can be considered to correspond to specific PID values.

[0040] Available media file format standards include the ISO base media file format (ISO / IEC 14496-12, which can be abbreviated as ISOBMFF) and the file format for NAL unit structured video (ISO / IEC 14496-15), which is derived from ISOBMFF.

[0041] A video codec consists of an encoder that converts an input video into a compressed representation suitable for storage / transmission, and a decoder that can restore the compressed video representation into a viewable form. A video encoder and / or a video decoder can also be separate from each other, i.e., they do not need to form a codec. Typically, an encoder discards some information of the original video sequence in order to represent the video in a more compact form (i.e., at a lower bit rate).

[0042] A typical hybrid video encoder, e.g., many encoder implementations of ITU-T H.263 and H.264, encodes video information in two phases. First, pixel values ​​of a particular picture area (or "block") are predicted, e.g., by motion compensation means (finding and pointing to an area in one of the previously coded video frames that closely corresponds to the block to be coded) or by spatial means (using pixel values ​​around the block to be coded in a specified manner). Then, the prediction error, i.e., the difference between the predicted block of pixels and the original block of pixels, is coded. This is typically done by transforming the pixel value differences using a specified transform (e.g., the discrete cosine transform (DCT) or a variant thereof), quantizing the coefficients, and entropy coding the quantized coefficients. By varying the fidelity of the quantization process, the encoder can control the balance between the precision of the pixel representation (picture quality) and the size of the resulting coded video representation (file size or transmission bit rate).

[0043] In temporal prediction, the source of prediction is a previously decoded picture (aka reference picture). In intra block copy (IBC, aka intra block copy prediction), prediction is applied similarly to temporal prediction, but the reference picture is the current picture, and only previously decoded samples are referenced in the prediction process. Inter-layer or inter-view prediction can be applied similarly to temporal prediction, but the reference picture is a decoded picture from another scalable layer or from another view, respectively. In some cases, inter prediction can refer to temporal prediction only, and in other cases, inter prediction can refer collectively to temporal prediction and any of intra block copy, inter-layer prediction, and inter-view prediction, which are performed by the same or similar process as temporal prediction. Inter prediction or temporal prediction may also be referred to as motion compensation or motion compensated prediction.

[0044] Motion compensation can be performed with either full-sample or sub-sample accuracy. In the case of full-sample accuracy motion compression, motion can be represented as motion vectors with integer values ​​of horizontal and vertical displacement, and the motion compensation process effectively copies samples from the reference picture using these displacements. In the case of sub-sample accuracy motion compensation, the motion vectors are represented by fractional or decimal values ​​of the horizontal and vertical components of the motion vector. If the motion vectors indicate non-integer positions in the reference picture, a sub-sample interpolation process is typically invoked to calculate predicted sample values ​​based on the reference samples and the selected sub-sample positions. The sub-sample interpolation process typically consists of horizontal filtering to compensate for the horizontal offset relative to the full sample positions followed by vertical filtering to compensate for the vertical offset relative to the full sample positions. However, the vertical processing can also occur before the horizontal processing in some environments.

[0045] Inter prediction, motion compensation, or motion compensated prediction, which can also be called temporal prediction, reduces temporal redundancy. In inter prediction, the source of prediction is a previously decoded picture. Intra prediction exploits the fact that adjacent pixels in the same picture may be correlated. Intra prediction can be performed in the spatial or transform domain, i.e., either sample values ​​or transform coefficients can be predicted. Intra prediction is generally exploited in intra coding, where inter prediction is not applied.

[0046] One result of the encoding procedure is a set of coding parameters, such as motion vectors and quantized transform coefficients. Many parameters can be efficiently entropy coded if they are first predicted from spatially or temporally neighboring parameters. For example, motion vectors can be predicted from spatially adjacent motion vectors, and only the difference to the motion vector predictor is coded. Prediction of coding parameters and intra prediction can be collectively referred to as in-picture prediction.

[0047] Figures 4a and 4b show an encoder and decoder suitable for utilizing embodiments of the present invention. A video codec consists of an encoder that converts an input video into a compressed representation suitable for storage / transmission, and a decoder that can decode the compressed video representation into a viewable form. Typically, the encoder discards and / or removes some information in the original video sequence in order to represent the video in a more compact form (i.e. at a lower bit rate). An example of the encoding process is shown in Figure 4a. Figure 4a shows the image (I n ), the predicted representation of the image block (P ’n ), prediction error signal (D n ), the reconstructed prediction error signal (D ’n ), preliminary reconstructed image (I ’n ), the final reconstructed image (R ’n ), transformation (T) and inverse transformation (T -1 ), quantization (Q) and inverse quantization (Q -1 ), Entropy coding (E), Reference Frame Memory (RFM), Inter prediction (P inter ), intra prediction (P intra ), mode selection (MS) and filtering (F).

[0048] An example of the decoding process is shown in Fig. 4b. Fig. 4b shows the predicted representation (P ’n ), the reconstruction prediction error signal (D ’n ), preliminary reconstructed image (I ’n ), the final reconstructed image (R ’n ), inverse transformation (T ―1 ), inverse quantization (Q -1 ), Entropy Decoding (E -1 ), reference frame memory (RFM), prediction (either inter or intra) (P), and filtering (F).

[0049] Many hybrid video encoders encode video information in two phases. First, pixel values ​​of a particular picture area (or "block") are predicted, for example by motion compensation means (finding and pointing to an area in one of the previously encoded video frames that closely corresponds to the block to be encoded) or by spatial means (using pixel values ​​around the block to be encoded in a specified manner). Then the prediction error, i.e., the difference between the predicted block of pixels and the original block of pixels, is encoded. This is typically done by transforming the pixel value differences using a specified transform (e.g., the discrete cosine transform (DCT) or a variant thereof), quantizing the coefficients, and entropy coding the quantized coefficients. By varying the fidelity of the quantization process, the encoder can control the balance between the accuracy of the pixel representation (picture quality) and the size of the resulting encoded video representation (file size or transmission bit rate). Video codecs may also provide transform skip modes that the encoder can choose to use. In transform skip mode, the prediction error is coded in the sample domain, for example, by deriving sample-wise difference values ​​for certain adjacent samples and coding the sample-wise difference values ​​with an entropy coder.

[0050] Entropy coding / decoding can be performed in many ways. For example, context-based coding / decoding can be applied, where both the encoder and the decoder modify the context state of the coding parameters based on previously coded / decoded coding parameters. The context-based coding can be, for example, Context-Adaptive Binary Arithmetic Coding (CABAC) or Context-Based Variable Length Coding (CAVLC) or any similar entropy coding. Entropy coding / decoding can alternatively or additionally be performed using a variable length coding scheme such as Huffman coding / decoding or Exponential-Golomb coding / decoding. The decoding of coding parameters from an entropy coded bitstream or codeword can be called parsing.

[0051] A phrase following a bitstream (e.g., points according to a bitstream) can be defined to indicate out-of-band transmission, signaling, or storage in a manner in which the out-of-band data is associated with the bitstream. A phrase decode following a bitstream, etc. can indicate decoding of referenced out-of-band data (which can be obtained from out-of-band transmission, signaling, or storage) associated with the bitstream. For example, a point following a bitstream can point to metadata of a container file that encapsulates the bitstream.

[0052] The H.264 / AVC standard was developed by the Joint Working Group (JVT) of the Video Coding Experts Group (VCEG) of the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) and the Moving Picture Experts Group (MPEG) of the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC). The H.264 / AVC standard was published by both parent standards organizations and is referred to as ITU-T Recommendation H.264 and ISO / IEC International Standard 14496-10, also known as MPEG-4 Part 10 Advanced Video Coding (AVC). There have been several versions of the H.264 / AVC standard that have integrated new extensions and features into the specification. These extensions include Scalable Video Coding (SVC) and Multiview Video Coding (MVC).

[0053] Version 1 of the High-Definition Video Coding (H.265 / HEVC aka HEVC) standard was developed by the Joint Collaborative Team - Video Coding (JCT-VC) of VCEG and MPEG. The standard was published by both parent standards organizations and is called ITU-T Recommendation H.265 and ISO / IEC International Standard 23008-2, also known as MPEG-H Part 2 High-Definition Video Coding (HEVC). Later versions of H.265 / HEVC include scalable, multiview, fidelity range, three-dimensional, and screen content coding extensions, which can be abbreviated as SHVC, MV-HEVC, REXT, 3D-HEVC, and SCC, respectively.

[0054] Versatile Video Coding (VVC) (MPEG-I Part 3), also known as ITU-T H.266, is a video compression standard developed by the Joint Working Group (JVET) of the Moving Image Experts Group (formally ISO / IEC JTC SC29 WG11) and the Video Coding Experts Group (VCEG) of the International Telecommunications Union (ITU), and is the successor to HEVC / H.265.

[0055] Some important definitions, bitstreams and coding structures, and concepts of H.264 / AVC and HEVC are presented in this paragraph as examples of video encoders, decoders, encoding methods, decoding methods, and bitstream structures, and embodiments can be implemented. Some important definitions, bitstreams and coding structures, and concepts of H.264 / AVC are the same as those of HEVC, and therefore they are described together below. Although the aspects of the present invention are not limited to H.264 / AVC or HEVC, the description part is presented for one possible basis in addition that the present invention can be partially or completely realized.

[0056] Like many earlier video coding standards, bitstream syntax and semantics as well as the decoding process for error-free bitstreams are specified in H.264 / AVC and HEVC. The encoding process is not specified, but the encoder must produce a conforming bitstream. Conformance between the bitstream and the decoder can be verified by a hypothetical reference decoder (HRD). The standard includes coding tools that help deal with transmission errors and losses, but the use of the tools in encoding is optional, and the decoding process is not prescribed for erroneous bitstreams.

[0057] The elementary units of the input to an H.264 / AVC or HEVC encoder and the output of an H.264 / AVC or HEVC decoder are pictures. A picture provided as an input to an encoder can be called a source picture, and a picture decoded by a decoder can be called a decoded picture.

[0058] The source and decoded pictures are each composed of one or more sample arrays, such as one of the following sets of sample arrays: -Luma(Y) only (monochrome). -Luma and two chromas (YCbCr or YCgCo). -Green, Blue and Red (GBR, also known as RGB). - Arrays representing other unspecified monochrome or tristimulus color sampling (eg, YZX, also known as XYZ).

[0059] In H.264 / AVC and HEVC, a picture can be either a frame or a field. A frame contains a matrix of luma samples and possibly corresponding chroma samples. A field is a set of alternate sample rows of a frame and can be used as an encoder input if the source signal is interlaced. The chroma sample array can be absent (so monochrome sampling can be used) or can be subsampled when compared to the luma sample array. The chroma format can be summarized as follows: -In monochrome sampling, there is only one sample array, which can be nominally thought of as the luma array. In -4:2:0 sampling, each of the two chroma arrays has 1 / 2 the height and 1 / 2 the width of the luma array. In -4:2:2 sampling, each of the two chroma arrays has the same height as the luma array and 1 / 2 the width of the luma array. In 4:4:4 sampling, when separate color planes are not used, each of the two chroma arrays has the same height and the same width as the luma array.

[0060] In H.264 / AVC and HEVC, the sample arrays can be encoded into the bitstream as separate color planes and each encoded color plane can be decoded separately from the bitstream. When separate color planes are used, each of these color planes is processed separately (by the encoder and / or decoder) as a picture by monochrome sampling.

[0061] Partitioning can be defined as the division of a set into subsets such that each element of the set is in a correct subset of the subsets.

[0062] The following representations may be used when describing HEVC encoding and / or decoding operations: A coding block may be defined as an NxN sample book for some value of N, such that the division of a coding tree block into coding blocks is a partition. A coding tree block (CTB) may be defined as an NxN sample block for some value of N, such that the division of components into coding tree blocks is a partition. A coding tree unit (CTU) may be defined as a coding tree block of luma samples, two corresponding coding tree blocks of chroma samples for a picture with three sample arrays, or a coding tree block of samples for a picture that is coded using three separate color planes and syntax structures used to code a monochrome picture or sample. A coding unit (CU) may be defined as a coding block of luma samples, two corresponding coding tree blocks of chroma samples for a picture with three sample arrays, or a coding block of samples for a picture that is coded using three separate color planes and syntax structures used to code a monochrome picture or sample. A CU with the largest allowed size can be named an LCU (largest coding unit) or a coding tree unit (CTU), and a video picture is divided into non-overlapping LCUs.

[0063] A CU consists of one or more Prediction Units (PUs) that define the prediction process for samples in the CU and one or more Transform Units (TUs) that define the prediction error coding process for samples of the CU. In general, a CU consists of a square block of samples with a size selectable from a predefined set of possible CU sizes. Each PU and TU is further divided into smaller PUs and TUs to increase the granularity of the prediction and prediction error coding processes, respectively. Each PU has prediction information associated with it that defines the type of prediction to be applied to pixels in this PU (e.g., motion vector information for inter-predicted PUs and intra prediction direction information for intra-predicted PUs).

[0064] Each TU may have associated therewith information (e.g., including DCT coefficient information) describing the prediction error decoding process of the samples within the TU. Typically, whether prediction error coding is applied to each CU is signaled at the CU level. If there is no prediction error residual associated with a CU, the CU is considered to have no TU. The division of an image into CUs, and the division of CUs into PUs and TUs, is typically signaled in the bitstream to allow a decoder to reproduce the intended structure of these units.

[0065] In HEVC, a picture can be partitioned into tiles, which are rectangular and contain an integer number of LCUs. In HEVC, the partition into tiles forms a regular grid, with the height and width of tiles differing from each other by at most one LCU. In HEVC, a slice is defined as an integer number of coding tree units contained in one independent slice segment and all the next dependent slice segments (if any) preceding the next independent slice segment (if any) in the same access unit. In HEVC, a slice segment is defined as an integer number of coding tree units that are consecutively ordered in the tile traversal and contained in a single NAL unit. The division of each picture into slice segments is a partitioning. In HEVC, an independent slice segment is defined as a slice segment in which the values ​​of syntax elements in the slice segment header are not inferred from the values ​​of the preceding slice segments, and a dependent slice segment is defined as a slice segment in which the values ​​of some syntax elements in the slice segment header are inferred from the values ​​of the preceding independent slice segments in the decoding order. In HEVC, a slice header is defined to be a slice segment header of an independent slice segment that is either the current slice segment or an independent slice segment preceding the current dependent slice segment, and a slice segment header is defined to be a part of a coded slice segment that contains data elements for the first or all coding tree units represented in the slice segment. If tiles are not used, CUs are scanned in the raster scan order of LCUs within a tile or picture. Within an LCU, CUs have a specific scan order.

[0066] The decoder reconstructs the output video by applying prediction means similar to the encoder (using motion or spatial information generated by the encoder and stored in the compressed representation) and prediction error decoding (the inverse operation of prediction error encoding, which recovers the quantized prediction error signal in the spatial pixel domain) to form a predicted representation of a pixel block. After applying the prediction and prediction error decoding means, the decoder sums the prediction and prediction error signals (pixel values) to form an output video frame. The decoder (and encoder) may also apply additional filtering means to improve the quality of the output video before passing it for display and / or storing it as a prediction reference for subsequent frames in the video sequence.

[0067] The filtering may include, for example, one or more of the following: deblocking, sample adaptive offset (SAO), and / or adaptive loop filtering (ALF). H.264 / AVC includes deblocking, while HEVC includes both deblocking and SAO.

[0068] In a typical video codec, the motion information is represented by a motion vector associated with each motion compensated image block, such as a prediction unit. Each of these motion vectors represents the displacement of an image block of a picture to be coded (at the encoder side) or decoded (at the decoder side) and a prediction source block in one of the previously coded or decoded pictures. To represent the motion vectors efficiently, the motion vectors are typically differentially coded with respect to a block-specific predicted motion vector. In a typical video codec, the predicted motion vector is generated in a predefined manner, for example, by calculating the median of the coded or decoded motion vectors of neighboring blocks. Another way to generate a motion vector prediction is to generate a list of candidate predictions from neighboring and / or co-located blocks of a temporal reference picture and signal the selected candidate as the motion vector predictor. In addition to predicting the motion vector value, the reference picture used for the motion compensated prediction can be predicted, and this prediction information can be represented, for example, by a reference index of a previously coded / decoded picture. The reference index is typically predicted from neighboring and / or co-located blocks of a temporal reference picture. Furthermore, common high-efficiency video codecs utilize an additional motion information encoding / decoding mechanism, often called merging / merge mode, in which all of the motion field information, including the motion vectors and corresponding reference picture indexes of each available reference picture list, is predicted and used without any modification / correction. Similarly, the prediction of the motion field information is performed using the motion field information of neighboring and / or co-located blocks of the temporal reference picture, and the motion field information to be used is signaled between a list of motion field lists filled with the motion field information of the available neighboring / co-located blocks.

[0069] In common video codecs, the prediction residual after motion compensation is first transformed by a transform kernel (such as DCT) and then encoded, because there is often some correction between the residuals, and the transform can help reduce this correction and provide more efficient encoding.

[0070] Video coding standards and specifications allow encoders to divide coded pictures into coded slices or the like. In-picture prediction is generally disabled across slice boundaries. Thus, slices can be considered as a way of dividing coded pictures into independently decodable pieces. In H.264 / AVC and HEVC, in-picture prediction can be disabled across slice boundaries. Thus, slices can be considered as a way of dividing coded pictures into independently decodable pieces, and thus slices are often considered elementary units for transmission. Often, encoders can indicate in the bitstream the type of in-picture prediction that is turned off across slice boundaries, and the decoder operation takes this information into account, for example, when concluding which prediction sources are available. For example, if neighboring CUs are in different slices, samples from the neighboring CUs are considered unavailable for intra prediction.

[0071] The elementary units at the output of an H.264 / AVC or HEVC encoder and at the input of an H.264 / AVC or HEVC decoder, respectively, are network abstraction layer (NAL) units. For transport over a packet-oriented network or storage into a structured file, the NAL units can be encapsulated in packets or similar structures. A byte stream format is specified in H.264 / AVC and HEVC for transmission or storage environments that do not provide a framing structure. The byte stream format separates NAL units from each other by appending a start code before each NAL unit. To avoid false detection of NAL unit boundaries, the encoder performs a byte-oriented start code emulation blocking algorithm and adds an emulation blocking byte to the NAL unit payload if a start code occurs otherwise. To allow for direct gateway operation between packet-oriented and stream-oriented systems, start code emulation blocking can always be performed regardless of whether the byte stream format is used. A NAL unit can be defined as a byte containing a syntax structure containing an indication of the type of data that follows and data in the form of RBSPs interspersed where necessary by emulation blocking bytes. A Raw Byte Sequence Payload (RBSP) can be defined as a syntax structure that contains an integer number of bytes that are encapsulated in a NAL unit. An RBSP is either empty or has the form of a string of data bits that contains a syntax element followed by an RBSP stop bit and followed by zero or more subsequent bits that are equal to 0.

[0072] A NAL unit consists of a header and a payload. In H.264 / AVC and HEVC, the NAL unit header indicates the type of the NAL unit.

[0073] In HEVC, a two-byte NAL unit header is used for all indicated NAL unit types. The NAL unit header contains one reserved bit, a six-bit NAL unit type indication, a three-bit nuh_temporal_id_plus1 indication for the temporal level (which can be required to be greater than or equal to one), and a six-bit nuh_layer_id syntax element. The temporal_id_plus1 syntax element can be considered as a temporal identifier for the NAL unit, and a zero-based TemporalID variable can be derived as follows: TemporalId=temporal_id_plus1-1. The abbreviation TID can be used synonymously with the TemporalId variable. A TemporalId equal to 0 corresponds to the lowest temporal level. The value of temporal_id_plus1 must be non-zero to prevent start code emulation involving two NAL unit header bytes. A bitstream generated by excluding all VCL NAL units with a TemporalId greater than or equal to the selected value and including all other VCL NAL units will remain conformant. As a result, a picture with TemporalId equal to tid_value does not use any picture with TemporalId greater than tid_value as an inter-prediction reference. A sub-layer or temporal sub-layer can be defined to be a temporal scalable layer (or temporal layer, TL) consisting of non-VCL NAL units associated with VCL NAL units having a particular value of the TemporalId variable. nuh_layer_id can be understood as a scalability layer identifier.

[0074] NAL units can be categorized into video coding layer (VCL) NAL units and non-VCL NAL units. A VCL NAL unit is typically a coded slice NAL unit. In HEVC, a VCL NAL unit contains a syntax element that represents one or more CUs.

[0075] A non-VCL NAL unit may be, for example, one of the following types: sequence parameter set, picture parameter set, supplemental enhancement information (SEI) NAL unit, access unit delimiter, end of sequence NAL unit, end of bitstream NAL unit, or filler data NAL unit. While a parameter set may be necessary for the reconstruction of a decoded picture, many of the other non-VCL NAL units are not necessary for the reconstruction of decoded sample values.

[0076] Parameters that remain unchanged throughout a coded video sequence may be included in a sequence parameter set. In addition to parameters that may be required by the decoding process, a sequence parameter set may optionally contain video utility information (VUI) including parameters that may be important for buffering, picture output timing, rendering, and resource reservation. In HEVC, a sequence parameter set RBSP contains parameters that may be referenced by one or more picture parameter sets RBSP or one or more SEI NAL units that contain a buffering period SEI message. A picture parameter set contains parameters that may not change for several coded pictures. A picture parameter set RBSP may contain parameters that may be referenced by a coded slice NAL unit of one or more coded pictures.

[0077] In HEVC, a video parameter set (VPS) can be defined as a syntax structure that contains zero or more syntax elements that apply to all coded video sequences, as determined by the content of syntax elements found in an SPS referenced by syntax elements found in a PPS referenced by syntax elements found in each slice segment header.

[0078] A video parameter set RBSP may contain parameters that may be referenced by one or more sequence parameter sets RBSP.

[0079] The relationship and hierarchy between video parameter sets (VPS), sequence parameter sets (SPS), and picture parameter sets (PPS) can be described as follows: VPS exists one level above SPS in the parameter set hierarchy and in the context of scalability and / or 3D video. VPS may contain parameters that are common to all slices across all (scalability or view) layers in all coded video sequences. SPS contains parameters that are common to all slices in a particular (scalability or view) layer in the entire coded video sequence and may be shared by multiple (scalability or view) layers. PPS contains parameters that are common to all slices in a particular layer representation (one scalability or view layer representation of one access unit) and may be shared by all slices in multiple layer representations.

[0080] The VPS can provide information about layer dependencies in the bitstream, as well as many other information that are applicable to all slices in all (scalability or view) layers in the entire coded video sequence. The VPS can be thought of as including two parts, a base VPS and a VPS extension part, where the VPS extension part can be optionally present.

[0081] The out-of-band transmission, signaling, or storage may additionally or alternatively be used for purposes other than resistance to transmission errors, such as ease of access or session negotiation. For example, a sample entry of a track of a file conforming to the ISO Base Media File Format may include a parameter set, and the encoded data of the bitstream is stored elsewhere in the file or in a separate file. The phrases according to the bitstream (e.g., pointing according to the bitstream) or according to an encoding unit of the bitstream (e.g., pointing according to an encoding tile) may be used in the claims and described embodiments to indicate out-of-band transmission, signaling, or storage in a manner in which the out-of-band data is associated with a bitstream or an encoding unit, respectively. The phrases according to the bitstream or decoding according to an encoding unit of the bitstream or the like may refer to decoding of the indicated out-of-band data (which may be obtained from the out-of-band transmission, signaling, or storage) associated with the bitstream or an encoding unit, respectively.

[0082] An SEI NAL unit can contain one or more SEI messages that are not necessary for decoding the output picture, but can assist in related processes such as picture output timing, rendering, error detection, error concealment, and resource reservation.

[0083] A coded picture is a coded representation of a picture.

[0084] In HEVC, a coded picture can be defined as a coded representation of a picture that includes all coding tree units of the picture. In HEVC, an access unit (AU) can be defined as a set of NAL units that are consecutive in decoding order and associated with each other according to a specified classification rule, and that includes at most one picture with any particular value of nuh_layer_id. In addition to including VCL NAL units of a coded picture, an access unit can also include non-VCL NAL units. The specified classification rule can, for example, associate pictures with the same output time or picture output count value to the same access unit.

[0085] A bitstream may be defined as a sequence of bits, in the form of a NAL unit stream or a byte stream forming a representation of a coded picture and associated data forming one or more coded video sequences. A first bitstream may be followed by a second bitstream of the same logical channel, such as in the same file or in the same connection of a communication protocol. An elementary stream (in the context of video coding) may be defined as a sequence of one or more bitstreams. The end of a first bitstream may be indicated by a specific NAL unit that may indicate an end of bitstream (EOB) NAL unit and is the last NAL unit of the bitstream. In HEVC and this current draft extension, the EOB NAL unit must have nuh_layer_id equal to 0.

[0086] In H.264 / AVC, a coded video sequence is defined to be a sequence of consecutive access units in decoding order from an IDR access unit, inclusively, to the next IDR access unit, exclusive, or to the end of the bitstream, if it occurs earlier.

[0087] In HEVC, a coded video sequence (CVS) can be defined as a sequence of access units consisting of, for example, an IRAP access unit, in decoding order, that has NoRaslOutputFlag equal to 1, followed by zero or more access units that are not IRAP access units with NoRaslOutputFlag equal to 1, including at most all subsequent access units, but not including any subsequent access units that are IRAP access units with NoRaslOutputFlag equal to 1. An IRAP access unit can be defined as an access unit whose base layer picture is an IRAP picture. The value of NoRastOutputFlag is equal to 1 for each IDR picture, each BLA picture, and each IRAP picture that is the first picture in this particular layer of the bitstream in decoding order and is the first IRAP picture following the end of the sequence NAL unit with the same value of nuh_layer_id in decoding order. There can be a means to provide the value of HandleCraAsBlaFlag to the decoder from an external entity, such as a player or receiver, that can control the decoder. HandleCraAsBlaFlag can be set to 1, for example, by a player that seeks a new position in the bitstream or tunes to a broadcast to begin decoding and then begins decoding from a CRA picture. When HandleCraAsBlaFlag is equal to 1 at a CRA picture, the CRA picture is treated and decoded as if a BLA picture were present.

[0088] In HEVC, a coded video sequence is indicated as ending when a specific NAL unit, which may additionally or alternatively be referred to (as specified above) as the end-of-sequence (EOS) NAL unit, appears in the bitstream and has a nuh_layer_id equal to 0.

[0089] Group of Pictures (GOP) and its properties can be defined as follows: A GOP can be decoded regardless of whether any previous pictures have been decoded. An open GOP is a group of pictures for which the pictures preceding the initial intra picture in output order cannot be correctly decoded when decoding starts from the initial intra picture of the open GOP. In other words, pictures of an open GOP can refer to pictures belonging to previous GOPs (in inter prediction). A specific NAL unit type, the CRA NAL unit type, can be used for this coded slice so that the HEVC decoder can recognize the intra picture that starts the open GOP. A closed GOP is a group of pictures for which all pictures can be correctly decoded when decoding starts from the initial intra picture of the closed GOP. In other words, pictures of a closed GOP do not refer to any pictures of previous GOPs. In H.264 / AVC and HEVC, a closed GOP can start with an IDR picture. In HEVC, a closed GOP can also start from a BLA_W_RADL or BLA_N_LP picture. Due to the high flexibility in the selection of reference pictures, an open GOP coding structure is potentially more efficient in compression compared to a closed GOP coding structure.

[0090] A decoded picture buffer (DPB) can be used in an encoder and / or a decoder. There are two reasons to buffer decoded pictures, such as reference for inter-prediction and reordering decoded pictures into output order. Since H.264 / AVC and HEVC provide a lot of flexibility for both reference picture marking and output reordering, separate buffers for reference picture buffering and output picture buffering may waste memory resources. Therefore, the DPB can include a unified decoded picture buffering process for reference pictures and output reordering. Decoded pictures can be removed from the DPB when they are not used as references and are not needed for output.

[0091] In many coding modes of H.264 / AVC and HEVC, the reference pictures for inter prediction are indicated by the index of the reference picture list. The index can be coded by variable length coding, which usually makes the small index have a short value of the corresponding syntax element. In H.264 / AVC and HEVC, two reference picture lists (reference picture list 0 and reference picture list 1) are generated for each bi-predicted (B) slice, and one reference picture list (reference picture list 0) is formed for each inter-coded (P) slice.

[0092] Many encoding standards, including H.264 / AVC and HEVC, may have a decoding process that derives a reference picture index of a reference picture list that can be used to indicate one of multiple reference pictures used for inter prediction of a particular block. The reference picture index may be coded into the bitstream by the encoder in some inter coding modes, or may be derived (by the encoder and decoder), for example, using neighboring blocks in some other inter coding modes.

[0093] The motion parameter types or motion information may include, but are not limited to, one or more of the following types: - an indication of the prediction type (e.g., intra-prediction, uni-prediction, bi-prediction) and / or the number of reference pictures; - an indication of a reference picture type such as inter (aka temporal) prediction, inter-layer prediction, inter-view prediction, view synthesis prediction (VSP), and inter-component prediction (which may be indicated per reference picture and / or per prediction type, and in some embodiments inter-view and view synthesis prediction may be considered together as one prediction direction) and / or short-term reference picture and / or long-term reference picture and / or inter-layer reference picture (which may be indicated per reference picture, for example); a reference index of a reference picture list and / or any other identifier of the reference picture (e.g. an index that can be indicated per reference picture, and the type of which can depend on the prediction direction and / or the reference picture type, and an index that can be accompanied by other relevant pieces of information such as a reference picture list or similar identifier to which the reference index applies), - horizontal motion vector components (e.g., components that can be indicated per prediction block or per reference index, etc.); - vertical motion vector component (e.g., a component that can be indicated per prediction block or per reference index, etc.); one or more parameters, such as a picture order count difference and / or a relative camera separation between the picture containing or associated with the motion parameter and its reference picture, that can be used for scaling the horizontal and / or vertical motion vector components in one or more motion vector prediction processes (one or more parameters can be indicated, for example, for each reference picture or each reference index, etc.), - coordinates of the block given by the motion parameters and / or the motion information, e.g. the coordinates of the top left sample of the block in luma sample units, - The motion parameters and / or extent of the block that the motion information gives (e.g. width and height).

[0094] Compared to previous video coding standards, the versatile video codec (H.266 / VVC) introduces several new coding tools, such as: Intra prediction -67 Intra Mode with Wide Angle Mode Extension -Block size and mode dependent 4-tap interpolation filter - Position-dependent intra-prediction combining (PDPC) -Cross-Component Linear Model Intra-Prediction (CCLM) -Multiple reference line intra prediction -Intra subpartition -Weighted intra prediction with matrix multiplication · Inter-picture prediction -Block motion copy with spatial, temporal, history-based and pairwise average merging candidates - Affine motion inter prediction - Sub-block based temporal motion vector prediction -Adaptive motion vector resolution - 8x8 block-based motion compression for temporal motion prediction - High definition (1 / 16 pel) motion vector storage and motion compensation with 8-tap interpolation filters for the luma component and 4-tap interpolation filters for the chroma component -Triangular partition - Intra and inter combined prediction -Merge with MVD (MMVD) -Symmetric MVD coding - Bidirectional Optical Flow - Decoder side motion vector adjustment - Bi-prediction with CU level weights Transformation, quantization and coefficient coding -Multiple primary transform options with DCT2, DCT7 and DCT8 -Secondary transformation of low frequency zone -Sub-block transformation of inter prediction residuals Dependent quantization with max QP increased from -51 to 63 - Transform coefficient coding with sign data hiding - Transform skip residual coding ·Entropy coding -Arithmetic coding engine with adaptive double window probability update In-loop filter -In-loop reshaping - Deblocking filter with powerful length filter -Sample Adaptive Offset -Adaptive Loop Filter Screen content encoding - Current picture reference with reference area constraints 360° video encoding -Horizontal wraparound motion compensation High-level syntax and parallel processing Reference picture management via direct reference picture list signaling - Tile Groups with Rectangle Tile Groups

[0095] Partitioning in VVC is performed similarly to HEVC, i.e., each picture is divided into coding tree units (CTUs). Pictures can also be subdivided into slices, tiles, bricks, and subpictures. CTUs can be divided into small CUs using a quadtree structure. Each CU can be split using nested multi-type trees, including quadtrees, binary trees, and binary splits. However, there are certain rules for inferring partitions to picture boundaries, and redundant split patterns are rejected in nested multi-type partitions.

[0096] In the new coding tools listed above, a cross-component linear model (CCLM) prediction mode is used in VVC to reduce cross-component redundancy, where a chroma sample is predicted based on the reconstructed luma sample of the same CU by using a linear model as follows: JPEG2025511015000002.jpg6150, where JPEG2025511015000003.jpg6150 represents the predicted chroma samples of a CU, JPEG2025511015000004.jpg6150 represents the downsampled reconstructed luma samples of the same CU.

[0097] Alternatively, the following formula can be used for CCLM: JPEG2025511015000005.jpg6150Here the >> operation indicates that the bits are shifted to the right by the value k.

[0098] The CCLM parameters (α and β) are derived by up to four neighboring chroma samples and their corresponding downsampled luma samples. Given that the current chroma block dimension is W×H, W′ and H′ are set as follows: -When LM mode is applied, W'=W, H'=H - When LM-A mode is applied, W'=W+H -When LM-L mode is applied, H'=H+W

[0099] Here, LM-A mode refers to linear model_top, where only the top template (i.e., sample values ​​from neighboring positions above the CU) are used to calculate the linear model coefficients. To obtain more samples, the top template is extended to (W+H). LM-L mode refers to linear model_left, where only the left template (i.e., sample values ​​from neighboring positions to the left of the CU) are used to calculate the linear model coefficients. To obtain more samples, the left template is extended to (H+W). For non-square blocks, the top template is extended to W+W and the left template is extended to H+H.

[0100] The top neighbouring positions are denoted as S[0, -1]…S[W' - 1, -1] and the left neighbouring positions are denoted as S[-1, 0]…S[-1, H' - 1]. Then the four samples are selected as follows: -When LM mode is applied and both upper and left neighbor samples are available, S[W' / 4, -1], S[ 3 * W' / 4, -1], S[ -1, H' / 4], S[ -1, 3 * H' / 4] -If LM-A mode is applied or only upper neighbor samples are available, S[ W' / 8, -1 ], S[ 3 * W' / 8, -1 ], S[ 5 * W' / 8, -1 ], S[ 7 * W' / 8, -1 ] If -LM_L mode is applied or only left neighbor samples are available, S[ -1, H' / 8 ], S[ -1, 3 * H' / 8 ], S[ -1, 5 * H' / 8 ], S[ -1, 7 * H' / 8 ]

[0101] The four neighboring luma samples of the selected position are downsampled four times and compared to obtain two small values, x0A and x1A, and two large values, x0B and x1B. The corresponding chroma sample values ​​are denoted as y0A, y1A, y0B, and y1B. Then xA, xB, yA, and yB are derived as follows: Xa=(x0A + x1A +1)>>1; Xb=(x0B + x1B +1)>>1; Ya=(y0A + y1A +1)>>1; Yb=(y0B + y1B +1)>>1 (Equation 2)

[0102] Finally, the linear model parameters α and β are obtained according to the following equations: JPEG2025511015000006.jpg10150 JPEG2025511015000007.jpg6150

[0103] FIG. 5 shows an example of the position of the left and top samples and the samples of the current block involved in the CCLM mode.

[0104] The division operation to calculate the parameter α is performed by a look-up table. To reduce the memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed in exponential notation. For example, diff is approximated by a 4-bit significant part and an exponent. As a result, the table of 1 / diff is reduced to 16 elements for 16 values ​​of the significant part as follows: DivTable [ ]={ 0, 7, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0} (Formula 5)

[0105] This offers the advantage of reducing both the computational complexity as well as the memory size required to store the required tables.

[0106] In order to match the chroma sample positions for 4:2:0 video sequences, two types of downsampling filters are applied to the luma samples to achieve a 2:1 downsampling ratio in both horizontal and vertical directions. The selection of the downsampling filter is indicated by the SPS level flag. The two downsampling filters are as follows, corresponding to "Type-0" and "Type-2" content, respectively: JPEG2025511015000008.jpg42150 JPEG2025511015000009.jpg11150

[0107] Note that only one luma line (a common line buffer in intra prediction) is used to generate downsampled luma samples when the upper reference line is on a CTU boundary.

[0108] This parameter computation is performed as part of the decoding process and not as an encoder search operation, and as a result no syntax is used to transmit the α and β values ​​to the decoder.

[0109] In chroma intra mode coding, a total of eight intra modes are possible for chroma intra mode coding. These modes include five traditional intra modes and three cross-component linear model modes (CCLM, LM_A, and LM_L). The chroma model signaling and derivation process is shown in Table 1. The chroma mode coding directly depends on the intra prediction mode of the corresponding luma block. Since separate block partition structures for luma and chroma components are supported in I slice, one chroma block can support multiple luma blocks. Therefore, in chroma DM mode, the intra prediction mode of the corresponding luma block that covers the center position of the current chroma block is directly inherited. TIFF2025511015000010.tif62142 Table 1

[0110] A single binarization table is used regardless of the value of sps_cclm_enabled_flag as shown in Table 2. TIFF2025511015000011.tif67134 Table 2

[0111] In Table 2, the first bin indicates whether it is regular (0) or LM mode (1). If it is LM mode, then the next bin indicates whether it is LM_CHROMA (0). If it is not LM_CHROMA, then one bin indicates whether it is LIM_L (0) or LM_A (1). In this case, when sps_cclm_enabled_flag is 0, the first bin of the binarization table of the corresponding intra_chroma_pred_mode can be discarded during entropy encoding. Or, in other words, the first bin is assumed to be 0 and therefore is not encoded. This single binarization table is used for both sps_cclm_enabled_flag equal to 0 and 1 cases. The first two bins of Table 2 are context coded by its own context model, and the remaining bins are bypass coded.

[0112] In addition, to reduce luma-chroma latency in the dual tree, when a 64x64 luma coding tree node is partitioned by not split (and intra sub-partition (ISP) is not used for 64x64 CUs) or QT, the chroma CUs of the 32x32 / 32x16 chroma coding tree nodes can use CCLM in the following manner: - If a 32x32 chroma node is not split or is not a partition QT split, all chroma CUs of the 32x32 node can use CCLM. - If a 32x32 chroma node is partitioned by horizontal BT and a 32x16 child node is not split or uses vertical BT split, all chroma CUs of the 32x16 chroma node can use CCLM.

[0113] In all other luma and chroma coding tree split conditions, CCLM is not allowed for the chroma CU.

[0114] Multi-Model LM (MMLM)

[0115] CCLM included in VVC is extended by adding three multi-model LM (MMLM) modes. In each MMLM mode, reconstructed neighboring samples are classified into two classes using a threshold that is the average value of the luma reconstructed neighboring samples. A linear model for each class is derived using the least mean square (LMS) method. In CCLM mode, the LMS method is also used to derive the linear model. Figures 6a and 6b show two luma-to-chroma models obtained for 17 luma (Y) thresholds in the sample domain and spatial domain, respectively. Each luma-to-chroma model has its own linear model parameters α and β. As can be seen in Figure 6b, each luma-to-chroma model corresponds to a spatial segmentation of the content (i.e., they correspond to different objects or textures in the scene).

[0116] Multi-Reference Line (MRL) Intra Prediction

[0117] Multi-reference line (MRL) intra prediction uses many reference lines for intra prediction. In Figure 7, an example of four reference lines is shown, where samples of segments A and F are not fetched from reconstructed neighboring samples, but are padded with the nearest samples from segments B and E, respectively. HEVC intra picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, two additional lines (reference line 1 and reference line 3) are used.

[0118] The index (mrl_idx) of the selected reference line is signaled and used to generate the intra predictor. For reference line idx greater than 0, include only additional reference line modes in the MPM list and signal only the mpm index without leaving any modes. The reference line index is signaled before the intra prediction modes, and planar modes are excluded from the intra prediction modes if a non-zero reference line index is signaled.

[0119] MRL is disabled for the first line of a block inside a CTU to avoid using extended reference samples outside the current CTU line. PDPC is also disabled when additional lines are used. In MRL mode, the derivation of DC values ​​in DC intra prediction modes for non-zero reference line indexes is aligned to the value of reference line index 0. MRL requires storage of three neighboring luma reference lines with the CTU to generate the prediction. The CCLM tool also requires three neighboring luma reference lines for this downsampling filter. The definition of MLR to use the same three lines is aligned to CCLM to reduce the decoder storage requirements.

[0120] Intra-Subpartition (ISP)

[0121] Intra Sub-Partition (ISP) divides a luma intra predicted block vertically or horizontally into 2 or 4 sub-partitions depending on the block size. For example, the minimum block size for ISP is 4x8 (or 8x4). If the block size is larger than 4x8 (or 8x4), the corresponding block is divided by 4 sub-partitions. M×128 JPEG2025511015000012.jpg6150 and 128xN Note that JPEG2025511015000013.jpg6170 ISP blocks may cause potential problems with 64x64 VDPU. For example, an Mx128 CU in the single tree case has an Mx128 luma TB and two corresponding M / 2x64 chroma TBs. If the CU uses ISP, then the luma TB is split into four Mx32 TBs (only horizontal splits are possible), each of which is smaller than a 64x64 block. However, in the current design of ISP chroma, the block is not split. Thus, both chroma components have a size larger than a 32x32 block. Similarly, a similar situation may occur with a 128xN CU using ISP. Thus, these two cases are also problematic in a 64x64 decoder pipeline. For this reason, the CU size that can use IDP is limited to a maximum of 64x64. All subpartitions satisfy the condition of having at least 16 samples.

[0122] Matrix Weighted Intra Prediction (MIP)

[0123] The matrix weighted intra prediction (MIP) method is a new intra prediction technique added to VVC. To predict samples of a rectangular block of width W and height H, matrix weighted intra prediction (MIP) takes as input one line of H reconstructed neighboring boundary samples on the left side of the block and one line of W reconstructed neighboring boundary samples above the block. If the reconstructed samples are not available, reconstructed samples are generated as is done in conventional intra prediction. The generation of the predicted signal is based on the following three steps, which are averaging, matrix vector multiplication and linear interpolation, as shown in Figure 8.

[0124] Inter Prediction in VVC

[0125] The merge list may include the following candidates: a) Spatial MVP from spatially neighboring CUs b) Time MVP from colocated CU c) History-based MVP from a FIFO table d) Pairwise Average MVP (using candidates already in the list) e) Zero MV.

[0126] The merge mode wide motion vector difference (MMVD) is for signaling MVD and resolution index after signaling merge candidates. In symmetric MVD, the motion information of list-1 is derived from the motion information of list-0 in the bi-prediction case.

[0127] In affine prediction, several motion vectors are indicated / signaled for different corners of a block and used to derive the motion vectors of sub-blocks. In affine merging, the affine motion information of a block is generated based on the regular or affine motion information of neighboring blocks.

[0128] In sub-block-based temporal motion vector prediction, the motion vectors of the sub-blocks of the current block are predicted from the appropriate sub-blocks of the reference frames as indicated by the motion vectors of spatial neighboring blocks (if available).

[0129] In adaptive motion vector resolution (AMVR), the precision of the MVD for each CU is signaled.

[0130] In bi-prediction with CU-level weights, the index indicates the weight value for the weighted average of the two predicted blocks. Bi-directional optical flow (BDOF) adjusts the motion vectors in the bi-prediction case. BDOF uses the signaled motion vectors to generate two predicted blocks. Then motion adjustments are calculated to minimize the error between the two predicted blocks using these gradient values. The final predicted block is refined using the motion adjustments and gradient values.

[0131] Therefore, it can be concluded that the objective of the linear model of the CCLM method used in VVC / H.266 is to determine the correlation of samples between two or more chroma channels (e.g., Cb and Cr) by deriving model parameters based on reconstructed samples in the neighborhood of a chroma block, collocated neighboring samples in a luma block, and reconstructed samples inside the collocated luma block.

[0132] However, the first-order linear model of CCLM is not adequate to model spatial correlations between luma and chroma samples (such as textures or surfaces).

[0133] Here we introduce an improved method to achieve cross-component prediction that is better suited to model the spatial correlation between luma and chroma samples.

[0134] 10. A method according to the embodiment shown in FIG. 9, comprising: receiving an image block unit of a frame (900), the image block unit including samples of color channels including at least one chrominance channel and one luminance channel; The method includes a step of reconstructing samples of a luminance channel of an image block unit (902); a step of determining a reference area for predicting samples of at least one color channel of the image block unit (904), where the reference area includes one or more of reference samples in a neighboring block of a current color channel / frame, in a neighboring block of a collocated block of a reference color channel / frame, and / or inside a collocated block of a reference color channel / frame (904); a step of determining filter coefficients of a filter for prediction based on the reference samples and a shape of the filter (906); and a step of predicting samples of at least one color channel of the image block unit by performing a convolution between the samples of the luminance channel and the filter coefficients (908).

[0135] Thus, the method predicts chroma samples using a convolution between a set of input data and a derived filter kernel at the decoder side. The shape of the filter and the number of coefficients are agreed upon by both the encoder and the decoder. The input data to the convolution can be indicated in a variety of ways, for example, it can include luma samples co-located with chroma samples, nearby co-located luma samples, or any other configuration of luma samples. In the filter coefficient derivation, a co-located reference sample area containing the reconstructed luma and chroma samples is defined for both luma and chroma prediction, where the reference sample can encompass any chroma and luma samples that have been reconstructed by both the encoder and the decoder. Once the reference sample is determined, the filter coefficients are derived based on the reference sample and the shape of the filter kernel. Finally, the predicted chroma block can be calculated by convolving the luma sample with the filter kernel.

[0136] Thus, the method enables the use of 2D filter kernels, thereby enhancing the modeling of spatial correlations (such as textures or surfaces) between luma and chroma samples.

[0137] According to one embodiment, the method includes the step of downsampling the samples of the luminance channel to correspond to a sample size of the chrominance samples before determining the filter coefficients.

[0138] Figure 10 shows an example of a collocated reference sample area consisting of reconstructed luma and chroma samples defined for both luma and chroma, where commonly used 4:2:0 chroma downsampling is applied. For the purpose of determining filter coefficients for predicting chroma samples, at least the corresponding reference luma samples can be downsampled to the dimension of the chroma samples. It should be noted that the reference sample area of ​​a given block can be, for example, 6 lines above and to the left as shown in Figure 10, but any number of reference lines that can be realized by both the encoder and the decoder can be used.

[0139] According to one embodiment, determining the filter coefficients includes obtaining statistics from reference samples for a linear regression based on the shape of the filter, and applying the linear regression method to the obtained statistics.

[0140] Once the reference samples have been determined, the filter coefficients can be derived by scanning the luma and chroma samples of the reference area, collecting available statistics from the reference samples based on the filter shape, and solving for the filter coefficients by applying some linear regression method to the obtained statistics, where various types of linear regression tools can be used, such as ordinary least squares estimation, orthogonal matching pursuit, optimized orthogonal matching pursuit, ridge regression, or least absolute value shrinkage selection operator.

[0141] According to one embodiment, the statistics include an autocorrelation matrix and a cross-correlation vector. Thus, based on the reference samples and the shape of the filter kernel, the autocorrelation matrix and the cross-correlation vector can be defined, and the filter coefficients can be solved, for example, by minimizing the squared error (or any other measure) based on the autocorrelation matrix and the cross-correlation vector.

[0142] According to one embodiment, the shape of the filter is a one-dimensional (1D) vector of filter coefficients or a two-dimensional (2D) array of filter coefficients.

[0143] FIG. 11 shows various examples of filter kernel dimensions, which can be, for example, 1x3 (1D vertical), 3x1 (1D horizontal), 3x3, 7x7, or any dimension. The filter kernel can be formed as a cross or diamond as shown in FIG. 11, or any given shape by selecting only a subset of all possible kernel positions. When indicating samples within a filter kernel, the following notations are used: North (up), East (right), South (down), West (left), and Center use the letters N, E, S, W, C as shown in FIG. 11.

[0144] We define the (possibly downsampled) luma samples as a 2D array y(x,y) indexed using the horizontal x and vertical y coordinates. We also define the colocated chroma samples as a 2D array C(x,y) and the filter kernel (i.e., coefficients) as a 3x3 array F(i,j). Then, at the sample level, the convolution between Y and F can be defined as JPEG2025511015000014.jpg16150

[0145] According to one embodiment, the method includes the step of applying auxiliary parameters from outside the shape of the filter to the convolution.

[0146] Other parameters or data terms outside the filter kernel can be applied as additional terms of the linear combination. Such terms can be nonlinear terms such as bias terms, spatial coordinates, recursively reconstructed chroma samples, or collocated luma samples or square roots of collocated luma sample values ​​up to a power of two. In general, the nonlinear terms can be any nonlinear function applied to either the reconstructed samples or other available data. More than one nonlinear term can be used to capture various kinds of nonlinear dependencies between luma and chroma sample values.

[0147] The introduction of parameters or data terms outside the filter kernel, especially non-linear terms, further extends the modeling of spatial correlation between luma and chroma samples.

[0148] When using auxiliary parameters such as nonlinear square root terms, the above convolution becomes: JPEG2025511015000015.jpg16150, where JPEG2025511015000016.jpg6150 are filter coefficients that lie outside the 2D filter kernel, but were obtained as part of the system of linear equations used to solve for the 2D filter coefficients as described above.

[0149] Another example of using an auxiliary parameter is to add a bias term to the convolution. JPEG2025511015000017.jpg16150

[0150] As another example, the luma sample squared can be used instead of or in addition to the square root term as follows: JPEG2025511015000018.jpg16150

[0151] JPEG2025511015000019.jpg6150 and Both the luma and chroma components (i.e., the CCCM coefficients) are obtained using reconstructed samples from the nearby neighborhood of the chroma block. However, CCCM coefficients derived from more distant samples need to be considered when the closer neighborhood cannot account for the cross-component relationships between luma and chroma.

[0152] According to one embodiment, the method includes storing filter coefficients in a memory and receiving an indication from an encoder to reuse at least a subset of previously stored filter coefficients to predict samples of at least one color channel of a current image block.

[0153] Thus, the encoder and decoder can store the CCCM coefficients in memory, allowing the encoder to test all or some subset of the CCCM coefficients obtained so far. Once a suitable set of CCCM coefficients is found, an index identifying these coefficients is signaled to the decoder. This type of reuse of earlier CCCM coefficients can be referred to as history-based CCCM.

[0154] According to one embodiment, the method includes the steps of determining samples outside the reference area, obtaining statistics from the samples outside the reference area for linear regression based on the shape of the filter, obtaining filter coefficients by applying a linear regression method to the obtained statistics, and storing the filter coefficients in a memory.

[0155] To further increase the count of filter coefficient sets, the encoder and decoder can include coefficients from non-CCM blocks in memory by scanning the luma and chroma samples of a reference area to collect available statistics from the reference samples based on the filter shape, and solve the filter coefficients by applying a linear regression method to the obtained statistics, which can be applied to all or a partial subset of the non-CCCM chroma blocks. The resulting CCCM coefficients are stored in memory.

[0156] Also, an autocorrelation matrix and a cross-correlation vector can be defined based on the reference samples and the shape of the filter kernel, and the filter coefficients can be solved, for example, by minimizing the squared error (or any other measure) based on the autocorrelation matrix and the cross-correlation vector. Furthermore, samples outside the reference area can be downsampled to correspond to the sample size of the chrominance samples before determining the filter coefficients.

[0157] To further improve the diversity of the obtained history-based CCCM coefficients, the encoder and decoder can also similarly perform the above steps of downsampling, obtaining statistics, and obtaining filter coefficients for the CCCM blocks. The resulting CCCM coefficients can be used to overwrite earlier CCCM coefficients or can be stored as additional CCCM coefficients.

[0158] By merging the locally derived CCCM coefficients with the history-based CCCM, a more refined chroma prediction can be obtained. JPEG2025511015000021.jpg6150 and Call it JPEG2025511015000022.jpg6150. Merge weights JPEG2025511015000023.jpg6150 is 0< JPEG2025511015000024.jpg6150<1.0, and the weighted CCCM coefficients are JPEG2025511015000025.jpg6150 and JPEG2025511015000026.jpg6150. This weighting can also be applied after convolution of the reconstructed chroma samples if the history-based CCCM filter structure does not match the locally derived CCCM filter structure. JPEG2025511015000027.jpg6150 is searched at the encoder side and signaled to the decoder. Multiple history-based and / or locally derived CCCM coefficients are JPEG2025511015000028.jpg6150 can be merged by searching and signaling the weights.

[0159] Model parameters or filter coefficients JPEG2025511015000029.jpg6150 and When computing JPEG2025511015000030.jpg6150, different choices can be made depending on whether one operates in single-tree mode (with three color components represented by a single coding unit) or in dual-tree mode (with a separate coding unit for luma and separate coding units for the two chroma components).

[0160] In dual-tree modes, where luma samples are available for a spatial area that typically covers a complete coding tree unit consisting of multiple coding units, luma samples below (S) and to the right (E) of a chroma coding unit can be made available. In such cases, the luma samples can be advantageously used for both the filter coefficient derivation and the actual sample prediction process that maps luma sample values ​​to chroma sample values ​​by a convolution operation.

[0161] However, in the single-tree case, such luma samples may not be available and may not be used by the model creation or sample prediction operations. The luma samples of the current coding unit may be obtained from inside the current coding unit when collecting statistics of the border samples immediately above the current coding unit, for example, if the south (S) sample below the center (C) sample is included in the filter kernel for CCCM model generation. However, since this type of operation requires the luma samples of the current coding unit to be available when generating a CCCM model for the chrominance components, it is not required to consider the implementation parallel processing aspect, thus preventing luma sample decoding from being performed in parallel with the CCCM model creation. To avoid this, the use of luma samples inside the current coding unit is allowed for dual-tree operations and disallowed for single-tree operations. Or alternatively, it can be signaled in the video bitstream if luma samples within the current coding unit are used for CCCM model creation and the decoder can configure itself for parallel or sequential processing of luma and chroma.

[0162] According to one embodiment, the method includes replacing any missing downsampled luminance samples by zero-valued samples.

[0163] In both the CCCM model creation and CCCM sample prediction process, some of the possibly downsampled luma samples required by the convolution kernels may not be available. For example, some samples may be outside the picture boundary or may belong to areas that have not yet been decoded. In such cases, these samples can be generated in various ways. For example, unavailable samples can be replaced by the nearest available samples. Such a process is commonly called padding. In the CCCM model creation stage, depending on the method selected to derive the CCCM coefficients, it may be advantageous to use zero-valued samples for unavailable samples instead of traditional padding means. This becomes particularly advantageous when using autocorrelation matrix-based methods for coefficient estimation, since zero-valued inputs do not introduce updates to such matrices and thus eliminate the potential negative effects of sample replacement of reference area boundaries.

[0164] There can be additional bitstream signaling to update the CCCM model derived from the decoded samples. For example, the update term JPEG2025511015000031.jpg6150 can be signaled, and the decoder can perform additional adjustments to other coefficients based on the decoded update terms.

[0165] Various measures can be taken to balance the various types of computational requirements of a coding unit, for example, the number and location of reference samples used in the CCCM model creation process can be determined based on the size of the coding unit, the number and spatial location of coefficients of the CCCM model can be adjusted based on the size of the coding unit, or the number and spatial location of coefficients of the CCCM model can be adjusted based on the number of CCCM models required within one coding unit.

[0166] Similarly, various measures can be taken to balance the memory and memory bandwidth needs of the CCCM process. For example, in some circumstances it is advantageous to limit the reference lines used above the coding tree unit. In such cases, different amounts of reference lines can be used in the CCCM model creation depending on whether the predicted sample block is at or near the top boundary of the coding tree unit or further away from this boundary. As an example, the number of reference lines required above the coding tree unit can be limited to three, or can be limited to two, or can be limited to one line of samples.

[0167] According to one embodiment, the method includes determining samples that do not belong to any object or texture inside the image block unit, and removing the samples from the reference area.

[0168] There can be an outlier removal process applied to the luma and chroma reference area samples to determine samples that do not belong to the object or texture inside the block. Such a process can be based on the determination of certain statistics or characteristics from the co-located luma samples inside the block. For example, one or more of the luma samples of the co-located block with the minimum and maximum intensity values ​​can be used as a measure to remove the reference area luma and possibly corresponding chroma samples from the CCCM parameter derivation process. Alternatively, the minimum and maximum values ​​can be used to clip the reference area samples that exceed the intensity range of the samples inside the block. In another method, clustering and / or classification methods can be used on the co-located luma samples of the block to determine the characteristics of the samples, and then the reference area samples that match the determined characteristics of the co-located luma samples are used for the CCCM parameter derivation.

[0169] Instead of one CCCM model, multiple CCCM models can be derived for a single block of chrominance samples. In such cases, various types of conditions can be used to select the CCCM model to apply to each chroma sample. For example, the average luma value of the reference sample can be used as a threshold when there are two CCCM models for a particular chroma block, and if the colocated luma sample is smaller than the threshold, the first CCCM model can be selected for the prediction of the chroma sample, otherwise the second CCCM model can be used for this chroma sample. Also, various conditions can be used to allow a block to have multiple CCCM models, for example multiple models are only possible if the chroma block has at least an indicated number of samples or at least an indicated number of reference samples are used for this block. This type of selection can be advantageous in cases where small blocks cannot benefit significantly from multiple CCCM models, but there are additional signaling and computational requirements typically associated with multiple models, making it less advantageous relative to small blocks compared to large blocks.

[0170] In the following, embodiments are disclosed regarding different implementation options in which the decoder derives and / or applies different parameters for the prediction model.

[0171] According to one embodiment, a set of chroma samples is predicted using a filtering operation that takes as input at least one luma sample colocated with the chroma sample, another luma sample at a different position, and a term derived from the luma sample or multiple luma samples using a non-linear operation, and the filtering operation outputs a predicted chroma sample.

[0172] According to one embodiment, any number of CCCM filters can be applied when predicting the chroma samples, for example, the chroma samples can be divided into two or more classes with each class using its own CCCM coefficients, similar to multi-model CCLM.

[0173] According to one embodiment, the nonlinear term of the convolution is defined as a nonlinear function of any reconstructed or otherwise available data, such as reconstructed luma or chroma samples.

[0174] The non-linear function used in the non-linear term can be, for example, a square root, or any exponential function, a sine function, a combination of multiple non-linear functions (linear or non-linear), or any other function that can be computed in the decoder.

[0175] According to one embodiment, the final prediction of a block is achieved by combining two or more of the CCCM predictions with different nonlinear terms. For example, one prediction can be achieved by a CCCM model without a nonlinear term and one or more predictions with the same or different nonlinear terms, and then the final prediction is calculated by a weighted average value of the predictions. The weight values ​​of such operations can be fixed or signaled in the bitstream, or can be determined based on certain factors such as block size, CCCM coefficients of different models, neighboring block prediction modes, etc.

[0176] According to one embodiment, the decision to use the nonlinear term in the CCCM model is determined based on texture analysis mechanisms on the reference samples and / or luma samples inside the block. For example, the average value or DC value of the reference area and the colocated luma samples can be used as a factor to determine the use of the nonlinear term. If the average values ​​are too different from each other (e.g., based on a certain threshold), then the nonlinear term is not used or vice versa. In another example, the slope of the linear relationship between the reference luma and chroma samples is used to determine the use of the nonlinear term in the CCCM. In this case, a portion of the reference luma and chroma samples is used in a linear model (e.g., linear regression), and if the slope of the linear model is less than or greater than a threshold, the nonlinear term is used in the CCCM process.

[0177] According to one embodiment, the binary selection of the use of the non-linear term is signaled by the encoder to the decoder.

[0178] According to one embodiment, the choice of the selected non-linear function is signaled by the encoder to the decoder at any given precision.

[0179] According to one embodiment, the shape, size, width, height, or other data that determines the selection of the filter kernel is signaled by the encoder to the decoder using any given precision. The signaling can be done at different levels, such as CU, PU, ​​TU, CTU, slice, tile, sub-picture, picture or sequence level.

[0180] As mentioned above, the filter kernel dimensions can be 1D or 2D, ranging from 1x1 to any dimension that can be realized in the decoder, and the shape of the filter kernel can be, for example, a row or column, a cross or diamond, or any shape that can be realized under the dimensions of the filter kernel.

[0181] According to one embodiment, the filter shape may cover only a subset of the elements covered by the dimensions of the filter kernel.

[0182] According to one embodiment, the shape and / or size of the filter kernel may be adaptively determined based on certain factors such as the block size, the availability of reference samples, the position of the block in the frame, the coding mode of neighboring blocks, etc.

[0183] According to one embodiment, the filter consists of central (colocated), north (top), south (bottom), west (left), and east (right) luma samples, a bias term, and a nonlinear term derived from the central luma sample.

[0184] According to one embodiment, the filter consists of central (colocated), south (bottom), west (left), and east (right) luma samples, a bias term, and a nonlinear term derived from the central luma sample.

[0185] According to one embodiment, the filter consists of central (collocated), north (up), south (down), west (left), and east (right) luma samples, a bias term, and the square root of the central luma sample.

[0186] According to one embodiment, the filter consists of central (collocated), north (up), south (down), west (left), and east (right) luma samples, a bias term, and a squared central luma sample.

[0187] According to one embodiment, the filter consists of center (colocated), north (top), south (bottom), west (left), and east (right) luma samples, a bias term, and a squared center luma sample that is scaled to have the same dynamic range as the luma samples.

[0188] According to one embodiment, the binary selection of the use of history-based CCCM is signaled by the encoder to the decoder.

[0189] According to one embodiment, the index (or position) of the selected history-based CCCM coefficient is signaled by the encoder to the decoder with any given precision.

[0190] According to one embodiment, the binary selection of the use of weighted history-based CCCM is signaled by the encoder to the decoder.

[0191] According to one embodiment, the weights of the history-based CCCM are signaled by the encoder to the decoder with any given precision.

[0192] It should be noted that all of the above options for history-based CCCM (including weighted displacement forms) can be signaled to the decoder by the encoder separately for each chroma component.

[0193] According to one embodiment, the history-based CCCM coefficients are computed (and / or updated) at the CU level or at the CTU level.

[0194] According to one embodiment, the correct CCCM coefficients from multiple sets of coefficients from history-based CCCM are determined at the decoder side. This can be done, for example, by using available coefficients from the history and predicting one or more of the reference area samples, and calculating the cost (e.g., distortion or difference) with respect to the reconstructed reference samples. The set of coefficients that results in the minimum cost is then selected and used to predict the samples inside the block.

[0195] According to one embodiment, the weighted history-based CCCM can also be determined based on one or more of the reference sample areas with different historical CCCM coefficients and different weight values, and a similar scheme by predicting the best performing set of coefficients and weights that provides the minimum cost to be used in predicting the samples of the block.

[0196] According to one embodiment, the fixed luma downsampling can be skipped and instead the full luma grid is used to derive the CCCM coefficients, thus obtaining an explicit downsampling filter for processing in the downsampled chroma case.

[0197] According to one embodiment, the binary choice of skipping the fixed luma downsampling is signaled by the encoder to the decoder.

[0198] According to one embodiment, the luma samples used for CCCM model generation are low-pass filtered.

[0199] According to one embodiment, the CCCM coefficients include additional data terms that lie outside the filter kernel, such as already reconstructed (ie, causal) chroma samples or spatial coordinates.

[0200] According to one embodiment, the selection of the additional data term is signaled by the encoder to the decoder at any given precision.

[0201] According to one embodiment, a recursive prediction is used that includes the causal chroma neighborhood in the prediction.

[0202] According to one embodiment, the binary choice of using recursive prediction is signaled by the encoder to the decoder.

[0203] According to one embodiment, the spatial coordinates of the reconstructed samples can be used as additional data terms for the filter, which allows modeling of the chroma samples using a surface model (such as planar or quadratic).

[0204] According to one embodiment, the binary selection of the use of spatial coordinates for filtering is signaled by the encoder to the decoder.

[0205] According to one embodiment, the selected surface model is signaled by the encoder to the decoder at any given precision.

[0206] According to one embodiment, one or more CCCM modes are applied as filtering and / or adjustment operations beyond the non-CCCM prediction. For example, CCCM filtering or adjustment can be applied to blocks predicted by an angle prediction mode or an existing CCLM mode. The parameters of the CCCM filter can be determined by using re-predicted neighboring samples with the non-CCCM prediction used to predict the inner samples of the block. The re-predicted and reconstructed neighboring samples are then used to derive the CCCM filter parameters.

[0207] According to one embodiment, the final prediction of a block is achieved by combining one or more of the non-CCCM predictions (e.g., angular intra prediction, CCLM) with one or more of the CCCM predictions. The weight of such a combination of different predictions can be fixed or signaled in the bitstream or can be determined based on the prediction modes of the neighboring blocks and / or the prediction modes of the collocate blocks.

[0208] According to one embodiment, the CCLM coefficients of a block are derived using a reference sample above the block, a reference sample to the left of the block, or both the reference samples above and to the left of the block. The choice of the reference sample for parameter derivation can be signaled in the bitstream or can be determined at the decoder side, for example by using a texture analysis mechanism on the entire reference samples.

[0209] According to one embodiment, the shape, size, width, height, or other data that determines the selection of reference samples for CCCM model generation is signaled by the encoder to the decoder.

[0210] According to one embodiment, the type of nonlinear term of the CCLM model is determined based on a texture analysis mechanism over the reference samples and / or the luma samples inside the block.

[0211] According to one embodiment, the reference channel can be luma, chroma, R, G, B, or any other color space format. For example, the CCCM can use reconstructed chroma Cb samples to predict chroma Cr samples, or vice versa.

[0212] According to one embodiment, the CCCM scheme can also be used to predict more than just chroma samples, for example, instead of using luma and chroma samples, an embodiment can use intra and inter samples to further adjust the inter samples to match the intra neighbors, or vice versa.

[0213] An apparatus according to one embodiment includes means for receiving an image block unit of a frame, the image block unit including color channel samples including at least one chrominance channel and one luminance channel; means for reconstructing the luminance channel samples of the image block unit; means for determining a reference area for predicting the at least one color channel sample of the image block unit, the reference area including one or more of reference samples in a neighboring block of a current color channel / frame, in a neighboring block of a collocated block of a reference color channel / frame, and / or inside a collocated block of a reference color channel / frame; means for determining filter coefficients of a filter for prediction based on the reference samples and a shape of the filter; and means for predicting the at least one color channel sample of the image block unit by performing a convolution between the luminance channel samples and the filter coefficients.

[0214] According to one embodiment, the apparatus comprises means for downsampling the samples of the luminance channel to correspond to a sample size of the chrominance samples before determining the filter coefficients.

[0215] According to one embodiment, the means for determining the filter coefficients includes means for obtaining statistics from reference samples for linear regression based on the shape of the filter, and means for obtaining the filter coefficients by applying a linear regression method to the obtained statistics.

[0216] According to one embodiment, the statistics include an autocorrelation matrix and a cross-correlation vector.

[0217] According to one embodiment, the shape of the filter is a one-dimensional (1D) vector of filter coefficients or a two-dimensional (2D) array of filter coefficients.

[0218] According to one embodiment, the device comprises means for applying auxiliary parameters from outside the shape of the filter to the convolution.

[0219] According to one embodiment, the apparatus includes means for inputting a luma sample collocated with an output chroma sample, at least one luma sample above the collocated sample, at least one luma sample below the collocated sample, at least one luma sample to the left of the collocated sample, at least one luma sample to the right of the collocated sample, and a bias term to a filter.

[0220] According to one embodiment, the apparatus includes means for inputting values ​​to the filter determined by applying a non-linear function to the values ​​of the collocated samples.

[0221] According to one embodiment, the apparatus includes means for storing filter coefficients in a memory and means for receiving an indication from an encoder to reuse at least a subset of previously stored filter coefficients to predict samples of at least one color channel of a current image block unit.

[0222] According to one embodiment, the apparatus includes means for determining samples outside a reference area, means for obtaining statistics from the samples outside the reference area for linear regression based on a shape of the filter, means for obtaining filter coefficients by applying a linear regression method to the obtained statistics, and means for storing the filter coefficients in a memory.

[0223] According to one embodiment, the apparatus includes means for replacing any missing downsampled luminance samples by zero value samples.

[0224] According to one embodiment, the apparatus includes means for determining samples that do not belong to any object or texture inside the image block unit, and means for removing the samples from the reference area.

[0225] In another aspect, an apparatus is provided that includes at least one processor and at least one memory having stored therein code, which, when executed by the at least one processor, causes the apparatus to perform at least the following steps: receiving an image block unit of a frame including samples of color channels including at least one chrominance channel and one luminance channel; reconstructing the samples of the luminance channel of the image block unit; determining a reference area for predicting the samples of the at least one color channel of the image block unit, the reference area including one or more of the reference samples in a neighboring block of a current color channel / frame, in a neighboring block of a reference color channel / frame, and / or inside a collocated block of a reference color channel / frame; determining filter coefficients of a filter for prediction based on the reference samples and a shape of the filter; and predicting the samples of the at least one color channel of the image block unit by performing a convolution between the samples of the luminance channel and the filter coefficients.

[0226] Such an apparatus may, for example, include the functional units disclosed in any of Figures 1, 2, 4a and 4b for implementing the embodiments.

[0227] Such an apparatus further includes code stored in the at least one memory that, when executed by the at least one processor, causes the apparatus to perform one or more of the embodiments disclosed herein.

[0228] FIG. 12 is a schematic diagram of an exemplary multimedia communication system in which various embodiments may be implemented. A data source 1510 provides a source signal in analog, uncompressed digital, or compressed digital format, or any combination of these formats. An encoder 1520 may include or be connected to pre-processing such as data format conversion and / or filtering of the source signal. The encoder 1520 encodes the source signal into an encoded media stream. It should be noted that the bitstream to be decoded may be received directly or indirectly from a remote device located within virtually any type of network. In addition, the bitstream may be received from local hardware or software. The encoder 1520 may encode more than one media type, such as audio and video, or may request more than one encoder 1520 to encode different media types of the source signal. The encoder 1520 may also take synthetically generated inputs, such as graphics and text, or may generate encoded bitstreams of synthetic media. In the following, only the processing of one encoded media stream of one media type is considered to simplify the explanation. However, it should be noted that a real-time broadcast service typically includes several streams (typically at least one audio, video and text subtitle stream). It should also be noted that a system may include many encoders, but only one encoder 1520 is shown in the figure for simplicity of illustration and without loss of generality. It should further be appreciated that although the text and examples contained herein may describe the encoding process in detail, those skilled in the art will understand that the same concepts and principles apply to the corresponding decoding process and vice versa.

[0229] The encoded media bitstreams can be transferred to storage 1530. Storage 1530 can include any type of mass memory that stores encoded media streams. The format of the encoded media bitstreams in storage 1530 can be an elementary self-contained bitstream format, or one or more encoded media bitstreams can be encapsulated in a container file, or the encoded media bitstreams can be encapsulated in a segment format suitable for DASH (or similar streaming systems) and stored as a sequence of segments. If one or more media bitstreams are encapsulated in a container file, a file generator (not shown) can be used to store the one or more media bitstreams in a file and generate file format metadata, which can also be stored in the file. Encoder 1520 or storage 1530 can include a file generator, or a file generator is operatively attached to either encoder 1520 or storage 1530. Some systems operate "live", i.e., omitting storage and forwarding the coded media bitstreams directly from the encoder 1520 to the sender 1540. The coded media bitstreams can then be forwarded to the sender 1540, also called a server, as needed. The format used for transmission can be an elementary self-contained bitstream format, a packet stream format, a segment format suitable for DASH (or similar streaming systems), or one or more coded media bitstreams can be encapsulated in a container file. The encoder 1520, storage 1530, and server 1540 can reside on the same physical device or can be included in separate devices.The encoder 1520 and server 1540 operate with live, real-time content, in which case the encoded media bitstream is typically not stored persistently but is buffered for short periods in the content encoder 1520 and / or server 1540 to eliminate processing delays, transmission delays, and fluctuations in the encoded media bitrate.

[0230] The server 1540 transmits the coded media bitstream using a communication protocol stack. The stack may include, but is not limited to, one or more of Real-time Transport Protocol (RTP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), Transmission Control Protocol (TCP), and Internet Protocol (IP). When the communication protocol stack is packet-oriented, the server 1540 encapsulates the coded media bitstream into packets. For example, when RTP is used, the server 1540 encapsulates the coded media bitstream into RTP packets according to an RTP payload format. Generally, each media type has a dedicated RTP payload format. It should be noted that the system can include more than one server 1540, but for simplicity, the following description considers only one server 1540.

[0231] If the media content is encapsulated in a container file for storage 1530 or for inputting data to the Transmitter 1540, the Transmitter 1540 may include or be operatively attached to a "sender file parser" (not shown). In particular, if a container file is not transmitted but at least one of the encapsulated media bitstreams is encapsulated for transport through a communication protocol, the sender file parser locates the appropriate portion of the encoded media bitstream to be transmitted through the communication protocol. The sender file parser may also assist in generating the correct format of the communication protocol, such as packet headers and payloads. The multimedia container file may include encapsulation instructions, such as ISOBMFF hint tracks, for encapsulation of at least one of the encapsulated media bitstreams in the communication protocol.

[0232] The server 1540 may or may not be connected to the gateway 1550 through a communication network, which may be, for example, a CDN, the Internet, and / or a combination of one or more access networks. The gateway may additionally or alternatively be referred to as a middlebox. In DASH, the gateway may be an edge server (of the CDN) or a web proxy. Note that for simplicity, the following description considers only one gateway 1550, although the system may generally include any number of gateways, etc. The gateway 1550 may perform various types of functions, such as converting a packet stream according to one communication protocol stack to another communication protocol stack, merging and forking data streams, and manipulating data streams according to downlink and / or receiver capabilities, such as controlling the bit rate of the forward stream according to pre-valence downlink network conditions. The gateway 1550 may be a server entity in various embodiments.

[0233] The system typically includes one or more receivers 1560 that can receive the transmitted signal, demodulate, and decapsulate into a coded media bitstream. The coded media bitstream can be transferred to a record storage 1570. The record storage 1570 can include any type of mass memory that stores the coded media bitstream. The record storage 1570 can alternatively or additionally include a computational memory such as a random access memory. The format of the coded media bitstream in the record storage 1570 can be an elementary self-contained bitstream format, or one or more coded media bitstreams can be encapsulated into a container file. When there are multiple coded media bitstreams, such as audio and video streams that are associated with each other, a container file is typically used, and the receiver 1560 includes or is attached to a container file generator that generates the container file from the input streams. Some systems operate "live", i.e., omit the record storage 1570, and transfer the coded media bitstream directly from the receiver 1560 to the decoder 1580. In some systems, only the most recent portion of the recorded stream, e.g., the most recent 10 minute excerpt of the recorded stream, is kept in recording storage 1570, while any earlier recorded data is discarded from recording storage 1570.

[0234] The encoded media bitstreams can be transferred from the archival storage 1570 to the decoder 1580. If there are many encoded media bitstreams, such as an audio stream and a video stream, that are associated with each other and encapsulated in a container file, or if a single media bitstream is encapsulated in a container file, for example for easy access, a file parser (not shown) is used to decapsulate each encoded media bitstream from the container file. The archival storage 1570 or the decoder 1580 can include the file parser, or the file parser is attached to either the archival storage 1570 or the decoder 1580. It should also be noted that while a system can include many decoders, only one decoder 1570 is shown here to simplify the explanation without loss of generality.

[0235] The encoded media bitstream may be further processed by a decoder 1570, the output of which is one or more uncompressed media streams. Finally, a renderer 1590 may play the uncompressed media streams, for example, via a loudspeaker or a display. The receiver 1560, recording storage 1570, decoder 1570, and renderer 1590 may reside in the same physical device or may be included in separate devices.

[0236] The Transmitter 1540 and / or the Gateway 1550 may be configured to perform, for example, switching between different viewpoints of the 360-degree video content, view switching, switching between different representations for bitrate adaptation and / or fast start-up, and / or the Transmitter 1540 and / or the Gateway 1550 may be configured to select a transmitted representation. Switching between different representations may occur for multiple reasons, such as in response to a request of the Receiver 1560 or pre-conditions such as the throughput of the network over which the bitstream is transmitted. In other words, the Receiver 1560 may initiate the switching between representations. The request from the receiver may be, for example, a request for a segment or sub-segment from a representation different from the previous one, a request for a change in the transmitted scalability layer and / or sub-layer, or a change in a rendering device with different capabilities compared to the previous one. The request for a segment may be an HTTP GET request. The request for a sub-segment may be an HTTP GET request with a byte range. Additionally or alternatively, bitrate adjustment or bitrate adaptation can be used, for example, to provide so-called fast start-up in streaming services, when the bitrate of the transmitted stream is lower than the channel bitrate after the start of streaming or random access, in order to achieve a buffer occupancy level that starts playback immediately and tolerates occasional packet delays and / or retransmissions. Bitrate adaptation can include multi-representation or layer-up switching and representation or layer-down switching operations performed in various orders.

[0237] The decoder 1580 may be configured to perform switching between different representations, e.g., switching between different viewpoints of 360-degree video content, view switching, bitrate adaptation and / or fast start-up, and / or the decoder 1580 may be configured to select the representation to be transmitted. Switching between different representations may be performed for a number of reasons, such as to achieve fast decoding operations or to adapt the transmitted bitstream, e.g., in terms of bitrate, to pre-conditions such as the throughput of the network through which the bitstream is transmitted. Fast decoding operations may be required, for example, when a device including the decoder 1580 is multitasking and uses computational resources for purposes other than decoding the video bitstream. In another embodiment, fast decoding operations may be required when the content is played at a faster pace than the standard playback speed, e.g., two or three times faster than conventional real-time playback speed.

[0238] Some embodiments are described above with respect to and / or using HEVC and / or VVC terminology, it should be understood that the embodiments may be implemented with any video encoder and / or video decoder as well.

[0239] The exemplary embodiments are described above in terms of an encoder, and it should be understood that the resulting bitstream and decoder may have corresponding elements. Similarly, where the exemplary embodiments are described in terms of a decoder, it should be understood that the encoder may have a structure and / or computer program for generating a bitstream that is decoded by the decoder. For example, some embodiments are described in terms of generating a predictive block as part of the encoding. The embodiments may be similarly implemented by generating a predictive block as part of the decoding, with differences in coding parameters such as horizontal and vertical offsets being decoded from the bitstream rather than being determined by the encoder.

[0240] The above-described embodiments of the invention describe the codec in terms of separate encoder and decoder devices to aid in understanding the processes involved. However, it will be understood that the devices, structures and operations may be implemented as a single encoder-decoder device / structure / operation. Furthermore, the encoder and decoder may share some or all common elements.

[0241] Although the above examples describe embodiments of the invention operating within a codec within an electronic device, it will be appreciated that the invention as defined in the claims may be implemented as part of any video codec. Thus, for example, embodiments of the invention may be implemented in a video codec capable of performing video encoding over a fixed or wired communications path.

[0242] Thus, the user equipment may include a video codec such as those described in the embodiments of the present invention above. It should be understood that the term user equipment is intended to cover any suitable type of wireless user equipment such as a mobile telephone, a portable data processing device or a portable web browser.

[0243] Furthermore, elements of public land mobile networks (PLMNs) may also include video codecs as described above.

[0244] In general, various embodiments of the present invention can be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, some aspects can be implemented in hardware and other aspects can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device, but the present invention is not limited thereto. Although various aspects of the present invention can be illustrated and described as block diagrams, flow diagrams, or using some other pictorial representations, it is well understood that these blocks, devices, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof, by way of non-limiting examples.

[0245] The embodiments of the invention can be implemented by computer software executable by a data processor of a mobile device such as a processor entity, or by hardware, or by a combination of software and hardware. Further in this regard, it should be noted that any block of the logic flow in the diagrams can represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software can be stored on physical media as memory chips, or memory blocks implemented in a processor, magnetic media such as hard disks or floppy disks, and optical media such as DVDs and CDs, which are data variants of this.

[0246] The memory may be of any type suitable for the local technology environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed and removable memories, etc. The data processor may be of any type suitable for the local technology environment and may include, by way of non-limiting examples, one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), and processors based on multi-core processor architectures.

[0247] Embodiments of the present invention may be implemented in a variety of components, such as integrated circuit modules. The design of integrated circuits is generally a highly automated process. Complex and powerful software tools are available to convert logic level designs into semiconductor circuit designs ready to be etched and formed into semiconductor substrates.

[0248] Programs such as those offered by Synopsys, Inc. of Mountain View, Calif., and Cadence Design of San Jose, Calif., use established design rules and libraries of pre-stored design modules to automatically route conductors and place components on a semiconductor chip. Once the design of a semiconductor circuit is complete, the resulting design is sent in a standardized electronic format (e.g., Opus, GDSII, etc.) to a semiconductor manufacturing facility or manufacturing "fab."

[0249] The foregoing description provides a complete and informative description of exemplary embodiments of the present invention, by way of illustrative and non-limiting examples. However, various modifications and adaptations will become apparent to those skilled in the art in view of the foregoing description, when read in conjunction with the accompanying drawings and the appended claims. However, all these and similar modifications of the teachings of the present invention are intended to fall within the scope of the present invention. [Explanation of symbols]

[0250] Receive 900 frames of image block units, the image block units including samples of color channels including at least one chrominance channel and one luminance channel. 902 Reconstruct the samples of the above luminance channel of the image block unit 904. Determine a reference area for predicting samples of at least one color channel of the image block unit, the reference area including one or more of the reference samples in a neighboring block of the current color channel / frame, in a neighboring block of the reference channel / frame, and / or inside a collocated block of the reference color channel / frame. 906 Determine filter coefficients of a filter for the prediction based on the reference samples and the shape of the filter. 908 predicting samples of at least one color channel of the image block unit by performing a convolution between the samples of the luminance channel and the filter coefficients.

Claims

1. means for receiving an image block unit of a frame, the image block unit including samples of color channels including at least one chrominance channel and one luminance channel; means for reconstructing samples of the luminance channel of the image block unit; means for determining a reference area for predicting samples of at least one color channel of the image block unit, the reference area including one or more of reference samples in a neighboring block in a current color channel / frame, in a neighboring collocated block of a reference color channel / frame, and / or inside the collocated block of a reference color channel / frame; means for determining filter coefficients of a filter for the prediction based on the reference samples and a shape of the filter; means for predicting samples of at least one color channel of the image block unit by performing a convolution between the samples of the luminance channel and the filter coefficients; 13. An apparatus comprising:

2. 2. The apparatus of claim 1, further comprising: means for downsampling the samples of the luminance channel to correspond to a sample size of the chrominance samples prior to determining the filter coefficients.

3. The means for determining the filter coefficients comprises: means for obtaining statistics from the reference samples for linear regression based on the shape of the filter; means for obtaining the filter coefficients by applying a linear regression method to the obtained statistics; 3. The apparatus of claim 1 or 2, comprising:

4. The apparatus of claim 3 , wherein the statistics include an autocorrelation matrix and a cross-correlation vector.

5. The apparatus of any of claims 1 to 4, wherein the shape of the filter is a one-dimensional (1D) vector of the filter coefficients or a two-dimensional (2D) array of the filter coefficients.

6. Apparatus according to any of the preceding claims, comprising means for applying auxiliary parameters from outside the shape of the filter to the convolution.

7. 7. The apparatus of claim 1, comprising: means for inputting to the filter a luma sample collocated with an output chroma sample, at least one luma sample above the collocated sample, at least one luma sample below the collocated sample, at least one luma sample to the left of the collocated sample, at least one luma sample to the right of the collocated sample, and a bias term.

8. The apparatus of claim 7 , further comprising: means for inputting to the filter a value determined by applying a non-linear function to values ​​of the collocated samples.

9. means for storing the filter coefficients in a memory; means for receiving an indication from an encoder to reuse at least a subset of previously stored filter coefficients to predict samples of at least one color channel of the current image block unit; The apparatus according to any one of claims 1 to 8, comprising:

10. means for determining samples outside said reference area; means for obtaining statistics from the samples outside the reference area for linear regression based on the shape of the filter; means for obtaining the filter coefficients by applying a linear regression method to the obtained statistics; means for storing said filter coefficients in said memory; The apparatus according to any one of claims 1 to 10, comprising:

11. Apparatus according to any of claims 2 to 10, including means for replacing any missing downsampled luminance samples by zero value samples.

12. means for determining samples that do not belong to any object or texture inside the image block unit; means for removing the sample from the reference area; 12. The apparatus of claim 1, comprising:

13. 1. A method comprising: receiving an image block unit of a frame, the image block unit including samples of color channels including at least one chrominance channel and one luminance channel; reconstructing samples of the luminance channel of the image block unit; determining a reference area for predicting samples of at least one color channel of the image block unit, the reference area including one or more of reference samples in a neighboring block of a current color channel / frame, in a neighboring collocated block of a reference color channel / frame, and / or inside the collocated block of a reference color channel / frame; determining filter coefficients of a filter for the prediction based on the reference samples and a shape of the filter; predicting samples of at least one color channel of the image block by performing a convolution between the samples of the luminance channel and the filter coefficients; The method includes:

14. 14. The method of claim 13, comprising downsampling the samples of the luminance channel to correspond to a sample size of the chrominance samples before determining the filter coefficients.

15. The step of determining the filter coefficients comprises: obtaining statistics from the reference samples for linear regression based on the shape of the filter; obtaining the filter coefficients by applying a linear regression method to the obtained statistics; The method of claim 13 or 14, comprising:

Images