Method and apparatus for cross-component prediction for video coding

Abstract

The present disclosure provides a method for decoding video data, comprising: obtaining a video block from a bitstream; obtaining an internal brightness sample point value of the video block, an external brightness sample point value of an external region of the video block and an external chrominance sample point value of the external region; determining a set of weighting coefficients corresponding to a filter shape based on the external luma sample value and the external chroma sample value; predicting, using the filter shape and the set of weighting coefficients, each of internal chroma sample values based on a plurality of corresponding luma sample values, the plurality of corresponding luminance sample values includes: one or more down-sampled luminance sample values associated with the filter shape, one or more non-down-sampled luminance sample values associated with the filter shape, and one or more non-linear luminance sample values; and obtaining a predicted video block using the predicted internal chroma sample values.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims priority to Provisional Application No. 63 / 425,929, filed on November 16, 2022. The entire content thereof is incorporated herein by reference in its entirety. Technical Field

[0003] This application relates to video encoding, decoding, and compression. More specifically, this application relates to methods and apparatuses for improving the encoding and decoding efficiency of image / video blocks to which cross - component prediction techniques are applied. Background Art

[0004] Digital video is supported by a wide variety of electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc. Electronic devices send and receive or otherwise convey digital video data across communication networks and / or store digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited memory resources of storage devices, video encoding and decoding can be used to compress video data according to one or more video encoding and decoding standards before the video data is conveyed or stored. For example, video encoding and decoding standards include Versatile Video Coding (VVC), Joint Exploration Test Model (JEM), High Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), Moving Picture Experts Group (MPEG) coding, etc. Video encoding and decoding typically utilize prediction methods (e.g., inter - frame prediction, intra - frame prediction, etc.), which utilize the redundancy inherent in video data. Video encoding and decoding aims to compress video data into a form that uses a lower bitrate while avoiding or minimizing degradation of video quality. Summary of the Invention

[0005] Embodiments of the present disclosure provide methods and apparatuses for improving the encoding and decoding efficiency of image / video blocks to which cross - component prediction techniques are applied.

[0006] A simplified summary of one or more aspects of the present disclosure is presented below in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects and is neither intended to identify the main or key elements of all aspects nor to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description presented later.

[0007] According to one aspect of the present disclosure, a method for decoding video data is provided. The method includes: receiving an encoded block of luminance samples for a block of the video data; decoding the encoded block of luminance samples to obtain the reconstructed luminance samples of the block; classifying the luminance samples of the block into one of a plurality of sample groups based on edge information of the luminance samples, wherein the luminance samples are obtained from one or more of the reconstructed luminance samples to correspond to chrominance samples of the block; and predicting the chrominance samples by applying one of a plurality of linear prediction models corresponding to the classified sample group to the luminance samples.

[0008] According to one aspect of the present disclosure, a computer system is provided, which includes one or more processors and one or more storage devices storing computer-executable instructions, and when the computer-executable instructions are executed, the one or more processors are caused to perform operations including: receiving an encoded block of luminance samples for a block of the video data; decoding the encoded block of luminance samples to obtain the reconstructed luminance samples of the block; classifying the luminance samples of the block into one of a plurality of sample groups based on edge information of the luminance samples, wherein the luminance samples are obtained from one or more of the reconstructed luminance samples to correspond to chrominance samples of the block; and predicting the chrominance samples by applying one of a plurality of linear prediction models corresponding to the classified sample group to the luminance samples.

[0009] According to one aspect of the present disclosure, a method for video decoding using an Edge Classification Linear Model (ELM) is provided. The method may include: receiving an encoded block of luminance samples for a first block of a video signal; decoding the encoded block of luminance samples to obtain reconstructed luminance samples; classifying the reconstructed luminance samples into a plurality of sample groups based on the direction and intensity of edge information; applying different linear prediction models to the reconstructed luminance samples in different sample groups; predicting chrominance samples for the first block of the video signal based on the applied linear prediction models.

[0010] According to one aspect of the present disclosure, a method for video decoding using a Filter-based Linear Model (FLM) is provided. The method may include: receiving an encoded block of luminance samples for a first block of a video signal; decoding the encoded block of luminance samples to obtain reconstructed luminance samples; determining a luminance sample region and a chrominance sample region to derive a Multiple Linear Regression (MLR) model; deriving the MLR model through pseudo-inverse matrix calculation; applying the MLR model to the reconstructed luminance samples; predicting chrominance samples for the first block of the video signal based on the applied MLR model.

[0011] According to one aspect of the present disclosure, a method for video decoding using a gradient linear model (GLM) is provided. The method may include: receiving an encoded block of luminance samples for a first block of a video signal; decoding the encoded block of luminance samples to obtain reconstructed luminance samples; developing a correlation between luminance AC information and chrominance intensity using the sample gradients; determining a luminance sample region and a chrominance sample region to derive a multiple linear regression (MLR) model; deriving the MLR model through pseudo-inverse matrix calculation; applying the MLR model to the reconstructed luminance samples; and predicting chrominance samples for the first block of the video signal based on the applied MLR model.

[0012] According to one aspect of the present disclosure, a method for video coding and decoding without using a downsampling process in a convolutional cross-component model (CCCM) is provided. The method may include: receiving an encoded block of luminance samples for a first block of a video signal; decoding the encoded block of luminance samples to obtain reconstructed luminance samples; using a non-downsampled luminance reference value and / or different selections of the non-downsampled luminance reference; determining a luminance sample region and a chrominance sample region to derive a convolutional cross-component model (CCCM); deriving the CCCM parameters through LDL decomposition; applying the CCCM to the reconstructed luminance samples; and predicting chrominance samples for the first block of the video signal based on the applied CCCM.

[0013] According to one aspect of the present disclosure, a method for video coding and decoding using LDL decomposition in a cross-component linear model (CCLM) / multi-model LM (MMLM) is provided. The method may include: receiving an encoded block of luminance samples for a first block of a video signal; decoding the encoded block of luminance samples to obtain reconstructed luminance samples; determining a luminance sample region and a chrominance sample region to derive a cross-component linear model (CCLM) / multi-model LM (MMLM); deriving the CCLM / MMLM parameters through LDL decomposition; applying the CCLM / MMLM to the reconstructed luminance samples; and predicting chrominance samples for the first block of the video signal based on the applied CCLM / MMLM.

[0014] According to one aspect of the present disclosure, a method for video coding and decoding using minimum sample constraints in FLM / GLM / ELM / CCCM is provided. The method may include: determining whether to apply an FLM / GLM / ELM / CCCM scheme in the intra prediction, where the number of samples is greater than or equal to a predefined number in the encoded block.

[0015] According to one aspect of the present disclosure, a method for video coding and decoding using non-downsampled luminance reference values and downsampled luminance reference values in CCCM is provided.

[0016] According to one aspect within the present disclosure, a method for video coding and decoding using multiple modes of a combination of FLM / GLM / ELM / CCCM / CCLM is provided.

[0017] According to one aspect of the present disclosure, a method for decoding video data is provided, including: obtaining a video block from a bitstream; obtaining internal luminance sample values of the video block, external luminance sample values of an external region of the video block, and external chrominance sample values of the external region; determining a set of weighting coefficients corresponding to a filter shape based on the external luminance sample values and the external chrominance sample values; using the filter shape and the set of weighting coefficients to predict each internal chrominance sample value among internal chrominance sample values based on multiple corresponding luminance sample values, where the multiple corresponding luminance sample values include: one or more downsampled luminance sample values associated with the filter shape, one or more non-downsampled luminance sample values associated with the filter shape, and one or more non-linear luminance sample values; and obtaining a predicted video block using the predicted internal chrominance sample values.

[0018] According to one aspect of the present disclosure, a method for encoding video data is provided, including: obtaining a video block; obtaining internal luminance sample values of the video block, external luminance sample values of an external region of the video block, and external chrominance sample values of the external region; determining a set of weighting coefficients corresponding to a filter shape based on the external luminance sample values and the external chrominance sample values; using the filter shape and the set of weighting coefficients to predict each internal chrominance sample value among internal chrominance sample values based on multiple corresponding luminance sample values, where the multiple corresponding luminance sample values include: one or more downsampled luminance sample values associated with the filter shape, one or more non-downsampled luminance sample values associated with the filter shape, and one or more non-linear luminance sample values; and generating a bitstream including an encoded video block by using the predicted internal chrominance sample values.

[0019] According to one aspect of the present disclosure, a computer system is provided, including: one or more processors; and one or more storage devices storing computer-executable instructions that, when executed, cause the one or more processors to perform the operations of the method of the present disclosure.

[0020] According to one aspect of the present disclosure, there is provided a computer program product storing computer-executable instructions that, when executed, cause one or more processors to perform operations of the method of the present disclosure.

[0021] According to one aspect of the present disclosure, there is provided a computer-readable storage medium storing instructions that, when executed by a computing device having one or more processors, cause the one or more processors to perform the decoding method of the present disclosure and store a bitstream to be decoded by the decoding method of the present disclosure.

[0022] According to one aspect of the present disclosure, there is provided a computer-readable storage medium storing instructions that, when executed by a computing device having one or more processors, cause the one or more processors to perform the encoding method of the present disclosure and store a bitstream generated by the encoding method of the present disclosure.

[0023] According to one aspect of the present disclosure, there is provided a computer-readable medium storing a bitstream, wherein the bitstream is to be decoded by performing operations of the method of the present disclosure.

[0024] According to one aspect of the present disclosure, there is provided a computer-readable medium storing a bitstream, wherein the bitstream is obtained by performing operations of the method of the present disclosure.

[0025] It should be understood that both the foregoing general description and the following detailed description are merely exemplary and do not limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings incorporated in this specification and constituting a part of this specification illustrate examples consistent with the present disclosure and, together with the description, are used to explain the principles of the present disclosure.

[0027] Figure 1 is a block diagram showing an exemplary system for encoding and decoding video blocks according to some implementations of the present disclosure.

[0028] Figure 2 is a block diagram showing an exemplary video encoder according to some implementations of the present disclosure.

[0029] Figure 3 is a block diagram showing an exemplary video decoder according to some implementations of the present disclosure.

[0030] Figures 4A to 4E is a block diagram showing how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some implementations of the present disclosure.

[0031] Figure 5 Shows the overall diagram of a block - based video encoder for VVC.

[0032] Figures 6A to 6E Is a schematic diagram showing block partitioning in VVC.

[0033] Figure 7 Shows the overall diagram of a video decoder for VVC.

[0034] Figure 8 Is a schematic diagram showing the positions of samples for deriving α and β.

[0035] Figures 9A to 9C Is a schematic diagram showing examples of MDLM, MDLM_L, and MDLM_T.

[0036] Figure 10 Is a schematic diagram showing an example of classifying adjacent samples into two groups.

[0037] Figure 11 Is a schematic diagram showing the inflection point T.

[0038] Figures 12A to 12B Is a schematic diagram showing the effect of the slope adjustment parameter "u".

[0039] Figure 13 Is a schematic diagram showing the co - located reconstructed luminance samples used.

[0040] Figure 14 Is a schematic diagram showing the adjacent reconstructed samples used.

[0041] Figures 15A to 15D Is a schematic diagram showing the process of intra - mode derivation on the decoder side.

[0042] Figure 16 Is a schematic diagram showing an example of four reference lines adjacent to a prediction block.

[0043] Figure 17 Is a schematic diagram showing the spatial part of a convolutional filter.

[0044] Figure 18 Is a schematic diagram showing the reference region (with its filling) used for deriving filter coefficients.

[0045] Figures 19A to 19B Is a schematic diagram showing that chrominance samples can be related to multiple luminance samples simultaneously.

[0046] Figure 20 Is a schematic diagram showing that coefficients / offsets of multiple (e.g., 6) luminance samples with respect to 1 chrominance sample are trained to linearly predict chrominance samples inside a CU.

[0047] Figure 21It is a schematic diagram showing that different chroma types / color formats can have different predefined filter shapes / taps.

[0048] Figure 22 It is a schematic diagram showing that FLM can use only the top or left luminance / chroma samples (extended) for parameter derivation.

[0049] Figure 23 It is a schematic diagram showing that FLM can use different lines for parameter derivation.

[0050] Figures 24A to 24D It is a schematic diagram showing the pre - operations before applying the MLR model (GLM 1 - tap / 2 - taps).

[0051] Figure 25 It is a schematic diagram showing examples of different shapes / numbers of filter taps.

[0052] Figure 26 It is a schematic diagram showing examples of different shapes / numbers of filter taps.

[0053] Figures 27A to 27B It is a schematic diagram showing examples of different shapes / numbers of filter taps.

[0054] Figures 28A to 28G It is a schematic diagram showing examples of different sets of filter taps.

[0055] Figures 29A to 29B It is a schematic diagram showing double training for implicit filter shape derivation.

[0056] Figure 30 It is a schematic diagram showing non - subsampled luminance samples.

[0057] Figure 31 Shows the workflow of a method for decoding video data according to one or more aspects of the present disclosure.

[0058] Figure 32 Shows the workflow of a method for encoding video data according to one or more aspects of the present disclosure.

[0059] Figure 33 It is a schematic diagram showing a computing environment coupled to a user interface according to some implementations of the present disclosure. Detailed Description

[0060] Reference will now be made in detail to specific implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth in order to assist in understanding the subject matter presented herein. However, the subject matter may be practiced using a variety of alternative approaches without departing from the scope of the claims and may be practiced without these specific details. For example, the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0061] It should be noted that the terms "first", "second", etc. used in the description, claims, and drawings of the present disclosure are used to distinguish objects and are not used to describe any particular order or sequence. It should be understood that the data used in this way may be interchanged under appropriate conditions, such that the embodiments of the present disclosure described herein may be implemented in an order other than those shown in the drawings or described in the present disclosure.

[0062] Figure 1 is a block diagram showing an exemplary system 10 for parallel encoding and decoding of video blocks according to some implementations of the present disclosure. As Figure 1 shown, system 10 includes a source device 12 that generates and encodes video data to be decoded by a destination device 14 at a later time. The source device 12 and the destination device 14 may include any of a variety of electronic devices, including cloud servers, server computers, desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, the source device 12 and the destination device 14 are equipped with wireless communication capabilities.

[0063] In some implementations, the destination device 14 may receive the encoded video data to be decoded via a link 16. The link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the destination device 14. In one example, the link 16 may include a communication medium such that the source device 12 can directly send the encoded video data to the destination device 14 in real time. The encoded video data may be modulated and sent to the destination device 14 according to a communication standard such as a wireless communication protocol. The communication medium may include any wireless or wired communication medium, such as the radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other device that may be used to facilitate communication from the source device 12 to the destination device 14.

[0064] In some other implementations, the encoded video data can be sent from the output interface 22 to the storage device 32. Subsequently, the encoded video data in the storage device 32 can be accessed by the target device 14 via the input interface 28. The storage device 32 can include any of a variety of distributed or local access data storage media, such as a hard disk drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing the encoded video data. In another example, the storage device 32 can correspond to a file server or another intermediate storage device that can hold the encoded video data generated by the source device 12. The target device 14 can access the stored video data from the storage device 32 via streaming or downloading. The file server can be any type of computer capable of storing the encoded video data and sending the encoded video data to the target device 14. Exemplary file servers include web servers (e.g., for websites), File Transfer Protocol (FTP) servers, network attached storage (NAS) devices, or local disk drives. The target device 14 can access the encoded video data through any standard data connection, including a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a Digital Subscriber Line (DSL), a cable modem, etc.), or a combination of both, which is suitable for accessing the encoded video data stored on the file server. The transmission of the encoded video data from the storage device 32 can be a streaming transmission, a download transmission, or a combination of both.

[0065] As Figure 1 shown, the source device 12 includes a video source 18, a video encoder 20, and an output interface 22. The video source 18 can include sources such as a video capture device, e.g., a camera, a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as the source video, or a combination of such sources. As an example, if the video source 18 is a camera of a security surveillance system, the source device 12 and the target device 14 can form a camera phone or a video phone. However, the implementations described in this application can generally be applied to video encoding and decoding and can be applied to wireless and / or wired applications.

[0066] The captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be sent directly to the target device 14 via the output interface 22 of the source device 12. The encoded video data can also (or alternatively) be stored on the storage device 32 for later access by the target device 14 or other devices for decoding and / or playback. The output interface 22 can also include a modem and / or a transmitter.

[0067] The target device 14 includes an input interface 28, a video decoder 30, and a display device 34. The input interface 28 may include a receiver and / or a modem and receives encoded video data via a link 16. The encoded video data transmitted via the link 16 or provided on a storage device 32 may include various syntax elements generated by the video encoder 20 for use by the video decoder 30 when decoding the video data. Such syntax elements may be included within the encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0068] In some implementations, the target device 14 may include a display device 34, which may be an integrated display device and an external display device configured to communicate with the target device 14. The display device 34 displays the decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0069] The video encoder 20 and the video decoder 30 may operate according to proprietary or industry standards, such as VVC, HEVC, MPEG-4 Part 10, AVC, or extensions of such standards. It should be understood that the present application is not limited to a particular video coding / decoding standard and may be applicable to other video coding / decoding standards. It is generally expected that the video encoder 20 of the source device 12 may be configured to encode video data according to any of these current or future standards. Similarly, it is generally also expected that the video decoder 30 of the target device 14 may be configured to decode video data according to any of these current or future standards.

[0070] The video encoder 20 and the video decoder 30 may each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When implemented partially in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0071] In some implementations, at least a portion of the components of source device 12 (e.g., video source 18, as described below with reference to Figure 2 The video encoder 20 described herein or components included in the video encoder 20 and the output interface 22) and / or at least a portion of the components of the target device 14 (eg, the input interface 28, as described below with reference to Figure 3 The video decoder 30 described herein or the components included in the video decoder 30 and the display device 34) may be operated in a cloud computing service network, which may provide software, platform and / or infrastructure, such as software as a service (SaaS), platform as a service (PaaS) or infrastructure as a service (IaaS). In some implementations, one or more components in the source device 12 and / or the target device 14 that are not included in the cloud computing service network may be provided in one or more client devices, and the one or more client devices may communicate with a server computer in the cloud computing service network via a wireless communication network (e.g., a cellular communication network, a short-range wireless communication network, or a global navigation satellite system (GNSS) communication network) or a wired communication network (e.g., a local area network (LAN) communication network or a power line communication (PLC) network). In an embodiment, at least a portion of the operations described herein may be implemented as a cloud-based service provided by one or more server computers, which are implemented by at least a portion of the components of the source device 12 and / or at least a portion of the components of the target device 14 in the cloud computing service network; and one or more other operations described herein may be implemented by one or more client devices. In some implementations, the cloud computing service network may be a private cloud, a public cloud, or a hybrid cloud. Without departing from the scope of the present disclosure, terms such as "cloud," "cloud computing," "cloud-based," etc., may be used interchangeably herein when appropriate. It should be understood that the present disclosure is not limited to implementation in the above-described cloud computing service network. Rather, the present disclosure may also be implemented in any other type of computing environment currently known or developed in the future.

[0072] Figure 2 is a block diagram illustrating an exemplary video encoder 20 according to some implementations described in the present application. The video encoder 20 can perform intra-frame and inter-frame predictive coding of video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence. It should be noted that the term "frame" can be used as a synonym for the term "image" or "picture" in the field of video coding and decoding.

[0073] like Figure 2As shown in the figure, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy encoding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a partitioning unit 45, an intra prediction processing unit 46, and an intra block copy (BC) unit 48. In some implementations, the video encoder 20 further includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63 (such as a deblocking filter) can be positioned between the adder 62 and the DPB 64 to filter block boundaries to remove block effect artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter such as a sample adaptive offset (SAO) filter, a cross-component sample adaptive offset (CCSAO) filter, and / or an adaptive loop filter (ALF) can be used to filter the output of the adder 62. It should be noted that for the CCSAO technology, the present application is not limited to the embodiments described herein, and alternatively, the present application can be applied to a case where an offset for any one of the luminance component, the Cb chrominance component, and the Cr chrominance component is selected for any other of the luminance component, the Cb chrominance component, and the Cr chrominance component to modify the any other component based on the selected offset. In addition, it should be noted that the first component mentioned herein can be any one of the luminance component, the Cb chrominance component, and the Cr chrominance component, the second component mentioned herein can be any other of the luminance component, the Cb chrominance component, and the Cr chrominance component, and the third component mentioned herein can be the remaining one of the luminance component, the Cb chrominance component, and the Cr chrominance component. In some examples, the loop filter can be omitted, and the decoded video block can be directly provided to the DPB 64 by the adder 62. The video encoder 20 can take the form of fixed or programmable hardware units, or can be divided among one or more of the fixed or programmable hardware units shown.

[0074] The video data memory 40 can store video data to be encoded by the components of the video encoder 20. The video data in the video data memory 40 can be obtained, for example, from Figure 1 the video source 18 shown in the figure. The DPB 64 is a buffer that stores reference video data (such as reference frames or pictures) used by the video encoder 20 (e.g., in intra or inter predictive coding modes) when encoding video data. The video data memory 40 and the DPB 64 can be formed by any one of a variety of memory devices. In various examples, the video data memory 40 can be on-chip with other components of the video encoder 20, or off-chip relative to those components.

[0075] AsFigure 2 As shown, after receiving video data, a segmentation unit 45 within a prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting a video frame into slices, tiles (e.g., a set of video blocks), or other larger coding units (CUs) according to a predefined segmentation structure such as a quadtree (QT) structure associated with the video data. A video frame is or can be regarded as a two-dimensional array or matrix of samples having sample values. Samples in the array can also be referred to as pixels or the basic units of an image. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the video frame. The video frame can be divided into multiple video blocks, for example, by using QT segmentation. A video block is again or can be regarded as a two-dimensional array or matrix of samples having sample values, but with dimensions smaller than those of the video frame. The number of samples in the horizontal and vertical directions (or axes) of the video block defines the size of the video block. The video block can also be segmented into one or more block partitions or sub-blocks (which can again form blocks) by, for example, iteratively using QT segmentation, binary tree (BT) segmentation, or ternary tree (TT) segmentation or any combination thereof. It should be noted that the term "block" or "video block" as used herein can be a part of a frame or picture, especially a rectangular (square or non-square) part. For example, referring to HEVC and VVC, a block or video block can be or correspond to a coding tree unit (CTU), CU, prediction unit (PU), or transform unit (TU), and / or can be or correspond to a corresponding block, for example, a coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB) and / or sub-block.

[0076] The prediction processing unit 41 can select one of multiple possible prediction coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one of multiple intra-prediction coding modes or one of multiple inter-prediction coding modes. The prediction processing unit 41 can provide the resulting intra- or inter-prediction coded block to an adder 50 to generate a residual block, and to an adder 62 to reconstruct the coded block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements such as motion vectors, intra-mode indicators, partition information, and other such syntax information to an entropy coding unit 56.

[0077] To select an appropriate intra-predictive coding mode for a current video block, the intra-prediction processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block with respect to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and the motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block with respect to one or more predictive blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can execute multiple coding channels, e.g., to select an appropriate coding mode for each block in the video data.

[0078] In some implementations, the motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern within the video frame sequence, the motion vector indicating the displacement of a video block within the current video frame relative to a predictive block within a reference video frame. The motion estimation performed by the motion estimation unit 42 is a process of generating a motion vector, which estimates the motion of a video block. For example, the motion vector can indicate the displacement of a video block within the current video frame or picture relative to a predictive block within the reference frame relative to the current block being encoded within the current frame. The predetermined pattern can specify video frames in the sequence as P frames or B frames. The intra BC unit 48 can determine a vector (e.g., a block vector) for intra BC coding in a manner similar to that of the motion vector determined by the motion estimation unit 42 for inter-prediction, or can utilize the motion estimation unit 42 to determine the block vector.

[0079] The predictive block for a video block can be or can correspond to a block or a reference block of a reference frame, which is considered to closely match the video block to be encoded in terms of pixel difference, and the pixel difference can be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 can calculate values at sub-integer pixel positions of the reference frames stored in the DPB 64. For example, the video encoder 20 can interpolate values at quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frames. Thus, the motion estimation unit 42 can perform a motion search with respect to full pixel positions and fractional pixel positions, and output a motion vector with fractional pixel accuracy.

[0080] The motion estimation unit 42 calculates a motion vector for a video block in an inter-predicted coded frame by comparing the position of the video block with the position of a predictive block of a reference frame selected from a first reference frame list (list 0) or a second reference frame list (list 1), and each of list 0 or list 1 identifies one or more reference frames stored in the DPB 64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44 and then to the entropy coding unit 56.

[0081] The motion compensation performed by the motion compensation unit 44 may involve extracting or generating a predictive block based on the motion vectors determined by the motion estimation unit 42. When receiving a motion vector for a current video block, the motion compensation unit 44 may locate the predictive block pointed to in one of the reference frame lists by the motion vector, retrieve the predictive block from the DPB 64, and forward the predictive block to the adder 50. The adder 50 then forms a residual video block of pixel differences by subtracting the pixel values of the predictive block provided by the motion compensation unit 44 from the pixel values of the current video block being encoded. The pixel differences forming the residual video block may include luminance component differences or chrominance component differences or both. The motion compensation unit 44 may also generate syntax elements associated with the video blocks of a video frame for use by the video decoder 30 when decoding the video blocks of the video frame. The syntax elements may include, for example, syntax elements defining the motion vectors used to identify the predictive blocks, any flags indicating the prediction mode, or any other syntax information described herein. It should be noted that the motion estimation unit 42 and the motion compensation unit 44 may be highly integrated but are shown separately for conceptual purposes.

[0082] In some implementations, the intra BC unit 48 may generate vectors and extract predictive blocks in a manner similar to that described above in connection with the motion estimation unit 42 and the motion compensation unit 44, but the predictive block is in the same frame as the current block being encoded and wherein the vector is referred to as a block vector rather than a motion vector. In particular, the intra BC unit 48 may determine the intra prediction mode to be used for encoding the current block. In some examples, the intra BC unit 48 may encode the current block using various intra prediction modes, for example, during a separate encoding pass, and test its performance through rate-distortion analysis. Next, the intra BC unit 48 may select an appropriate intra prediction mode to use among the various tested intra prediction modes and generate an intra mode indicator accordingly. For example, the intra BC unit 48 may use rate-distortion analysis for the various tested intra prediction modes to calculate rate-distortion values and select, among the tested modes, the intra prediction mode having the best rate-distortion characteristics as the appropriate intra prediction mode to use. Rate-distortion analysis generally determines the amount of distortion (or error) between the encoded block and the original unencoded block that was encoded to produce the encoded block, as well as the bit rate (i.e., the number of bits) used to produce the encoded block. The intra BC unit 48 may calculate a ratio based on the distortion and rate of the various encoded blocks to determine which intra prediction mode exhibits the best rate-distortion value for the block.

[0083] In other examples, the intra BC unit 48 may perform such functions for intra BC prediction according to the implementations described herein, in whole or in part, using the motion estimation unit 42 and the motion compensation unit 44. In any case, for intra block copy, the predictive block may be a block that is considered to closely match the block to be encoded in terms of pixel differences, which may be determined by SAD, SSD, or other difference metrics, and the identification of the predictive block may include the calculation of values for sub-integer pixel positions.

[0084] Regardless of whether the predictive block is from the same frame according to intra prediction or a different frame according to inter prediction, the video encoder 20 may form a residual video block by subtracting the pixel values of the predictive block from the pixel values of the current video block being encoded, thereby forming pixel difference values. The formation of the pixel difference values of the residual video block may include both the luminance component difference and the chrominance component difference.

[0085] The intra prediction processing unit 46 may intra predict the current video block as an alternative to the inter prediction performed by the motion estimation unit 42 and the motion compensation unit 44 or the intra block copy prediction performed by the intra BC unit 48, as described above. In particular, the intra prediction processing unit 46 may determine an intra prediction mode for encoding the current block. To this end, the intra prediction processing unit 46 may encode the current block using various intra prediction modes, for example, during a separate encoding pass, and the intra prediction processing unit 46 (or in some examples, the mode selection unit) may select an appropriate intra prediction mode to use from the tested intra prediction modes. The intra prediction processing unit 46 may provide information indicating the selected intra prediction mode for the block to the entropy coding unit 56. The entropy coding unit 56 may encode the information indicating the selected intra prediction mode into the bitstream.

[0086] After the prediction processing unit 41 determines a predictive block for the current video block via inter prediction or intra prediction, the summer 50 forms a residual video block by subtracting the predictive block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to the transform processing unit 52. The transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform such as a discrete cosine transform (DCT) or a conceptually similar transform.

[0087] The transform processing unit 52 may send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting the quantization parameter. In some examples, the quantization unit 54 may then perform a scan of the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 may perform this scan.

[0088] After quantization, entropy encoding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. Then the encoded bitstream can be sent to video decoder 30 as shown in Figure 1 , or archived in storage device 32 as shown in Figure 1 for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode the motion vectors and other syntax elements of the current video frame being encoded.

[0089] Inverse quantization unit 58 and inverse transform processing unit 60 respectively apply inverse quantization and inverse transform to reconstruct the residual video block in the pixel domain to generate a reference block for prediction of other video blocks. As mentioned above, motion compensation unit 44 may generate a motion-compensated predictive block from one or more reference blocks of frames stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the predictive block to calculate sub-integer pixel values used in motion estimation.

[0090] Adder 62 adds the reconstructed residual block to the motion-compensated predictive block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. Then, the reference block can be used by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 as a predictive block for inter prediction of another video block in a subsequent video frame.

[0091] Figure 3 is a block diagram illustrating an exemplary video decoder 30 according to some implementations of the present application. Video decoder 30 includes video data memory 79, entropy decoding unit 80, prediction processing unit 81, inverse quantization unit 86, inverse transform processing unit 88, adder 90, and DPB 92. Prediction processing unit 81 also includes motion compensation unit 82, intra prediction unit 84, and intra BC unit 85. Video decoder 30 may perform a decoding process that is generally inverse to the encoding process described above with respect to video encoder 20 in conjunction with Figure 2 . For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, while intra prediction unit 84 may generate prediction data based on an intra prediction mode indicator received from entropy decoding unit 80.

[0092] In some examples, units of the video decoder 30 may be assigned tasks to perform implementations of the present application. Additionally, in some examples, implementations of the present disclosure may be divided among one or more of the units of the video decoder 30. For example, the intra BC unit 85 may perform implementations of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra BC unit 85, and the functionality of the intra BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0093] The video data memory 79 may store video data (such as an encoded video bitstream) to be decoded by other components of the video decoder 30. The video data stored in the video data memory 79 may be obtained, for example, from a storage device 32, via a wired or wireless network communication of the video data, or from a local video source (such as a camera) by accessing a physical data storage medium (e.g., a flash drive or a hard disk). The video data memory 79 may include a coded picture buffer (CPB) that stores encoded video data from the encoded video bitstream. The DPB 92 of the video decoder 30 stores reference video data used when decoding video data by the video decoder 30 (e.g., in an intra or inter predictive coding mode). The video data memory 79 and the DPB 92 may be formed of any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, in Figure 3 FIG. [not provided in the original, so left as is] the video data memory 79 and the DPB 92 are depicted as two separate components of the video decoder 30. However, those skilled in the art will appreciate that the video data memory 79 and the DPB 92 may be provided by the same memory device or separate memory devices. In some examples, the video data memory 79 may be on-chip or off-chip relative to other components of the video decoder 30.

[0094] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of an encoded video frame and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantized coefficients, motion vectors, or intra prediction mode indicators, as well as other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra prediction mode indicators, as well as other syntax elements, to the prediction processing unit 81.

[0095] When a video frame is encoded as an intra-predicted (I) frame or an intra-coded prediction block in other types of frames, the intra prediction unit 84 of the prediction processing unit 81 may generate prediction data for a video block of the current video frame based on the signaled intra prediction mode and reference data from previously decoded blocks of the current frame.

[0096] When a video frame is encoded as an inter-predicted (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for a video block of the current video frame based on the motion vectors and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks may be generated from a reference frame within one of the reference frame lists. The video decoder 30 may construct the reference frame lists (list 0 and list 1) using default construction techniques based on the reference frames stored in the DPB 92.

[0097] In some examples, when a video block is encoded according to the intra BC mode described herein, the intra BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block may be within the reconstructed region of the same picture as the current video block defined by the video encoder 20.

[0098] The motion compensation unit 82 and / or the intra BC unit 85 determine prediction information for a video block of the current video frame by parsing the motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded. For example, the motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode (e.g., intra or inter prediction) used to encode the video block of the video frame, the inter prediction frame type (e.g., B or P), construction information for one or more of the reference frame lists for the frame, the motion vector for each inter-predicted video block of the frame, the inter prediction state for each inter-predicted video block of the frame, and other information used to decode the video block in the current video frame.

[0099] Similarly, the intra BC unit 85 may use some of the received syntax elements (e.g., flags) to determine that the current video block is predicted using the following information: the intra BC mode, construction information for which video blocks of the frame are within the reconstructed region and should be stored in the DPB 92, the block vector for each intra BC predicted video block of the frame, the intra BC prediction state for each intra BC predicted video block of the frame, and other information used to decode the video block in the current video frame.

[0100] The motion compensation unit 82 may also perform interpolation using an interpolation filter such as the one used by the video encoder 20 during the encoding of video blocks to calculate the interpolated values for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 may determine the interpolation filter used by the video encoder 20 from the received syntax elements and use the interpolation filter to generate a predictive block.

[0101] The dequantization unit 86 dequantizes the quantized transform coefficients provided in the bitstream and the entropy decoding unit 80 entropy decodes using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or conceptually similar inverse transform process) to the transform coefficients in order to reconstruct the residual block in the pixel domain.

[0102] After the motion compensation unit 82 or the intra BC unit 85 generates a predictive block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 and the corresponding predictive block generated by the motion compensation unit 82 and the intra BC unit 85. A loop filter 91 (such as a deblocking filter, SAO filter, CCSAO filter, and / or ALF) may be positioned between the adder 90 and the DPB 92 to further process the decoded video block. In some examples, the loop filter 91 may be omitted and the decoded video block may be directly provided to the DPB 92 by the adder 90. Then, the decoded video blocks in a given frame are stored in the DPB 92, which stores the reference frames used for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 may also store the decoded video for later presentation on a display device (such as Figure 1 display device 34).

[0103] In a typical video coding process, a video sequence generally includes an ordered collection of frames or pictures. Each frame may include three arrays of samples, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luminance samples. SCb is a two-dimensional array of Cb chrominance samples. SCr is a two-dimensional array of Cr chrominance samples. In other cases, the frame may be monochrome and thus include only one two-dimensional array of luminance samples.

[0104] As Figure 4AAs shown, video encoder 20 (or more specifically, partitioning unit 45) generates an encoded representation of a frame by first dividing the frame into a set of CTUs. A video frame may include an integer number of CTUs that are sequentially ordered in a raster scan order from left to right and top to bottom. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by video encoder 20 in the sequence parameter set such that all CTUs in the video sequence have the same size of one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that the present application is not necessarily limited to a specific size. As Figure 4B As shown, each CTU may include one CTB of luminance samples, two corresponding coding tree blocks of chrominance samples, and syntax elements used to encode the samples of the coding tree blocks. The syntax elements describe the nature of different types of units of the encoded pixel block and how the video sequence can be reconstructed at video decoder 30, including inter-frame or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In a monochrome picture or a picture with three separate color planes, a CTU may include a single coding tree block and syntax elements used to encode the samples of the coding tree block. The coding tree block may be an N×N sample block.

[0105] To achieve better performance, video encoder 20 may recursively perform tree partitioning, such as binary tree partitioning, ternary tree partitioning, quadtree partitioning, or a combination thereof, on the coding tree blocks of the CTU and divide the CTU into smaller CUs. As Figure 4C Depicted, the 64x64 CTU 400 is first divided into four smaller CUs, each CU having a block size of 32x32. Among the four smaller CUs, CU 410 and CU 420 are each divided into four CUs with a block size of 16×16. The two 16×16 CUs 430 and 440 are each further divided into four CUs with a block size of 8×8. Figure 4D The depiction shows as Figure 4C A quadtree data structure depicting the final result of the partitioning process of CTU 400 as depicted, where each leaf node of the quadtree corresponds to a CU of a corresponding size ranging from 32x32 to 8x8. Similar to Figure 4B the CTU depicted, each CU may include a CB of luminance samples of a frame of the same size and two corresponding coding blocks of chrominance samples, and syntax elements used to encode the samples of the coding blocks. In a monochrome picture or a picture with three separate color planes, a CU may include a single coding block and a syntax structure used to encode the samples of the coding block. It should be noted that Figure 4C and Figure 4DThe quadtree splitting depicted is for illustrative purposes only, and a CTU can be split into CUs to adapt to varying local characteristics based on quadtree / tritree / binary tree splitting. In a multi-type tree structure, a CTU is partitioned by a quadtree structure, and each quadtree leaf CU can be further partitioned by binary tree and tritree structures. As Figure 4E shown in

[0106] In some implementations, the video encoder 20 may further split the coding block of a CU into one or more MxN PBs. A PB is a rectangular (square or non-square) block of samples on which the same prediction (inter-frame or intra-frame) is applied. The PU of a CU may include a PB of luma samples, two corresponding PBs of chroma samples, and syntax elements used to predict the PB. In a monochrome picture or a picture with three separate color planes, the PU may include a single PB and a syntax structure used to predict the PB. The video encoder 20 may generate predictive luma, Cb, and Cr blocks for the luma, Cb, and Cr PBs of each PU of the CU.

[0107] The video encoder 20 may use intra-frame prediction or inter-frame prediction to generate a predictive block for the PU. If the video encoder 20 uses intra-frame prediction to generate a predictive block for the PU, then the video encoder 20 may generate the predictive block for the PU based on the decoded samples of the frame associated with the PU. If the video encoder 20 uses inter-frame prediction to generate a predictive block for the PU, then the video encoder 20 may generate the predictive block for the PU based on the decoded samples of one or more frames different from the frame associated with the PU.

[0108] After the video encoder 20 generates predictive luma, Cb, and Cr blocks for one or more PUs of a CU, the video encoder 20 may generate a luma residual block for the CU by subtracting the predictive luma block of the CU from its original luma coding block, such that each sample in the luma residual block of the CU indicates the difference between the luma sample in one of the predictive luma blocks of the CU and the corresponding sample in the original luma coding block of the CU. Similarly, the video encoder 20 may generate a Cb residual block and a Cr residual block for the CU, such that each sample in the Cb residual block of the CU indicates the difference between the Cb sample in one of the predictive Cb blocks of the CU and the corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU may indicate the difference between the Cr sample in one of the predictive Cr blocks of the CU and the corresponding sample in the original Cr coding block of the CU.

[0109] In addition, as Figure 4CAs shown, video encoder 20 can use quadtree partitioning to decompose the luminance, Cb, and Cr residual blocks of a CU into one or more luminance, Cb, and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) block of samples on which the same transform is applied. The TUs of a CU can include a transform block of luminance samples, two corresponding transform blocks of chrominance samples, and syntax elements used to transform the samples of the transform block. Thus, each TU of a CU can be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with a TU can be a sub-block of the luminance residual block of the CU. The Cb transform block can be a sub-block of the Cb residual block of the CU. The Cr transform block can be a sub-block of the Cr residual block of the CU. In a monochrome picture or a picture with three separate color planes, a TU can include a single transform block and a syntax structure used to transform the samples of the transform block.

[0110] Video encoder 20 can apply one or more transforms to the luminance transform block of a TU to generate a luminance coefficient block of the TU. A coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be scalars. Video encoder 20 can apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block of the TU. Video encoder 20 can apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block of the TU.

[0111] After generating a coefficient block (e.g., a luminance coefficient block, a Cb coefficient block, or a Cr coefficient block), video encoder 20 can quantize the coefficient block. Quantization generally refers to a process in which transform coefficients are quantized to possibly reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After video encoder 20 quantizes the coefficient block, video encoder 20 can entropy code the syntax elements indicating the quantized transform coefficients. For example, video encoder 20 can perform CABAC on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 can output a bitstream that includes a sequence of bits forming a representation of the encoded frame and associated data, which is stored in storage device 32 or sent to destination device 14.

[0112] After receiving the bitstream generated by the video encoder 20, the video decoder 30 may parse the bitstream to obtain syntax elements from the bitstream. The video decoder 30 may reconstruct a frame of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing video data is generally inverse to the encoding process performed by the video encoder 20. For example, the video decoder 30 may perform an inverse transform on a coefficient block associated with a TU of a current CU to reconstruct a residual block associated with the TU of the current CU. The video decoder 30 also reconstructs an encoded block of the current CU by adding samples of a predictive block of a PU for the current CU to corresponding samples of a transform block of the TU of the current CU. After reconstructing the encoded block of each CU of a frame, the video decoder 30 may reconstruct the frame.

[0113] As described above, video coding and decoding mainly uses two modes (i.e., intra-frame prediction (or intra-prediction) and inter-frame prediction (or inter-prediction)) to achieve video compression. It should be noted that IBC can be regarded as intra-frame prediction or a third mode. Between the two modes, since motion vectors are used to predict the current video block from a reference video block, inter-frame prediction contributes more to coding and decoding efficiency than intra-frame prediction.

[0114] However, with continuously improving video data capture techniques for preserving details in video data and finer video block sizes, the amount of data required to represent the motion vectors for the current frame also increases significantly. One way to overcome this challenge is to benefit from the fact that not only do adjacent CU groups in both the spatial and temporal domains have similar video data for prediction purposes, but the motion vectors between these adjacent CUs are also similar. Therefore, it is possible to use the motion information of spatially adjacent CUs and / or temporally co-located CUs as an approximation of the motion information (e.g., motion vectors) of the current CU by exploring the spatial and temporal correlations of the current CU (which is also referred to as the "Motion Vector Prediction Value (MVP)" of the current CU).

[0115] Instead of encoding the actual motion vector of the current CU determined by the motion estimation unit 42 as described above in conjunction with Figure 2 the motion vector difference (MVD) of the current CU is generated by subtracting the motion vector prediction value of the current CU from the actual motion vector of the current CU. By doing so, it is not necessary to encode the motion vectors determined by the motion estimation unit 42 for each CU of a frame into the video bitstream, and the amount of data used to represent the motion information in the video bitstream can be significantly reduced.

[0116] Similar to the process of selecting a predictive block in a reference frame during inter prediction of a coding block, both the video encoder 20 and the video decoder 30 need to adopt a set of rules to construct a motion vector candidate list (also referred to as a "merge list") for the current CU using those potential candidate motion vectors associated with spatially neighboring CUs and / or temporally collocated CUs of the current CU, and then select a member from the motion vector candidate list as the motion vector prediction value for the current CU. By doing so, it is not necessary to send the motion vector candidate list itself from the video encoder 20 to the video decoder 30, and the index of the selected motion vector prediction value within the motion vector candidate list is sufficient for the video encoder 20 and the video decoder 30 to use the same motion vector prediction value within the motion vector candidate list to encode and decode the current CU.

[0117] Introduction

[0118] Various video coding and decoding techniques can be used to compress video data. Video coding and decoding are performed according to one or more video coding and decoding standards. For example, video coding and decoding standards include Versatile Video Coding (VVC), High Efficiency Video Coding (H.265 / HEVC), Advanced Video Coding (H.264 / AVC), Moving Picture Experts Group (MPEG) coding and decoding, etc. Video coding and decoding generally utilize prediction methods (e.g., inter prediction, intra prediction, etc.), which utilize the redundancy present in video images or sequences. An important goal of video coding and decoding techniques is to compress video data into a form that uses a lower bit rate while avoiding or minimizing degradation of video quality.

[0119] The first version of the VVC standard was completed in July 2020, which provides approximately 50% bit rate savings or equivalent perceived quality compared to the existing generation video coding and decoding standard HEVC. Although the VVC standard provides significant coding and decoding improvements over its predecessors, there is evidence that additional coding and decoding tools can be utilized to achieve excellent coding and decoding efficiency. Recently, the Joint Video Exploration Team (JVET) under the collaboration of ITU-T VCEG and ISO / IEC MPEG has started exploring advanced techniques that can achieve substantial enhancements in coding and decoding efficiency beyond VVC. In April 2021, a software codebase called the Enhanced Compression Model (ECM) was established for future video coding and decoding exploration work. The ECM reference software is based on the VVC Test Model (VTM) developed by JVET for VVC, where several existing modules (e.g., intra / inter prediction, transform, loop filtering, etc.) are further extended and / or improved. In the future, any new coding and decoding tools beyond the VVC standard can be integrated into the ECM platform and tested using the JVET Common Test Conditions (CTC).

[0120] Similar to all the aforementioned video coding standards, ECM is constructed on a block-based hybrid video coding framework. Figure 5 A block diagram of a general block-based hybrid video coding system is shown. The input video signal (referred to as a coding unit (CU)) is processed block by block. In ECM-1.0, a CU can be up to 128x128 pixels. However, similar to VVC, a coding tree unit (CTU) is split into CUs to adapt to varying local characteristics based on a quadtree / binary tree / trinary tree. In the multi-type tree structure, a CTU is first split by a quadtree structure. Then, each quadtree leaf node can be further split by binary tree and trinary tree structures. As Figure 6A , Figure 6B , Figure 6C , Figure 6D and Figure 6E shown, there are five splitting types: quaternary split, vertical binary split, horizontal binary split, vertical extended quaternary split, and horizontal extended quaternary split.

[0121] In Figure 5In [the context], spatial prediction and / or temporal prediction can be performed. Spatial prediction (or "intra prediction") uses pixels of samples from encoded neighboring blocks (which are referred to as reference samples) in the same video picture / slice to predict the current video block. Spatial prediction reduces the spatial redundancy inherent in the video signal. Temporal prediction (also referred to as "inter prediction" or "motion-compensated prediction") uses the reconstructed pixels from encoded video pictures to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in the video signal. The temporal prediction signal of a given CU is typically signaled by one or more motion vectors (MVs) indicating the amount and direction of motion between the current CU and its temporal reference. Additionally, if multiple reference pictures are supported, a reference picture index is also sent, which is used to identify which reference picture in the reference picture memory stores the temporal prediction signal. After spatial and / or temporal prediction, the mode decision block in the encoder selects the best prediction mode, for example, based on the rate-distortion optimization method. Then, the predicted block is subtracted from the current video block; and the prediction residual is decorrelated and quantized using a transform. The quantized residual coefficients are dequantized and inverse-transformed to form the reconstructed residual, which is then added back to the predicted block to form the reconstructed signal of the CU. Further loop filtering (such as the deblocking filter, sample adaptive offset (SAO), and adaptive loop filter (ALF)) can be applied to the reconstructed CU before placing the reconstructed CU in the reference picture memory and being used to encode future video blocks. To form the output video bitstream, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy coding unit to be further compressed and packed to form the bitstream. It should be noted that as used herein, the term "block" or "video block" can be a part of a frame or picture, especially a rectangular (square or non-square) part. For example, referring to HEVC and VVC, a block or video block can be or correspond to a coding tree unit (CTU), coding unit (CU), prediction unit (PU), or transform unit (TU), and / or can be or correspond to the corresponding block, for example, a coding tree block (CTB), coded block (CB), prediction block (PB), or transform block (TB) and / or sub-block.

[0122] Figure 7 FIG. shows a general block diagram of a block-based video decoder. The video bitstream is first entropy decoded at the entropy decoding unit. The coding mode and prediction information are sent to the spatial prediction unit (if intra-coded) or the temporal prediction unit (if inter-coded) to form the predicted block. The residual transform coefficients are sent to the inverse quantization unit and the inverse transform unit to reconstruct the residual block. Then, the predicted block and the residual block are added together. The reconstructed block can further undergo loop filtering before it is stored in the reference picture memory. Then, the reconstructed video in the reference picture memory is sent out to drive the display device and is used to predict future video blocks.

[0123] The main focus of this disclosure is to further enhance the encoding and decoding efficiency of the encoding and decoding tools for cross-component prediction and cross-component linear model (CCLM) applied in ECM. Below, some relevant encoding and decoding tools in ECM are briefly reviewed. After that, some deficiencies in the existing design of CCLM are discussed. Finally, solutions are provided to improve the existing CCLM prediction design.

[0124] Cross-component linear model prediction

[0125] To reduce cross-component redundancy, the cross-component linear model (CCLM) prediction mode is used in VVC, for which the chroma samples are predicted based on the reconstructed luminance samples of the same CU by using the following linear model:

[0126] pred C (i,j) = α · rec L ′(i,j) + β (1)

[0127] where pred C (i,j) represents the predicted chroma samples in the CU, and rec L ′(i,j) represents the downsampled reconstructed luminance samples obtained by downsampling the reconstructed luminance samples rec L (i,j). The above α and β are linear model parameters, which are derived from at most four adjacent chroma samples and their corresponding downsampled luminance samples, which can be called adjacent luminance-chroma sample pairs. Assuming that the current chroma block has a size of W×H, then W' and H' are obtained as follows:

[0128] – When the LM mode is applied, W’ = W, H’ = H;

[0129] – When the LM-A mode is applied, W’ = W + H;

[0130] – When the LM-L mode is applied, H’ = H + W;

[0131] where in the LM mode, the upper and left samples of the CU are used together to calculate the linear model coefficients; in the LM_A mode, only the upper samples of the CU are used to calculate the linear model coefficients; and in the LM_L mode, only the left samples of the CU are used to calculate the linear model coefficients.

[0132] If the positions of the upper adjacent samples of the chroma block are represented as S[0, -1] …… S[W'-1, -1], and the positions of the left adjacent samples of the chroma block are represented as S[-1, 0] …… S[-1, H'-1], then the positions of the four adjacent chroma samples are selected as follows:

[0133] – When the LM mode is applied and both the upper adjacent sample and the left adjacent sample are available, S[W' / 4, -1], S[3*W' / 4,

[0134] -1], S[-1, H' / 4], S[-1, 3*H' / 4] are selected as the positions of the four adjacent chrominance samples;

[0135] – When the LM-A mode is applied or only the upper adjacent sample is available, S[W' / 8, -1], S[3*W' / 8, -1], S[5*W' / 8,

[0136] -1], S[7*W' / 8, -1] are selected as the positions of the four adjacent chrominance samples;

[0137] – When the LM-L mode is applied or only the left adjacent sample is available, S[-1, H' / 8], S[-1, 3*H' / 8], S[-1, 5*H' / 8],

[0138] S[-1, 7*H' / 8] are selected as the positions of the four adjacent chrominance samples.

[0139] Four adjacent luma samples corresponding to the selected positions are obtained through a downsampling operation, and the four obtained adjacent luma samples are compared four times to find two larger values: x 0 A and x 1 A , and two smaller values: x 0 B and x 1 B . The chrominance sample values corresponding to the two larger values and the two smaller values are respectively denoted as y 0 A , y 1 A , y 0 B and y 1 B . Then, X a 、X b 、Y a and Y b are derived as:

[0140] X a =(x 0 A +x 1 A +1)>>1;

[0141] X b =(x 0 B +x 1B (+1) >> 1;

[0142] Y a = (y 0 A + y 1 A (+1) >> 1;

[0143] Y b = (y 0 B + y 1 B (+1) >> 1 (2)

[0144] Finally, the linear model parameters α and β are obtained according to the following equations.

[0145]

[0146] β = Y b - α · X b (4)

[0147] Figure 8 An example showing the positions of the left and upper samples involved in the CCLM mode and the samples of the current block, including the positions of the left and upper samples of the N×N chrominance block in the CU and the positions of the left and upper samples of the 2N×2N luminance block in the CU.

[0148] The division operation for calculating the parameter α is implemented using a lookup table. To reduce the memory required for storing the table, the diff value (the difference between the maximum and minimum values) and the parameter α are represented by exponential notation. For example, diff is approximated to 4 significant bits and an exponent. Thus, for 16 values of the significand as follows, the table for 1 / diff is reduced to 16 elements:

[0149] DivTable[] = {0, 7, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0} (5)

[0150] This will have the benefit of reducing both the computational complexity and the memory size required for storing the table.

[0151] In addition to the upper and left templates being used together to calculate the linear model coefficients, they can alternatively be used in two other LM modes (referred to as the LM_A and LM_L modes).

[0152] In the LM_T mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is extended to (W + H) samples. In the LM_L mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is extended to (H + W) samples.

[0153] In the LM_LT mode, the left template and the upper template are used to calculate the linear model coefficients.

[0154] To match the chrominance sample positions of the 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples to achieve a 2:1 downsampling ratio in both the horizontal and vertical directions. The selection of the downsampling filter is specified by the SPS level flag.

[0155] The two downsampling filters are as follows, corresponding to "type 0" and "type 2" content respectively.

[0156]

[0157] Note that when the above reference line is at the CTU boundary, only one luma line (the common line buffer in intra prediction) is used to obtain the downsampled luma samples.

[0158] This parameter calculation is performed as part of the decoding process, not just as an encoder search operation. Therefore, no syntax is used to convey the values of α and β to the decoder.

[0159] For chrominance intra mode coding, a total of 8 intra modes are allowed for chrominance intra mode coding. These modes include five traditional intra modes and three cross-component linear model modes (CCLM, LM_A, and LM_L). The chrominance mode signaling and derivation process are shown in Table 1. The chrominance mode coding directly depends on the intra prediction mode of the corresponding luma block. Since a separate block partitioning structure for the luma component and the chrominance component is enabled in the I slice, one chrominance block can correspond to multiple luma blocks. Therefore, for the chrominance DM mode, the intra prediction mode of the corresponding luma block covering the center position of the current chrominance block is directly inherited.

[0160] Table 1 - Deriving Chrominance Prediction Modes from Luma Modes when CCLM is Enabled

[0161]

[0162] Regardless of the value of sps_cclm_enabled_flag, a single binarization table is used, as shown in Table 2.

[0163] Table 2 - Unified Binarization Table for Chrominance Prediction Modes

[0164] Value of intra_chroma_pred_mode Binary bit string 4 00 0 0100 1 0101 2 0110 3 0111 5 10 6 110 7 111

[0165] In Table 2, the first binary bit indicates whether it is in the normal mode (0) or the LM mode (1). If it is in the LM mode, then the next binary bit indicates whether it is LM_CHROMA (0). If it is not LM_CHROMA, then the next binary bit indicates whether it is LM_L (0) or LM_A (1). For this case, when the sps_cclm_enabled_flag is 0, the first binary bit of the binary table for the corresponding intra_chroma_pred_mode can be discarded before entropy encoding / decoding. Or, in other words, the first binary bit is inferred to be 0 and thus not encoded / decoded. This single binary table is used for both cases where the sps_cclm_enabled_flag is equal to 0 and 1. The first two binary bits in Table 2 are context-encoded using their own context model, and the remaining binary bits are bypass-encoded.

[0166] Furthermore, to reduce the luminance-chrominance delay in the dual tree, when a 64x64 luminance coding tree node is split using no split (and ISP is not used for 64x64 CUs) or QT, the chroma CUs in the 32x32 / 32x16 chroma coding tree nodes are allowed to use CCLM in the following way:

[0167] – If the 32x32 chroma node is not split or split by a split QT, all chroma CUs in the 32x32 node can use CCLM

[0168] – If the 32x32 chroma node is split using a horizontal BT and the 32x16 child node is not split or split using a vertical BT, all chroma CUs in the 32x16 chroma node can use CCLM.

[0169] Under all other luminance and chroma coding tree split conditions, CCLM is not allowed for chroma CUs.

[0170] During ECM development, the simplified derivation (min-max approximation) of α and β was removed. Instead, a linear least squares solution derivation model for α and β between the causal reconstruction data of the downsampled luminance samples and the causal chroma samples.

[0171]

[0172] where Rec C (i) and Rec’ L (i) indicate the reconstructed chroma samples and the downsampled reconstructed luminance samples around the target block, and I indicates the total number of samples of the adjacent data.

[0173] The LM_A and LM_L modes are also known as the Multi-Directional Linear Model (MDLM). Figure 9A An example of MDLM operation when the block content cannot be predicted from the L-shaped reconstruction region is shown. Figure 9B MDLM_L which derives the CCLM parameters using only the left reconstruction samples is shown. Figure 9C MDLM_T which derives the CCLM parameters using only the top reconstruction samples is shown.

[0174] JCTVC-C206: Integration

[0175] The integration for the above-discussed Least Mean Square (LMS) (refer to equations (8)-(9)) has been proposed as an improvement to CCLM. The initial integration design of LMS CCLM was first proposed in JCTVC-C206. Then the method was improved through a series of simplifications, including reducing the α precision n α from 13 to 7 in JCTVC-F0233 / I0178, reducing the maximum multiplier bit width in JCTVC-I0151, and reducing the division LUT entries from 64 to 32 in JCTVC-H0490 / I0166, finally resulting in the ECM LMS version.

[0176] Basic algorithm

[0177] As discussed in equation (1), the integration design utilizes the linear relationship to model the correlation between the luminance signal and the chrominance signal. The chrominance value is predicted based on the reconstructed luminance values of the co-located blocks.

[0178] The luminance component and the chrominance component have different sampling rates in YUV420 sampling. The sampling rate of the chrominance component is half of the sampling rate of the luminance component and has a 0.5 pixel phase difference in the vertical direction. The reconstructed luminance needs to be downsampled in the vertical direction and subsampled in the horizontal direction to match the size of the chrominance signal. For example, the downsampling can be achieved as follows:

[0179] Rec L ′(i,j) = (rec L (2i,2j) + rec L (2i,2j + 1)) >> 1 (10)

[0180] Integer implementation

[0181] In equation (8), floating-point operations are required to calculate the linear model parameter α to maintain high data accuracy. And when α is represented by a floating-point value, floating-point multiplication is involved in equation (1). In this section, an integer implementation of the algorithm is designed. Specifically, using n αThe fractional part of the parameter α is quantified by the bit data accuracy. The value of the parameter α is represented by the enlarged and rounded integer value α' and a' = a × (1 << n α ). Then, the linear model of Equation (1) is changed to:

[0182] pred C [x,y] = (α' · Rec L '[x,y] >> n α ) + β' (11)

[0183] where β′ is the rounded value of the floating-point β, and α' can be calculated as follows.

[0184]

[0185] It is proposed to replace the division operation of Equation (12) by table lookup and multiplication. First, A2 is scaled down to reduce the table size. A1 is also scaled down to avoid product overflow. Then, in A2, only the most significant bits defined by the value are retained, and the other bits are set to zero. The approximate value A2' can be calculated as:

[0186]

[0187] where [...] means a rounding operation, and can be calculated as:

[0188]

[0189] where bdepth(A2) means the bit depth of the value A2.

[0190] The same operation is performed on A1 as follows:

[0191]

[0192] Considering the quantized representations of A1 and A2, Equation (12) can be rewritten as follows.

[0193]

[0194] where is represented as a lookup table of length to avoid division.

[0195] In the simulation, the constant parameters are set to:

[0196] ·n α equal to 13, which is a trade-off between data accuracy and computational cost.

[0197] · Equal to 6, resulting in a lookup table size of 64. When bdepth(A2) < 6 (e.g., A2 < 32), the table size can be further reduced to 32 by expansion.

[0198] ·n table Equal to 15, resulting in a 16-bit data representation of the table elements.

[0199] · Is set to 15 to avoid product overflow and maintain 16-bit multiplication.

[0200] Finally, α' is clipped to [-2 -15 , 2 15 -1] to maintain 16-bit multiplication in Equation (11). Using this clipping, when n α Is equal to 13, the actual a value is limited to [-4, 4), which is useful for preventing error magnification.

[0201] Using the calculated parameter α', the parameter β′ is calculated as follows:

[0202]

[0203] Where the division in the above equation can be simply replaced by a shift because the value I is a power of 2.

[0204] JCTVC-I0166: Simplified parameter calculation

[0205] Similar to that discussed above for Equation (1), in HM6.0, an intra prediction mode called LM is applied to use the reconstruction of the co-located luma PU and predict the chroma PU based on a linear model. The parameters of the linear model consist of the slope (a >> k) and the y-intercept (b) derived from adjacent luma and chroma pixels using the least mean square solution. The value of the predicted sample predSamples[x, y] is derived as follows, where x, y = 0…nS - 1, and nS specifies the block size of the current chroma PU:

[0206] predSamples[x, y] = Clip1 C (((p Y ’[x, y] * a) >> k) + b), where x, y = 0…nS - 1 (17)

[0207] Where P Y ’[x, y] is the reconstructed pixel from the corresponding luma component. When the coordinates x and y are equal to or greater than 0, P Y ’ is the reconstructed pixel from the co-located luma PU. When x or y is less than 0, P Y ’ is the reconstructed adjacent pixel of the co-located luma PU.

[0208] Some intermediate variables L, C, LL, LC, k2, and k3 during the derivation process are derived as follows:

[0209]

[0210] k2 = Log2((2 * nS) >> k3) (18 - 5)

[0211] k3 = Max(0, BitDepth C + Log2(nS) - 14) (18 - 6)

[0212] Therefore, variables a, b, and k can be derived as follows:

[0213] a1 = (LC << k2) – L * C (19 - 1)

[0214] a2 = (LL << k2) – L * L (19 - 2)

[0215] k1 = Max(0, Log2(abs(a2)) - 5) – Max(0, Log2(abs(a1)) - 14) + 2 (19 - 3)

[0216] a1s = a1 >> Max(0, Log2(abs(a1)) - 14) (19 - 4)

[0217] a2s = abs(a2 >> Max(0, Log2(abs(a2)) - 5)) (19 - 5)

[0218] a3 = a2s < 1? 0 : Clip3(-2 15 , 2 15 -1, a1s * lmDiv + (1 << (k1 - 1)) >> k1) (19 - 6)

[0219] a = a3 >> Max(0, Log2(abs(a3)) - 6) (19 - 7)

[0220] k = 13 – Max(0, Log2(abs(a)) - 6) (19 - 8)

[0221] b = (L – ((a * C) >> k1) + (1 << (k2 - 1))) >> k2 (19 - 9) where lmDiv is specified in a lookup table of 63 entries (i.e., Table 3), which is generated online as follows:

[0222] lmDiv(a2s) = ((1 << 15) + a2s / 2) / a2s (20)

[0223] Table 3 - Specification of lmDiv

[0224]

[0225]

[0226] In equation (19-6), a1s is a 16-bit signed integer, and lmDiv is a 16-bit unsigned integer. Therefore, a 16-bit multiplier and 16-bit storage are required. It is proposed to reduce the bit depth of the multiplier to an internal bit depth and reduce the size of the look-up table, as detailed below.

[0227] Reduced multiplier bit depth

[0228] The bit depth of a1s is reduced to the internal bit depth by changing equation (19-4) to the following:

[0229] a1s = a1 >> Max(0, Log2( abs( a1 ) ) – (BitDepth C – 2)) (21)

[0230] The value of lmDiv with the internal bit depth is implemented using the following equation (22) and stored in the look-up table:

[0231] lmDiv(a2s) = ( (1 << (BitDepth C -1)) + a2s / 2 ) / a2s (22)

[0232] Table 4 shows an example of an internal bit depth of 10.

[0233] Table 4 - Specification of lmDiv with an internal bit depth equal to 10

[0234] a2s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 lmDiv 512 256 171 128 102 85 73 64 57 51 47 43 39 37 34 32 a2s 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 lmDiv 30 28 27 26 24 23 22 21 20 20 19 18 18 17 17 16 a2s 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 lmDiv 16 15 15 14 14 13 13 13 12 12 12 12 11 11 11 11 a2s 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 lmDiv 10 10 10 10 10 9 9 9 9 9 9 9 8 8 8

[0235] Equations (19-3) and (19-8) are also modified as follows:

[0236] k1 = Max( 0, Log2( abs( a2 ) ) - 5 ) – Max( 0, Log2( abs( a1 ) ) –(BitDepth C – 2) ) (23-1)

[0237] k = BitDepth C – 1 – Max( 0, Log2( abs( a ) ) - 6 ) (23-2)

[0238] Reduced lookup table entries

[0239] It is also proposed to reduce the entries from 63 to 32 and reduce the bits per entry from 16 to 10, as shown in Table 5. By doing so, almost 70% memory savings can be achieved. The corresponding changes in Equation (19-6), Equation (20), and Equation (19-8) are as follows:

[0240] a3 = a2s < 32? 0 : Clip3( -2 15 , 2 15 -1, a1s*lmDiv + ( 1 << ( k1 - 1 )) >> k1 ) (24-1)

[0241] lmDiv(a2s) = ((1 << (BitDepth C +4)) + a2s / 2 ) / a2s (24-2)

[0242] k = BitDepth C + 4 – Max( 0, Log2( abs( a ) ) - 6 ) (24-3)

[0243] Table 5 - Specification of lmDiv with internal bit depth equal to 10

[0244] a2s 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 lmDiv 512 496 482 468 455 443 431 420 410 400 390 381 372 364 356 349 a2s 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 lmDiv 341 334 328 321 315 309 303 298 293 287 282 278 273 269 264 260

[0245] Multi-model linear model prediction

[0246] In ECM-1.0, a multi-model LM (MMLM) prediction mode is proposed, for which chrominance samples are predicted based on the reconstructed luminance samples of the same CU by using the following two linear models:

[0247]

[0248] where pred C (i,j) represents the predicted chrominance sample in the CU, and rec L ′(i,j) represents the downsampled reconstructed luminance sample of the same CU. Threshold is calculated as the average of adjacent reconstructed luminance samples. Figure 10 An example of classifying adjacent samples into two groups based on the value Threshold is shown. For each group, the parameters αi and βi (where i is equal to 1 and 2 respectively) are obtained from two samples within the group (which are the minimum luminance sample A(X A 、Y A ) and the maximum luminance sample B(X B 、Y B)) is derived from the linear relationship between the luminance value and the chrominance value. Here, X A 、Y A are the x - coordinate (i.e., the luminance value) and y - coordinate (i.e., the chrominance value) of sample point A, and X B 、Y B are the x - coordinate and y - coordinate of sample point B. The linear model parameters α and β are obtained according to the following equations.

[0249]

[0250] β = y A −αx A (26)

[0252] Such a method is also called the min - max method. The division in the above equations can be avoided and replaced by multiplication and shift.

[0253] For a coding block with a square shape, the above two equations are directly applied. For a non - square coding block, the adjacent sample points on the longer boundary are first downsampled to have the same number of sample points as those used for the shorter boundary.

[0254] Except for the scenario where the upper template and the left template are used together to calculate the linear model coefficients, the two templates can also alternatively be used in the other two MMLM modes (referred to as MMLM_A and MMLM_L modes).

[0255] In the MMLM_A mode, only the pixel sample points in the upper template are used to calculate the linear model coefficients. To obtain more sample points, the upper template is extended to a size of (W + W). In the MMLM_L mode, only the pixel sample points in the left template are used to calculate the linear model coefficients. To obtain more sample points, the left template is extended to a size of (H + H).

[0256] Note that when the above reference line is at the CTU boundary, only one luminance row (which is stored in the line buffer for intra - prediction) is used to obtain the downsampled luminance sample points.

[0257] For chrominance intra mode coding, a total of 11 intra modes are allowed for chrominance intra mode coding. These modes include five traditional intra modes and six cross-component linear model modes (CCLM, LM_A, LM_L, MMLM, MMLM_A, and MMLM_L). The chrominance mode signaling and derivation process are shown in Table 6. The chrominance mode coding directly depends on the intra prediction mode of the corresponding luma block. Since a separate block partitioning structure for the luma component and chrominance component is enabled in the I slice, a chrominance block can correspond to multiple luma blocks. Therefore, for the chrominance DM mode, the intra prediction mode of the corresponding luma block covering the center position of the current chrominance block is directly inherited.

[0258] Table 6 - Deriving Chrominance Prediction Modes from Luma Modes when MMLM_ is Enabled

[0259]

[0260] Adaptive enabling of LM and MMLM for prediction

[0261] The MMLM and LM modes can also be used together in an adaptive manner. For MMLM, the two linear models are as follows:

[0262]

[0263] where pred C (i,j) represents the predicted chrominance sample in the CU, and rec L ′(i,j) represents the downsampled reconstructed luma sample of the same CU. Threshold can be simply determined based on the luma and chrominance averages and their minimum and maximum values. Figure 11 An example of classifying adjacent samples into two groups based on the inflection point T indicated by the arrow is shown. The linear model parameters α1 and β1 are derived from the linear relationship between the luma values and chrominance values from two samples (which are the minimum luma samples A(X A 、Y A )) and Threshold(X T 、Y T ). The linear model parameters α2 and β2 are derived from the linear relationship between the luma values and chrominance values from two samples (which are the maximum luma samples B(X B 、Y B )) and Threshold(X T 、Y T ). Here, X A 、Y A are the x coordinate (i.e., luma value) and y coordinate (i.e., chrominance value) of sample A, and X B 、Y Bare the x - coordinate and y - coordinate values of sample point B. The linear model parameters α i and β i are obtained according to the following equations, where i equals 1 and 2 respectively.

[0264]

[0265] β2 = Y T −α2X T (28)

[0266] For the coding block with a square shape, the above equations are directly applied. For a non - square coding block, the adjacent sample points on the longer boundary are first downsampled to have the same number of sample points as that of the shorter boundary.

[0267] Except for the scenario where the upper template and the left template are used together to determine the linear model coefficients, the two templates can also be alternatively used in the other two MMLM modes (referred to as MMLM_A and MMLM_L modes respectively).

[0268] In the MMLM_A mode, only the pixel sample points in the upper template are used to calculate the linear model coefficients. To obtain more sample points, the upper template is extended to the size of (W + W). In the MMLM_L mode, only the pixel sample points in the left template are used to calculate the linear model coefficients. To obtain more sample points, the left template is extended to the size of (H + H).

[0269] Note that when the above reference line is at the CTU boundary, only one luminance row (which is stored in the line buffer for intra - frame prediction) is used to obtain the downsampled luminance sample points.

[0270] For chrominance intra - mode coding, there is a conditional check used to select the LM mode (CCLM, LM_A, and LM_L) or the multi - model LM mode (MMLM, MMLM_A, and MMLM_L). The conditional check is as follows:

[0271]

[0272] where BlkSizeThres LM represents the minimum block size of the LM mode, and BlkSizeThres MM represents the minimum block size of the MMLM mode. The symbol d represents a predetermined threshold value. In one example, d can be taken as 0. In another example, d can be taken as 8.

[0273] For chroma intra mode coding, a total of 8 intra modes are allowed for chroma intra mode coding. These modes include five traditional intra modes and three cross-component linear model modes. The chroma mode signaling and derivation process are shown in Table 1. It should be noted that for a given CU, if it is coded in the linear model mode, whether it is the conventional single model LM mode or the MMLM mode is determined based on the above condition check. Different from the case shown in Table 6, there is no separate MMLM mode to be signaled. The chroma mode coding directly depends on the intra prediction mode of the corresponding luma block. Since the separate block partitioning structure for the luma component and the chroma component is enabled in the I slice, one chroma block can correspond to multiple luma blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited.

[0274] Slope adjustment for CCLM

[0275] During the development of ECM, scale (slope) adjustment for CCLM was proposed as a further improvement, for example, as described in JVET-Y0055 / Z0049.

[0276] As discussed above, CCLM uses a model with 2 parameters to map luma values to chroma values. The scale parameter "a" and the bias parameter "b" are defined for the following mapping:

[0277] chromaVal = a * lumaVal + b (30)

[0278] It is proposed to signal the adjustment "u" of the scale parameter to update the model to the following form:

[0279] chromaVal = a’ * lumaVal + b’ (31)

[0280] where a’=a+u, and b’=b-u*y r .

[0281] With this selection, the mapping function tilts or rotates around the point with the luma value y r It is proposed to use the average of the reference luma samples used in model creation as y r so as to provide a meaningful modification to the model. Figures 12A to 12B The effect of the scale adjustment parameter "u" is shown, where, Figure 12A shows the model created without the scale adjustment parameter "u", and Figure 12B shows the model created with the scale adjustment parameter "u".

[0282] In one example, the scaling parameter is provided as an integer between -4 and 4, inclusive, and signaled in the bitstream. The unit of the scaling parameter is 1 / 8 of the chroma sample value per luma sample value (for 10-bit content).

[0283] In one example, the CCLM model that adjusts the reference samples ("LM_CHROMA_IDX" and "MMLM_CHROMA_IDX") available for both above and to the left of the block being used but not for the "unilateral" mode is adjusted. This selection is based on a trade-off between coding efficiency and complexity.

[0284] When applying scaling to the multi-mode CCLM model, two models can be adjusted and thus up to two scale updates can be signaled for a single chroma block.

[0285] To implement scaling at the encoder, the encoder can perform a SATD-based search for the optimal value of the scale update for Cr and a similar SATD-based search for Cb. If either result is a non-zero scaling parameter, the combined scaling adjustment pair (the SATD-based update for Cr, the SATD-based update for Cb) is included in the RD check list of the TU.

[0286] Fusion of chroma intra prediction modes

[0287] During the development of ECM, JVET-Y0092 / Z0051 proposed the fusion of chroma intra modes.

[0288] The intra prediction modes enabled for the chroma component in ECM-4.0 are six cross-component linear model (LM) modes (including CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L, and MMLM_T modes), the direct mode (DM), and four default chroma intra prediction modes. The four default modes are given by the list {0, 50, 18, 1}, and if the DM mode already belongs to the list, the mode in the list will be replaced with mode 66.

[0289] The decoder-side intra mode derivation (DIMD) method for luma intra prediction is included in ECM-4.0. First, the horizontal gradient and the vertical gradient are calculated for each reconstructed luma sample of the L-shaped template of the second adjacent row and column of the current block to construct the histogram of gradients (HoG). Then, the two intra prediction modes with the largest and the second largest histogram magnitude values are mixed with the planar mode to generate the final prediction value of the current luma block.

[0290] To improve the coding efficiency of chrominance intra prediction, two methods are proposed, including decoder-side derived chrominance intra prediction mode (DIMD chrominance) and the fusion of non-LM mode and MMLM_LT mode.

[0291] DIMD chroma mode

[0292] In the first embodiment, a DIMD chrominance mode is proposed. The proposed DIMD chrominance mode uses the DIMD derivation method to derive the chrominance intra prediction mode of the current block based on the co-located reconstructed luma samples. Specifically, as Figure 13 shown, the horizontal gradient and vertical gradient are calculated for each co-located reconstructed luma sample of the current chrominance block to construct the HoG. Then, the intra prediction mode with the maximum histogram magnitude value is used to perform the chrominance intra prediction of the current chrominance block.

[0293] When the intra prediction mode derived from the DIMD chrominance mode is the same as the intra prediction mode derived from the DM mode, the intra prediction mode with the second largest histogram magnitude value is used as the DIMD chrominance mode.

[0294] A CU-level flag is signaled to indicate whether the proposed DIMD chrominance mode is applied, as shown in Table 7.

[0295] Table 7. Binarization process of intra_chroma_pred_mode for the proposed method

[0296] intra_chroma_pred_mode Binary bit string Chroma intra mode 0 1100 List[0] 1 1101 List[1] 2 1110 List[2] 3 1111 List[3] 4 10 DIMD chroma 5 0 DM

[0297] Fusion of chroma intra prediction modes

[0298] In the second embodiment, the fusion of chrominance intra prediction modes is proposed, where the DM mode and four default modes can be fused with the MMLM_LT mode as follows:

[0299] pred = (w0 * pred0 + w1 * pred1 + (1 ≤ ≤ (shift - 1))) >> shift

[0300] where pred0 is the predicted value obtained by applying the non-LM mode, pred1 is the predicted value obtained by applying the MMLM_LT mode, and pred is the final predicted value of the current chrominance block. The two weights w0 and w1 are determined by the intra prediction modes of the adjacent chrominance blocks, and shift is set to be equal to 2. Specifically, when both the upper adjacent block and the left adjacent block are encoded with the LM mode, {w0, w1} = {1, 3}; when both the upper adjacent block and the left adjacent block are encoded with the non-LM mode, {w0, w1} = {3, 1}; otherwise, {w0, w1} = {2, 2}.

[0301] For syntax design, if the non-LM mode is selected, then a flag is signaled to indicate whether to apply fusion. And the proposed fusion is only applied to I slices.

[0302] Combination of DIMD chroma mode and fusion of chroma intra prediction modes

[0303] In the third embodiment, the fusion of the DIMD chroma mode and the intra-chroma prediction mode is combined. Specifically, the DIMD chroma mode described in the first embodiment is applied, and for I slices, the DM mode, the four default modes, and the DIMD chroma mode can be fused with the MMLM_LT mode using the weights described in the second embodiment, while for non-I slices, only the DIMD chroma mode can be fused with the MMLM_LT mode using equal weights.

[0304] Combination of DIMD chroma mode with reduced processing and fusion of chroma intra prediction modes

[0305] In the fourth embodiment, the fusion of the DIMD chroma mode with reduced processing and the intra-chroma prediction mode is combined. Specifically, the DIMD chroma mode with reduced processing derives the intra mode based on the neighboring reconstructed Y, Cb, and Cr samples in the second adjacent rows and columns, as Figure 14 shown. The other parts are the same as those in the third embodiment.

[0306] Decoder-side intra mode derivation (DIMD)

[0307] In one embodiment, when DIMD is applied, two intra modes are derived from the reconstructed neighboring samples, and using the weights derived from the gradient as described in JVET-O0449, these two predicted values are combined with the planar mode predicted value, as Figures 15A to Figure 15D shown. The division operation in the weight derivation is performed using the same lookup table (LUT)-based integration scheme used by CCLM. For example, the division operation Orient = G y / G x in the orientation calculation is computed by the following LUT-based scheme:

[0308] x = Floor(Log2(Gx))

[0309] normDiff = ((Gx << 4) >> x) & 15

[0310] x += (3 + (normDiff!= 0)? 1 : 0)

[0311] Orient = (Gy * (DivSigTable[normDiff] | 8) + (1 << (x - 1))) >> x

[0312] Among them

[0313] DivSigTable

[16] = {0, 7, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0}

[0314] The derived intra mode is included in the main list of the most probable intra modes (MPM), so the DIMD process is performed before constructing the MPM list. The main derived intra mode of the DIMD block is stored together with the block and is used for the construction of the MPM list of adjacent blocks.

[0315] Figures 15A to 15D The steps of intra mode derivation at the decoder side are shown, where the intra prediction direction is estimated in the absence of intra mode signaling. As Figure 15A shown in the first step includes estimating the gradient of each sample point (for the light gray sample points as shown in Figure 15A ). As Figure 15B shown in the second step includes mapping the gradient value to the closest prediction direction within [2, 66]. As Figure 15C shown in the third step includes selecting 2 prediction directions, where for each prediction direction, all the absolute gradients Gx and Gy of the adjacent pixels with that direction are added, and the top 2 directions are selected. As Figure 15D shown in the fourth step includes performing weighted intra prediction using the selected directions.

[0316] Multi-reference line (MRL) intra prediction

[0317] Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. In Figure 16 , an example of 4 reference lines is depicted, where the sample points of segments A and F are not extracted from the reconstructed adjacent sample points, but are filled with the closest sample points from segments B and E respectively. Intra prediction within the HEVC picture uses the closest reference line (i.e., reference line 0). In MRL, 2 additional lines (reference line 1 and reference line 3) are used.

[0318] The index (mrl_idx) of the selected reference line is signaled and is used to generate the intra prediction value. For reference line indices greater than 0, only the additional reference line modes in the MPM list are included, and only the mpm index is signaled in the absence of remaining modes. The reference line index is signaled before the intra prediction mode, and in the case of signaling a non-zero reference line index, the planar mode is excluded from the intra prediction mode.

[0319] Disable MRL for the first line of blocks within a CTU to prevent the use of extended reference samples outside the current CTU line. Additionally, disable PDPC when using additional lines. For the MRL mode, the derivation of the DC value in the DC intra prediction mode for non-zero reference line indices is consistent with the derivation of the DC value for reference line index 0. MRL requires storing 3 adjacent luma reference lines per CTU to generate predictions. The Cross-Component Linear Model (CCLM) tool also requires 3 adjacent luma reference lines for its downsampling filter. The definition of MRL using the same 3 lines is aligned with CCLM to reduce the storage requirements for the decoder.

[0320] Convolutional cross-component model (CCCM) for intra prediction

[0321] During ECM development, a Convolutional Cross-Component Model (CCCM) for chroma intra modes was proposed.

[0322] It is proposed to apply the Convolutional Cross-Component Model (CCCM) to predict chroma samples from the reconstructed luma samples in a similar spirit to that accomplished by the current CCLM mode. Similar to CCLM, when using chroma subsampling, the reconstructed luma samples are downsampled to match the lower resolution chroma grid.

[0323] In addition, similar to CCLM, there is an option to use a single model or a multi-model variant of CCCM. The multi-model variant uses two models, one model derived for samples above the average luma reference value and another model for the remaining samples (following the spirit of the CCLM design). The multi-model CCCM mode can be selected for a PU with at least 128 available reference samples.

[0324] Convolutional filter

[0325] The proposed convolutional 7-tap filter consists of a 5-tap plus-shaped spatial component, a non-linear term, and a bias term. The input to the spatial 5-tap component of the filter consists of the following terms: the center (C) luma sample (which is co-located with the chroma sample to be predicted) and its above / north (N), below / south (S), left / west (W), and right / east (E) neighbors, as Figure 17 shown.

[0326] The non-linear term P is expressed as the square of the center luma sample C and scaled to the sample value range of the content:

[0327] P = (C * C + midVal) >> bitDepth

[0328] That is, for 10-bit content, it is calculated as:

[0329] P = (C * C + 512) >> 10

[0330] The bias term B represents a scalar offset between the input and the output (similar to the offset term in CCLM) and is set to the middle chroma value (512 for 10-bit content).

[0331] The output of the filter is calculated as the convolution between the filter coefficients c i and the input values and is clipped to the range of valid chroma samples:

[0332] predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B

[0333] Calculation of filter coefficients

[0334] The filter coefficients c are calculated by minimizing the MSE between the predicted chroma samples and the reconstructed chroma samples in the reference region. i . Figure 18 A reference region consisting of 6 rows of chroma samples above and to the left of the PU is shown. The reference region extends one PU width to the right below the PU boundary and one PU height. The region is adjusted to include only available samples. An extension of the region shown in blue is required to support the "side samples" of the plus-shaped spatial filter and is filled when in an unavailable region.

[0335] The MSE minimization is performed by calculating the autocorrelation matrix of the luminance input and the cross-correlation vector between the luminance input and the chroma output. The autocorrelation matrix is LDL-factorized, and back substitution is used to calculate the final filter coefficients. This process generally follows the calculation of the ALF filter coefficients in ECM. However, LDL factorization is chosen instead of Cholesky factorization to avoid using square root operations. The proposed method uses only integer operations.

[0336] Bitstream signaling

[0337] The use of the mode is signaled using the CABAC-encoded PU-level flag. A new CABAC context is included to support this. When it comes to signaling, CCCM is considered a sub-mode of CCLM. That is, the CCCM flag is signaled only when the intra prediction mode is LM_CHROMA_IDX (to enable single-mode CCCM) or MMLM_CHROMA_IDX (to enable multi-model CCCM).

[0338] Encoder operation

[0339] The encoder performs two new RD checks in the chroma prediction mode loop, one for checking the single-model CCCM mode and one for checking the multi-model CCCM mode.

[0340] In existing CCLM or MMLM designs, adjacent reconstructed luma-chroma samples are classified into one or more sample groups based on a Threshold value that only considers the luma DC value. That is, the luma-chroma sample pairs are classified only by considering the intensity of the luma samples. However, the luma component typically retains a large amount of texture, and the current luma sample can be highly correlated with adjacent luma samples. Such inter-sample correlation (AC correlation) can be beneficial for the classification of luma-chroma sample pairs and can bring additional coding and decoding efficiency.

[0341] As Figure 19A shown, CCLM assumes that a given chroma sample is only correlated with the corresponding luma sample (L0.5, which can be regarded as a fractional luma sample position), and uses simple linear regression (SLR) estimated with ordinary least squares (OLS) to predict the given chroma sample. However, as Figure 19B shown, in some video content, a chroma sample can be simultaneously correlated with multiple luma samples (AC or DC related), so a multiple linear regression (MLR) model can further improve the prediction accuracy.

[0342] Although the CCCM mode can enhance the intra prediction efficiency, there is room for further improving its performance. At the same time, some parts of the existing CCCM mode also need to be simplified for efficient codec hardware implementation or improved for better coding and decoding efficiency. In addition, the trade-off between its implementation complexity and its coding and decoding efficiency benefits needs to be further improved.

[0343] Edge classification linear model (ELM)

[0344] To improve the encoding and decoding efficiency of the luminance component and the chrominance component, contrary to the above implementation that only considers the luminance DC value, a classifier that considers luminance edges or AC information is introduced. In addition to the existing MMLM with classification, the present disclosure provides exemplary classifiers. The process of generating the linear prediction model for different sample groups can be similar to CCLM or MMLM (e.g., via the least squares method or the simplified min-max method, etc.), but with different metrics for classification. Different classifiers can be used to classify adjacent luminance samples and / or the luminance samples corresponding to the chrominance samples to be predicted (e.g., of adjacent luminance-chrominance sample pairs). The luminance samples corresponding to the chrominance samples can be obtained by a downsampling operation to match the positions of the corresponding chrominance samples in a 4:2:0 video sequence. For example, the luminance samples corresponding to the chrominance samples can be obtained by performing a downsampling operation on more than one (e.g., 4) reconstructed luminance samples corresponding to the chrominance samples (e.g., located around the chrominance samples). Alternatively, for example, in the case of a 4:4:4 video sequence, the luminance samples can be directly obtained from the reconstructed luminance samples. Alternatively, the luminance samples can be obtained from the corresponding ones among the reconstructed luminance samples at the corresponding co-location positions of the corresponding chrominance samples. For example, the luminance sample to be classified can be obtained from one of the four reconstructed luminance samples corresponding to the chrominance sample at the upper left position of the four reconstructed luminance samples, which can be regarded as the co-location position for the chrominance sample.

[0345] The first classifier can classify the luminance samples according to the edge strength of the luminance samples. For example, a direction (e.g., 0 degrees, 45 degrees, or 90 degrees, etc.) can be selected to calculate the edge strength. The direction can be formed by the current sample and the adjacent sample along the direction (e.g., the adjacent sample at the upper right of the current sample at 45 degrees). The edge strength can be calculated by subtracting the neighbor sample from the current sample. The edge strength can be quantized into one of M segments by M - 1 thresholds, and the first classifier can use M categories to classify the current sample. Alternatively or additionally, N directions can be formed by the current sample and N adjacent samples along N directions. N edge strengths can be calculated by subtracting the N adjacent samples from the current sample respectively. Similarly, if each of the N edge strengths can be quantized into one of M segments by M - 1 thresholds, the first classifier can use MN categories to classify the current sample.

[0346] A second classifier can be used for classification according to local patterns. For example, the current luminance sample Y0 can be compared with N adjacent luminance samples Yi. If the value of Y0 is greater than the value of Yi, the score can be incremented by 1; otherwise, the score can be decremented by 1. The score can be quantized to form K categories. The second classifier can classify the current sample into one of the K categories. For example, the adjacent luminance samples can be obtained from four neighbors located above, to the left, to the right, and below the current luminance sample (i.e., no diagonal neighbors).

[0347] It is conceivable that multiple first classifiers, second classifiers, or different instances of the first classifier or the second classifier or other classifiers described herein can be combined. For example, the first classifier can be combined with an existing classifier based on the MMLM threshold. For another example, instance A of the first classifier can be combined with another instance B of the first classifier, where instances A and B adopt different directions (e.g., the vertical direction and the horizontal direction, respectively).

[0348] Those skilled in the art will understand that although the existing CCLM design in the VVC standard is used as the basic CCLM method in the description, the proposed cross-component method described in the present disclosure can also be applied to other predictive coding and decoding tools with a similar design concept. For example, for chrominance from luminance (CfL) in the AV1 standard, the proposed method can also be applied by dividing luminance-chrominance sample pairs into multiple sample groups.

[0349] Those skilled in the art will understand that Y / Cb / Cr can also be represented as Y / U / V in the field of video coding and decoding. If the video data is in RGB format, the proposed method can also be applied, for example, by simply mapping the YUV representation to GBR.

[0350] Filter-based linear model (FLM)

[0351] The following introduces a filter-based linear model (FLM) using an MLR model to consider the possibility that a chrominance sample can be simultaneously related to multiple luminance samples.

[0352] For the chrominance sample to be predicted, the reconstructed co-located and adjacent luminance samples can be used to predict the chrominance sample to capture the inter-sample correlation between the co-located luminance sample, the adjacent luminance sample, and the chrominance sample. The reconstructed luminance samples are linearly weighted and combined with an "offset" to generate the predicted chrominance sample (C: predicted chrominance sample, L i : the i-th reconstructed co-located or adjacent luminance sample, α i(where α: filter coefficient, β: offset, N: filter tap), as shown in the following equation (32-1). Note that linear weighting plus the offset value directly forms the predicted chrominance sample (which can be low-pass or high-pass depending on the video content), and then it is added to the residual to form the reconstructed chrominance sample.

[0353]

[0354] In some implementations similar to CCCM, the offset term can also be implemented as the intermediate chrominance value B (512 for 10-bit content) multiplied by another coefficient, as shown in the following equation (32-2).

[0355]

[0356] For a given CU, the top and left reconstructed luma and chroma samples can be used to derive or train the FLM parameters (α i, , β). Similar to CCLM, α i and β can be derived via OLS. The top training samples and left training samples are collected, and a pseudo-inverse matrix is calculated at both the encoder side and the decoder side to derive the parameters, and then these parameters are used to predict the chroma samples in a given CU. Let N denote the number of filter taps applied to the luma samples, M denote the total number of top and left reconstructed luma-chroma sample pairs used for training the parameters, denote the luma sample with the i-th sample pair and the j-th filter tap, and C i denote the chroma sample with the i-th sample pair. The following equations show the derivation of the pseudo-inverse matrix A + and also the parameters. Figure 20 Shows an example where N is 6 (6-tap), M is 8, the top 2 rows and left 3 columns of luma samples and the top 1 row and left 1 column of chroma samples are used to derive or train the parameters.

[0357]

[0358] b = Ax

[0359] x = (A T A) -1 A T b = A + b (33)

[0360] Note that the chroma samples can be predicted by only α i without the offset β, which can be a subset of the proposed method.

[0361] Note that, although the existing CCLM design in the VVC standard is used as the basic CCLM method in the following description, for those skilled in the art of video coding and decoding, the proposed cross-component method described in this disclosure can also be applied to other predictive coding tools with a similar design concept. For example, for the chroma from luma (CfL) in the AV1 standard, the proposed FLM can also be applied by including multiple luma samples into the MLR model.

[0362] The proposed ELM / FLM / GLM (discussed below) can be directly extended to the CfL design in the AV1 standard, which explicitly sends the model parameters (α,β). For example, (in the 1-tap case) at the encoder, α and / or β are derived at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level and signaled to the decoder for the CfL mode.

[0363] Filter shape

[0364] To further improve the coding and decoding performance, additional designs can be used in the FLM prediction. As Figure 20 shown and discussed above, a 6-tap luma filter is used for the FLM prediction. However, although the multi-tap filter can well fit the training data (e.g., the top and left adjacent reconstructed luma and chroma samples), in some cases, the training data does not capture all the characteristics of the test data, but it may lead to overfitting and may not be able to well predict the test data (i.e., the chroma block samples to be predicted). Moreover, different filter shapes can well adapt to different video block contents, resulting in more accurate predictions.

[0365] To solve this problem, the filter shape and the number of filter taps can be predefined, or signaled or switched at the sequence parameter set (SPS), adaptive parameter set (APS), picture parameter set (PPS), picture header (PH), slice header (SH), region, CTU, CU, sub-block, or sample level. The set of filter shape candidates can be predefined, and the selection of the set of filter shape candidates can be signaled or switched at the SPS, APS, PPS, PH, SH, region, CTU, CU, sub-block, or sample level. Different components (e.g., U and V) can have different filter switching controls. For example, the set of filter shape candidates (e.g., indicated by indices 0 to 5) can be predefined, and filter shape (1, 2) can represent a 2-tap luma filter, filter shape (1, 2, 4) can represent a 3-tap luma filter, etc., as in Figure 20As shown in. The filter shape selection for the U and V components can be switched in the PH or at the CU or CTU level. Note that N taps can represent N taps with or without the offset β as described herein. An example is given in Table 8 below.

[0366] Table 8 - Exemplary Signaling and Switching for Different Filter Shapes

[0367]

[0368] The FLM or CCCM filter shape can include non - linear terms. For example, for the CCCM filter, the following equation can be used to predict the chroma sample value:

[0369] predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B, P=(C * C + midVal)>>bitDepth

[0370] Wherein, the filter corresponds to the weighted coefficients c0, c1,..., c6 of the central (C) luma sample value (which is co - located with the chroma sample to be predicted), its upper / north (N), lower / south (S), right / east (E) and left / west (W) neighbors, the non - linear term P and the bias term B. The values used to derive the non - linear term P can be a combination of the current and adjacent luma samples, but are not limited to C * C. For example, P can be derived as follows:

[0371] P=(Q * R + midVal)>>bitDepth

[0372] Wherein, Q and R represent the values used to derive the non - linear term P.

[0373] Q and R can be a linear combination of the current and adjacent luma samples in the down - sampled domain (e.g., Q and R are pre - processed luma samples obtained through a weighted average operation) or without any down - sampling process.

[0374] For example, each of Q and R can be selected from one of the N, S, E, W, and C luma sample values, e.g., Q * R = C * N, C * S, C * E, C * W, S * N or N * N, etc.; or both Q and R can be equal to the average of the N, S, E, and W luma sample values, i.e., Q = R=(N + S + E + W) / 4; or Q is equal to the C luma sample value and R is equal to the average of the N, S, E, and W luma sample values, i.e., Q = C and R=(N + S + E + W) / 4.

[0375] The different values (Q and R) used to derive the non - linear terms are considered different filter shapes and can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub - block / sample level. The set of filter shape candidates can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub - block / sample level.

[0376] Different chroma types and / or color formats can have different predefined filter shapes and / or taps. For example, the predefined filter shapes (1, 2, 4, 5) can be used for 4:2:0 type 0, the predefined filter shapes (0, 1, 2, 4, 7) can be used for 4:2:0 type 2, and the predefined filter shapes (1, 4) can be used for 4:2:2, and the predefined filter shapes (0, 1, 2, 3, 4, 5) can be used for 4:4:4, as Figure 21 shown.

[0377] In another aspect of the present disclosure, the unavailable luma and chroma samples for deriving the MLR model can be filled from the available reconstructed samples. For example, if using a 6 - tap (0, 1, 2, 3, 4, 5) filter as in Figure 21 , for a CU located at the left picture boundary, the left column including samples (0, 3) is unavailable (outside the picture boundary), so the sample (0, 3) is a duplicate fill from samples (1, 4) to apply the 6 - tap filter. Note that the filling process can be applied to training data (top and left adjacent reconstructed luma and chroma samples) and test data (luma and chroma samples in the CU).

[0378] The number of one or more shapes / filter taps can be used for FLM prediction, examples are shown in Figure 25 , Figure 26 and Figures 27A to Figures 27B shown. One or more sets of filter taps can be used for FLM prediction, examples are shown in Figures 28A to Figures 28G shown.

[0379] Implicit filter shape derivation

[0380] The filter shape candidates can be implicitly derived without explicit signaling bits. For example, the filter shape candidates can be the filter shape candidates of FLM or GLM (discussed below). In another example, the filter shape candidates can be the cross - shaped filter for CCCM, Figure 25 , Figure 26 , Figure 27A , Figure 27B andFigures 28A to 28G Any of the filters shown or other filters mentioned in this disclosure. Since longer filter taps always theoretically fit better in the training data (template region), but may overfit, the "N-fold cross-validation" technique well-known in the machine learning field can be used to train the filter coefficients. This technique divides the available training data into N sets, and uses some sets for training and other sets for validation.

[0381] The following example relates to implicit filter shape derivation for FLM prediction:

[0382] Step 1: Determine M filter shape candidates for predicting the chrominance sample values of the current CU;

[0383] Step 2: Divide the available L-shaped template region outside the CU into N regions, denoted as R0, R1,... R N-1 , that is, divide the training data into N sets for N-fold training, where the luminance sample values and chrominance sample values of the available template region are known values;

[0384] Step 3: Apply each of the M filter shape candidates to a part of the available template region, that is, one or more of all N regions R0, R1,... R N-1 ;

[0385] Step 4: Derive M sets of filter coefficients corresponding to the M filter shape candidates, denoted as F0, F1,... F M- 1;

[0386] Step 5: Apply the derived sets of F0, F1,... F M-1 filter coefficients to another part of the available template region to predict the chrominance sample values based on

[0387] the corresponding luminance sample values, where the other part of the available template region is different from the part of the available template region mentioned in Step 3;

[0388] Step 6: For each of the M filter shapes, accumulate the errors (denoted as E0, E1,... E M-1 ) between the predicted chrominance sample values and the known chrominance sample values in another part of the available template region by the sum of absolute differences (SAD), sum of squared differences (SSD), or sum of absolute transform differences (SATD);

[0389] Step 7: Sort and select K minimum errors (denoted as E’0, E’1,... E’ K-1 ), which correspond to K filter shapes and K sets of filter coefficients; and

[0390] Step 8: Select one filter shape candidate from the K filter shape candidates to be applied to the current CU for chrominance prediction. If K is greater than 1, the decoder can still receive a signal indicating the applied filter from the encoder. However, if K is 1, the signaling can be omitted, and the filter with the minimum accumulated error is determined as the applied filter.

[0391] Figure 29A and Figure 29B illustrates an example of dual training for implicit filter shape derivation. For the current chrominance CU prediction (blue region), Figure 29A illustrates that in the template region, the even row region R0 (yellow) is used for training / deriving 4 sets of filter coefficients, and the odd row region R1 (red) is used for verification / comparison of the 4 sets of filter coefficients and sorting of the costs; Figure 29B illustrates that in the template region, R0 (yellow) and R1 (red) are interleaved. It should be understood that R0 and R1 can be swapped in these examples.

[0392] In one example, one of the 4 filter shape candidates is to be selected as the applied filter, and the L-shaped template region is divided into even-numbered and odd-numbered rows or columns. These steps include:

[0393] Pre-define 4 filter shape candidates for the current CU (e.g., 4 filter shape candidates from Figure 25 );

[0394] Divide the available L-shaped template region (e.g., 6 chrominance rows and columns for CCCM, note that in the CCCM design, each chrominance sample involves 6 luminance samples for downsampling) into 2 regions denoted as R0, R1, where, for example, R0 consists of even rows or columns, and R1 consists of odd rows or columns ( Figure 29A illustrates the following example: in the template region, the even row region R0 is used for training / deriving 4 sets of filter coefficients, and the odd row region R1 is used for verification / comparison and sorting of the costs of the 4 sets of filter coefficients);

[0395] Independently apply the 4 filter shape candidates to a part of the available template region (e.g., region R0);

[0396] Derive 4 sets of filter coefficients for the 4 filter shapes, denoted as F0, F1,... F3;

[0397] Apply the derived sets of filter coefficients F0, F1,... F3 to another part of the available template region (e.g., region R1) respectively to predict the corresponding chrominance sample values;

[0398] For each of the four filters, accumulate the errors (denoted as E0, E1, …, E3) between the predicted chroma sample values and the known chroma sample values in another part of the available template region (e.g., region R1) by SAD, SSD, or SATD; and

[0399] Select one filter shape candidate (denoted as E’0) among the four filter shape candidates with the minimum accumulated error among E0, E1, …, E3, which corresponds to a filter shape and a set of filter coefficients. In this example, only the filter shape candidate with the minimum accumulated error will be determined as the applied filter, and then the decoder does not need to receive a signal indicating the applied filter.

[0400] In one example, the L-shaped template region can be divided into interleaved parts when K = 2. These steps include:

[0401] Determine four filter shape candidates for the current CU;

[0402] Divide the available L-shaped template region (e.g., 6 chroma rows or columns for CCCM) into two regions denoted as R0 and R1, where the luma samples in R0 and R1 are interleaved as shown in the following table:

[0403] <![CDATA[R0]]> <![CDATA[R1]]> <![CDATA[R1]]> <![CDATA[R0]]>

[0404] or

[0405] <![CDATA[R1]]> <![CDATA[R0]]> <![CDATA[R0]]> <![CDATA[R1]]>

[0406] and Figure 29B Show an example where in the template region, R0 and R1 are interleaved;

[0407] Apply the four filter shape candidates independently to a part of the available template region (e.g., region R0);

[0408] Derive four sets of filter coefficients (denoted as F0, F1, …, F3) for the four filter shapes respectively;

[0409] Apply the derived sets of F0, F1, …, F3 filter coefficients to another part of the available template region (e.g., region R1) respectively to predict the corresponding chroma sample values;

[0410] For each of the four filter shapes, accumulate the errors (denoted as E0, E1, …, E3) between the predicted chroma sample values and the known chroma sample values in another part of the available template region (e.g., region R1) by SAD, SSD, or SATD;

[0411] Sort and select two smallest accumulated errors (denoted as E'0, E'1) from E0, E1, …… E3, which correspond to two filter shapes and two sets of filter coefficients; and

[0412] Based on the signal received from the encoder, select one filter shape candidate from two filter shape candidates to be applied to the current CU for chrominance prediction.

[0413] Note that the method of implicit filter shape derivation can also be used to determine whether to introduce a non - linear term in the CCCM filter coefficients (treating with / without non - linear terms as different filter shapes).

[0414] Although the above shows examples for the CCCM filter, it should be understood that the non - linear term P can also be included in the FLM filter (e.g., the 3*2 filter as shown in Figure 20 ), and be derived in a similar way as discussed above.

[0415] In one example, the step of dividing the available L - shaped template region can be omitted. In this example, M sets of filter coefficients can be derived based on the sample values from the available template region, and then be applied back to the available template region respectively to predict the corresponding chrominance sample values for the accumulated error.

[0416] Matrix derivation

[0417] As mentioned above, the MLR model (linear equation) must be derived at both the encoder and the decoder. According to one or more aspects of the present disclosure, several methods are proposed to derive the pseudo - inverse matrix A + , or directly solve the linear equation. Other known methods such as Newton's method, Cayley - Hamilton method and eigenvalue decomposition (as mentioned in https: / / en.wikipedia.org / wiki / Invertible_matrix ) can also be applied.

[0418] In the present disclosure, for simplicity, A + can be denoted as A -1 . The linear equation can be solved as follows:

[0419] 1. Solve A -1 through the adjoint matrix (adjA), closed - form, analytical solution:

[0420] The following shows an nxn general form, a 2x2 and a 3x3 case. If the FLM uses 3x3, two scalers plus an offset need to be solved.

[0421] b = Ax, x=(A T A)-1 A T b = A + b, represented as A -1 b

[0422]

[0423] By removing the (n - 1) x (n - 1) sub - matrix of the j - th row and the i - th column

[0424]

[0425] 2. Gauss - Jordan elimination method

[0426] The Gauss - Jordan elimination method can be used to solve the linear equations through the augmented matrix [A I n and a series of elementary row operations to obtain the reduced row echelon form matrix [I|X]. 2x2 and 3x3 examples are shown below.

[0427]

[0428] 3. Cholesky decomposition

[0429] To solve Ax = b, A can first be decomposed by the Cholesky - Crout algorithm, resulting in an upper triangular matrix and a lower triangular matrix, and a forward substitution plus a back substitution can be applied serially to obtain the solution. A 3x3 example is shown below.

[0430]

[0431] In addition to the above examples, some conditions require special handling. For example, if some conditions lead to the inability to solve the linear equations, default values can be used to fill the chrominance prediction values. For example, when 1 << (bitDepth - 1), meanC, meanL, or meanC - meanL (average current chrominance or other chrominance, from available luminance values, or a subset of the FLM - reconstructed adjacent regions) is predefined, the default value can be predefined, or signaled or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub - block / sample level.

[0432] The following example represents the case when the matrix A cannot be solved, where the default prediction value can be assigned to the entire current block:

[0433] 1. Solve by closed - form (analytical, adjoint matrix), but A is singular (i.e., detA = 0);

[0434] 2. Solve by Cholesky decomposition, but A cannot be Cholesky decomposed, g jj <REG_SQR, where REG_SQR is a small value, which can be predefined, or signaled or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0435] Applied area

[0436] Figure 20 Illustrate a typical case of deriving FLM parameters using the top 2 luma lines and / or the left 3 luma lines and the top 1 chroma line and / or the left 1 chroma line. However, as mentioned above, due to different block contents and the reconstruction quality of different neighboring samples, using different regions for parameter derivation can bring coding and decoding benefits. The following presents several ways to select the regions applied for parameter derivation:

[0437] 1. Similar to MDLM, FLM derivation can use only the top or left luma and / or chroma samples to derive parameters. Whether to use FLM, FLM_L, or FLM_T can be predefined, or signaled or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Assume that the current chroma block has a size of W×H, then W' and H' are obtained as follows:

[0438] – When applying the FLM mode, W’ = W, H’ = H;

[0439] – When applying the FLM_T mode, W’ = W + We; where We represents the extended top luma / chroma samples;

[0440] – When applying the FLM_L mode, H’ = H + He; where He represents the extended left luma / chroma samples.

[0441] The number of extended luma / chroma samples (We, He) can be predefined, or signaled or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0442] For example, predefined (We, He) = (H, W) as VVC CCLM, or predefined (W, H) as ECM CCLM. Unavailable (We, He) luma / chroma samples can be filled by repeating the nearest (horizontal, vertical) luma / chroma samples.

[0443] Figure 22Show the description of FLM_L and FLM_T (e.g., less than 4 taps). When applying FLM_L or FLM_T, only H’ or W’ luminance / chrominance samples are used for parameter derivation respectively.

[0444] 2. Similar to MRL, different line indices can be predefined, signaled, or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level to indicate the selected luminance-chrominance sample pairs. This can benefit from the different reconstruction qualities of different line samples.

[0445] Figure 23 Show that similar to MRL, FLM can use different lines for parameter derivation (e.g., less than 4 taps). For example, FLM can use the light blue / yellow luminance and / or chrominance samples of index 1.

[0446] 3. Expand the CCLM region and use all the top N rows and / or the left M rows for parameter derivation. Figure 23 Show that all the dark blue and light blue and dark yellow and light yellow regions can be used at once. Training with a larger region (data) can result in a more robust MLR model.

[0447] It should be understood that the luminance sample values of the external region of the video block to be decoded can be referred to as "external luminance sample values", and the chrominance sample values of the external region can be referred to as "external chrominance sample values" throughout the present disclosure.

[0448] Syntax

[0449] The corresponding syntax can be defined in Table 9 as follows for FLM prediction. Wherein, FLC represents fixed length code, TU represents truncated unary code, EGk represents exponential Golomb code of order k, where k can be fixed, or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level, SVLC represents signed EG0, and UVLC represents unsigned EG0.

[0450] Example of Table 9 - FLM syntax

[0451]

[0452]

[0453] Note that the binarization of each syntax element can be changed.

[0454] Gradient linear model (GLM)

[0455] A new method for cross-component prediction is proposed based on the existing linear model design to further improve the encoding and decoding accuracy and efficiency. The main aspects of the proposed method are detailed as follows.

[0456] Although the FLM discussed above provides the best flexibility (leading to the best performance), if the number of filter taps goes up, many unknown parameters need to be solved. When the inverse matrix is ​​larger than 3×3, the closed form derivation is not suitable (too many multipliers) and an iterative method like Cholesky's is required, which will burden the decoder processing cycle. In this section, a pre-operation before applying the linear model is proposed, including exploiting the correlation between the luminance AC information and the chrominance intensity using the sample gradients. With the help of the gradients, the number of filter taps can be effectively reduced.

[0457] Please note that the methods / examples in this section can be combined / reused from any of the designs discussed above, including but not limited to classification, filter shapes, matrix derivation (with special processing), application areas, syntax. In addition, the methods / examples listed in this section can also be applied to any of the designs discussed above to have better performance and certain complexity tradeoffs.

[0458] Please note that the reference samples / training templates / reconstructed neighboring regions used in this article generally refer to luma samples used to derive MLR model parameters, which are then applied to internal luma samples in a CU to predict chroma samples in the CU.

[0459] Filter shape

[0460] According to the proposed method, instead of directly using luma sample intensity values ​​as input to the linear model, pre-operations (e.g., pre-linear weighting, sign, scaling / abs, thresholding, ReLU) can be applied to reduce the dimensionality of unknown parameters. In one example, the pre-operations can include calculating sample differences based on luma sample values. As understood by those skilled in the art, sample differences can be characterized as gradients, and therefore the new method is also referred to as a gradient linear model (GLM) in some embodiments.

[0461] Please note that the following detailed description discusses scenarios in which the proposed pre-operation can be reused for the SLR model (also known as the 1-tap case) / combined with the SLR model, and can be reused for the MLR model (also known as the multi-tap case, e.g., 2-tap) / combined with the MLR model.

[0462] For example, instead of applying a 2-tap to the 2 luma samples, the 2 luma samples may be pre-operated on and then a simpler 1-tap may be applied to reduce complexity. Figures 24A to 24DShows some examples for 1-tap / 2-tap (with offset) pre-operations, where the 2-tap coefficients are represented as (a, b). Note that each circle shown as in Figures 24A to 24D represents an illustrative chrominance position in the YUV 4:2:0 format. As discussed above, in the YUV 4:2:0 format, the luminance sample corresponding to a chrominance sample can be obtained by performing a downsampling operation on more than one (e.g., 4) reconstructed luminance samples corresponding to (e.g., around) the chrominance sample. In other words, the chrominance position can correspond to one or more luminance samples including co-located luminance samples. Different 1-tap patterns are designed for gradient calculation for different gradient directions and using different "interpolated" luminance samples (weighted to different luminance positions). For example, in Figure 24A , 24C and 24D show a typical filter [1, 0, -1; 1, 0, -1], which represents the following operations:

[0463]

[0464] where, rec L represents the reconstructed luminance sample value and Rec L ″(i,j) represents the pre-operation luminance sample value. Also note that the 1-tap filters shown as in Figure 24A , Figure 24C and Figure 24D can be understood as an alternative to the downsampling filter with modified filter coefficients used in CCLM (please refer to equations (6)-(7)).

[0465] The pre-operation can be based on gradient, edge direction (detection), pixel intensity, pixel change, pixel variance, Roberts / Prewitt / compass / Sobel / Laplacian operator, high-pass filter (by calculating gradient or other related operators), low-pass filter (by performing weighted average operation), etc. The edge direction detectors listed in the examples can be extended to different edge directions. For example, 1-tap (1, -1) or 2-tap (a, b) are applied along different directions to detect different edge gradients. The filter shape / coefficients can be symmetric about the chrominance position, such as the example in Figures 24A to 24D (420 type-0 case).

[0466] Pre - operation parameters (coefficients, signs, scaling / absolute value, thresholding, ReLU) can be fixed, or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub - block / sample level. Note that in the example, if multiple coefficients are applied to a sample (e.g., - 1, 4), they can be combined (e.g., 3) to reduce operations.

[0467] In one example, the pre - operation can involve calculating the sample difference of the luma sample values. Alternatively, the pre - operation can include performing downsampling through a weighted average operation. In some cases, the pre - operation can be applied repeatedly. For example, a template filter can be applied to a template using a low - pass smoothing FIR filter [1, 2, 1] / 4 or [1, 2, 1; 1, 2, 1] / 8 (i.e., downsampling) to remove outliers, and then a 1 - tap GLM filter can be applied to calculate the sample difference to derive a linear model. It is conceivable that the sample difference can also be calculated and then downsampling can be implemented.

[0468] In one example, the pre - operation coefficients (eventually applied (e.g., 3) or intermediate applied (e.g., - 1, 4) to each luma sample) can be limited to values that are powers of 2 to save multipliers.

[0469] In one aspect of the present disclosure, the proposed new method can be reused in combination with the CCLM discussed above, which utilizes a simple linear regression (SLR) model and uses a corresponding luma sample value to predict a chroma sample value. This is also referred to as the 1 - tap case. In this case, deriving the linear model also includes deriving the scale parameter α and the offset parameter β by using the adjacent luma sample values and adjacent chroma sample values of the pre - operation. Alternatively, the linear model can be rewritten as:

[0470] C = α·L+β (35) where L here represents the "pre - operation" luma sample. The parameter derivation of the 1 - tap GLM can reuse the CCLM design, but consider the directional gradient (which can have a high - pass filter). In one example, the scale parameter α can be derived by utilizing a division lookup table detailed below to achieve simplification.

[0471] In one example, when combining the GLM with the SLR model, the scale parameter α and the offset parameter β can be derived by utilizing the min - max method discussed above. Specifically, the scale parameter α and the offset parameter β can be derived by: comparing the adjacent luma sample values of the pre - operation to determine the minimum luma sample value Y A and the maximum luma sample value Y B ; respectively determining the corresponding chroma sample values X for the minimum luma sample value Y A and the maximum luma sample value Y B determine the corresponding chroma sample values XA and X B ; and based on the minimum luminance sample value Y A , the maximum luminance sample value Y B and the corresponding chrominance sample value X A and X B , the scale parameter α and the offset parameter β are derived according to the following equation:

[0472]

[0473] β = Y A - αX A (36)

[0474] In one example, when combining GLM with the SLR model, the scaling discussed above can be reused. In this case, the encoder can determine the scaling value (e.g., "u") to be signaled in the bitstream and add the scaling value to the derived scale parameter α. The decoder can determine the scaling value (e.g., "u") from the bitstream and add the scaling value to the derived scale parameter α. The added value is ultimately used to predict the internal chrominance sample value.

[0475] In one aspect of the present disclosure, the proposed new method can be reused for or combined with FLM, which utilizes a multiple linear regression (MLR) model and uses multiple luminance sample values to predict chrominance sample values. This is also referred to as the multi-tap case, e.g., 2-tap. In this case, the linear model can be rewritten as:

[0476]

[0477] C M-1 = α·L M-1 + β

[0478]

[0479] b = Ax (37)

[0480] x = (A T A) -1 A T b = A + b

[0481]

[0482] In this case, multiple scale parameters α and offset parameter β can be derived by using pre - processed adjacent luminance sample values and adjacent chrominance sample values. In one example, the offset parameter β is optional. In one example, at least one of the multiple scale parameters α can be derived by exploiting sample differences. Additionally, another one of the multiple scale parameters α can be derived by exploiting the down - sampled luminance sample values. In one example, at least one of the multiple scale parameters α can be derived by using horizontal or vertical sample differences calculated based on the down - sampled adjacent luminance sample values. In other words, the linear model can combine multiple scale parameters α associated with different pre - operations.

[0483] Implicit filter shape derivation

[0484] In one example, instead of explicitly signaling the selected filter shape index, the direction - oriented filter shape used can be derived at the decoder to save bit overhead. For example, at the decoder, several directional gradient filters can be applied to each reconstructed luminance sample of the L - shaped template for the i - th adjacent row and column of the current block. Then, the filtered values (gradients) can be accumulated separately for each direction of the several directional gradient filters. In the example, the accumulated value is the accumulated value of the absolute values of the corresponding filtered values. After the accumulation, the direction of the directional gradient filter for which the accumulated value is the largest can be determined as the derived (luminance) gradient direction. For example, a gradient histogram (HoG) can be constructed to determine the maximum value. The derived direction can be further applied as the direction for predicting chrominance samples in the current block.

[0485] The following example involves reusing the decoder - side intra - mode derivation (DIMD) method used for intra - frame prediction of luminance included in ECM - 4.0:

[0486] Step 1: Apply 2 directional gradient filters (3x3 horizontal / vertical Sobel) to each reconstructed luminance sample of the L - shaped template for the 2nd adjacent row and column of the current block;

[0487] Step 2: Accumulate the filtered values (gradients) of each of the directional gradient filters by SAD (sum of absolute differences);

[0488] Step 3: Construct a gradient histogram (HoG) based on the accumulated filtered values; and

[0489] Step 4: The maximum value in the HoG is determined as the derived (luminance) gradient direction, based on which the GLM filter can be determined.

[0490] In one example, if the shape candidates are [-1, 0, 1; -1, 0, 1] (horizontal) and [1, 2, 1; -1, -2, -1] (vertical), when the maximum value is associated with the horizontal shape, the shape [-1, 0, 1; -1, 0, 1] is used for GLM-based chrominance prediction.

[0491] The gradient filter used to derive the gradient direction can be the same as or different from the GLM filter in shape. For example, both filters can be horizontal [-1, 0, 1; -1, 0, 1], or the two filters can have different shapes, and the GLM filter can be determined based on the gradient filter.

[0492] Classification

[0493] The proposed GLM can be combined with the MMLM or ELM discussed above. When combined with classification, each group can share or have its own filter shape, where the syntax indicates the shape of each group. For example, as an exemplary classifier, the horizontal gradient grad_hor can be classified into a first group corresponding to a first linear model, and the vertical gradient grad_ver can be classified into a second group corresponding to a second linear model. In one example, the horizontal luminance style can be generated only once.

[0494] Further possible classifiers are also provided as follows. Using the classifier, the adjacent and intra-luminance-chrominance sample pairs of the current video block can be classified into multiple groups based on one or more thresholds. Note that, as discussed above, each adjacent / intra-chrominance sample and its corresponding luminance sample can be referred to as a luminance-chrominance sample pair. One or more thresholds are associated with the intensity of the adjacent / intra-luminance samples. In this case, each group in the multiple groups corresponds to a respective one of the multiple linear models.

[0495] When combined with the MMLM classifier, the following operations can be performed: classify the adjacent reconstructed luminance-chrominance sample pairs of the current video block into 2 groups based on Threshold; derive different linear models for different groups, where the derivation process can be GLM-simplified, i.e., using the above pre-operations to reduce the number of taps; similarly classify the luminance-chrominance sample pairs (intra-luminance-chrominance sample pairs, where each intra-luminance-chrominance sample pair in the intra-luminance-chrominance sample pairs includes an intra-chrominance sample value to be predicted using the derived linear model) within the CU into 2 groups based on Threshold; apply different linear models to the reconstructed luminance samples in different groups; and predict the chrominance samples in the CU based on the different classified linear models.

[0496]

[0497] where, rec L ′(i,j) can be the downsampled reconstructed luminance sample; rec C (i,j) can be the reconstructed chrominance sample (note: only neighbors are available); Threshold can be the average of adjacent reconstructed luminance samples. Note that the number of classes (2) can be extended to multiple classes by increasing the number of thresholds (e.g., equally divided based on the minimum / maximum of adjacent reconstructed (downsampled) luminance samples, fixed, or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level).

[0498] In one example, instead of the MMLM luminance DC intensity, the filtered value of the FLM / GLM applied to adjacent luminance samples is used for classification. For example, if a 1-tap (1, -1) GLM is applied, the average AC value (physical meaning) is used. The process can be: classifying adjacent reconstructed luminance-chrominance sample pairs into K groups based on one or more filter shapes, one or more filtered values, and K - 1 thresholds Ti; deriving different MLR models for different groups, where the derivation process can be GLM simplified, i.e., using the above pre-operations to reduce the number of taps; similarly classifying the luminance-chrominance sample pairs (internal luminance-chrominance sample pairs, where each internal luminance-chrominance sample pair in the internal luminance-chrominance sample pairs includes the internal chrominance sample value to be predicted using the derived linear model) within the CU into K groups based on one or more filter shapes, one or more filtered values, and K - 1 thresholds Ti; applying different linear models to the reconstructed luminance samples in different groups; predicting the chrominance samples in the CU based on different classification linear models. Where, Threshold can be predefined (e.g., 0, or can be a table), or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, Threshold can be the average AC value (filtered value) of adjacent reconstructed (which can be downsampled) luminance samples (2 groups), or equally divided based on the minimum / maximum AC (K groups).

[0499] It is also proposed to combine GLM with an ELM classifier. As Figures 24A to 24D shown, one filter shape (e.g., 1-tap) can be selected to calculate the edge intensity. The direction is determined as the direction along which the sample differences between the current sample and the samples of N adjacent samples (e.g., all 6 luminance samples) are calculated. For example, Figure 24AThe filter (shape [1, 0, -1; 1, 0, -1]) at the upper middle in it indicates the horizontal direction because the sample differences between the samples in the horizontal direction can be calculated, while the filter (shape [1, 2, 1; -1, -2, -1]) below it indicates the vertical direction because the sample differences between the samples in the vertical direction can be calculated. The positive and negative coefficients in each filter in the filter implement the calculation of the sample differences. Then, the processing may include: calculating an edge strength through the filtered values (e.g., equivalent); quantizing the edge strength into M segments through M - 1 thresholds Ti; classifying the current sample using K classes (e.g., K == M); deriving different MLR models for different groups, where the derivation process may be GLM simplified, i.e., using the above pre-operations to reduce the number of taps; classifying the luminance-chrominance sample pairs within the CU into K groups; applying different MLR models to the reconstructed luminance samples in different groups; and predicting the chrominance samples in the CU based on the MLR models of different classifications. Note that the filter shape used for classification may be the same as or different from the filter shape used for MLR prediction. Both the number M - 1 of thresholds and the threshold values Ti may be fixed, or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Additionally, other classifiers / combined classifiers discussed in ELM may also be used for FLM and / or GLM.

[0500] If the classified samples in a group are less than a quantity (e.g., predefined 4), the default values mentioned when discussing the matrix derivation of the MLR model can be applied to the group parameters (α i, , β). If the corresponding adjacent reconstructed samples are not available for the selected LM mode, then the default values can be applied. For example, when the MMLM_L mode is selected but the left sample is invalid.

[0501] Simplification and unification

[0502] To further improve the coding and decoding efficiency, several methods related to the simplification of GLM are introduced as follows.

[0503] Matrix / parameter derivation in FLM requires floating-point operations (e.g., closed-form division), which are expensive for decoder hardware. Therefore, a fixed-point design is needed. For the 1-tap GLM case, it can be regarded as a modified luminance reconstruction sample generation of CCLM (e.g., horizontal gradient direction, from CCLM[1, 2, 1; 1, 2, 1] / 8 to GLM[-1, 0, 1; -1, 0, 1]). The original CCLM process can be reused for GLM, including fixed-point operations, MDLM downsampling, division tables, application of size limits, min-max approximation, and scaling. For all entries, the 1-tap GLM can have its own configuration or share the same design as CCLM. For example, a simplified min-max method is used to derive the parameters (instead of LMS), and it is combined with scaling after the GLM model is derived. In this case, the center point (luminance value y r ) for rotating the slope becomes the average of the reference luminance samples "gradient". Another example, when the GLM is enabled for this CU, it is inferred that the CCLM slope adjustment is disabled and there is no need to signal the syntax related to slope adjustment.

[0504] For example, this section adopts typical case reference samples (1 row above and 1 column to the left). Note that, as Figure 23 shown, the extended reconstruction area can also use simplifications with the same spirit and can have syntax indicating specific areas (such as MDLM, MRL).

[0505] Note that the following aspects can be combined and applied jointly. For example, combining reference sample downsampling and division tables to perform the division process.

[0506] When applying classification (MMLM / ELM), the same or different simplification operations can be applied to each group. For example, before applying a right shift, the samples of each group are filled to the target sample number respectively, and then the same derivation process and the same division table are applied.

[0507] Note that the method of implicit filter shape derivation can also be used to determine whether to disable the downsampling process in the CCCM filter coefficients (treating the use / non-use of the downsampling process as different filter shapes).

[0508] Fixed-point implementation

[0509] The 1-tap case can reuse the CCLM design. Division by n can be achieved by a right shift, and division by A2 can be achieved by a LUT. Including n involved in the integrated design of LMS CCLM α 、n A1 、n A2 、r A1 、r A2 、ntable The integration parameters for the intermediate parameters (equations (19)-(20)) used to derive the linear model can be the same as or have different values from those of the CCLM to achieve higher precision. The integration parameters can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level, and can be conditional on the sequence bit depth. For example, n table = bitdepth + 4.

[0510] MDLM downsampling

[0511] When the GLM is combined with the MDLM, the total number of samples present for parameter derivation may not be a power-of-two value, and padding to a power-of-two is required to replace division with a right-shift operation. For example, for an 8x4 chroma CU, the MDLM requires W + H = 12 samples, where only 8 samples (reconstructed) are available for the MDLM_T, and then the downsampled 4 samples (0, 2, 4, 6) can be equally padded. The code for implementing such an operation is shown below:

[0512]

[0513] Other padding methods can also be applied, such as repetitive / mirror padding relative to the last adjacent sample (rightmost / lowest).

[0514] The padding method for the GLM can be the same as or different from that for the CCLM.

[0515] Note that in the ECM version, for an 8x4 chroma CU, the MDLM_T / MDLM_L require 2T / 2L = 16 / 8 samples respectively. In such a case, the same padding method can be applied to meet the target power-of-two number of samples.

[0516] Division LUT

[0517] The division LUTs proposed for the CCLM / LIC (local illumination compensation) in the development of known standards such as AVC / HEVC / AV1 / VVC / AVS can be used for GLM division. For example, reusing the LUT in JCTVC-I0166 for the case of bit depth = 10 (Table 4). The division LUT can be different from that of the CCLM. For example, the CCLM uses the min-max method with a division table as in Equation 5, but the GLM uses a 32-entry LMS division LUT as in Table 5.

[0518] When GLM is combined with MMLM, the meanL value may not always be positive (e.g., using filtered / gradient values to classify groups), so it is necessary to extract sgn(meanL), and use abs(meanL) to look up the division LUT. Note that the division LUT used for MMLM classification and parameter derivation can be different. For example, a lower-precision LUT (such as the LUT in the min-max method) is used for mean classification, and a higher-precision LUT (such as in LMS) is used for parameter derivation.

[0519] Size limit and latency constraint

[0520] Similar to the CCLM design, some size limits can be applied to ELM / FLM / GLM. For example, the same constraints can be applied to the luminance-chrominance delay in the dual tree.

[0521] The size limits can be based on the CU area / width / height / depth. The thresholds can be predefined or signaled at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, for the chrominance CU area, the predefined threshold can be 128.

[0522] In one example, at least one pre-operation is performed in response to determining that a video block meets an enabling threshold, where the enabling threshold is associated with the area, width, height, or partition depth of the video block. Specifically, the enabling threshold can define the minimum or maximum area, width, height, or partition depth of the video block. As understood by those skilled in the art, the video block can include the current chrominance block and its co-located luminance block. It is also proposed to jointly apply the above enabling threshold to the current chrominance block and its co-located luminance block. For example, at least one pre-operation is performed in response to determining that the enabling threshold is met for both the current chrominance block and its co-located luminance block.

[0523] Line buffer reduction

[0524] Similar to the CCLM design, if the co-located luminance region of the current chrominance CU contains the first row within a CTU, the generation of the top template samples can be limited to 1 row to reduce the CTU row line buffer storage. Note that when the above reference line is at the CTU boundary, only one luminance line (the general line buffer in intra prediction) is used to obtain the downsampled luminance samples.

[0525] For example, in Figure 22In [specific context], if the co-located luma region of the current chroma CU contains the first row within a CTU, the top template can be restricted to using only 1 row (but not 2) for parameter derivation (other CUs can still use 2 rows). When processing the CTU line by line at the decoder hardware, this saves luma sample line buffer storage. Several methods can be used to achieve line buffer reduction. Note that the example of the limited "1" row can be extended to N rows with similar operations. Similarly, 2-tap or multi-tap can also apply such operations. When applying multi-tap, chroma samples may also need to apply operations.

[0526] For example, Figure 24A the 1-tap filter [1, 0, -1; 1, 0, -1] shown in [reference] is used as an example for illustration. This filter can be reduced to [0, 0, 0; 1, 0, -1], that is, only the coefficients of the lower row are used. Alternatively, the luma samples of the limited upper row can be filled from the luma samples of the lower row (repetition, mirroring, 0, meanL, meanC, etc.).

[0527] Taking N = 4 as an example, that is, the video block is located at the top boundary of the current CTU, and the adjacent luma sample values and corresponding chroma sample values of the top 4 rows are used to derive the linear model. Note that the corresponding chroma sample values can refer to the adjacent chroma sample values of the corresponding top 4 rows (for example, for the YUV 4:4:4 format). Alternatively, the corresponding chroma sample values can refer to the adjacent chroma sample values of the corresponding top 2 rows (for example, for the YUV 4:2:0 format). In this case, the adjacent luma sample values and corresponding chroma sample values of the top 4 rows can be divided into two regions: the first region including valid sample values (for example, the luma sample values and corresponding chroma sample values of a nearest row) and the second region including invalid sample values (for example, the luma sample values and corresponding chroma sample values of the other three rows). Then the coefficients of the filter corresponding to the sample positions that do not belong to the first region can be set to zero, so that only the sample values from the first region are used to calculate the sample difference. For example, as discussed above, in this case, the filter [1, 0, -1; 1, 0, -1] can be reduced to [0, 0, 0; 1, 0, -1]. Alternatively, the nearest sample values in the first region can be filled into the second region, so that the filled sample values can be used to calculate the sample difference.

[0528] Fusion of chrominance intra prediction modes

[0529] In one example, since GLM can be regarded as a special CCLM mode, the fusion design can be reused or have its own way. Multiple (two or more) weights can be applied to generate the final prediction value. For example,

[0530] pred = (w0 * pred0 + w1 * pred1 + (1 << (shift - 1))) >> shift

[0531] where pred0 is a prediction value based on a non-LM mode, and pred1 is a prediction value based on a GLM, or

[0532] pred0 is a prediction value based on one of the CCLMs (including all MDLM / MMLM), and pred1 is a prediction value based on a GLM, or

[0533] pred0 is a prediction value based on a GLM, and pred1 is a prediction value based on a GLM.

[0534] Depending on whether the adjacent blocks are encoded using CCLM / GLM / other coding modes or block size / width / height, different I / P / B slices can have different designs for the weights w0 and w1.

[0535] For example, the design for the weights can be determined by the intra prediction mode of the adjacent chrominance blocks, and the shift is set to be equal to 2. Specifically, when both the upper and left adjacent blocks are encoded using the LM mode, then {w0, w1} = {1, 3}; when both the upper and left adjacent blocks are encoded using a non-LM mode, then {w0, w1} = {3, 1}; otherwise {w0, w1} = {2, 2}. For non-I slices, both w0 and w1 can be set to be equal to 2.

[0536] For the syntax design, if the non-LM mode is selected, a flag is signaled to indicate whether fusion is applied.

[0537] 1-tap linear model

[0538] As described above, the 1-tap GLM has a good gain-complexity trade-off because it can reuse the existing CCLM module without introducing additional derivations. According to one or more aspects of the present disclosure, such a 1-tap design can be further extended or generalized.

[0539] In one aspect of the present disclosure, for the chrominance samples to be predicted, a single corresponding luminance sample L can be generated by combining the co-located luminance sample and adjacent luminance samples. For example, the combination can be a combination of different linear filters, such as, for example, a combination of a high-pass gradient filter (GLM) and a low-pass smoothing filter (e.g., the [1, 2, 1; 1, 2, 1] / 8 FIR downsampling filter that can typically be used in CCLM); and / or a combination of a linear filter and a non-linear filter (e.g., having an nth power, e.g., L n, n can be a positive number, a negative number, or a +- fraction (e.g., +1 / 2, square root, or +3, cube, which can be rounded and rescaled to the bit-depth dynamic range)).

[0540] In one aspect of the present disclosure, the combination can be applied repeatedly. For example, the combination of GLM and [1, 2, 1; 1, 2, 1] / 8 FIR can be applied to the reconstructed luminance samples, and then a non-linear power of 1 / 2 can be applied. For example, the non-linear filter can be implemented as a LUT (look-up table). For example, for bit-depth = 10, n-th power, n = 1 / 2, LUT[i] = (int)(sqrt(i)+0.5)<<5, i = 0 to 1023, where 5 will be scaled to the dynamic range of bit-depth = 10. When the linear filter cannot efficiently handle the luminance-chrominance relationship, the non-linear filter can provide an option. Whether to use the non-linear term can be predefined, or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0541] In one or more of the above aspects of the present disclosure, GLM can refer to a generalized linear model (which can be used to linearly or non-linearly generate a single luminance sample, and the single generated luminance sample can be fed into the CCLM linear model to derive the parameters of the CCLM linear model), and the linear / non-linear generation can be referred to as a general style. Different gradients or general styles can be combined to form another style. For example, the gradient style can be combined with the CCLM downsampling value; the gradient style can be combined with the non-linear L 2 value; the gradient style can be combined with another gradient style. The two gradient styles to be combined can have different directions or the same direction. For example, [1, 1, 1; -1, -1, -1] and [1, 2, 1; -1, -2, -1], which both have a vertical direction, can be combined, or [1, 1, 1; -1, -1, -1] and [1, 0, -1; 1, 0, -1], which have vertical and horizontal directions, can be combined, as Figures 24A to 24D shown. The combination can include addition, subtraction, or linear weighting.

[0542] GLM applied to the downsampling domain

[0543] As described above, the pre - operations can be applied repeatedly, and the GLM can be applied on the pre - linearly weighted / pre - operated samples. For example, similar to CCLM, a template filtering can be applied to the luminance samples to remove outliers and generate down - sampled luminance samples (a down - sampled luminance sample corresponding to a chrominance sample) using a low - pass smoothing FIR filter [1, 2, 1; 1, 2, 1] / 8 (i.e., the CCLM down - sampling smoothing filter). And after that, a 1 - tap GLM can be applied to the smoothed down - sampled luminance samples to derive the MLR model.

[0544] Some gradient filter patterns, such as 3x3 Sobel or Prewitt operators, can be applied to the down - sampled luminance samples. The following table shows some of the gradient filter patterns.

[0545]

[0546]

[0547] The gradient filter patterns can be combined with other gradient / general filter patterns in the down - sampled luminance domain. In one example, the combined filter pattern can be applied to the down - sampled luminance samples. For example, the combined filter pattern can be derived by performing addition or subtraction operations on the corresponding coefficients of the gradient filter pattern and a DC / low - pass - based filter pattern (such as filter pattern [0, 0, 0; 0, 1, 0; 0, 0, 0] or [1, 2, 1; 2, 4, 1; 1, 2, 1]). In another example, the combined filter pattern is derived by performing addition or subtraction operations on the coefficients of the gradient filter pattern and a non - linear value such as L 2 The combined filter pattern is derived by performing addition or subtraction operations on the coefficients of the gradient filter pattern and another gradient filter pattern with a different or the same direction. In another example, the combined filter pattern is derived by performing a linear weighting operation on the coefficients of the gradient filter pattern.

[0548] The GLM applied on the down - sampled domain can be adapted to the CCCM framework, but may sacrifice high - frequency accuracy because of the low - pass smoothing applied before applying the GLM.

[0549] GLM used as the input to CCCM

[0550] As shown above, similar to CCLM, CCCM applies luminance downsampling before convolution. When chroma subsampling is used, the reconstructed luminance samples are downsampled to match the lower-resolution chroma grid. Since the 1-tap GLM can also be regarded as changing the CCLM downsampling filter coefficients (e.g., from [1, 2, 1; 1, 2, 1] / 8 to [1, 2, 1, -1, -2, -1], i.e., from low-pass to high-pass), GLM can be used as the input to CCCM. Specifically, the gradient filter of GLM uses gradient-based coefficients (e.g., [1, 2, 1, -1, -2, -1]) to replace the luminance downsampling filter ([1, 2, 1; 1, 2, 1] / 8). In this case, the CCCM operation becomes a "linear / nonlinear combination of gradients", as shown by the following equation:

[0551] predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B

[0552] where C, N, S, E, W, P are the gradients of the current or adjacent samples (compared to the original downsampled values of CCCM). The related GLM methods described in this disclosure can be applied in the same way before entering the CCCM convolution, e.g., classification, separate Cb / Cr control, syntax, style combination, PU size limit, etc.

[0553] The gradient-based coefficient replacement can be applied to specific CCCM taps. In addition, not only high-pass coefficient replacement can be used, but also low-pass / band-pass / all-pass coefficient replacement can be used. The replacement can be combined with the FLM / CCCM shape switch discussed above (resulting in a different number of taps). For example, Figures 24A to 24D the gradient style in can be used for replacement. In one example, the operations for applying GLM as the input to CCCM include: predefining one or more coefficient candidates for CCCM / FLM downsampling; determining the CCCM / FLM filter shape and the number of filter taps for the CU; applying different CCLM downsampling coefficients to different filter taps, where the coefficients can be high-pass filters (GLM) or low-pass / band-pass / all-pass filters; generating the downsampled luminance samples for the CCCM input samples (using the applied coefficients); and feeding the generated downsampled luminance samples into the CCCM process.

[0554] Some examples for changing the CCLM downsampling filter coefficients are shown below:

[0555] Example 1:

[0556] The candidate filters are [1, 2, 1; 1, 2, 1] / 8 and [1, 0, -1; 1, 0, -1]; predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B, using a typical CCCM cross shape, 7-tap; N, S, W, E use the filter [1, 2, 1; 1, 2, 1] / 8, maintaining the original CCLM downsampling filter; and C, P use the filter [1, 0, -1; 1, 0, -1], i.e., the horizontal gradient filter, and then P physically means the gradient ^ 2.

[0557] Example 2:

[0558] The candidate filters are [1, 2, 1; 1, 2, 1] / 8, [1, 0, -1; 1, 0, -1], [1, 2, 1; -1, -2, -1], [2, 1, -1; 1, 1, -1, -2] and [-1, 1, 2; -2, -1, 1]; predChromaVal = c0C0 + c1C1 + c2C2 + c3C3 + c4C4 + c5P + c6B; C0 uses the filter [1, 2, 1; 1, 2, 1] / 8, maintaining the original CCLM downsampling filter; C1 uses the filter [1, 0, -1; 1, 0, -1]; C2 uses the filter [1, 2, 1; -1, -2, -1]; C3 uses the filter [2, 1, -1; 1, -1, -2]; C4 uses the filter [-1, 1, 2; -2, -1, 1]; C5 uses the filter [1, 2, 1; 1, 2, 1] / 8, maintaining the original CCLM downsampling filter; C0 to C5, and P have the same downsampled luminance position (in the typical CCCM cross shape == C); and C1 to C4 are generated by Sobel-based gradient filters based on different directions (as Figures 24A to 24D shown).

[0559] Example 3:

[0560] The candidate filters are [1, 2, 1; 1, 2, 1] / 8, [1, 0, -1; 1, 0, -1], [1, 2, 1; -1, -2, -1], [0, 1, 1; 0, 1, 1], and [1, 1, 0; 1, 1, 0]; redChromaVal = c0C0 + c1C1 + c2C2 + c3C3 + c4C4 + c5P + c6B; C0 uses the filter [1, 2, 1; 1, 2, 1] / 8, maintaining the original CCLM downsampling filter; C1 uses the filter [1, 0, -1; 1, 0, -1]; C2 uses the filter [1, 2, 1; -1, -2, -1]; C3 uses the filter [0, 1, 1; 0, 1, 1]; C4 uses the filter [1, 1, 0; 1, 1, 0]; C5 uses the filter [1, 2, 1; 1, 2, 1] / 8, maintaining the original CCLM downsampling filter; C0 to C5, and P have the same downsampled luma position (in the typical CCCM cross shape == C); C1 to C2 are generated by a gradient filter based on Sobel in different directions (as shown in Figures 24A to 24D as shown); and C3 to C4 are generated by a low-pass filter.

[0561] Which CCCM / FLM taps for applying coefficient replacement can be predefined (as in the above examples), or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0562] For each CCCM / FLM tap, the coefficient candidates for CCCM / FLM downsampling can be predefined, or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0563] Example 4:

[0564] The candidate filters are [1, 2, 1; 1, 2, 1] / 8, [1, 0, -1; 1, 0, -1], [1, 2, 1; -1, -2, -1], [2, 1, -1; 1, -1, -2], and [-1, 1, 2; -2, -1, 1]; predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B, using the typical CCCM cross shape, 7 taps; C uses a switched downsampling filter among 5 candidate filters; N, S, W, E use the filter [1, 2, 1; 1, 2, 1] / 8, maintaining the original CCLM downsampling filter; and P uses a switched downsampling filter among 5 candidate filters.

[0565] Example 5:

[0566] The candidate filters are: [1, 2, 1; 1, 2, 1] / 8, [1, 0, -1; 1, 0, -1], [1, 2, 1; -1, -2, -1], [0, 1, 1; 0, 1, 1], and [1, 1, 0; 1, 1, 0]; predChromaVal = c0C + c1W + c2E + c3P + c4B, i.e., a horizontal minus sign shape, 5-tap; C uses the switched-downsampling filter among the 5 candidate filters; W and E use the switched-downsampling filters among the 3 candidate filters: [1, 2, 1; 1, 2, 1] / 8, [0, 1, 1; 0, 1, 1], and [1, 1, 0; 1, 1, 0]; and P uses the filter [1, 2, 1; 1, 2, 1] / 8, maintaining the original CCLM downsampling filter.

[0567] Syntax

[0568] In one or more aspects of the present disclosure, one or more syntaxes may be introduced to indicate information about the GLM. Examples of the GLM syntax are shown in Table 10 below.

[0569] Table 10

[0570]

[0571]

[0572] FLC: Fixed Length Code

[0573] TU: Truncated Unary Code

[0574] EGk: Exponential Golomb code of order k, where k may be fixed, or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0575] SVLC: Signed EG0

[0576] UVLC: Unsigned EG0

[0577] Note that the binarization of each syntax element may be changed.

[0578] In one aspect of the present disclosure, the GLM on / off control for Cb / Cr components can be either jointly or separately. For example, at the CU level, a single flag can be used to indicate whether GLM is active for that CU. If it is active, a single flag can be used to indicate whether both Cb / Cr are active. If it is not active, the single flag indicating Cb or Cr is active. When Cb and / or Cr are active, the filter index / gradient (generic) pattern can be signaled separately. All flags can have their own context models or be bypass-coded.

[0579] In another aspect of the present disclosure, whether to signal the GLM on / off flag can depend on the luminance / chrominance coding mode and / or CU size. For example, in the ECM5 chrominance intra-frame mode syntax, when applying MMLM or MMLM_L or MMLM_T, GLM can be inferred as off; when the CU area < A, where A can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level; if combined with CCCM, when CCCM is on, GLM can be inferred as off.

[0580] Note that when GLM is combined with MMLM, different models can share the same gradient / generic pattern or have their own gradient / generic pattern.

[0581]

[0582] When GLM is combined with CCCM / FLM, if the current CU is implemented as CCCM / FLM, then the CU-level GLM enable flag can be inferred as off.

[0583] hasGlmFlag &=!pu.cccmFlag;

[0584] CCCM without using downsampling processing

[0585] CCCM needs to process the downsampled luminance reference values before calculating the model parameters and applying the CCCM model, which burdens the decoder processing cycle. In this section, a CCCM that does not utilize the downsampling process is proposed, including different options that utilize non-downsampled luminance references and / or non-downsampled luminance references. One or more filter shapes can be used for the purposes described below.

[0586] In one example, a convolutional 7-tap filter may include a 5-tap plus-shaped spatial component, a non-linear term, and a bias term. The input to the spatial 5-tap component of the filter includes a central (C) non-downsampled luminance sample (which is co-located with the chrominance sample to be predicted) and its non-downsampled upper or north (N), lower or south (S), left or west (W), and right or east (E) neighbors, as Figure 17 shown.

[0587] The non-linear term P is represented as the square of the central luminance sample C and scaled to the range of the sample values of the content:

[0588] P = (C * C + midVal) >> bitDepth

[0589] That is, for 10-bit content, it is calculated as:

[0590] P = (C * C + 512) >> 10

[0591] The bias term B represents a scalar offset between the input and the output (similar to the offset term in CCLM) and is set to the mid-chrominance value (512 for 10-bit content).

[0592] The output of the filter is calculated as the convolution between the filter coefficients ci and the input values and is clipped to the range of valid chrominance samples:

[0593] predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B

[0594] In another example, a convolutional 7-tap filter may include a 6-tap rectangular-shaped spatial component and a bias term. The input to the spatial 6-tap component of the filter includes a central (b) non-downsampled luminance sample (which is co-located with the chrominance sample to be predicted) and its downsampled lower-left or southwest (d), lower-right or southeast (f), lower or south (e), left or west (a), and right or east (c) neighbors, as Figure 25 shown in Shape 1.

[0595] The bias term B represents a scalar offset between the input and the output (similar to the offset term in CCLM) and is set to the mid-chrominance value (512 for 10-bit content).

[0596] The output of the filter is calculated as the convolution between the filter coefficients ci and the input values and is clipped to the range of valid chrominance samples:

[0597] predChromaVal = c0a + c1b + c2c + c3d + c4e + c5f + c6B

[0598] In yet another example, a convolutional 8-tap filter can consist of a 6-tap rectangular-shaped spatial component, a non-linear term, and a bias term. The input to the spatial 6-tap component of the filter consists of: a center (b) non-downsampled luma sample (which is co-located with the chroma sample to be predicted) and its non-downsampled lower-left or southwest (d), lower-right or southeast (f), lower or south (e), left or west (a), and right or east (c) neighbors, as Figure 25 shown in Shape 1 of

[0599] The non-linear term P is expressed as the square of the center luma sample (b) and scaled to the sample value range of the content:

[0600] P = (b * b + midVal) >> bitDepth

[0601] That is, for 10-bit content, it is calculated as:

[0602] P = (b * b + 512) >> 10

[0603] The bias term B represents a scalar offset between the input and the output (similar to the offset term in CCLM) and is set to the middle chroma value (512 for 10-bit content).

[0604] The output of the filter is calculated as the convolution between the filter coefficients ci and the input values and is clipped to the range of valid chroma samples:

[0605] predChromaVal = c0a + c1b + c2c + c3d + c4e + c5f + c6P + c7B

[0606] It should be understood that the examples shown above are only sample examples, and other implementations may be possible without departing from the present disclosure, such as having more or fewer taps and having Figure 25 any of the shapes shown in

[0607] In yet another example, a convolutional 9-tap filter can consist of a 6-tap rectangular-shaped spatial component, two non-linear terms, and a bias term. The input to the spatial 6-tap component of the filter consists of: a center (b) non-downsampled luma sample (which is co-located with the chroma sample to be predicted and is substantially at the center of the filter shape) and its non-downsampled lower-left / southwest (d), lower-right / southeast (f), lower / south (e), left / west (a), and right / east (c) neighbors, as Figure 25 shown in Shape 1 of

[0608] The non - linear terms P and Q are two non - linear luma sample values, which are respectively represented as the powers of the luma sample values of the center (b) luma sample and the powers of the luma sample values of the lower / south (e) luma sample, and then scaled to the sample value range of the content:

[0609] P = (b * b + midVal) >> bitDepth;

[0610] Q = (e * e + midVal) >> bitDepth.

[0611] That is to say, for 10 - bit content, it is calculated as:

[0612] P = (b * b + 512) >> 10;

[0613] Q = (e * e + 512) >> 10.

[0614] The bias term B is a bias value representing the scalar offset between the input and the output (similar to the offset term in CCLM) and is set to the middle chroma value (512 for 10 - bit content).

[0615] The output of the filter is calculated as the convolution between the filter coefficients ci and the input values and is clipped to the range of valid chroma samples:

[0616] predChromaVal = c0a + c1b + c2c + c3d + c4e + c5f + c6P + c7Q + c8B

[0617] It should be understood that the non - linear terms P and Q can be represented as the powers of any luma sample value of the non - decimated luma samples of the filter. The two non - linear terms P and Q are only exemplary, and the corresponding chroma sample values can be calculated based on one or more non - linear values.

[0618] In another example, as shown in Figure 30 the convolutional 9 - tap filter can include a 6 - tap spatial term, two non - linear terms, and a bias term. The 6 - tap spatial term corresponds to 6 adjacent luma samples (i.e., L0, L1,..., L5) of the chroma sample to be predicted (i.e., the triangle in Figure 30 ). Note that the two adjacent original reconstructed luma samples used in the non - linear terms can be replaced by any two adjacent original reconstructed luma samples, i.e., without downsampling as shown in Figure 30 (i.e., L0, L1,..., L5).

[0619]

[0620] where α i is related to L iassociated coefficients, and β is an offset. Similar to the existing CCCM design, up to 6 rows / columns of chroma samples above and to the left of the current CU are used to derive the filter coefficients. The filter coefficients are derived based on the same LDL decomposition method used in the CCCM. The convolutional 9-tap filter can be signaled as an additional CCCM model in addition to the existing CCCM model. For signaling, when CCCM is selected, a single flag is signaled and used for both chroma components to indicate whether to apply the default CCCM model or the proposed additional CCCM model.

[0621] In yet another example, a convolutional N-tap (N is an integer and greater than 1) filter can consist of an (N - 1 - M)-tap (M is an integer) spatial term, M non-linear terms, and a bias term. The (N - 1 - M)-tap spatial term corresponds to the neighboring luma samples of the chroma sample to be predicted (i.e., L0, L1, …, L N-1-M ). Note that the neighboring original reconstructed luma samples used in the spatial term and the non-linear terms can be replaced by any neighboring original reconstructed luma samples, i.e., without downsampling (i.e., L0, L1, …, L N-1-M ).

[0622]

[0623] where α i is the coefficient associated with L i , and β is an offset. Similar to the existing CCCM design, up to 6 rows / columns of chroma samples above and to the left of the current CU are used to derive the filter coefficients. The filter coefficients are derived based on the same LDL decomposition method used in the CCCM. This method can be signaled as an additional CCCM model in addition to the existing CCCM model. For signaling, when CCCM is selected, a single flag is signaled and used for both chroma components to indicate whether to apply the default CCCM model or the proposed additional CCCM model.

[0624] Note that the methods / examples in this section can be combined / reused with the methods mentioned above, including but not limited to the methods related to classification, filter shape, matrix derivation (with special processing), application area, and syntax. In addition, the methods / examples listed in this section can also be applied together with the methods / examples (more taps) above to achieve better performance and certain complexity trade-offs.

[0625] In the present disclosure, reference samples / training templates / reconstructed neighboring regions generally refer to the luma samples used to derive the MLR model parameters, and then the MLR model parameters are applied to the internal luma samples in a CU to predict the chroma samples in the CU.

[0626] According to one or more embodiments of the present disclosure, the reference sample / training template / reconstruction adjacent region may be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, Figure 9A Examples showing an L-shaped reconstruction region, a left / top reconstruction region for deriving parameters.

[0627] Filter shape

[0628] The number of one or more shapes / filter taps may be used for CCCM prediction, as Figure 25 , Figure 26 and Figures 27A to 27B shown. One or more sets of filter taps may be used for FLM prediction, examples of which are shown in Figures 28A to 28G . The selected luminance reference value is non-downsampled. The number of one or more predefined shapes / filter taps may be used for CCCM prediction based on previously decoded information at the TB / CB / slice / picture / sequence level.

[0629] Although a multi-tap filter can well fit the training data (i.e., the top / left adjacent reconstructed luminance / chrominance samples), in some cases, this training data does not capture all the characteristics of the test data, and it may lead to overfitting and may not well predict the test data (i.e., the chrominance block samples to be predicted). Moreover, different filter shapes can well adapt to different video block contents, resulting in more accurate prediction. To solve this problem, the filter shape / number of filter taps may be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. The set of filter shape candidates may be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Different components (U / V) may have different filter switch controls. For example, as shown in the following table, the predefined set of filter shape candidates (index = 0 to 5) and filter shapes (1, 2) represent a 2-tap luminance filter, while filter shapes (1, 2, 4) represent a 3-tap luminance filter as shown in Figure 20 . The filter shape selection for the U / V components may be switched in the PH or at the CU / CTU level. Note that, as described above, N taps may represent N taps with or without an offset β.

[0630]

[0631]

[0632] Different chroma types / color formats may have different predefined filter shapes / taps. For example, predefined filter shapes are used for 420 type 0: (1, 2, 4, 5), 420 type 2: (0, 1, 2, 4, 7), 422: (1, 4), 444: (0, 1, 2, 3, 4, 5), as Figure 21 shown.

[0633] Unavailable luma / chroma samples for deriving the MLR model can be filled from available reconstructed samples. For example, if a 6-tap (0, 1, 2, 3, 4, 5) filter as Figure 21 is used, for a CU located at the left picture boundary, the left column including (0, 3) is unavailable (outside the picture boundary), so (0, 3) is filled as a duplicate from (1, 4) to apply the 6-tap filter. Note that the filling process is applied in both the training data (top / left adjacent reconstructed luma / chroma samples) and the test data (luma / chroma samples in a CU).

[0634] According to one or more embodiments of the present disclosure, unavailable luma / chroma samples for deriving the MLR model can be skipped and not used. Then, the filling process is not required for unavailable luma / chroma samples.

[0635] CCLM / MMLM with LDL decomposition

[0636] CCCM needs to process LDL decomposition to calculate the model parameters of the CCCM model, avoiding the use of square root operations and only requiring integer operations. In this section, CCLM / MMLM with LDL decomposition is proposed. As mentioned above, LDL decomposition can also be used in ELM / FLM / GLM.

[0637] Note that the methods / examples in this section can be combined / reused with the methods mentioned above, including but not limited to methods related to classification, filter shapes, matrix derivation (with special handling), application areas, and syntax. In addition, the methods / examples listed in this section can also be applied together with the methods / examples above to achieve better performance and certain complexity trade-offs.

[0638] In the present disclosure, reference samples / training templates / reconstructed adjacent regions generally refer to luma samples used to derive the MLR model parameters, and then the MLR model parameters are applied to the internal luma samples in a CU to predict the chroma samples in the CU.

[0639] CCLM / MMLM with extended range

[0640] One or more reference sample points can be used for CCLM / MMLM prediction, i.e., as Figure 18 shown, the reference region can be the same as the reference region in CCCM. Based on the previous decoded information at the TB / CB / slice / picture / sequence level, different reference regions can be used for CCLM / MMLM prediction.

[0641] Although the training data with multiple reference regions can well fit the calculation of model parameters, in some cases, the training data does not capture all the characteristics of the test data, but it may lead to overfitting and may not be able to predict the test data well (i.e., the chrominance block sample points to be predicted). In addition, different reference regions can well adapt to different video block contents, resulting in more accurate prediction. To solve this problem, the reference shape / number of the reference region can be predefined, or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample point level. The reference region candidate set can be predefined, or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample point level. Different components (U / V) can have different reference region switch controls. For example, a predefined reference region candidate set (index = 0 to 4), as shown in the following table. The reference region selection of the U / V component can be switched in the PH or at the CU / CTU level. Different chrominance types / color formats can have different predefined reference regions.

[0642]

[0643]

[0644] The unavailable luma / chroma sample points for deriving the MLR model can be filled with available reconstructed sample points, and the filling process is applied to both the training data (top / left adjacent reconstructed luma / chroma sample points) and the test data (luma / chroma sample points in the CU).

[0645] According to one or more embodiments of the present disclosure, the unavailable luma / chroma sample points for deriving the MLR model can be skipped and not used. Then, no filling process is required for the unavailable luma / chroma sample points.

[0646] FLM / GLM / ELM / CCCM with minimum sample limit

[0647] The FLM needs to process the downsampled luminance reference values and calculate the model parameters, which burdens the decoder processing cycle, especially for small blocks. In this section, an FLM with a minimum sample limit is proposed. For example, the FLM is only used for samples greater than a predefined number (such as 64, 128). One or more different limits can be used for this purpose. For example, the FLM is only used for samples greater than a predefined number (such as 256) in a single model, and the FLM is only used for samples greater than a predefined number (such as 128) in a multi-model.

[0648] According to one or more embodiments of the present disclosure, the predefined minimum number of samples for a single model can be greater than or equal to the predefined minimum number of samples for a multi-model. For example, the FLM / GLM / ELM / CCCM is only used for samples greater than or equal to a predefined number (such as 128) in a single model, and the FLM / GLM / ELM / CCCM is only used for samples greater than or equal to a predefined number (such as 256) in a multi-model.

[0649] According to one or more embodiments of the present disclosure, the predefined minimum number of samples for the FLM / GLM / ELM can be greater than or equal to the predefined minimum number of samples for the CCCM. For example, the CCCM is only used for samples greater than or equal to a predefined number (such as 0) in a single model, and the CCCM is only used for samples greater than or equal to a predefined number (such as 128) in a multi-model. The FLM is only used for samples greater than or equal to a predefined number (such as 128) in a single model, and the FLM is only used for samples greater than or equal to a predefined number (such as 256) in a multi-model.

[0650] Note that the methods / examples in this section can be combined / reused with the methods mentioned above, including but not limited to the methods related to classification, filter shape, matrix derivation (with special processing), application area, and syntax. In addition, the methods / examples listed in this section can also be applied together with the methods / examples above (more taps) to achieve better performance and certain complexity trade-offs.

[0651] Multi-mode of the combination of FLM / GLM / ELM / CCCM / CCLM

[0652] According to one or more embodiments of the present disclosure, two models in multiple modes of the FLM / GLM / ELM / CCCM / CCLM can be further combined to bring additional encoding and decoding efficiency. For example, the parameters of the CCCM(ci) and GLM(a, b) are first derived separately, and then the weight (w i ) between the CCCM and the GLM is derived by linear regression. Finally, the weighted CCCM and GLM are used to predict the chrominance samples from the reconstructed luminance samples.

[0653] GLMpredChromaVal = a * lumaVal + b

[0654] CCCMpredChromaVal = c0 * C + c1 * N + c2 * S + c3 * E + c4 * W + c5 * P + c6 * B

[0655] FinalpredChromaVal = w0 * GLMpredChromaVal + w1 * CCCMpredChromaVal

[0656] According to one or more embodiments of the present disclosure, there is a flag signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub - block / sample level to indicate whether to use the combined mode.

[0657] According to one or more embodiments of the present disclosure, instead of explicitly signaling the selected mode flag, the mode flag can be derived at the decoder to save bit overhead.

[0658] (1) Determine M combined candidates for the current CU

[0659] (2) Divide the available L - shaped template region into N regions, denoted as R0, R1,... R N-1

[0660] (Divide the training data into N sets, N - fold training)

[0661] (3) Independently apply the M combined candidates to a part of the available template region (which can be a single or multiple regions among R0, R1,... R N-1 )

[0662] (4) Derive M sets of filter coefficients (according to M filter shapes), denoted as F0, F1,... F M-1

[0663] (5) Apply the derived sets of F0, F1,... F M-1 filter coefficients to the other part of the available template region, different from the part of the available template in (3)

[0664] (6) Accumulate the errors through SAD, SSD or SATD, denoted as E0, E1,... E M-1

[0665] (7) Sort and select K minimum errors, denoted as E'0, E'1,... E' K-1 , which correspond to K combined filters / K sets of filter coefficients

[0666] Signal and select one combination of K filters to be applied to the current CU for chrominance prediction; if K is 1, signaling is not required (the filter shape with the minimum error is the applied combined filter).

[0667] For example,

[0668] (1) Pre - define 3 filter candidates for the current CU, such as CCCM, GLM, and the combination of CCCM and GLM

[0669] (2) Divide the available L - shaped template region (6 chrominance rows / columns for CCCM, note that in the CCCM design, each chrominance sample involves 6 luma samples for down - sampling) into 2 regions, denoted as R0 and R1

[0670] For example, even rows / columns: R0, odd rows / columns: R1

[0671] Figure 29A Show the following example: in the template region, the even - row region R0 is used to train / deduce 3 sets of filter coefficients, and the odd - row region R1 is used to verify / compare and rank the costs of the 3 sets of filter coefficients.

[0672] (3) Independently apply the 3 filter candidates to a part of the available template region (a single R0)

[0673] (4) Deduce 3 sets of filter coefficients (according to 4 filter shapes), denoted as F0, F1, F2

[0674] (5) Apply the deduced sets of filter coefficients F0, F1, F2 to another part of the available template region different from the part in (3) (R1).

[0675] (6) Accumulate the errors through SAD, SSD, or SATD, denoted as E0, E1, E2

[0676] (7) Rank and select the 1 minimum error, denoted as E'0, which corresponds to 1 filter shape / 1 set of filter coefficients.

[0677] (8) If K is 1, signaling is not required (the filter with the minimum error is the applied filter shape)

[0678] Note that the methods / examples in this section can be combined / reused from the methods mentioned in all sections, including but not limited to classification, filter shape, matrix derivation (with special processing), application area, syntax. In addition, the methods / examples listed in this section can also be applied to all sections for better performance and certain complexity trade - offs.

[0679] CCCM with non-downsampled and downsampled luminance reference values

[0680] Only the downsampled luminance reference values can be used in the CCCM to calculate the model parameters and apply the CCCM model. In this section, the non-downsampled luminance reference values are also used in the CCCM to calculate the model parameters and apply the CCCM model, including using the non-downsampled luminance reference values and the downsampled luminance reference values at different or the same positions. As described above, one or more filter shapes can be used for this purpose.

[0681] In one example, a convolutional 8-tap filter can include a 6-tap rectangular-shaped spatial component, a downsampled luminance sample, and a bias term. The input to the spatial 6-tap component of the filter includes the center (b) non-downsampled luminance sample (which is co-located with the chroma sample to be predicted) and its non-downsampled lower-left or southwest (d), lower-right or southeast (f), lower or south (e), left or west (a), and right or east (c) neighbors, as shown in Shape 1 of Figure 25 and the center (C) downsampled luminance sample (which is co-located with the chroma sample to be predicted).

[0682] The bias term B represents a scalar offset between the input and the output (similar to the offset term in the CCLM) and is set to the middle chroma value (512 for 10-bit content).

[0683] The output of the filter is calculated as the convolution between the filter coefficients ci and the input values and is clipped to the range of valid chroma samples:

[0684] predChromaVal = c0a + c1b + c2c + c3d + c4e + c5f + c6C + c7B

[0685] In another example, a convolutional 9-tap filter can consist of a 6-tap rectangular-shaped spatial component, two non-linear terms, and a bias term. The input to the spatial 6-tap component of the filter consists of the following terms: the center (b) non-downsampled luminance sample (which is co-located with the chroma sample to be predicted and is substantially located at the center of the filter shape) and its non-downsampled lower-left / southwest (d), lower-right / southeast (f), lower / south (e), left / west (a), and right / east (c) neighbors, as shown in Shape 1 of Figure 25 and the center (C) downsampled luminance sample (which is co-located with the chroma sample to be predicted (e.g., the value of the center (C) downsampled luminance sample is determined by a weighted average operation)).

[0686] The non - linear terms P and Q are two non - linear luma sample values, which are respectively represented as the powers of the luma sample values of the center (b) non - downsampled luma samples and the powers of the luma sample values of the center (C) downsampled luma samples, and then scaled to the sample value range of the content:

[0687] P = (b * b+midVal) >> bitDepth;

[0688] Q = (C * C+midVal) >> bitDepth.

[0689] That is to say, for 10 - bit content, it is calculated as:

[0690] P = (b * b+512) >> 10;

[0691] Q = (C * C+512) >> 10.

[0692] The bias term B is a bias value representing the scalar offset between the input and the output (similar to the offset term in CCLM) and is set to the middle chroma value (512 for 10 - bit content).

[0693] The output of the filter is calculated as the convolution between the filter coefficients ci and the input values and is clipped to the range of valid chroma samples:

[0694] predChromaVal = c0a + c1b + c2c + c3d + c4e + c5f + c6P + c7Q + c8B

[0695] It should be understood that the non - linear terms P and Q can be represented as the powers of any luma sample values of the downsampled / non - downsampled luma samples for predicting the corresponding chroma sample values. The two non - linear terms P and Q are only exemplary, and the corresponding chroma sample values can be calculated based on one or more non - linear values.

[0696] As mentioned above, the non - linear term Q can also be represented as the power of the non - downsampled luma samples of the filter. In such an example, the convolutional 9 - tap filter can consist of a 6 - tap rectangular - shaped spatial component, two non - linear terms, and a bias term. The input of the 6 - tap spatial component of the filter consists of the following terms: the center (b) non - downsampled luma sample (which is co - located with the chroma sample to be predicted and is basically located at the center of the filter shape) and its non - downsampled lower - left / south - west (d), lower - right / south - east (f), lower / south (e), left / west (a), and right / east (c) neighbors, as Figure 25 shown in Shape 1 of

[0697] The non - linear terms P and Q are two non - linear luma sample values, which are respectively represented as the power of the luma sample value of the center (b) luma sample and the power of the luma sample value of the lower / south (e) luma sample, and then scaled to the sample value range of the content:

[0698] P = (b * b+midVal) >> bitDepth;

[0699] Q = (e * e+midVal) >> bitDepth.

[0700] That is to say, for 10 - bit content, it is calculated as:

[0701] P = (b * b+512) >> 10;

[0702] Q = (e * e+512) >> 10.

[0703] The bias term B is a bias value representing the scalar offset between the input and the output (similar to the offset term in CCLM) and is set to the middle chroma value (512 for 10 - bit content).

[0704] The output of the filter is calculated as the convolution between the filter coefficients ci and the input values and is clipped to the range of valid chroma samples:

[0705] predChromaVal = c0a + c1b + c2c + c3d + c4e + c5f + c6P + c7Q + c8B

[0706] In another example, the convolutional 9 - tap filter can include the original CCCM 7 - tap terms corresponding to the downsampled reconstructed luma samples and 2 - tap spatial terms corresponding to 2 adjacent original reconstructed luma samples, that is, without downsampling (i.e., L2, L3) as shown in Figure 30 to predict chroma samples. The CCCM 7 - tap filter consists of a 5 - tap plus - shaped spatial component, non - linear terms, and a bias term. The input of the spatial 5 - tap component of the filter consists of the following terms: the center (C) luma sample (which is co - located with the chroma sample to be predicted), and its upper / north (N), lower / south (S), left / west (W), and right / east (E) neighbors, as shown in Figure 17 as shown.

[0707] The non - linear term P is represented as the square of the center luma sample C and is scaled to the sample value range of the content:

[0708] P = (C * C+midVal) >> bitDepth

[0709] That is to say, for 10 - bit content, it is calculated as:

[0710] P = (C * C + 512) >> 10

[0711] The bias term B represents a scalar offset between the input and the output (similar to the offset term in CCLM) and is set to the middle chroma value (512 for 10-bit content).

[0712] The output of the filter is calculated as the convolution between the filter coefficients ci and the input values and is clipped to the range of valid chroma samples:

[0713] predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B + c7L2 + c8L3

[0714] Note that L2, L3 can be replaced by any two adjacent original reconstructed luma samples, i.e., without downsampling as shown in Figure 30 (i.e., L0, L1, …, L5).

[0715] In another example, a convolutional N-tap (N is an integer and greater than 7) filter can include the original CCCM 7-tap terms corresponding to the downsampled reconstructed luma samples, the (N - 7 - M)-tap (M is an integer) spatial terms corresponding to the adjacent original reconstructed luma samples of the chroma samples, and M non-linear terms, i.e., without downsampling to the chroma samples as shown in Figure 30 (i.e., L0, L1, …, L5). The CCCM 7-tap filter consists of a 5-tap plus-shaped spatial component, non-linear terms, and a bias term. The input to the spatial 5-tap component of the filter consists of the following terms: the center (C) luma sample (which is co-located with the chroma sample to be predicted) and its above / north (N), below / south (S), left / west (W), and right / east (E) neighbors, as shown in Figure 17 .

[0716] The non-linear term P is represented as the square of the center luma sample C and is scaled to the sample value range of the content:

[0717] P = (C * C + midVal) >> bitDepth

[0718] That is, for 10-bit content, it is calculated as:

[0719] P = (C * C + 512) >> 10

[0720] The bias term B represents a scalar offset between the input and the output (similar to the offset term in CCLM) and is set to the middle chroma value (512 for 10-bit content).

[0721] The output of the filter is calculated as the convolution between the filter coefficients ci and the input values, and is clipped to the range of valid chroma samples:

[0722]

[0723] Note that the original reconstructed luma samples used in the spatial and non - linear terms can be replaced by any adjacent original reconstructed luma samples, i.e., as Figure 30 shown, without downsampling (i.e., L0, L1, …, L5).

[0724] It should be understood that the filter shape is not limited to the plus shape used by the CCCM and can be any shape as desired. For example, the filter shape can be any of the shapes shown in Figure 25 .

[0725] Note that the methods / examples in this section can combine / reuse from the methods mentioned above, including but not limited to classification, filter shape, matrix derivation (with special handling), application area, syntax. In addition, the methods / examples listed in this section can also be applied to the methods (more taps) mentioned above to have better performance and certain complexity trade - offs.

[0726] In this disclosure, reference samples / training templates / reconstructed adjacent regions generally refer to the luma samples used to derive the MLR model parameters, and then the MLR model parameters are applied to the internal luma samples in a CU to predict the chroma samples in the CU.

[0727] According to one or more embodiments of this disclosure, the CCCM without a downsampling process and the CCCM with a downsampling process can be used with different taps or shapes. For example, the number of filter shapes / filter taps with downsampled values and without downsampled values can be predefined, or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub - block / sample level.

[0728] In one example, a filter with downsampled values and a filter without downsampled values can be signaled / switched at the CU level. The filter with downsampled values is the same as the CCCM. The filter without downsampled values can consist of (N - 1 - M) taps (N and M are integers) in the spatial term, M non - linear terms, and a bias term. The (N - 1 - M) - tap spatial term corresponds to the adjacent original reconstructed luma samples of the chroma sample to be predicted, i.e., as Figure 30 shown, without downsampling (i.e., L0, L1, …, L5).

[0729]

[0730] where α i is a coefficient associated with L i and β is an offset.

[0731] In another example, one filter with a downsampling value and two filters without a downsampling value can be signaled / switched at the CU level. The filter with the downsampling value is the same as the CCCM. One filter without a downsampling value can be composed of an (N - 1 - M)-tap (where N and M are integers) spatial term, M non-linear terms, and a bias term. The (N - 1 - M)-tap spatial term corresponds to the adjacent original reconstructed luma samples of the chroma sample to be predicted, i.e., as Figure 30 shown, without downsampling (i.e., L0, L1,..., L5).

[0732]

[0733] where α i is a coefficient associated with L i and β is an offset. Another filter without a downsampling value can be composed of an (N - 2)-tap (where N is an integer) spatial term, one non-linear term, and a bias term. The (N - 2)-tap spatial term corresponds to the adjacent original reconstructed luma samples of the chroma sample to be predicted, i.e., without downsampling.

[0734]

[0735] where α i is a coefficient associated with L i and β is an offset.

[0736] According to one or more embodiments of the present disclosure, the reference samples / training templates / reconstructed neighboring regions for the derivation of multi-model parameters and the reference samples / training templates / reconstructed neighboring regions for calculating the multi-model thresholds can be different.

[0737] According to one or more embodiments of the present disclosure, the reference samples / training templates / reconstructed neighboring regions can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, Figure 9A examples showing an L-shaped reconstructed region, a left / top reconstructed region for deriving parameters are shown.

[0738] Figure 31 The workflow of method 3100 for encoding video data according to one or more aspects of the present disclosure is shown.

[0739] At step 3110, method 3100 includes obtaining a video block from a bitstream.

[0740] At step 3120, method 3100 includes obtaining internal luminance sample values of the video block, external luminance sample values of an external region of the video block, and external chrominance sample values of the external region.

[0741] At step 3130, method 3100 includes determining a set of weighting coefficients corresponding to a filter shape based on the external luminance sample values and the external chrominance sample values.

[0742] At step 3140, method 3100 includes predicting each of the internal chrominance sample values based on the filter shape and the set of weighting coefficients, based on a plurality of corresponding luminance sample values, where the plurality of corresponding luminance sample values includes: one or more downsampled luminance sample values associated with the filter shape, one or more non-downsampled luminance sample values associated with the filter shape, and one or more non-linear luminance sample values.

[0743] At step 3150, method 3100 includes obtaining a predicted video block using the predicted internal chrominance sample values.

[0744] In one example, determining the set of weighting coefficients corresponding to the filter shape includes: determining the set of weighting coefficients based on a plurality of external chrominance sample values and external luminance sample values corresponding to each of the plurality of external chrominance sample values, where the external luminance sample values corresponding to each of the plurality of external chrominance sample values includes: one or more downsampled luminance sample values associated with the filter shape, one or more non-downsampled luminance sample values associated with the filter shape, and one or more non-linear luminance sample values.

[0745] In one example, the filter shape is associated with five luminance sample points, which include: a first luminance sample point located at the center of the filter shape; a second luminance sample point below the first luminance sample point; a third luminance sample point to the left of the first luminance sample point; a fourth luminance sample point to the right of the first luminance sample point; and a fifth luminance sample point above the first luminance sample point.

[0746] In one example, the filter shape is associated with six luminance sample points including: a first luminance sample point located at the center of the filter shape; a second luminance sample point below the first luminance sample point; a third luminance sample point to the left of the first luminance sample point; a fourth luminance sample point to the right of the first luminance sample point; a fifth luminance sample point below and to the left of the first luminance sample point; and a sixth luminance sample point below and to the right of the first luminance sample point.

[0747] In one example, the set of weighting coefficients further includes a weighting coefficient for a bias value.

[0748] In one example, one or more non-linear luma sample values include non-linear luma sample values derived by scaling the square of the value of a first luma sample to the luma sample value range of a video block.

[0749] In one example, the filter shape is a first filter shape, and the first filter shape may be signaled at the coding unit (CU) level to switch to a second filter shape for predicting each of the intra chroma sample values based on a plurality of corresponding luma sample values.

[0750] In one example, the plurality of corresponding luma sample values associated with the second filter shape include: one or more downsampled luma sample values associated with the second filter shape, and one or more non-linear luma sample values.

[0751] In one example, the plurality of corresponding luma sample values associated with the second filter shape include: one or more non-downsampled luma sample values associated with the filter shape, and one or more non-linear luma sample values.

[0752] In one example, the first filter shape and the second filter shape use different external luma sample values and external chroma sample values to determine respective sets of weighting coefficients.

[0753] Figure 32 A workflow of a method 3200 for encoding video data according to one or more aspects of the present disclosure is shown.

[0754] At step 3210, method 3200 includes obtaining a video block.

[0755] At step 3220, method 3200 includes obtaining the intra luma sample values of the video block, the external luma sample values of an external region of the video block, and the external chroma sample values of the external region.

[0756] At step 3230, method 3200 includes determining a set of weighting coefficients corresponding to a filter shape based on the external luma sample values and the external chroma sample values.

[0757] At step 3240, method 3200 includes predicting each of the intra chroma sample values based on the filter shape and the set of weighting coefficients, based on a plurality of corresponding luma sample values, where the plurality of corresponding luma sample values include: one or more downsampled luma sample values associated with the filter shape, one or more non-downsampled luma sample values associated with the filter shape, and one or more non-linear luma sample values.

[0758] At step 3250, method 3200 includes generating a bitstream including the encoded video block by using the predicted intra chroma sample values.

[0759] In one example, determining a set of weighted coefficients corresponding to a filter shape includes: determining the set of weighted coefficients based on a plurality of external chrominance sample values and external luminance sample values corresponding to each of the plurality of external chrominance sample values, wherein the external luminance sample values corresponding to each of the plurality of external chrominance sample values include: one or more downsampled luminance sample values associated with the filter shape, one or more non-downsampled luminance sample values associated with the filter shape, and one or more non-linear luminance sample values.

[0760] In one example, the filter shape is associated with five luminance samples, which include: a first luminance sample located at the center of the filter shape; a second luminance sample below the first luminance sample; a third luminance sample to the left of the first luminance sample; a fourth luminance sample to the right of the first luminance sample; and a fifth luminance sample above the first luminance sample.

[0761] In one example, the filter shape is associated with six luminance samples including: a first luminance sample located at the center of the filter shape; a second luminance sample below the first luminance sample; a third luminance sample to the left of the first luminance sample; a fourth luminance sample to the right of the first luminance sample; a fifth luminance sample in the lower left of the first luminance sample; and a sixth luminance sample in the lower right of the first luminance sample.

[0762] In one example, the set of weighted coefficients further includes a weighted coefficient for a deviation value.

[0763] In one example, one or more non-linear luminance sample values include non-linear luminance sample values derived by scaling the square of the value of the first luminance sample to the luminance sample value range of the video block.

[0764] In one example, the filter shape is a first filter shape, and the first filter shape can be signaled at the coding unit (CU) level to switch to a second filter shape for predicting each of the internal chrominance sample values based on a plurality of corresponding luminance sample values.

[0765] In one example, the plurality of corresponding luminance sample values associated with the second filter shape include: one or more downsampled luminance sample values associated with the second filter shape, and one or more non-linear luminance sample values.

[0766] In one example, the plurality of corresponding luminance sample values associated with the second filter shape include: one or more non-downsampled luminance sample values associated with the filter shape, and one or more non-linear luminance sample values.

[0767] In one example, the first filter shape and the second filter shape use different external luminance sample values and external chrominance sample values to respectively determine sets of weighting coefficients.

[0768] Figure 33 FIG. shows a computing environment 3310 coupled to a user interface 3350. The computing environment 3310 can be part of a data processing server. The computing environment 3310 includes a processor 3320, a memory 3330, and an input / output (I / O) interface 3340.

[0769] The processor 3320 generally controls the overall operation of the computing environment 3310, such as operations associated with display, data acquisition, data communication, and image processing. The processor 3320 can include one or more processors to execute instructions to perform all or some of the steps in the above methods. In addition, the processor 3320 can include one or more modules that facilitate the interaction between the processor 3320 and other components. The processor can be a central processing unit (CPU), a microprocessor, a single-chip machine, a graphics processing unit (GPU), etc.

[0770] The memory 3330 is configured to store various types of data to support the operation of the computing environment 3310. The memory 3330 can include predetermined software 3332. Examples of such data include instructions for any application or method operating on the computing environment 3310, video data sets, image data, etc. The memory 3330 can be implemented by using any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disks, or optical disks.

[0771] The I / O interface 3340 provides an interface between the processor 3320 and peripheral interface modules (such as a keyboard, click wheel, buttons, etc.). The buttons can include, but are not limited to, a home button, a start scan button, and a stop scan button. The I / O interface 3340 can be coupled to an encoder and a decoder.

[0772] In an embodiment, a non-transitory computer-readable storage medium is also provided, which includes, for example in the memory 3330, a plurality of programs executable by the processor 3320 in the computing environment 3310 for performing the above methods and / or storing a bitstream generated by the above encoding method or a bitstream decoded by the above decoding method. In one example, the plurality of programs can be executed by the processor 3320 in the computing environment 3310 to (e.g., from Figure 2The video encoder 20) therein receives a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames and / or one or more associated syntax elements, etc.), and can also be executed by the processor 3320 in the computing environment 3310 to perform the decoding method described above according to the received bitstream or data stream. In another example, multiple programs can be executed by the processor 3320 in the computing environment 3310 to perform the above encoding method to encode video information (e.g., video blocks representing video frames and / or one or more associated syntax elements, etc.) into a bitstream or data stream, and can also be executed by the processor 3320 in the computing environment 3310 to send the bitstream or data stream (e.g., to Figure 3 the video decoder 30) therein. Alternatively, a non-transitory computer-readable storage medium can store therein a bitstream or data stream, the bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames and / or one or more associated syntax elements, etc.) generated by an encoder (e.g., Figure 2 the video encoder 20) therein using, for example, the encoding method described above for use by a decoder (e.g., Figure 3 the video decoder 30) therein when decoding video data. The non-transitory computer-readable storage medium can be, for example, ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0773] In an embodiment, a bitstream generated by the above encoding method or a bitstream to be decoded by the above decoding method is provided. In an embodiment, a bitstream is provided that includes encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method.

[0774] In an embodiment, a computing device is further provided, which includes one or more processors (e.g., the processor 3320) and a non-transitory computer-readable storage medium or memory 3330 storing therein multiple programs executable by the one or more processors, wherein the one or more processors are configured to execute the above method when executing the multiple programs.

[0775] In an embodiment, a computer program product having instructions for storing or transmitting a bitstream is further provided, the bitstream including encoded video information generated by the above encoding method or encoded video information to be decoded by the above decoding method. In one embodiment, a computer program product is further provided, which includes in, for example, the memory 3330 multiple programs executable by the processor 3320 in the computing environment 3310 for performing the above method. For example, the computer program product can include a non-transitory computer-readable storage medium.

[0776] In an embodiment, the computing environment 3310 may be implemented with one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described methods.

[0777] In an embodiment, a method for storing a bitstream is also provided, including storing the bitstream on a digital storage medium, where the bitstream includes encoded video information generated by the above-described encoding method or encoded video information to be decoded by the above-described decoding method.

[0778] In an embodiment, a method for transmitting the bitstream generated by the above-described encoder is also provided. In an embodiment, a method for receiving the bitstream to be decoded by the above-described decoder is also provided.

[0779] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or to limit the present disclosure. Many modifications, variations, and alternative implementations will be apparent to those of ordinary skill in the art in light of the foregoing description and the teachings presented in the associated drawings.

[0780] Unless otherwise specifically stated, the order of the steps of the methods according to the present disclosure is merely illustrative, and the steps of the methods according to the present disclosure are not limited to the order specifically described above, but may be changed according to actual conditions. In addition, at least one step of the methods according to the present disclosure may be adjusted, combined, or deleted according to actual requirements.

[0781] The examples were chosen and described to explain the principles of the present disclosure and to enable others of ordinary skill in the art to understand the present disclosure for various implementations and to best utilize the basic principles and various implementations with various modifications suitable for the particular purposes contemplated. Accordingly, it should be understood that the scope of the present disclosure is not limited to the specific examples of the disclosed implementations, and modifications and other implementations are intended to be included within the scope of the present disclosure.

Claims

1. A method for decoding video data, comprising: obtaining a video block from a bitstream; obtaining internal luma sample values of the video block, external luma sample values of an external region of the video block, and external chroma sample values of the external region; determining a set of weighting coefficients corresponding to a filter shape based on the external luma sample values and the external chroma sample values; predicting each of the internal chroma sample values based on the filter shape and the set of weighting coefficients, based on a plurality of corresponding luma sample values, wherein the plurality of corresponding luma sample values include: one or more downsampled luma sample values associated with the filter shape, one or more non-downsampled luma sample values associated with the filter shape, and one or more non-linear luma sample values; and obtaining a predicted video block using the predicted internal chroma sample values.

2. The method according to claim 1, wherein Determining the set of weighting coefficients corresponding to the filter shape includes: determining the set of weighting coefficients based on a plurality of external chroma sample values and external luma sample values corresponding to each of the plurality of external chroma sample values, wherein the external luma sample values corresponding to each of the plurality of external chroma sample values include: one or more downsampled luma sample values associated with the filter shape, one or more non-downsampled luma sample values associated with the filter shape, and one or more non-linear luma sample values.

3. The method according to claim 1, wherein The filter shape is associated with five luma samples including: a first luma sample located at the center of the filter shape; a second luma sample below the first luma sample; a third luma sample to the left of the first luma sample; a fourth luma sample to the right of the first luma sample; and a fifth luma sample above the first luma sample.

4. The method according to claim 1, wherein The filter shape is associated with six luma samples including: a first luma sample located at the center of the filter shape; a second luma sample below the first luma sample; a third luma sample to the left of the first luma sample; a fourth luma sample to the right of the first luma sample; a fifth luma sample in the lower left of the first luma sample; and a sixth luma sample in the lower right of the first luma sample.

5. The method according to claim 1, wherein, The set of weighting coefficients further includes a weighting coefficient for a deviation value.

6. The method according to claim 3 or 4, wherein The one or more non-linear luma sample values include non-linear luma sample values derived by scaling the square of the value of the first luma sample to the luma sample value range of the video block.

7. The method according to claim 1, wherein The filter shape is a first filter shape, and the first filter shape can be signaled at the coding unit (CU) level to switch to a second filter shape for predicting each of the internal chroma sample values based on a plurality of corresponding luma sample values.

8. The method according to claim 7, wherein The plurality of corresponding luma sample values associated with the second filter shape include: one or more downsampled luma sample values associated with the second filter shape, and one or more non-linear luma sample values.

9. The method according to claim 7, wherein The plurality of corresponding luma sample values associated with the second filter shape includes: one or more non-downsampled luma sample values associated with the filter shape, and one or more non-linear luma sample values.

10. The method according to claim 7, wherein, The first filter shape and the second filter shape use different external luma sample values and external chroma sample values to respectively determine the set of weighted coefficients.

11. A method for encoding video data, comprising: obtaining a video block; obtaining internal luma sample values of the video block, external luma sample values of an external region of the video block, and external chroma sample values of the external region; determining a set of weighted coefficients corresponding to a filter shape based on the external luma sample values and the external chroma sample values; predicting each of the internal chroma sample values based on a plurality of corresponding luma sample values by using the filter shape and the set of weighted coefficients, wherein the plurality of corresponding luma sample values includes: one or more downsampled luma sample values associated with the filter shape, one or more non-downsampled luma sample values associated with the filter shape, and one or more non-linear luma sample values; and generating a bitstream including the encoded video block by using the predicted internal chroma sample values.

12. The method according to claim 11, wherein, Determining the set of weighted coefficients corresponding to the filter shape includes: determining the set of weighted coefficients based on a plurality of external chroma sample values and external luma sample values corresponding to each of the plurality of external chroma sample values, wherein the external luma sample values corresponding to each of the plurality of external chroma sample values includes: one or more downsampled luma sample values associated with the filter shape, one or more non-downsampled luma sample values associated with the filter shape, and one or more non-linear luma sample values.

13. The method according to claim 11, wherein, The filter shape is associated with five luma samples including: a first luma sample located at the center of the filter shape; a second luma sample below the first luma sample; a third luma sample to the left of the first luma sample; a fourth luma sample to the right of the first luma sample; and a fifth luma sample above the first luma sample.

14. The method according to claim 11, wherein, The filter shape is associated with six luma samples including: a first luma sample located at the center of the filter shape; a second luma sample below the first luma sample; a third luma sample to the left of the first luma sample; a fourth luma sample to the right of the first luma sample; a fifth luma sample in the lower left of the first luma sample; and a sixth luma sample in the lower right of the first luma sample.

15. The method according to claim 11, wherein, The set of weighted coefficients further includes a weighted coefficient for a deviation value.

16. The method according to claim 13 or 14, wherein, The one or more non-linear luma sample values include non-linear luma sample values derived by scaling the square of the value of the first luma sample to the luma sample value range of the video block.

17. The method according to claim 11, wherein, The filter shape is a first filter shape, and the first filter shape can be signaled at the coding unit (CU) level to switch to a second filter shape for predicting each of the intra chroma sample values based on a plurality of corresponding luma sample values.

18. The method according to claim 17, wherein, The plurality of corresponding luma sample values associated with the second filter shape include: one or more downsampled luma sample values associated with the second filter shape, and one or more non-linear luma sample values.

19. The method according to claim 17, wherein The plurality of corresponding luma sample values associated with the second filter shape include: one or more non-downsampled luma sample values associated with the filter shape, and one or more non-linear luma sample values.

20. The method according to claim 17, wherein, The first filter shape and the second filter shape use different external luma sample values and external chroma sample values to respectively determine the set of weighting coefficients.

21. A computer system, comprising: one or more processors; and one or more storage devices storing computer-executable instructions that, when executed, cause the one or more processors to perform the operations of the method according to any one of claims 1-20.

22. A computer program product storing computer-executable instructions that, when executed, cause one or more processors to perform the operations of the method according to any one of claims 1-20.

23. A computer-readable storage medium storing instructions that, when executed by a computing device having one or more processors, cause the one or more processors to perform the decoding method according to any one of claims 1-10, and store a bitstream to be decoded by the decoding method according to any one of claims 1-10.

24. A computer-readable storage medium storing instructions that, when executed by a computing device having one or more processors, cause the one or more processors to perform the encoding method according to any one of claims 11-20, and store a bitstream generated by the encoding method according to any one of claims 11-20.

25. A computer-readable medium storing a bitstream, wherein, The bitstream is to be decoded by performing the operations of the method according to any one of claims 1-10.

26. A computer-readable medium for storing a bitstream, wherein, The bitstream is obtained by performing the operations of the method according to any one of claims 11-20.

Images