Cross-component prediction of chroma samples

Abstract

During video encoding or decoding, cross-component prediction may be used to predict chroma samples from collocated reconstructed luma samples. The prediction may be based on a gradient of collocated reconstructed luminance samples, a downsampled value of collocated reconstructed luminance samples, or a combination thereof. An exemplary method includes determining a first value associated with a chroma sample by applying a first gradient pattern to reconstructed values of a first plurality of luma samples; determining a second value associated with the chroma sample by applying a downsampling filter to the reconstructed values of a second plurality of luma samples; and predicting the chroma sample based on the first value and the second value.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This disclosure claims priority to U.S. Provisional Application No. 63 / 350,764, filed on June 9, 2022; U.S. Provisional Application No. 63 / 358,172, filed on July 4, 2022; U.S. Provisional Application No. 63 / 402,944, filed on August 31, 2022; and U.S. Application No. 18 / 325,704, filed on May 30, 2023. The entire contents of all of the above applications are incorporated herein by reference. Technical Field

[0002] The present disclosure relates generally to video processing, and more particularly, to methods and systems for predicting chroma samples based on collocated luma samples. Background Art

[0003] A video is a set of static images (or "frames") that capture visual information. In order to reduce storage memory and transmission bandwidth, videos can be compressed before storage or transmission, and decompressed before display. The compression process is often referred to as encoding, while the decompression process is often referred to as decoding. There are various video coding formats that use standardized video coding techniques, the most common of which are based on prediction, transform, quantization, entropy coding, and loop filtering. Video coding standards that specify specific video coding formats, such as the High Efficiency Video Coding (HEVC / H.265) standard, the Versatile Video Coding (VVC / H.266) standard, and the AVS standard, are developed by standardization organizations. With the application of more and more advanced video coding technologies in video standards, the coding efficiency of new video coding standards is getting higher and higher. Summary of the invention

[0004] Embodiments of the present disclosure relate to, in some embodiments, a computer-implemented method comprising: determining a first value associated with a chroma sample by applying a first gradient pattern to reconstructed values ​​of a first plurality of luma samples; determining a second value associated with the chroma sample by applying a downsampling filter to a plurality of reconstructed values ​​of a second plurality of luma samples; and predicting the chroma sample based on the first value and the second value.

[0005] The embodiment of the present disclosure also provides a device for processing video data. The system includes: a memory storing a set of instructions; and one or more processors configured to execute the set of instructions to enable the device to perform: determining a first value associated with a chroma sample by applying a first gradient pattern to a reconstructed value of a first plurality of luma samples; determining a second value associated with the chroma sample by applying a downsampling filter to a reconstructed value of a second plurality of luma samples; and predicting the chroma sample based on the first value and the second value.

[0006] Embodiments of the present disclosure also provide a non-transitory computer-readable medium storing a bitstream of a video for processing according to a method, the method comprising: determining a first value associated with a chroma sample by applying a first gradient pattern to reconstructed values ​​of a first plurality of luma samples; determining a second value associated with the chroma sample by applying a downsampling filter to reconstructed values ​​of a second plurality of luma samples; and predicting the chroma sample based on the first value and the second value.

[0007] The embodiments of the present disclosure also provide a non-transitory computer-readable medium storing a set of instructions, the set of instructions being executable by one or more processors of a device to cause the device to initiate a method for processing video data. The method includes: determining a first value associated with a chroma sample by applying a first gradient pattern to a reconstructed value of a first plurality of luma samples; determining a second value associated with the chroma sample by applying a downsampling filter to a reconstructed value of a second plurality of luma samples; and predicting the chroma sample based on the first value and the second value.

[0008] Embodiments of the present disclosure also provide a method for processing video data, the method comprising: predicting a chroma sample from a collocated luma sample associated with the chroma sample, wherein the prediction is based on a nonlinear model defining a nonlinear relationship between a predicted value of the chroma sample and a value associated with the collocated luma sample.

[0009] The embodiment of the present disclosure also provides a device for processing video data. The system includes: a memory storing a set of instructions; and one or more processors configured to execute the set of instructions to enable the device to perform: predicting a chroma sample from a collocated luma sample associated with the chroma sample; wherein the prediction is based on a nonlinear model defining a nonlinear relationship between a predicted value of the chroma sample and a value associated with the collocated luma sample.

[0010] Embodiments of the present disclosure also provide a non-transitory computer-readable medium storing a bitstream of a video for processing according to a method, the method comprising: predicting a chroma sample from a collocated luminance sample associated with the chroma sample, wherein the prediction is based on a non-linear model defining a non-linear relationship between a predicted value of the chroma sample and a value associated with the collocated luminance sample.

[0011] Embodiments of the present disclosure also provide a non-transitory computer-readable medium storing a set of instructions executable by one or more processors of a device to cause the device to initiate a method for processing video data, the method comprising: predicting a chroma sample from a collocated luma sample associated with the chroma sample, wherein the prediction is based on a non-linear model defining a non-linear relationship between a predicted value of the chroma sample and a value associated with the collocated luma sample.

[0012] An embodiment of the present disclosure also provides a computer program product, comprising: computer program instructions, the computer program instructions causing a computer to execute a method comprising: predicting a chroma sample from a collocated luminance sample associated with the chroma sample, wherein the prediction is based on a nonlinear model defining a nonlinear relationship between a predicted value of the chroma sample and a value associated with the collocated luminance sample.

[0013] An embodiment of the present disclosure also provides a computer program, which causes a computer to execute a method including: predicting a chroma sample from a collocated luminance sample associated with the chroma sample, wherein the prediction is based on a nonlinear model defining a nonlinear relationship between a predicted value of the chroma sample and a value associated with the collocated luminance sample. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Embodiments and various aspects of the present disclosure are illustrated in the following detailed description and accompanying drawings.The various features illustrated in the drawings are not drawn to scale.

[0015] Figure 1 is a schematic diagram of the structure of an example video sequence according to some embodiments of the present disclosure.

[0016] Figure 2A is a schematic diagram illustrating an example encoding process of a hybrid video coding system according to some embodiments of the present disclosure.

[0017] Figure 2B is a schematic diagram illustrating another example encoding process of a hybrid video coding system according to some embodiments of the present disclosure.

[0018] Figure 3A is a schematic diagram illustrating an example decoding process of a hybrid video coding system according to some embodiments of the present disclosure.

[0019] Figure 3B is a schematic diagram illustrating another example decoding process of a hybrid video coding system according to some embodiments of the present disclosure.

[0020] Figure 4 is a block diagram of an example device for encoding or decoding a video according to some embodiments of the present disclosure.

[0021] Figure 5 is a schematic diagram illustrating an example method for deriving parameters of a cross-component model using neighboring samples according to some embodiments of the present disclosure.

[0022] Figure 6 is a schematic diagram illustrating another example method for deriving parameters of a cross-component model using neighboring samples according to some embodiments of the present disclosure.

[0023] Figure 7 is a schematic diagram illustrating an example method for deriving parameters of a gradient model using samples of adjacent rows according to some embodiments of the present disclosure.

[0024] Figure 8 is a schematic diagram illustrating another example method for deriving parameters of a gradient model using samples of adjacent rows according to some embodiments of the present disclosure.

[0025] Fig. 9 is a flow chart of an example method for predicting chroma samples using a cross-component nonlinear model according to some embodiments of the present disclosure.

[0026] Fig.10 is a flow chart of an example method for predicting chroma samples using a gradient model according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0027] Reference will now be made in detail to exemplary embodiments, examples of which are shown in the accompanying drawings. Specific embodiments are described below with reference to the accompanying drawings, in which, unless otherwise specified, the same numbers in different drawings represent the same or similar elements. The implementations set forth in the following description of the exemplary embodiments do not represent all implementations consistent with the present invention. Instead, they are merely examples of devices and methods consistent with aspects related to the present invention as described in the appended claims. Specific aspects of the present disclosure are described in more detail below. In the event of a conflict with a term or definition incorporated by reference, the terms and definitions provided herein shall prevail.

[0028] Embodiments provided in the present disclosure relate to encoding and decoding video information, and more specifically, to methods and systems for predicting chrominance samples based on one or more collocated luminance samples, such a process being referred to as cross-component prediction throughout the disclosure. As described in detail below, cross-component prediction may employ a cross-component linear model (CCLM), a cross-component nonlinear model (CCNLM), a gradient model, or a combination thereof.

[0029] The disclosed CCLM, CCNLM and gradient model are used to encode or decode video data. Video is a set of static images (or "frames") arranged in a time sequence to store visual information. A video capture device (e.g., a camera) can be used to capture and store those images in time sequence, and a video playback device (e.g., a television, a computer, a smart phone, a tablet computer, a video player, or any end-user terminal with a display function) can be used to display such images in time sequence. In addition, in some applications, the video capture device can send the captured video to a video playback device (e.g., a computer with a monitor) in real time, such as for monitoring, conferencing, or live broadcasting.

[0030] In order to reduce the storage space and transmission bandwidth required for such applications, the video can be compressed before storage and transmission, and decompressed before display. Compression and decompression can be implemented by software executed by a processor (e.g., a processor of a general-purpose computer) or dedicated hardware. The module for compression is generally referred to as an "encoder", and the module for decompression is generally referred to as a "decoder". Encoders and decoders can be collectively referred to as "codecs". Encoders and decoders can be implemented as any of various suitable hardware, software, or combinations thereof. For example, the hardware implementation of encoders and decoders can include circuits such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, or any combination thereof. The software implementation of encoders and decoders can include program code, computer executable instructions, firmware, or any suitable computer-implemented algorithm or process fixed in a computer-readable medium. Video compression and decompression can be implemented by various algorithms or standards, such as MPEG-1, MPEG-2, MPEG-4, H.26x series, etc. In some applications, a codec can decompress a video from a first coding standard and recompress the decompressed video using a second coding standard, in which case the codec can be referred to as a "transcoder".

[0031] The video encoding process can identify and retain useful information that can be used to reconstruct the image, and ignore unimportant reconstruction information. If the ignored unimportant information cannot be completely reconstructed, such an encoding process can be called "lossy". Otherwise, it can be called "lossless". Most encoding processes are lossy, which is a trade-off to reduce the required storage space and transmission bandwidth.

[0032] Useful information about the image being encoded (referred to as the "current image") includes changes relative to a reference image (e.g., a previously encoded and reconstructed image). Such changes can include changes in pixel position, brightness, or color, with position changes being of greatest interest. Changes in the position of a group of pixels representing an object can reflect the object's motion between the reference image and the current image.

[0033] A picture that is encoded without reference to another picture (i.e., it is its own reference picture) is called an "I-picture." If some or all blocks (a block generally refers to a portion of a video picture) in a picture are predicted using intra-prediction or inter-prediction with a reference picture (e.g., unidirectional prediction), the picture is called a P-picture. If at least one block in a picture is predicted using two reference pictures (e.g., bidirectional prediction), the picture is called a "B-picture."

[0034] Figure 1 , Figure 2A , Figure 2B , Figure 3A , Figure 3B and Figure 4 The general aspects of the video encoding / decoding apparatus and processing used in the disclosed embodiments are shown. Specifically, Figure 1 The structure of an example video sequence 100 according to some embodiments of the present disclosure is shown. The video sequence 100 may be a live video or a video that has been captured and archived. The video 100 may be a real-life video, a computer-generated video (e.g., a computer game video), or a combination of the two (e.g., a real video with an augmented reality effect). The video sequence 100 may be input from a video capture device (e.g., a camera), a video archive containing previously captured videos (e.g., a video file stored in a storage device), or a video feed interface (e.g., a video broadcast transceiver) that receives video from a video content provider.

[0035] like Figure 1 As shown, video sequence 100 may include a series of images arranged in time along a timeline, including images 102, 104, 106, and 108. Images 102-106 are consecutive, and there are more images between images 106 and 108. Figure 1 , picture 102 is an I-picture, and its reference picture is picture 102 itself. Picture 104 is a P-picture, and its reference picture is picture 102, as indicated by the arrow. Picture 106 is a B-picture, and its reference pictures are pictures 104 and 108, as indicated by the arrow. In some embodiments, the reference picture of a picture (e.g., picture 104) may not be immediately before or after the picture. For example, the reference picture of picture 104 may be the picture before picture 102. It should be noted that the reference pictures of pictures 102-106 are merely examples, and the present disclosure is not limited to such pictures. Figure 1Example of a reference image shown.

[0036] Typically, video codecs do not encode or decode the entire image at once due to the computational complexity of the encoding and decoding tasks. Instead, they may segment the image into basic segments and encode or decode the image segments one by one. In this disclosure, such basic segments are referred to as basic processing units ("BPUs"). For example, Figure 1 Structure 110 in shows an example structure of an image of video sequence 100 (e.g., any of images 102-108). In structure 110, the image is divided into 4×4 basic processing units, whose boundaries are shown as dashed lines. In some embodiments, the basic processing unit may be referred to as a "macroblock" in some video coding standards (e.g., the MPEG family, H.261, H.263, or H.264 / AVC), or as a "coding tree unit" ("CTU") in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing unit may have a variable size in the image, such as 128×128, 64×64, 32×32, 16×16, 4×8, 16×32, or pixels of any shape and size. The size and shape of the basic processing unit may be selected for the image based on a balance between coding efficiency and the level of detail to be maintained in the basic processing unit.

[0037] A basic processing unit may be a logical unit that may include a set of different types of video data stored in a computer memory (e.g., in a video frame buffer). For example, a basic processing unit of a color image may include a luma component (Y) representing achromatic luminance information, one or more chroma components (e.g., Cb and Cr) representing color information, and associated syntax elements, where the luma and chroma components may have the same size as the basic processing unit. In some video coding standards (e.g., H.265 / HEVC or H.266 / VVC), luma and chroma components may be referred to as "coding tree blocks" ("CTBs"). Any operation performed on a basic processing unit may be repeated for each of its luma and chroma components.

[0038] Video encoding has multiple stages of operation, examples of which are Figure 2A-2B and Figure 3A-3BAs shown. For each stage, the size of the basic processing unit may still be too large for processing, so it can be further divided into segments referred to as "basic processing subunits" in this disclosure. In some embodiments, the basic processing subunit may be referred to as a "block" in some video coding standards (e.g., MPEG family, H.261, H.263 or H.264 / AVC), or as a "coding unit" ("CU") in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing subunit may have the same size as the basic processing unit or a smaller size than the basic processing unit. Similar to the basic processing unit, the basic processing subunit is also a logical unit, which may include a set of different types of video data (e.g., Y, Cb, Cr and associated syntax elements) stored in a computer memory (e.g., in a video frame buffer). Any operation performed on the basic processing subunit can be repeated for each of its luminance and chrominance components. It should be noted that this division can be performed to a further level according to processing needs. It should also be noted that different schemes can be used to divide the basic processing unit at different stages.

[0039] For example, in the mode decision phase (an example of which is in Figure 2B As shown in , the encoder can decide what prediction mode (e.g., intra prediction or inter prediction) to use for a basic processing unit, which may be too large to make such a decision. The encoder can divide the basic processing unit into multiple basic processing sub-units (e.g., CUs in H.265 / HEVC or H.266 / VVC) and decide the prediction type for each individual basic processing sub-unit.

[0040] For another example, in the prediction phase (the example in Figure 2A-2B ), the encoder can perform prediction operations at the level of basic processing sub-units (e.g., CUs). However, in some cases, the basic processing sub-units may still be too large to process. The encoder can further divide the basic processing sub-units into smaller segments (e.g., called "prediction blocks" or "PBs" in H.265 / HEVC or H.266 / VVC), at which prediction operations can be performed.

[0041] For another example, in the transformation phase (the example in Figure 2A-2B), the encoder can perform transform operations on the residual basic processing sub-unit (e.g., CU). However, in some cases, the basic processing sub-unit may still be too large to process. The encoder can further divide the basic processing sub-unit into smaller segments (e.g., called "transform blocks" or "TBs" in H.265 / HEVC or H.266 / VVC), at which level transform operations can be performed. It should be noted that the division scheme of the same basic processing sub-unit can be different in the prediction stage and the transform stage. For example, in H.265 / HEVC or H.266 / VVC, the prediction blocks and transform blocks of the same CU can have different sizes and numbers.

[0042] exist Figure 1 In the structure 110, the basic processing unit 112 is further divided into 3×3 basic processing sub-units, whose boundaries are shown by dotted lines. Different basic processing units of the same image can be divided into basic processing sub-units in different schemes.

[0043] In some embodiments, in order to provide parallel processing capabilities and fault tolerance for video encoding and decoding, the image can be divided into regions for processing, so that for the region of the image, the encoding or decoding process can be independent of information from any other region of the image. In other words, each region of the image can be processed separately. By doing so, the codec can process different regions of the image in parallel, thereby improving coding efficiency. In addition, when the data of a region is damaged during processing or lost during network transmission, the codec can correctly encode or decode other regions of the same image without relying on the damaged or lost data, thereby providing fault tolerance. In some video coding standards, the image can be divided into different types of regions. For example, H.265 / HEVC and H.266 / VVC provide two types of regions: "slices" and "tiles". It should also be noted that different images of the video sequence 100 can have different partitioning schemes for dividing the image into regions.

[0044] For example, in Figure 1 In FIG. 1 , the structure 110 is divided into three regions 114, 116, and 118, whose boundaries are shown as solid lines inside the structure 110. The region 114 includes four basic processing units. The regions 116 and 118 each include six basic processing units. It should be noted that Figure 1 The basic processing units, basic processing sub-units, and structural areas in 110 are merely examples, and the present disclosure does not limit the embodiments thereof.

[0045] Figure 2A FIG. 2 is a schematic diagram of an exemplary encoding process 200A according to an embodiment of the present disclosure. For example, the encoding process 200A may be performed by an encoder. Figure 2AAs shown, the encoder may encode the video sequence 202 into a video bitstream 228 according to process 200A. Figure 1 The video sequence 100 in FIG. 200 may include a set of images (referred to as “original images”) arranged in time sequence. Figure 1 In the structure 110 in the video sequence 202, each original image of the video sequence 202 can be divided into basic processing units, basic processing sub-units or regions for processing by the encoder. In some embodiments, the encoder can perform process 200A at the level of basic processing units for each original image of the video sequence 202. For example, the encoder can perform process 200A in an iterative manner, wherein the encoder can encode the basic processing unit in one iteration of process 200A. In some embodiments, the encoder can perform process 200A in parallel for regions (e.g., regions 114-118) of each original image of the video sequence 202.

[0046] refer to Figure 2A , the encoder may feed the basic processing unit of the original image of the video sequence 202 (referred to as the "original BPU") to the prediction stage 204 to generate prediction data 206 and prediction BPU 208. The encoder may subtract the prediction BPU 208 from the original BPU to generate a residual BPU 210. The encoder may feed the residual BPU 210 to the transform stage 212 and the quantization stage 214 to generate quantized transform coefficients 216. The encoder may feed the prediction data 206 and the quantized transform coefficients 216 to the binary encoding stage 226 to generate a video bitstream 228. The components 202, 204, 206, 208, 210, 212, 214, 216, 226, and 228 may be referred to as the "forward path". During the process 200A, after the quantization stage 214, the encoder may feed the quantized transform coefficients 216 to the inverse quantization stage 218 and the inverse transform stage 220 to generate a reconstructed residual BPU 222. The encoder may add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a prediction reference 224, which is used in the prediction phase 204 of the next iteration of the process 200A. The components 218, 220, 222, and 224 of the process 200A may be referred to as a "reconstruction path." The reconstruction path may be used to ensure that both the encoder and the decoder use the same reference data for prediction.

[0047] The encoder can iteratively perform process 200A to encode each original BPU of the encoded original image (in the forward path) and generate a prediction reference 224 for the next original BPU of the encoded original image (in the reconstruction path). After encoding all the original BPUs of the original image, the encoder can continue to encode the next image in the video sequence 202.

[0048] Referring to process 200A, an encoder may receive a video sequence 202 generated by a video acquisition device (eg, a camera). As used herein, the term "receiving" may refer to any action of receiving, inputting, acquiring, retrieving, obtaining, reading, accessing, or inputting data in any manner.

[0049] In the prediction phase 204, at the current iteration, the encoder may receive the original BPU and the prediction reference 224 and perform a prediction operation to generate the prediction data 206 and the prediction BPU 208. The prediction reference 224 may be generated from the reconstruction path of the previous iteration of the process 200A. The purpose of the prediction phase 204 is to reduce information redundancy by extracting the prediction data 206 that can be used to reconstruct the original BPU into the prediction BPU 208 from the prediction data 206 and the prediction reference 224.

[0050] Ideally, the predicted BPU 208 can be the same as the original BPU. However, due to non-ideal prediction and reconstruction operations, the predicted BPU 208 is usually slightly different from the original BPU. In order to record these differences, after generating the predicted BPU 208, the encoder can subtract it from the original BPU to generate a residual BPU 210. For example, the encoder can subtract the value of the corresponding pixel of the predicted BPU 208 (e.g., grayscale value or RGB value) from the value of the pixel of the original BPU. Each pixel of the residual BPU 210 can have a residual value as a result of this subtraction between the corresponding pixels of the original BPU and the predicted BPU 208. Compared with the original BPU, the predicted data 206 and the residual BPU 210 can have fewer bits, but they can be used to reconstruct the original BPU without significant quality degradation. Therefore, the original BPU is compressed.

[0051] To further compress the residual BPU 210, in the transform stage 212, the encoder can reduce the spatial redundancy of the residual BPU 210 by decomposing it into a set of two-dimensional "base patterns". Each base pattern is associated with a "transform coefficient". The base patterns can have the same size (e.g., the size of the residual BPU 210), and each base pattern can represent a changing frequency (e.g., the frequency of brightness changes) component of the residual BPU 210. None of the base patterns can be reproduced from any combination (e.g., linear combination) of any other base patterns. In other words, the decomposition can decompose the changes of the residual BPU 210 into the frequency domain. This decomposition is similar to the discrete Fourier transform of a function, where the base pattern is similar to the basis functions of the discrete Fourier transform (e.g., trigonometric functions), and the transform coefficients are similar to the coefficients associated with the basis functions.

[0052] Different transform algorithms can use different basic modes. Various transform algorithms can be used in the transform stage 212, such as discrete cosine transform, discrete sine transform, etc. The transform at the transform stage 212 is reversible. That is, the encoder can restore the residual BPU 210 by the inverse operation of the transform (called "inverse transform"). For example, in order to restore the pixels of the residual BPU 210, the inverse transform can be to multiply the values ​​of the corresponding pixels of the basic mode by the corresponding associated coefficients and add the products to produce a weighted sum. For video coding standards, both the encoder and the decoder can use the same transform algorithm (and therefore have the same basic mode). Therefore, the encoder can only record the transform coefficients, from which the decoder can reconstruct the residual BPU 210 without receiving the basic mode from the encoder. Compared with the residual BPU 210, the transform coefficients can have fewer bits, but they can be used to reconstruct the residual BPU 210 without significant quality degradation. Therefore, the residual BPU 210 is further compressed.

[0053] The encoder may further compress the transform coefficients in the quantization stage 214. During the transform process, different basic modes may represent different frequencies of change (e.g., frequencies of brightness changes). Because the human eye is generally better at recognizing low-frequency changes, the encoder may ignore information about high-frequency changes without causing noticeable quality degradation in decoding. For example, in the quantization stage 214, the encoder may generate quantized transform coefficients 216 by dividing each transform coefficient by an integer value (referred to as a "quantization parameter") and rounding the quotient to its nearest integer. After such an operation, some transform coefficients of the high-frequency basic mode may be converted to zero, and the transform coefficients of the low-frequency basic mode may be converted to smaller integers. The encoder may ignore the quantized transform coefficients 216 of zero value, whereby the transform coefficients are further compressed. The quantization process is also reversible, where the quantized transform coefficients 216 may be reconstructed as transform coefficients in an inverse operation of quantization (referred to as "inverse quantization").

[0054] Because the encoder ignores the remainder of the division in the rounding operation, the quantization stage 214 may be lossy. Generally, the quantization stage 214 may contribute the most information loss in the process 200A. The greater the information loss, the fewer the number of bits required to quantize the transform coefficients 216. To achieve different levels of information loss, the encoder may use different quantization parameter values ​​or any other parameters of the quantization process.

[0055] In the binary encoding stage 226, the encoder may encode the prediction data 206 and the quantized transform coefficients 216 using a binary encoding technique, such as entropy coding, variable length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless or lossy compression algorithm. In some embodiments, in addition to the prediction data 206 and the quantized transform coefficients 216, the encoder may encode other information in the binary encoding stage 226, such as the prediction mode used in the prediction stage 204, the parameters of the prediction operation, the transform type at the transform stage 212, the parameters of the quantization process (e.g., quantization parameters), the encoder control parameters (e.g., bit rate control parameters), etc. The encoder may use the output data of the binary encoding stage 226 to generate a video bitstream 228. In some embodiments, the video bitstream 228 may be further packaged for network transmission.

[0056] Referring to the reconstruction path of process 200A, at the inverse quantization stage 218, the encoder may perform inverse quantization on the quantized transform coefficients 216 to generate reconstructed transform coefficients. At the inverse transform stage 220, the encoder may generate a reconstructed residual BPU 222 based on the reconstructed transform coefficients. The encoder may add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224 to be used in the next iteration of process 200A.

[0057] It should be noted that other variations of process 200A may be used to encode video sequence 202. In some embodiments, the stages of process 200A may be performed by the encoder in a different order. In some embodiments, one or more stages of process 200A may be combined into a single stage. In some embodiments, a single stage of process 200A may be divided into multiple stages. For example, transform stage 212 and quantization stage 214 may be combined into a single stage. In some embodiments, process 200A may include additional stages. In some embodiments, process 200A may omit Figure 2A one or more stages in a process.

[0058] Figure 2B A schematic diagram of another example encoding process 200B according to an embodiment of the present disclosure is shown. Process 200B can be modified from process 200A. For example, process 200B can be used by an encoder that complies with a hybrid video coding standard (e.g., H.26x series). Compared with process 200A, the forward path of process 200B also includes a mode decision stage 230, and the prediction stage 204 is divided into a spatial prediction stage 2042 and a temporal prediction stage 2044, and the reconstruction path of process 200B further includes a loop filtering stage 232 and a buffer 234.

[0059] In general, prediction techniques can be divided into two types: spatial prediction and temporal prediction. Spatial prediction (e.g., intra-image prediction or "intra-frame prediction") can use pixels from one or more already encoded adjacent BPUs in the same image to predict the current BPU. That is, the prediction reference 224 in spatial prediction may include adjacent BPUs. Spatial prediction can reduce the spatial redundancy inherent in the image. Temporal prediction (e.g., inter-image prediction or "inter-frame prediction") can use areas from one or more already encoded images to predict the current BPU. That is, the prediction reference 224 in temporal prediction may include encoded images. Temporal prediction can reduce the temporal redundancy inherent in the image.

[0060] Referring to process 200B, in the forward path, the encoder performs prediction operations in the spatial prediction stage 2042 and the temporal prediction stage 2044. For example, in the spatial prediction stage 2042, the encoder may perform intra-frame prediction. For the original BPU of the image being encoded, the prediction reference 224 may include one or more neighboring BPUs that have been encoded (in the forward path) and reconstructed (in the reconstruction path) in the same image. The encoder may generate the predicted BPU 208 by interpolating the neighboring BPUs. Interpolation techniques may include, for example, linear interpolation or interpolation, polynomial interpolation or interpolation, etc. In some embodiments, the encoder may perform interpolation at the pixel level, such as by interpolating the value of the corresponding pixel of each pixel of the predicted BPU 208. The neighboring BPUs used for interpolation may be located in various directions relative to the original BPU, such as in the vertical direction (e.g., at the top of the original BPU), the horizontal direction (e.g., on the left side of the original BPU), the diagonal direction (e.g., at the bottom left, bottom right, top left, or top right of the original BPU), or any direction defined in the video coding standard used. For intra prediction, the prediction data 206 may include, for example, the location (eg, coordinates) of the used neighboring BPU, the size of the used neighboring BPU, interpolation parameters, the direction of the used neighboring BPU relative to the original BPU, and the like.

[0061] For another example, in the temporal prediction stage 2044, the encoder may perform inter-frame prediction. For the original BPU of the current image, the prediction reference 224 may include one or more images (referred to as "reference images") that have been encoded (in the forward path) and reconstructed (in the reconstruction path). In some embodiments, the reference image may be encoded and reconstructed by BPUs one by one. For example, the encoder may add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a reconstructed BPU. When all reconstructed BPUs of the same image are generated, the encoder may generate a reconstructed image as a reference image. The encoder may perform a "motion estimation" operation to search for a matching area in a range (referred to as a "search window") of the reference image. The position of the search window in the reference image may be determined based on the position of the original BPU in the current image. For example, the search window may be centered at a position in the reference image having the same coordinates as the original BPU in the current image, and may extend outward by a predetermined distance. When the encoder identifies (e.g., by using a pel recursive algorithm, a block matching algorithm, etc.) an area similar to the original BPU in the search window, the encoder may determine such an area as a matching area. The matching region may have a different size than the original BPU (e.g., smaller, equal, larger, or having a different shape). Because the reference image and the current image are temporally separated on the timeline (e.g., as Figure 1 ), so the matching area can be considered to "move" to the location of the original BPU over time. The encoder can record the direction and distance of this movement as a "motion vector". Figure 1 The encoder can search for matching areas and determine the motion vector associated with each reference image. In some embodiments, the encoder can assign weights to the pixel values ​​of the matching areas of each matching reference image.

[0062] Motion estimation may be used to identify various types of motion, such as translation, rotation, scaling, etc. For inter-frame prediction, the prediction data 206 may include, for example, the location (e.g., coordinates) of the matching region, the motion vector associated with the matching region, the number of reference images, the weights associated with the reference images, etc.

[0063] To generate the predicted BPU 208, the encoder may perform a "motion compensation" operation. Motion compensation may be used to reconstruct the predicted BPU 208 based on the prediction data 206 (e.g., motion vectors) and the prediction reference 224. For example, the encoder may move a matching area of ​​a reference image according to the motion vector, where the encoder may predict the original BPU of the current image. Figure 1The encoder may move the matching region of the reference image according to the motion vectors and the average pixel value of the matching region. In some embodiments, if the encoder has assigned weights to the pixel values ​​of the matching regions of the matching reference image, the encoder may add the weighted sum of the pixel values ​​of the moving matching regions.

[0064] In some embodiments, inter-frame prediction can be unidirectional or bidirectional. Unidirectional inter-frame prediction can use one or more reference images in the same temporal direction relative to the current image. For example, Figure 1 The picture 104 in is a unidirectional inter-frame prediction picture, where the reference picture (i.e., picture 102) precedes the picture 104. Bidirectional inter-frame prediction can use one or more reference pictures in both temporal directions relative to the current picture. For example, Figure 1 Picture 106 in is a bi-directional inter-predicted picture, where the reference pictures (ie, pictures 104 and 108) are relative to picture 104 in both temporal directions.

[0065] Still referring to the forward path of process 200B, after spatial prediction 2042 and temporal prediction stage 2044, at mode decision stage 230, the encoder can select a prediction mode (e.g., one of intra prediction or inter prediction) for the current iteration of process 200B. For example, the encoder can perform a rate-distortion optimization technique, in which the encoder can select a prediction mode to minimize the value of a cost function based on the bit rate of a candidate prediction mode and the distortion of a reconstructed reference image under the candidate prediction mode. Based on the selected prediction mode, the encoder can generate a corresponding prediction BPU 208 and prediction data 206.

[0066] In the reconstruction path of process 200B, if the intra-frame prediction mode has been selected in the forward path, after generating the prediction reference 224 (e.g., the current BPU that has been encoded and reconstructed in the current image), the encoder can feed the prediction reference 224 directly to the spatial prediction stage 2042 for later use (e.g., for interpolating the next BPU of the current image). If the inter-frame prediction mode has been selected in the forward path, after generating the prediction reference 224 (e.g., the current image in which all BPUs have been encoded and reconstructed), the encoder can feed the prediction reference 224 to the loop filter stage 232. At this stage, the encoder can apply a loop filter to the prediction reference 224 to reduce or eliminate distortion (e.g., blocking artifacts) introduced by the inter-frame prediction. The encoder can apply various loop filter techniques at the loop filter stage 232, such as deblocking, sampling adaptive compensation, adaptive loop filter, etc. The loop filtered reference pictures may be stored in a buffer 234 (or "decoded picture buffer") for later use (e.g., as inter-prediction reference pictures for future pictures of the video sequence 202). The encoder may store one or more reference pictures in the buffer 234 for use at the temporal prediction stage 2044. In some embodiments, the encoder may encode the parameters of the loop filter (e.g., loop filter strength) as well as the quantized transform coefficients 216, the prediction data 206, and other information at the binary encoding stage 226.

[0067] Figure 3A FIG. 3 is a schematic diagram showing an exemplary decoding process 300A according to an embodiment of the present disclosure. The process 300A may correspond to Figure 2A 300A. In some embodiments, process 300A may be similar to the reconstruction path of process 200A. The decoder may decode video bitstream 228 into video stream 304 according to process 300A. Video stream 304 may be very similar to video sequence 202. However, due to information loss during compression and decompression (e.g., Figure 2A-2B 214), typically, the video stream 304 is different from the video sequence 202. Figure 2A-2B 200A and 200B in the decoder, the decoder may perform process 300A at a basic processing unit (BPU) level for each picture encoded in the video bitstream 228. For example, the decoder may perform process 300A in an iterative manner, where the decoder may decode a basic processing unit in one iteration of process 300A. In some embodiments, the decoder may perform process 300A in parallel for regions (e.g., regions 114-118) of each picture encoded in the video bitstream 228.

[0068] like Figure 3AAs shown, the decoder may feed a portion of the video bitstream 228 associated with a basic processing unit (referred to as an "encoded BPU") of an encoded picture to a binary decoding stage 302, where the decoder may decode the portion into prediction data 206 and quantized transform coefficients 216. The decoder may feed the quantized transform coefficients 216 to an inverse quantization stage 218 and an inverse transform stage 220 to generate a reconstructed residual BPU 222. The decoder may feed the prediction data 206 to the prediction stage 204 to generate the prediction BPU 208. The decoder may add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224. In some embodiments, the prediction reference 224 may be stored in a buffer (e.g., a decoded picture buffer in a computer memory). The decoder may feed the prediction reference 224 to the prediction stage 204 for performing a prediction operation in the next iteration of the process 300A.

[0069] The decoder may iteratively perform process 300A to decode each coded BPU of a coded picture and generate a prediction reference 224 for the next coded BPU of the coded picture. After decoding all coded BPUs of a coded picture, the decoder may output the picture to a video stream 304 for display and continue decoding the next coded picture in the video bitstream 228.

[0070] At the binary decoding stage 302, the decoder may perform the inverse operation of the binary coding technique used by the encoder (e.g., entropy coding, variable length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless compression algorithm). In some embodiments, in addition to the prediction data 206 and the quantized transform coefficients 216, the decoder may decode other information at the binary decoding stage 302, such as the prediction mode, parameters of the prediction operation, the transform type, parameters of the quantization process (e.g., quantization parameters), encoder control parameters (e.g., bit rate control parameters), etc. In some embodiments, if the video bitstream 228 is transmitted over the network in the form of packets, the decoder may depacketize the video bitstream 228 before feeding it to the binary decoding stage 302.

[0071] Figure 3B A schematic diagram of another example decoding process 300B according to an embodiment of the present disclosure is shown. Process 300B can be modified from process 300A. For example, process 300B can be used by a decoder that complies with a hybrid video coding standard (e.g., H.26x series). Compared to process 300A, process 300B additionally divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044, and additionally includes a loop filtering stage 232 and a buffer 234.

[0072] In process 300B, for an encoded basic processing unit (referred to as a "current BPU") of a decoded encoded image (referred to as a "current image"), the prediction data 206 decoded by the decoder from the binary decoding stage 302 may include various types of data, depending on what prediction mode the encoder used to encode the current BPU. For example, if the encoder uses intra-frame prediction to encode the current BPU, the prediction data 206 may include a prediction mode indicator (e.g., a flag value) indicating intra-frame prediction, parameters of the intra-frame prediction operation, etc. The parameters of the intra-frame prediction operation may include, for example, the location (e.g., coordinates) of one or more neighboring BPUs used as a reference, the size of the neighboring BPU, parameters of interpolation, the direction of the neighboring BPU relative to the original BPU, etc. For another example, if the current BPU is encoded by inter-frame prediction used by the encoder, the prediction data 206 may include a prediction mode indicator (e.g., a flag value) indicating inter-frame prediction, parameters of the inter-frame prediction operation, etc. The parameters of the inter-frame prediction operation may include, for example, the number of reference images associated with the current BPU, the weights respectively associated with the reference images, the positions (e.g., coordinates) of one or more matching regions in the corresponding reference images, one or more motion vectors respectively associated with the matching regions, and the like.

[0073] Based on the prediction mode indicator, the decoder can decide whether to perform spatial prediction (e.g., intra-frame prediction) in the spatial prediction stage 2042 or to perform temporal prediction (e.g., inter-frame prediction) in the temporal prediction stage 2044. The details of performing such spatial prediction or temporal prediction are described in Figure 2B After performing such spatial prediction or temporal prediction, the decoder may generate a predicted BPU 208, and the decoder may add the predicted BPU 208 and the reconstructed residual BPU 222 to generate a prediction reference 224, as shown in FIG. Figure 3A As described in.

[0074] In process 300B, the decoder can feed the prediction reference 224 to the spatial prediction stage 2042 or the temporal prediction stage 2044 for performing a prediction operation in the next iteration of process 300B. For example, if the current BPU is decoded using intra-frame prediction in the spatial prediction stage 2042, then after generating the prediction reference 224 (e.g., the decoded current BPU), the decoder can feed the prediction reference 224 directly to the spatial prediction stage 2042 for later use (e.g., for interpolating the next BPU of the current image). If the current BPU is decoded using inter-frame prediction in the temporal prediction stage 2044, then after generating the prediction reference 224 (e.g., the reference image in which all BPUs are decoded), the encoder can feed the prediction reference 224 to the loop filter stage 232 to reduce or eliminate distortion (e.g., blocking artifacts). The decoder can Figure 2BThe loop filter is applied to the prediction reference 224 in the manner shown. The loop filtered reference picture may be stored in a buffer 234 (e.g., a decoded picture buffer in a computer memory) for later use (e.g., as an inter-prediction reference picture for future encoded pictures of the video bitstream 228). The decoder may store one or more reference pictures in the buffer 234 for use at the temporal prediction stage 2044. In some embodiments, when the prediction mode indicator of the prediction data 206 indicates that inter-frame prediction was used to encode the current BPU, the prediction data may further include parameters of the loop filter (e.g., loop filter strength).

[0075] Figure 4 4 is a block diagram of an example apparatus 400 for encoding or decoding a video according to an embodiment of the present disclosure. Figure 4 As shown, the device 400 may include a processor 402. When the processor 402 executes the instructions described herein, the device 400 may become a special-purpose machine for video encoding or decoding. The processor 402 may be any type of circuit capable of manipulating or processing information. For example, the processor 402 may include any number of central processing units (or "CPUs"), graphics processing units (or "GPUs"), neural processing units ("NPUs"), microcontroller units ("MCUs"), optical processors, programmable logic controllers, microprocessors, digital signal processors, intellectual property (IP) cores, programmable logic arrays (PLAs), programmable array logic (PALs), general array logic (GALs), complex programmable logic devices (CPLDs), a field programmable gate array (FPGA), a system on a chip (SoC), an application-specific integrated circuit (ASIC), and the like. In some embodiments, the processor 402 may also be a group of processors grouped into a single logical component. For example, as Figure 4 As shown, processor 402 may include multiple processors, including processor 402a, processor 402b, and processor 402n.

[0076] The apparatus 400 may also include a memory 404 configured to store data (eg, instruction sets, computer codes, intermediate data, etc.). Figure 4As shown, the stored data may include program instructions (e.g., for implementing stages in process 200A, 200B, 300A, or 300B) and data for processing (e.g., video sequence 202, video bitstream 228, or video stream 304). Processor 402 may access program instructions and data for processing (e.g., via bus 410), and execute program instructions to perform operations or manipulations on data for processing. Memory 404 may include a high-speed random access storage device or a non-volatile storage device. In some embodiments, memory 404 may include any number of random access memories (RAM), read-only memories (ROM), optical disks, magnetic disks, hard drives, solid-state drives, flash drives, secure digital (SD) cards, memory sticks, compact flash (CF) cards, etc. Any combination. Memory 404 may also be a group of memories grouped into a single logical component ( Figure 4 not shown).

[0077] The bus 410 may be a communication device that transmits data between components within the apparatus 400, such as an internal bus (eg, a CPU-memory bus), an external bus (eg, a Universal Serial Bus port, a Peripheral Component Interconnect Express port), or the like.

[0078] For ease of explanation and without ambiguity, in the present disclosure, the processor 402 and other data processing circuits are collectively referred to as "data processing circuits". The data processing circuits may be implemented entirely as hardware, or as a combination of software, hardware, or firmware. In addition, the data processing circuits may be a single separate module, or may be fully or partially combined into any other component of the device 400.

[0079] The device 400 may also include a network interface 406 to provide wired or wireless communication with a network (e.g., the Internet, an intranet, a local area network, a mobile communication network, etc.). In some embodiments, the network interface 406 may include any number of network interface controllers (NICs), radio frequency (RF) modules, transponders, transceivers, modems, routers, gateways, wired network adapters, wireless network adapters, Bluetooth adapters, infrared adapters, near field communication (“NFC”) adapters, cellular network chips, etc.

[0080] In some embodiments, the apparatus 400 may optionally further include a peripheral interface 408 to provide a connection to one or more peripheral devices. Figure 4 As shown, peripheral devices may include, but are not limited to, a cursor control device (e.g., a mouse, touch pad, or touch screen), a keyboard, a display (e.g., a cathode ray tube display, a liquid crystal display, or a light emitting diode display), a video input device (e.g., a camera or an input interface coupled to a video archive), and the like.

[0081] It should be noted that the video codec (e.g., a codec that performs the process 200A, 200B, 300A, or 300B) can be implemented as any combination of any software or hardware modules in the device 400. For example, some or all stages of the process 200A, 200B, 300A, or 30013 can be implemented as one or more software modules of the device 400, such as a program instance that can be loaded into the memory 404. For another example, some or all stages of the process 200A, 200B, 300A, or 300B can be implemented as one or more hardware modules of the device 400, such as a dedicated data processing circuit (e.g., FPGA, ASIC, NPU, etc.).

[0082] The present disclosure provides a video encoding and decoding method for predicting chrominance samples based on juxtaposed luminance samples using a cross-component linear model (CCLM), a cross-component nonlinear model (CCNLM), or a gradient model. The disclosed models can be freely combined to perform chrominance sample prediction. The disclosed cross-component prediction method can reduce cross-component redundancy and can be used in conjunction with any image / video coding standard, such as Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), AOMedia Video 1 (AV1), Joint Photographic Experts Group (JPEG), Moving Picture Experts Group (MPEG), etc.

[0083] According to some embodiments, the CCLM may be used to predict the chrominance samples of a block from the collocated reconstructed luma samples via a linear model as in Equation 1. pred C (i,j)=α·rec′ L (i,j)+β (Formula 1) where pred C (i, j) represents the predicted value of the chroma sample in the current block, and rec′ L (i, j) represents the reconstructed value of the collocated luminance samples of the same block downsampled for non-4:4:4 color formats; (i, j) are the coordinates of the sample in the block; the coefficients α and β can represent a linear model, and the values ​​of the two coefficients are derived based on the reconstructed samples adjacent to the current block at the encoder and decoder sides without explicit signal notification.

[0084] The following describes three CCLM models in detail: CCLM_LT, CCLM_L, and CCLM_T. These three modes differ in the location of the reconstructed neighboring samples used for the derivation of the linear model parameters (α and β). The neighboring samples reconstructed above refer to the CCLM_T mode, while the neighboring samples reconstructed on the left refer to the CCLM_L mode. In the CCLM_LT mode, both the above and left reconstructed neighboring samples are used.

[0085] In the signaling of the chroma intra mode, a flag indicating whether CCLM is applied is first signaled. If the flag is signaled as true, it is further signaled which of the three CCLM modes is applied.

[0086] In the disclosed embodiment, downsampling of reconstructed luma samples may be used. In order to match the chroma sample positions of a 4:2:0 or 4:2:2 color format video sequence, two types of downsampling filters shown in Formula 2 and Formula 3 may be applied to luma samples, both of which have a downsampling ratio of 2 to 1 in both the horizontal and vertical directions. Based on the flag of the SPS level, a 2-dimensional 6-tap or 5-tap filter is applied to the luma samples within the current block and its neighboring luma samples. When the flag of the SPS level is equal to 1, it signals the prediction process to operate in a manner designed for chroma sample positions that are not vertically shifted relative to the corresponding luma sample positions, and a 5-tap filter is used; and when the flag of the SPS level is equal to 0, it signals the prediction process to operate in a manner designed as follows: the chroma sample positions are shifted downward by 0.5 in units of luma samples relative to the corresponding luma sample positions, and a 6-tap filter is used. If the top row of the current block is a CTU boundary, an exception occurs. In this case, a one-dimensional 3-tap filter as shown in Formula 4 is applied to the above-mentioned neighboring luma samples to avoid using more than one luma row above the CTU boundary. f3=(1 2 1) (Formula 4)

[0087] The downsampling process using the above-mentioned filter can be expressed by the following formula, where Formula 5, Formula 6, and Formula 7 correspond to the filters in Formula 2, Formula 3, and Formula 4, respectively. rec L ′(i,j)=[rec L (2i-1,2j)+2·rec L (2i,2j)+rec L (2i+1,2j)+2]>>2 (Formula 7) where rec L Represents the reconstructed value of the collocated luminance samples, rec′ L Represents the reconstructed value of the downsampled and concatenated luma samples.

[0088] The linear model coefficients α and β are derived based on reconstructed neighboring chroma samples and their corresponding reconstructed luma samples, which are downsampled for non-4:4:4 color formats at the encoder and decoder sides to avoid any signaling overhead.

[0089] In the original version of the CCLM model, the linear minimum mean square error (LMMSE) estimator is used to derive the parameters: Among them, rec′ L (n) represents the reconstructed value of the downsampled adjacent luminance samples, rec C (n) represents the reconstructed value of the adjacent chrominance samples, and N is the total number of adjacent samples used. Figure 5 Neighboring samples for deriving parameters of a cross-component model according to some embodiments of the present disclosure are shown. Figure 5 In , neighboring samples used to derive model parameters are shown as circles. Figure 5 As shown, for a W×H chroma CU, CCLM_LT mode uses the W adjacent samples above and the H adjacent samples on the left ( Figure 5 A), CCLM_L mode uses the left adjacent (H+W) samples ( Figure 5 B), and CCLM_T mode uses the above adjacent (W+H) samples ( Figure 5 C).

[0090] In some embodiments, to ensure that the number of samples used to derive linear model parameters is a power of 2, the number of adjacent samples used to derive model parameters may be increased or decreased.

[0091] For example, to reduce computational complexity, only four adjacent samples may be used to derive model parameters. Figure 6 is a schematic diagram illustrating a method for deriving model parameters using neighboring samples according to some embodiments of the present disclosure. Figure 6 In , neighboring samples used to derive model parameters are shown as circles. Figure 6 In A, for a W×H chroma block (e.g., an 8×8 chroma CU), the four neighboring samples used in CCLM_LT mode are the samples at the positions of W / 4 and 3w / 4 at the top boundary and at the positions of H / 4 and 3h / 4 at the left boundary. Figure 6 As shown in B, for CCLM_L mode, the left boundary is extended to the size of (W+H) samples, and the four samples used for model parameter derivation are located at positions (W+H) / 8, 3(W+H) / 8, 5(W+H) / 8, and 7(W+H) / 8. Figure 6As shown in C, for the CCLM_T mode, the top boundary is extended to the size of (W+H) samples, and the four samples used for model parameter derivation are located at positions (W+H) / 8, 3(W+H) / 8, 5(W+H) / 8 and 7(W+H) / 8.

[0092] The four reconstructed downsampled adjacent luma samples at the chosen position are compared four times to find the two smaller values: and and two larger values: and Their corresponding reconstructed chrominance sample values ​​are expressed as and Then L min ,L max ,C min and C max It is derived as:

[0093] Finally, the linear model coefficients α and β are obtained according to the following formula. β=C min -α·L min (Formula 12)

[0094] The division operation for calculating the parameter α is implemented using a lookup table. To reduce the memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are represented by exponential notation. For example, a 4-bit significant part and an exponent are used to approximate the diff. Therefore, the table for 1 / diff is reduced to 16 elements for 16 significant values, as shown below: DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0}(Formula 13)

[0095] This will have the benefit of reducing the complexity of the calculations as well as the memory size required to store the required tables.

[0096] In the disclosed embodiment, a multi-model CCLM can be used. The CCLM can be extended by adding three multi-model CCLM (MMLM) modes MMLM_LT, MMLM_L, and MMLM_T. The differences between the three modes are the same as the differences between CCLM_LT, CCLM_L, and CCLM_T modes, namely the positions of the reconstructed neighboring samples used for the derivation of the linear model parameters (α and β). In each MMLM mode, there may be more than one linear model between the luminance and chrominance in the block. First, the reconstructed neighboring samples are classified into two categories using a threshold value that is the average value of the values ​​of the neighboring samples reconstructed by the luminance. Then, using the above-mentioned LMMSE method, each class is treated as an independent training set to derive a linear model. Subsequently, the reconstructed luminance samples of the current block are also classified based on the same rules. Finally, the chrominance samples are predicted by the luminance samples reconstructed differently in different categories.

[0097] In the disclosed embodiment, a gradient linear model (GLM) method may be used. Compared to CCLM, GLM utilizes the gradient of luminance samples to derive a linear model instead of downsampling the reconstructed luminance samples. In other words, the gradient G is used in the CCLM process instead of using the filter in Formulas 4-6. The other designs of CCLM (e.g., parameter derivation, linear transformation of predicted samples) remain unchanged. The gradient G can be calculated by one of four Sobel-based gradient modes:

[0098] By using the above-mentioned gradient mode, the gradient G may be calculated by the following formula, wherein Formula 18, Formula 19, Formula 20, and Formula 21 correspond to the gradient modes in Formula 14, Formula 15, Formula 16, and Formula 17, respectively.

[0099] The linear model parameters α and β are derived based on the corresponding gradients G of the reconstructed adjacent chroma samples and the collocated reconstructed luma samples at the encoder and decoder sides by the same method of CCLM (e.g., LMMSE method). Then, the chroma samples of the block can be predicted from the gradients of the collocated reconstructed luma samples by the linear model: pred C (i,j)=α·G L (i,j)+β (Formula 22)

[0100] For signaling, when CCLM mode is enabled for the current CU, two flags are signaled separately for Cb and Cr components to indicate whether GLM is enabled for the components; if GLM is enabled for a component, a syntax element is further signaled to select one of four gradient modes for gradient calculation. In some embodiments, the syntax element is encoded using a two-bit fixed length code.

[0101] In some embodiments, GLM can only be used for some CCLM modes. For example, GLM can only be used for CCLM_LT mode, that is, for CCLM_LT mode, some syntax elements are signaled to indicate whether GLM is enabled and which gradient mode is used. If GLM is enabled for CCLM_LT mode, the gradient G of the upper and left adjacent reconstruction brightness samples is used to replace the downsampled reconstruction adjacent brightness samples in the linear model parameter derivation process, and only the signal linear model is used in the current block. If GLM is disabled for CCLM_LT mode, the original CCLM_LT mode is applied. There is no change for other CCLM modes (that is, CCLM_L, CCLM_T and three MMLM modes). For another example, GLM can only be used for CCLM_LT mode and MMLM_LT mode. For another example, GLM can only be used for CCLM_LT mode, CCLM_L mode and CCLM_T mode. For another example, GLM can be used for all six CCLM modes.

[0102] When GLM is applied to the MMLM mode, a multi-model GLM (MMGLM) method is used. In the MMGLM mode, there may be more than one linear model between the gradient G in the block and the chrominance samples. When implementing the MMGLM method, the gradients of the reconstructed adjacent samples are first classified into two categories using a threshold value, which is the average value of the gradient values ​​of the adjacent samples of the luminance reconstruction. Then, using the above-mentioned LMMSE method, each classification is regarded as an independent training set to derive a linear model. Subsequently, the gradients of the reconstructed luminance samples of the current block are also classified based on the same rules. Finally, the chrominance samples are predicted by the gradients of the luminance samples reconstructed differently in different categories.

[0103] In some embodiments, the GLM method supports 16 gradient modes. That is, the gradient G can be calculated by one of the 16 gradient modes according to the following formula. A syntax element is signaled to indicate which of the 16 gradient modes is used.

[0104] In some embodiments, the downsampled reconstructed luma samples and the gradient of the reconstructed luma samples are used together to derive a linear model. The linear model parameters α and β are derived at the encoder and decoder side by the same method of CCLM (e.g., LMMSE method) based on the reconstructed adjacent chroma samples and the corresponding gradient G of the juxtaposed reconstructed luma samples and the downsampled reconstructed luma samples. The value of the downsampled reconstructed luma sample can be obtained by one of the downsampling filters described above. Then, the chroma samples of the block can be predicted by the linear model from the gradient of the juxtaposed reconstructed luma samples and the value of the downsampled reconstructed luma samples: pred C (i,j)=α·(G L (i,j)+rec′ L (i,j))+β (Formula 39)

[0105] In some embodiments, a flag is signaled to indicate which GLM method in Equations 23-38 is used.

[0106] In the study of video coding technology, it has been noted that there are correlations between different color components. CCLM assumes that there is a linear correlation between chrominance samples and luma samples at corresponding positions in the coding block. However, this linear relationship may not be suitable for all coding blocks. Sometimes, even for local textures, there may be a more complex relationship between luma and chroma, and in this case, it may be more appropriate to use a nonlinear relationship to make assumptions.

[0107] In addition, in the GLM method, the chroma sample is predicted by constructing the relationship between the gradients of the reconstructed luma sample and the chroma sample. However, when the gradient of the reconstructed luma sample of the current block is very close, this method may have difficulty in accurately predicting the chroma sample. Even if the values ​​of the downsampled reconstructed luma samples are introduced in the GLM method corresponding to Formula 39, these values ​​always have the same parameters as the gradient of the reconstructed luma sample, which is not appropriate in some cases.

[0108] In this disclosure, it is proposed to predict the chroma samples of a block from the collocated reconstructed luma samples by using a non-linear model.

[0109] In an exemplary embodiment, a cross-component nonlinear model (CCNLM) is used to predict the chrominance samples of the current block from the downsampled reconstructed luma samples, as shown in Equation 40. pred C (i,j)=a n ·(rec′ L (i,j)) n +a n-1 ·(rec′ L (i,j)) n-1 +…+a1·rec′L (i,j) + a0 (Formula 40) where pred C (i,j) represents the predicted value of the chrominance samples in the current block, and rec′ L (i,j) represents the reconstructed value of the collocated luma samples of the same block downsampled for non-4:4:4 color formats; (i,j) are the coordinates of the samples in the block; the coefficients a0, …, a n are non-linear model parameters that can represent a non-linear model; the value of n can be any positive integer greater than 1, e.g., n = 2.

[0110] According to some embodiments, some terms in Formula 40 can be removed. For example, when the value of n is equal to 2 and the term with power 1 is removed, Formula 40 can be rewritten as: pred C (i,j) = a2 · (rec′ L (i,j)) 2 + a0 (Formula 41)

[0111] According to some embodiments, in order to make the coefficients of each term have a similar order of magnitude, each term can be multiplied by different adjustment factors as: pred C (i,j) = a n · factor n · (rec′ L (i,j)) n + a n-1 · factor n-1 · (rec′ L (i,j)) n-1 + … + a1 · factor1 · rec′ L (i,j) + a0 · factor0 (Formula 42) factor m = 1 / bitDepth m-1 (1 < m <= n). Here, the bit depth refers to the number of bits used to indicate the color of the image.

[0112] According to some embodiments, some coefficients in Formula 40 are restricted to be the same. For example, when the value of n is equal to 2, and the terms with power 1 and power 2 have the same coefficient, then Formula 40 can be rewritten as: pred C (i,j) = a2 · ((rec′ L (i,j)) 2 + rec′ L (i,j)) + a0 (Formula 43)

[0113] According to some embodiments, the above embodiments can be freely combined. For example, the value of n is equal to 2, and each term is multiplied by a different adjustment factor as formula 44, where factor0=bitDepth / 2, factor1=1, factor2=1 / bitDepth. For another example, the value of n is equal to 2, and each term is multiplied by a different adjustment factor, and the term with the power of 1 and the term with the power of 2 have the same coefficient as in Formula 45, where factor0=bitDepth / 2,factor1=1,factor2=1 / bitDepth pred C (i,j)=a2·(rec′ L (i,j)) 2 factor2+a1*rec′ L (i,j)·factor1+a0·factor0 (Formula 44) pred C (i,j)=a2·((rec′ L (i,j)) 2 factor2+rec′ L (i,j)·factor1)+a0·factor0(Formula 45)

[0114] According to some embodiments, the model parameters a0, ..., a n is derived based on the original chroma samples in the current block and the collocated original luma samples of the same block, which are downsampled and signaled to the bitstream in the case of non-4:4:4 color format at the encoder side. At the decoder, the model parameters a0, ..., a are decoded from the bitstream n .

[0115] According to some embodiments, the model parameters a0, ..., a n is derived based on reconstructed adjacent chroma samples and their corresponding reconstructed luma samples, which are downsampled at the encoder and decoder sides for the case of non-4:4:4 color formats to avoid any signaling overhead.

[0116] For example, the neighboring samples used to derive the nonlinear model parameters in the proposed method can be the same as the neighboring samples used to derive the linear model parameters in the aforementioned CCLM method.

[0117] For another example, the neighboring samples used to derive the nonlinear model parameters in the proposed method may be the reconstructed samples in x neighboring rows and columns of the current block, where x may be any positive integer, such as x=3.

[0118] As another example, in the proposed method, a subset of neighboring samples is used to derive nonlinear model parameters, for example, 4 samples.

[0119] According to some embodiments, the model parameters a0, ..., a are derived by the least mean square (LMS) method. n The LMS method derives the parameters a0,…,a by minimizing the mean square error (MSE) between the predicted and reconstructed values ​​of adjacent chrominance samples. n Specifically, the parameters a0,…,a n The initial value of can be used to derive the predicted values ​​of the adjacent chrominance samples. Then, the parameters a0,…,a can be adjusted by minimizing the MSE between the predicted and reconstructed values ​​of the adjacent chrominance samples. n The value of .

[0120] For example, a gradient descent method may be used to minimize the MSE. As is understood in the art, a gradient descent method is an iterative first-order optimization algorithm capable of finding a local minimum / maximum of a given function (e.g., the MSE between predicted and reconstructed values ​​of adjacent chrominance samples).

[0121] For another example, the system of equations for optimizing the partial derivatives of the MSE is written in a matrix multiplication form containing a system of linear equations, and the model parameters can be derived by solving the system of linear equations by Gaussian elimination.

[0122] For another example, the autocorrelation matrix of the reconstructed values ​​of the downsampled adjacent luma samples and the cross-correlation vector between the reconstructed values ​​of the downsampled adjacent luma samples and the reconstructed values ​​of the adjacent chroma samples are calculated. The autocorrelation matrix is ​​decomposed by LU or LDL or Cholesky, and the parameters a0,…,a n Use back substitution to calculate.

[0123] According to some embodiments, similar to the CCLM method described above, when using neighboring samples to derive model parameters a0, ..., a n When , the proposed CCNLM method can support different modes based on the positions of adjacent samples. For example, three CCNLM modes are supported: CCNLM_LT, CCNLM_L, and CCNLM_T. These three modes are used for nonlinear model parameters (a0,…,a n) differ in the positions of the reconstructed neighboring samples derived from the CCNLM_T mode. The neighboring samples reconstructed above refer to the CCNLM_T mode, while the neighboring samples reconstructed to the left refer to the CCNLM_L mode. In the CCNLM_LT mode, both the neighboring samples reconstructed above and to the left refer to the CCNLM_L mode.

[0124] According to some embodiments, similar to the above-mentioned CCLM method, a multi-model CCNLM (MMNLM) method is proposed. Specifically, there may be more than one nonlinear model between the luminance and chrominance in a block. For example, adjacent samples are used to derive model parameters. First, the reconstructed adjacent samples are classified into two categories using a threshold (e.g., the average value of the values ​​of adjacent samples of luminance reconstruction). Then, using the above-mentioned method, each class is regarded as an independent training set to derive a nonlinear model. Subsequently, the reconstructed luminance samples of the current block are also classified based on the same rules. Finally, the chrominance samples are predicted by the luminance samples reconstructed differently in different categories.

[0125] In some embodiments, when using neighboring samples to derive model parameters (a0, ..., a n When , MMNLM can support different modes based on the positions of adjacent samples. For example, three MMNLM modes are supported: MMNLM_LT, MMNLM_L, and MMNLM_T. These three modes are used for nonlinear model parameters ((a0,…,a n ) differ in the positions of the reconstructed neighboring samples derived from the MMNLM_T mode. The neighboring samples reconstructed above refer to the MMNLM_T mode, and the neighboring samples reconstructed to the left refer to the MMNLM_L mode. In the MMNLM_LT mode, both the above and left reconstructed neighboring samples refer to the MMNLM_LT mode.

[0126] According to some embodiments, six CCLM modes are supported: CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L, and MMLM_T. In the disclosed embodiments, the various modes of the proposed CCNLM can be used to replace some or all of the CCLM modes.

[0127] According to some embodiments, the proposed CCNLM is used to replace all the above CCLM modes. That is, the six CCNLM modes CCNLM_LT, CCNLM_L, CCNLM_T, MMNLM_LT, MMNLM_L and MMNLM_T are used to replace the six CCLM modes respectively. And there is no additional syntax signal notification.

[0128] According to some embodiments, the proposed CCNLM is used to replace some of the above-mentioned CCLM modes.

[0129] For example, the proposed CCNLM is used to replace the CCLM_LT mode. Thus, CCNLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L and MMLM_T are supported. For the CCNLM_LT mode, a nonlinear model between luminance and chrominance is derived, and for the other modes, a linear model is derived.

[0130] For another example, the proposed CCNLM is used to replace the CCLM_LT mode and the MMLM_LT mode. Thus, CCNLM_LT, CCLM_L, CCLM_T, MMNLM_LT, MMLM_L and MMLM_T are supported. For the CCNLM_LT mode and the MMLM_LT mode, a nonlinear model between luminance and chrominance is derived, and for the other modes, a linear model is derived.

[0131] For another example, the proposed CCNLM is used to replace CCLM_LT mode, CCLM_L mode and CCLM_T mode. Thus, CCNLM_LT, CCNLM_L, CCNLM_T, MMLM_LT, MMLM_L and MMLM_T are supported. For CCNLM_LT mode, CCNLM_L mode and CCNLM_T mode, a nonlinear model between luminance and chrominance is derived, and a linear model is derived for other modes.

[0132] According to some embodiments, an explicit signaling method is used to decide whether to use the proposed CCNLM method, i.e., whether to use a linear model or a nonlinear model. For example, after the CCLM mode is signaled, a flag is signaled to indicate whether to use the proposed CCNLM method. In other words, some syntax elements are first signaled to indicate the positions of neighboring samples used in model parameter derivation (LT, L or T) and whether a single model or a dual model is used in the current block. Then a flag is signaled to indicate whether to use a linear model or a nonlinear model.

[0133] According to some embodiments, the explicit signaling method supports only some of the six CCNLM modes.

[0134] According to some embodiments, two flags are signaled to indicate whether to use the proposed CCNLM method for Cb and Cr, respectively.

[0135] According to some embodiments, an implicit method is used to decide whether to use the proposed CCNLM method, ie, to use a linear model or a nonlinear model.

[0136] For example, based on the reconstructed adjacent samples, a linear model and a proposed nonlinear model are derived. Then, the two models are used to predict adjacent chrominance samples from the reconstructed adjacent luminance samples, respectively. The prediction values ​​obtained by the two types of models are used to calculate the sum of absolute differences (SAD) or the sum of absolute transform differences (SATD) between the reconstructed values ​​of adjacent chrominance samples, respectively. Finally, the model with the smaller SAD or SATD is used to predict the chrominance samples of the current block.

[0137] For another example, the gradients of neighboring luma samples and neighboring chroma samples are used to decide whether to use a linear model or a non-linear model.

[0138] For another example, the size of the current chrominance block is used to determine whether to use a linear model or a nonlinear model. For example, if the area of ​​the current chrominance block is greater than a threshold (eg, 256), the nonlinear model is used; otherwise, the linear model is used.

[0139] In the disclosed embodiments, the CCNLM method may be combined with the GLM method. According to some embodiments, the proposed CCNLM method is combined with the GLM method, where, as shown in Equation 46, the chrominance samples of the current block are predicted based on the gradient G of the collocated reconstructed luminance samples. pred C (i, j) = a n ·(G L (i, j)) n +a n-1 ·(G L (i, j)) n-1 +…+a1·G L (i, j) + a0 (Formula 46) where pred C (i, j) represents the predicted value of the chrominance sample in the current block, G L (i, j) represents the corresponding gradient G of the collocated reconstructed luminance samples of the same block; (i, j) is the coordinate of the sample in the block; coefficients a0, ..., a n is a nonlinear model parameter that can represent a nonlinear model; the value of n can be any positive integer greater than 1, for example, n=2.

[0140] According to some embodiments, some terms in Formula 46 may be removed. For example, when the value of n is equal to 2 and the terms with a power of 1 are deleted, Formula 46 may be rewritten as: pred C (i, j) = a2·(G L (i, j)) 2 +a0 (Formula 47)

[0141] According to some embodiments, in order to make the coefficients of each term in Equation 46 have similar orders of magnitude, each term may be multiplied by a different adjustment factor as follows: pred C (i, j) = a n ·factor n ·(G L (i, j)) n +a n-1 ·factor n-1 ·(G L (i, j)) n-1 +…+a1·factor1·G L (i, j) + a0 factor0 (Formula 48) For example, factor0 = bitDepth / 2, factor1 = 1, factor m =1 / bitD epth m-1 (1<m<=n).

[0142] According to some embodiments, in some terms in Equation 46, the gradient G L (i, j) can be reconstructed by downsampling the collocated luma samples rec′ L (i, j) is replaced. rec′ L (i, j) can be obtained by one of the downsampling filters mentioned above.

[0143] For example, only the highest-order terms in G(i, j) are rec′ L (i, j) instead: pred C (i, j) = a n ·(rec′ L (i, j)) n +a n-1 ·(G L (i, j)) n-1 +…+a1·G L (i, j)+a0 (Formula 49)

[0144] For another example, only the lowest-order terms in G(i, j) are rec′ L (i, j) instead: pred C (i, j) = a n ·(G L (i, j)) n +a n-1 ·(G L (i, j)) n-1 +…+a1·rec′ L(i, j)+a0 (Formula 50)

[0145] For another example, only the lowest-order term G(i, j) is not included in rec′ L (i, j) is replaced, and for other items, G(i, j) is replaced by rec′ L (i, j) replacement: pred C (i, j) = a n ·(rec′ L (i, j)) n +a n-1 ·(rec′ L (i, j)) n-1 +…+a1·G L (i, j) + a0 (Formula 51)

[0146] According to some embodiments, in some terms in Equation 46, the gradient G L (i, j) can be obtained by reconstructing the value rec of the collocated luminance sample L (2i, 2j) instead.

[0147] According to some embodiments, the above four embodiments can be freely combined.

[0148] For example, if n is equal to 2, use the adjustment factors, factor0 = bitDepth / 2, factor1 = 1, factor2 = 1 / bitDepth: pred C (i, j) = a2·(G L (i, j)) 2 / bitDepth+a1·G L (i,j)+a0·bitDepth / 2 (Formula 52)

[0149] As another example, in the above example, the highest order term G(i, j) is represented by rec′ L (i, j) instead: pred C (i, j) = a2·(rec′ L (i, j)) 2 / bitDepth+a1·G L (i,j)+a0·bitDepth / 2 (Formula 53)

[0150] The samples used to derive the non-linear model may be the original chroma samples in the current block and the collocated original luma samples of the same block, which are downsampled at the encoder side for the case of non-4:4:4 color formats and signaled to the bitstream. Alternatively, the samples used to derive the non-linear model may be the reconstructed adjacent chroma samples and their corresponding luma samples, which are downsampled at the encoder and decoder side for the case of non-4:4:4 color formats to avoid any signaling overhead.

[0151] The nonlinear model derivation method may be one of the least mean square (LMS) methods mentioned above.

[0152] According to some embodiments, the proposed CCNLM method is always combined with the GLM method. That is, when the GLM method is used for chrominance blocks, a nonlinear model is used to predict the chrominance samples.

[0153] According to some embodiments, an explicit signaling method is used to decide whether to combine the proposed CCNLM method with the GLM method, i.e., whether to use a linear model or a nonlinear model for the GLM. For example, after signaling a GLM flag, if the GLM mode is enabled, a flag is further signaled to indicate whether a nonlinear model is used.

[0154] According to some embodiments, an implicit method is used to decide whether to combine the proposed CCNLM method with the GLM method, that is, whether to use a linear model or a nonlinear model for the GLM. For example, the implicit method can be the above-mentioned implicit method for deciding whether to use the proposed CCNLM method, that is, whether to use a linear model or a nonlinear model.

[0155] According to some embodiments, the proposed CCNLM method can be combined with only some GLM modes. For example, the GLM method can be used for CCLM_LT mode, CCLM_L mode and CCLM_T mode, and CCNLM can only be combined with GLM when CCLM_LT mode is selected. For another example, the GLM method can be used for all six CCLM modes, and CCNLM can only be combined with GLM when CCLM_LT mode is selected.

[0156] According to some embodiments, the above-described embodiments can be freely combined.

[0157] For example, GLM is only available in CCLM_LT mode, and when GLM is enabled, the proposed CCNLM method is always combined with the GLM method as Equation 52. The LDL decomposition method is used to derive nonlinear parameters from the gradients of reconstructed neighboring chrominance samples and reconstructed neighboring luma samples.

[0158] The present disclosure also provides a new GLM method, in which the chrominance samples of the current block are predicted based on the gradient G of the juxtaposed reconstructed luminance samples and the downsampled reconstructed luminance samples. The prediction is based on formula 54: pred C (i, j) = a2·G L (i, j)+a1·rec′ L (i, j)+a0 (Formula 54) where pred C (i, j) represents the predicted value of the chrominance sample in the current block, G L (i, j) denotes the corresponding gradient G of the collocated reconstructed luminance samples of the same block, and rec L ′(i, j) represents the reconstructed value of the collocated luminance samples of the same block downsampled by one of the above downsampling filters for non-4:4:4 color formats; (i, j) are the coordinates of the sample in the block; coefficients a2, a1 and a0 are model parameters that can represent pred C (i, j), G L (i, j) and rec′ L The relationship between (i, j).

[0159] In some embodiments, another GLM method is proposed, as shown in Formula 55: pred C (i, j) = a2·(G L (i, j)+rec′ L (i,j))+a1·rec′ L (i, j)+a0 (Formula 55)

[0160] The coefficients in the GLM method corresponding to equations 54 and 55, respectively, can be derived based on the values ​​of the reconstructed adjacent chroma samples, the corresponding gradient G of the juxtaposed reconstructed adjacent luminance samples, and the values ​​of the corresponding reconstructed adjacent luminance samples. The model parameters are derived by performing the derivation on both the encoder and decoder sides using one of the above methods of least mean squares (LMS). For example, the coefficients can be derived using Gaussian elimination or LDL decomposition.

[0161] In some embodiments, only one of the above GLM methods is supported.

[0162] For example, the GLM method corresponding to Formula 54 is supported. For the chrominance block, a flag is signaled in the bitstream to indicate whether GLM is used. If this flag indicates that GLM is used, the GLM method corresponding to Formula 54 is used to predict the chrominance samples in the current block.

[0163] In some embodiments, two GLM methods are supported, and a flag is signaled to indicate which of the two GLM methods is used.

[0164] In one example, the GLM methods corresponding to Formula 22 and Formula 54 are supported. For chroma blocks, a flag is signaled in the bitstream to indicate whether GLM is used. If this flag indicates that GLM is used, another flag is signaled to indicate which of the two GLM methods corresponding to Formula 22 and Formula 54, respectively, is used to predict the chroma samples in the current block.

[0165] In another example, GLM methods corresponding to Formula 39 and Formula 54, respectively, are supported.

[0166] In another example, GLM methods corresponding to Formula 54 and Formula 55, respectively, are supported.

[0167] In some embodiments, three GLM methods are supported, and a syntax element is signaled to indicate which GLM method is used.

[0168] For example, GLM methods corresponding to Formula 22, Formula 39, and Formula 54 are supported. For chrominance blocks, a flag is signaled in the bitstream to indicate whether GLM is used. If this flag indicates that GLM is used, a syntax element is signaled to indicate which of the three GLM methods corresponding to Formula 22, Formula 39, and Formula 54 are used to predict the chrominance samples in the current block.

[0169] In some embodiments, four GLM methods are supported, and a syntax element is signaled to indicate which GLM method is used.

[0170] For example, GLM methods corresponding to Formula 22, Formula 39, Formula 54, and Formula 55 are supported. For chroma blocks, a flag is signaled in the bitstream to indicate whether GLM is used. If this flag indicates that GLM is used, a syntax element is signaled to indicate which of the three GLM methods corresponding to Formula 22, Formula 39, Formula 54, and Formula 55 are used to predict the chroma samples in the current block.

[0171] In some embodiments, for the two GLM methods corresponding to Formula 54 and Formula 55, each term may be multiplied by different adjustment factors according to Formula 56 and Formula 57, respectively. For example, factor2 is equal to 1, factor1 is equal to 1, and factor0 is equal to 512. pred C (i,j)=a2·factor2·G L (i,j)+a1·factor1·rec L′(i,j)+a0·factor0(Formula 56) pred C (i, j) = a2 · factor2 · (G L (i, j)+rec′ L (i, j))+a1·factor1·rec′ L (i, j) + a0 factor0 (Formula 57)

[0172] In some embodiments, the value rec′ of the reconstructed luma sample downsampled in the GLM method corresponding to Equation 54 is L (i, j), the nth power of the brightness sample value can be reconstructed by downsampling (rec′ L (i, j)) n Replace, as shown in Formula 58: pred C (i, j) = a2·G L (i,j)+a1·(rec′ L (i, j)) n +a0 (Formula 58)

[0173] In some embodiments, the value rec′ of the reconstructed luminance sample downsampled in the GLM method corresponding to Equation 55 L (i, j), the value of the brightness sample can be reconstructed by downsampling to the power of n (rec′ L (i, j)) n to replace.

[0174] In one example, only the rec′ in the first term L (i, j) is (rec′ L (i, j)) n Replace as follows: pred C (i, j) = a2·(G L (i,j)+(rec′ L (i, j)) n )+a1·rec′ L (i, j)+a0 (Formula 59)

[0175] In another example, only rec′ in the second term L (i, j) is (rec′ L (i, j)) n Replace as follows: pred C (i, j) = a2·(G L (i, j)+rec′ L(i,j))+a1·(rec′ L (i, j)) n +a0 (Eq.60)

[0176] In another example, all rec′ in the first and second terms L (i, j) are all (rec′ L (i, j)) n Replace as follows: pred C (i, j) = a2·(G L (i,j)+(rec′ L (i, j)) n )+a1·(rec′ L (i, j)) n +a0 (Formula 61)

[0177] In some embodiments, the values ​​rec′ of the downsampled reconstructed luminance samples in the two GLM methods corresponding to Equation 54 and Equation 55, respectively L (i, j) can be replaced by the value of the reconstructed luminance sample. For example, for a 4:2:0 color format video sequence, rec L (2i, 2j) to replace rec′ L (i, j), where (i, j) are the coordinates of the sample in the current chroma block.

[0178] In the above-described GLM method, the gradient G may be calculated by one of four Sobel-based gradient modes as shown in equations 14-17 or one of 16 gradient modes as shown in equations 23-38.

[0179] In some embodiments, another novel GLM method is proposed, in which the chrominance samples of the current block are predicted from multiple gradients of the collocated reconstructed luma samples, as shown in Formula 62: pred C (i, j) = a n ·G Lk (i, j)+…+a1·G L1 (i, j)+a0 (Formula 62) where pred C (i, j) represents the predicted value of the chrominance sample in the current block; G L1 (i, j) to G Ln (i, j) represents k different gradients of the juxtaposed reconstructed luminance samples of the same block; (i, j) is the coordinate of the sample in the block; coefficients a0 to a nare model parameters, which can be derived at the encoder and decoder side based on the values ​​of reconstructed adjacent chroma samples and k different gradients of concatenated reconstructed adjacent luma samples by one of the above methods for deriving model parameters using least mean squares (LMS). For example, the coefficients can be derived using an LDL decomposition method.

[0180] In one example, two gradients are used simultaneously to derive the prediction value of the chrominance sample, as shown in Equation 63. The two gradients G can be calculated according to the horizontal gradient mode and the vertical gradient mode as shown in Equation 14 and Equation 15. L1 (i, j) and G L2 (i, j). pred C (i, j) = a2·GL2(i, j) + a1·GL1(i, j) + a0 (Formula 63)

[0181] In another example, four gradients are used simultaneously to derive the prediction value of the chroma sample, as shown in Equation 64. The four gradients G L1 (i, j) to G L4 (i, j) can be calculated according to the four gradient modes as shown in equations 14-17. pred C (i, j) = a4·G L4 (i,j)+a3·G L3 (i, j)+a2·G L2 (i,j)+a1·G L1 (i, j)+a0(public Formula 64)

[0182] In another example, the number of gradients and gradient modes used together to derive prediction values ​​for chroma samples may be determined by a signaled syntax element.

[0183] In some embodiments, another new GLM method is proposed, in which the chrominance samples of the current block are predicted based on multiple gradients of the collocated reconstructed luma samples and the values ​​of the collocated downsampled reconstructed luma samples, as shown in Formula 65: pred C (i, j) = a n ·G Lk (i, j)+…+a2·G L1 (i, j)+a1·rec′ L (i, j)+a0 (Formula 65) where pred C (i, j) represents the predicted value of the chrominance sample in the current block; G L1 (i, j) to G Lk(i, j) represents k different gradients of the collocated reconstructed luminance samples of the same block; rec′ L (i, j) represents the reconstructed value of the collocated luminance samples of the same block downsampled by one of the above downsampling filters for non-4:4:4 color formats; (i, j) is the coordinate of the sample in the block; coefficients a0 to a n are model parameters, which can be derived at the encoder and decoder side based on the values ​​of reconstructed adjacent chroma samples and the gradients of concatenated adjacent luminance samples by one of the above methods for deriving model parameters using least mean squares (LMS). For example, the coefficients can be derived using an LDL decomposition method.

[0184] In one example, the two gradients of the collocated reconstructed luma samples and the values ​​of the collocated downsampled reconstructed luma samples are used together to derive the predicted values ​​of the chroma samples, as shown in Equation 66. The two gradients G can be calculated according to the horizontal gradient pattern and the vertical gradient pattern as shown in Equation 14 and Equation 15. L1 (i, j) and G L2 (i, j). pred C (i, j) = a3·G L2 (i, j)+a2·G L1 (i, j)+a1·rec′ L (i, j) + a0 (Formula 66)

[0185] In another example, four gradients of the collocated reconstructed luma samples are used together with the values ​​of the collocated downsampled reconstructed luma samples to derive the predicted values ​​of the chroma samples, as shown in Equation 67. The four gradients G L1 (i, j) to G L4 (i, j) can be calculated according to the gradient mode as shown in formulas 14-17. pred C (i, j) = a5·G L4 (i,j)+a4·G L3 (i,j)+a3·G L2 (i, j)+a2·G L1 (i, j)+a1·rec′ L (i,j)+a0 (Formula 67)

[0186] In another example, the number of gradients and gradient modes used together to derive prediction values ​​for chroma samples may be determined by a signaled syntax element.

[0187] In some embodiments, nonlinear terms are added to the values ​​of the concatenated downsampled reconstructed luma samples to calculate chroma samples. For example, as shown in Formula 68, nonlinear terms are added to the GLM method corresponding to Formula 62 to calculate chroma samples. In another example, as represented by Formula 69, nonlinear terms are added to the GLM method corresponding to Formula 65 to calculate chroma samples. The value of x can be any positive integer greater than 1. In one example, the value of x is equal to 2. pred C (i, j) = a n ·G Lk (i, j)+…+a2·G L1 (i,j)+a1·(rec′ L (i, j)) x +a0 (Formula 68) pred C (i, j) = a n ·G Lk (i, j)+…+a3·G L1 (i, j)+a2·rec′ L (i,j)+a1·(rec′ L (i, j)) x +a0 (Formula 69)

[0188] In some embodiments, two GLM methods corresponding to Formula 62 and Formula 65 are supported, and a block-level flag is signaled in the bitstream to indicate which GLM method is used to predict the current chrominance block. That is, a block-level flag is signaled to indicate whether the value of the concatenated downsampled reconstructed luma sample is used to predict the chroma sample.

[0189] In some embodiments, the number of supported gradients is modified.

[0190] In one example, only one gradient is supported. For example, only the horizontal gradient mode corresponding to Formula 14 is supported.

[0191] In another example, a slice level syntax element is signaled to indicate the number of gradient modes supported in the current slice.

[0192] In some embodiments, the coding method of the gradient mode is modified. A truncated unary code is used to encode which gradient mode is used. For example, four gradient modes as shown in formulas 14 to 17 are supported, and a truncated unary code is used to encode the syntax element indicating which of the four gradient modes is used, as shown in Table 1. Table 1: Exemplary truncated unary codewords for different gradient modes Gradient Mode Truncating unary codewords The horizontal gradient pattern corresponding to Equation 14 0 The vertical gradient pattern corresponding to Equation 15 10 The diagonal gradient pattern corresponding to Equation 16 110 The diagonal gradient pattern corresponding to Equation 17 111

[0193] In some embodiments, the Cb and Cr components may share the same GLM enable flag.

[0194] In one example, one flag is signaled to indicate whether GLM is used for Cb and Cr components. If the flag indicates that GLM is used, two syntax elements are signaled to indicate which gradient mode is used for Cb and Cr, respectively.

[0195] In another example, a flag is signaled to indicate whether GLM is used for the two components Cb and Cr. If the flag indicates that GLM is used, a syntax element is signaled to indicate which gradient mode is used for the two components Cb and Cr.

[0196] In some embodiments, multiple rows are used to derive GLM model parameters. Specifically, n chromaticity adjacent rows can be used to derive GLM model parameters, where the value of n can be equal to any positive integer greater than 1. For example, the value of n is equal to 6.

[0197] In one example, the GLM supports CCLM_LT, CCLM_L, and CCLM_T modes, and uses 6 chroma neighbor rows to derive GLM parameters. Figure 7 is a schematic diagram illustrating a method for deriving parameters of a gradient model using samples on adjacent rows according to some embodiments of the present disclosure. Figure 7 In FIG. 1 , samples on adjacent rows of a chrominance block are shown as circles. Specifically, Figure 7 As shown, for a W×H chrominance block: When GLM is used in CCLM_LT mode, the upper 6 rows of samples and the left 6 rows of samples are used to derive GLM parameters ( Figure 7 A); When GLM is used in CCLM_L mode, the left 6 rows of samples and the lower left 6 rows of samples are used to derive GLM parameters ( Figure 7 B); and when GLM is used in CCLM_T mode, the upper 6 rows of samples and the upper right 6 rows of samples are used to derive GLM parameters ( Figure 7 C).

[0198] In another example, the GLM supports CCLM_LT, CCLM_L, and CCLM_T modes, and uses 6 chroma neighbor rows to derive GLM parameters. Figure 8 is a schematic diagram illustrating a method for deriving parameters of a gradient model using samples on adjacent rows according to some embodiments of the present disclosure. Figure 8 The samples on adjacent rows of the chrominance block are shown as circles. Specifically, Figure 8 As shown, for a W×H chrominance block: When GLM is used in CCLM_LT mode, 6 rows of samples on the upper side, 6 rows of samples on the upper right side, 6 rows of samples on the left side, and 6 rows of samples on the lower left side are used to derive GLM parameters ( Figure 8 A); When GLM is used in CCLM_L mode, the left 6 rows of samples and the lower left 6 rows of samples are used to derive GLM parameters ( Figure 8 B); and when GLM is used in CCLM_T mode, the upper 6 rows of samples and the upper right 6 rows of samples are used to derive GLM parameters ( Figure 8 C).

[0199] In some embodiments, when the GLM is used in CCLM_LT mode, the upper left neighboring samples are also included in the samples used to derive the GLM model parameters.

[0200] In some embodiments, the above-mentioned extension of the number of rows used to derive GLM model parameters is only applied to some GLM modes. For example, two GLM methods corresponding to Formula 22 and Formula 54, respectively, can be supported, and a flag is signaled to indicate which GLM method is used for the current block. Then, when the GLM method corresponding to Formula 22 is used, only one adjacent row is used to derive the GLM model parameters, and when the GLM method corresponding to Formula 54 is used, six adjacent rows are used to derive the GLM model parameters.

[0201] In some embodiments, a variation of the multi-model GLM method is proposed. In the multi-model GLM method, more than one model can be used to predict the chrominance samples in the block. The values ​​rec′ of the reconstructed luminance samples can be based on the value of the gradient G, the sign of the gradient G, the absolute value of the gradient G, or the downsampled L (i, j) to complete the classification. Each class is then treated as an independent training set to derive the model parameters that indicate the relationship between the gradient G and the value of the chrominance sample, or the value rec′ of the downsampled reconstructed luminance sample L The relationship between (i,j) and the value of the chrominance samples. Finally, the chrominance samples are predicted differently in different classes.

[0202] In one example, the gradients of the reconstructed adjacent samples are first classified into two categories by the sign of the gradient. That is, positive gradients are in the first category and negative gradients are in the second category. Then, using the above-mentioned LMMSE method, each class is regarded as an independent training set to derive a linear model. Subsequently, the gradients of the reconstructed luma samples of the current block are also classified based on the same rules. Finally, according to the method in Formula 22, the chroma samples are predicted by the gradients of the luma samples reconstructed differently in different categories.

[0203] In another example, the gradients of the reconstructed adjacent samples are first classified into two categories using a threshold, which is the average of the absolute values ​​of the gradients of the adjacent samples of the luminance reconstruction. That is, the gradients whose absolute values ​​are greater than or equal to the threshold are classified into the first category, and the gradients whose absolute values ​​are less than the threshold are classified into the second category. Then, using the above-mentioned LMMSE method, each class is regarded as an independent training set to derive a linear model. Subsequently, the gradients of the reconstructed luminance samples of the current block are also classified based on the same rules. Finally, for the first category, the chrominance samples are predicted from the gradients according to Formula 22. For the second category, the chrominance samples are predicted from the downsampled reconstructed luminance samples according to Formula 1.

[0204] In some embodiments, the above-described embodiments related to the disclosed GLM methods can be freely combined.

[0205] In one example, only one GLM method corresponding to equation 54 is supported. For chroma blocks, one flag is signaled to indicate whether GLM is used for the two components Cb and Cr. If the flag indicates that GLM is used for the two components Cb and Cr, one syntax element is signaled by truncating the unary code to indicate which of the four gradient modes corresponding to equations 14 to 17 is used for the two components Cb and Cr. Six adjacent rows are used to derive the model parameters.

[0206] In another example, two GLM methods corresponding to Formula 22 and Formula 54, respectively, are supported. For the chroma block, a flag is signaled to indicate whether GLM is used for the two components Cb and Cr. If the flag indicates that GLM is used, another flag is signaled to indicate which of the two GLM methods is used, and a syntax element is signaled by truncating a unary code to indicate which of the four gradient modes corresponding to Formulas 14 to 17 is used for the two components Cb and Cr. When the GLM method corresponding to Formula 22 is used, only one adjacent row is used to derive the model parameters, and when the GLM method corresponding to Formula 54 is used, six adjacent rows are used to derive the model parameters.

[0207] In some embodiments, when predicting chroma samples, an offset may be subtracted from the term of the downsampled reconstructed luma sample. The offset may be different for each block. For example, the offset may be equal to the first downsampled reconstructed luma sample in the current block or may be related to the bit depth. For example, the method in equation 54 may be modified as follows: pred C (i,j)=a2·G L (i,j)+a1·(rec′ L (i,j)-offset)+a0 (Formula 70)

[0208] The above-described embodiments may be performed as a part of video data processing such as an encoding process or a decoding process. Fig. 9 900 is a flow chart of a method 900 for predicting chroma samples using a cross-component nonlinear model according to some embodiments of the present disclosure. When predicting chroma samples, the method 900 may be performed by an encoder (e.g., by Figure 2A Process 200A or Figure 2B 200B) or by a decoder (e.g., Figure 3A Process 300A or Figure 3B 300B) or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 900 may be performed by a processor 402 of the computer system. In some embodiments, the method 900 may be implemented by a computer program product contained in a computer-readable medium, including a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by the device 400. Fig. 9 As shown, method 900 includes the following steps 910 to 920.

[0209] At step 910, a processor (e.g., Figure 4 The processor 402 of FIG. 40 trains a cross-component nonlinear model (CCNLM). The CCNLM defines a nonlinear relationship between the predicted chrominance samples and the juxtaposed reconstructed luminance samples. For example, the CCNLM may be a nonlinear model as described in Equation 40. The model parameters in Equation 40, i.e., a n ,a n-1 ,...a1 and a0 can be any value. For example, parameter a n ,a n-1 , ...at least one of a1 and a0 may be equal to zero. In some embodiments, the model parameter a n ,a n-1 ,...a1 and a0 are determined based on the bit depth of the image so that all model parameters have similar orders of magnitude. In the case of non-4:4:4 color formats, such as when the color format is 4:2:2 or 4:2:0, rec′ in Formula 40 L (i,j) is the downsampled value of a number of reconstructed luma samples associated with coordinate (i,j).

[0210] During model training, the processor uses the training dataset to derive the model parameters a n ,a n-1,...a1 and a0. In some embodiments, the training may be performed by an encoder, and the training data set includes the original chrominance samples of the encoded blocks and the original collocated luminance samples of the encoded blocks. After the training is completed, the encoder may transfer the derived model parameters a n ,a n-1 ,...a1 and a0 are encoded in the bitstream to be sent to the decoder. In some embodiments, the training data set includes reconstructed adjacent chrominance samples of the coded blocks and reconstructed adjacent luminance samples of the coded blocks. Such training based on reconstructed adjacent luminance or chrominance samples can be performed at the encoder side or the decoder side, and therefore does not require the derived model parameters a n ,a n-1 , ... explicit signaling of a1 and a0. In some embodiments, training can be performed by minimizing the mean square error (MSE) between the reconstructed adjacent luma samples and the predicted adjacent chroma samples of the chroma block. In some embodiments, training can be performed using CCLM_LT mode, CCLM_L mode, or CCLM_T mode. CCLM_LT mode uses the reconstructed luma / chroma samples adjacent to the top and the reconstructed luma / chroma samples adjacent to the left as training data (e.g., Figure 5 A, Figure 6 A). CCLM_L mode uses the left adjacent reconstructed luma / chroma samples as training data (e.g. Figure 5 B, Figure 6 B). CCLM_T mode uses the upper adjacent reconstructed luma / chroma samples as training data (e.g. Figure 5 C, Figure 6 C).

[0211] Return to reference Fig. 9 , in step 920, the processor predicts the chrominance samples based on the CCNLM. In some embodiments, more than one nonlinear model may be used to predict the chrominance samples in the coding block. For example, the processor may perform a comparison of the reconstructed luminance samples in the coding block with one or more thresholds, and classify the reconstructed luminance samples in the coding block into multiple categories based on the comparison. The threshold may be a statistical value of the reconstructed luminance samples in the coding block, such as the mean or median of the reconstructed luminance samples. The processor may then apply multiple different nonlinear models to multiple classes, respectively. In some embodiments, such multi-model training may be performed using the MMNLM_LT mode, the MMNLM_L mode, or the MMNLM_T mode. The MMNLM_LT mode uses the reconstructed luminance / chrominance samples adjacent to the top and the reconstructed luminance / chrominance samples adjacent to the left as training data (e.g., Figure 5 A, Figure 6 A). The MMNLM_L mode uses the left adjacent reconstructed luma / chroma samples as training data (e.g., Figure 5B, Figure 6 B). The MMNLM_T mode uses the upper adjacent reconstructed luma / chroma samples as training data (e.g., Figure 5 C, Figure 6 C). In some embodiments, the nonlinear model used in method 900 may also include a term represented by a gradient of juxtaposed reconstructed luminance samples. For example, a gradient-based nonlinear model may be defined as one of equations 46-53. The details of combining the gradient with the nonlinear model are described above in conjunction with these equations.

[0212] Fig.10 1 is a flow chart of a method 1000 for predicting chroma samples using a cross-component nonlinear model according to some embodiments of the present disclosure. When predicting chroma samples, the method 1000 may be performed by an encoder (e.g., by Figure 2A Process 200A or Figure 2B 200B) or by a decoder (e.g., by Figure 3A Process 300A or Figure 3B 300B) or by a device (e.g., Figure 4 For example, one or more processors (e.g., Figure 4 The method 1000 may be performed by a processor 402 of the computer system. In some embodiments, the method 1000 may be implemented by a computer program product contained in a computer-readable medium, the computer program product comprising a computer (e.g., Figure 4 Computer executable instructions such as program codes executed by the device 400). Fig.10 As shown, method 1000 includes the following steps 1010 to 1020.

[0213] At step 1010, a processor (e.g., Figure 4 The processor 402 of the embodiment trains a gradient model. In some embodiments, the gradient model may be a linear model that specifies a linear relationship between a predicted chroma sample and a collocated reconstructed luma sample (e.g., Formula 22, Formula 54). In some embodiments, the gradient model may be a nonlinear model that specifies a nonlinear relationship between a predicted chroma sample and a collocated reconstructed luma sample (e.g., Formula 46). In some embodiments, the gradient model may be a hybrid model based on both the gradient of the collocated reconstructed luma sample and the value of the collocated reconstructed luma sample (e.g., Formula 39, Formula 49). In some embodiments, the gradient model may be a model that defines a relationship between more than one gradient of a predicted chroma sample and a collocated reconstructed luma sample (e.g., Formula 62). Each of the one or more gradients is determined by applying different gradient patterns to multiple reconstructed luma samples associated with the collocated reconstructed luma samples. For example, the gradient pattern may be a pattern defined by Formulas 23-38).

[0214] During model training, the processor uses the training data set to derive model parameters. In some embodiments, the training data set includes the current coding block (e.g., Figure 7 A) is one or more rows of upper adjacent samples. The training data set includes one or more rows of upper right adjacent samples of the coding block (for example, Figure 7 C. Figure 8 A. Figure 8 C). In some embodiments, the training data set includes one or more rows of left-side adjacent samples of the coding block (e.g., Figure 7 A). In some embodiments, the training data set includes one or more rows of lower left adjacent samples of the coding block (e.g., Figure 7 B, Figure 8 A, Figure 8 B) In some embodiments, the training data set includes one or more top left neighboring samples of the coding block.

[0215] Return to reference Fig.10 , in step 1020, the processor predicts chrominance samples based on the gradient model. In some embodiments, more than one gradient model may be used to predict chrominance samples in a coding block. For example, the processor may perform a comparison of the reconstructed luminance samples in the coding block with one or more thresholds, and classify the reconstructed luminance samples in the coding block into a first plurality of categories based on the comparison. The threshold may include an absolute value of the gradient of the reconstructed luminance sample, a sign of the gradient of the reconstructed luminance sample, or a downsampled value of the reconstructed luminance sample. The processor may then apply a plurality of different gradient models to the first plurality of categories, respectively. In some embodiments, such multi-model training may be performed by classifying the reconstructed adjacent luminance and chrominance samples into a second plurality of categories using a threshold. Multiple gradient models may then be trained using the second plurality of categories, respectively.

[0216] In some embodiments, a non-transitory computer-readable storage medium is also provided. In some embodiments, the medium may store all or part of a video bitstream encoded or decoded according to the disclosed cross-component prediction method. In addition, the video bitstream may include a flag or syntax element that signals the disclosed cross-component model or gradient model. For example, the video bitstream may include a flag indicating whether the CCNLM or gradient model is enabled. As another example, the video bitstream may include a syntax element that signals whether more than one nonlinear model or gradient model is used to predict chrominance samples in a coding block from juxtaposed reconstructed luminance samples.

[0217] In some embodiments, a non-transitory computer-readable storage medium may store instructions executable by a device (such as the disclosed encoder and decoder) for performing the above method. Common forms of non-transitory media include, for example, floppy disks, hard disks, solid-state drives, tapes or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with hole patterns, RAM, PROM and EPROM, FLASH-EPROM or any other flash memory, NVRAM, caches, registers, any other storage chips or cassettes and networked versions thereof. The device may include one or more processors (CPUs), input / output interfaces, network interfaces, and / or memories.

[0218] In some embodiments, a computer program product is provided, wherein the program product comprises: computer program instructions, wherein the computer program instructions enable a computer to execute the steps of the method described in any embodiment of the present disclosure.

[0219] In some embodiments, a computer program is provided, which enables a computer to execute the steps of the method described in any embodiment of the present disclosure.

[0220] The embodiments may be further described using the following terms:

[0221] 1. A video processing method, comprising: determining a first value associated with a chroma sample by applying a first gradient pattern to a plurality of reconstructed values ​​of a first plurality of luma samples; determining a second value associated with the chroma sample by applying a downsampling filter to a plurality of reconstructed values ​​of a second plurality of luma samples; and The chroma samples are predicted based on the first value and the second value.

[0222] 2. The method of clause 0, wherein the first plurality of luma samples and the second plurality of luma samples each comprise at least one of: the chroma sample's collocated luma sample, or The adjacent luma samples of the collocated luma sample.

[0223] 3. The method of clause 0, wherein applying the first gradient pattern to the plurality of reconstructed values ​​of the first plurality of luma samples comprises: Encode or decode syntax elements in a bitstream; and A first gradient mode is selected from a plurality of gradient modes based on the syntax element.

[0224] 4. A method as described in clause 0, wherein predicting the chroma samples based on the first value and the second value comprises: The chroma samples are predicted based on a first parameter associated with the first value, a second parameter associated with the second value, and a third parameter.

[0225] 5. A method according to clause 0, wherein the prediction of the chrominance samples is based on pred C (i,j)=a1·G L (i,j)+a2·rec′ L (i,j)+a3, where: (i, j) represents the coordinates of the chrominance sample, pred C (i,j) is the predicted value of the chrominance sample, G L (i,j) is the first value, rec′ L (i, j) is the second value, and a1, a2 and a3 are the first parameter, the second parameter and the third parameter respectively.

[0226] 6. A method according to clause 0, wherein the third parameter is determined based on the bit depth of the image.

[0227] 7. A method as described in clause 0, wherein the chroma samples belong to a coding block, and the method further comprises: The first parameter, the second parameter and the third parameter are determined based on one or more neighboring samples of the coding block.

[0228] 8. A method as described in clause 0, wherein the one or more neighboring samples of the coding block include at least one of: The adjacent luma samples of the coding block, or Neighboring chroma samples of the coding block.

[0229] 9. A method as described in clause 0, wherein the one or more neighboring samples of the coding block are selected from: One or more rows of upper adjacent samples or left adjacent samples of the coding block.

[0230] 10. The method according to clause 0, further comprising: Encode or decode syntax elements in a bitstream; and The number of the one or more rows is determined based on the syntax element, wherein: If the syntax element is equal to a third value, the number of the one or more rows is 1, and If the syntax element is equal to a fourth value, then the number of the one or more rows is greater than one.

[0231] 11. The method according to clause 0, further comprising: Encode or decode syntax elements in a bitstream, The one or more neighboring samples of the coding block are determined based on the syntax element, wherein: If the syntax element is equal to a third value, selecting the one or more adjacent samples from upper adjacent samples and left adjacent samples of the coding block, If the syntax element is equal to a fourth value, selecting the one or more neighboring samples from upper neighboring samples of the coding block, and If the syntax element is equal to a fifth value, the one or more neighboring samples are selected from left neighboring samples of the coding block.

[0232] 12. A method as described in clause 0, wherein predicting the chroma samples based on the first value and the second value comprises: A Cb component and a Cr component of the chroma sample are predicted based on the first value and the second value.

[0233] 13. The method of clause 0, wherein predicting the chrominance samples further comprises: predicting the chrominance samples based on a plurality of values, Wherein each of the plurality of values ​​is determined by applying a different gradient pattern to reconstructed values ​​of the first plurality of luma samples.

[0234] 14. The method according to clause 0, further comprising: Encode or decode syntax elements in a bitstream, Wherein the syntax element indicates whether the prediction of the chroma sample is based on both the first value and the second value.

[0235] 15. A video processing device, comprising: a memory storing a set of instructions; and one or more processors configured to execute the set of instructions to cause the apparatus to perform: determining a first value associated with a chroma sample by applying a first gradient pattern to a plurality of reconstructed values ​​of a first plurality of luma samples; determining a second value associated with the chroma sample by applying a downsampling filter to a plurality of reconstructed values ​​of a second plurality of luma samples; and Chroma samples are predicted based on the first value and the second value.

[0236] 16. The apparatus of clause 0, wherein the first plurality of luma samples and the second plurality of luma samples each comprise at least one of: the chroma sample's collocated luma sample, or The adjacent luma samples of the collocated luma sample.

[0237] 17. The apparatus of clause 0, wherein, in applying the first gradient pattern to the plurality of reconstructed values ​​of the first plurality of luma samples, the one or more processors are configured to execute the set of instructions to cause the apparatus to perform: Encode or decode syntax elements in a bitstream; and A first gradient mode is selected from a plurality of gradient modes based on the syntax element.

[0238] 18. The apparatus of clause 0, wherein, in predicting the chroma samples based on the first value and the second value, the one or more processors are configured to execute the set of instructions to cause the apparatus to perform: The chroma samples are predicted based on a first parameter associated with the first value, a second parameter associated with the second value, and a third parameter.

[0239] 19. The apparatus of clause 0, wherein the prediction of the chrominance samples is based on pred C (i,j)=a1·G L (i,j)+a2·rec′ L (i,j)+a3, where: (i, j) represents the coordinates of the chrominance sample, pred C (i,j) is the predicted value of the chrominance sample, G L (i,j) is the first value, rec′ L (i, j) is the second value, and a1, a2 and a3 are the first parameter, the second parameter and the third parameter respectively.

[0240] 20. An apparatus as described in clause 0, wherein the third parameter is determined based on a bit depth of the image.

[0241] 21. The apparatus of clause 0, wherein the chroma samples belong to a coding block, and wherein the one or more processors are configured to execute the set of instructions to cause the apparatus to perform: The first parameter, the second parameter and the third parameter are determined based on one or more neighboring samples of the coding block.

[0242] 22. An apparatus according to clause

[0241] , wherein the one or more neighboring samples of the coding block include at least one of the following: The adjacent luma samples of the coding block, or Neighboring chroma samples of the coding block.

[0243] 23. An apparatus according to clause

[0241] , wherein the one or more adjacent samples of the coding block are selected from: One or more rows of upper adjacent samples or left adjacent samples of the coding block.

[0244] 24. An apparatus as described in clause

[0243] , wherein the one or more processors are configured to execute the set of instructions to cause the apparatus to perform: Encode or decode syntax elements in a bitstream; and The number of the one or more rows is determined based on the syntax element, wherein: If the syntax element is equal to a third value, the number of the one or more rows is 1, and If the syntax element is equal to a fourth value, then the number of the one or more rows is greater than one.

[0245] 25. An apparatus as described in clause

[0241] , wherein the one or more processors are configured to execute the set of instructions to cause the apparatus to perform: Encode or decode syntax elements in a bitstream, The one or more neighboring samples of the coding block are determined based on the syntax element, wherein: If the syntax element is equal to a third value, selecting the one or more adjacent samples from upper adjacent samples and left adjacent samples of the coding block, If the syntax element is equal to a fourth value, selecting the one or more neighboring samples from upper neighboring samples of the coding block, and If the syntax element is equal to a fifth value, the one or more neighboring samples are selected from left neighboring samples of the coding block.

[0246] 26. The apparatus of clause 0, wherein, in predicting the chroma samples based on the first value and the second value, the one or more processors are configured to execute the set of instructions to cause the apparatus to perform: A Cb component and a Cr component of the chroma sample are predicted based on the first value and the second value.

[0247] 27. The apparatus of clause 0, wherein, in predicting the chroma samples, the one or more processors are configured to execute the set of instructions to cause the apparatus to perform: predicting the chrominance samples based on a plurality of values, Wherein each of the plurality of values ​​is determined by applying a different gradient pattern to reconstructed values ​​of the first plurality of luma samples.

[0248] 28. The apparatus of clause 0, wherein the one or more processors are configured to execute the set of instructions to cause the apparatus to perform: Encode or decode syntax elements in a bitstream, Wherein the syntax element indicates whether the prediction of the chroma sample is based on both the first value and the second value.

[0249] 29. A non-transitory computer-readable medium storing a bitstream of a video for processing according to a method, comprising: determining a first value associated with a chroma sample by applying a first gradient pattern to reconstructed values ​​of a first plurality of luma samples; determining a second value associated with the chroma sample by applying a downsampling filter to reconstructed values ​​of a second plurality of luma samples; and Chroma samples are predicted based on the first value and the second value.

[0250] 30. The non-transitory computer-readable medium of clause

[0249] , wherein the first plurality of luma samples and the second plurality of luma samples each include at least one of: the chroma sample's collocated luma sample, or The adjacent luma samples of the collocated luma sample.

[0251] 31. The non-transitory computer-readable medium of clause

[0249] , wherein applying the first gradient pattern to the reconstructed values ​​of the first plurality of luma samples comprises: Encode or decode syntax elements in a bitstream; and A first gradient mode is selected from a plurality of gradient modes based on the syntax element.

[0252] 32. The non-transitory computer-readable medium of clause

[0249] , wherein predicting the chroma samples based on the first value and the second value comprises: The chroma samples are predicted based on a first parameter associated with the first value, a second parameter associated with the second value, and a third parameter.

[0253] 33. A non-transitory computer-readable medium according to clause

[0252] , wherein the prediction of the chrominance samples is based on pred C (i,j)=a1·G L (i,j)+a2·rec′ L (i,j)+a3, where: (i, j) represents the coordinates of the chrominance sample, pred C (i,j) is the predicted value of the chrominance sample, G L (i,j) is the first value, rec ′ L′ L (i, j) is the second value, and a1, a2 and a3 are the first parameter, the second parameter and the third parameter respectively.

[0254] 34. A non-transitory computer-readable medium as described in clause

[0252] , wherein the third parameter is determined based on the bit depth of the image.

[0255] 35. The non-transitory computer-readable medium of clause

[0252] , wherein the chroma samples belong to a coding block, and the method further comprises: The first parameter, the second parameter and the third parameter are determined based on one or more neighboring samples of the coding block.

[0256] 36. The non-transitory computer-readable medium of clause

[0255] , wherein the one or more neighboring samples of the coding block include at least one of: The adjacent luma samples of the coding block, or Neighboring chroma samples of the coding block.

[0257] 37. The non-transitory computer-readable medium of clause

[0255] , wherein the one or more neighboring samples of the coding block are selected from: One or more rows of upper adjacent samples or left adjacent samples of the coding block.

[0258] 38. The non-transitory computer-readable medium of clause

[0257] , further comprising: Encode or decode syntax elements in a bitstream; and The number of the one or more rows is determined based on the syntax element, wherein: If the syntax element is equal to a third value, the number of the one or more rows is 1, and If the syntax element is equal to a fourth value, then the number of the one or more rows is greater than one.

[0259] 39. The non-transitory computer-readable medium of clause

[0255] , further comprising: encoding or decoding a syntax element in a bitstream, and determining the one or more neighboring samples of the coding block based on the syntax element, wherein: If the syntax element is equal to a third value, one or more adjacent samples are selected from upper adjacent samples and left adjacent samples of the coding block, If the syntax element is equal to a fourth value, one or more neighboring samples are selected from upper neighboring samples of the coding block, and If the syntax element is equal to a fifth value, the one or more neighboring samples are selected from left neighboring samples of the coding block.

[0260] 40. The non-transitory computer-readable medium of clause

[0249] , wherein predicting the chroma samples based on the first value and the second value comprises: A Cb component and a Cr component of the chroma sample are predicted based on the first value and the second value.

[0261] 41. The non-transitory computer-readable medium of clause

[0249] , wherein predicting the chrominance samples further comprises: predicting the chrominance samples based on a plurality of values, Wherein each of the plurality of values ​​is determined by applying a different gradient pattern to reconstructed values ​​of the first plurality of luma samples.

[0262] 42. The non-transitory computer-readable medium of clause

[0249] , further comprising: Encode or decode syntax elements in a bitstream, Wherein the syntax element indicates whether the prediction of the chroma sample is based on both the first value and the second value.

[0263] 43. A video processing method, comprising: predicting a chroma sample from a collocated luma sample associated with the chroma sample, Therein, the prediction is based on a non-linear model defining a non-linear relationship between predicted values ​​of the chrominance samples and values ​​associated with the collocated luma samples.

[0264] 44. A method according to clause

[0263] , wherein the nonlinear model is represented by a nonlinear equation: pred C (i,j)=a n ·(rec′ L (i,j)) n +a n-1 ·(rec′ L (i,j)) n-1 +…+a1·rec′L (i,j)+a0, in: (i, j) represents the coordinates of the chrominance sample, pred C (i,j) is the predicted value of the chrominance sample, rec′ L (i,j) is the value associated with the collocated luminance sample, n is an integer equal to or greater than 2, and a n ,a n-1 ,...a1 and a0 are the parameters of the nonlinear model.

[0265] 45. A method according to clause

[0264] , wherein parameter a n ,a n-1 ,...at least one of a1 and a0 is equal to zero.

[0266] 46. ​​A method according to clause

[0264] , wherein parameter a n ,a n-1 ,...At least one of a1 and a0 is determined based on the bit depth of the image.

[0267] 47. A method according to clause

[0264] , wherein the chroma samples belong to the coding block, and the method further comprises: The parameter a is determined based on the neighboring samples of the coding block. n ,a n-1 ,...a1 and a0.

[0268] 48. The method of clause

[0263] , further comprising: A value associated with the collocated luma sample is determined by applying a downsampling filter to reconstructed values ​​of a plurality of luma samples associated with the collocated luma sample.

[0269] 49. A method according to clause

[0263] , wherein more than one non-linear model is used to predict chrominance samples in a coding block, and the method further comprises: classifying luma samples in the coding block into a plurality of categories; and Apply multiple nonlinear models to multiple classes separately.

[0270] 50. A method according to clause

[0263] , wherein the non-linear model further defines a relationship between the predicted value of the chrominance sample and a gradient value associated with the collocated luma sample, wherein: The gradient value is determined based on applying a gradient pattern to reconstructed values ​​of a plurality of luma samples associated with the collocated luma sample.

[0271] 51. The method of clause

[0270] , wherein the nonlinear model comprises a n ·(G L (i,j)) n At least one of the following: (i, j) represents the coordinates of the chrominance sample, G L (i, j) is the gradient value, n is an integer equal to or greater than 1, and a n are the parameters of the nonlinear model.

[0272] 52. The method of clause

[0271] , wherein the nonlinear model further comprises a m ·(rec′ L (i, j)) m At least one of the following: rec′ L (i, j) is the value associated with the collocated luminance sample, m is an integer equal to or greater than 1, and a m are the parameters of the nonlinear model.

[0273] 53. A video processing device, comprising: a memory storing a set of instructions; and one or more processors configured to execute the set of instructions to cause the apparatus to perform: predicting a chroma sample from a collocated luma sample associated with the chroma sample, Therein, the prediction is based on a non-linear model defining a non-linear relationship between predicted values ​​of the chrominance samples and values ​​associated with the collocated luma samples.

[0274] 54. An apparatus according to clause

[0273] , wherein the nonlinear model is represented by a nonlinear equation: pred C (i, j) = a n ·(rec′ L (i, j)) n +a n x1·(rec′ L (i, j)) n-1 +…+a1·rec′ L (i, j)+a0, in: (i, j) represents the coordinates of the chrominance sample, pred C (i, j) is the predicted value of the chrominance sample, rec′ L (i, j) is the value associated with the collocated luminance sample, n is an integer equal to or greater than 2, and a n , a n-1 , ...a1 and a0 are the parameters of the nonlinear model.

[0275] 55. An apparatus according to clause

[0274] , wherein parameter a n , a n At least one of -1, ...a1 and a0 is equal to zero.

[0276] 56. An apparatus according to clause

[0274] , wherein parameter a n , a n At least one of -1, . . . a1 and a0 is determined based on the bit depth of the image.

[0277] 57. The apparatus of clause

[0274] , wherein the chroma samples belong to a coding block, and wherein the one or more processors are configured to execute the set of instructions to cause the apparatus to perform: The parameter a is determined based on the neighboring samples of the coding block. n , a n-1 , ...a1 and a0.

[0278] 58. An apparatus as described in clause

[0273] , wherein the one or more processors are configured to execute the set of instructions to cause the apparatus to perform: A value associated with the collocated luma sample is determined by applying a downsampling filter to reconstructed values ​​of a plurality of luma samples associated with the collocated luma sample.

[0279] 59. An apparatus as described in clause

[0273] , wherein more than one non-linear model is used to predict chrominance samples in a coding block, and the one or more processors are configured to execute the set of instructions to cause the apparatus to perform: classifying luma samples in the coding block into a plurality of categories; and Apply multiple nonlinear models to multiple classes separately.

[0280] 60. An apparatus as described in clause

[0273] , wherein the non-linear model further defines a relationship between the predicted value of the chroma sample and the gradient value associated with the collocated luma sample, wherein: The gradient value is determined based on applying a gradient pattern to reconstructed values ​​of a plurality of luma samples associated with the collocated luma samples.

[0281] 61. The apparatus of clause

[0280] , wherein the nonlinear model comprises a n ·(G L (i, j)) n At least one of the following: (i, j) represents the coordinates of the chrominance sample, G L (i, j) is the gradient value, n is an integer equal to or greater than 1, and a n are the parameters of the nonlinear model.

[0282] 62. The apparatus of clause

[0281] , wherein the nonlinear model further comprises a m ·(rec′ L (i, j)) m At least one of the following: rec′ L (i, j) is the value associated with the collocated luminance sample, m is an integer equal to or greater than 1, and a m are the parameters of the nonlinear model.

[0283] 63. A non-transitory computer-readable medium storing a bitstream of a video for processing according to a method, comprising: predicting a chroma sample from a collocated luma sample associated with the chroma sample, Therein, the prediction is based on a non-linear model defining a non-linear relationship between predicted values ​​of the chrominance samples and values ​​associated with the collocated luma samples.

[0284] 64. The non-transitory computer-readable medium of clause

[0283] , wherein the nonlinear model is represented by a nonlinear equation: pred C (i, j) = a n ·(rec′ L (i, j)) n +a n-1 ·(rec′ L (i, j)) n-1 +…+a1·rec′ L (i, j)+a0, in: (i, j) represents the coordinates of the chrominance sample, pred C (i, j) is the predicted value of the chrominance sample, rec′ L (i, j) is the value associated with the collocated luminance sample, n is an integer equal to or greater than 2, and a n , a n-1 , ...a1 and a0 are the parameters of the nonlinear model.

[0285] 65. The non-transitory computer-readable medium of clause

[0284] , wherein parameter a n , a n-1 , ...at least one of a1 and a0 is equal to zero.

[0286] 66. The non-transitory computer-readable medium of clause

[0284] , wherein the parameter a n , a n-1 , ...at least one of a1 and a0 is determined based on the bit depth of the image.

[0287] 67. The non-transitory computer-readable medium of clause

[0284] , wherein the chroma samples belong to a coding block, and the method further comprises: The parameter a is determined based on neighboring samples of the coding block n , a n-1 , ...a1 and a0.

[0288] 68. The non-transitory computer-readable medium of clause

[0283] , further comprising: A value associated with the collocated luma sample is determined by applying a downsampling filter to reconstructed values ​​of a plurality of luma samples associated with the collocated luma sample.

[0289] 69. The non-transitory computer-readable medium of clause

[0283] , wherein more than one non-linear model is used to predict chrominance samples in a coding block, and the method further comprises: classifying luma samples in the coding block into a plurality of categories; and Apply multiple nonlinear models to multiple classes separately.

[0290] 70. The non-transitory computer-readable medium of clause

[0283] , wherein the non-linear model further defines a relationship between the predicted value of the chroma sample and a gradient value associated with the collocated luma sample, wherein: A gradient value is determined based on applying a gradient pattern to reconstructed values ​​of a plurality of luma samples associated with the collocated luma samples.

[0291] 71. The non-transitory computer-readable medium of clause

[0290] , wherein the nonlinear model comprises a n ·(G L (i,j)) n At least one of the following: (i, j) represents the coordinates of the chrominance sample, G L (i, j) is the gradient value, n is an integer equal to or greater than 1, and a n are the parameters of the nonlinear model.

[0292] 72. The non-transitory computer-readable medium of clause

[0291] , wherein the nonlinear model further comprises a m ·(rec′ L (i,j)) m At least one of the following: rec′ L (i,j) is the value associated with the collocated luminance sample, m is an integer equal to or greater than 1, and a m are the parameters of the nonlinear model.

[0293] 73. A computer program product comprising computer program instructions, and the computer program instructions enable a computer to perform the video processing method according to any one of clauses 1 to 14 and 43 to 52.

[0294] 74. A computer program, wherein the computer program enables a computer to perform the video processing method according to any one of clauses 1 to 14 and 43 to 52.

[0295] It should be noted that the relational terms such as "first", "second", etc. herein are only used to distinguish one entity or operation from another entity or operation, and do not require or imply any actual relationship or order between these entities or operations. In addition, the words "include", "have", "include", "includes", etc. and other similar forms are intended to be equivalent in meaning and open-ended, because one or more items after any of these words do not mean an exhaustive list of such one or more items, or mean limited to the one or more items listed.

[0296] As used herein, unless otherwise specifically stated, the term "or" encompasses all possible combinations unless not feasible. For example, if it is stated that a database may include A or B, then unless otherwise specifically stated or not feasible, the database may include A, or B, or A and B. As a second example, if it is stated that a database may include A, B, or C, then unless otherwise specifically stated or not feasible, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0297] It should be understood that the above embodiments can be implemented by hardware, or software (program code), or a combination of hardware and software. If implemented by software, it can be stored in the above-mentioned computer-readable medium. When executed by a processor, the software can execute the disclosed method. The computing unit and other functional units described in this disclosure can be implemented by hardware, or software, or a combination of hardware and software. It will also be understood by those of ordinary skill in the art that multiple modules / units in the above-mentioned modules / units can be combined into one module / unit, and each of the above-mentioned modules / units can be further divided into multiple sub-modules / sub-units.

[0298] In the foregoing description, embodiments have been described with reference to many specific details that may vary with implementation. Certain adjustments and modifications may be made to the described embodiments. Other embodiments will be apparent to those skilled in the art by considering the specification and practice of the invention disclosed herein. The description and embodiments are to be considered exemplary only, with the true scope and spirit of the invention being indicated by the appended claims. The sequence of steps shown in the figures is also intended to be for illustrative purposes only and is not intended to be limited to any particular sequence of steps. Therefore, it will be appreciated by those skilled in the art that these steps may be performed in different orders while implementing the same method.

[0299] In the drawings and the specification, exemplary embodiments have been disclosed. However, many variations and modifications may be made to these embodiments. Therefore, although specific terms are employed, they are used only in a general and descriptive sense and not for the purpose of limitation.

Claims

1. A video processing method, characterized in that: include: determining a first value associated with a chroma sample by applying a first gradient pattern to reconstructed values ​​of a first plurality of luma samples; determining a second value associated with the chroma sample by applying a downsampling filter to reconstructed values ​​of a second plurality of luma samples; as well as The chroma samples are predicted based on the first value and the second value.

2. The method according to claim 0, wherein: The first plurality of luma samples and the second plurality of luma samples each include at least one of: the collocated luma sample of the chroma sample, or The adjacent luma samples of the collocated luma sample.

3. The method according to claim 0 or 2, wherein: Applying the first gradient pattern to the reconstructed values ​​of the first plurality of luma samples comprises: Encode or decode syntax elements in a bitstream; as well as A first gradient mode is selected from a plurality of gradient modes based on the syntax element.

4. The method of any one of claims 0 to 3, wherein predicting the chroma samples based on the first value and the second value comprises: The chroma samples are predicted based on a first parameter associated with the first value, a second parameter associated with the second value, and a third parameter.

5. The method according to claim 0, wherein: The prediction of the chrominance samples is based on pred C (i,j)=a1·G L (i,j)+a2·rec′ L (i,j)+a3, where: (i, j) represents the coordinates of the chrominance sample, pred C (i,j) is the predicted value of the chrominance sample, G L (i,j) is the first value, rec′ L (i, j) is the second value, and a1, a2, and a3 are the first parameter, the second parameter, and the third parameter, respectively.

6. The method according to claim 0, wherein: The third parameter is determined based on the bit depth of the image.

7. The method according to claim 0, characterized in that: The chroma samples belong to a coding block, and the method further comprises: The first parameter, the second parameter and the third parameter are determined based on one or more neighboring samples of the coding block.

8. The method according to claim 0, wherein: The one or more neighboring samples of the coding block include at least one of the following: The adjacent luma samples of the coding block, or Neighboring chroma samples of the coding block.

9. The method according to claim 0, wherein the one or more adjacent samples of the coding block are selected from: One or more rows of upper adjacent samples or left adjacent samples of the coding block.

10. The method according to claim 0, further comprising: Encode or decode syntax elements in a bitstream; and The number of the one or more rows is determined based on the syntax element, wherein: If the syntax element is equal to a third value, the number of the one or more rows is 1, and If the syntax element is equal to a fourth value, then the number of the one or more rows is greater than one.

11. The method according to claim 0, characterized in that: Also includes: Encode or decode syntax elements in a bitstream, The one or more neighboring samples of the coding block are determined based on the syntax element, wherein: If the syntax element is equal to a third value, selecting the one or more neighboring samples from the upper neighboring samples and the left neighboring samples of the coding block, If the syntax element is equal to a fourth value, one or more neighboring samples are selected from upper neighboring samples of the coding block, and If the syntax element is equal to a fifth value, the one or more neighboring samples are selected from left neighboring samples of the coding block.

12. The method according to any one of claims 0 to 11, characterized in that Predicting the chroma sample based on the first value and the second value comprises: A Cb component and a Cr component of the chroma sample are predicted based on the first value and the second value.

13. The method according to claim 0, characterized in that: The predicting of the chroma samples further comprises: predicting the chrominance samples based on a plurality of values, Wherein each of the plurality of values ​​is determined by applying a different gradient pattern to reconstructed values ​​of the first plurality of luminance samples.

14. The method according to claim 0, characterized in that Also includes: Encode or decode syntax elements in a bitstream, Wherein the syntax element indicates whether the prediction of the chroma sample is based on both the first value and the second value.

15. A video processing device, comprising: a memory storing a set of instructions; and one or more processors configured to execute the set of instructions to cause the apparatus to perform: determining a first value associated with a chroma sample by applying a first gradient pattern to reconstructed values ​​of a first plurality of luma samples; determining a second value associated with the chroma sample by applying a downsampling filter to reconstructed values ​​of a second plurality of luma samples; as well as The chroma samples are predicted based on the first value and the second value.

16. The apparatus of claim 0, wherein the first plurality of luma samples and the second plurality of luma samples each comprise at least one of: the collocated luma sample of the chroma sample, or The adjacent luma samples of the collocated luma sample.

17. The device according to claim 0 or 16, wherein: When applying the first gradient pattern to the reconstructed values ​​of the first plurality of luma samples, the one or more processors are configured to execute the set of instructions to cause the apparatus to perform: Encode or decode syntax elements in a bitstream; as well as The first gradient mode is selected from a plurality of gradient modes based on the syntax element.

18. The device according to any one of claims 0 to 17, wherein: In predicting the chroma sample based on the first value and the second value, the one or more processors are configured to execute the set of instructions to cause the apparatus to perform: The chroma samples are predicted based on a first parameter associated with the first value, a second parameter associated with the second value, and a third parameter.

19. The apparatus of claim 0, wherein the prediction of the chrominance samples is based on pred C (i,j)=a1·G L (i,j)+a2·rec′ L (i,j)+a3, where: (i, j) represents the coordinates of the chrominance sample, pred C (i,j) is the predicted value of the chrominance sample, G L (i,j) is the first value, rec′ L (i, j) is the second value, and a1, a2, and a3 are the first parameter, the second parameter, and the third parameter, respectively.

20. A non-transitory computer-readable medium storing a bitstream of a video for processing according to a method, comprising: determining a first value associated with a chroma sample by applying a first gradient pattern to reconstructed values ​​of a first plurality of luma samples; determining a second value associated with the chroma sample by applying a downsampling filter to reconstructed values ​​of a second plurality of luma samples; as well as The chroma samples are predicted based on the first value and the second value.

21. A computer program product, characterized in that The method comprises computer program instructions, wherein the computer program instructions enable a computer to execute the video processing method according to any one of claims 1 to 14.

22. A computer program, characterized in that The computer program enables a computer to execute the video processing method according to any one of claims 1 to 14.

Images