Offset value determination of chroma (CfL) mode according to luminance

Abstract

Various embodiments described herein include methods and systems for coding video. In one aspect, a video bitstream includes different coded blocks of a current image frame, and a cross-component intra prediction mode is signaled. The method includes, for each different coding block, identifying a chroma component, a luma sample co-located with the chroma component, and an adjacent luma sample of the luma sample. In one embodiment, a method includes deriving a respective bias value based on a set of luma samples within a reference region of a respective coded block in a current image frame, the reference region including one or more coded blocks adjacent to and decoded prior to the respective coded block. The method further includes combining the luma samples, adjacent luma samples, and corresponding offset values to generate chroma components, and reconstructing different coded blocks including the chroma components of each different coded block.

Description

Related Applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 444,180, filed on February 8, 2023, entitled “Improved Bias Value for CfL Mode,” and this application is a continuation of and claims priority to U.S. Patent Application No. 18 / 497,896, filed on October 30, 2023, entitled “Bias Value Determination for Chroma-for-Luma (CfL) Mode.” Technical Field

[0002] The disclosed embodiments relate generally to coding video, including but not limited to systems and methods for determining chroma samples from associated luma samples in cross-component intra prediction of video data. Background Art

[0003] Digital video is supported by various electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc. Electronic devices transmit and receive or otherwise communicate digital video data over a communication network, and / or store the digital video data on a storage device. Due to the limited bandwidth capacity of a communication network and the limited memory resources of a storage device, video coding may be used to compress the video data according to one or more video coding standards before transmitting or storing the video data.

[0004] A variety of video codec standards have been developed. For example, video coding standards include AOMedia Video 1 (AV1), Versatile Video Coding (VVC), Joint Exploration test Model (JEM), High-Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), and Moving Picture Expert Group (MPEG) coding. Video coding typically utilizes prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that exploit the redundancy inherent in video data. Video coding aims to compress video data into a form that uses a lower bit rate while avoiding or minimizing the degradation of video quality.

[0005] HEVC (also known as H.265) is a video compression standard designed as part of the MPEG-H project. ITU-T and ISO / IEC released the HEVC / H.265 standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). Versatile Video Coding (VVC) (also known as H.266) is a video compression standard intended to be a successor to HEVC. ITU-T and ISO / IEC released the VVC / H.266 standard in 2020 (version 1) and 2022 (version 2). AV1 is an open video coding format designed as an alternative to HEVC. On January 8, 2019, the verified version 1.0.0 of the specification with Errata 1 was released. Summary of the invention

[0006] As mentioned above, encoding (compression) reduces bandwidth and / or storage space requirements. As described in detail later, both lossless compression and lossy compression can be used. Lossless compression refers to a technique that can reconstruct an exact copy of the original signal from the compressed original signal via a decoding process. Lossy compression refers to a decoding / decoding process in which the original video information is not fully retained during decoding and cannot be fully restored during decoding. When lossy compression is used, the reconstructed signal may not be the same as the original signal, but the distortion between the original signal and the reconstructed signal is small enough to make the reconstructed signal useful for the intended application. The amount of tolerable distortion depends on the application. For example, users of certain consumer video streaming applications can tolerate higher distortion than users of movie or television broadcast applications. The compression ratio achievable by a specific decoding algorithm can be selected or adjusted to reflect various distortion tolerances: higher tolerable distortion generally allows decoding algorithms that produce higher losses and higher compression ratios.

[0007] The present disclosure describes the application of bias terms in a cross-component intra prediction (CCIP) mode for determining each of a plurality of chroma samples of a current decoding block based on one or more luma samples to implement cross-component intra prediction of video data. For example, the CCIP mode includes a cross-component linear model (CCLM) mode and a convolutional cross-component mode (CCCM). In the CCLM mode, chroma samples are converted based on a linear model according to a reconstructed luma sample co-located with the chroma sample. In the CCCM mode, chroma samples are predicted based on a plurality of reconstructed luma samples positioned based on a filter shape of a filter with a plurality of weighting factors. In both the CCLM mode and the CCCM mode, when chroma subsampling is used, the reconstructed luma samples are downsampled to match a chroma grid of a lower resolution, and there is an option to use a single model or a multi-model variant. In some embodiments, the multi-model variant uses two models, one derived based on luma samples above an average luma reference value and the other derived based on the remaining luma samples. Furthermore, in some embodiments, for prediction units having at least 128 available reference samples, such a multi-model CCCM mode is selected.

[0008] In some embodiments, the CCCM mode is associated with a filter (e.g., a 7-tap filter, a 5-tap filter) having multiple weighting factors. Multiple weighting factors are applied together with two additional weighting factors to combine luma samples, nonlinear terms, and bias terms. For example, a cross-shaped 5-tap filter with five inputs consists of a central (C) luma sample juxtaposed with a central chroma sample to be predicted and four adjacent luma samples, the four adjacent luma samples including an upper / north (N) adjacent sample, a lower / south (S) adjacent sample, a left / west (W) adjacent sample, and a right / east (E) adjacent sample. The nonlinear term P represents the square of the central luma sample C scaled to a sample value range. The bias term B represents a scalar offset between the input and the output, and, for example, the bias term B is set to an intermediate chroma value (512 for 10-bit content). In some embodiments, the output of the filter is determined as the convolution between the weighting factor ci (also referred to as the filter coefficient ci) and the input luma sample, and is clipped to the range of valid chroma samples. Various embodiments of the present application are intended to determine a bias term B in cross-component intra-frame prediction of video data (e.g., in CCCM mode), for example, by adaptively deriving a corresponding bias value for each decoding block of a current image frame based on a set of one or more luminance samples within a reference area of ​​the corresponding decoding block.

[0009] According to some embodiments, a method for decoding a video is provided. The method includes receiving a video bitstream including a plurality of different decoding blocks of a current image frame. The video bitstream includes a syntax element for a cross-component intra prediction (CCIP) mode, which indicates that each chrominance component of the plurality of different decoding blocks is determined based on one or more luminance samples. The method also includes deriving a corresponding bias value based on a set of one or more luminance samples in a reference area of ​​a corresponding decoding block in the current image frame. The reference area includes one or more decoding blocks that are adjacent to the corresponding decoding block and decoded before the corresponding decoding block. The method also includes reconstructing a plurality of decoding blocks including the corresponding decoding block based at least on the derived bias value.

[0010] According to some embodiments, a method for encoding a video is provided. The method includes encoding each corresponding decoding block of a plurality of different decoding blocks of a current image frame. Encoding each corresponding decoding block also includes: identifying a first chrominance component of the corresponding decoding block and a first luminance sample co-located with the first chrominance component in the corresponding decoding block, identifying a plurality of adjacent luminance samples of the first luminance sample, and deriving a corresponding bias value based on a set of one or more luminance samples in a reference area of ​​the corresponding decoding block in the current image frame. The reference area includes one or more decoding blocks adjacent to the corresponding decoding block and to be decoded before the corresponding decoding block. The method also includes generating a video bitstream including the plurality of different decoding blocks of the current image frame. The video bitstream includes a syntax element for a cross-component intra prediction (CCIP) mode, which indicates that each chrominance component of the plurality of different decoding blocks is determined based on one or more luminance samples.

[0011] According to some embodiments, a computing system is provided, such as a streaming system, a server system, a personal computer system, or other electronic device. The computing system includes a control circuit system and a memory storing one or more instruction sets. The one or more instruction sets include instructions for performing any of the methods described herein. In some embodiments, the computing system includes an encoder component and / or a decoder component.

[0012] According to some embodiments, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores one or more instruction sets executed by a computing system. The one or more instruction sets include instructions for performing any of the methods described herein.

[0013] Therefore, devices and systems utilizing methods for decoding video are disclosed. Such methods, devices, and systems may supplement or replace conventional methods, devices, and systems for decoding video.

[0014] The features and advantages described in the specification are not necessarily exhaustive, and in particular, some additional features and advantages will be apparent to one of ordinary skill in the art in view of the drawings, description, and claims provided in the present disclosure. Furthermore, it should be noted that the language used in the specification is primarily selected for readability and instructional purposes, and is not selected to describe or limit the subject matter described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order that the present disclosure may be understood in more detail, a more specific description may be made by reference to the features of various embodiments, some of which are illustrated in the accompanying drawings. However, the accompanying drawings illustrate only the relevant features of the present disclosure and are therefore not to be considered limiting, as those skilled in the art will appreciate after reading the present disclosure that the description may allow for other effective features.

[0016] Figure 1 is a block diagram illustrating an example communication system in accordance with some implementations.

[0017] Figure 2A is a block diagram illustrating example elements of an encoder component according to some implementations.

[0018] Figure 2B is a block diagram illustrating example elements of a decoder component according to some implementations.

[0019] Figure 3 is a block diagram illustrating an example server system according to some implementations.

[0020] Figure 4 An example scheme for generating chroma samples from multiple luma samples in CCCM mode is shown according to some embodiments.

[0021] Figure 5A is a diagram of example reference regions including multiple reference regions for a coding block in a current image frame according to some embodiments.

[0022] Figure 5B is a diagram of another example reference region adjusted to include only available samples of a coding block in a current image frame according to some embodiments.

[0023] 6A to 6F is a block diagram of six example filter shapes for filters applied to combine luma samples in CCCM mode according to some embodiments.

[0024] Figure 7 is a flow chart illustrating a method 600 of decoding a video according to some implementations.

[0025] According to common practice, the various features shown in the drawings are not necessarily drawn to scale, and like reference numerals may be used to denote like features throughout the specification and drawings. DETAILED DESCRIPTION

[0026] The present disclosure describes the application of bias terms in a cross-component intra prediction (CCIP) mode for determining each of a plurality of chroma samples of a current decoding block based on one or more luma samples to implement cross-component intra prediction of video data. For example, the CCIP mode includes a cross-component linear model (CCLM) mode and a convolutional cross-component mode (CCCM). In the CCLM mode, a reconstructed luma sample co-located with a chroma sample is converted into a chroma sample based on a linear model. In the CCCM mode, a chroma sample is predicted based on a plurality of reconstructed luma samples positioned based on a filter shape of a filter with a plurality of weighting factors. In both the CCLM mode and the CCCM mode, when chroma subsampling is used, the reconstructed luma sample is downsampled to match a chroma grid of a lower resolution, and there is an option to use a single model or a multi-model variant. In some embodiments, the multi-model variant uses two models, one derived based on luma samples above an average luma reference value and the other derived based on the remaining luma samples. Furthermore, in some embodiments, for prediction units having at least 128 available reference samples, such a multi-model CCCM mode is selected.

[0027] In some embodiments, the CCCM mode is associated with a filter (e.g., a 7-tap filter, a 5-tap filter) having multiple weighting factors. Multiple weighting factors are applied together with two additional weighting factors to combine luma samples, nonlinear terms, and bias terms. For example, a cross-shaped 5-tap filter with five inputs consists of a center (C) luma sample juxtaposed with a center chroma sample to be predicted and four adjacent luma samples, the four adjacent luma samples including an upper / north (N) adjacent sample, a lower / south (S) adjacent sample, a left / west (W) adjacent sample, and a right / east (E) adjacent sample. The nonlinear term P represents the square of the center luma sample C scaled to a sample value range. The bias term B represents a scalar offset between the input and the output, and, for example, the bias term B is set to an intermediate chroma value (512 for 10-bit content). Various embodiments of the present application are directed to determining a bias term B in cross-component intra prediction of video data (e.g., in CCCM mode), for example, by adaptively deriving a corresponding bias value for each coding block of a current image frame based on a set of one or more luma samples within a reference region of the corresponding coding block. In addition, in some embodiments, the output of the filter is determined as a convolution between a weighting factor ci (also referred to as a filter coefficient ci) and an input luma sample, and is clipped to a range of valid chroma samples.

[0028] Figure 1 1 is a block diagram illustrating a communication system 100 according to some embodiments. The communication system 100 includes a source device 102 and a plurality of electronic devices 120 (e.g., electronic device 120-1 to electronic device 120-m) communicatively coupled to each other via one or more networks. In some embodiments, the communication system 100 is, for example, a streaming system for use with video-enabled applications such as video conferencing applications, digital TV applications, and media storage and / or distribution applications.

[0029] The source device 102 includes a video source 104 (e.g., a camera component or a media storage device) and an encoder component 106. In some embodiments, the video source 104 is a digital camera (e.g., configured to create an uncompressed video sample stream). The encoder component 106 generates one or more coded video bitstreams based on the video stream. Compared with the coded video bitstream 108 generated by the encoder component 106, the video stream from the video source 104 may be a high data volume. Because the coded video bitstream 108 has a lower data volume (less data) than the video stream from the video source, the coded video bitstream 108 requires less bandwidth to transmit and requires less storage space to store than the video stream from the video source 104. In some embodiments, the source device 102 does not include an encoder component 106 (e.g., configured to transmit uncompressed video data to the network 110).

[0030] One or more networks 110 represent any number of networks that transmit information between source device 102, server system 112, and / or electronic device 120, including, for example, wired (wired) and / or wireless communication networks. One or more networks 110 can exchange data in circuit switching channels and / or packet switching channels. Representative networks include telecommunication networks, local area networks, wide area networks, and / or the Internet.

[0031] One or more networks 110 include a server system 112 (e.g., a distributed / cloud computing system). In some embodiments, the server system 112 is a streaming server (e.g., configured to store and / or distribute video content, such as an encoded video stream from a source device 102), or includes the streaming server. The server system 112 includes a decoder component 114 (e.g., configured to encode and / or decode video data). In some embodiments, the decoder component 114 includes an encoder component and / or a decoder component. In various embodiments, the decoder component 114 is instantiated as hardware, software, or a combination thereof. In some embodiments, the decoder component 114 is configured to decode the encoded video bitstream 108 and re-encode the video data using different encoding standards and / or methods to generate encoded video data 116. In some embodiments, the server system 112 is configured to generate multiple video formats and / or encodings based on the encoded video bitstream 108.

[0032] In some embodiments, the server system 112 functions as a Media-Aware Network Element (MANE). For example, the server system 112 can be configured to prune the encoded video bitstream 108 to customize a potentially different bitstream for one or more of the electronic devices 120. In some embodiments, the MANE is provided separately from the server system 112.

[0033] The electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, the decoder component 122 is configured to decode the encoded video data 116 to generate an outgoing video stream that can be presented on a display or other type of presentation device. In some embodiments, one or more of the electronic devices 120 do not include a display component (e.g., communicatively coupled to an external display device and / or include a media storage device). In some embodiments, the electronic device 120 is a streaming client. In some embodiments, the electronic device 120 is configured to access the server system 112 to obtain the encoded video data 116.

[0034] The source device and / or multiple electronic devices 120 are sometimes referred to as “end devices” or “user devices.” In some embodiments, one or more of the electronic devices 120 and / or the source device 102 are examples of server systems, personal computers, portable devices (e.g., smart phones, tablets, or laptops), wearable devices, video conferencing devices, and / or other types of electronic devices.

[0035] In an example operation of the communication system 100, the source device 102 transmits a coded video bitstream 108 to a server system 112. For example, the source device 102 can decode a picture stream captured by the source device. The server system 112 receives the coded video bitstream 108 and can decode and / or encode the coded video bitstream 108 using a decoder component 114. For example, the server system 112 can apply a coding that is more optimal for network transmission and / or storage to the video data. The server system 112 can transmit the coded video data 116 (e.g., one or more decoded video bitstreams) to one or more of the electronic devices 120. Each electronic device 120 can decode the coded video data 116 to recover and optionally display the video picture.

[0036] In some embodiments, the transmission discussed above is a unidirectional data transmission. Unidirectional data transmission is sometimes used for media service applications, etc. In some embodiments, the transmission discussed above is a bidirectional data transmission. Bidirectional data transmission is sometimes used for video conferencing applications, etc. In some embodiments, the encoded video bitstream 108 and / or the encoded video data 116 are encoded and / or decoded according to any video coding / compression standard described herein, such as HEVC, VVC, and / or AV1.

[0037] Figure 2A1 is a block diagram illustrating example elements of an encoder component 106 according to some embodiments. The encoder component 106 receives a source video sequence from a video source 104. In some embodiments, the encoder component includes a receiver (e.g., a transceiver) component configured to receive the source video sequence. In some embodiments, the encoder component 106 receives the video sequence from a remote video source (e.g., a video source that is a component of a device different from the encoder component 106). The video source 104 can provide a source video sequence in the form of a digital video sample stream, which can have any suitable bit depth (e.g., 8 bits, 10 bits, or 12 bits), any color space (e.g., BT.601 Y CrCb or RGB), and any suitable sampling structure (e.g., Y CrCb 4:2:0 or Y CrCb 4:4:4). In some embodiments, the video source 104 is a storage device that stores previously captured / prepared videos. In some embodiments, the video source 104 is a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual pictures that are imparted with motion when viewed sequentially. The pictures themselves may be organized as a spatial array of pixels, where each pixel may include one or more samples, depending on the sampling structure, color space, etc. used. The relationship between pixels and samples may be readily understood by one of ordinary skill in the art. The following description focuses on samples.

[0038] The encoder component 106 is configured to decode and / or compress the pictures of the source video sequence into a decoded video sequence 216 in real time or under other time constraints required by the application. Implementing the appropriate decoding speed is a function of the controller 204. In some embodiments, the controller 204 controls other functional units as described below and is functionally coupled to the other functional units. The parameters set by the controller 204 may include rate control related parameters (e.g., picture skipping, quantizer and / or lambda value of rate distortion optimization techniques), picture size, picture group (Group Of Pictures, GOP) layout, maximum motion vector search range, etc. Other functions of the controller 204 can be easily identified by those of ordinary skill in the art, as they may belong to the encoder component 106 optimized for a specific system design.

[0039] In some embodiments, the encoder component 106 is configured to operate in a decoding loop. In a simplified example, the decoding loop includes a source decoder 202 (e.g., responsible for creating symbols, such as a symbol stream, based on the input picture to be decoded and the reference picture) and a (local) decoder 210. The decoder 210 reconstructs the symbols to create sample data in a similar manner to the (remote) decoder (when the compression between the symbols and the decoded video bitstream is lossless). The reconstructed sample stream (sample data) is input to the reference picture memory 208. Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the contents of the reference picture memory 208 are also bit-accurate between the local encoder and the remote encoder. In this way, the prediction part of the encoder interprets the same sample values ​​as the decoder would interpret when using prediction during decoding as reference picture samples. The principles of reference picture synchronization (and the drift that occurs when synchronization cannot be maintained, for example due to channel errors) are known to those of ordinary skill in the art.

[0040] The operation of decoder 210 may be combined with a remote decoder, for example, as follows Figure 2B The operation of the decoder component 122 described in detail is the same. However, briefly refer to Figure 2B Since the symbols are available and the encoding / decoding of the symbols into a decoded video sequence by the entropy decoder 214 and the parser 254 can be lossless, the entropy decoding portion of the decoder component 122 including the buffer memory 252 and the parser 254 may not be fully implemented in the local decoder 210.

[0041] At this point, it can be observed that any decoder technology other than the parsing / entropy decoding present in the decoder must also be present in the corresponding encoder in substantially the same functional form. For this reason, the disclosed subject matter focuses on the decoder operation. Since the encoder technology is opposite to the decoder technology described comprehensively, the description of the encoder technology can be simplified. A more detailed description is only required and provided in some parts below.

[0042] As part of its operation, the source decoder 202 can perform motion compensated predictive decoding, which predictively decodes the input frame with reference to one or more previously decoded frames in the video sequence that are designated as reference image frames. In this way, the decoding engine 212 decodes the differences between the pixel blocks of the input frame and the pixel blocks of the reference image frame, which can be selected as the prediction reference for the input frame. The controller 204 can manage the decoding operations of the source decoder 202, including, for example, setting parameters and subgroup parameters for encoding video data.

[0043] The decoder 210 decodes the decoded video data of the frame that can be designated as the reference image frame based on the symbols created by the source decoder 202. The operation of the decoding engine 212 can advantageously be a lossy process. When the decoded video data is in the video decoder ( Figure 2A When decoded at a remote video decoder (not shown), the reconstructed video sequence may be a copy of the source video sequence with some errors. Decoder 210 repeats the decoding process that may be performed on the reference image frame by the remote video decoder, and may cause the reconstructed reference image frame to be stored in reference picture memory 208. In this manner, encoder component 106 locally stores a copy of the reconstructed reference image frame that has common content (absent transmission errors) with the reconstructed reference image frame to be obtained by the remote video decoder.

[0044] The predictor 206 may perform a prediction search for the decoding engine 212. That is, for a new frame to be decoded, the predictor 206 may search the reference picture memory 208 for sample data (as candidate reference pixel blocks) or specific metadata such as reference picture motion vectors, block shapes, etc. that may be used as appropriate prediction references for the new picture. The predictor 206 may operate on a sample block by pixel block basis to find an appropriate prediction reference. In some cases, as determined by the search results obtained by the predictor 206, the input picture may have prediction references taken from multiple reference pictures stored in the reference picture memory 208.

[0045] The outputs of all the above-mentioned functional units may be subjected to entropy decoding in entropy decoder 214. Entropy decoder 214 converts the symbols generated by the various functional units into a decoded video sequence by losslessly compressing the symbols according to techniques known to those skilled in the art (e.g., Huffman coding, variable length coding, and / or arithmetic coding).

[0046] In some embodiments, the output of the entropy decoder 214 is coupled to a transmitter. The transmitter can be configured to buffer the decoded video sequence created by the entropy decoder 214 in preparation for transmission via a communication channel 218, which can be a hardware / software link to a storage device that will store the encoded video data. The transmitter can be configured to merge the decoded video data from the source decoder 202 with other data to be transmitted, such as decoded audio data and / or an auxiliary data stream (source not shown). In some embodiments, the transmitter can transmit additional data along with the encoded video. The source decoder 202 may include such data as part of the decoded video sequence. The additional data may include time / space / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, Supplementary Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, etc.

[0047] The controller 204 can manage the operation of the encoder component 106. During decoding, the controller 204 can assign a specific decoding picture type to each decoded picture, which may affect the decoding technology applied to the corresponding picture. For example, a picture can be assigned as an intra picture (I picture), a predictive picture (P picture), or a bidirectional predictive picture (B picture). Intra-pictures can be decoded and decoded without using any other frame in the sequence as a prediction source. Some video codecs allow different types of intra-pictures, including, for example, independent decoder refresh (IDR) pictures. Those of ordinary skill in the art understand those variations of I pictures and their corresponding applications and features, and therefore are not repeated here. Predictive pictures can be decoded and decoded using inter-prediction or intra-prediction that predicts sample values ​​for each block using at most one motion vector and reference index. Bidirectional predictive pictures can be decoded and decoded using inter-prediction or intra-prediction that predicts sample values ​​for each block using at most two motion vectors and reference indexes. Similarly, multi-predictive pictures can use more than two reference pictures and associated metadata to reconstruct a single block.

[0048] A source picture may typically be spatially subdivided into blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples, respectively), and decoded on a block-by-block basis. Blocks may be predictively decoded with reference to other (already decoded) blocks, which are determined by the coding assignments applied to the block's corresponding pictures. For example, blocks of an I picture may be non-predictively decoded, or blocks of an I picture may be predictively decoded (spatial prediction or intra prediction) with reference to already decoded blocks of the same picture. Blocks of pixels of a P picture may be non-predictively decoded via spatial prediction or via temporal prediction with reference to one previously decoded reference picture. Blocks of a B picture may be non-predictively decoded via spatial prediction or via temporal prediction with reference to one or two previously decoded reference pictures.

[0049] Video can be captured as multiple source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often referred to simply as intra-prediction) exploits spatial correlations in a given picture, while inter-picture prediction exploits (temporal or other) correlations between pictures. In an example, a particular picture in encoding / decoding, which is referred to as the current picture, is divided into blocks. In the case where a block in the current picture is similar to a reference block in a reference picture that has been previously decoded and is still buffered in the video, the block in the current picture can be decoded by a vector called a motion vector. The motion vector points to a reference block in a reference picture, and in the case of using multiple reference pictures, can have a third dimension that identifies the reference picture.

[0050] The encoder component 106 may perform decoding operations according to a predetermined video decoding technique or standard, such as any described herein. In operation of the encoder component 106, the encoder component 106 may perform various compression operations, including predictive decoding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the decoded video data may conform to the syntax specified by the video decoding technique or standard used.

[0051] Figure 2B is a block diagram illustrating example elements of decoder component 122 according to some implementations. Figure 2B The decoder component 122 in is coupled to the channel 218 and the display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to the loop filter 256 and configured to transmit data to the display 124 (eg, via a wired or wireless connection).

[0052] In some embodiments, the decoder component 122 includes a receiver coupled to the channel 218 and configured to receive data from the channel 218 (e.g., via a wired or wireless connection). The receiver can be configured to receive one or more decoded video sequences to be decoded by the decoder component 122. In some embodiments, the decoding of each decoded video sequence is independent of other decoded video sequences. Each decoded video sequence can be received from the channel 218, which can be a hardware / software link to a storage device storing coded video data. The receiver can receive the coded video data and other data, such as decoded audio data and / or auxiliary data streams, which can be forwarded to their respective use entities (not depicted). The receiver can separate the decoded video sequence from the other data. In some embodiments, the receiver receives additional (redundant) data along with the coded video. The additional data can be included as part of the decoded video sequence. The decoder component 122 can use the additional data to decode the data and / or more accurately reconstruct the original video data. The additional data can be in the form of, for example, time, space or SNR enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0053] According to some embodiments, decoder component 122 includes a buffer memory 252, a parser 254 (sometimes also referred to as an entropy decoder), a scaler / inverse transform unit 258, an intra picture prediction unit 262, a motion compensated prediction unit 260, an aggregator 268, a loop filter unit 256, a reference picture memory 266, and a current picture memory 264. In some embodiments, decoder component 122 is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuit systems. In some embodiments, decoder component 122 is at least partially implemented in software.

[0054] The buffer memory 252 is coupled between the channel 218 and the parser 254 (e.g., to combat network jitter). In some embodiments, the buffer memory 252 is separate from the decoder component 122. In some embodiments, a separate buffer memory is provided between the output of the channel 218 and the decoder component 122. In some embodiments, in addition to the buffer memory 252 internal to the decoder component 122 (e.g., the buffer memory 252 is configured to handle playout timing), a separate buffer memory is provided external to the decoder component 122 (e.g., to combat network jitter). When receiving data from a store / forward device with sufficient bandwidth and controllability or from an isochronous network, the buffer memory 252 may not be required, or the buffer memory 252 may be smaller. In order to make the most of a packet network such as the Internet, the buffer memory 252 may be required, and the buffer memory 252 may be relatively large and may advantageously have an adaptive size, and may be implemented at least in part in an operating system or similar element (not depicted) external to the decoder component 122.

[0055] The parser 254 is configured to reconstruct symbols 270 from the decoded video sequence. The symbols may include, for example, information for managing the operation of the decoder component 122 and / or information for controlling a rendering device such as the display 124. The control information for the rendering device may be in the form of, for example, a Supplementary Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not depicted). The parser 254 parses (entropy decodes) the decoded video sequence. The decoding of the decoded video sequence may be performed according to a video decoding technique or standard and may follow principles known to those skilled in the art, including: variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser 254 may extract a subgroup parameter set for at least one of the subgroups of pixels in the video decoder from the decoded video sequence based on at least one parameter corresponding to the group. Subgroups may include Group of Pictures (GOP), pictures, tiles, slices, macroblocks, Coding Units (CU), blocks, Transform Units (TU), Prediction Units (PU), etc. The parser 254 may also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the decoded video sequence.

[0056] Depending on the type of decoded video picture or part thereof (e.g., inter- and intra-pictures, inter- and intra-blocks) and other factors, the reconstruction of symbol 270 may involve multiple different units. Which units are involved and how they are involved may be controlled by subgroup control information parsed from the decoded video sequence by parser 254. For the sake of brevity, such subgroup control information flow between parser 254 and the following multiple units is not depicted.

[0057] In addition to the functional blocks already mentioned, decoder component 122 can be conceptually subdivided into a plurality of functional units as described below. In the actual implementation operation under commercial constraints, many of these units interact closely with each other and can be integrated with each other at least in part. However, for the purpose of describing the disclosed subject matter, it is kept to be conceptually subdivided into the following functional units.

[0058] The sealer / inverse transform unit 258 receives the quantized transform coefficients and control information (such as which transform to use, block size, quantization factor, and / or quantization scaling matrix) as symbols 270 from the parser 254. The sealer / inverse transform unit 258 may output a block including sample values, which may be input into the aggregator 268.

[0059] In some cases, the output samples of the sealer / inverse transform unit 258 belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed pictures, but can use predictive information from previously reconstructed portions of the current picture. Such predictive information can be provided by the intra picture prediction unit 262. The intra picture prediction unit 262 can use the surrounding already reconstructed information obtained from the current (partially reconstructed) picture in the current picture memory 264 to generate a block of the same size and shape as the block under reconstruction. The aggregator 268 can add the prediction information that has been generated by the intra picture prediction unit 262 to the output sample information as provided by the sealer / inverse transform unit 258 on a per-sample basis.

[0060] In other cases, the output samples of the scaler / inverse transform unit 258 belong to an inter-frame decoding block and potentially to a motion compensated block. In such a case, the motion compensated prediction unit 260 can access the reference picture memory 266 to obtain samples for prediction. After the obtained samples are motion compensated according to the symbols 270 belonging to the block, these samples can be added to the output of the scaler / inverse transform unit 258 (called residual samples or residual signals in this case) by the aggregator 268 to generate output sample information. The address within the reference picture memory 266 from which the motion compensated prediction unit 260 obtains the predicted samples can be controlled by a motion vector. The motion vector can be available to the motion compensated prediction unit 260 in the form of a symbol 270, which can have, for example, an X component, a Y component, and a reference picture component. Motion compensation can also include interpolation of sample values ​​such as those obtained from the reference picture memory 266 when using sub-sample accurate motion vectors, motion vector prediction mechanisms, etc.

[0061] The output samples of aggregator 268 may be subjected to various loop filtering techniques in loop filter unit 256. The video compression techniques may include in-loop filter techniques controlled by parameters included in the decoded video bitstream and available to loop filter unit 256 as symbols 270 from parser 254, but the video compression techniques may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of a decoded picture or a decoded video sequence, as well as to previously reconstructed and loop filtered sample values.

[0062] The output of the loop filter unit 256 may be a sample stream that may be output to a rendering device such as the display 124 and stored in the reference picture memory 266 for future inter-picture prediction.

[0063] Once reconstructed, certain decoded pictures may be used as reference pictures for future prediction. Once a decoded picture is reconstructed, and the decoded picture has been identified as a reference picture (e.g., by parser 254), the current reference picture may become part of reference picture memory 266, and a new current picture memory may be reallocated before starting reconstruction of a subsequent decoded picture.

[0064] The decoder component 122 may perform decoding operations according to a predetermined video compression technique that may be recorded in a standard such as any of the standards described herein. As specified in a video compression technology document or standard and particularly in a configuration file therein, a decoded video sequence may conform to the syntax specified by the video compression technology or standard used in the sense that the decoded video sequence follows the syntax of the video compression technology or standard. In addition, in order to conform to some video compression technologies or standards, the complexity of the decoded video sequence may be within a range as defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstructed sample rate (measured in, for example, megasamples per second), the maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further defined by a Hypothetical Reference Decoder (HRD) specification and metadata for HRD buffer management signaled in the decoded video sequence.

[0065] Figure 3 is a block diagram illustrating a server system 112 according to some embodiments. The server system 112 includes a control circuit system 302, one or more network interfaces 304, a memory 314, a user interface 306, and one or more communication buses 312 for interconnecting these components. In some embodiments, the control circuit system 302 includes one or more processors (e.g., CPU, GPU, and / or DPU). In some embodiments, the control circuit system includes one or more field-programmable gate arrays (FPGA), hardware accelerators, and / or one or more integrated circuits (e.g., application-specific integrated circuits).

[0066] The network interface 304 can be configured to interface with one or more communication networks (e.g., wireless, wired, and / or optical networks). The communication network can be local, wide area, metropolitan area network, vehicle-mounted and industrial, real-time, delay-tolerant, etc. Examples of communication networks include: local area networks such as Ethernet; wireless LAN (Local Area Network, LAN); cellular networks including GSM (Global System for Mobile Communications, GSM), 3G (the Third Generation, 3G), 4G (the Fourth Generation, 4G), 5G (the Fifth Generation, 5G), LTE (Long Term Evolution, LTE), etc.; TV wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV; vehicle and industrial networks including CANBus, etc. Such communications can be one-way reception only (e.g., broadcast TV), one-way transmission only (e.g., CAN bus to certain CAN bus devices), or bidirectional (e.g., to other computer systems using a local area digital network or a wide area digital network). Such communications can include communications to one or more cloud computing networks.

[0067] The user interface 306 includes one or more output devices 308 and / or one or more input devices 310. The input device 310 may include one or more of the following: a keyboard, a mouse, a touch pad, a touch screen, a data glove, a joystick, a microphone, a scanner, a camera, etc. The output device 308 may include one or more of the following: an audio output device (e.g., a speaker), a visual output device (e.g., a display or monitor), etc.

[0068] The memory 314 may include high-speed random access memory (e.g., DRAM, SRAM, DDR RAM, and / or other random access solid-state memory devices) and / or non-volatile memory (e.g., one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid-state storage devices). The memory 314 optionally includes one or more storage devices remote from the control circuit system 302. The memory 314, or alternatively, the non-volatile solid-state memory device within the memory 314, includes a non-transitory computer-readable storage medium. In some embodiments, the memory 314 or the non-transitory computer-readable storage medium of the memory 314 stores the following programs, modules, instructions, and data structures, or a subset or superset thereof: Operating system 316, which includes procedures for handling various basic system services and for performing hardware-related tasks; A network communications module 318 for connecting the server system 112 to other computing devices via one or more network interfaces 304 (eg, via wired and / or wireless connections); A decoding module 320 for performing various functions related to encoding and / or decoding data, such as video data. In some embodiments, the decoding module 320 is an instance of the decoder component 114. The decoding module 320 includes, but is not limited to, one or more of the following: o a decoding module 322 for performing various functions related to decoding encoded data, such as those previously described with respect to the decoder component 122; and o an encoding module 340 for performing various functions related to encoding data, such as those previously described with respect to the encoder component 106; and Picture memory 352 for storing pictures and picture data, for example for use with decoding module 320. In some embodiments, picture memory 352 includes one or more of: reference picture memory 208, buffer memory 252, current picture memory 264, and reference picture memory 266.

[0069] In some embodiments, the decoding module 322 includes a parsing module 324 (e.g., configured to perform the various functions previously described with respect to the parser 254), a transform module 326 (e.g., configured to perform the various functions previously described with respect to the scaler / inverse transform unit 258), a prediction module 328 (e.g., configured to perform the various functions previously described with respect to the motion compensated prediction unit 260 and / or the intra-picture prediction unit 262), and a filter module 330 (e.g., configured to perform the various functions previously described with respect to the loop filter 256).

[0070] In some embodiments, the encoding module 340 includes a code module 342 (e.g., configured to perform various functions previously described with respect to the source decoder 202 and / or the decoding engine 212) and a prediction module 344 (e.g., configured to perform various functions previously described with respect to the predictor 206). In some embodiments, the decoding module 322 and / or the encoding module 340 includes Figure 3 For example, a shared prediction module is used by both the decoding module 322 and the encoding module 340.

[0071] Each of the above identified modules stored in the memory 314 corresponds to an instruction set for performing the functions described herein. The above identified modules (e.g., instruction sets) do not need to be implemented as separate software programs, processes, or modules, and therefore various subsets of these modules can be combined or otherwise rearranged in various embodiments. For example, the decoding module 320 optionally does not include a separate decoding module and encoding module, but uses the same module set to perform two groups of functions. In some embodiments, the memory 314 stores a subset of the above identified modules and data structures. In some embodiments, the memory 314 stores additional modules and data structures not described above, such as an audio processing module.

[0072] although Figure 3 The server system 112 is shown according to some embodiments, but Figure 3 The present invention is intended more as a functional description of various features that may be present in one or more server systems rather than as a block diagram of the embodiments described herein. In practice, and as recognized by one of ordinary skill in the art, items shown separately may be combined and some items may be separated. For example, Figure 3 Some of the items shown individually in the figure can be implemented on a single server, and a single item can be implemented by one or more servers. The actual number of servers used to implement the server system 112 and how the features are distributed among them will vary depending on the implementation and, optionally, will depend in part on the amount of data traffic that the server system handles during peak usage periods and during average usage periods.

[0073] Figure 4 An example scheme 400 for generating chrominance samples 402 from a plurality of luma samples 404 in CCCM mode according to some embodiments is shown. In some embodiments, each of a plurality of different decoding blocks 406 (e.g., a first decoding block 406A, a second decoding block 406B) of a current image frame 408 is decoded in a cross-component intra prediction (CCIP) mode. In CCIP mode, the decoder 122 ( Figure 2B ) determines each chrominance sample 402 of a plurality of coded blocks 406 based on one or more luma samples 404 that have been reconstructed. The CCIP mode includes the CCCM mode, in which the chrominance samples 402 are each determined based on the one or more luma samples 404 that have been reconstructed. i) combines a plurality of reconstructed luma samples 404 (e.g., samples 404C, 404N, 404S, 404W, and 404E) to generate a first chroma sample 402C. For each corresponding decoding block 406 (e.g., first decoding block 406A), the plurality of reconstructed luma samples include a first luma sample 404C of the corresponding decoding block 406 that is co-located with the first chroma sample 402C and a plurality of adjacent luma samples 404x, the adjacent luma samples 404x being located adjacent to the first luma sample 404C, for example, based on a filter shape of a filter. A corresponding bias value B is determined based on a set of one or more luma samples 404 within a reference region 412 of the corresponding decoding block 406 in the current image frame 408. The reference region 412 includes one or more decoding blocks that are adjacent to the corresponding decoding block 406A and decoded before the corresponding decoding block 406A. The first luma sample 402C, the plurality of adjacent luma samples 404x, and the corresponding bias value B are combined to generate the first chroma sample 402C. The decoder 122 reconstructs a plurality of different decoded blocks 406 including the first chroma component 402C of each corresponding decoded block 406. In some embodiments, the first decoded block 406A is different from the second decoded block 406B, and the corresponding bias values ​​B of the first decoded block 406A and the second decoded block 406B are determined based on different reference regions of the decoded blocks 406A and 406B.

[0074] The filter shape of the filter determines the number of the plurality of adjacent luma samples 404x and their positions relative to the first luma sample 404C. For example, a cross-shaped 5-tap filter with five inputs is composed of the first luma sample 404C and four adjacent luma samples 404x co-located with the chroma sample 402C to be predicted, including the north / upper adjacent sample 404N, the south / lower adjacent sample 404S, the west / left adjacent sample 404W, and the east / right adjacent sample 404E. Each of the first luma sample 404C and the adjacent luma samples 404x corresponds to a corresponding weighting factor (e.g., c0, c1, c2, c3, or c4), which is also referred to as a filter coefficient.

[0075] In some embodiments, the plurality of weighting factors includes weighting factors (e.g., c0 to c4) applied as filter coefficients in the filter and two additional weighting factors (e.g., c5, c6). According to equation 414, a weighted combination of the linear components of luma samples 404C and 404x, the nonlinear term P, and the bias term B is determined and applied to generate the first chroma sample 402C. In some embodiments, the weighting factor c0 corresponding to the first luma sample 404C is greater than the remaining weighting factors (e.g., c1 to c6) in the plurality of weighting factors. As shown below, the nonlinear term P represents the square of the first luma sample 404C scaled to the sample value range: P=(C*C+midVal)>>bitDepth (1) Where C represents the value of the first luma sample 404C, midVal represents the average of the two limits of the range of luma values ​​rounded to an integer, and bitDepth is the bit depth of the luma value. For example, the bit depth of the luma value is 10, and the nonlinear term P is represented as: P=(C*C+512)>>10 (2)

[0076] The bias term B represents a scalar offset between the input and the output, and, for example, is set to an intermediate chroma value (512 for 10-bit content). In some embodiments, the output of the filter is determined as the convolution between the filter coefficients and the input (e.g., the first luma sample 404C and the adjacent luma sample 404x), and is clipped to the range of valid chroma samples 404.

[0077] In some embodiments, the reference area 412 of each corresponding decoding block 406 includes one or more decoding blocks decoded before the corresponding decoding block 406 (eg, Figure 4 8 decoding blocks in ). In some embodiments, the subset of one or more decoding blocks is immediately adjacent to the corresponding decoding block 406. In some embodiments, the subset of one or more decoding blocks is separated from the corresponding decoding block 406 by one or more decoding blocks. In some embodiments, the reference area 412 includes at least a portion of a plurality of rows above the corresponding decoding block 406 and / or a portion of a plurality of columns to the left of the corresponding decoding block 406. For example, the reference area 412 includes at least a portion of a plurality of rows above the corresponding decoding block 406 and / or a portion of a plurality of columns to the left of the corresponding decoding block 406. Figure 4, the reference area 412 includes 7 rows of chroma samples above the corresponding decoding block 406 and 9 columns of chroma samples to the left of the corresponding decoding block 406. In some embodiments, the reference area 412 extends one decoding block width to the right of the right boundary of the corresponding decoding block 406, and extends one decoding block height to below the bottom boundary of the corresponding decoding block 406. In some embodiments, the reference area 412 is adjusted to include only available samples. The extension 412E to the reference area 412 is needed to support the side samples of the cross filter, and the extension 412E is filled in the unavailable area.

[0078] In some embodiments, for the corresponding decoding block 406, the set of one or more luma samples within the reference region 412 includes a single luma sample (e.g., 404R1) within the reference region 412 of the corresponding decoding block 406 in the current image frame 408. Alternatively, in some embodiments, for the corresponding decoding block 406, the corresponding bias value B is equal to an average value of a set of one or more luma samples including multiple luma samples (e.g., 404R1, 404R2, and 404R3) within the reference region 412 of the corresponding decoding block 406 in the current image frame 408.

[0079] In some embodiments, for a corresponding coding block 406, the set of one or more luma samples within the reference area 412 includes one or more rows 410R of luma samples above the corresponding decoding block 406 in the reference area 412, and the corresponding bias value is equal to the median luma value of the one or more rows of luma samples. Alternatively, in some embodiments, for a corresponding decoding block 406, the set of one or more luma samples within the reference area 412 includes one or more rows 410R of luma samples above the corresponding decoding block 406 in the reference area 412, and the set of one or more luma samples includes a reference luma sample 404R4 physically located substantially at the center of the one or more rows 410R of luma samples. The corresponding bias value is equal to the luma value of the reference luma sample 404R4. In some embodiments, each row 410R of luma samples includes an even number of luma samples, and the reference luma sample 404R4 corresponds to two luma samples immediately adjacent to the center. The corresponding bias value B is equal to the average of the two luma samples immediately adjacent to the center.

[0080] In some embodiments, for a corresponding decoding block 406 (e.g., block 406A), the set of one or more luma samples 404 of the reference area 412 includes one or more columns 410C of luma samples located to the left of the corresponding decoding block 406 in the reference area 412, and the corresponding bias value B is equal to the median luma value of the one or more columns 410C of luma samples. Alternatively, in some embodiments, for a corresponding decoding block 406 (e.g., block 406A), the set of one or more luma samples 402 of the reference area 412 includes one or more columns 410C of luma samples located to the left of the corresponding decoding block 406 in the reference area 412, and the set of one or more luma samples 404 includes a reference luma sample 404R5 physically located substantially at the center of the one or more columns 410C of luma samples. The corresponding bias value B is equal to the luma value of the reference luma sample 404R5. In some embodiments, the column 410C of luma samples includes an even number of luma samples, and the reference luma sample 404R5 corresponds to two luma samples that are adjacent to the center. The corresponding bias value B is equal to the average of the two luma samples that are adjacent to the center.

[0081] In some embodiments, as described above, the first luma sample 404C (C), a plurality of adjacent luma samples 404x (e.g., N, S, W, E), and a bias value B are used to generate a first chroma component 402C (preChromaVal) based on equation 414 as shown below: predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B Where c0 to c6 are weighting factors. In some embodiments, the decoder 122 reduces each of the first luma sample 404C and the plurality of adjacent luma samples 404x by an offset luma value before applying them to equation 414. In addition, in some embodiments, the decoder 122 clips each of the first luma sample 404C and the plurality of adjacent luma samples 404x before applying them to equation 414, wherein the first luma sample 404C and the plurality of adjacent luma samples 404x have been reduced by the offset luma value. In addition, in some embodiments, the offset luma value is selected from an average value of the first luma sample 404C and the plurality of adjacent luma samples 404x, a top reference area 402T ( Figure 5A ), the left reference area 402L ( Figure 5A), the luma value of a single luma sample 404O located at the upper left corner of the corresponding decoding block 406, the median luma value of one or more rows 410R of luma samples located above the corresponding decoding block 406 within the reference area 412, the luma value of a luma sample 404R4 physically located substantially at the center of the one or more rows 410R of luma samples, the median luma value of one or more columns 410C of luma samples located to the left of the corresponding decoding block 406 within the reference area 412, and the luma value of a luma sample 404R5 physically located substantially at the center of the one or more columns 410C of luma samples.

[0082] In some embodiments, the CCIP mode includes a multi-chroma (multi-CfL) mode according to luma. The decoder 122 determines to encode a plurality of different decoding blocks 405 in the multi-CfL mode. According to the determination to encode a corresponding decoding block 406 in the multi-CfL mode, a plurality of adjacent luma samples 404x of a first luma sample 404C of the corresponding decoding block are identified based on a filter shape of a filter, and a first chroma component 402C of the corresponding decoding block 406 is a weighted sum of the first luma sample 404C, the plurality of adjacent luma samples 404x, the output of a nonlinear function of the first luma sample 404C based on equation 414, and a corresponding bias value B. In some embodiments, the first luma sample 404C, the plurality of adjacent luma samples 404x, and the corresponding bias value B are combined using a plurality of weighting factors to generate the first chroma component 402C, and each weighting factor includes an integer part and a fractional part.

[0083] In some embodiments, the filter shape of the filter is a cross shape and has 5 taps. According to the filter shape, the plurality of adjacent luma samples 404x include four adjacent luma samples 404N, 404S, 404W, and 404E arranged in a cross shape with the first luma sample 404C as the center. In addition, in some embodiments, the plurality of weighting factors of the filter include 7 weighting factors corresponding to the first luma sample 404C, the four adjacent luma samples 404x, the output of the nonlinear function of the first luma sample 404C (e.g., P), and the bias component (e.g., B), respectively.

[0084] In some embodiments, when chroma subsampling is used, the reconstructed luma samples 404 are downsampled to match the lower resolution chroma grid. In some embodiments, a CCCM mode is implemented using a single model or a multi-model variant. The single model uses a single equation 414 including a single set of weighting factors for determining the chroma samples 402 of the corresponding decoding block 406. Each of the single set of weighting factors is received using a video bitstream, or each of the single set of weighting factors is derived from the reconstructed samples of the reference area 412. Conversely, in some embodiments, the multi-model variant uses two models corresponding to two sets of weighting factors. One model is derived based on samples above the average luma reference value of the reference area 412, and the other model is derived based on the remaining luma samples of the reference area 412. In addition, in some embodiments, such a multi-model CCCM mode is used to reconstruct a prediction unit (e.g., a corresponding decoding block 406) with at least 128 available reference samples.

[0085] In some embodiments, a plurality of weighting factors (e.g., c0 to c6 in equation 414) are determined by minimizing a mean square error (MSE) between predicted chroma samples 402 and reconstructed chroma samples 402 in a reference region 412. MSE minimization is performed by computing an autocorrelation matrix of luma samples 404 in the reference region 412 and a cross-correlation vector between luma samples 404 and chroma samples 402 in the reference region 412. The autocorrelation matrix is ​​processed using LDL decomposition, and a plurality of weighting factors are calculated using back substitution. The processing generally follows the calculation of filter coefficients of an adaptive loop filter (ALF) in enhanced compression model (ECM) video coding. The LDL decomposition does not use square root operations, only integer arithmetic operations.

[0086] In some embodiments, the use of CCCM mode is signaled in the video bitstream 116 using a flag decoded with context-adaptive binary arithmetic coding (CABAC) at the prediction unit level. A new CABAC context is included to support CCCM mode. In some cases, CCCM is considered a sub-mode of CCLM. If the intra prediction mode is LM_CHROMA_IDX (single-mode CCCM enabled) or MMLM_CHROMA_IDX (multi-model CCCM enabled), the CCCM flag is signaled in the video bitstream 116.

[0087] In some embodiments, the encoder 106 encodes each corresponding decoding block 406 of the plurality of different decoding blocks of the current image frame 408 into a video bitstream. The encoder 106 identifies the first chroma component 402C of the corresponding decoding block 406 and the first luma sample 404C co-located with the first chroma component 402C in the corresponding decoding block 406, and a plurality of adjacent luma samples 404x of the first luma sample 402C, for example, based on the filter shape of the filter. The corresponding offset value B is derived based on a set of one or more luma samples 404 within a reference area 412 of the corresponding decoding block 406 in the current image frame 408. The reference area 412 includes one or more decoding blocks 406 that are adjacent to the corresponding decoding block 406 and will be decoded before the corresponding decoding block 406. The corresponding offset value is optionally sent to the decoder 122 along with the video bitstream. The encoder 106 generates a video bitstream including the plurality of different decoding blocks of the current image frame. The video bitstream signals a cross-component intra prediction (CCIP) mode indicating that each chrominance component of a plurality of different coding blocks is determined based on one or more luma samples.

[0088] Figure 5A 4 is a diagram of an example reference region 412 including multiple reference regions of a decoded block 406 in a current image frame 408 according to some embodiments. In some embodiments, the corresponding decoded block 406 of the current image frame 408 includes a plurality of weighting factors (e.g., Figure 4 The decoder 122 ( FIG. 2 ) derives a corresponding bias value B of the corresponding decoded block 406 based on a set of one or more luma samples 404 within a reference area 412 of the corresponding decoded block 406 in the current image frame 408. The reference area 412 includes one or more decoded blocks that are adjacent to the corresponding decoded block 406 and decoded before the corresponding decoded block 406. The first luma sample 402C, the plurality of adjacent luma samples 404x, and the corresponding bias value B are combined to generate the first chroma sample 402C. The decoder 122 reconstructs the corresponding decoded block 406 including the first chroma sample 402C.

[0089] In some embodiments, one or more decoding blocks 412T and 412L of the reference area 412 are immediately adjacent to the corresponding decoding block 406. In some embodiments not shown, one or more decoding blocks are separated from the corresponding decoding block 406 by the one or more decoding blocks.

[0090] In some implementations, the corresponding decoding block 406 (e.g., Figure 4 5, in this example, the reference area 412 includes the top reference area 412T and the left reference area 412L. Each reference area includes one or more decoding blocks. In other words, in some embodiments, the reference area 412 includes at least a portion of a plurality of rows above the corresponding decoding block 406 and / or a portion of a plurality of columns to the left of the corresponding decoding block 406. For example, referring to FIG. 5, the reference area 412 includes a first portion of 7 rows of chroma samples above the corresponding decoding block 406 and a second portion of 9 columns of chroma samples to the left of the corresponding decoding block 406. The first portion is determined by the length of the corresponding decoding block 406, and the second portion is determined by the width of the corresponding decoding block 406. In some embodiments, the reference area 412 extends one decoding block width to the right of the right border of the corresponding decoding block 406, and extends one decoding block height to below the bottom border of the corresponding decoding block 406. In some embodiments, the reference area 412 is adjusted to include only available samples. The extension 412E to the reference area 412 is required to support the side samples of the cross-shaped spatial filter, and the extension 412E is filled in the unavailable area.

[0091] Figure 5B 4 is a diagram of another example reference region 412 that is adjusted to include only available samples of a decoding block 406 in a current image frame 408 according to some embodiments. In some embodiments, the reference region 412 is adjusted to include only available samples. In other words, for each corresponding decoding block 406, the decoder 122 (FIG. 2) identifies the reference region 412 of the corresponding decoding block 406. Based on determining that one or more color components are not available in a set of columns or rows adjacent to a boundary, the decoder 122 excludes the set of columns or rows from the reference region 412. The boundary includes one of a slice boundary, a super block boundary, a tile boundary, a decoding block boundary, and an image frame boundary. For example, the current image frame 408 has a rightmost column 502 of incomplete luma samples. Based on determining that the rightmost column 502 of luma samples is incomplete, the decoder 122 excludes the rightmost column 502 of luma samples so that it is not applied to determine the corresponding bias value of the corresponding decoding block 406.

[0092] 6A to 6F6 is a block diagram of six example filter shapes 600A to 600F of the filter 600 applied to combine luma samples 404 in CCCM mode according to some embodiments. Each filter shape 600A to 600F is optionally applied to a sequence of image frames including a current image frame 408, a current image frame 408, a slice of a current image frame 408, a super block including a corresponding decoding block 406, or a corresponding decoding block 406. Fig. 6A , the filter shape 600A corresponds to an odd number of horizontal luma samples (eg, 3 horizontal luma samples 404W, 404C, and 404E) centered around the first luma sample 404C (P0). Referring to Figure 6B , the filter shape 600B corresponds to an odd number of vertical luma samples (eg, 3 vertical luma samples 404N, 404C, and 404S) centered around the first luma sample 404C (P0). Figure 6C , the filter shape 600C corresponds to an array of 3×3 luma samples (P0 to P8) centered around the first luma sample 404C (P0). Fig.6D and Fig. 6E , filter shape 600C corresponds to a subset of the array of 3×3 luma samples (P0 to P6) centered around the first luma sample 404C and missing two angular luma samples (eg, Fig.6D The upper left brightness sample and the lower right brightness sample in Fig.6D The lower left and upper right luminance samples in ). Fig. 6F , the filter shape is a cross and corresponds to an odd number of horizontal luma samples (e.g., 5 horizontal luma samples) centered around the first luma sample 404C and an odd number of vertical luma samples (e.g., 5 vertical luma samples) centered around the same first luma sample 404C.

[0093] In some embodiments, the video bitstream received by the decoder 122 includes a respective filter index for each respective decoding block 406 of a plurality of different decoding blocks. For each respective decoding block 406 (e.g., Figure 4406A in the decoder 122), the decoder 122 adaptively selects one of the multiple different filters based on the corresponding filter index. Each different filter has a respective filter shape and multiple weighting factors (also referred to as filter coefficients). The first luma sample 404C, multiple adjacent luma samples 404x, and the corresponding bias value B are combined using at least the multiple weighting factors of the selected one of the multiple different filters. In addition, in some embodiments, the multiple different filters include one or more of the following: (1) a first filter 600C, which includes nine weighting factors and is configured to generate a first chroma component 402C as a weighted combination of at least the first luma sample 404C (P0) and eight adjacent luma samples 404x (P1 to P8) directly surrounding the first luma sample 404C (P0), (2) a second filter 600D or 600E, which includes seven weighting factors and is configured to generate a first chroma component 402C as a weighted combination of at least the first luma sample 404C (P0) and eight adjacent luma samples 404x (P1 to P8) directly surrounding the first luma sample 404C (P0). The first chrominance component 402C is a weighted combination of the luma sample 404C (P0) and six adjacent luma samples 404x (P1 to P6) symmetrically positioned relative to the first luma sample 404C (P0), and (3) a cross filter 600F, which includes 4k+1 weighting factors and is configured to generate the first chrominance component 402C as a weighted combination of at least the first luma sample 404C and k adjacent luma samples 404x from each of two row directions and two column directions immediately adjacent to the first luma sample 404C, where k is a positive integer (e.g., 2). In addition, in some embodiments, the first chrominance component 402C is also generated based on at least one of the following: a fixed bias value, a weighted and adjustable bias value, and a second order or higher order of at least one of the first luma sample 404C and the plurality of adjacent luma samples 404x.

[0094] Reference Fig. 6F In some embodiments, the decoder 122 applies a cross filter 600F including 4k+1 weighting factors to combine the first luma sample 404C, a plurality of adjacent luma samples 404x, and the bias value B. Each of the two row directions and the two column directions corresponds to k adjacent luma samples immediately adjacent to the first luma component. In addition, in some embodiments, the k adjacent luma samples 404x in each of the two row directions are combined in second order into the first chroma component 402C, and the k adjacent luma samples 404x in each of the two column directions are linearly combined into the first chroma component 402C. Alternatively, in some embodiments, the k adjacent luma samples 404x in each of the two column directions are combined in second order into the first chroma component 402C, and the k adjacent luma samples 404x in each of the two row directions are linearly combined into the first chroma component 402C.

[0095] Figure 7 is a flowchart illustrating a method 700 for decoding a video according to some embodiments. The method 700 may be performed at a computing system (e.g., a server system 112, a source device 102, or an electronic device 120) having a control circuit system and a memory storing instructions for execution by the control circuit system. In some embodiments, the method 700 is applied in conjunction with one or more video codecs, including but not limited to AV1, AV2, HEVC, VVC, and ECM. In some embodiments, the method 700 is performed by executing instructions stored in a memory of the computing system (e.g., a decoding module 320 of the memory 314). In some embodiments, the current image frame 408 includes a corresponding decoding block 406. In some embodiments, the method 700 is applied to use one color component to predict another color component, and one or more color components need to be downsampled. In addition, in some embodiments, the method 700 is applied to use a red color component to predict a green or blue color component.

[0096] In some embodiments, the decoder 122 receives (702) a video bitstream including a plurality of different decoded blocks of a current image frame, and applies (710) a multi-CfL mode, wherein the bias value B is determined based on a set of one or more luma samples within the reference region 412. For example, the bias value is an average of the one or more luma samples within the reference region 412 and is recorded as AvgLuma. In some embodiments, based on the following equation 414 ( Figure 4 ) to represent (714) the chrominance value of the corresponding decoded block 406: predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6AvgLuma (3)

[0097] In some embodiments, the bias value B is set to the value of a predefined adjacent sample in the reference area 412 of the corresponding decoded block 406 in the current image frame 408. For example, the bias value B is set to the luma value of one of the upper left luma sample 404R2 and the lower right luma sample 404R3, or the chroma value of the chroma sample co-located with the luma samples 404R2 and 404R3.

[0098] In some embodiments, the offset value B is an average value of the adjacent reconstructed samples above the reference area 412 of the corresponding decoding block 406. In some embodiments, the offset value B is an average value of the adjacent reconstructed samples to the left of the reference area 412 of the corresponding decoding block 406.

[0099] In some embodiments, the offset value B may be a middle / median pixel in one or more upper rows in the reference area 412 of the corresponding decoding block 406. In some embodiments, the offset value B may be a middle / median pixel in one or more left columns in the reference area 412 of the corresponding decoding block 406.

[0100] In some embodiments, the first luma sample 404C and a plurality of adjacent luma samples 404x (e.g., N, S, C, W, E) are offset by an offset value to reduce the value range. In some embodiments, the offset value is the average value of the luma values ​​of the corresponding decoding block 406. In some embodiments, the offset value is the average value of the adjacent reconstructed samples above and / or to the left of the reference area 412 of the corresponding decoding block 406. In some embodiments, the offset value is the upper left pixel value of the luma and chroma of the corresponding decoding block 406. In some embodiments, the offset value is the middle / median pixel in one or more upper rows in the reference area 412 of the corresponding decoding block 406. In some embodiments, the offset value is the middle / median pixel in one or more left columns in the reference area 412 of the corresponding decoding block 406. In some embodiments, after the luma samples 404C and 404x (e.g., N, S, C, W, E) are subtracted by the offset value, the offset luma sample values ​​404C and 404x are also clipped to a predefined range.

[0101] In some embodiments, multiple CfL is applied during boundary processing, and instead of filling in unavailable samples, only available samples are involved in the multiple CfL calculation process. Figure 5B , one less column 502 is used on the right boundary. The boundary is one of a slice boundary, a CTU boundary, a super block boundary, a tile boundary, and a picture boundary.

[0102] In some embodiments, one or more left samples immediately adjacent to the left border are unavailable, and the left border column is reduced (e.g., moved one or more columns to the right). In some embodiments, one or more top samples immediately adjacent to the top border are unavailable, and the top border row is reduced (e.g., moved one or more rows down). In some embodiments, one or more right samples immediately adjacent to the right border are unavailable, and the right border column is reduced (e.g., moved one or more columns to the left). In some embodiments, one or more bottom samples immediately adjacent to the bottom border are unavailable, and the bottom border line is reduced (e.g., moved one or more rows up).

[0103] In some embodiments, the filter applied to predict the first chrominance sample 402C based on the first luma sample has a filter shape and a plurality of filter coefficients. The filter shape and the number of filter coefficients are optionally predefined or adaptively selected. In addition, in some embodiments, the filter is adaptively selected from a plurality of filters. The index of the selected filter is signaled to the video bitstream. In some embodiments, unavailable samples near the border are filled according to the picture size. In the example, reference Figure 6C , the predefined filter includes a center position P0 and eight surrounding positions P1 to P8. The first chroma sample 402C is predicted as follows: predChrome=c0P0+c1P1+c2P2+c3P3+c4P4+c5P5+c6P6 (4) Using at least 8 luma chroma samples, coefficients may be calculated at both the encoder 106 and the decoder 122. Additional constant bias terms, bias terms equal to the luma average, or higher order terms may optionally be applied to determine the first chroma sample 402.

[0104] In some implementations, unavailable samples near the border are filled according to the image size. Fig.6D or Fig. 6E , the predefined filter includes a center position P0 and six surrounding positions P1 to P6. The first chroma sample 402C is predicted as follows: predChrome=c0P0+c1P1+c2P2+c3P3+c4P4+c5P5+c6P6 (5) Using at least 6 luma chroma samples, the coefficients are calculated at both the encoder 106 and the decoder 122. Optionally, an additional constant bias term, a bias term equal to the luma average, or a higher order term is applied to determine the first chroma sample 402.

[0105] In some implementations, unavailable samples near the border are filled according to the image size. Fig. 6F , the predefined filter is a cross filter and includes a center position P0 and eight surrounding positions P1 to P8 arranged in a cross shape. The first chroma sample 402C is predicted as follows: predChrome=c0P0+c1P1+c2P2+c3P3+c4P4+c5P5+c6P6+c7P7+c8P8 (6) Using at least 8 luma chroma samples, the coefficients may be calculated at both the encoder 106 and the decoder 122. Optionally, an additional constant bias term, a bias term equal to the luma average, or a higher order term may be applied to determine the first chroma sample 402.

[0106] In some embodiments, one of the plurality of filters is adaptively selected to generate the first chrominance sample 402C based on the first luma sample 404C and the associated adjacent luma sample 404x.

[0107] In some embodiments, multiple higher order terms are used during chroma sample prediction to increase the weight of a subset of sample positions. For example, luma samples located at horizontal positions (including the center position) are associated with the second order, while luma samples associated with vertical positions use linear terms. A first luma sample 404C (C), a plurality of adjacent luma samples 404x (e.g., N, S, W, E), and a bias value B are combined to generate a first chroma component 402C (preChromaVal) based on equation 414 as shown below: predChromaVal=c0C*C+c1N+c2S+c3E*E+c4W*W+c5B (7)

[0108] In another example, the vertical position (including the center position) uses the second order, while the horizontal position uses a linear term. The first luma sample 404C (C), a plurality of adjacent luma samples 404x (e.g., N, S, W, E), and the bias value B are combined to generate the first chroma component 402C (preChromaVal) based on equation 414 as shown below: predChromaVal=c0C*C+c1N*N+c2S*S+c3E+c4W+c5B (8)

[0109] In some embodiments, the first chroma sample 402C is fully connected to all corresponding luma samples 404 (e.g., in the corresponding decoding block 406), and the downsampling operation is performed after the fully connected layer. The neural network coefficients of the fully connected layer are pre-trained and stored at both the encoder side and the decoder side.

[0110] although Figure 7 Some logical stages are shown in a particular order, but stages that are not order-dependent may be reordered, and other stages may be combined or split. Some reordering or other groupings not specifically mentioned are obvious to one of ordinary skill in the art, so the ordering and groupings presented herein are not exhaustive. In addition, it should be appreciated that the stages may be implemented in hardware, firmware, software, or any combination thereof.

[0111] We now turn to some example implementations.

[0112] (A1) In some implementations, a method 700 for decoding video data is implemented. The method 700 includes receiving (702) a video bitstream including a plurality of different decoded blocks of a current image frame. The video bitstream includes (704) a syntax element for a cross-component intra prediction (CCIP) mode, the syntax element indicating whether each chrominance component of the plurality of different decoded blocks is determined based on one or more luma samples. The method (700) also includes deriving (710) a corresponding bias value of the corresponding decoded block based on a set of one or more luma samples within a reference area of ​​the corresponding decoded block in the current image frame. The reference area includes (712) one or more decoded blocks adjacent to the corresponding decoded block and decoded before the corresponding decoded block. The method also includes reconstructing (716) a plurality of decoded blocks including the corresponding decoded block based at least on the derived bias value.

[0113] (A2) In some embodiments of A1, for a corresponding decoding block, the set of one or more luma samples includes a single luma sample within a reference region of the corresponding decoding block in the current image frame.

[0114] (A3) In some embodiments of A1, for a corresponding decoding block, a corresponding offset value is equal to an average value of a set of one or more luminance samples of a plurality of luminance samples within a reference area including the corresponding decoding block in the current image frame.

[0115] (A4) In an embodiment of any one of items A1 to A3, for a corresponding decoding block, a set of one or more luma samples includes one or more luma sample rows above the corresponding decoding block in a reference area, and the corresponding bias value is equal to a median luma value of the one or more luma sample rows.

[0116] (A5) In an embodiment of any one of items A1 to A3, for a corresponding decoding block, a set of one or more luma samples includes one or more luma sample rows above the corresponding decoding block within a reference area, and a set of one or more luma samples includes a reference luma sample physically located substantially at the center of the one or more luma sample rows, and wherein the corresponding bias value is equal to the luma value of the reference luma sample.

[0117] (A6) In an embodiment of any one of items A1 to A3, for a corresponding decoding block, a set of one or more luma samples includes one or more luma sample columns located to the left of the corresponding decoding block in a reference area, and the corresponding bias value is equal to a median luma value of the one or more luma sample columns.

[0118] (A7) In an embodiment of any one of A1 to A3, for a corresponding coding block, the set of one or more luma samples includes one or more luma sample columns located to the left of the corresponding coding block within a reference region, and the set of one or more luma samples includes a reference luma sample physically located substantially at the center of the one or more luma sample columns. The corresponding offset value is equal to the luma value of the reference luma sample.

[0119] (A8) In an embodiment of any one of items A1 to A7, method 700 also includes identifying (706) a first chrominance component of a corresponding decoding block and a first luminance component co-located with the first chrominance component in the corresponding decoding block; identifying (708) multiple adjacent luminance components of the first luminance component; and combining (714) the first luminance component, the multiple adjacent luminance components, and the corresponding bias value to generate a first chrominance component.

[0120] (A9) In some embodiments of A8, combining the first luma sample, a plurality of adjacent luma samples, and corresponding bias values ​​to generate a first chroma component further comprises reducing the first luma sample and each of the plurality of adjacent luma samples using the offset luma value. In some embodiments of A8, combining the first luma sample, a plurality of adjacent luma samples, and corresponding bias values ​​to generate a first chroma component further comprises clipping the first luma sample and each of the plurality of adjacent luma samples that have been reduced using the offset luma value.

[0121] (A10) In some embodiments of A8, the offset luminance value is one of: an average value of a first luminance sample and multiple adjacent luminance samples, an average value of luminance samples in a top reference area of ​​a reference area located above a corresponding decoding block, an average value of luminance samples in a left reference area of ​​a reference area located to the left of a corresponding decoding block, a luminance value of a single luminance sample located at the upper left corner of the corresponding decoding block, a median luminance value of one or more luminance sample rows above the corresponding decoding block within the reference area, a luminance value of a luminance sample physically located substantially at the center of one or more luminance sample rows, a median luminance value of one or more luminance sample columns located to the left of the corresponding decoding block within the reference area, and a luminance value of a luminance sample physically located substantially at the center of one or more luminance sample columns.

[0122] (A11) In some embodiments of A8, the video bitstream includes corresponding filter indices for corresponding decoding blocks of the plurality of different decoding blocks. Method 700 also includes, for the corresponding decoding block, adaptively selecting one of a plurality of different filters based on the corresponding filter index, each different filter having a corresponding filter shape and a plurality of weighting factors. The first luma sample, the plurality of adjacent luma samples, and the corresponding offset value are combined using at least the plurality of weighting factors of the selected one of the plurality of different filters.

[0123] (A12) In some embodiments of A11, the multiple different filters include one or more of the following: a first filter, which includes nine weighting factors and is configured to generate a first chrominance component as a weighted combination of at least a first luma sample and eight adjacent luma samples directly surrounding the first luma sample; a second filter, which includes seven weighting factors and is configured to generate a first chrominance component as a weighted combination of at least the first luma sample and six adjacent luma samples directly surrounding the first luma sample and symmetrically positioned relative to the first luma sample; and a cross filter, which includes 4k+1 weighting factors and is configured to generate a first chrominance component as a weighted combination of at least the first luma sample and k adjacent luma samples from each of two row directions and two column directions immediately adjacent to the first luma sample, where k is a positive integer.

[0124] (A13) In some embodiments of A12, the first chrominance component is also generated based on at least one of: a weighted and adjustable bias value, and a second order or higher order of at least one of the first luma sample and a plurality of adjacent luma samples.

[0125] (A14) In an embodiment of any one of A1 to A13, method 700 further includes: for a corresponding decoding block, identifying a reference area of ​​the corresponding decoding block, including excluding a set of columns or rows from the reference area based on determining that one or more color components are unavailable in a set of columns or rows adjacent to a boundary.

[0126] (A15) In an embodiment of A14, the boundary includes one of a slice boundary, a super block boundary, a tile boundary, a decoding block boundary and an image frame boundary.

[0127] (A16) In an embodiment of any one of A1 to A15, method 700 further includes identifying a first chrominance component of a corresponding decoding block and a first luminance component co-located with the first chrominance component in the corresponding decoding block; identifying multiple adjacent luminance components of the first luminance component; and combining the first luminance component, the multiple adjacent luminance components, and the corresponding bias value to generate a first chrominance component. Generating the first chrominance component further includes: applying a cross filter including 4k+1 weighting factors to combine a first luminance sample, multiple adjacent luminance samples, and the corresponding bias value. Each of the two row directions and the two column directions corresponds to k adjacent luminance samples immediately adjacent to the first luminance component. The k adjacent luminance samples in each of the two row directions are combined into the first chrominance component in the second order, and the k adjacent luminance samples in each of the two column directions are linearly combined into the first chrominance component.

[0128] (A17) In an embodiment of any one of A1 to A15, method 700 further includes identifying a first chrominance component of a corresponding decoding block and a first luminance component co-located with the first chrominance component in the corresponding decoding block; identifying multiple adjacent luminance components of the first luminance component; and combining the first luminance component, the multiple adjacent luminance components, and the corresponding bias value to generate a first chrominance component. Generating the first chrominance component further includes: applying a cross filter including 4k+1 weighting factors to combine a first luminance sample, multiple adjacent luminance samples, and the corresponding bias value. Each of the two row directions and the two column directions corresponds to k adjacent luminance samples immediately adjacent to the first luminance component. The k adjacent luminance samples in each of the two column directions are combined into the first chrominance component in the second order, and the k adjacent luminance samples in each of the two row directions are linearly combined into the first chrominance component.

[0129] (A18) In an embodiment of any one of A1 to A17, for a corresponding decoding block, a reference area of ​​the corresponding decoding block includes one or more of the following: an upper left reference area, a top reference area, an upper right reference area, a lower left reference area, and a left reference area.

[0130] (A19) In some embodiments of any one of A1 to A18, the CCIP mode includes a multi-chrominance (multi-CfL) mode based on luma. The method 700 also includes determining to encode a plurality of different decoding blocks in the multi-CfL mode. Based on determining to encode a corresponding decoding block in the multi-CfL mode, a plurality of adjacent luma samples of a first luma sample of the corresponding decoding block are identified based on a filter shape of a filter, and a first chroma component of the corresponding decoding block is a weighted sum of the first luma sample, a plurality of adjacent luma samples, an output of a nonlinear function of the first luma sample, and a corresponding bias value.

[0131] (A20) In some embodiments of any one of A1 to A19, a first luma sample, a plurality of adjacent luma samples, and a corresponding bias value are combined using a plurality of weighting factors to generate a first chroma component, and each weighting factor includes an integer part and a fractional part.

[0132] (A21) In some implementations, a method for encoding video data is implemented. The method includes encoding each corresponding decoding block of a plurality of different decoding blocks of a current image frame by identifying a first chrominance component of the corresponding decoding block and a first luma sample co-located with the first chrominance component in the corresponding decoding block, identifying a plurality of adjacent luma samples of the first luma sample, and deriving a corresponding bias value based on a set of one or more luma samples in a reference area of ​​the corresponding decoding block in the current image frame. The reference area includes one or more decoding blocks adjacent to the corresponding decoding block and to be decoded before the corresponding decoding block. The method also includes generating a video bitstream including the plurality of different decoding blocks of the current image frame. The video bitstream signals a cross-component intra prediction (CCIP) mode indicating that each chrominance component of the plurality of different decoding blocks is determined based on one or more luma samples.

[0133] On the other hand, some embodiments include a computing system (e.g., server system 112) that includes a control circuit system (e.g., control circuit system 302) and a memory (e.g., memory 314) coupled to the control circuit system, the memory storing one or more instruction sets configured to be executed by the control circuit system, the one or more instruction sets including instructions for performing any of the methods described herein (e.g., A1 to A21 above).

[0134] On the other hand, some embodiments include a non-transitory computer-readable storage medium storing one or more instruction sets for execution by a control circuit system of a computing system, the one or more instruction sets including instructions for performing any of the methods described herein (e.g., A1 to A21 above).

[0135] The proposed methods may be used alone or in combination in any order. In addition, each of the methods (or implementations), encoders, and decoders may be implemented by a processing circuit system (e.g., one or more processors or one or more integrated circuits). For example, one or more processors execute a program stored in a non-transitory computer-readable medium. In the following, the term block may be interpreted as a prediction block, a decoding block, or a decoding unit, i.e., a CU.

[0136] It should be understood that although the terms "first", "second", etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element.

[0137] The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present claims. As used in the description of the embodiments and the appended claims, the singular forms "a", "an" and "the" are also intended to include plural forms unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more associated listed items. It will also be understood that when the terms "include" and / or "comprising" are used in this specification, it indicates the presence of the features, integers, steps, operations, elements and / or parts, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or their groups.

[0138] As used herein, the term “if” may be interpreted as “when the precondition is true” or “after the precondition is true” or “in response to determining that the precondition is true” or “in accordance with determining that the precondition is true” or “in response to detecting that the precondition is true”, depending on the context. Similarly, the phrase “if it is determined that [the precondition is true]” or “if [the precondition is true]” or “when [the precondition is true]” may be interpreted as “after determining that the precondition is true” or “in response to determining that the precondition is true” or “in accordance with determining that the precondition is true” “after detecting that the precondition is true” or “in response to detecting that the precondition is true”, depending on the context.

[0139] For purposes of illustration, the foregoing description has been described with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present claims to the precise forms disclosed. In light of the above teachings, many modifications and variations are possible. The embodiments are chosen and described in order to best illustrate the principles of operation and practical application, thereby enabling others skilled in the art to implement.

Claims

1. A method for decoding video data, comprising: Receiving a video bitstream including a plurality of coding blocks of a current image frame, wherein the video bitstream includes a syntax element for a cross-component intra prediction (CCIP) mode, the syntax element indicating whether each chrominance component of the plurality of different coding blocks is determined based on one or more luma components; deriving a corresponding bias value based on a set of one or more luminance components within a reference area of ​​a corresponding decoding block in the current image frame, wherein the reference area includes one or more decoding blocks adjacent to the corresponding decoding block and decoded before the corresponding decoding block; and A plurality of decoding blocks including the corresponding decoding block are reconstructed based at least on the derived offset value.

2. The method according to claim 1, wherein: For the corresponding coding block, the set of one or more luminance components includes a single luminance component within the reference region of the corresponding coding block in the current image frame.

3. The method according to claim 1, wherein: For the corresponding decoding block, the corresponding offset value is equal to an average value of the set of the one or more luminance components of the plurality of luminance components within the reference area including the corresponding decoding block in the current image frame.

4. The method according to claim 1, wherein: For the corresponding coding block, the set of one or more luma components includes one or more luma component rows in the reference area above the corresponding coding block, and the corresponding offset value is equal to a median luma value of the one or more luma component rows.

5. The method according to claim 1, wherein: For the corresponding decoding block, the set of one or more luminance components includes one or more luminance component rows above the corresponding decoding block within the reference area, and the set of one or more luminance components includes a reference luminance component that is physically substantially located at the center of the one or more luminance component rows, and wherein the corresponding bias value is equal to the luminance value of the reference luminance component.

6. The method according to claim 1, wherein: For the corresponding coding block, the set of one or more luma components includes one or more luma component columns located to the left of the corresponding coding block in the reference area, and the corresponding offset value is equal to a median luma value of the one or more luma component columns.

7. The method according to claim 1, wherein: For the corresponding decoding block, the set of one or more luminance components includes one or more luminance component columns located to the left of the corresponding decoding block within the reference area, and the set of one or more luminance components includes a reference luminance component physically located substantially at the center of the one or more luminance component columns, and wherein the corresponding bias value is equal to the luminance value of the reference luminance component.

8. The method according to claim 1, further comprising: Identify a first chrominance component of the corresponding decoding block and a first luminance component co-located with the first chrominance component in the corresponding decoding block; identifying a plurality of adjacent luminance components of the first luminance component; as well as The first luma component, the plurality of adjacent luma components, and the corresponding offset values ​​are combined to generate the first chroma component.

9. The method of claim 8, combining the first luma component, the plurality of adjacent luma components, and the corresponding offset values ​​to generate the first chroma component further comprising: The first luminance component and each of the plurality of adjacent luminance components are reduced using an offset luminance value.

10. The method of claim 9, combining the first luma component, the plurality of adjacent luma components, and the corresponding offset values ​​to generate the first chroma component further comprising: The first luminance component and each of the plurality of adjacent luminance components that have been reduced using the offset luminance value are clipped.

11. The method according to claim 9, wherein: The offset brightness value is one of the following: an average value of the first brightness component and the plurality of adjacent brightness components; an average value of luminance components in a top reference area of ​​the reference area located above the corresponding decoding block; an average value of a luminance component in a left reference area of ​​the reference area located on the left side of the corresponding decoding block; a luminance value of a single luminance component located at the upper left corner of the corresponding decoding block; a median luma value of one or more luma component rows above the corresponding decoding block within the reference region; a luma value of a luma component physically located substantially at a center of the one or more luma component rows; a median luma value of one or more luma component columns to the left of the corresponding decoding block within the reference region; as well as A luma value of a luma component physically located substantially at a center of the one or more luma component columns.

12. A computing system comprising: Control circuit system; as well as A memory storing one or more programs configured to be executed by the control circuit system, the one or more programs further comprising instructions for: Receiving a video bitstream including a plurality of different coding blocks of a current image frame, wherein the video bitstream includes a syntax element for a cross-component intra prediction (CCIP) mode, the syntax element indicating that each chrominance component of the plurality of different coding blocks is determined based on one or more luma components; deriving a corresponding bias value based on a set of one or more luminance components within a reference area of ​​a corresponding decoding block in the current image frame, wherein the reference area includes one or more decoding blocks adjacent to the corresponding decoding block and decoded before the corresponding decoding block; and A plurality of decoding blocks including the corresponding decoding block are reconstructed based at least on the derived offset value.

13. The computing system of claim 12, wherein the one or more programs further comprise instructions for: For the corresponding decoding block, identifying the reference region of the corresponding decoding block includes excluding a set of columns or rows from the reference region based on determining that one or more color components are not available in the set of columns or rows adjacent to a boundary.

14. The computing system of claim 13, wherein: The boundary includes one of a slice boundary, a super block boundary, a tile boundary, a decoding block boundary and an image frame boundary.

15. The computing system of claim 12, wherein the one or more programs further comprise instructions for: Identify a first chrominance component of the corresponding decoding block and a first luminance component co-located with the first chrominance component in the corresponding decoding block; identifying a plurality of adjacent luminance components of the first luminance component; as well as The first luma component, the plurality of adjacent luma components, and the corresponding offset values ​​are combined to generate the first chroma component.

16. The computing system of claim 15, wherein: The video bitstream includes a corresponding filter index for the corresponding decoding block of the plurality of different decoding blocks, and the one or more programs further include instructions for performing the following operations for the corresponding decoding block: Adaptively selecting one of a plurality of different filters based on the corresponding filter index, each different filter having a corresponding filter shape and a plurality of weighting factors, wherein the first luminance component, the plurality of adjacent luminance components, and the corresponding bias value are combined using at least a plurality of weighting factors of the selected one of the plurality of different filters.

17. The computing system of claim 16, wherein: The plurality of different filters include one or more of the following: a first filter comprising nine weighting factors and configured to generate the first chrominance component as a weighted combination of at least the first luma component and eight adjacent luma components immediately surrounding the first luma component; a second filter comprising seven weighting factors and configured to generate the first chrominance component as a weighted combination of at least the first luma component and six adjacent luma components immediately surrounding the first luma component and positioned symmetrically with respect to the first luma component; as well as A cross filter comprising 4k+1 weighting factors and configured to generate the first chrominance component as a weighted combination of at least the first luminance component and k adjacent luminance components from each of two row directions and two column directions immediately adjacent to the first luminance component, wherein k is a positive integer.

18. The computing system of claim 17, wherein: The first chroma component is also generated based on at least one of: a weighted and adjustable bias value and a second order or higher order of the first luma component and at least one luma component of the plurality of adjacent luma components.

19. A non-transitory computer-readable storage medium storing one or more programs for execution by control circuitry of a computing system, the one or more programs comprising instructions for: A video bitstream including a plurality of different decoded blocks of a current image frame is received, wherein: The video bitstream includes a syntax element for a cross-component intra prediction (CCIP) mode, the syntax element indicating that each chroma component of the plurality of different coding blocks is determined based on one or more luma components; deriving a corresponding bias value based on a set of one or more luminance components within a reference region of a corresponding decoding block in the current image frame, wherein the reference region includes one or more decoding blocks adjacent to the corresponding decoding block and decoded before the corresponding decoding block; as well as A plurality of decoding blocks including the corresponding decoding block are reconstructed based at least on the derived offset value.

20. The non-transitory computer readable storage medium of claim 19, wherein the one or more programs further comprise instructions for: Identify a first chrominance component of the corresponding decoding block and a first luminance component co-located with the first chrominance component in the corresponding decoding block; identifying a plurality of adjacent luminance components of the first luminance component; as well as combining the first luma component, the plurality of adjacent luma components, and the corresponding bias values ​​to generate the first chroma component; wherein generating the first chrominance component further comprises applying a cross filter including 4k+1 weighting factors to combine the first luminance component, the plurality of adjacent luminance components, and the corresponding offset value; wherein each of the two row directions and the two column directions corresponds to k adjacent brightness components immediately adjacent to the first brightness component; The k adjacent luminance components in each of the two row directions are combined into the first chrominance component in the second order, and the k adjacent luminance components in each of the two column directions are linearly combined into the first chrominance component.

21. The non-transitory computer readable storage medium of claim 19, wherein the one or more programs further comprise instructions for: Identify a first chrominance component of the corresponding decoding block and a first luminance component co-located with the first chrominance component in the corresponding decoding block; identifying a plurality of adjacent luminance components of the first luminance component; as well as combining the first luma component, the plurality of adjacent luma components, and the corresponding bias values ​​to generate the first chroma component; wherein generating the first chrominance component further comprises applying a cross filter including 4k+1 weighting factors to combine the first luminance component, the plurality of adjacent luminance components, and the corresponding offset value, wherein each of the two row directions and the two column directions corresponds to k adjacent brightness components immediately adjacent to the first brightness component; The k adjacent luminance components in each of the two column directions are combined into the first chrominance component in the second order, and the k adjacent luminance components in each of the two row directions are linearly combined into the first chrominance component.

22. The non-transitory computer readable storage medium of claim 19, wherein: The reference region of the corresponding decoding block includes one or more of: an upper left reference region, a top reference region, an upper right reference region, a lower left reference region, and a left reference region.

Images