Determining bias values ​​for chroma forma (CfL) mode

Abstract

Various implementations described herein include methods and systems for coding video. In one aspect, a video bitstream includes different coding blocks of a current image frame and signals an inter-component intra-prediction mode. The method includes, for each different coding block, identifying a chroma component, a luma sample co-located with the chroma component, and a neighboring luma sample of the luma sample. The method includes deriving a respective bias value based on a set of luma samples in a reference area of ​​the respective coding block, the reference area including one or more coding blocks adjacent to the respective coding block and decoded prior to the respective coding block in the current image frame. The method further includes combining the luma sample, the neighboring luma sample, and the respective bias value to generate a chroma component, and reconstructing the different coding blocks including the chroma component of each different coding block.

Description

[Technical Field]

[0001] Related Applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 444,180, entitled "Improved Bias Value for CfL Mode," filed February 8, 2023, and is a continuation of and claims priority to U.S. Patent Application No. 18 / 497,896, entitled "Bias Value Determination for Chroma-for-Luma (CfL) Mode," filed October 30, 2023.

[0002] FIELD OF THE INVENTION

[0002] The disclosed embodiments relate generally to video coding, including, but not limited to, systems and methods for determining chroma samples from associated luma samples in cross-component intra prediction of video data. [Background technology]

[0003]

[0003] Digital video is supported by a variety of electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video teleconferencing devices, and video streaming devices. The electronic devices transmit, receive, or otherwise communicate digital video data over communication networks and / or store the digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited memory resources of storage devices, video coding may be used to compress the video data according to one or more video coding standards before the video data is communicated or stored.

[0004]

[0004] Multiple video codec standards have been developed. For example, video coding standards include AOMedia Video 1 (AV1), Generic Video Coding (VVC), Joint Exploration and Test Model (JEM), High Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), and Moving Picture Experts Group (MPEG) coding. Video coding generally uses prediction methods (e.g., inter-prediction, intra-prediction, etc.) that exploit the redundancy inherent in video data. Video coding aims to compress video data into a format that uses a lower bitrate while avoiding or minimizing degradation of video quality.

[0005]

[0005] HEVC, also known as H.265, is a video compression standard designed as part of the MPEG-H project. ITU-T and ISO / IEC published the HEVC / H.265 standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). Generic Video Coding (VVC), also known as H.266, is a video compression standard intended as the successor to HEVC. ITU-T and ISO / IEC published the VVC / H.266 standard in 2020 (version 1) and 2022 (version 2). AV1 is an open video coding format designed as a replacement for HEVC. Validated version 1.0.0, accompanied by Errata 1 of the specification, was released on January 8, 2019. Summary of the Invention [Problem to be solved by the invention]

[0006] As mentioned above, encoding (compression) reduces bandwidth and / or storage space requirements. As will be described in detail later, both lossless and lossy compression can be employed. Lossless compression refers to a technique in which an exact copy of the original signal can be reconstructed from the compressed original signal through a decoding process. Lossy compression refers to a coding / decoding process in which the original video information is not always fully preserved during coding and not always fully recoverable during decoding. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signal is made small enough to make the reconstructed signal useful for the target application. The amount of acceptable distortion depends on the application. For example, users of some consumer video streaming applications may tolerate higher distortion than users of movie or television broadcast applications. The compression ratio achievable by a particular coding algorithm can be selected or adjusted to reflect various distortion tolerances, with higher tolerable distortion generally allowing for coding algorithms that result in higher loss and higher compression ratios. [Means for solving the problem]

[0007] This disclosure describes applying a bias term to implement inter-component intra prediction of video data in a cross-component intra prediction (CCIP) mode in which each of multiple chroma samples of a current coding block is determined based on one or more luma samples. For example, CCIP modes include a cross-component linear model mode (CCLM) and a convolutional cross-component mode (CCCM). In CCLM mode, a chroma sample is transformed from a reconstructed luma sample co-located with the chroma sample based on a linear model. In CCCM mode, a chroma sample is predicted from multiple reconstructed luma samples positioned based on the filter shape of a filter having multiple weighting factors. In both CCLM and CCCM modes, when chroma subsampling is used, the reconstructed luma sample is downsampled to match a low-resolution chroma grid, and there are options to use a single model or a multi-model variant. In some embodiments, the multi-model variant uses two models: one model derived based on luma samples above an average luma reference value and another model derived based on the remaining luma samples. Furthermore, in some embodiments, such a multi-model CCCM mode is selected for prediction units that have at least 128 reference samples available.

[0008] In some embodiments, the CCCM mode is associated with a filter having multiple weighting factors (e.g., a 7-tap filter, a 5-tap filter). The multiple weighting factors are applied in conjunction with two additional weighting factors to combine the luma sample, the nonlinear term, and the bias term. For example, a cross-shaped 5-tap filter has five inputs: a central (C) luma sample co-located with the central chroma sample to be predicted, and four adjacent luma samples, including the above / north (N), below / south (S), left / west (W), and right / east (E) neighbors. The nonlinear term P represents the square of the central luma sample C scaled to the sample value range. The bias term B represents a scalar offset between the input and the output, and is set, for example, to the midpoint chroma value (512 for 10-bit content). In some embodiments, the output of the filter is calculated based on the weighting factors ci (filter coefficients c i ) and the input luma samples, and is clipped to a range of valid chroma samples. Various embodiments of the present application are directed to determining the bias term B in inter-component intra prediction of video data (e.g., CCCM mode) by adaptively deriving a respective bias value for each coding block of a current image frame based on, for example, a set of one or more luma samples in a reference area of ​​the respective coding block.

[0009] According to some embodiments, a method of video coding is provided. The method includes receiving a video bitstream including multiple different coding blocks of a current image frame. The video bitstream includes a syntax element for a cross-component intra prediction (CCIP) mode indicating that each chroma component of the multiple different coding blocks is determined based on one or more luma samples. The method further includes deriving a respective bias value based on a set of one or more luma samples in a reference area of ​​the respective coding block in the current image frame. The reference area includes one or more coding blocks that are adjacent to the respective coding block and decoded prior to the respective coding block. The method further includes reconstructing the multiple coding blocks including the respective coding block based at least on the derived bias value.

[0010] According to some embodiments, a method of video encoding is provided. The method includes encoding each respective coding block of a plurality of different coding blocks of a current image frame. Encoding each respective coding block further includes identifying a first chroma component of the respective coding block and a first luma sample co-located with the first chroma component of the respective coding block, identifying a plurality of neighboring luma samples of the first luma sample, and deriving a respective bias value based on a set of one or more luma samples in a reference area of ​​the respective coding block in the current image frame. The reference area includes one or more coding blocks that are neighboring the respective coding block and decoded prior to the respective coding block. The method further includes generating a video bitstream including the plurality of different coding blocks of the current image frame. The video bitstream includes a syntax element for a cross-component intra prediction (CCIP) mode indicating that each chroma component of the plurality of different coding blocks is determined based on the one or more luma samples.

[0011] According to some embodiments, a computing system, such as a streaming system, a server system, a personal computer system, or other electronic device, is provided. The computing system includes control circuitry and a memory that stores one or more sets of instructions. The one or more sets of instructions include instructions for performing any of the methods described herein. In some embodiments, the computing system includes an encoder component and / or a decoder component.

[0012] According to some embodiments, a non-transitory computer-readable storage medium is provided that stores one or more sets of instructions for execution by a computing system, the one or more sets of instructions including instructions for performing any of the methods described herein.

[0013]

[0013] Accordingly, disclosed are methods, as well as devices and systems, for coding video, which may complement or replace conventional methods, devices, and systems for video coding.

[0014]

[0014] The features and advantages described herein are not necessarily all-inclusive, and in particular, some additional features and advantages will become apparent to those skilled in the art in view of the drawings, specification, and claims provided in this disclosure. Moreover, it should be noted that the language used herein has been selected primarily for readability and educational purposes, and has not necessarily been selected to define or limit the subject matter described herein.

[0015]

[0015] So that the present disclosure may be understood in detail, a more particular description may be made by reference to the features of various embodiments, some of which are illustrated in the accompanying drawings, which, however, illustrate only pertinent features of the present disclosure and should not therefore be considered necessarily limiting, since the description may lead to other useful features, as will be appreciated by those skilled in the art upon reading the present disclosure. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a block diagram illustrating an exemplary communication system, according to some embodiments. [Figure 2A]

[0017] FIG. 2 is a block diagram illustrating exemplary elements of an encoder component, according to some embodiments. [Figure 2B]

[0018] 3 is a block diagram illustrating exemplary elements of a decoder component, according to some embodiments. [Figure 3]

[0019] FIG. 1 is a block diagram illustrating an exemplary server system, according to some embodiments. [Figure 4]

[0020] A diagram showing an example scheme for generating chroma samples from multiple luma samples in CCCM mode, according to some embodiments. [Figure 5A]

[0021] 1 is a diagram of an exemplary reference area including multiple reference regions for a coding block in a current image frame, according to some embodiments. [Figure 5B]

[0022] FIG. 10 is a diagram of another exemplary reference area adjusted to include only available samples of coding blocks in a current image frame, according to some embodiments. [Figure 6A]

[0023] FIG. 10 is a structural diagram of an example filter shape of a filter applied to combine luma samples in CCCM mode, according to some embodiments. [Figure 6B]

[0023] FIG. 1 is a structural diagram of an exemplary filter shape of a filter applied to combine luma samples in CCCM mode, according to some embodiments. [Figure 6C]

[0023] FIG. 1 is a structural diagram of an exemplary filter shape of a filter applied to combine luma samples in CCCM mode, according to some embodiments. [Figure 6D]

[0023] FIG. 1 is a structural diagram of an exemplary filter shape of a filter applied to combine luma samples in CCCM mode, according to some embodiments. [Figure 6E]

[0023] FIG. 1 is a structural diagram of an exemplary filter shape of a filter applied to combine luma samples in CCCM mode, according to some embodiments. [Figure 6F]

[0023] FIG. 1 is a structural diagram of an exemplary filter shape of a filter applied to combine luma samples in CCCM mode, according to some embodiments. [Figure 7]

[0024] 1 is a flow diagram illustrating a method for coding video according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0017]

[0025] According to common practice, the various features illustrated in the drawings are not necessarily drawn to scale and like reference numerals may be used to denote like features throughout the specification and figures.

[0018]

[0026] This disclosure describes applying a bias term to implement inter-component intra prediction of video data in a cross-component intra prediction (CCIP) mode in which each of multiple chroma samples of a current coding block is determined based on one or more luma samples. For example, CCIP modes include a cross-component linear model mode (CCLM) and a convolutional cross-component mode (CCCM). In CCLM mode, a chroma sample is transformed from a reconstructed luma sample co-located with the chroma sample based on a linear model. In CCCM mode, a chroma sample is predicted from multiple reconstructed luma samples positioned based on the filter shape of a filter having multiple weighting factors. In both CCLM and CCCM modes, when chroma subsampling is used, the reconstructed luma sample is downsampled to match a low-resolution chroma grid, and there is the option of using a single model or a multi-model variant. In some embodiments, the multi-model variant uses two models: one model derived based on luma samples above an average luma reference value and another model derived based on the remaining luma samples. Furthermore, in some embodiments, such a multi-model CCCM mode is selected for prediction units that have at least 128 reference samples available.

[0019]

[0027] In some embodiments, the CCCM mode is associated with a filter having multiple weighting factors (e.g., a 7-tap filter, a 5-tap filter). The multiple weighting factors are applied in conjunction with two additional weighting factors to combine the luma sample, the nonlinear term, and the bias term. For example, a cross-shaped 5-tap filter has five inputs: a central (C) luma sample co-located with the central chroma sample to be predicted, and four adjacent luma samples including an above / north (N), a below / south (S), a left / west (W), and a right / east (E) adjacent sample. The nonlinear term P represents the square of the central luma sample C scaled to the sample value range. The bias term B represents a scalar offset between the input and output, and is set to, for example, the midpoint chroma value (512 for 10-bit content). Various embodiments of the present application are directed to determining a bias term B in inter-component intra prediction of video data (e.g., CCCM mode) by, for example, adaptively deriving a respective bias value for each coding block of a current image frame based on a set of one or more luma samples in a reference area of ​​the respective coding block. Furthermore, in some embodiments, the output of the filter is determined as a convolution between weighting factors c i (also referred to as filter coefficients c i ) and the input luma samples, and is clipped to a range of valid chroma samples.

[0020]

[0028] 1 is a block diagram illustrating a communication system 100 according to some embodiments. Communication system 100 includes a source device 102 and multiple electronic devices 120 (e.g., electronic devices 120-1 through 120-m) communicatively coupled to each other via one or more networks. In some embodiments, communication system 100 is a streaming system for use with video-enabled applications, such as, for example, video conferencing applications, digital TV applications, and media storage and / or distribution applications.

[0021]

[0029] Source device 102 includes a video source 104 (e.g., a camera component or media storage) and an encoder component 106. In some embodiments, video source 104 is a digital camera (e.g., configured to create an uncompressed video sample stream). Encoder component 106 generates one or more encoded video bitstreams from the video stream. The video stream from video source 104 may be of a high data volume compared to encoded video bitstream 108 generated by encoder component 106. Because encoded video bitstream 108 has a lower data volume (less data) compared to the video stream from the video source, encoded video bitstream 108 requires less bandwidth to transmit and less storage space to store compared to the video stream from video source 104. In some embodiments, source device 102 does not include encoder component 106 (e.g., configured to transmit uncompressed video data to network(s) 110).

[0022]

[0030] The one or more networks 110 represent any number of networks that convey information between the source device 102, the server system 112, and / or the electronic device 120, including, for example, wireline (wired) and / or wireless communication networks. The one or more networks 110 may exchange data in circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet.

[0023]

[0031] The one or more networks 110 include a server system 112 (e.g., a distributed / cloud computing system). In some embodiments, the server system 112 is or includes a streaming server (e.g., configured to store and / or distribute video content, such as an encoded video stream from the source device 102). The server system 112 includes a coder component 114 (e.g., configured to encode and / or decode video data). In some embodiments, the coder component 114 includes an encoder component and / or a decoder component. In various embodiments, the coder component 114 is instantiated as hardware, software, or a combination thereof. In some embodiments, the coder component 114 is configured to decode the encoded video bitstream 108 and re-encode the video data using different encoding standards and / or methodologies to generate encoded video data 116. In some embodiments, the server system 112 is configured to generate multiple video formats and / or encodings from the encoded video bitstream 108.

[0024]

[0032] In some embodiments, server system 112 functions as a media-aware network element (MANE). For example, server system 112 may be configured to prune encoded video bitstream 108 to accommodate potentially different bitstreams to one or more of electronic devices 120. In some embodiments, a MANE is provided separately from server system 112.

[0025]

[0033] Electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, decoder component 122 is configured to decode encoded video data 116 to generate an outgoing video stream that can be rendered on a display or other type of rendering device. In some embodiments, one or more of electronic devices 120 do not include a display component (e.g., are communicatively coupled to an external display device and / or include media storage). In some embodiments, electronic device 120 is a streaming client. In some embodiments, electronic device 120 is configured to access server system 112 to obtain encoded video data 116.

[0026]

[0034] The source device and / or the plurality of electronic devices 120 may be referred to as “terminal devices” or “user devices.” In some embodiments, the source device 102 and / or one or more of the electronic devices 120 are instances of a server system, a personal computer, a portable device (e.g., a smartphone, tablet, or laptop), a wearable device, a videoconferencing device, and / or other types of electronic devices.

[0027]

[0035] In an exemplary operation of communication system 100, source device 102 transmits encoded video bitstream 108 to server system 112. For example, source device 102 may code a stream of pictures captured by the source device. Server system 112 may receive encoded video bitstream 108 and decode and / or encode encoded video bitstream 108 using coder component 114. For example, server system 112 may apply coding to the video data, which is more optimal for network transmission and / or storage. Server system 112 may transmit encoded video data 116 (e.g., one or more coded video bitstreams) to one or more of electronic devices 120. Each electronic device 120 may decode the encoded video data 116 to recover the video pictures and, optionally, display them.

[0028]

[0036] In some embodiments, the transmission described above is a unidirectional data transmission. The unidirectional data transmission may be utilized in media serving applications, etc. In some embodiments, the transmission described above is a two-way data transmission. The two-way data transmission may be utilized in video conferencing applications, etc. In some embodiments, the coded video bitstream 108 and / or the coded video data 116 are coded and / or decoded according to any of the video coding / compression standards described herein, such as HEVC, VVC, and / or AV1.

[0029]

[0037] 2A is a block diagram illustrating exemplary elements of the encoder component 106, according to some embodiments. The encoder component 106 receives a source video sequence from a video source 104. In some embodiments, the encoder component includes a receiver (e.g., a transceiver) component configured to receive the source video sequence. In some embodiments, the encoder component 106 receives a video sequence from a remote video source (e.g., a video source that is a component of a different device than the encoder component 106). The video source 104 may provide the source video sequence in the form of a digital video sample stream that may be of any suitable bit depth (e.g., 8-bit, 10-bit, or 12-bit), any color space (e.g., BT.601 Y CrCb, or RGB), and any suitable sampling structure (e.g., Y CrCb 4:2:0 or Y CrCb 4:4:4). In some embodiments, the video source 104 is a storage device that stores previously captured / prepared video. In some embodiments, the video source 104 is a camera that captures local image information as a video sequence. Video data may be provided as multiple individual pictures that, when viewed in sequence, impart motion. The pictures themselves may be organized as a spatial array of pixels, each of which may contain one or more samples depending on the sampling structure, color space, etc., in use. Those skilled in the art can readily understand the relationship between pixels and samples. The following discussion focuses on samples.

[0030]

[0038] The encoder component 106 is configured to code and / or compress pictures of a source video sequence into a coded video sequence 216 in real time or under other time constraints required by an application. Enforcing an appropriate coding rate is one function of the controller 204. In some embodiments, the controller 204 controls and is operatively coupled to other functional units as described below. Parameters set by the controller 204 may include rate control-related parameters (e.g., picture skip, quantizer, and / or λ value for rate-distortion optimization techniques), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. Those skilled in the art can readily identify other functions of the controller 204, as they may pertain to the encoder component 106 being optimized for a certain system design.

[0031]

[0039] In some embodiments, the encoder component 106 is configured to operate in a coding loop. In a simplified example, the coding loop includes a source coder 202 (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be coded and one or more reference pictures) and a (local) decoder 210. The decoder 210 reconstructs the symbols to create sample data in a manner similar to a (remote) decoder (when the compression between the symbols and the coded video bitstream is lossless). The reconstructed sample stream (sample data) is input to a reference picture memory 208. Because decoding of the symbol stream yields bit-exact results independent of the decoder location (local or remote), the contents in the reference picture memory 208 are also bit-exact between the local and remote encoders. In this way, the predictive portion of the encoder interprets the same sample values ​​as reference picture samples that the decoder would interpret when using prediction during decoding. This principle of reference picture synchrony (and the resulting drift if synchrony cannot be maintained, for example due to channel errors) is known to those skilled in the art.

[0032]

[0040] The operation of decoder 210 may be the same as that of a remote decoder, such as decoder component 122, which is described in detail below in connection with Figure 2B. However, with brief reference to Figure 2B, because symbols are available and the encoding / decoding of symbols into a coded video sequence by entropy coder 214 and parser 254 may be lossless, the entropy decoding portion of decoder component 122, including buffer memory 252 and parser 254, may not be fully implemented in local decoder 210.

[0033]

[0041] An observation that can be made at this point is that any decoder technology other than parsing / entropy decoding that is present in a decoder must necessarily be present, in a substantially equivalent functional form, in the corresponding encoder. For this reason, the disclosed subject matter focuses on decoder operation. A description of the encoder technology may be omitted, since the encoder technology is the inverse of the decoder technology that will be comprehensively described. Only in some areas is further detail required, which is provided below.

[0034]

[0042] As part of its operation, source coder 202 may perform motion-compensated predictive coding, which predictively codes an input frame with reference to one or more previously coded frames from a video sequence designated as reference image frames. In this manner, coding engine 212 codes differences between pixel blocks of the input frame and pixel blocks of reference image frame(s) that may be selected as prediction reference(s) for the input frame. Controller 204 may manage the coding operations of source coder 202, including, for example, setting parameters and subgroup parameters used to encode the video data.

[0035]

[0043] The decoder 210 decodes coded video data of frames that may be designated as image frames based on symbols created by the source coder 202. The operation of the coding engine 212 may advantageously be a lossy process. When the coded video data is decoded in a video decoder (not shown in FIG. 2A ), the reconstructed video sequence may be a copy of the source video sequence with some errors. The decoder 210 replicates the decoding process that may be performed on reference image frames by a remote video decoder, which may cause the reconstructed reference image frames to be stored in the reference picture memory 208. In this way, the encoder component 106 locally stores copies of reconstructed reference image frames having common content as the reconstructed reference image frames that will be obtained by the remote video decoder (without transmission errors).

[0036]

[0044] The predictor 206 may perform the predictive search for the coding engine 212. That is, for a new frame to be coded, the predictor 206 may search the reference picture memory 208 for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that can serve as suitable predictive references for the new picture. The predictor 206 may operate on a sample block-by-pixel block basis to find suitable predictive references. In some cases, as determined by the search results obtained by the predictor 206, the input picture may have predictive references drawn from multiple reference pictures stored in the reference picture memory 208.

[0037]

[0045] The output of all the above-mentioned functional units may undergo entropy coding in entropy coder 214. Entropy coder 214 turns the symbols produced by the various functional units into a coded video sequence by losslessly compressing them according to techniques known to those skilled in the art (e.g., Huffman coding, variable length coding, and / or arithmetic coding).

[0038]

[0046] In some embodiments, the output of the entropy coder 214 is coupled to a transmitter. The transmitter may be configured to buffer the coded video sequence(s) created by the entropy coder 214 to prepare them for transmission over a communication channel 218, which may be a hardware / software link to a storage device that will store the coded video data. The transmitter may be configured to merge the coded video data from the source coder 202 with other data to be transmitted, such as coded audio data and / or an auxiliary data stream (source not shown). In some embodiments, the transmitter may transmit additional data along with the coded video. The source coder 202 may include such data as part of the coded video sequence. The additional data may include other forms of redundant data, such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, supplemental enhancement information (SEI) messages, visual usability information (VUI) parameter set fragments, etc.

[0039]

[0047] The controller 204 may manage the operation of the encoder component 106. During coding, the controller 204 may assign a coded picture type to each coded picture, which may affect the coding technique applied to the respective picture. For example, a picture may be assigned as an intra picture (I picture), a predicted picture (P picture), or a bidirectionally predicted picture (B picture). An intra picture may be coded and decoded without using any other frame in the sequence as a source of prediction. Some video codecs allow different types of intra pictures, including, for example, independent decoder refresh (IDR) pictures. Those skilled in the art are aware of these variations of I pictures and their respective applications and characteristics, and therefore, they will not be repeated here. A predicted picture may be coded and decoded using intra prediction or inter prediction, using at most one motion vector and reference index to predict sample values ​​for each block. Bidirectionally predicted pictures can be coded and decoded using intra or inter prediction, using at most two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multi-predicted pictures can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0040]

[0048] A source picture is generally spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and may be coded block-by-block. Blocks may be predictively coded with reference to other (already coded) blocks determined by the coding assignment applied to the block's respective picture. For example, blocks of an I-picture may be non-predictively coded, or they may be predictively coded with reference to already coded blocks of the same picture (spatial prediction or intra-prediction). Pixel blocks of a P-picture may be non-predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be non-predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.

[0041]

[0049] Video may be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra-prediction) exploits spatial correlation within a given picture, while inter-picture prediction exploits correlation (temporal or other) between pictures. In one example, a particular picture to be coded / decoded, called the current picture, is partitioned into blocks. When a block in the current picture is similar to a reference block in a previously coded and still buffered reference picture in the video, the block in the current picture may be coded by a vector called a motion vector. A motion vector points to a reference block in a reference picture and may have a third dimension that identifies the reference picture if multiple reference pictures are in use.

[0042]

[0050] Encoder component 106 may perform coding operations in accordance with a given video coding technique or standard, such as any of those described herein. In doing so, encoder component 106 may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the coded video data may conform to a syntax specified by the video coding technique or standard being used.

[0043]

[0051] 2B is a block diagram illustrating exemplary elements of the decoder component 122, according to some embodiments. The decoder component 122 in FIG. 2B is coupled to the channel 218 and the display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to the loop filter 256 and configured to transmit data to the display 124 (e.g., via a wired or wireless connection).

[0044]

[0052] In some embodiments, the decoder component 122 includes a receiver coupled to the channel 218 and configured to receive data from the channel 218 (e.g., via a wired or wireless connection). The receiver may be configured to receive one or more coded video sequences to be decoded by the decoder component 122. In some embodiments, the decoding of each coded video sequence is independent of the other coded video sequences. Each coded video sequence may be received from the channel 218, which may be a hardware / software link to a storage device that stores the coded video data. The receiver may receive the coded video data along with other data, e.g., coded audio data and / or auxiliary data streams, which may be forwarded to their respective using entities (not shown). The receiver may separate the coded video sequence from the other data. In some embodiments, the receiver receives additional (redundant) data along with the coded video. The additional data may be included as part of the coded video sequence(s). The additional data may be used by the decoder component 122 to decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or SNR enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0045]

[0053] According to some embodiments, the decoder component 122 includes a buffer memory 252, a parser 254 (sometimes referred to as an entropy decoder), a scaler / inverse transform unit 258, an intra-picture prediction unit 262, a motion compensation prediction unit 260, an aggregator 268, a loop filter unit 256, a reference picture memory 266, and a current picture memory 264. In some embodiments, the decoder component 122 is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuitry. In some embodiments, the decoder component 122 is implemented at least partially in software.

[0046]

[0054] The buffer memory 252 is coupled intermediate the channel 218 and the parser 254 (e.g., to eliminate network jitter). In some embodiments, the buffer memory 252 is separate from the decoder component 122. In some embodiments, a separate buffer memory is provided between the output of the channel 218 and the decoder component 122. In some embodiments, in addition to the buffer memory 252 internal to the decoder component 122 (e.g., configured to handle playout timing), a separate buffer memory is provided external to the decoder component 122 (e.g., to eliminate network jitter). When receiving data from a storage / forwarding device of sufficient bandwidth and controllability or from an isosynchronous network, the buffer memory 252 may not be needed or may be small. For use with best-effort packet networks such as the Internet, the buffer memory 252 may be needed and may be relatively large, advantageously adaptively sized, and at least partially implemented in an operating system or similar element (not shown) external to the decoder component 122.

[0047]

[0055] Parser 254 is configured to reconstruct symbols 270 from the coded video sequence. These symbols may include, for example, information used to manage the operation of decoder component 122 and / or information for controlling a rendering device such as display 124. The control information for the rendering device(s) may be in the form of, for example, a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). Parser 254 parses (entropy decodes) the coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow principles well known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. Parser 254 may extract from the coded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to that group. The subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. Parser 254 may also extract information from the coded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0048]

[0056] The reconstruction of symbols 270 may involve several different units, depending on the type of coded video picture or portion thereof (e.g., inter-picture and intra-picture, inter-block and intra-block, etc.), and other factors. Which units are involved and how they are involved may be controlled by subgroup control information parsed from the coded video sequence by parser 254. The flow of such subgroup control information between parser 254 and the following units is not shown for clarity.

[0049]

[0057] Beyond the functional blocks already mentioned, the decoder component 122 may be conceptually subdivided into several functional units, which are described below. In an actual implementation operating under commercial constraints, many of these units will interact closely with each other and may be, at least partially, integrated with each other. However, for purposes of describing the disclosed subject matter, the conceptual subdivision into the following functional units will be maintained.

[0050]

[0058] The scaler / inverse transform unit 258 receives the quantized transform coefficients as well as control information (such as which transform to use, block size, quantization factor, and / or quantization scaling metric) from the parser 254 as symbol(s) 270. The scaler / inverse transform unit 258 may output blocks containing sample values ​​that may be input to the aggregator 268.

[0051]

[0059] In some cases, the output samples of the scaler / inverse transform unit 258 relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by the intra-picture prediction unit 262. The intra-picture prediction unit 262 may generate blocks of the same size and shape as the block being reconstructed using surrounding already reconstructed information fetched from the current (partially reconstructed) picture from the current picture memory 264. The aggregator 268 may add, on a sample-by-sample basis, the prediction information generated by the intra-picture prediction unit 262 to the output sample information provided by the scaler / inverse transform unit 258.

[0052]

[0060] In other cases, the output samples of the scalar / inverse transform unit 258 relate to inter-coded and potentially motion-compensated blocks. In such cases, the motion-compensated prediction unit 260 can access the reference picture memory 266 to fetch samples used for prediction. After motion-compensating the fetched samples according to symbols 270 related to the block, these samples can be added by the aggregator 268 to the output of the scalar / inverse transform unit 258 (in this case, referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory 266 from which the motion-compensated prediction unit 260 fetches prediction samples can be controlled by motion vectors. The motion vectors can be available to the motion-compensated prediction unit 260, for example, in the form of symbols 270 that can have X, Y, and reference picture components. Motion compensation can also include interpolation of sample values ​​fetched from the reference picture memory 266 when sub-sample exact motion vectors are in use, motion vector prediction mechanisms, and the like.

[0053]

[0061] The output samples of aggregator 268 may undergo various loop filtering techniques in loop filter unit 256. Video compression techniques may include in-loop filter techniques controlled by parameters included in the coded video bitstream and made available to loop filter unit 256 as symbols 270 from parser 254, but may also be responsive to meta-information obtained during decoding of a coded picture or previous portion (in decoding order) of a coded video sequence, as well as to previously reconstructed and loop-filtered sample values.

[0054]

[0062] The output of the loop filter unit 256 may be a sample stream that may be output to a render device such as the display 124, as well as stored in the reference picture memory 266 for use in future inter-picture prediction.

[0055]

[0063] Some coded pictures, when reconstructed, may be used as reference pictures for future prediction. Once a coded picture is reconstructed and the coded picture is identified as a reference picture (e.g., by parser 254), the current reference picture may become part of reference picture memory 266, and a fresh current picture memory may be reallocated before beginning reconstruction of a subsequent coded picture.

[0056]

[0064] Decoder component 122 may perform decoding operations according to a predetermined video compression technology, which may be documented in a standard, such as any of the standards described herein. A coded video sequence may conform to the syntax specified by the video compression technology or standard being used, in that it follows the syntax of the video compression technology or standard as specified in the video compression technology document or standard, and in particular, in a profile document therein. Also, for compliance with some video compression technologies or standards, the complexity of a coded video sequence may be within limits defined by the level of the video compression technology or standard. In some cases, the level restricts a maximum picture size, a maximum frame rate, a maximum reconstruction sample rate (e.g., measured in megasamples per second), a maximum reference picture size, etc. The limits set by the level may, in some cases, be further restricted through a hypothetical reference decoder (HRD) specification and metadata for HRD buffer management signaled in the coded video sequence.

[0057]

[0065] 3 is a block diagram illustrating a server system 112, according to some embodiments. The server system 112 includes a control circuit 302, one or more network interfaces 304, a memory 314, a user interface 306, and one or more communication buses 312 for interconnecting these components. In some embodiments, the control circuit 302 includes one or more processors (e.g., a CPU, a GPU, and / or a DPU). In some embodiments, the control circuit includes one or more field programmable gate arrays (FPGAs), hardware accelerators, and / or one or more integrated circuits (e.g., application specific integrated circuits).

[0058]

[0066] The network interface(s) 304 may be configured to interface with one or more communication networks (e.g., wireless networks, wireline networks, and / or optical networks). The communication networks may be local, wide-area, metropolitan, vehicular and industrial, real-time, delay-tolerant, etc. Examples of communication networks include local area networks such as Ethernet, wireless LANs, cellular networks, to include GSM, 3G, 4G, 5G, LTE, etc.; TV wireline or wireless wide-area digital networks, to include cable TV, satellite TV, and terrestrial broadcast TV; vehicular and industrial, to include CANbus; etc. Such communications may be unidirectional receive-only (e.g., broadcast TV), unidirectional transmit-only (e.g., CANbus to some CANbus devices), or bidirectional (e.g., with other computer systems using local or wide-area digital networks). Such communications may include communications to one or more cloud computing networks.

[0059]

[0067] The user interface 306 includes one or more output devices 308 and / or one or more input devices 310. The input device(s) 310 may include one or more of a keyboard, a mouse, a trackpad, a touchscreen, a data glove, a joystick, a microphone, a scanner, a camera, etc. The output device(s) 308 may include one or more of an audio output device (e.g., a speaker), a visual output device (e.g., a display or monitor), etc.

[0060]

[0068] Memory 314 may include high-speed random-access memory (such as DRAM, SRAM, DDR RAM, and / or other random-access solid-state memory devices) and / or non-volatile memory (such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid-state storage devices). Memory 314 optionally includes one or more storage devices located remotely from control circuitry 302. Memory 314, or alternatively, the non-volatile solid-state memory device(s) within memory 314, comprises a non-transitory computer-readable storage medium. In some embodiments, memory 314 or the non-transitory computer-readable storage medium of memory 314 stores the following programs, modules, instructions, and data structures, or a subset or superset thereof: an operating system 316, which includes procedures for handling various basic system services and for performing hardware-dependent tasks; a network communications module 318 used to connect the server system 112 to other computing devices via one or more network interfaces 304 (e.g., via wired and / or wireless connections); A coding module 320 for performing various functions related to encoding and / or decoding data, such as video data. In some embodiments, the coding module 320 is an instance of the coder component 114. The coding module 320 may include, but is not limited to: a decoding module 322 for performing various functions related to decoding the encoded data, such as those previously described with respect to the decoder component 122; an encoding module 340 for performing various functions related to encoding data, such as those previously described with respect to the encoder component 106; and A picture memory 352 for storing pictures and picture data, e.g., for use with the coding module 320. In some embodiments, the picture memory 352 includes one or more of the reference picture memory 208, the buffer memory 252, the current picture memory 264, and the reference picture memory 266.

[0061]

[0069] In some embodiments, the decoding module 322 includes a parsing module 324 (e.g., configured to perform various functions previously described with respect to the parser 254), a transform module 326 (e.g., configured to perform various functions previously described with respect to the scalar / inverse transform unit 258), a prediction module 328 (e.g., configured to perform various functions previously described with respect to the motion compensation prediction unit 260 and / or the intra-picture prediction unit 262), and a filter module 330 (e.g., configured to perform various functions previously described with respect to the loop filter 256).

[0062]

[0070] In some embodiments, encoding module 340 includes a code module 342 (e.g., configured to perform various functions previously described with respect to source coder 202 and / or coding engine 212) and a prediction module 344 (e.g., configured to perform various functions previously described with respect to predictor 206). In some embodiments, decoding module 322 and / or encoding module 340 include a subset of the modules shown in Figure 3. For example, a shared prediction module is used by both decoding module 322 and encoding module 340.

[0063]

[0071] Each of the above-identified modules stored in memory 314 corresponds to a set of instructions for performing functions described herein. The above-identified modules (e.g., sets of instructions) need not be implemented as separate software programs, procedures, or modules; thus, various subsets of these modules may be combined or possibly rearranged in various embodiments. For example, coding module 320 optionally does not include separate decoding and encoding modules, but rather uses the same set of modules to perform both sets of functions. In some embodiments, memory 314 stores a subset of the above-identified modules and data structures. In some embodiments, memory 314 stores additional modules and data structures not described above, such as an audio processing module.

[0064]

[0072] 3 illustrates a server system 112 according to some embodiments, although FIG. 3 is intended as a functional illustration of various features that may be present in one or more server systems, rather than a structural schematic of the embodiments described herein. In practice, and as will be recognized by those skilled in the art, items shown separately may be combined and some items may be separated. For example, some items shown separately in FIG. 3 may be implemented on a single server, and a single item may be implemented by one or more servers. The actual number of servers used to implement server system 112 and how features are allocated among them may vary from implementation to implementation and optionally depend in part on the amount of data traffic the server system handles during peak usage periods as well as during average usage periods.

[0065]

[0073] 4 illustrates an example scheme 400 for generating chroma samples 402 from multiple luma samples 404 in CCCM mode, according to some embodiments. In some embodiments, each of multiple different coding blocks 406 (e.g., a first coding block 406A, a second coding block 406B) of a current image frame 408 is coded in a cross-component intra-prediction (CCIP) mode. In CCIP mode, the decoder 122 (FIG. 2B) determines each chroma sample 402 of the multiple coding blocks 406 based on one or more reconstructed luma samples 404. CCIP mode uses multiple weighting factors (c iThe CCCM mode includes a CCCM mode in which a first chroma sample 402C is generated by combining multiple reconstructed luma samples (e.g., samples 404C, 404N, 404S, 404W, and 404E) using a bias value B. For each coding block 406 (e.g., first coding block 406A), the multiple reconstructed luma samples include a first luma sample 404C of the respective coding block 406 that is co-located with the first chroma sample 402C and multiple adjacent luma samples 404x located adjacent to the first luma sample 404C, for example, based on a filter shape of a filter. Each bias value B is determined based on a set of one or more luma samples 404 in a reference area 412 of the respective coding block 406 in the current image frame 408. The reference area 412 includes one or more coding blocks that are adjacent to the respective coding block 406A and that are decoded prior to the respective coding block 406A. The first luma sample 402C, the plurality of neighboring luma samples 404x, and the respective bias values ​​B are combined to generate the first chroma sample 402C. The decoder 122 reconstructs the plurality of different coding blocks 406, including the first chroma component 402C of each respective coding block 406. In some embodiments, the first coding block 406A is different from the second coding block 406B, and the respective bias values ​​B of the first coding block 406A and the second coding block 406B are determined based on different reference areas of the coding blocks 406A and 406B.

[0066]

[0074] The filter shape of the filter determines the number of neighboring luma samples 404x and their locations relative to the first luma sample 404C. For example, a cross-shaped 5-tap filter has five inputs: a first luma sample 404C co-located with the chroma sample 402C to be predicted, and four neighboring luma samples 404x, including a north / above neighboring sample 404N, a south / below neighboring sample 404S, a west / left neighboring sample 404W, and an east / right neighboring sample 404E. Each of the first luma sample 404C and the neighboring luma samples 404x corresponds to a respective weighting factor (e.g., c0, c1, c2, c3, or c4), also referred to as a filter coefficient.

[0067]

[0075] In some embodiments, the plurality of weighting factors includes weighting factors (e.g., c0-c4) that are applied as filter coefficients of the filter, and two additional weighting factors (e.g., c5, c6). A weighted combination is determined for the linear components of the luma samples 404C and 404x, the nonlinear term P, and the bias term B, and is applied to generate the first chroma sample 402C according to equation 414. In some embodiments, the weighting factor c0 corresponding to the first luma sample 404C is greater than the remaining weighting factors (e.g., c1-c6) of the plurality of weighting factors. The nonlinear term P represents the square of the first luma sample 404C scaled to the sample value range as follows: P=(C*C+midVal)>>bitDepth (1) where C represents the value of the first luma sample 404C, midVal represents the rounded integer average of the two boundaries of the range of luma values, and bitDepth represents the bit depth of the luma values. For example, if the bit depth of the luma values ​​is 10, then the nonlinear term P can be expressed as: P=(C*C+512)>>10 (2)

[0068]

[0076] The bias term B represents a scalar offset between the input and the output, and is set, for example, to an intermediate chroma value (512 for 10-bit content). In some embodiments, the output of the filter is determined as the convolution of the filter coefficients with the input (e.g., the first luma sample 404C and the neighboring luma sample 404x) and clipped to the range of valid chroma samples 404.

[0069]

[0077] In some embodiments, the reference area 412 of each respective coding block 406 includes one or more coding blocks (e.g., eight coding blocks in FIG. 4 ) that are decoded prior to the respective coding block 406. In some embodiments, a subset of the one or more coding blocks is immediately adjacent to the respective coding block 406. In some embodiments, the subset of the one or more coding blocks is separated from the respective coding block 406 by one or more coding blocks. In some embodiments, the reference area 412 includes at least a portion of rows above the respective coding block 406 and / or a portion of columns to the left of the respective coding block 406. For example, with reference to FIG. 4 , the reference area 412 includes seven rows of chroma samples above the respective coding block 406 and nine columns of chroma samples to the left of the respective coding block 406. In some embodiments, the reference area 412 extends one coding block width to the right of the right boundary of the respective coding block 406 and one coding block height below the bottom boundary of the respective coding block 406. In some embodiments, the reference area 412 is adjusted to include only the available samples. An extension 412E to the reference area 412 is required to support the samples on the sides of the cross filter, and is padded in areas that are not available.

[0070]

[0078] In some embodiments, for each coding block 406, the set of one or more luma samples in the reference area 412 includes a single luma sample (e.g., 404R1) in the reference area 412 of the respective coding block 406 in the current image frame 408. Alternatively, in some embodiments, for each coding block 406, the respective bias value B is equal to the average of a set of one or more luma samples including multiple luma samples (e.g., 404R1, 404R2, and 404R3) in the reference area 412 of the respective coding block 406 in the current image frame 408.

[0071]

[0079] In some embodiments, for each coding block 406, the set of one or more luma samples in the reference area 412 includes one or more rows of luma samples 410R above the respective coding block 406 in the reference area 412, and the respective bias values ​​are equal to the center luma value of the one or more rows of luma samples. Alternatively, in some embodiments, for each coding block 406, the set of one or more luma samples in the reference area 412 includes one or more rows of luma samples 410R above the respective coding block 406 in the reference area 412, and the set of one or more luma samples includes a reference luma sample 404R4 that is located approximately physically in the center of the one or more rows of luma samples 410R. The respective bias values ​​are equal to the luma value of the reference luma sample 404R4. In some embodiments, each row 410R of luma samples includes an even number of luma samples, and the reference luma sample 404R4 corresponds to the two luma samples immediately adjacent to the center. Each bias value B is equal to the average of the two luma samples immediately adjacent to the center.

[0072]

[0080] In some embodiments, for each coding block 406 (e.g., block 406A), the set of one or more luma samples 404 in the reference area 412 includes one or more columns of luma samples 410C located to the left of the respective coding block 406 in the reference area 412, and the respective bias values ​​B are equal to the center luma value of the one or more columns of luma samples 410C. Alternatively, in some embodiments, for each coding block 406 (e.g., block 406A), the set of one or more luma samples 402 in the reference area 412 includes one or more columns of luma samples 410C located to the left of the respective coding block 406 in the reference area 412, and the set of one or more luma samples 404 includes a reference luma sample 404R5 located approximately physically in the center of the one or more columns of luma samples 410C. The respective bias values ​​B are equal to the luma value of the reference luma sample 404R5. In some embodiments, the column of luma samples 410C includes an even number of luma samples, and the reference luma sample 404R5 corresponds to the two luma samples immediately adjacent to the center, with each bias value B equal to the average of the two luma samples immediately adjacent to the center.

[0073]

[0081] In some embodiments, as described above, the first luma sample 404C(C), the plurality of adjacent luma samples 404x (e.g., N, S, W, E), and the bias value B for generating the first chroma component 402C(preChromaVal) based on equation 414 are as follows: predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B where c0 through c6 are weighting factors. In some embodiments, decoder 122 subtracts first luma sample 404C and each of the plurality of neighboring luma samples 404x by the offset luma value before applying equation 414. Further, in some embodiments, decoder 122 clips first luma sample 404C and each of the plurality of neighboring luma samples 404x that have been subtracted by the offset luma value before applying equation 414. In addition, in some embodiments, the offset luma value is selected from the following: an average of first luma sample 404C and each of the plurality of neighboring luma samples 404x, an average of luma samples in a top reference region 402T (FIG. 5A) of a reference area 412 located above each coding block 406, an average of luma samples in a left reference region 402L (FIG. 5A) of a reference area located to the left of each coding block 406, an average of luma samples in a left reference region 402L (FIG. 5A) of a reference area located to the left of each coding block 406, or a luma value of a single luma sample 404O located in the top-left corner of each coding block 406. the luma value of one or more rows of luma samples 410R located above each coding block 406 in the reference area 412, the luma value of luma sample 404R4 located approximately in the physical center of one or more rows of luma samples 410R, the luma value of one or more columns of luma samples 410C located to the left of each coding block 406 in the reference area 412, and the luma value of luma sample 404R5 located approximately in the physical center of one or more columns of luma samples 410C.

[0074]

[0082] In some embodiments, the CCIP mode includes a multi-chroma-from-luma (multi-CfL) mode. The decoder 122 determines that multiple different coding blocks 405 are coded in the multi-CfL mode. In accordance with determining that each coding block 406 is coded in the multi-CfL mode, multiple neighboring luma samples 404x of a first luma sample 404C of the respective coding block are identified based on a filter shape of a filter, and a first chroma component 402C of the respective coding block 406 is a weighted sum of the first luma sample 404C, the multiple neighboring luma samples 404x, an output of a nonlinear function of the first luma sample 404C, and a respective bias value B based on equation 414. In some embodiments, the first luma sample 404C, the multiple neighboring luma samples 404x, and the bias value B are combined using multiple weighting factors to generate the first chroma component 402C, where each weighting factor includes an integer portion and a fractional portion.

[0075]

[0083] In some embodiments, the filter shape of the filter is cross-shaped and has five taps. According to the filter shape, the plurality of neighboring luma samples 404x includes four neighboring luma samples 404N, 404S, 404W, and 404E arranged in a cross shape centered on the first luma sample 404C. Furthermore, in some embodiments, the plurality of weighting factors of the filter include seven weighting factors corresponding to the first luma sample 404C, the four neighboring luma samples 404x, the output of the nonlinear function of the first luma sample 404C (e.g., P), and the bias component (e.g., B), respectively.

[0076]

[0084] In some embodiments, when chroma subsampling is used, the reconstructed luma samples 404 are downsampled to match the low-resolution chroma grid. In some embodiments, the CCCM mode is implemented using a single-model or multi-model variant. The single-model uses a single equation 414 containing a single set of weighting factors to determine the chroma samples 402 of each coding block 406. Each of the single set of weighting factors is received in the video bitstream or derived from reconstructed samples of the reference area 412. Conversely, in some embodiments, the multi-model variant uses two models corresponding to two sets of weighting factors. One model is derived based on samples above the average luma reference value of the reference area 412, and the other model is derived based on the remaining luma samples of the reference area 412. Furthermore, in some embodiments, such a multi-model CCCM mode is used to reconstruct prediction units (e.g., each coding block 406) that have at least 128 reference samples available.

[0077]

[0085] In some embodiments, the weighting factors (e.g., c0 through c6 in equation 414) are determined by minimizing the mean square error (MSE) between the predicted chroma samples 402 and the reconstructed chroma samples 402 in the reference area 412. Minimizing the MSE is performed by calculating an autocorrelation matrix of the luma samples 404 and a cross-correlation vector between the luma samples 404 and the chroma samples 402 in the reference area 412. The autocorrelation matrix is ​​processed with an LDL decomposition, and the weighting factors are calculated using backsubstitution. This process roughly follows the calculation of filter coefficients of an adaptive loop filter (ALF) in enhanced compression model (ECM) video coding. The LDL decomposition does not use square root operations, but only integer arithmetic.

[0078]

[0086] In some embodiments, the use of CCCM mode is signaled in the video bitstream 116 with a context-adaptive binary arithmetic coding (CABAC) coded flag at the prediction unit level. To support CCCM mode, a new CABAC context is included. In some circumstances, CCCM is considered a submode of CCLM. If the intra-prediction mode is LM_CHROMA_IDX (enables single-mode CCCM) or MMLM_CHROMA_IDX (enables multi-model CCCM), a CCCM flag is signaled in the video bitstream 116.

[0079]

[0087] In some embodiments, the encoder 106 encodes each respective coding block 406 of a plurality of different coding blocks of a current image frame 408 into a video bitstream. The encoder 106 identifies a first chroma component 402C of each coding block 406, a first luma sample 404C co-located with the first chroma component 402C in the respective coding block 406, and a plurality of neighboring luma samples 404x of the first luma sample 402C, for example, based on a filter shape of a filter. Each bias value is derived based on a set of one or more luma samples 404 in a reference area 412 of the respective coding block 406 in the current image frame 408. The reference area 412 includes one or more coding blocks 406 that are adjacent to the respective coding block 406 and that are decoded prior to the respective coding block 406. The respective bias values ​​are optionally transmitted to the decoder 122 along with the video bitstream. The encoder 106 generates a video bitstream including the plurality of different coding blocks of the current image frame. The video bitstream signals a cross-component intra-prediction (CCIP) mode, which indicates that each chroma component of multiple different coding blocks is determined based on one or more luma samples.

[0080]

[0088] 5A is a diagram of an exemplary reference area 412 including multiple reference regions for a coding block 406 in a current image frame 408, according to some embodiments. In some embodiments, each coding block 406 in the current image frame 408 includes a first chroma sample 402C, which is generated by combining multiple reconstructed luma samples 404 (e.g., samples 404C, 404N, 404S, 404W, and 404E) using multiple weighting factors (e.g., c0-c6 in equation 414 of FIG. 4). The decoder 122 (FIG. 2) derives a respective bias value B for each coding block 406 based on the set of one or more luma samples 404 in the reference area 412 for each coding block 406 in the current image frame 408. The reference area 412 includes one or more coding blocks that are adjacent to and decoded prior to the respective coding block 406. The first luma sample 402C, the plurality of neighboring luma samples 404x, and the respective bias values ​​B are combined to generate the first chroma sample 402C. The decoder 122 reconstructs each coding block 406 that includes the first chroma sample 402C.

[0081]

[0089] In some embodiments, one or more coding blocks 412T and 412L of the reference area 412 are immediately adjacent to the respective coding block 406. In some embodiments not shown, the one or more coding blocks are separated from the respective coding block 406 by one or more coding blocks.

[0082]

[0090] In some embodiments, the reference area 412 of each coding block 406 (e.g., block 406A in FIG. 4 ) includes one or more of an upper-left reference region 412TL, an upper reference region 412T, an upper-right reference region 412TR, a lower-left reference region 412BL, and a left reference region 412L. Referring to FIG. 5 , in this example, the reference area 412 includes a top reference region 412T and a left reference region 412L. Each of the reference regions includes one or more coding blocks. Stated differently, in some embodiments, the reference area 412 includes at least a portion of multiple rows above each coding block 406 and / or a portion of multiple columns to the left of each coding block 406. For example, referring to FIG. 5 , the reference area 412 includes a first portion of seven rows of chroma samples above each coding block 406 and a second portion of nine columns of chroma samples to the left of each coding block 406. The first portion is determined by the length of each coding block 406, and the second portion is determined by the width of each coding block 406. In some embodiments, the reference area 412 extends one coding block width to the right of the right boundary of each coding block 406 and one coding block height below the bottom boundary of each coding block 406. In some embodiments, the reference area 412 is adjusted to include only available samples. An extension 412E to the reference area 412 is required to support the side samples of the cross-shaped spatial filter, and is padded in areas that are not available.

[0083]

[0091] FIG. 5B is a diagram of another exemplary reference area 412 adjusted to include only available samples of a coding block 406 in a current image frame 408, according to some embodiments. In some embodiments, the reference area 412 is adjusted to include only available samples. In other words, the decoder 122 (FIG. 2) identifies, for each coding block 406, the reference area 412 for the respective coding block 406. Following a determination that one or more color components are not available in a set of columns or rows immediately adjacent to a boundary, the decoder 122 excludes that set of columns or rows from the reference area 412. The boundary includes one of a slice boundary, a superblock boundary, a tile boundary, a coding block boundary, and an image frame boundary. For example, the current image frame 408 has a right-most column 502 of luma samples that is incomplete. Following a determination that the right-most column 502 of luma samples is incomplete, the decoder 122 excludes the right-most column 502 of luma samples from being applied to determine the respective bias value of the respective coding block 406.

[0084]

[0092] 6A-6F are structural diagrams of six exemplary filter shapes 600A-600F of a filter 600 applied to combine luma samples 404 in CCCM mode, according to some embodiments. Each filter shape 600A-600F is optionally applied to a sequence of image frames including the current image frame 408, the current image frame 408, a slice of the current image frame 408, a superblock including a respective coding block 406, or a respective coding block 406. With reference to FIG. 6A, filter shape 600A corresponds to an odd number of horizontal luma samples (e.g., three horizontal luma samples 404W, 404C, and 404E) centered about the first luma sample 404C(P0). With reference to FIG. 6B, filter shape 600B corresponds to an odd number of vertical luma samples (e.g., three vertical luma samples 404N, 404C, and 404S) centered about the first luma sample 404C(P0). With reference to Figure 6C, filter shape 600C corresponds to an array of 3x3 luma samples (P0-P8) centered at the first luma sample 404C (P0). With reference to Figures 6D and 6E, filter shape 600C corresponds to a subset (P0-P6) of the array of 3x3 luma samples centered at the first luma sample 404C and missing two corner luma samples (e.g., the top-left and bottom-right luma samples in Figure 6D, and the bottom-left and top-right luma samples in Figure 6F). With reference to Figure 6F, the filter shape is cross-shaped and corresponds to an odd number of horizontal luma samples (e.g., 5 horizontal luma samples) centered at the first luma sample 404C and an odd number of vertical luma samples (e.g., 5 vertical luma samples) centered at the same first luma sample 404C.

[0085]

[0093] In some embodiments, the video bitstream received by the decoder 122 includes a respective filter index for each coding block 406 of a plurality of different coding blocks. For each coding block 406 (e.g., 406A in FIG. 4), the decoder 122 adaptively selects one of a plurality of different filters based on the respective filter index. Each different filter has a different filter shape and multiple weighting factors (also referred to as filter coefficients). The first luma sample 404C, the multiple neighboring luma samples 404x, and the bias value B are combined using the multiple weighting factors of at least the selected filter from the multiple different filters. Further, in some embodiments, the plurality of different filters include one or more of the following: (1) a first filter 600C including nine weighting factors and configured to generate the first chroma component 402C as a weighted combination of at least the first luma sample 404C(P0) and eight neighboring luma samples 404x (P1-P8) that immediately surround the first luma sample 404C(P0); (2) a second filter 600C including seven weighting factors and configured to generate the first chroma component 402C as a weighted combination of at least the first luma sample 404C(P0) and eight neighboring luma samples 404x (P1-P8) that immediately surround the first luma sample 404C(P0); (3) a second filter 600D or 600E configured to generate the first chroma component 402C as a weighted combination of six neighboring luma samples 404x (P1-P6) located symmetrically with respect to the first luma sample 404C (P0); and (4) a cross filter 600F including 4k+1 weighting factors and configured to generate the first chroma component 402C as a weighted combination of at least the first luma sample 404C and k neighboring luma samples 404x that are immediately neighbors of the first luma sample 404C from each of two row directions and two column directions, where k is a positive integer (e.g., 2).Furthermore, in some embodiments, the first chroma component 402C is generated further based on at least one of the following: a fixed bias value, a weighted and adjustable bias value, and a second or higher order of at least one of the first luma sample 404C and the plurality of neighboring luma samples 404x.

[0086]

[0094] 6F , in some embodiments, the decoder 122 applies a cross filter 600F including 4k+1 weighting factors to combine the first luma sample 404C, the multiple neighboring luma samples 404x, and the bias value B. Each of the two row directions and two column directions corresponds to k neighboring luma samples that are immediately adjacent to the first luma sample. Furthermore, in some embodiments, the k neighboring luma samples 404x in each of the two row directions are combined into the first chroma component 402C in a quadratic manner, and the k neighboring luma samples 404x in each of the two column directions are combined into the first chroma component 402C linearly. Alternatively, in some embodiments, k adjacent luma samples 404x in each of two column directions are combined quadratically into the first chroma component 402C, and k adjacent luma samples 404x in each of two row directions are linearly combined into the first chroma component 402C.

[0087]

[0095] FIG. 7 is a flow diagram illustrating a method 700 for coding video, according to some embodiments. The method 700 may be implemented in a computing system (e.g., the server system 112, the source device 102, or the electronic device 120) having control circuitry and memory storing instructions for execution by the control circuitry. In some embodiments, the method 700 is applied in conjunction with one or more video codecs, including, but not limited to, AV1, AV2, HEVC, VVC, and ECM. In some embodiments, the method 700 is implemented by executing instructions stored in a memory (e.g., the coding module 320 of the memory 314) of the computing system. In some embodiments, the current image frame 408 includes each coding block 406. In some embodiments, the method 700 is applied to predict one color component using another color component, and downsampling is required for one or more color components. Furthermore, in some embodiments, the method 700 is applied to predict a green or blue color component using a red component.

[0088]

[0096] In some embodiments, decoder 122 receives (702) a video bitstream including multiple different coding blocks of a current image frame and applies (710) a multi-CfL mode in which a bias value B is determined based on a set of one or more luma samples in reference area 412. For example, the bias value is an average of one or more luma samples in reference area 412, denoted AvgLuma. In some embodiments, the chroma value of each coding block 406 is expressed (714) based on equation 414 (FIG. 4) as follows: predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6AvgLuma (3)

[0089]

[0097] In some embodiments, the bias value B is set as the value of a predefined neighboring sample in the reference area 412 of each coding block 406 in the current image frame 408. For example, the bias value B is set to the luma value of one of the top-left luma sample 404R2 and the bottom-right luma sample 404R3, or the chroma value of the chroma sample co-located with the luma samples 404R2 and 404R3.

[0090]

[0098] In some embodiments, the bias value B is the average of the neighboring reconstructed samples above the reference area 412 of the respective coding block 406. In some embodiments, the bias value B is the average of the neighboring reconstructed samples to the left of the reference area 412 of the respective coding block 406.

[0091]

[0099] In some embodiments, the bias value B may be the middle / center pixel of one or more lines above in the reference area 412 of the respective coding block 406. In some embodiments, the bias value B may be the middle / center pixel of one or more lines to the left in the reference area 412 of the respective coding block 406.

[0092]

[0100] In some embodiments, the first luma sample 404C and multiple neighboring luma samples 404x (e.g., N, S, C, W, E) are offset by an offset value to reduce the range of values. In some embodiments, the offset value is an average of the luma values ​​of the respective coding block 406. In some embodiments, the offset value is an average of neighboring reconstructed samples above and / or to the left of the reference area 412 of the respective coding block 406. In some embodiments, the offset value is the top-left luma and chroma pixel value of the respective coding block 406. In some embodiments, the offset value is the middle / center pixel of one or more lines above in the reference area 412 of the respective coding block 406. In some embodiments, the offset value is the middle / center pixel of one or more lines to the left in the reference area 412 of the respective coding block 406. In some embodiments, after the luma samples 404C and 404x (e.g., N, S, C, W, E) are subtracted by the offset value, the offset luma sample values ​​404C and 404x are further clipped to a predefined range.

[0093]

[0101] In some embodiments, multi-CfL is applied during boundary processing, and instead of padding unavailable samples, only available samples are included in the multi-CfL calculation process. Referring to Figure 5B, one less column 502 is used at the right boundary. This boundary is one of a slice boundary, a CTU boundary, a superblock boundary, a tile boundary, and a picture boundary.

[0094]

[0102] In some embodiments, one or more left samples adjacent to the left boundary are unavailable, and the left boundary line is reduced (e.g., shifted to the right by one or more columns). In some embodiments, one or more top samples adjacent to the top boundary are unavailable, and the top boundary line is reduced (e.g., shifted down by one or more rows). In some embodiments, one or more right samples adjacent to the right boundary are unavailable, and the right boundary line is reduced (e.g., shifted left by one or more columns). In some embodiments, one or more bottom samples adjacent to the bottom boundary are unavailable, and the bottom boundary line is reduced (e.g., shifted up by one or more rows).

[0095]

[0103] In some embodiments, the filter applied to predict the first chroma sample 402C based on the first luma sample has a filter shape and several filter coefficients. The filter shape and the number of filter coefficients are optionally predefined or adaptively selected. Furthermore, in some embodiments, the filter is adaptively selected from multiple filters. The index of the selected filter is signaled in the video bitstream. In some embodiments, depending on the picture size, unavailable samples near the border are padded. As an example, referring to FIG. 6C, the predefined filter includes a center position P0 and eight peripheral positions P1-P8. The first chroma sample 402C is predicted as follows: predChrome=c0P0+c1P1+c2P2+c3P3+c4P4+c5P5+c6P6+c7P7+c8P8 (4) With at least eight luma chroma samples, the coefficients can be calculated in both the encoder 106 and the decoder 122. To determine the first chroma sample 402, an additional constant bias term, a bias term equal to the luma mean, or a higher order term is optionally applied.

[0096]

[0104] In some embodiments, depending on the picture size, unavailable samples near the border are padded. As an example, referring to Figure 6D or 6E, a predefined filter includes a center position P0 and six peripheral positions P1-P6. The first chroma sample 402C is predicted as follows: predChrome=c0P0+c1P1+c2P2+c3P3+c4P4+c5P5+c6P6 (5) Using at least six luma chroma samples, coefficients are calculated in both the encoder 106 and the decoder 122. To determine the first chroma sample 402, an additional constant bias term, a bias term equal to the luma mean, or a higher order term is optionally applied.

[0097]

[0105] In some embodiments, depending on the picture size, unavailable samples near the border are padded. As an example, referring to FIG. 6F, the predefined filter is a cross-shaped filter, including a center position P0 and eight peripheral positions P1-P8 arranged in a cross shape. The first chroma sample 402C is predicted as follows: predChrome=c0P0+c1P1+c2P2+c3P3+c4P4+c5P5+c6P6+c7P7+c8P8 (6) With at least eight luma chroma samples, the coefficients can be calculated in both the encoder 106 and the decoder 122. To determine the first chroma sample 402, an additional constant bias term, a bias term equal to the luma mean, or a higher order term is optionally applied.

[0098]

[0106] In some embodiments, one of the multiple filters is adaptively selected to generate the first chroma sample 402C from the first luma sample 404C and the associated neighboring luma sample 404x.

[0099]

[0107] In some embodiments, during chroma sample prediction, multiple higher-order terms are used to weight a subset of sample positions. For example, luma samples located at horizontal positions (including the center position) are quadratically related, while luma samples associated with vertical positions use linear terms. The first luma sample 404C(C), multiple neighboring luma samples 404x (e.g., N, S, W, E), and bias value B are combined to generate the first chroma component 402C(preChromaVal) based on equation 414 as follows: predChromaVal=c0C*C+c1N+c2S+c3E*E+c4W*W+c5B (7)

[0100]

[0108] In another example, vertical positions (including the center position) use quadratic terms, and horizontal positions use linear terms. The first luma sample 404C(C), multiple adjacent luma samples 404x (e.g., N, S, W, E), and bias value B are combined to generate the first chroma component 402C(preChromaVal) based on equation 414 as follows: predChromaVal=c0C*C+c1N*N+c2S*S+c3E+c4W+c5B (8)

[0101]

[0109] In some embodiments, the first chroma sample 402C is fully connected with all corresponding luma samples 404 (e.g., each coding block 406), and the downsampling operation occurs after the fully connected layer. The neural network coefficients of the fully connected layer are pre-trained and stored on both the encoder and decoder sides.

[0102]

[0110] 7 shows some logical stages in a particular order, but order-independent stages may be rearranged and other stages may be combined or separated. Any rearrangement or other grouping not specifically described will be apparent to one of ordinary skill in the art, and therefore the ordering and grouping presented herein is not exhaustive. Moreover, it should be recognized that the stages may be implemented in hardware, firmware, software, or any combination thereof.

[0103]

[0111] Reference will now be made to some exemplary embodiments.

[0104]

[0112] (A1) In some implementations, a method 700 for decoding video data is performed. The method 700 includes receiving a video bitstream including multiple different coding blocks of a current image frame (702). The video bitstream includes a syntax element for a cross-component intra prediction (CCIP) mode indicating whether each chroma component of the multiple different coding blocks is determined based on one or more luma samples (704). The method 700 further includes deriving a respective bias value for each coding block based on a set of one or more luma samples in a reference area of ​​the respective coding block in the current image frame (710). The reference area includes one or more coding blocks that are adjacent to the respective coding block and decoded prior to the respective coding block (712). The method further includes reconstructing the multiple coding blocks including the respective coding block based at least on the derived bias value (716).

[0105]

[0113] (A2) In some embodiments of A1, for each coding block, the set of one or more luma samples includes a single luma sample within a reference area of ​​the respective coding block in the current image frame.

[0106]

[0114] (A3) In some embodiments of A1, for each coding block, the respective bias value is equal to the average of a set of one or more luma samples including multiple luma samples within a reference area of ​​the respective coding block in the current image frame.

[0107]

[0115] (A4) In some embodiments of A1-A3, for each coding block, the set of one or more luma samples includes one or more rows of luma samples above the respective coding block in the reference area, and each bias value is equal to the median luma value of the one or more rows of luma samples.

[0108]

[0116] (A5) In some embodiments of A1-A3, for each coding block, the set of one or more luma samples includes one or more rows of luma samples above the respective coding block in the reference area, and the set of one or more luma samples includes a reference luma sample located approximately physically in the center of the one or more rows of luma samples, each bias value equal to the luma value of the reference luma sample.

[0109]

[0117] (A6) In some embodiments of A1-A3, for each coding block, the set of one or more luma samples includes one or more columns of luma samples located to the left of the respective coding block in the reference area, and each bias value is equal to the median luma value of the one or more columns of luma samples.

[0110]

[0118] (A7) In some embodiments of A1-A3, for each coding block, the set of one or more luma samples includes one or more columns of luma samples located to the left of the respective coding block in the reference area, and the set of one or more luma samples includes a reference luma sample located approximately physically in the center of the one or more columns of luma samples, each bias value equal to the luma value of the reference luma sample.

[0111]

[0119] (A8) In some embodiments of any of A1-A7, the method 700 further comprises identifying (706) a first chroma component of each coding block and a first luma component co-located with the first chroma component of each coding block; The method further includes identifying a plurality of neighboring luma components of the first luma component (708) and combining the first luma component, the plurality of neighboring luma components, and their respective bias values ​​to generate a first chroma component (714).

[0112]

[0120] (A9) In some embodiments of A8, combining the first luma sample, the plurality of adjacent luma samples, and the respective bias values ​​to generate the first chroma component further includes subtracting the first luma sample and each of the plurality of adjacent luma samples with an offset luma value. In some embodiments of A8, combining the first luma sample, the plurality of adjacent luma samples, and the respective bias values ​​to generate the first chroma component further includes clipping the first luma sample and each of the plurality of adjacent luma samples that have been subtracted with the offset luma value.

[0113]

[0121] (A10) In some embodiments of A8, the offset luma value is one of the following: an average of the first luma sample and multiple adjacent luma samples, an average of luma samples in an upper reference region of a reference area located above the respective coding block, an average of luma samples in a left reference region of a reference area located to the left of the respective coding block, a luma value of a single luma sample located in the upper left corner of the respective coding block, a central luma value of one or more rows of luma samples located above the respective coding block in the reference area, a luma value of a luma sample located approximately in the physical center of the one or more rows of luma samples, a central luma value of one or more columns of luma samples located to the left of the respective coding block in the reference area, and a luma value of a luma sample located approximately in the physical center of the one or more columns of luma samples.

[0114]

[0122] (A11) In some embodiments of A8, the video bitstream includes a respective filter index for each coding block of the plurality of different coding blocks. The method 700 further includes, for each coding block, adaptively selecting one of the plurality of different filters based on the respective filter index, each different filter having a respective filter shape and a plurality of weighting factors. The first luma sample, the plurality of neighboring luma samples, and the respective bias values ​​are combined using the plurality of weighting factors of the selected filter of at least the plurality of different filters.

[0115]

[0123] (A12) In some embodiments of A11, the plurality of different filters include one or more of the following: a first filter including nine weighting factors and configured to generate the first chroma component as a weighted combination of at least the first luma sample and eight neighboring luma samples that immediately surround the first luma sample; a second filter including seven weighting factors and configured to generate the first chroma component as a weighted combination of at least the first luma sample and six neighboring luma samples that immediately surround the first luma sample and are located symmetrically relative to the first luma sample; and a cross filter including 4k+1 weighting factors and configured to generate the first chroma component as a weighted combination of at least the first luma sample and k neighboring luma samples that are immediately adjacent to the first luma sample from each of two row directions and two column directions, where k is a positive integer.

[0116]

[0124] (A13) In some embodiments of A12, the first chroma component is generated further based on at least one of the following: a weighted and adjustable bias value, and at least one second-order or higher order of the first luma sample and a plurality of neighboring luma samples.

[0117]

[0125] (A14) In any of the embodiments of A1-A13, method 700 further includes, for each coding block, identifying a reference area for the respective coding block, including excluding a set of columns or rows immediately adjacent to a boundary from the reference area in accordance with a determination that one or more color components are not available in that set of columns or rows.

[0118]

[0126] (A15) In some embodiments of A14, the boundary includes one of a slice boundary, a superblock boundary, a tile boundary, a coding block boundary, and an image frame boundary.

[0119]

[0127] (A16) In some embodiments of any of A1-A15, method 700 further includes identifying a first chroma component of each coding block and a first luma component co-located with the first chroma component in the respective coding block; identifying multiple neighboring luma components of the first luma component; and combining the first luma component, the multiple neighboring luma components, and their respective bias values ​​to generate the first chroma component. Generating the first chroma component further includes applying a cross filter including 4k+1 weighting factors to combine the first luma sample, the multiple neighboring luma samples, and their respective bias values. Each of two row directions and two column directions corresponds to k neighboring luma samples immediately adjacent to the first luma sample. The k neighboring luma samples in each of the two row directions are combined into the first chroma component quadratically, and the k neighboring luma samples in each of the two column directions are linearly combined into the first chroma component.

[0120]

[0128] (A17) In some embodiments of any of A1 to A15, method 700 further includes identifying a first chroma component of each coding block and a first luma component co-located with the first chroma component in the respective coding block; identifying multiple neighboring luma components of the first luma component; and combining the first luma component, the multiple neighboring luma components, and their respective bias values ​​to generate the first chroma component. Generating the first chroma component further includes applying a cross filter including 4k+1 weighting factors to combine the first luma sample, the multiple neighboring luma samples, and their respective bias values. Each of two row directions and two column directions corresponds to k neighboring luma samples immediately adjacent to the first luma sample. The k neighboring luma samples in each of the two column directions are combined into the first chroma component quadratically, and the k neighboring luma samples in each of the two row directions are linearly combined into the first chroma component.

[0121]

[0129] (A18) In some embodiments of any of A1 to A17, for each coding block, the reference area of ​​each coding block includes one or more of a top-left reference region, a top reference region, a top-right reference region, a bottom-left reference region, and a left reference region.

[0122]

[0130] (A19) In some embodiments of A1-A18, the CCIP mode includes a multi-chroma-from-luma (multi-CfL) mode. Method 700 further includes determining that a plurality of different coding blocks are to be coded in the multi-CfL mode. In accordance with determining that each coding block is to be coded in the multi-CfL mode, a plurality of neighboring luma samples of a first luma sample of each coding block are identified based on a filter shape of a filter, and a first chroma component of each coding block is a weighted sum of the first luma sample, the plurality of neighboring luma samples, an output of a nonlinear function of the first luma sample, and a respective bias value.

[0123]

[0131] (A20) In some embodiments of A1-A19, the first luma sample, the plurality of adjacent luma samples, and their respective bias values ​​are combined using a plurality of weighting factors to generate the first chroma component, each weighting factor including an integer portion and a fractional portion.

[0124]

[0132] (A21) In some implementations, a method for encoding video data is implemented. The method includes encoding each respective coding block of a plurality of different coding blocks of a current image frame by identifying a first chroma component of the respective coding block and a first luma sample co-located with the first chroma component of the respective coding block, identifying multiple neighboring luma samples of the first luma sample, and deriving a respective bias value based on a set of one or more luma samples in a reference area of ​​the respective coding block in the current image frame. The reference area includes one or more coding blocks that are neighboring the respective coding block and decoded prior to the respective coding block. The method further includes generating a video bitstream including the plurality of different coding blocks of the current image frame. The video bitstream signals a cross-component intra prediction (CCIP) mode indicating that each chroma component of the plurality of different coding blocks is determined based on the one or more luma samples.

[0125]

[0133] In another aspect, some embodiments include a computing system (e.g., server system 112) including control circuitry (e.g., control circuitry 302) and a memory (e.g., memory 314) coupled to the control circuitry, wherein the memory stores one or more sets of instructions configured to be executed by the control circuitry, the one or more sets of instructions including instructions for performing any of the methods described herein (e.g., A1-A21 above).

[0126]

[0134] In yet another aspect, some embodiments include a non-transitory computer-readable storage medium storing one or more sets of instructions for execution by control circuitry of a computing system, the one or more sets of instructions including instructions for performing any of the methods described herein (e.g., A1-A21 above).

[0127]

[0135] The proposed methods can be used separately or combined in any order. Furthermore, each of the methods (or embodiments), the encoder, and the decoder can be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). For example, one or more processors execute a program stored in a non-transitory computer-readable medium. Hereinafter, the term block may be interpreted as a prediction block, a coding block, or a coding unit (CU).

[0128]

[0136] Although terms such as "first," "second," etc. may be used herein to describe various elements, it will be understood that these elements are not to be limited by these terms and are merely used to distinguish one element from another.

[0129]

[0137] The terminology used herein is for the purpose of describing particular embodiments only and does not limit the scope of the claims. As used in the description of these embodiments and in the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly dictates otherwise. It will also be understood that the term "and / or," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will further be understood that the terms "comprises" and / or "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0130]

[0138] The term "if," as used herein, may be interpreted to mean "when," or "upon," or "in response to determining," or "in accordance with a determination," or "in response to detecting" that a stated condition precedent is true, depending on the context. Similarly, the phrase "if determined [the stated condition precedent is true]," or "if [the stated condition precedent is true]," or "when [the stated condition precedent is true]," may be interpreted to mean "upon determining," or "in response to determining," or "in accordance with a determination," or "upon detecting," or "in response to detecting" that a stated condition precedent is true, depending on the context.

[0131]

[0139] The foregoing description has been set forth with reference to specific embodiments for purposes of explanation. However, the above exemplary description is not intended to be exhaustive or to limit the scope of the claims to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described in order to best explain the principles of operation and practical application, thereby enabling others skilled in the art.

Claims

1. 1. A method for decoding video data, comprising: receiving a video bitstream including a plurality of coding blocks of a current image frame, the video bitstream including a syntax element for a cross-component intra prediction (CCIP) mode indicating whether each chroma component of the plurality of different coding blocks is determined based on one or more luma components; deriving a respective bias value based on a set of one or more luma components in a reference area of ​​each coding block in the current image frame, the reference area including one or more coding blocks that are adjacent to and decoded prior to the respective coding block; reconstructing the plurality of coding blocks including the respective coding blocks based at least on the derived bias values; A method comprising:

2. The method of claim 1 , wherein, for each coding block, the set of one or more luma components includes a single luma component within the reference area of ​​the respective coding block in the current image frame.

3. 2. The method of claim 1 , wherein, for the respective coding block, the respective bias value is equal to an average of the set of one or more luma components comprising multiple luma components in the reference area of ​​the respective coding block in the current image frame.

4. 2. The method of claim 1 , wherein, for the respective coding block, the set of one or more luma components includes luma components of one or more rows above the respective coding block in the reference area, and the respective bias value is equal to a median luma value of the luma components of the one or more rows.

5. 2. The method of claim 1 , wherein, for the respective coding blocks, the set of one or more luma components includes luma components of one or more rows above the respective coding block in the reference area, the set of one or more luma components includes a reference luma component located approximately physically in the center of the one or more rows of luma components, and the respective bias values ​​are equal to the luma value of the reference luma component.

6. 2. The method of claim 1 , wherein, for the respective coding blocks, the set of one or more luma components includes luma components of one or more columns located to the left of the respective coding block in the reference area, and the respective bias values ​​are equal to a median luma value of the luma components of the one or more columns.

7. 2. The method of claim 1 , wherein, for the respective coding blocks, the set of one or more luma components includes one or more columns of luma components located to the left of the respective coding block in the reference area, the set of one or more luma components includes a reference luma component located approximately physically in the center of the one or more columns of luma components, and the respective bias values ​​are equal to the luma value of the reference luma component.

8. identifying a first chroma component of the respective coding block and a first luma component co-located with the first chroma component of the respective coding block; identifying a plurality of neighboring luma components of the first luma component; combining the first luma component, the plurality of adjacent luma components, and the respective bias values ​​to generate a first chroma component; The method of claim 1 further comprising:

9. combining the first luma component, the plurality of adjacent luma components, and the respective bias values ​​to generate the first chroma component; subtracting the first luma component and each of the plurality of adjacent luma components by an offset luma value; The method of claim 8 further comprising:

10. combining the first luma component, the plurality of adjacent luma components, and the respective bias values ​​to generate the first chroma component; clipping the first luma component subtracted by the offset luma value and each of the plurality of adjacent luma components; 10. The method of claim 9, further comprising:

11. The offset luma value is an average of the first luma component and the plurality of adjacent luma components; an average of luma components in an upper reference region of the reference area located above each coding block; an average of luma components in a left reference region of the reference area located to the left of each coding block; a luma value of a single luma component located at the upper left corner of each of the coding blocks; a median luma value of one or more rows of luma components located above the respective coding block in the reference area; a luma value of a luma component located approximately in the physical center of said one or more rows of luma components; a median luma value of luma components in one or more columns located to the left of the respective coding block in the reference area; and a luma value of a luma component located approximately in the physical center of the one or more columns of luma components; 10. The method of claim 9, wherein the

12. 1. A computing system comprising: a control circuit; a memory storing one or more programs configured to be executed by the control circuitry; wherein the one or more programs are receiving a video bitstream including a plurality of different coding blocks of a current image frame, the video bitstream including a syntax element for a cross-component intra prediction (CCIP) mode indicating that each chroma component of the plurality of different coding blocks is determined based on one or more luma components; deriving a respective bias value based on a set of one or more luma components in a reference area of ​​each coding block in the current image frame, the reference area including one or more coding blocks that are adjacent to and decoded prior to the respective coding block; reconstructing the plurality of coding blocks including the respective coding blocks based at least on the derived bias values; 20. The computing system of claim 19, further comprising instructions for:

13. the one or more programs: identifying the reference area for each coding block, including excluding a set of columns or rows from the reference area in accordance with a determination that the one or more color components are not available in a set of columns or rows immediately adjacent to a boundary for the respective coding block; The computing system of claim 12 further comprising instructions for:

14. The computing system of claim 13 , wherein the boundary comprises one of a slice boundary, a superblock boundary, a tile boundary, a coding block boundary, and an image frame boundary.

15. the one or more programs: identifying a first chroma component of the respective coding block and a first luma component co-located with the first chroma component of the respective coding block; identifying a plurality of neighboring luma components of the first luma component; combining the first luma component, the plurality of adjacent luma components, and the respective bias values ​​to generate the first chroma component; The computing system of claim 12 further comprising instructions for:

16. the video bitstream including a respective filter index for each coding block of the plurality of different coding blocks, and the one or more programs, for each coding block: adaptively selecting one of a plurality of different filters based on the respective filter index, each different filter having a respective filter shape and a plurality of weighting factors, and wherein the first luma component, the plurality of adjacent luma components, and the respective bias values ​​are combined using at least the plurality of weighting factors of the selected filter of the plurality of different filters. The computing system of claim 15 further comprising instructions for:

17. the plurality of different filters: a first filter including nine weighting factors and configured to generate the first chroma component as a weighted combination of at least the first luma component and eight neighboring luma components immediately surrounding the first luma component; a second filter including seven weighting factors and configured to generate the first chroma component as a weighted combination of at least the first luma component and six neighboring luma components that immediately surround and are symmetrically positioned relative to the first luma component; and a cross filter including 4k+1 weighting factors and configured to generate the first chroma component as a weighted combination of at least the first luma component and k immediately neighboring luma components of the first luma component from each of two row directions and two column directions, where k is a positive integer; 17. The computing system of claim 16, comprising one or more of:

18. 20. The computing system of claim 17, wherein the first chroma component is generated further based on at least one of a weighted, adjustable bias value and second or higher order ones of the first luma component and at least one of the plurality of adjacent luma components.

19. 1. A non-transitory computer-readable storage medium storing one or more programs for execution by control circuitry of a computing system, the one or more programs comprising: receiving a video bitstream including a plurality of different coding blocks of a current image frame, the video bitstream including a syntax element for a cross-component intra prediction (CCIP) mode indicating that each chroma component of the plurality of different coding blocks is determined based on one or more luma components; deriving a respective bias value based on a set of one or more luma components in a reference area of ​​each coding block in the current image frame, the reference area including one or more coding blocks that are adjacent to and decoded prior to the respective coding block; reconstructing the plurality of coding blocks including the respective coding blocks based at least on the derived bias values; A non-transitory computer-readable storage medium containing instructions for:

20. the one or more programs: identifying a first chroma component of the respective coding block and a first luma component co-located with the first chroma component of the respective coding block; identifying a plurality of neighboring luma components of the first luma component; combining the first luma component, the plurality of adjacent luma components, and the respective bias values ​​to generate the first chroma component; further comprising instructions for generating the first chroma component further includes applying a cross filter including 4k+1 weighting factors to combine the first luma component, the plurality of adjacent luma components, and the respective bias values; each of two rows and two columns corresponding to k immediate neighboring luma components of the first luma component; 20. The non-transitory computer-readable storage medium of claim 19, wherein the k adjacent luma components in each of the two row directions are combined with the first chroma component quadratically, and the k adjacent luma components in each of the two column directions are combined with the first chroma component linearly.

21. the one or more programs: identifying a first chroma component of the respective coding block and a first luma component co-located with the first chroma component of the respective coding block; identifying a plurality of neighboring luma components of the first luma component; combining the first luma component, the plurality of adjacent luma components, and the respective bias values ​​to generate the first chroma component; further comprising instructions for generating the first chroma component further includes applying a cross filter including 4k+1 weighting factors to combine the first luma component, the plurality of adjacent luma components, and the respective bias values; each of two rows and two columns corresponding to k immediate neighboring luma components of the first luma component; 20. The non-transitory computer-readable storage medium of claim 19, wherein the k adjacent luma components in each of the two column directions are combined with the first chroma component quadratically and the k adjacent luma components in each of the two row directions are combined with the first chroma component linearly.

22. 20. The non-transitory computer-readable storage medium of claim 19, wherein the reference area of ​​the respective coding block comprises one or more of a top-left reference region, a top reference region, a top-right reference region, a bottom-left reference region, and a left reference region.

Images