Adaptive blending for cross-component prediction

Abstract

A luma prediction block of a luma block is obtained. An initial chroma prediction block of a chroma block corresponding to the luma block is obtained. Filter coefficients are derived based on the luma prediction block and the initial chroma prediction block. A luma blending component is obtained by applying the filter coefficients to a reconstructed luma block that is obtained by adding the luma prediction block to a luma residual block. A final chroma prediction block is obtained by blending, based on a blending mode, the luma blending component with the initial chroma prediction block. The blending is performed by combining, pixel-location wise, the luma blending component and the initial chroma prediction block using a weighted average using blending weights specified by the blending mode. The chroma block is reconstructed by adding the final chroma prediction block to a chroma residual block.

Description

ADAPTIVE BLENDING FOR CROSS-COMPONENT PREDICTIONBACKGROUND

[0001] Digital video streams may represent video using a sequence of frames or still images. Digital video can be used for various applications including, for example, video conferencing, high-definition video entertainment, video advertisements, or sharing of usergenerated videos. A digital video stream can contain a large amount of data and consume a significant amount of computing or communication resources of a computing device for processing, transmission, or storage of the video data. Various approaches have been proposed to reduce the amount of data in video streams, including encoding or decoding techniques.SUMMARY

[0002] An aspect of the disclosed implementation is a method for decoding a block that includes a luma block and a chroma block. The method includes obtaining a luma prediction block of the luma block; obtaining an initial chroma prediction block of the chroma block corresponding to the luma block; deriving filter coefficients based on the luma prediction block and the initial chroma prediction block; obtaining a luma blending component by applying the filter coefficients to a reconstructed luma block that is obtained by adding the luma prediction block to a luma residual block; obtaining a final chroma prediction block by blending, based on a blending mode, the luma blending component with the initial chroma prediction block, where the blending is performed by combining, pixel-location wise, the luma blending component and the initial chroma prediction block using a weighted average using weights specified by the blending mode; and reconstructing the chroma block by adding the final chroma prediction block to a chroma residual block.

[0003] In certain implementations, the filter coefficients are derived to minimize a mean square within a reference area. The method may further involve decoding the blending mode from a compressed bitstream, where the blending mode may be decoded from a header of the chroma block or a header of the block. The method may also include decoding a maximal blending weighting index and a blending index indicative of the blending mode, wherein the blending index is less than or equal to the maximal blending weighting index.

[0004] In some implementations, a first blending weight used for samples close to a topleft comer of the luma blending component is not smaller than a second blending weight used for samples close to a bottom-right comer of the luma blending component. The blending weights may be derived conditionally without signaling in a compressed bitstream based on an absence of luma residues for the luma block while cross-component inter-prediction is signaled as on for the block, resulting in a derived blending factor of 1 for pure crosscomponent prediction.

[0005] The blending mode may indicate that same weights are used for each pixel location, or alternatively, that the blending weights used for combining values at corresponding locations are based on the location relative to a top-left comer of the block. The method may further include determining not to use cross -component prediction in conjunction with transform skip, wherein if inter cross-component prediction is on for the block, a transform skip flag for the chroma block is not signaled but derived as off for the block.

[0006] Another aspect of the disclosed implementation is a non-transitory computer- readable storage medium having stored thereon an encoded bitstream. The encoded bitstream includes a luma residual block for a luma block, a chroma residual block for a chroma block corresponding to the luma block, and a blending mode indicator specifying blending weights for blending a luma blending component with an initial chroma prediction block. The luma blending component is derivable by applying filter coefficients to a reconstructed luma block obtained by adding a luma prediction block to the luma residual block. The filter coefficients are derivable based on the luma prediction block and the initial chroma prediction block. A final chroma prediction block is obtainable by combining, pixel-location wise, the luma blending component and the initial chroma prediction block based on the blending weights specified by the blending mode indicator. The chroma block is reconstructable by adding the final chroma prediction block to the chroma residual block during decoding.

[0007] In some implementations, the encoded bitstream further includes a maximal blending weighting index that defines an upper limit for a blending index specifying the blending weights. The maximal blending weighting index may be encoded at a level no lower than a slice, such as in a sequence parameter set, a picture parameter set, a frame header, or a slice header. The blending mode indicator may include a blending index that is less than or equal to the maximal blending weighting index, indicating specific weights for the luma blending component and the initial chroma prediction block.

[0008] The blending index may be selectable from a set of values including a first value indicating no blending, a second value indicating a predetermined weight pair, and a third value indicating a pure cross -component blending mode. The blending weights may be derivable without signaling when the luma residual block contains no residues and crosscomponent inter-prediction is signaled as on.

[0009] The blending mode indicator may specify uniform weights for all pixel locations or a non-uniform blending mode in which weights vary based on pixel location, such as a gradational blending scheme transitioning from higher values near the top-left comer to lower values near the bottom-right comer. The encoded bitstream may include syntax indicating that transform skip is not used with inter cross-component prediction, and may further include a flag for inter cross-component prediction merge mode, where the blending mode indicator is implicitly selected from a merge list when the flag is set.

[0010] These and other aspects of the present disclosure are disclosed in the following detailed description of the embodiments, the appended claims and the accompanying figures.

[0011] It will be appreciated that aspects can be implemented in any convenient form. For example, aspects may be implemented by appropriate computer programs which may be carried on appropriate carrier media which may be tangible carrier media (e.g. disks) or intangible carrier media (e.g. communications signals). Aspects may also be implemented using suitable apparatus which may take the form of programmable computers running computer programs arranged to implement the methods and / or techniques disclosed herein. Aspects can be combined such that features described in the context of one aspect may be implemented in another aspect.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The description herein makes reference to the accompanying drawings described below, wherein like reference numerals refer to like parts throughout the several views.

[0013] FIG. 1 is a schematic of a video encoding and decoding system.

[0014] FIG. 2 is a block diagram of an example of a computing device that can implement a transmitting station or a receiving station.

[0015] FIG. 3 is a diagram of a typical video stream to be encoded and subsequently decoded.

[0016] FIG. 4 is a block diagram of an encoder according to implementations of this disclosure.

[0017] FIG. 5 is a block diagram of a decoder according to implementations of this disclosure.

[0018] FIG. 6 is a block diagram of a representation of a portion of a frame according to implementations of this disclosure.

[0019] FIG. 7 A illustrates an example of cross-component prediction mode referred to as convolutional cross -component model (CCCM) mode.

[0020] FIG. 7B illustrates an example of cross-component prediction mode referred to as cross-component linear model (CCLM) mode.

[0021] FIG. 7C illustrates an example of cross-component prediction mode referred to as Inter CCCM mode.

[0022] FIG. 8 is a flowchart of an example of a technique for adaptive blending for crosscomponent prediction.

[0023] FIG. 9 illustrates an example of an encoded bitstream that illustrates adaptive blending for cross-component prediction.DETAILED DESCRIPTION

[0024] Video compression schemes may include breaking respective images, or frames, of a video stream into smaller portions, such as blocks, or coding tree units (CTUs), and generating an encoded bitstream using techniques to limit the information included for respective CTUs thereof. The bitstream can be decoded to re-create the source frames from the limited information. Encoding CTUs to or decoding CTUs from a bitstream can include predicting the values of pixels or CTUs based on similarities with other pixels or CTUs in the same frame which have already been coded. Those similarities can be determined using intra prediction, which attempts to predict the pixel values of a coding unit (CU) of a CTU using pixels peripheral to the CU (e.g., pixels that are in the same frame as the CU, but which are outside the CU).

[0025] A CU includes a luminance, also referred to as luma, component and two chrominance, also referred to as chroma, components. These luma and chroma components may in some case be referred to as a luma block and chroma blocks. The luma component of a CU may, for example, be expressed within a Y plane of the CU and the chroma components may be expressed either within U and V planes or Cr and Cb planes of the CU. The luma component is understood to include some number of luma samples and each chroma component is understood to include some number of chroma samples. Generally, the luma samples provide measures of brightness throughout a subject CU and thus represents thestructural qualities of the video content of the subject CU, whereas the chroma samples provide measures of color throughout the subject CU. Because of this, conventional video compression schemes often use finer prediction approaches for predicting luma components of CUs than for chroma components thereof. Such schemes may also use approaches directed to predicting those chroma components from the predicted luma components.

[0026] To reduce the cross-component redundancy, several models for predicting chroma blocks from luma blocks have been developed. Due to the high correlation between luma and chroma, cross-component prediction can be more efficient than chroma only prediction in some cases, such as when a block is rich in textures. Examples of such cross-component prediction modes are described herein, such as those described with respect to FIGS. 7A-7C. One such mode, described with respect to FIG. 7C, is the Inter Convolutional crosscomponent (Inter CCCM) mode. Briefly, in the Inter CCCM mode, cross -component filters are derived using the prediction blocks of luma and chroma. The derived filters are applied to the reconstructed luma block and blended with the prediction blocks of chroma to produce the final chroma prediction blocks. In the blending process of the Inter CCCM mode, fixed blending patterns are used. For example, in the blending process, the filtered reconstructed luma blocks use a blending weight of 0.75 and chroma prediction blocks use a blending weight of 0.25. However, this fixed blending is not always efficient.

[0027] Implementations according to this disclosure improve upon the fixed blending patterns of Inter CCCM. Adaptive blending strategies for combining the filtered reconstructed luma blocks with chroma prediction blocks that produce chroma prediction blocks with higher efficiency than the fixed blending approach are described. For example, blending weights may be dynamically adjusted to differ from block to block. For example, different blending patterns can be used: uniform, where each sample within a block gets the same weight; and non-uniform, where the weight changes depending on the sample's position in the block. The choice of blending pattern can be signaled in the bitstream, such as in the form of a blending index. The blending indices may range from indicating no blending to full cross-component blending and can also signify non-uniform blending. Additionally, in some cases, the blending weights may be inferred without signaling. For example, a weight of 1 may be used for blocks without luma residue.

[0028] Further details of techniques for adaptive blending for cross-component prediction are described herein with initial reference to a system in which they can be implemented. FIG. 1 is a schematic of a video encoding and decoding system 100. A transmitting station 102 can be, for example, a computer having an internal configuration of hardware such asthat described in FIG. 2. However, other implementations of the transmitting station 102 are possible. For example, the processing of the transmitting station 102 can be distributed among multiple devices.

[0029] A network 104 can connect the transmitting station 102 and a receiving station 106 for encoding and decoding of the video stream. Specifically, the video stream can be encoded in the transmitting station 102, and the encoded video stream can be decoded in the receiving station 106. The network 104 can be, for example, the Internet. The network 104 can also be a local area network (LAN), wide area network (WAN), virtual private network (VPN), cellular telephone network, or any other means of transferring the video stream from the transmitting station 102 to, in this example, the receiving station 106.

[0030] The receiving station 106, in one example, can be a computer having an internal configuration of hardware such as that described in FIG. 2. However, other suitable implementations of the receiving station 106 are possible. For example, the processing of the receiving station 106 can be distributed among multiple devices.

[0031] Other implementations of the video encoding and decoding system 100 are possible. For example, an implementation can omit the network 104. In another implementation, a video stream can be encoded and then stored for transmission at a later time to the receiving station 106 or any other device having memory. In one implementation, the receiving station 106 receives (e.g., via the network 104, a computer bus, and / or some communication pathway) the encoded video stream and stores the video stream for later decoding. In an example implementation, a real-time transport protocol (RTP) is used for transmission of the encoded video over the network 104. In another implementation, a transport protocol other than RTP may be used, e.g., a video streaming protocol based on the Hypertext Transfer Protocol (HTTP).

[0032] When used in a video conferencing system, for example, the transmitting station 102 and / or the receiving station 106 may include the ability to both encode and decode a video stream as described below. For example, the receiving station 106 could be a video conference participant who receives an encoded video bitstream from a video conference server (e.g., the transmitting station 102) to decode and view and further encodes and transmits his or her own video bitstream to the video conference server for decoding and viewing by other participants.

[0033] FIG. 2 is a block diagram of an example of a computing device 200 that can implement a transmitting station or a receiving station. For example, the computing device 200 can implement one or both of the transmitting station 102 and the receiving station 106of FIG. 1. The computing device 200 can be in the form of a computing system including multiple computing devices, or in the form of one computing device, for example, a mobile phone, a tablet computer, a laptop computer, a notebook computer, a desktop computer, and the like.

[0034] A processor 202 in the computing device 200 can be a conventional central processing unit. Alternatively, the processor 202 can be another type of device, or multiple devices, capable of manipulating or processing information now existing or hereafter developed. For example, although the disclosed implementations can be practiced with one processor as shown (e.g., the processor 202), advantages in speed and efficiency can be achieved by using more than one processor.

[0035] A memory 204 in computing device 200 can be a read only memory (ROM) device or a random access memory (RAM) device in an implementation. However, other suitable types of storage device can be used as the memory 204. The memory 204 can include code and data 206 that is accessed by the processor 202 using a bus 212. The memory 204 can further include an operating system 208 and application programs 210, the application programs 210 including at least one program that permits the processor 202 to perform the techniques described herein. For example, the application programs 210 can include applications 1 through N, which further include a video coding application that performs the techniques described herein. The computing device 200 can also include a secondary storage 214, which can, for example, be a memory card used with a mobile computing device.Because the video communication sessions may contain a significant amount of information, they can be stored in whole or in part in the secondary storage 214 and loaded into the memory 204 as needed for processing.

[0036] The computing device 200 can also include one or more output devices, such as a display 218. The display 218 may be, in one example, a touch sensitive display that combines a display with a touch sensitive element that is operable to sense touch inputs. The display 218 can be coupled to the processor 202 via the bus 212. Other output devices that permit a user to program or otherwise use the computing device 200 can be provided in addition to or as an alternative to the display 218. When the output device is or includes a display, the display can be implemented in various ways, including by a liquid crystal display (LCD), a cathode-ray tube (CRT) display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0037] The computing device 200 can also include or be in communication with an image-sensing device 220, for example, a camera, or any other image- sensing device 220now existing or hereafter developed that can sense an image such as the image of a user operating the computing device 200. The image- sensing device 220 can be positioned such that it is directed toward the user operating the computing device 200. In an example, the position and optical axis of the image-sensing device 220 can be configured such that the field of vision includes an area that is directly adjacent to the display 218 and from which the display 218 is visible.

[0038] The computing device 200 can also include or be in communication with a soundsensing device 222, for example, a microphone, or any other sound-sensing device now existing or hereafter developed that can sense sounds near the computing device 200. The sound- sensing device 222 can be positioned such that it is directed toward the user operating the computing device 200 and can be configured to receive sounds, for example, speech or other utterances, made by the user while the user operates the computing device 200.

[0039] Although FIG. 2 depicts the processor 202 and the memory 204 of the computing device 200 as being integrated into one unit, other configurations can be utilized. The operations of the processor 202 can be distributed across multiple machines (wherein individual machines can have one or more processors) that can be coupled directly or across a local area or other network. The memory 204 can be distributed across multiple machines such as a network-based memory or memory in multiple machines performing the operations of the computing device 200. Although depicted here as one bus, the bus 212 of the computing device 200 can be composed of multiple buses. Further, the secondary storage 214 can be directly coupled to the other components of the computing device 200 or can be accessed via a network and can comprise an integrated unit such as a memory card or multiple units such as multiple memory cards. The computing device 200 can thus be implemented in a wide variety of configurations.

[0040] FIG. 3 is a diagram of an example of a video stream 300 to be encoded and subsequently decoded. The video stream 300 includes a video sequence 302. At the next level, the video sequence 302 includes a number of adjacent frames 304. While three frames are depicted as the adjacent frames 304, the video sequence 302 can include any number of adjacent frames 304. The adjacent frames 304 can then be further subdivided into individual frames, for example, a frame 306. At the next level, the frame 306 can be divided into a series of planes or segments 308. The segments 308 can be subsets of frames that permit parallel processing, for example. The segments 308 can also be subsets of frames that can separate the video data into separate colors. For example, a frame 306 of color video data can includea luminance plane and two chrominance planes. The segments 308 may be sampled at different resolutions.

[0041] Whether or not the frame 306 is divided into segments 308, the frame 306 may be further subdivided into blocks 310, which can contain data corresponding to, for example, 16x16 pixels in the frame 306. The blocks 310 can also be arranged to include data from one or more segments 308 of pixel data. The blocks 310 can also be of any other suitable size such as 4x4 pixels, 8x8 pixels, 16x8 pixels, 8x16 pixels, 16x16 pixels, or larger. Unless otherwise noted, the terms block and macroblock are used interchangeably herein.

[0042] FIG. 4 is a block diagram of an encoder 400 according to implementations of this disclosure. The encoder 400 can be implemented, as described above, in the transmitting station 102, such as by providing a computer software program stored in memory, for example, the memory 204. The computer software program can include machine instructions (i.e., executable instructions) that, when executed by a processor such as the processor 202, cause the transmitting station 102 to encode video data in the manner described in FIG. 4. The encoder 400 can also be implemented as specialized hardware included in, for example, the transmitting station 102. In one particularly desirable implementation, the encoder 400 is a hardware encoder.

[0043] The encoder 400 has the following stages to perform the various functions in a forward path (shown by the solid connection lines) to produce an encoded or compressed bitstream 420 using the video stream 300 as input: an intra / inter prediction stage 402, a transform stage 404, a quantization stage 406, and an entropy encoding stage 408. The encoder 400 may also include a reconstruction path (shown by the dotted connection lines) to reconstruct a frame for encoding of future blocks. In FIG. 4, the encoder 400 has the following stages to perform the various functions in the reconstruction path: a dequantization stage 410, an inverse transform stage 412, a reconstruction stage 414, and a loop filtering stage 416. Other structural variations of the encoder 400 can be used to encode the video stream 300.

[0044] When the video stream 300 is presented for encoding, respective adjacent frames 304, such as the frame 306, can be processed in units of blocks. At the intra / inter prediction stage 402, respective blocks can be encoded using intra-frame prediction (also called intraprediction) or inter- frame prediction (also called inter-prediction). In any case, a prediction block can be formed. In the case of intra-prediction, a prediction block may be formed from samples in the current frame that have been previously encoded and reconstructed. In the caseof inter-prediction, a prediction block may be formed from samples in one or more previously constructed reference frames.

[0045] Next, the prediction block can be subtracted from the current block at the intra / inter prediction stage 402 to produce a residual block (also called a residual). The transform stage 404 transforms the residual into transform coefficients in, for example, the frequency domain using block-based transforms. The quantization stage 406 converts the transform coefficients into discrete quantum values, which are referred to as quantized transform coefficients, using a quantizer value or a quantization level. For example, the transform coefficients may be divided by the quantizer value and truncated.

[0046] The quantized transform coefficients are then entropy encoded by the entropy encoding stage 408. The entropy-encoded coefficients, together with other information used to decode the block (which may include, for example, syntax elements such as used to indicate the type of prediction used, transform type, motion vectors, a quantizer value, or the like), are then output to the compressed bitstream 420. The compressed bitstream 420 can be formatted using various techniques, such as variable length coding (VLC) or arithmetic coding. The compressed bitstream 420 can also be referred to as an encoded video stream or encoded video bitstream, and the terms will be used interchangeably herein.

[0047] The reconstruction path (shown by the dotted connection lines) can be used to ensure that the encoder 400 and a decoder 500 (described below with respect to FIG. 5) use the same reference frames to decode the compressed bitstream 420. The reconstruction path performs functions that are similar to functions that take place during the decoding process (described below with respect to FIG. 5), including dequantizing the quantized transform coefficients at the dequantization stage 410 and inverse transforming the dequantized transform coefficients at the inverse transform stage 412 to produce a derivative residual block (also called a derivative residual). At the reconstruction stage 414, the prediction block that was predicted at the intra / inter prediction stage 402 can be added to the derivative residual to create a reconstructed block. The loop filtering stage 416 can be applied to the reconstructed block to reduce distortion such as blocking artifacts.

[0048] Other variations of the encoder 400 can be used to encode the compressed bitstream 420. In some implementations, a non-transform based encoder can quantize the residual signal directly without the transform stage 404 for certain blocks or frames. In some implementations, an encoder can have the quantization stage 406 and the dequantization stage 410 combined in a common stage.

[0049] FIG. 5 is a block diagram of a decoder 500 according to implementations of this disclosure. The decoder 500 can be implemented in the receiving station 106, for example, by providing a computer software program stored in the memory 204. The computer software program can include machine instructions that, when executed by a processor such as the processor 202, cause the receiving station 106 to decode video data in the manner described in FIG. 5. The decoder 500 can also be implemented in hardware included in, for example, the transmitting station 102 or the receiving station 106.

[0050] The decoder 500, similar to the reconstruction path of the encoder 400 discussed above, includes in one example the following stages to perform various functions to produce an output video stream 516 from the compressed bitstream 420: an entropy decoding stage 502, a dequantization stage 504, an inverse transform stage 506, an intra / inter prediction stage 508, a reconstruction stage 510, a loop filtering stage 512, and a deblocking filtering stage 514. Other structural variations of the decoder 500 can be used to decode the compressed bitstream 420.

[0051] When the compressed bitstream 420 is presented for decoding, the data elements within the compressed bitstream 420 can be decoded by the entropy decoding stage 502 to produce a set of quantized transform coefficients. The dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by the quantizer value), and the inverse transform stage 506 inverse transforms the dequantized transform coefficients to produce a derivative residual that can be identical to that created by the inverse transform stage 412 in the encoder 400. Using header information decoded from the compressed bitstream 420, the decoder 500 can use the intra / inter prediction stage 508 to create the same prediction block as was created in the encoder 400 (e.g., at the intra / inter prediction stage 402).

[0052] At the reconstruction stage 510, the prediction block can be added to the derivative residual to create a reconstructed block. The loop filtering stage 512 can be applied to the reconstructed block to reduce blocking artifacts. Other filtering can be applied to the reconstructed block. In this example, the deblocking filtering stage 514 is applied to the reconstructed block to reduce blocking distortion, and the result is output as the output video stream 516. The output video stream 516 can also be referred to as a decoded video stream, and the terms will be used interchangeably herein. Other variations of the decoder 500 can be used to decode the compressed bitstream 420. In some implementations, the decoder 500 can produce the output video stream 516 without the deblocking filtering stage 514.

[0053] FIG. 6 is a block diagram of a representation of a portion 600 of a frame, such as the frame 306 of FIG. 3, according to implementations of this disclosure. As shown, the portion 600 of the frame includes four 64x64 blocks 610, which may be referred to as superblocks, in two rows and two columns in a matrix or Cartesian plane. A superblock can have a larger or a smaller size. For example, a superblock can be 128x128. A superblock can also be referred to as a coding tree block (CTB). While FIG. 6 is explained with respect to a superblock of size 64x64, the description is easily extendable to larger (e.g., 128x128) or smaller superblock sizes.

[0054] In an example, a superblock can be a basic or maximum coding unit (CU). Each superblock can include four 32x32 blocks 620. Each 32x32 block 620 can include four 16x16 blocks 630. Each 16x16 block 630 can include four 8x8 blocks 640. Each 8x8 block 640 can include four 4x4 blocks 650. Each 4x4 block 650 can include 16 pixels, which can be represented in four rows and four columns in each respective block in the Cartesian plane or matrix. The pixels can include information representing an image captured in the frame, such as luminance information, color information, and location information. In an example, a block, such as a 16xl6-pixel block as shown, can include a luminance block 660, which can include luminance pixels 662; and two chrominance blocks 670 / 680, such as a U or Cb chrominance block 670, and a V or Cr chrominance block 680. The chrominance blocks 670 / 680 can include chrominance pixels 690. For example, the luminance block 660 can include 16x16 luminance pixels 662, and each chrominance block 670 / 680 can include 8x8 chrominance pixels 690, as shown. Although one arrangement of blocks is shown, any arrangement can be used. Although FIG. 6 shows NxN blocks, in some implementations, NxM, where N^M, blocks can be used. For example, 32x64 blocks, 64x32 blocks, 16x32 blocks, 32x16 blocks, or any other size blocks can be used. In some implementations, Nx2N blocks, 2NxN blocks, or a combination thereof can be used.

[0055] In some implementations, video coding can include ordered block-level coding. Ordered block-level coding can include coding blocks of a frame in an order, such as rasterscan order, wherein blocks can be identified and processed starting with a block in the upper left comer of the frame, or a portion of the frame, and proceeding along rows from left to right and from the top row to the bottom row, identifying each block in turn for processing. For example, the superblock in the top row and left column of a frame can be the first block coded, and the superblock immediately to the right of the first block can be the second block coded. The second row from the top can be the second row coded, such that the superblock inthe left column of the second row can be coded after the superblock in the rightmost column of the first row.

[0056] In an example, coding a block can include using quad-tree coding, which can include coding smaller block units with a block in raster-scan order. The 64x64 superblock shown in the bottom-left corner of the portion of the frame shown in FIG. 6, for example, can be coded using quad-tree coding in which the top-left 32x32 block can be coded, then the top-right 32x32 block can be coded, then the bottom-left 32x32 block can be coded, and then the bottom-right 32x32 block can be coded. Each 32x32 block can be coded using quad-tree coding in which the top-left 16x16 block can be coded, then the top-right 16x16 block can be coded, then the bottom-left 16x16 block can be coded, and then the bottom-right 16x16 block can be coded. Each 16x16 block can be coded using quad-tree coding in which the top-left 8x8 block can be coded, then the top-right 8x8 block can be coded, then the bottom-left 8x8 block can be coded, and then the bottom-right 8x8 block can be coded. Each 8x8 block can be coded using quad-tree coding in which the top-left 4x4 block can be coded, then the topright 4x4 block can be coded, then the bottom-left 4x4 block can be coded, and then the bottom-right 4x4 block can be coded. In some implementations, 8x8 blocks can be omitted for a 16x16 block, and the 16x16 block can be coded using quad-tree coding in which the top-left 4x4 block can be coded, and then the other 4x4 blocks in the 16x16 block can be coded in raster-scan order.

[0057] In an example, video coding can include compressing the information included in an original, or input, frame by omitting some of the information in the original frame from a corresponding encoded frame. For example, coding can include reducing spectral redundancy, reducing spatial redundancy, reducing temporal redundancy, or a combination thereof.

[0058] In an example, reducing spectral redundancy can include using a color model based on a luminance component (Y) and two chrominance components (U and V or Cb and Cr), which can be referred to as the YUV or YCbCr color model or color space. Using the YUV color model can include using a relatively large amount of information to represent the luminance component of a portion of a frame and using a relatively small amount of information to represent each corresponding chrominance component for the portion of the frame. For example, a portion of a frame can be represented by a high-resolution luminance component, which can include a 16x16 block of pixels, and by two lower resolution chrominance components, each of which representing the portion of the frame as an 8x8 block of pixels. A pixel can indicate a value (e.g., a value in the range from 0 to 255) and canbe stored or transmitted using, for example, eight bits. Although this disclosure is described with reference to the YUV color model, any color model can be used.

[0059] FIGS. 7A-7C describe examples of cross-component prediction modes. Each of cross-component prediction modes reduces redundancy among color components, such as the luminance, or luma, component (Y), and the two chrominance, or chroma, components (U and V or Cb and Cr).

[0060] FIG. 7A illustrates an example 700 of cross-component prediction mode referred to as CCCM mode. The example 700 illustrates pixels that are filled with different patterns. The example 700 is used to illustrate a current block template and is also used to illustrate a reference block template.

[0061] When used to describe a current block template, a block 708 illustrates a current block (i.e., the block being decoded). While the block 708 is shown as being of size 8x4, the disclosure is not so limited. The block 708 can be of any other size. Pixels filled with a pattern 702 are pixels of the current block of a current frame. Pixels filled with a pattern 706 (or a subset thereof, as further described herein) illustrate reconstructed pixels of the current frame. Pixels filled with a pattern 704 are pixels that are not available and may contain a padding value (i.e., are set to a padding value). Depending on the neighborhood used for a filter, one or more pixels used by the filter may not be available (such as because these pixels are outside the frame boundary or are outside a largest coding unit that includes the block 708). As such, a padding value may be used (e.g., assumed) for such pixels.

[0062] When used to describe a reference block template, the block 708 illustrates a reference block in a reference frame. Again, while the block 708 is shown as being of size 8x4, the disclosure is not so limited. The block 708 can be of a size corresponding to the size of the current block. As such, pixels filled with the pattern 702 are pixels of the reference block of a reference frame. Pixels filled with the pattern 706 (or a subset thereof, as further described herein) illustrate reconstructed pixels of the reference frame. Pixels filled with the pattern 704 are pixels that are not available and may contain a padding value.

[0063] The template may include a top region 710 that may include 1 to N (where N>1) rows of pixels. The template may include a top-right region 712 that includes 1 to N rows. The template may include a left region 714 of 1 to M (where M>1) columns of pixels. The template may include a bottom- left region 716 of 1 to M (where M>1) columns of pixels.

[0064] In an example, N=M. In an example, if the current block is a luma block, then the template can be 4-sample wide. If the current block is a chroma block, the template (i.e., a chroma template) may be based on the chroma color format. For example, for 4:4:4 content,the chroma template can also be 4-sample wide; and for 4:2:0 or 4:2:2 color formats, the chroma template can be 2- sample wide. In an example, when the top-right region 712 is available, only a 4x4 luma block at the top-right is included in the template. Similarly, if the bottom-left region 716 is available, only a 4x4 luma block at bottom-right is included in the template. The chroma template can be adjusted accordingly based on the chroma color format. In another example, the top template may always be 1-sample wide for both luma and chroma while the left template may be 4-sample wide for luma.

[0065] In an example, the filter coefficients include at least two coefficients. In an example, the filter coefficients include more than two coefficients for at least one of the color components (i.e., at least of the luma or the chroma component). In an example, the number (i.e., cardinality) of the filter coefficients can be decoded from the compressed bitstream. For example, an indicator of the number of filter coefficients can be decoded from the compressed bitstream. That is, if the indicator of the number of coefficient is the first value (e.g., 0), then no filtering is performed on the prediction block. If the indicator of the number of the filter coefficients is a second value (e g., 1), then two filter coefficients are derived; and if the indicator of the number of filter coefficients is a third value (e.g., 2), then more than two filter coefficients are derived.

[0066] The filter is a convolutional filter. The filter coefficients (Cj) can be obtained by minimizing an error metric between the first reconstructed pixels and the second reconstructed pixels. The error metric can be a mean square error (MSE) between pixel values of the respective reconstructed pixels. The error can be a sum of absolute differences (SAD) error between the pixel values of the reconstructed pixels. Any other suitable error metric can be used. In an example, an MSE minimization can be performed by calculating an autocorrelation matrix for the luma input and a cross-correlation vector between the luma input and chroma output. The autocorrelation matrix is LDL decomposed and the final filter coefficients are computed using back-substitution.

[0067] In an example, the number of coefficients to be obtained depends on which pixels within the neighborhood of the intermediate prediction pixel to which the filter is to be applied are used in the filtering. The pixels within the neighborhood of an intermediate prediction pixel that are used for filtering are referred to herein as at least a subset of pixels of the neighborhood.

[0068] An example 718 illustrates a neighborhood of an intermediate pixel 720 of an intermediate prediction block. The example 718 illustrates a 3x3 neighborhood. However, the neighborhood can be larger or smaller, rectangular, or some other shape (e.g., diamond). Theexample 718 illustrates that pixels 722, 724, 726, and 728 (i.e., pixels to the north, east, south, and west of the intermediate pixel 720, respectively) are used in the filtering. As such, the filter coefficients include at least five coefficients: one coefficient to be used with each of the pixels 720 to 728.

[0069] As such, the filter is a 5-tap filter and the prediction pixel corresponding to the intermediate pixel 720 can be obtained using equation (1), where (i=0, ..., 4) are the filter coefficients, pred is the filter pixel of the final prediction block. Equation (1) is shown as further including a constant term (i.e., cs), which may also be derived and used ins some implementations . pred = c0C -I- cqN + c2S + c3E -I- c4W -I- c5(1)

[0070] In an example, one or more but not all filter coefficients may be further refined after being derived. As such, the obtained filter coefficients may be considered to be predicted filter coefficients. The difference (i.e., a coefficient refinement value) between a predicted filter coefficient and the actual value of the filter coefficient may be signaled in the compressed bitstream. As such, obtaining the filter coefficients for the filter can include obtaining a predicted filter coefficient for a filter coefficient of the filter coefficients; decoding, from the compressed bitstream, a coefficient refinement value; and adjusting the predicted filter coefficient using the coefficient refinement value to obtain the filter coefficient.

[0071] In another example, a 7-tap convolutional filter may be used to obtain the chroma prediction block from a luminance prediction block. The convolutional filter may include a 5- tap plus sign shape spatial component, a nonlinear term, and a bias term. The input to the spatial 5-tap component of the filter consists of a center (C) luma sample which is collocated with the chroma sample to be predicted and its above / north (N), below / south (S), left / west (W) and right / east (E) neighbors, as described above with respect to equation (1). As such, the prediction pixel can be obtained using equation (2): predChromaVal = c0C + qN + c2S + c3E + c4W + c5P + c6B (2)

[0072] In equation (2), P is the non-linear term. In an example, the nonlinear term P can be represented as power of two of the center luma sample C (i.e., intermediate pixel 720) and scaled to the sample value range of the content using equation (3).P = (C2+ midVai) » itDepth. (3)

[0073] To illustrate, assuming 10-bit content, then P is calculated as P = ( C2+ 512 ) » 10. Other non-linear terms are possible. The bias term B, when used, can represent a scalaroffset between the input and output. The coefficients ctcan be obtained in a similar way as described above with respect to equation (1). For example, The coefficients ctcan be obtained by minimizing MSE between predicted and reconstructed chroma samples in a reference area. In equation (3), C, N, S, E, and W correspond to the values of the luma prediction values, such as shown in the example 718.

[0074] FIG. 7B illustrates an example 730 of cross-component prediction mode referred to as CCLM mode. In CCLM, a prediction obtained for a luma block can be used to obtain the prediction for a corresponding chroma block. Said another way, in the case that the current block is a luma block, a chroma prediction block for a chroma block corresponding to the current block is derived from the final prediction block. In an example, a 3x3 luma filter plus 1x1 chroma filter plus a DC value may be used. Alternatively, a 3x3 luma filter plus 3x3 chroma filter plus a DC value may be used.

[0075] In CCLM, chroma samples are predicted based on the reconstructed luma samples of the same coding unit (which may be referred to as a largest coding unit, a macroblock, or other such nomenclature) of the current block by using a linear model that is according to equation (4): predc(i,j) = a x recj(i,y) + ft (4)

[0076] In equation (4), predc(i,j) represents the chroma sample predictions, recz(i,j) represents a down-sampled reconstructed luma predictions of the current luma block. Downsampling is performed in the case that the chroma samples and the luma samples do not have the same resolution. For example, down-sampling may be performed in that case that a 4:2:2 or a 4:2:0 format is used. The down-sampling aligns the resolution of luma and chroma blocks.

[0077] The cross-component parameters (a and ft) can be derived with at most four neighboring chroma samples and their corresponding down-sampled luma samples. FIG. 7B illustrates an example 730 of the locations of left and above samples and the sample of the current block involved in the cross-component filtering mode. In some implementations, a division operation to obtain, such as to calculate, the a parameter is implemented using a look-up table. With respect to a luma block 732 (e.g., a reconstructed luma block), the locations of left and above samples are shown as filled circles, such as a filled circle 734. With respect to a chroma block 736, the locations of left and above samples are shown as filled circles, such as a filled circle 738.

[0078] The reconstructed luma down-sampling, such as for 4:2:0 video, wherein four lumasamples correspond to a respective, collocated, chroma sample, may include using one or more, such as two, down-sampling filters, which are applied to the reconstructed luma samples to obtain a two-to-one down-sampling ratio in the horizontal and vertical directions. In the down-sampled luma block a luma sample corresponds to a respective, collocated, chroma sample. In the absence of down- sampling, the chroma pixel location collocated with a group of luma pixels, or samples, such as four luma pixels, or samples, may correspond with a luma pixel location of a luma pixel, or sample, from the group of luma pixels, or samples, such as the top-left luma pixel, or sample, or may correspond with a spatial center of the group of luma pixels, or samples, or may be otherwise oriented with respect to the group of luma pixels, or samples.

[0079] A first, or type-0, 4:2:0 chroma format content, filter (fi) may be expressed as the following: / 0 1 0\ fi = 1 4 1 .\0 1 0 /

[0080] A second, or type-2, 4:2:0 chroma format content, filter ( / 2) may be expressed as the following:

[0081] The down-sampling filters may be indicated using a flag, such as a sequence parameter set (SPS) level flag. In some implementations, a, such as one, luma line (general line buffer in intra prediction) is used to obtain the down-sampled luma samples wherein the upper reference line is at the coding tree unit boundary.

[0082] FIG. 7C illustrates an example 740 of cross-component prediction mode referred to as Inter CCCM mode. The Inter CCCM mode may also be referred to as a crosscomponent residual prediction mode. The Inter CCCM mode can be used in the case that a block is inter predicted or is predicted using Intra Block Copy (IBC) and when there are nonzero luma residuals. In the IBC mode, a block within the same picture is predicted by utilizing a block (i.e., displacement) vector to copy samples from another location of the same picture. The cross-component filters (i.e., the filter coefficients) can be derived using the prediction signals of the luma block and the chroma block, as shown in a diagram 746. The derived filters can be applied to the reconstructed luma signal producing a final chroma prediction.

[0083] The Inter CCCM mode can be used for blocks with rich chroma textures. As such, cross-component prediction using the luma component alone may not produce sufficientlyacceptable results (e.g., reconstruction of the chroma blocks). Thus, while inter prediction is used for the luma component, for each of the chroma components, a cross component prediction that uses the reconstructed luma block and the inter-prediction of the chroma component are combined (blended) to obtain a prediction for the chroma component, as described herein. In the conventional Inter CCCM mode, the blending weights are fixed: 75% of the chroma component prediction is obtained from the reconstructed luma block and 25% is obtained from the inter-prediction of the chroma block. However, the disclosure herein discloses techniques for varying such fixed weighting factors.

[0084] The cross-component filters can be derived using the prediction signals of luma and chroma. The derived filters can be applied to the reconstructed luma signal producing the final chroma predictions. The filtering can use an 8-tap filter. The 8-tap filter can consist of 6 spatial luma samples, a nonlinear term, and a bias term. The spatial luma samples (L0,...,L5) can be obtained from the luma grid selecting the 6 luma samples closest to the chroma position C without down-sampling. For example, given, a chroma pixel 742, the luma values of the predictions of the luma pixels at locations 744A-744F can be used. The predicted chroma value for the chroma pixel 742 can be obtained using equation (5): predChromaVal = c0L0+ cqLx + c2L2+ c3L3+ c4I4+ c5I5+ c6* nonlinear((L0+ L3+ 1) » 1) + c7B (5)

[0085] In equation (5), c0to c7are filter coefficients, nonlinear can be an operator that is as described with respect to equation (3), and B is a bias term.

[0086] The diagram 746 illustrates a decoding process, which can be implemented by the decoder 500 of FIG. 5, of cross-component prediction using Inter CCCM. The process involves deriving cross-component filters from the prediction signals of both the luma (Y) (e.g., a prediction 748) and chroma (Cb and Cr) blocks (e.g., predictions 754A and 754B, respectively). The filter coefficients are derived, as described above, at a block 756. In the CCCM mode described with respect to FIG. 7A, an L-shaped (e.g., above and left) reference area is used to derive the coefficients in that mode. In the Inter CCCM mode, the reference areas in the reference frames are used.

[0087] The filter coefficients are then subsequently applied, at a block 758, to a reconstructed luma signal 752 (obtained by adding the prediction 748 to a luma residual 750) to obtain luma blending components 759A and 759B, respectively. The luma blending components 759 A and 759B are then blended, at adders 760A and 760B, with the predictions 754 A and 754B, respectively, to produce final chroma prediction blocks 761 A and 76 IB,respectively. The final chroma prediction block 761 A is added, at adder 764A, to a residual Cb 762A to obtain a reconstructed Cb block 766A; and the final chroma prediction block 76 IB is added, at adder 764B, to a residual Cr 762B to obtain a reconstructed Cr block 766B. The blending weights are fixed and are set, empirically, to 0.75 and 0.25.

[0088] Another cross -component prediction mode is referred to as Inter cross-component prediction (CCP) merge mode. The Inter CCP merge mode can be used for chroma interpredicted blocks. The Inter CCP merge models include all possible cross-component prediction models described herein. That is the models used with Inter CCP can be any of a CCCM model, a CCLM model, and / or an Inter CCCM model.

[0089] When a block is coded with a cross-component prediction mode, the model (e.g., the filter coefficients) used for the block is retained (e.g., saved) for potential use by a subsequently coded block using Inter CCP. That is, a retained model may be inherited (e.g., used) for the coding a subsequent chroma intra and / or inter blocks.

[0090] A flag is signaled to indicate whether a chroma block that is inter predicted is further coded using the Inter CCP mode. If the Inter CCP merge mode is used, a CCP merge list is constructed. The details of the merge list construction are not pertinent to the understanding of this adaptive blending for cross-component prediction. After the CCP merge list is constructed, the candidate with the lowest template cost is selected for the chroma inter block. That is, template matching is performed with respect to each of the candidates to find the model associated with the lowest cost. The candidate model corresponding to the lowest cost is then selected. As such, no signaling is necessary to indicate the candidate to be selected. The chroma inter block is then predicted in the same way as that of Inter CCCM. That is, the motion compensation predicted samples are blended with the cross-component predicted samples to form the final prediction. Conventionally, the blending weights are fixed, as described above.

[0091] To restate, a flag is firstly used to indicate whether the Inter CCP merge is used for a current chroma block that is inter-predicted. If Inter CCP merge is used, a CCP model is implicitly selected from a CCP merge list. The final prediction of the current chroma inter block is formed by combining the motion-compensation predicted signals and the crosscomponent predicted signals derived using the selected CCP model. The weights for combining predictions are fixed to (WCCP, Winter) = (3 / 4, 1 / 4).

[0092] FIG. 8 is a flowchart of an example of a technique 800 for adaptive blending for cross-component prediction. The technique 800 can be implemented, for example, as a software program that may be executed by computing devices such as transmitting station102 or receiving station 106. The software program can include machine-readable instructions (i.e., executable instructions) that may be stored in a memory such as the memory 204 or the secondary storage 214, and that, when executed by a processor, such as the processor 202, may cause the computing device to perform the technique 800. The technique 800 may be implemented in whole or in part in the intra / inter prediction stage 508 of the decoder 500. The technique 800 can be implemented using specialized hardware or firmware. Multiple processors, memories, or both, may be used. The technique 800 decodes a block that includes a luma block and a chroma block. The block is decoded using a modified Inter CCCM mode where the blending weights are not fixed.

[0093] At 802, a luma prediction block is obtained for the luma block. The luma prediction block can be the prediction 748 of FIG. 7C. In an example, the luma prediction block is obtained using inter prediction. As such, a motion vector and a reference frame may be decoded from a compressed bitstream and used to obtain the luma prediction block. In another example, the luma prediction block can be obtained using intra block copy according to a displacement vector decoded from the compressed bitstream. As such, the predicted luma sample values are obtained based on previously coded and reconstructed luma samples within the same frame.

[0094] At 804, an initial chroma prediction block of the chroma block corresponding to the luma block is obtained. The initial chroma prediction block can be one of the prediction 754A or the prediction 754B of FIG. 7C. The initial chroma prediction block can be obtained using inter-prediction or using IBC. The initial chroma prediction block is also referred as the temporal prediction.

[0095] At 806, filter coefficients are derived based on the luma prediction block and the initial chroma prediction block. The filter coefficient can be derived as described with respect to the block 756 of FIG. 7C. As such, the filter coefficients for a cross-component filter are computed by assessing the relationship between luma and chroma predicted values to minimize a prediction error, which can be based on mean square error (MSE) or any other suitable error metric. That is, the filter coefficients can be derived to minimize a mean square error within a reference area.

[0096] At 808, a luma blending component is obtained. The luma blending component is also referred to as the cross component prediction. The luma blending component can be one of the luma blending components 759A or 759B corresponding the chroma block. As such, the luma blending component can be obtained by applying the derived filter coefficients to a reconstructed luma block (such as the reconstructed luma signal 752 of FIG. 7C), whichinvolves modifying the reconstructed luma samples with the filter coefficients to generate a component that will be used in conjunction with the initial chroma prediction to form a more accurate chroma prediction. Each of the values of the luma blending component can be calculated as described with respect to the chroma pixel 742 of FIG. 7C.

[0097] At 810, a final chroma prediction block is obtained by blending, based on a blending mode, the luma blending component (i.e., the cross-component prediction) with the initial chroma prediction block (i.e., the temporal prediction). That is, the previously generated luma blending component is merged or combined with the initial chroma prediction using blending weights that are derived based on the blending mode to yield the final predicted chroma samples. The blending mode may be decoded from the compressed bitstream or may be derived. The blending mode may be decoded from a header of the chroma block or from a header of the block.

[0098] A blending mode refers to a configuration or strategy used to combine a luma blending component with an initial chroma prediction block to generate a final chroma prediction block during decoding. The blending mode specifies how blending weights, defined as a pair of values (wL, wc), where wLis the weight applied to the luma blending component and wcis the weight applied to the initial chroma prediction block, with wL+ wc= 1, are assigned to each pixel location within the block

[0099] The blending mode can be indicated at the block level. As such, one block may use one set of blending weights (e.g., 0.75 / 0.25) and another may use different blending weights (e.g., 1 / 0).

[0100] The blending mode may indicate uniform blending. That is, the same blending weights (x, 1 — x) are used for each pixel location in a block. The blending mode may indicate non-uniform blending. That is, different weights may be used for different pixel locations in the block. Stated another way, the blending weights may be position dependent.

[0101] In an example of non-uniform blending, a gradational blending scheme is implemented where the weighting factors assigned to the cross-component prediction and temporal prediction are varied across the block in a position-dependent manner. To illustrate, the weighting for cross-component prediction is initiated at 0.75 for pixels situated in the top left comer of the block and similarly for adjacent pixels. Upon progression towards a diagonal line bisecting the block, the weighting factor uniformly transitions to 0.5 for all such aligned pixels. As the weighting evaluation proceeds towards the bottom right comer of the block, the weighting factor assigned to the cross -component prediction progressivelydiminishes to a value of 0.25, concurrently increasing the weighting factor for the temporal prediction to 0.75. As such, a first blending weight used for samples close to a top-left corner of the luma blending component may not be smaller than a second blending weight used for samples close to a bottom-right comer of the luma blending component. This implementation facilitates a dynamic blending where the weight for the cross-component prediction is maximized proximal to the top left comer and gradually minimized towards the bottom right comer, thereby adapting the blending strategy to optimize the prediction accuracy across the block.

[0102] A rationale for the non-uniform weighting can be as follows: In the CCCM mode, blending weights are derived from an L- shaped neighborhood, suggesting higher accuracy for pixels nearer the top-left corner of the block, due to closer proximity to the reference samples. Conversely, pixels towards the bottom-right corner may be less precisely predicted due to distance from these reference samples. In the Inter CCCM mode, both of the crosscomponent and temporal predictions are available for use. Therefore, if the cross -component prediction is less reliable, particularly for pixels further from the reference neighborhood, it becomes beneficial to assign greater weight to the temporal prediction to improve prediction accuracy. This position-dependent blending allows for tailored weighting strategies across the block, enhancing the prediction quality by dynamically adjusting to the precision of the crosscomponent prediction at different locations within the block.

[0103] In an example, a maximal blending weighting index can be signaled in the compressed bitstream at a level no lower than a slice, such as in sequence parameter set (SPS), picture parameter set (PPS), frame header, or slice header. At the block level, a blending index, indicative of the blending mode, can be decoded after decoding a syntax element that indicates that the Inter CCCM mode is to be applied to the block. The weighting index can be signaled in truncated unary code.

[0104] The maximal blending weighting index sets the ceiling for the blending index. As such, the number of bits required to signal the blending index at the block level can be reduced, because the maximal index confines the range of possible values, allowing for a shorter code (e.g., fewer bits) for smaller indices and precluding the need for encoding larger, unused indices. As such, the technique 800 may include decoding, from the compressed bitstream, a maximal blending weighting index; and decoding a blending index indicative of the blending mode, where the blending index is less than or equal to the maximal blending weighting index.

[0105] The blending index can be a value that is selected from a set of possible indices that includes 5 values (e.g., 0 to 4). A first value (e.g., index 0) indicates that Inter CCCM is not to be performed for the block, which is equivalent to using blending weights of 0 / 0; a second value (e.g., index 1) indicates to use the blending weights 0.75 / 0.25 (i.e., 0.75 for the cross-component prediction or luma blending component; and 0.25 for the temporal prediction or the initial chroma prediction block); a third value (e.g., index 2), which may be referred to as a pure cross -component blending mode, indicates to use the blending weights 1 / 0 (i.e., 1 for the cross-component prediction or luma blending component; and 0 for the temporal prediction or the initial chroma prediction block); a fourth value (e.g., index 3) indicates to use the blending weights 0.25 / 0.75 (i.e., 0.25 for the cross-component prediction or luma blending component; and 0.75 for the temporal prediction or the initial chroma prediction block); and a fifth value (e.g., index 4) is for cross-component blending weighting factor of 0.5.

[0106] As such, the blending index indicates a first weight to use with the luma blending component and a second weight to use with the initial chroma prediction block. The maximal index may be signaled as being smaller than the fifth value (e.g., index 4). In an example, the set of possible indices may include indices indicating non-uniform blending.

[0107] In an example, the blending weights can be derived (e.g., inferred) and the compressed bitstream does not include a blending index for the block. In an example, the blending weights can be inferred to correspond to the pure cross-component blending mode (i.e., the third value and blending weights of 1 / 0). It has been observed that the coding of screen content can benefit (e.g., results in improved reconstruction) when using these blending weights. Typically, when screen content is coded, the residuals are zeros. That is, the prediction 748 of FIG. 7C would be signaled to be all zeros. As such, if a syntax element in the compressed bitstream indicates to decode the block using the Inter CCCM mode and the luma residuals are zero, then the blending weights can be inferred to be 1 / 0. That is, the cross-component prediction (i.e., the luma blending component) is used but the temporal prediction (i.e., the initial chroma prediction block) is not considered.

[0108] At 812, the chroma block is reconstructed by adding the final chroma prediction block to a chroma residual block.

[0109] In some implementations, the technique 800 determines not to use crosscomponent prediction (e.g., Inter CCCM or Inter CCP merge) in conjunction with transform skip, where if inter cross-component prediction is on for a block, the transform skip flags for the chroma components are not signaled but derived as off for the block. In theseimplementation, cross-component prediction, such as Inter CCCM and Inter CCP merge, is not used together with transform skip. If one is on (e.g., signaled), the other can be inferred to be off.

[0110] The transform skip mode, which omits the transformation process and transmits pixel-domain residuals, may be less efficient when combined with cross-component prediction. Particularly, it has been observed that when there are no luma residuals, the use of cross-component prediction without transformation is not typically beneficial for standard camera-captured video content; it may be more applicable to screen content. In the absence of a transformation process, signaling a non-zero coefficient results in its direct addition to the prediction at a specific position (i.e., the position of the non-zero coefficient).

[0111] However, if a transformation, such as a low-frequency or DC transform, were applied, the influence of a single non-zero residual would be distributed across the entire block, introducing additional detail. Conversely, with transform skip, this influence is localized to a specific position unless numerous non-zero residues are present. Therefore, under these conditions, the advantage provided by cross-component prediction may not be realized to its fullest potential, implying that when Inter CCCM is active, the benefits of skipping the transform may not warrant the associated signaling. Consequently, the transform skip flags for the Cb and Cr components may not be signaled and are instead derived to be off, precluding their use when cross-component prediction modes are in operation for a block. This conditional signaling strategy simplifies the bitstream and may potentially improve the overall efficiency of the codec's operation.

[0112] In alternative implementations, the operational status of cross-component prediction for a block may provide a contextual basis for signaling transform skip flags for the luma and chroma components (Cb and Cr). Divergent from the approach described above, where transform skip is entirely precluded in the presence of Inter CCCM, transform skip may be signaled and utilized, albeit within a modified context that accounts for the presence of cross-component prediction. Herein, transform skip may be associated with a higher bit cost, representing a trade-off that may yield better performance than the outright deactivation of the transform skip feature.

[0113] In some implementations, Inter CCCM without luma residues is not used together with IBC. If a block is IBC without luma residues, flag(s) for inter CCCM is (are) not further signaled and is (are) derived as off because it has been observed that, in such cases, IBC provides no benefits.

[0114] For simplicity of explanation, the technique 800 of FIG. 8, is depicted and described as a series of steps or operations. However, the steps or operations in accordance with this disclosure can occur in various orders and / or concurrently. Additionally, other steps or operations not presented and described herein may be used. Furthermore, not all illustrated steps or operations may be required to implement a technique in accordance with the disclosed subject matter.

[0115] FIG. 9 illustrates an example of an encoded bitstream 900 that illustrates adaptive blending for cross-component prediction. The encoded bitstream 900 includes a luma residual block 902 for a luma block, a chroma residual block 904 for a chroma block corresponding to the luma block, and a blending mode indicator 906 specifying (e.g., indicative of) blending weights for blending a luma blending component with an initial chroma prediction block.

[0116] The luma blending component can be derived by applying filter coefficients to a reconstructed luma block, which is obtained by adding a luma prediction block to the luma residual block 902. The filter coefficients can be derived based on the luma prediction block and the initial chroma prediction block, enabling the blending process to leverage the correlation between luma and chroma components for improved prediction efficiency.

[0117] A final chroma prediction block is obtainable by combining, pixel-location wise, the luma blending component with the initial chroma prediction block using the blending weights specified by the blending mode indicator 906. The chroma block is then reconstructable by adding the final chroma prediction block to the chroma residual block 904 during decoding.

[0118] The blending mode indicator 906 can specify that the same blending weights are used for each pixel location within the chroma block, therewith indicating uniform blending. Alternatively, the blending mode indicator 906 can specify a non-uniform blending mode in which the blending weights vary based on pixel location within the chroma block. In such cases, the non-uniform blending mode may be used to assign (e.g., by a decoder) a first blending weight to samples near a top-left comer of the luma blending component that is not smaller than a second blending weight assigned to samples near a bottom-right corner, facilitating position-dependent blending. That is, this non-uniform mode can implement a gradational blending scheme, with the blending weights transitioning from a higher value near the top-left corner to a lower value near the bottom-right comer of the luma blending component, optimizing prediction accuracy by adapting to the spatial characteristics of the block. The blending weights specified by the blending mode indicator 906 can be used in aweighted average operation to combine the luma blending component and the initial chroma prediction block.

[0119] The encoded bitstream 900 may include a maximal blending index 908, which defines an upper limit for a blending index that specifies the blending weights for blending the luma blending component with the initial chroma prediction block. The maximal blending index 908 sets a ceiling for allowable blending indices, reducing signaling overhead. The maximal blending index 908 may be encoded at a level no lower than a slice, such as in one of a sequence parameter set, a picture parameter set, a frame header, or a slice header, ensuring it is available for decoding decisions at appropriate hierarchical levels. The blending mode indicator 906 may include a blending index 910 that is less than or equal to the maximal blending index 908.

[0120] The blending index 910 may indicate a first weight for the luma blending component and a second weight for the initial chroma prediction block, therewith providing specific weighting factors for the blending process. The blending index 910 can be selectable from a set of values, including a first value indicating no blending, a second value indicating a first predetermined weight pair (e.g., 0.75 for the luma blending component and 0.25 for the initial chroma prediction block), and a third value indicating a pure cross-component blending mode where the luma blending component is fully weighted (e.g., 1 for the luma blending component and 0 for the initial chroma prediction block).

[0121] The encoded bitstream 900 may include a syntax element 912 indicating that inter cross-component prediction is on (e.g., enabled or is to be performed by the decoder). When present, the syntax element 912 signals to the decoder that modes like Inter CCCM are active, and in such cases, a transform skip flag for the chroma block is not signaled but derived as off, implying that transform skip is disabled for the chroma block to optimize coding efficiency. This exclusion of the transform skip flag when inter cross-component prediction is used further reinforces that transform skip is off, simplifying the bitstream structure.

[0122] The encoded bitstream 900 may include a merge flag 914 indicating whether inter cross-component prediction merge mode is used. When set, the merge flag 914 causes the blending mode indicator 906 to be implicitly selected, by the decoder, from a merge list, enabling the decoder to choose blending weights based on predefined candidates.

[0123] In some implementations, the encoded bitstream 900 may exclude signaling for inter cross-component prediction when the luma residual block 902 contains no residues and the luma block is coded using intra block copy, implying that inter cross-component prediction is off, which is beneficial for certain content types like screen content. In suchscenarios, the blending weights can be derived without signaling in the encoded bitstream 900, resulting in a blending weight of 1 for the luma blending component and 0 for the initial chroma prediction block, further optimizing decoding for blocks with no luma residuals.

[0124] The aspects of encoding and decoding described above illustrate some examples of encoding and decoding techniques. However, it is to be understood that encoding and decoding, as those terms are used in the claims, could mean compression, decompression, transformation, or any other processing or change of data.

[0125] The word “example” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “example” is not necessarily to be construed as being preferred or advantageous over other aspects or designs. Rather, use of the word “example” is intended to present concepts in a concrete fashion. As used in this application, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless specified otherwise or clearly indicated otherwise by the context, the statement “X includes A or B” is intended to mean any of the natural inclusive permutations thereof. That is, if X includes A; X includes B; or X includes both A and B, then “X includes A or B” is satisfied under any of the foregoing instances. In addition, the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more,” unless specified otherwise or clearly indicated by the context to be directed to a singular form. Moreover, use of the term “an implementation” or the term “one implementation” throughout this disclosure is not intended to mean the same embodiment or implementation unless described as such.

[0126] Implementations of the transmitting station 102 and / or the receiving station 106 (and the algorithms, methods, executable instructions, etc., stored thereon and / or executed thereby, including by the encoder 400 and the decoder 500) can be realized in hardware, software, or any combination thereof. The hardware can include, for example, computers, intellectual property (IP) cores, application- specific integrated circuits (ASICs), programmable logic arrays, optical processors, programmable logic controllers, microcode, microcontrollers, servers, microprocessors, digital signal processors, or any other suitable circuit. In the claims, the term “processor” should be understood as encompassing any of the foregoing hardware, either singly or in combination. The terms “signal” and “data” are used interchangeably. Further, portions of the transmitting station 102 and the receiving station 106 do not necessarily have to be implemented in the same manner.

[0127] Further, in one aspect, for example, the transmitting station 102 or the receiving station 106 can be implemented using a general purpose computer or general purposeprocessor with a computer program that, when executed, carries out any of the respective methods, algorithms, and / or instructions described herein. In addition, or alternatively, for example, a special purpose computer / processor can be utilized which can contain other hardware for carrying out any of the methods, algorithms, or instructions described herein.

[0128] The transmitting station 102 and the receiving station 106 can, for example, be implemented on computers in a video conferencing system. Alternatively, the transmitting station 102 can be implemented on a server, and the receiving station 106 can be implemented on a device separate from the server, such as a handheld communications device. In this instance, the transmitting station 102, using an encoder 400, can encode content into an encoded video signal and transmit the encoded video signal to the communications device. In turn, the communications device can then decode the encoded video signal using a decoder 500. Alternatively, the communications device can decode content stored locally on the communications device, for example, content that was not transmitted by the transmitting station 102. Other suitable transmitting and receiving implementation schemes are available. For example, the receiving station 106 can be a generally stationary personal computer rather than a portable communications device, and / or a device including an encoder 400 may also include a decoder 500.

[0129] Further, all or a portion of implementations of the present disclosure can take the form of a computer program product accessible from, for example, a computer-usable or computer-readable medium. A computer-usable or computer-readable medium can be any device that can, for example, tangibly contain, store, communicate, or transport the program for use by or in connection with any processor. The medium can be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable mediums are also available.

[0130] The above-described embodiments, implementations, and aspects have been described in order to facilitate easy understanding of this disclosure and do not limit this disclosure. On the contrary, this disclosure is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope is to be accorded the broadest interpretation as is permitted under the law so as to encompass all such modifications and equivalent arrangements.

Claims

What is claimed is:

1. A method for decoding a block that includes a luma block and a chroma block, comprising: obtaining a luma prediction block of the luma block; obtaining an initial chroma prediction block of the chroma block corresponding to the luma block; deriving filter coefficients based on the luma prediction block and the initial chroma prediction block; obtaining a luma blending component by applying the filter coefficients to a reconstructed luma block that is obtained by adding the luma prediction block to a luma residual block; obtaining a final chroma prediction block by blending, based on a blending mode, the luma blending component with the initial chroma prediction block, wherein the blending is performed by combining, pixel-location wise, the luma blending component and the initial chroma prediction block using a weighted average using blending weights specified by the blending mode; and reconstructing the chroma block by adding the final chroma prediction block to a chroma residual block.

2. The method of claim 1, wherein the filter coefficients are derived to minimize a mean square within a reference area.

3. The method of claim 1, further comprising: decoding the blending mode from a compressed bitstream.

4. The method of claim 3, wherein the blending mode is decoded from a header of the chroma block or a header of the block.

5. The method of claim 3, further comprising: decoding, from the compressed bitstream, a maximal blending weighting index; and decoding a blending index indicative of the blending mode, wherein the blending index is less than or equal to the maximal blending weighting index.

6. The method of claim 5, wherein a first blending weight used for samples close to a top-left corner of the luma blending component is not smaller than a second blending weight used for samples close to a bottom-right corner of the luma blending component.

7. The method of claim 3, wherein decoding the blending mode from the compressed bitstream comprises: decoding a blending index indicative of the blending mode, wherein the blending index indicates a first weight to use with the luma blending component and a second weight to use with the initial chroma prediction block.

8. The method of any one of claims 1 to 3, further comprising: deriving the blending weights conditionally without signaling in a compressed bitstream, wherein the blending weights are derived based on an absence of luma residues for the luma block while cross -component inter-prediction is signaled as on for the block, resulting in a derived blending factor of 1 for pure cross-component prediction.

9. The method of any one of claims 1 to 3, wherein the blending mode indicates that same weights are used for each pixel location.

10. The method of any one of claims 1 to 3, wherein the blending mode indicates that the blending weights used for combining a first value at a location of the luma blending component and a second value at the location of the initial chroma prediction block are based on the location relative to a top-left corner of the block.

11. The method of any one of claims 1 to 3, further comprising: determining not to use cross -component prediction in conjunction with transform skip, wherein if inter cross -component prediction is on for the block, a transform skip flag for the chroma block is not signaled but derived as off for the block.

12. A device, comprising: a processor that is configured to perform the method of one of claims 1-11.

13. A device, comprising:a memory; and a processor, the processor configured to execute instructions stored in the memory to perform the method of one of claims 1-11.

14. A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising operations that perform the method of one of claims 1-11.

15. A non-transitory computer-readable storage medium having stored thereon an encoded bitstream, wherein the encoded bitstream is configured for decoding by the method of one of claims 1-11.

16. A non-transitory computer-readable storage medium having stored thereon an encoded bitstream, the encoded bitstream comprising: a luma residual block for a luma block; a chroma residual block for a chroma block corresponding to the luma block; and a blending mode indicator specifying blending weights for blending a luma blending component with an initial chroma prediction block, wherein the luma blending component is derivable by applying filter coefficients to a reconstructed luma block obtained by adding a luma prediction block to the luma residual block, wherein the filter coefficients are derivable based on the luma prediction block and the initial chroma prediction block, wherein a final chroma prediction block is obtainable by combining, pixellocation wise, the luma blending component and the initial chroma prediction block based on the blending weights specified by the blending mode indicator, and wherein the chroma block is reconstructable by adding the final chroma prediction block to the chroma residual block during decoding.

17. The non-transitory computer-readable storage medium of claim 16, wherein the encoded bitstream further comprises a maximal blending weighting index, wherein the maximal blending weighting index defines an upper limit for a blending index that specifies the blending weights for blending the luma blending component with the initial chroma prediction block.

18. The non-transitory computer-readable storage medium of claim 17, wherein the maximal blending weighting index encoded at a level no lower than a slice.

19. The non-transitory computer-readable storage medium of claim 17, wherein the maximal blending weighting index is signaled in one of a sequence parameter set, a picture parameter set, a frame header, or a slice header.

20. The non-transitory computer-readable storage medium of claim 17, wherein the blending mode indicator includes a blending index that is less than or equal to the maximal blending weighting index.

21. The non-transitory computer-readable storage medium of claim 20, wherein the blending index indicates a first weight for the luma blending component and a second weight for the initial chroma prediction block.

22. The non-transitory computer-readable storage medium of claim 20, wherein the blending index is selectable from a set of values including a first value indicating no blending, a second value indicating a first predetermined weight pair, and a third value indicating a pure cross-component blending mode.

23. The non-transitory computer-readable storage medium of any one of claims 16 to 17, wherein the blending weights are derivable without signaling in the encoded bitstream when the luma residual block contains no residues and cross-component inter-prediction is signaled as on, resulting in a blending weight of 1 for the luma blending component and 0 for the initial chroma prediction block.

24. The non-transitory computer-readable storage medium of claim 16, wherein the blending mode indicator specifies that same blending weights are used for each pixel location within the chroma block.

25. The non-transitory computer-readable storage medium of claim 16, wherein the blending mode indicator specifies a non-uniform blending mode in which the blending weights vary based on pixel location within the chroma block.

26. The non-transitory computer-readable storage medium of claim 25, wherein the non-uniform blending mode assigns a first blending weight to samples near a top-left comer of the luma blending component that is not smaller than a second blending weight assigned to samples near a bottom-right comer of the luma blending component.

27. The non-transitory computer-readable storage medium of claim 25, wherein the non-uniform blending mode implements a gradational blending scheme with the blending weights transitioning from a higher value near a top-left corner to a lower value near a bottom-right corner of the luma blending component.

28. The non-transitory computer-readable storage medium of claim 16, wherein the encoded bitstream further comprises a syntax element indicating that inter crosscomponent prediction is on, and wherein a transform skip flag for the chroma block is not signaled but derived as off.

29. The non-transitory computer-readable storage medium of claim 16, wherein the encoded bitstream excludes a transform skip flag for the chroma block when inter crosscomponent prediction is used, implying transform skip is off.

30. The non-transitory computer-readable storage medium of claim 16, wherein the blending weights are used in a weighted average operation to combine the luma blending component and the initial chroma prediction block.

31. The non-transitory computer-readable storage medium of claim 16, wherein the encoded bitstream further comprises: a flag indicating whether inter cross-component prediction merge mode is used, and wherein the blending mode indicator is implicitly selected from a merge list when the flag is set.

32. The non-transitory computer-readable storage medium of claim 16, wherein the encoded bitstream excludes signaling for inter cross -component prediction when the luma residual block contains no residues and the block is coded using intra block copy, implying inter cross-component prediction is off.

Images