Region-based inter-component prediction

Abstract

Region-based inter-component prediction improves convolutional cross-component mode (CCCM) prediction by allowing filter coefficients for predicting chroma samples from luma samples to be derived for an entire region of a frame of a video stream, such as a coding tree unit (CTU), without requiring such filter coefficients to be derived for each individual coding unit (CU). Deriving filter coefficients for an entire region, rather than for each individual CU during processing, significantly reduces latency during video coding, allowing CCCM prediction to be used for hardware coder implementations.

Description

[Background technology]

[0001] A digital video stream may represent a video using a sequence of frames or still images. Digital video can be used for a variety of applications, such as video conferencing, high-definition video entertainment, video advertising, or sharing user-generated videos. Digital video streams can contain large amounts of data, consuming significant computing or communication resources of computing devices to process, transmit, or store the video data. Various approaches, such as encoding or decoding techniques, have been proposed to reduce the amount of data in a video stream. Summary of the Invention

[0002] Among other things, systems and techniques for region-based cross-component prediction are disclosed herein.

[0003] A method for region-based inter-component prediction according to an embodiment of the present disclosure includes identifying a region in a frame to encode or decode, determining regional filter coefficients for the region, determining an input value for a current luma sample in a portion of the region, determining a predicted chroma sample for the current luma sample based on the input value and the regional filter coefficients, and encoding or decoding the predicted chroma sample.

[0004] In some implementations of the method, determining the regional filter coefficients includes deriving at least a portion of the regional filter coefficients based on one or both of the region or adjacent regions.

[0005] In some implementations of the method, deriving at least a portion of the regional filter coefficients based on one or both of the region and the neighboring regions includes minimizing a mean square error between the predicted chroma samples and the reconstructed chroma samples within a reference area of ​​the frame.

[0006] In some implementations of the method, the mean squared error is performed using chroma samples from a padding area outside the region.

[0007] In some implementations of the method, determining the region filter coefficients includes decoding, from a bitstream associated with the frame, one or more syntax elements used to signal the region filter coefficients.

[0008] In some implementations of the method, the method includes determining to use domain filter coefficients to determine the predicted chroma sample based on the classification of the current luma sample.

[0009] In some implementations of the method, a portion of the region is a coding unit, and different region filter coefficients are used to determine the second predicted chroma sample based on a classification of the second current luma sample within the coding unit.

[0010] In some implementations of the method, identifying the region includes decoding one or more syntax elements associated with the region signaled in the bitstream.

[0011] In some implementations of the method, determining the predicted chroma sample includes determining spatial weight values ​​for an area of ​​a portion of the region according to a prediction approach to use for the area, and determining the predicted chroma sample using the spatial weight values.

[0012] In some implementations of the method, a portion of the region is a coding unit, and regional filter coefficients are determined for use in multiple coding units of the region.

[0013] In some implementations of the method, the size of the region is greater than the minimum chroma unit size.

[0014] In some implementations of the method, the regions are coding tree units of size 128x128 or 64x64.

[0015] An apparatus for region-based inter-component prediction according to an embodiment of the present disclosure includes a memory and a processor, wherein the processor is configured to execute instructions stored in the memory for determining regional filter coefficients for a region in a frame to encode or decode; determining a first predicted chroma sample for a first luma sample in a first portion of the region based on an input value of a first luma sample and based on the regional filter coefficients; determining a second predicted chroma sample for a second luma sample in a second portion of the region based on an input value of a second luma sample and based on the regional filter coefficients; and encoding or decoding the first predicted chroma sample and the second predicted chroma sample.

[0016] In some implementations of the apparatus, a first portion of the domain filter coefficients are signaled in a bitstream associated with a frame, and a second portion of the domain filter coefficients are derived based on video data in the frame.

[0017] In some implementations of the apparatus, the region is a current coding tree unit, and the region filter coefficients are derived using reconstructed chroma samples from one or more neighboring coding tree units of the current coding tree unit.

[0018] In some implementations of the apparatus, the regional filter coefficients are used for both the first predicted chroma sample and the second predicted chroma sample based on the classification of the first luma sample and the second luma sample.

[0019] In some implementations of the apparatus, the classification is based on one or more of gradient, direction, or pixel value band.

[0020] A non-transitory computer-readable storage device according to an embodiment of the present disclosure includes program instructions executable by one or more processors, which, when executed, cause the one or more processors to perform operations for region-based inter-component prediction, the operations including determining filter coefficients to use for predicting chroma samples within a plurality of coding units of a coding tree unit in a frame for encoding or decoding, determining a current luma sample within a coding unit of the plurality of coding units, determining a predicted chroma sample of the current luma sample based on an input value and the filter coefficients, and encoding or decoding the predicted chroma sample.

[0021] In some implementations of the non-transitory computer-readable storage device, determining the filter coefficients includes one of deriving the filter coefficients based on one or both of the coding tree unit or neighboring coding tree units of the coding tree unit, decoding one or more syntax elements used to signal the filter coefficients from a bitstream associated with the frame, or deriving a first portion of the filter coefficients and decoding a second portion of the filter coefficients from the bitstream.

[0022] In some implementations of the non-transitory computer-readable storage device, determining the predicted chroma sample includes determining spatial weight values ​​for an area of ​​the coding unit according to a prediction approach to use for the area, and determining the predicted chroma sample using the spatial weight values.

[0023] These and other aspects of the present disclosure are set forth in the following detailed description of the embodiments, the appended claims, and the accompanying drawings.

[0024] This description makes reference to the accompanying drawings, which are listed below, in which like reference numerals refer to like parts throughout the several views. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a schematic diagram of an example video encoding and decoding system; [Figure 2] 1 is a block diagram of an example of a computing device that can implement a transmitting station or a receiving station. [Figure 3] FIG. 1 is a diagram of an example of a video stream to be encoded and decoded. [Figure 4] FIG. 2 is a block diagram of an example encoder. [Figure 5] FIG. 2 is a block diagram of an example decoder. [Figure 6] 1 is a diagram of an example of a portion of a video frame. [Figure 7] 10 shows an example of a reference area for region-based inter-component prediction. [Figure 8] 1 shows an example of a neighborhood of luma samples used to predict chroma samples. [Figure 9] 1 illustrates exemplary resolutions of luma and chroma blocks. [Figure 10] FIG. 1 is a flowchart diagram of an example technique for region-based inter-component prediction. DETAILED DESCRIPTION OF THE INVENTION

[0026] Video compression schemes may include dividing each image or frame of a video stream into smaller portions, such as blocks or coding tree units (CTUs), and generating an encoded bitstream using techniques that limit the information contained in each CTU. The bitstream can be decoded to recreate the source frame from the limited information. Encoding or decoding a CTU into or from a bitstream may include predicting values ​​of pixels or CTUs based on similarities with other pixels or CTUs in the same frame that have already been coded. These similarities may be determined using intra-prediction, which attempts to predict pixel values ​​of a coding unit (CU) of a CTU using pixels surrounding the CU (CU) (e.g., pixels in the same frame as the CU but outside the CU). During encoding, the result of an intra-prediction mode performed on a CU is a prediction unit (PU). A prediction residual may be determined based on the difference between pixel values ​​of the CU and the PU. The prediction residual and the intra-prediction mode used to finally obtain the prediction residual can then be encoded into a bitstream. During decoding, the prediction residual is reconstructed into a CU using the PU generated based on the intra-prediction mode, and then included in the output video stream.

[0027] A CU includes one luminance component, also referred to as a luma component, and two chrominance components, also referred to as chroma components. These luma and chroma components are sometimes referred to as luma blocks and chroma blocks. For example, the luma component of a CU may be represented in the Y plane of the CU, and the chroma components may be represented in either the U and V planes or the Cr and Cb planes of the CU. It is understood that a luma component includes several luma samples, and each chroma component includes several chroma samples. Generally, luma samples provide a measure of the overall brightness of the target CU and thus represent the structural quality of the video content of the target CU, while chroma samples provide a measure of the overall color of the target CU. For this reason, conventional video compression schemes often use a more precise prediction approach to predict the luma component of a CU than its chroma components. Such schemes may also utilize approaches aimed at predicting those chrominance components from a predicted luma component.

[0028] One example of such a luma-to-chroma prediction approach is the cross-component linear model (CCLM) prediction proposed for use in the H.266 codec, also known as Versatile Video Coding (VVC), which is used for intra-predicted CUs to predict chroma signals based on weighted luma signals. In CCLM prediction, the chroma samples of a CU are calculated as pred_C(i,j) = α *The CCLM prediction parameters α and β are weights derived from up to four neighboring chroma samples and their corresponding downsampled luma samples using one or more lookup tables. Downsampling aligns the resolutions of the luma and chroma components of a CU. In particular, if the resolutions of the luma and chroma components are already equal (e.g., 4:4:4), the downsampling operation may be omitted. However, if the resolutions of the luma and chroma components are unequal (e.g., 4:2:0), and thus the chroma components are generally smaller than the luma components, one or more downsampling filters may be applied to the luma samples in the luma component in both the horizontal and vertical directions. Examples of downsampling filters may include Type-0, where each chroma sample resides between two vertical luma samples across the CU, and Type-2, where there is a chroma sample for every luma sample across the CU. Due to the high correlation between luma and chroma values, CCLM prediction is generally more efficient than traditional chroma space prediction approaches when CUs are rich in texture, especially chroma texture.

[0029] While CCLM prediction offers advantages over historical approaches for predicting chroma from luma, there may be opportunities to further improve the accuracy and / or efficiency of CCLM prediction. One such opportunity involves a new approach to predicting chroma from luma that builds on CCLM prediction, called convolutional cross-component model (CCCM) prediction. CCCM prediction uses a 7-tap filter that includes a 5-tap spatial component, a 1-tap nonlinear term, and a 1-tap bias term. The spatial components include the current luma sample C and four neighboring samples, called N, S, E, and W (e.g., arranged in a plus, x, diamond, or other shape, with C at the center in any such case). The nonlinear term P is expressed as a power of 2 of C, where P = (C * The bias term B is scaled to the sample value range of the content, expressed as C+midVal)>>bitDepth, where bitDepth represents the bit precision of the video content and midVal is the midpoint chroma value within that bit precision. For example, for 10-bit video content, bitDepth equals 10 and midVal equals 512. The bias term B, similar to the offset term in CCLM prediction, represents a scalar offset between the input and output and is set to the midpoint chroma value of the bit precision (e.g., 512 for 10-bit video content), so B equals midVal.

[0030] The output of CCCM prediction, i.e., the predicted chroma value based on C, is calculated using the filter coefficient c i The predicted chroma value, predChromaVal, is calculated as the convolution between c0C+c1N+c2S+c3E+c4W+c5P+c6B (where i is a value between 0 and 6 inclusive) and the input value, clipped to the range of valid chroma samples. The predicted chroma value, predChromaVal, is expressed as predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B. The filter coefficients, c iis determined by minimizing the mean squared error (MSE) between predicted and reconstructed chroma samples within a reference area corresponding to one or more CTUs, including the current CTU with the CU being predicted. In one example, the reference area may include N (e.g., 6) lines of chroma samples above the CU and to the left of the CU; accordingly, the reference area may extend one CU width to the right of the CU boundary and one CU height below the CU boundary. The reference area is adjusted to include only available chroma samples. An extension of the reference area, represented as one sample surrounding the periphery of the actual reference area, may be provided to support chroma samples along the edges of the reference area if such edge samples would otherwise be unavailable. MSE minimization is performed by calculating the autocorrelation matrix of the luma input samples and the cross-correlation vector between the luma input samples and the predicted chroma output samples.

[0031] Although CCCM prediction offers many improvements over CCLM prediction alone, it is not without drawbacks. In particular, CCCM prediction uses the filter coefficient c iTo derive , several 64-bit division operations by an arbitrary denominator must be performed. Because these division operations must be performed sequentially due to the nature of solving the function, each filter coefficient value is represented using a relatively large number of bits (e.g., using 22-bit precision). Therefore, there is typically a long latency incurred by CCCM prediction to derive the filter coefficients. This latency is particularly noticeable in hardware coders (i.e., combined hardware encoder and decoder, or separate hardware encoder and decoder), which are limited to a specific amount of processing per cycle and generally have a limited cycle budget for small CUs. These limitations necessarily prevent CCCM prediction from being implemented in hardware coders, because hardware coders must be designed to handle worst-case scenarios (e.g., requiring CCCM prediction for every sample within an entire CU). In particular, in such worst-case scenarios, it would be impossible to play video at a desired frame rate (e.g., 30 frames per second) if there is not enough time to process chroma samples within each CU of each frame. Therefore, it is desirable to modify the CCCM prediction to make it usable in hardware coder implementations.

[0032] Embodiments of the present disclosure address problems such as these by using a region-based approach to inter-component prediction in which CCCM prediction filter coefficients are determined for and used across all CUs in a relatively large region (e.g., a CTU). Deriving filter coefficients once for an entire region of a frame, rather than for each individual CU, substantially reduces the latency of the coding process for enabling a hardware coder to perform CCCM prediction by eliminating the need to perform a highly resource-intensive filter coefficient derivation calculation sequence for each CU. A region generally corresponds to a single CTU in a frame, but in any case, must be larger than the minimum chroma unit size allowed by the target video codec. Therefore, the size of a given region may be signaled in the bitstream. The filter coefficients for a given region may be derived based on spatially neighboring regions in the frame, signaled in the bitstream (e.g., in an adaptation parameter set (APS) or slice header), or both; for example, some of the filter coefficients for the region are derived and other filter coefficients are signaled. In some cases, multiple filter coefficient sets may be used within a single region. For example, different filter sets may be used based on the classification of the reconstructed luma sample used to predict a target chroma sample, where a first set of filter coefficients may be used to predict a first chroma sample in a given region and a second set of filter coefficients may be used to predict a second chroma sample in that same region. In some cases, different filter shapes may be used for inter-component prediction. In some cases, the region-based inter-component prediction approaches disclosed herein may be combined with CCLM prediction approaches to improve prediction accuracy within regions of particular types and / or sizes, e.g., as described above.

[0033] Although reference is made herein to CTUs, CUs, PUs, etc., commonly used in video codecs such as H.265 and H.266, referred to as High-Efficiency Video Coding (HEVC), by way of example, embodiments of the present disclosure may also be used with other video coding structures. In one particular, but non-limiting, example, embodiments of the present disclosure may also be used with superblocks, macroblocks, blocks, etc., as commonly used in video codecs such as VP9, ​​AV1, and the currently under development AV2. Accordingly, references herein to particular video coding structures such as CTUs, CUs, PUs, etc., shall be considered representations of non-limiting example video coding structures in which embodiments of the present disclosure may be used.

[0034] Further details of techniques for region-based inter-component prediction are described herein, first with reference to systems in which such techniques can be implemented. Figure 1 is a schematic diagram of an example video encoding and decoding system 100. The transmitting station 102 can be, for example, a computer having a hardware configuration such as that illustrated in Figure 2. However, other implementations of the transmitting station 102 are possible. For example, the processing of the transmitting station 102 can be distributed across multiple devices.

[0035] A network 104 may connect the transmitting station 102 and the receiving station 106 for encoding and decoding a video stream. Specifically, the video stream may be encoded at the transmitting station 102, and the encoded video stream may be decoded at the receiving station 106. The network 104 may be, for example, the Internet. The network 104 may also be a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a cellular network, or any other means for transferring a video stream from the transmitting station 102 to the receiving station 106 in this example.

[0036] The receiving station 106 can, in one example, be a computer having a hardware configuration such as that illustrated in Figure 2. However, other suitable implementations of the receiving station 106 are possible. For example, the processing of the receiving station 106 can be distributed among multiple devices.

[0037] Other implementations of the video encoding and decoding system 100 are possible. For example, in an implementation, the network 104 may be omitted. In other implementations, the video stream may be encoded and then stored for later transmission to the receiving station 106 or any other device having memory. In one implementation, the receiving station 106 receives the encoded video stream (e.g., via the network 104, a computer bus, and / or some other communication path) and stores the video stream for later decoding. In an example implementation, the Real-Time Transport Protocol (RTP) is used to transmit the encoded video over the network 104. In other implementations, a transport protocol other than RTP (e.g., the Hypertext Transfer Protocol (HTTP) video streaming protocol) may be used.

[0038] For example, when used in a videoconferencing system, transmitting station 102 and / or receiving station 106 may include the functionality to both encode and decode video streams, as described below. For example, receiving station 106 may be a videoconferencing participant that receives, decodes, and views an encoded video bitstream from a videoconferencing server (e.g., transmitting station 102), and also encodes and sends its own video bitstream to the videoconferencing server for decoding and display by other participants.

[0039] In some implementations, the video encoding and decoding system 100 may instead be used to encode and decode data other than video data. For example, the video encoding and decoding system 100 may be used to process image data. The image data may include blocks of data from an image (e.g., CTUs of a frame of a video stream). In such implementations, the transmitting station 102 may be used to encode the image data, and the receiving station 106 may be used to decode the image data.

[0040] Alternatively, the receiving station 106 may represent a computing device that stores encoded image data for later use, such as after receiving the encoded or pre-encoded image data from the transmitting station 102. As a further alternative, the transmitting station 102 may represent a computing device that decodes image data, such as before transmitting the decoded image data to the receiving station 106 for display.

[0041] Figure 2 is a block diagram of an example computing device 200 capable of implementing a transmitting station or a receiving station. For example, computing device 200 may implement one or both of transmitting station 102 and receiving station 106 of Figure 1. Computing device 200 may be in the form of a computing system including multiple computing devices, or in the form of a single computing device, such as, for example, a mobile phone, a tablet computer, a laptop computer, a notebook computer, a desktop computer, etc.

[0042] Processor 202 in computing device 200 may be a conventional central processing unit. Alternatively, processor 202 may be any other type of device, or multiple devices, now existing or later developed, that can manipulate or process information. For example, while the disclosed embodiments may be implemented with a single processor (e.g., processor 202) as shown, advantages in speed and efficiency may be achieved by using multiple processors.

[0043] In an embodiment, memory 204 in computing device 200 may be a read-only memory (ROM) device or a random-access memory (RAM) device, although other suitable types of storage devices may be used as memory 204. Memory 204 may include code and data 206 accessed by processor 202 using bus 212. Memory 204 may further include an operating system 208 and application programs 210, which include at least one program that enables processor 202 to perform the techniques described herein. For example, application programs 210 may include applications 1-N, and may further include, among other things, encoding and / or decoding software that performs the enhanced multi-stage intra-prediction described herein.

[0044] Computing device 200 may also include secondary storage 214, which may be, for example, a memory card used in a mobile computing device. A video communication session may contain a significant amount of information, all or part of which may be stored in secondary storage 214 and loaded into memory 204 as needed for processing.

[0045] Computing device 200 may also include one or more output devices, such as a display 218. Display 218, in one example, may be a touch-sensitive display that combines a display with touch-sensing elements operable to sense touch input. Display 218 may be coupled to processor 202 via bus 212. In addition to, or instead of, display 218, other output devices may be provided that enable a user to program or otherwise use computing device 200. When the output device is or includes a display, the display may be implemented in various ways, such as, for example, a liquid crystal display (LCD), a cathode ray tube (CRT) display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0046] Computing device 200 may also include or communicate with image sensing device 220, such as a camera or some other now existing or later developed image sensing device 220, that can sense images, such as an image of a user operating computing device 200. Image sensing device 220 may be positioned to face the user operating computing device 200. In one example, the position and optical axis of image sensing device 220 may be configured to be directly adjacent to display 218 and have a field of view that includes the area from which display 218 is viewable.

[0047] Computing device 200 may also include or communicate with a sound sensing device 222, such as a microphone or any other now existing or later developed sound sensing device that can sense sounds near computing device 200. Sound sensing device 222 may be positioned to face toward a user operating computing device 200 and may be configured to receive sounds, such as speech or other vocalizations, made by the user while the user is operating computing device 200.

[0048] 2, the processor 202 and memory 204 of computing device 200 are shown as integrated into one unit, but other configurations are available. The operations of processor 202 may be distributed across multiple machines (each machine may have one or more processors) that may be coupled directly or through a local area or other network. Memory 204 may be distributed across multiple machines, such as network-based memory or memory in multiple machines that perform the operations of computing device 200.

[0049] Although shown here as a single bus, bus 212 of computing device 200 may be comprised of multiple buses. Additionally, secondary storage 214 may be directly connected to other components of computing device 200 or may be accessible over a network, and may include an integrated unit such as a memory card, or multiple units such as multiple memory cards. Accordingly, computing device 200 may be implemented in a wide variety of configurations.

[0050] 3 is a diagram of an example video stream 300 to be encoded and decoded. The video stream 300 includes a video sequence 302. At the next level, the video sequence 302 includes a number of adjacent video frames 304. Although three frames are shown as adjacent frames 304, the video sequence 302 may include any number of adjacent frames 304. The adjacent frames 304 may be further subdivided into individual video frames, such as frame 306.

[0051] At the next level, the frame 306 can be divided into a series of planes, slices, or segments 308. A segment 308 can be a subset of a frame that allows for parallel processing, for example. A segment 308 can also be a subset of a frame that allows for separation of video data into individual colors. For example, a frame 306 of color video data can include one luminance plane and two chrominance planes. The segments 308 may be sampled at different resolutions.

[0052] Regardless of whether the frame 306 is divided into segments 308, the frame 306 may be further subdivided into CTUs 310, which may contain data corresponding to, for example, N by M pixels in the frame 306, where N and M may refer to the same or different integer values. The CTUs 310 may also be arranged to contain data from one or more slices 308 of pixel data. The CTUs 310 may be of any suitable size, such as 4 by 4 pixels, 8 by 8 pixels, 16 by 8 pixels, 8 by 16 pixels, 16 by 16 pixels, or larger, up to a maximum size which may be 128 by 128 pixels or other N by M pixel size.

[0053] Figure 4 is a block diagram of an example encoder 400. The encoder 400, as described above, may be implemented in the transmitting station 102, such as by providing a computer software program stored in a memory, such as memory 204. The computer software program may include machine instructions that, when executed by a processor, such as processor 202, cause the transmitting station 102 to encode video data in the manner described in Figure 4. The encoder 400 may also be implemented as dedicated hardware, for example, included in the transmitting station 102. In some implementations, the encoder 400 is a hardware encoder.

[0054] The encoder 400 has the following stages that perform various functions in a forward path (shown by solid connecting lines) to generate an encoded or compressed bitstream 420 using the video stream 300 as input: an intra / inter prediction stage 402, a transform stage 404, a quantization stage 406, and an entropy encoding stage 408. The encoder 400 may also include a reconstruction path (shown by dotted connecting lines) to reconstruct frames for encoding future CTUs. In FIG. 4, the encoder 400 has the following stages to perform various functions in the reconstruction path: a dequantization stage 410, an inverse transform stage 412, a reconstruction stage 414, and a loop filtering stage 416. Other structural variations of the encoder 400 may also be used to encode the video stream 300.

[0055] In some cases, the functions performed by encoder 400 may occur after filtering of video stream 300. That is, video stream 300 may undergo preprocessing in accordance with one or more embodiments of this disclosure before encoder 400 receives video stream 300. Alternatively, encoder 400 may itself perform such preprocessing on video stream 300 before proceeding to perform the functions described with respect to FIG. 4, e.g., before processing video stream 300 in intra / inter prediction stage 402.

[0056] When the video stream 300 is presented for encoding after preprocessing has been performed, each adjacent frame 304, e.g., frame 306, can be processed on a CTU-by-CTU basis. In the intra / inter prediction stage 402, each CU of a CTU can be encoded using intra-frame prediction (also called intra-prediction) or inter-frame prediction (also called inter-prediction). In either case, a PU can be formed. In the case of intra-prediction, a PU may be formed from samples in a current frame that have been previously encoded and reconstructed. In the case of inter-prediction, a PU may be formed from samples in one or more previously constructed reference frames.

[0057] Next, in the intra / inter prediction stage 402, the PU can be subtracted from the CU to generate a prediction residual, also called a residual. The transform stage 404 transforms the residual into transform coefficients, e.g., in the frequency domain using a block-based transform. The quantization stage 406 converts the transform coefficients into discrete quantized values, called quantized transform coefficients, using quantizer values ​​or quantization levels. For example, the transform coefficients may be divided by the quantizer value and truncated.

[0058] The quantized transform coefficients are then entropy encoded by an entropy encoding stage 408. The entropy encoded coefficients, along with other information used to decode the CU (which may include, for example, syntax elements such as those used to indicate the type of prediction used, the transform type, motion vectors, quantizer values, etc.), are output in a compressed bitstream 420. The compressed bitstream 420 can be formatted using a variety of techniques, such as variable length coding or arithmetic coding. The compressed bitstream 420 can also be referred to as an encoded video stream or encoded video bitstream, and these terms are used interchangeably herein.

[0059] A reconstruction path (represented by a dotted connecting line) can be used to ensure that the encoder 400 and the decoder 500 (described below with respect to FIG. 5 ) use the same reference frame to decode the compressed bitstream 420. The reconstruction path performs functions similar to those performed during the decoding process (described below with respect to FIG. 5 ), including dequantizing the quantized transform coefficients in a dequantization stage 410 and inverse transforming the dequantized transform coefficients in an inverse transform stage 412 to generate a differential prediction residual (also called a derivative prediction residual). In a reconstruction stage 414, the PU predicted in the intra / inter prediction stage 402 can be added to the differential residual to create a reconstructed CU. A loop filtering stage 416 can apply an in-loop filter or other filter to the reconstructed CU to reduce distortions such as blocking artifacts. Examples of filters that may be applied in loop filtering stage 416 include, but are not limited to, a deblocking filter, a directional enhancement filter, and a loop restoration filter.

[0060] Other variations of the encoder 400 can be used to encode the compressed bitstream 420. In some implementations, a non-transform-based encoder can directly quantize the residual signal for a particular CU, CTU, or frame without using the transform stage 404. In some implementations, the encoder may have the quantization stage 406 and dequantization stage 410 combined into a common stage.

[0061] Figure 5 is a block diagram of an example decoder 500. The decoder 500 can be implemented in the receiving station 106, for example, by providing a computer software program stored in memory 204. The computer software program can include machine instructions that, when executed by a processor, such as processor 202, cause the receiving station 106 to decode video data in the manner described in Figure 5. The decoder 500 can also be implemented in hardware, for example, included in the transmitting station 102 or the receiving station 106. In some implementations, the decoder 500 is a hardware decoder.

[0062] The decoder 500, similar to the reconstruction path of the encoder 400 described above, in one example includes the following stages to perform various functions for generating an output video stream 516 from a compressed bitstream 420: an entropy decoding stage 502, a dequantization stage 504, an inverse transform stage 506, an intra / inter prediction stage 508, a reconstruction stage 510, a loop filtering stage 512, and a post filter stage 514. Other structural variations of the decoder 500 can be used to decode the compressed bitstream 420.

[0063] When the compressed bitstream 420 is presented for decoding, the data elements in the compressed bitstream 420 may be decoded by an entropy decoding stage 502 to generate a set of quantized transform coefficients. A dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by a quantizer value), and an inverse transform stage 506 inverse transforms the dequantized transform coefficients to generate differential residuals that may be identical to those produced by the inverse transform stage 412 in the encoder 400. Using header information decoded from the compressed bitstream 420, the decoder 500 may use an intra / inter prediction stage 508 to create PUs that are identical to those produced in the encoder 400 (e.g., by the intra / inter prediction stage 402).

[0064] In the reconstruction stage 510, the PU may be added to the differential residual to create a reconstructed CU. A loop filtering stage 512 may be applied to the reconstructed CU to reduce blocking artifacts. Examples of filters that may be applied in the loop filtering stage 512 include, but are not limited to, a deblocking filter, a directional enhancement filter, and a loop restoration filter. Other filtering may be applied to the reconstructed CU. In this example, a post-filter stage 514 is applied to the reconstructed CU to reduce blocking distortion, and the result is output as an output video stream 516. The output video stream 516 may also be referred to as a decoded video stream, and these terms are used interchangeably herein.

[0065] Other variations of the decoder 500 can be used to decode the compressed bitstream 420. In some implementations, the decoder 500 can generate the output video stream 516 without or otherwise omitting the post-filter stage 514.

[0066] FIG. 6 is a diagram of an example portion of a video frame 600, which may be, for example, frame 306 shown in FIG. 3. Video frame 600 includes several 64×64 CTUs, such as four 64×64 CTUs 610 in two rows and two columns in a matrix or Cartesian plane, as shown. Each 64×64 CTU 610 may include up to four 32×32 CUs 620. Each 32×32 CU 620 may include up to four 16×16 CUs 630. Each 16×16 CU 630 may include up to four 8×8 CUs 640. Each 8×8 CU 640 may include up to four 4×4 CUs 650. Each 4×4 CU 650 may include 16 pixels, which may be represented by four rows and four columns for each CU in a Cartesian plane or matrix.

[0067] In some implementations, video frame 600 may include CTUs larger than 64x64 and / or CUs smaller than 4x4. Video frame 600 may be partitioned into various arrangements according to features within video frame 600 and / or other criteria. While one arrangement of CUs is shown, any arrangement may be used. While FIG. 6 shows NxN CTUs and CUs, in some implementations, NxM CTUs and / or CUs may be used, where N and M are different numbers. For example, 32x64 CTUs, 64x32 CTUs, 16x32 CUs, 32x16 CUs, or any other size may be used. In some implementations, Nx2N CTUs or CUs, 2NxN CTUs or CUs, or a combination thereof may be used.

[0068] A pixel may include information representing the image captured in video frame 600, such as luminance information, color information, and position information. In some implementations, a block, such as the 16x16 pixel block shown, may include a luminance block 660, which may have luminance pixels 662, and two chrominance blocks 670, 680, such as a U or Cb chrominance block 670 and a V or Cr chrominance block 680. The chrominance blocks 670, 680 may include chrominance pixels 690. For example, as shown, the luminance block 660 may include 16x16 luminance pixels 662, and each chrominance block 670, 680 may include 8x8 chrominance pixels 690.

[0069] In some implementations, coding the video frame 600 may include ordered block-level coding. Ordered block-level coding may include coding the CUs of the video frame 600 in an order, such as a raster-scan order, where the CUs may be identified and processed starting with the CTU in the upper-left corner of the video frame 600, or a portion of the video frame 600, and proceeding from left to right along the rows, and from the top row to the bottom row, identifying each CU for processing in turn. For example, the 64x64 CTU in the top row and left column of the video frame 600 may be the first CTU coded, and the 64x64 CTU immediately to the right of the first CTU may be the second CTU coded. The second-to-top row may be the second row coded, so the 64x64 CTU in the left column of the second row may be coded after the 64x64 CTU in the right-most column of the first row.

[0070] In some implementations, coding the CTUs of video frame 600 may include using quadtree coding, which may include coding smaller CUs within the CTU in raster-scan order. For example, the 64x64 CTU shown in the bottom left corner of a portion of video frame 600 may be coded using quadtree coding, with the top-left 32x32 CU coded, then the top-right 32x32 CU coded, then the bottom-left 32x32 CU coded, then the bottom-right 32x32 CU coded. Each 32x32 CU may be coded using quadtree coding, with the top-left 16x16 CU coded, then the top-right 16x16 CU coded, then the bottom-left 16x16 CU coded, then the bottom-right 16x16 CU coded. Each 16x16 CU may be coded using quad-tree coding, where the top-left 8x8 CU may be coded, then the top-right 8x8 CU may be coded, then the bottom-left 8x8 CU may be coded, then the bottom-right 8x8 CU may be coded. Each 8x8 CU may be coded using quad-tree coding, where the top-left 4x4 CU may be coded, then the top-right 4x4 CU may be coded, then the bottom-left 4x4 CU may be coded, then the bottom-right 4x4 CU may be coded. In some implementations, the 8x8 CU may be omitted for the 16x16 CU, and the 16x16 CU may be coded using quad-tree coding, where the top-left 4x4 CU may be coded, then the remaining 4x4 CUs in the 16x16 CU may be coded in raster-scan order.

[0071] In some implementations, coding video frame 600 may include encoding information contained in the original version of the image or video frame, such as by omitting some of the information in the original version of the image or video frame from the corresponding encoded image or video frame. For example, coding may include reducing spectral redundancy, reducing spatial redundancy, or a combination thereof. Reducing spectral redundancy may include using a color model based on one luminance component (Y) and two chrominance components (U and V or Cb and Cr), which may be referred to as a YUV or YCbCr color model or color space. Using the YUV color model may involve using a relatively large amount of information to represent the luminance component of a portion of video frame 600 and a relatively small amount of information to represent each of the corresponding chrominance components of the portion of video frame 600. For example, a portion of a video frame 600 may be represented by one high-resolution luminance component, which may include a 16x16 block of luma samples, and two lower-resolution chrominance components, each representing a portion of the image as an 8x8 block of chroma samples. The samples may represent values, e.g., in a range of 0 to 255, and may be stored or transmitted, e.g., using 8 bits. While this disclosure is described with reference to the YUV color model, other color models may be used. Reducing spatial redundancy may include transforming the CUs to the frequency domain, e.g., using a discrete cosine transform. For example, a unit of an encoder may perform a discrete cosine transform using transform coefficient values ​​based on spatial frequency.

[0072] Although described herein with reference to a matrix or Cartesian representation of video frame 600 for clarity, video frame 600 may be stored, transmitted, processed, or any combination thereof in a data structure such that pixel values ​​and / or luma and chroma samples can be efficiently represented for video frame 600. For example, video frame 600 may be stored, transmitted, processed, or any combination thereof in a two-dimensional data structure such as a matrix as shown, or in a one-dimensional data structure such as a vector array. Furthermore, although described herein as illustrating a chrominance-subsampled image in which U and V have half the resolution of Y, video frame 600 may have a different configuration for its color channels. For example, with further reference to the YUV color space, full resolution may be used for all color channels of video frame 600. In other examples, color spaces other than the YUV color space may be used to represent the resolution of the color channels of video frame 600.

[0073] FIG. 7 illustrates an example of a reference area 700 for region-based inter-component prediction. The reference area 700 shows chroma samples of a CTU, certain of which are filled with patterns 702, 704, and 706. In particular, the chroma samples filled with pattern 702 correspond to the current PU 708 undergoing prediction, the chroma samples filled with pattern 704 are reconstructed chroma samples that can be used to predict the chroma samples filled with pattern 702, and the chroma samples filled with pattern 706 represent a padding area used to extend the reference area to accommodate predictions of chroma samples located along the edges of the chroma samples filled with pattern 704. The padding area surrounds some or all of the periphery of the reference area 700 and is one or more chroma samples wide. In the example illustrated in FIG. 7, the padding area is one chroma sample wide and is illustrated based on a single chroma sample having pattern 706 adjacent to each of the outermost chroma samples filled with pattern 704. As described below, when using four adjacent samples (e.g., N, S, E, and W) to determine the CCCM filter coefficients for the current luma sample, the padding area ensures that all four adjacent sample areas are available, even for samples along the edges of the portion of reference area 700 filled with pattern 704. In that the chroma samples filled with pattern 706 are not available within the CTU itself, they may be understood to include (i.e., be set to) padding values. Although PU 708 is shown as being of size 8x4, this disclosure is not limited to any particular PU size.

[0074] The reference area 700 may include a top region 710, which may include rows 1 through N (where N>1). The reference area 700 may include a top right region 712, which may include rows 1 through N. The reference area 700 may include a left region 714, which may include columns 1 through M (where M>1). The reference area 700 may include a bottom left region 716, which may include columns 1 through M (where M>1). In an example, N=M. The reference area 700 may be based on a chroma color format. For example, for 4:4:4 content, the reference area 700 may be four samples wide, and for 4:2:0 or 4:2:2 color formats, the reference area 700 may be two samples wide. In an example, if the top right region 712 is available, only the top right 4x4 luma block is included in the reference area 700. Similarly, if the bottom-left region 716 is available, then only the bottom-right 4x4 luma block is included in the reference area 700. The reference area 700 can be adjusted accordingly based on the chroma color format. In another example, the top region 710 can always be one sample wide for both luma and chroma, and the left region 714 can be four samples wide for luma.

[0075] While conventional approaches to CCCM prediction require deriving filter coefficients for each PU, such as PU 708, individually and therefore for a very small portion of reference area 700, the region-based inter-component prediction disclosed herein involves deriving filter coefficients for the entire reference area 700. In this manner, reference area 700 corresponds to a region of the frame undergoing prediction, and more specifically, corresponds to a CTU within that frame that includes PU 708. However, in some cases, reference area 700 may correspond in whole or in part to multiple CTUs, such as the CTU that includes PU 708 and one or more of that CTU's neighboring CTUs.

[0076] 8 illustrates an example neighborhood 800 of luma samples 802 used to predict chroma samples. Neighborhood 800 shows an example 3×3 neighborhood. In some cases, neighborhood 800 may be larger or smaller than 3×3, and / or neighborhood 800 may be a shape other than a square, such as a non-square rectangle or diamond. Luma sample 802 is located within the center of neighborhood 800. Neighboring luma samples 804, 806, 808, and 810 surround luma sample 802, which is labeled C to indicate it is the current luma sample being processed, and are used to predict the chroma samples of luma sample 802. In the illustrated example, luma samples 804, 806, 808, and 810 are labeled using directional designations N, S, E, and W (i.e., north, south, east, and west), respectively, relative to the location of luma sample 802. Luma sample 802 and neighboring luma samples 804, 806, 808, and 810 together comprise the values ​​of the 5-tap spatial components used in CCCM prediction, and these values ​​are used to calculate the predicted chroma sample for luma sample 802, expressed as predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B, where filter coefficients c i is derived for the entire region using region-based inter-component prediction as disclosed herein, not just for the CU containing the luma samples 802.

[0077] 9 shows example resolutions of luma and chroma blocks. As mentioned above, to ensure that appropriate luma samples are used to predict the chroma samples of a given CU 900, it may be desirable to downsample (i.e., reduce the resolution of) the luma blocks of the CU being processed so that the resulting resolution of that luma block is the same as the resolution of the CU's chroma blocks. For example, downsampling may be performed if the resolutions of the luma and chroma blocks are initially provided in a format such as 4:2:0. However, if the resolutions of the luma and chroma blocks of a given CU are already the same (e.g., 4:4:4), the downsampling operation for that CU may be skipped.

[0078] Further details of techniques for region-based inter-component prediction will now be described. Figure 10 is a flowchart diagram of an example technique 1000 for region-based inter-component prediction. Technique 1000 may be performed, for example, in whole or in part, in a prediction stage of an encoder used to encode the video stream (e.g., intra / inter prediction stage 402) or in a prediction stage of a decoder used to decode the bitstream (e.g., intra / inter prediction stage 508).

[0079] Technique 1000 can be implemented as a software program that can be executed by a computing device, such as transmitting station 102 or receiving station 106, for example. For example, the software program can include machine-readable instructions that can be stored in a memory, such as memory 204 or secondary storage 214, and that, when executed by a processor, such as processor 202, can cause the computing device to perform technique 1000. Technique 1000 can be implemented using dedicated hardware or firmware. For example, a hardware component, such as a hardware coder, can be configured to perform technique 1000. As described above, some computing devices may have multiple memories or processors, and the operations described in technique 1000 can be distributed using multiple processors, memories, or both. For ease of explanation, technique 1000 is shown and described herein as a series of steps or operations. However, steps or operations in accordance with the present disclosure can occur in various orders and / or simultaneously. Furthermore, other steps or operations not shown and described herein may be used. Furthermore, not all illustrated steps or operations are required to implement a technique in accordance with the disclosed subject matter.

[0080] At 1002, a region within a current frame being processed (i.e., encoded or decoded) is identified. The region may be, for example, a CTU. During encoding, the region may be identified during frame partitioning as a single CTU. During decoding, the region may be identified using one or more syntax elements signaled in the bitstream. The region has a size larger than the minimum chroma block size. For example, the region may be 128x128 or 64x64.

[0081] In 1004, the region filter coefficients are determined for this region. The region filter coefficients are the CCCM prediction filter coefficients (i.e., the filter coefficients c i) Determining the regional filter coefficients may include deriving the regional filter coefficients based on one or more previously coded spatially neighboring regions, identifying the regional filter coefficients using one or more syntax elements signaled in the bitstream, or both. Deriving the regional filter coefficients includes minimizing the MSE between predicted and reconstructed chroma samples within a reference area, e.g., reference area 700 shown in FIG. 7. Thus, while previous CCCM prediction approaches derive filter coefficients for individual CUs and thus use predicted and reconstructed chroma samples that are restricted to only a portion of the reference area corresponding to a given CU, deriving the regional filter coefficients includes minimizing the MSE using the entire reference area. However, in some cases, some or all of the padding portion of the reference area (e.g., samples having pattern 706 shown in FIG. 7) may be excluded from the regional filter coefficient determination process. As shown with respect to FIG. 9, if downsampling is performed, the downsampling may be performed before the regional filter coefficients are determined.

[0082] In some implementations, regional filter coefficients can be derived for the identified region using chroma samples reconstructed from one or more other regions. For example, if the identified region is the current CTU, the regional filter coefficients can be derived using chroma samples reconstructed from one or both of the CTU adjacent to the left of the current CTU or the CTU adjacent above the current CTU. In another example, if the identified region is the current CTU, the regional filter coefficients can be derived using chroma samples reconstructed from one or more of the CTU adjacent to the top left of the current CTU, the CTU adjacent to the top right of the current CTU, the CTU adjacent to the bottom left of the current CTU, or the CTU adjacent to the bottom right of the current CTU.

[0083] During encoding, the region filter coefficients for the region are derived, but during decoding, the region filter coefficients for the region may be derived and / or signaled. For example, signaling the region filter coefficients may include explicitly or implicitly signaling the region filter coefficients in a bitstream, such as in an adaptation parameter set, slice header, or other structure available for storing syntax elements for use in decoding encoded video data from the bitstream. In some cases, some but not all of the region filter coefficients for a region may be signaled. In such cases, the remaining region filter coefficients may be derived as described above. For example, in such cases, a first subset of the region filter coefficients may be signaled and a second subset of the region filter coefficients may be derived.

[0084] Furthermore, in some cases, one or more region filter coefficients signaled in the bitstream may be refined as part of the process for determining the region filter coefficients. For example, refining the region filter coefficients may include deriving region filter coefficients as described above and comparing the derived region filter coefficients with the signaled region filter coefficients. In some such cases, if the comparison indicates that the derived region filter coefficients lie within a first threshold range of the signaled region filter coefficients, the signaled region filter coefficients or the derived region filter coefficients may be used as the refined region filter coefficients. In other such cases, if the comparison indicates that the derived region filter coefficients lie outside the first threshold range of the signaled region filter coefficients but within a second threshold range thereof, the signaled region filter coefficients and the derived region filter coefficients may be combined (e.g., averaged) to generate the refined region filter coefficients. In yet other such cases, if the comparison indicates that there are derived domain filter coefficients outside both the first and second threshold ranges of the signaled domain filter coefficients, the derived domain filter coefficients may be used as refined domain filter coefficients. Other examples are possible.

[0085] In some implementations, determining regional filter coefficients may include determining multiple sets of regional filter coefficients for the region. For example, different sets of regional filter coefficients may be determined based on different classifications of reconstructed luma samples for the region. In such cases, luma samples corresponding to the same classification may be understood to share the same set of regional filter coefficients. The classifications may be derived in parallel, and therefore there is no dependency between them. The classifications may be based on, for example, gradient, direction, pixel value band, etc. For example, gradient-based classifications may be derived at the 4x4 luma block level. In another example, band-based classifications may be derived at the 2x2 luma block level based on the average value of a given 2x2 luma block. In some cases, overlapped classifications may be used, in which a sample may be counted in more than one classification. For overlapped classifications, different weights may be used. For example, a sample may have a larger weight if it is directly classified into a target classification than if it is not. In some cases, padding (e.g., pixel repetition) may be used for classification and prediction where luma samples are needed but not reconstructed.

[0086] At 1006, an input value is determined for a current luma sample within the region. In particular, the current luma sample is located within a sub-portion of the region, e.g., the CU or PU undergoing prediction. The input value includes the current luma sample, the number of neighboring luma samples of the current luma sample, and the bit precision of the video data being encoded or decoded. For example, the input values ​​may correspond to 7 taps of a 7-tap filter used for CCCM prediction, which are expressed as powers of 2 for the current luma sample C, the four neighboring luma samples N, S, E, and W, C, and a nonlinear term P (e.g., P=(C *C+midVal)>>bitDepth, where bitDepth represents the bit precision of the video content and midVal is the midpoint chroma value within that bit precision), as well as a bias term B, which is expressed as a scalar offset between the input and output and is set to the midpoint chroma value of the bit precision, similar to the offset term in CCLM prediction.

[0087] The current luma sample and the number of neighboring luma samples are identified using a filter. In one example, the filter may be applied to a 3x3 neighborhood within the CU containing the current luma sample, as shown in Figure 8. The filter may have a plus shape as used by the example of Figure 8, so that the number of neighboring luma samples includes four neighboring luma samples, labeled N, S, E, and W as shown in Figure 8, but other example shapes may be used, and neighborhoods of other sizes may be used. For example, an x-shaped filter may be used for a 3x3 neighborhood, a diamond-shaped filter may be used for a 5x5 neighborhood, and so on. In some implementations, a filter with a number of coefficients below a threshold may be used for CUs below a specified size (e.g., 8x8), and / or a filter with a number of coefficients above a threshold may be used for CUs above the specified size.

[0088] At 1008, a predicted chroma sample is determined based on the input value of the current luma sample and the domain filter coefficients. For example, a predicted chroma sample, represented as predChromaVal, may be determined by calculating predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B, where C, N, S, E, W, P, and B are the input value of the current luma sample, and c0, c1, c2, c3, c4, c5, and c6 are domain filter coefficients.

[0089] In some implementations, the predicted chroma sample may be determined by a weighted combination (e.g., average) of a first predicted chroma sample determined as described above (i.e., using the region-based inter-component prediction disclosed herein) and a second predicted chroma sample determined using CCLM prediction. Thus, the predicted chroma sample may be determined for a portion of the region (e.g., the CU containing the current luma sample) by a weighted combination (e.g., average) of sample values ​​using spatial weighting values ​​determined according to the prediction approach for that portion. For example, the weighting values ​​used may depend on the sample position relative to the CU, such that larger weighting values ​​are used for region-based inter-component prediction in the bottom and / or right portions of the CU and larger weighting values ​​are used for CCLM prediction in the top and / or left portions of the CU. This may be desirable because CCLM prediction generally adapts well to local textures, while the region-based inter-component prediction disclosed herein generally adapts well to large regions. The weighting values ​​may be predefined (eg, for use during encoding) or may be signaled within the bitstream (eg, for use during decoding).

[0090] In some implementations, the regional filter coefficients used to determine the predicted chroma sample may be a first set of regional filter coefficients, and a second set of regional filter coefficients may be used to determine a second predicted chroma sample for the identified region. For example, the particular regional filter coefficients to use for the predicted chroma sample and the second predicted chroma sample may be based on classification information of the luma samples corresponding to those predicted chroma samples.

[0091] At 1010, the predictive chroma sample is encoded (e.g., into a bitstream) or decoded (e.g., for output in an output video stream) depending on whether technique 1000 is performed during encoding or decoding. In some implementations, the predictive chroma sample may be reconstructed for use in predicting one or more other chroma samples within a region (e.g., within the same CU or PU in which the current luma sample is located and to which the predictive chroma sample corresponds).

[0092] Technique 1000 describes an approach for predicting chroma samples corresponding to luma samples, which in some cases may include predicting chroma samples for each luma sample accordingly (e.g., within a CU, CTU, or otherwise), and in some cases technique 1000 may be used to predict chroma samples for some but not all luma samples.

[0093] The encoding and decoding aspects described above illustrate some examples of encoding and decoding techniques, although it should be understood that encoding and decoding, as the terms are used in the claims, may mean compressing, decompressing, converting, or other data processing or modification.

[0094] The word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" should not necessarily be construed as preferred or advantageous over other aspects or designs. Rather, use of the word "exemplary" is intended to present concepts in a concrete manner. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clearly indicated otherwise by context, the statement "X includes A or B" is intended to refer to its natural inclusive permutations. That is, if X includes A, if X includes B, or if X includes both A and B, then "X includes A or B" is satisfied in each of the foregoing cases. Additionally, the articles "a" and "an," as used in this application and the appended claims, should generally be construed to mean "one or more" unless otherwise specified or unless the context clearly indicates a singular reference. Furthermore, the use of the term "embodiment" or "one embodiment" throughout this disclosure is not intended to refer to the same embodiment unless specifically described as such.

[0095] Implementations of the transmitting station 102 and / or receiving station 106 (and the algorithms, methods, instructions, etc. stored therein and / or executed thereby, e.g., by the encoder 400 and decoder 500, or other encoders or decoders disclosed herein) can be realized in hardware, software, or any combination thereof. Hardware can include, for example, computers, intellectual property (IP) cores, application specific integrated circuits (ASICs), programmable logic arrays, optical processors, programmable logic controllers, microcode, microcontrollers, servers, microprocessors, digital signal processors, or any other suitable circuitry. In the claims, the term "processor" should be understood to include any of the foregoing hardware, alone or in combination. The terms "signal" and "data" are used interchangeably. Furthermore, portions of the transmitting station 102 and receiving station 106 need not necessarily be implemented in the same way.

[0096] Furthermore, in one aspect, for example, the transmitting station 102 or the receiving station 106 may be implemented using a general-purpose computer or processor with a computer program that, when executed, performs any of the respective methods, algorithms, and / or instructions described herein. Additionally or alternatively, a special-purpose computer / processor may be utilized that may include, for example, other hardware for performing any of the methods, algorithms, or instructions described herein.

[0097] The transmitting station 102 and the receiving station 106 can be implemented on computers, for example, within a videoconferencing system. Alternatively, the transmitting station 102 can be implemented on a server, and the receiving station 106 can be implemented on a device separate from the server, such as a handheld communication device. In this case, the transmitting station 102 can encode content into an encoded video signal and transmit the encoded video signal to the communication device. The communication device can then decode the encoded video signal. Alternatively, the communication device can decode content stored locally on the communication device, e.g., content not transmitted by the transmitting station 102. Other suitable transmitting and receiving implementations are also possible. For example, the receiving station 106 can be a generally stationary personal computer rather than a portable communication device.

[0098] Furthermore, all or part of the embodiments of the present disclosure may take the form of a computer program product accessible, for example, from a computer-usable or computer-readable medium. The computer-usable or computer-readable medium may be, for example, any device that can tangibly store, store, communicate, or transfer a program for use by or in connection with any processor. The medium may include, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable media may also be used.

[0099] The above-described embodiments and other aspects have been set forth to facilitate understanding of the present disclosure, but are not intended to limit the present disclosure. On the contrary, the present disclosure is intended to cover various modifications and equivalent arrangements that fall within the scope of the appended claims, which scope is to be accorded the broadest interpretation permitted under law so as to encompass all such modifications and equivalent arrangements.

Claims

1. 1. A method for region-based inter-component prediction, comprising: identifying a region within a frame to encode or decode; determining regional filter coefficients for the region; determining an input value of a current luma sample within a portion of the region; determining a predicted chroma sample for the current luma sample based on the input value and the region filter coefficients; encoding or decoding the predicted chroma samples; A method comprising:

2. Determining the region filter coefficients comprises: The method of claim 1 , comprising deriving at least a portion of the region filter coefficients based on one or both of the region and adjacent regions.

3. Deriving at least the portion of the region filter coefficients based on one or both of the region or the adjacent regions may include: The method of claim 2 , comprising minimizing a mean square error between predicted and reconstructed chroma samples within a reference area of ​​the frame.

4. The method of claim 3 , wherein the mean squared error is performed using chroma samples from a padding area outside the region.

5. Determining the region filter coefficients comprises: The method of claim 1 , comprising decoding one or more syntax elements used to signal the region filter coefficients from a bitstream associated with the frame.

6. 6. The method of claim 1, comprising determining to use the regional filter coefficients for determining the predicted chroma sample based on a classification of the current luma sample.

7. the portion of the region is a coding unit; The method of claim 6 , wherein different regional filter coefficients are used to determine a second predicted chroma sample based on a classification of a second current luma sample within the coding unit.

8. identifying the region 6. The method of claim 1, 2, 3, 4, or 5, comprising decoding one or more syntax elements associated with the region signaled in a bitstream.

9. Determining the predicted chroma samples includes: determining spatial weight values ​​for areas of the portion of the region according to a predictive approach to use for the areas; determining the predicted chroma samples using the spatial weight values; 6. The method of claim 1, 2, 3, 4, or 5, comprising:

10. 6. The method of claim 1, wherein the portion of the region is a coding unit, and the region filter coefficients are determined for use with multiple coding units of the region.

11. The method of claim 1 , wherein the size of the region is greater than a minimum chroma unit size.

12. The method of claim 11 , wherein the regions are coding tree units of size 128×128 or 64×64.

13. 1. An apparatus for region-based inter-component prediction, comprising: The apparatus includes a memory and a processor; The processor includes: determining regional filter coefficients for a region within a frame to encode or decode; determining a first prediction chroma sample for the first luma sample in a first portion of the region based on an input value of the first luma sample and based on the region filter coefficients; determining a second prediction chroma sample for the second luma sample in a second portion of the region based on the input value of the second luma sample and based on the region filter coefficients; encoding or decoding the first predicted chroma sample and the second predicted chroma sample; 20. An apparatus configured to execute instructions for:

14. 14. The apparatus of claim 13, wherein a first portion of the domain filter coefficients are signaled in a bitstream associated with the frame, and a second portion of the domain filter coefficients are derived based on video data in the frame.

15. 14. The apparatus of claim 13, wherein the region is a current coding tree unit, and the region filter coefficients are derived using reconstructed chroma samples from one or more neighboring coding tree units of the current coding tree unit.

16. 16. The apparatus of claim 13, 14, or 15, wherein the regional filter coefficients are used for both the first predicted chroma sample and the second predicted chroma sample based on classifications of the first luma sample and the second luma sample.

17. The apparatus of claim 16 , wherein the classification is based on one or more of gradient, direction, or pixel value band.

18. a non-transitory computer-readable storage device containing program instructions executable by one or more processors, the program instructions, when executed, causing the one or more processors to perform region-based inter-component prediction operations; The calculation is determining filter coefficients to use to predict chroma samples in a plurality of coding units of a coding tree unit in a frame for encoding or decoding; determining a current luma sample within a coding unit of the plurality of coding units; determining a predicted chroma sample for the current luma sample based on an input value and the filter coefficients; encoding or decoding the predicted chroma samples; a non-transitory computer readable storage device including:

19. Determining the filter coefficients comprises: deriving the filter coefficients based on one or both of the coding tree unit and neighboring coding tree units of the coding tree unit; decoding, from a bitstream associated with the frame, one or more syntax elements used to signal the filter coefficients; or deriving a first portion of the filter coefficients and decoding a second portion of the filter coefficients from the bitstream; 20. The non-transitory computer-readable storage device of claim 18, comprising one of:

20. Determining the predicted chroma samples includes: determining spatial weight values ​​for areas of the coding unit according to a prediction approach to use for the areas; determining the predicted chroma samples using the spatial weight values; 20. The non-transitory computer-readable storage device of claim 18, comprising:

Images