Scaling in cross-component sample offset

Abstract

Methods, devices, and non-transitory computer-readable storage medium are provided for use in video encoding or decoding where at least one parameter of a cross-component sample offset filter is scaled to produce at least one scaled parameter, and the cross-component sample offset filter is applied to reconstructed pixels of a video frame to produce filtered pixels using the at least one scaled parameter. The scaling can be based on a scaling factor encoded in a compressed bitstream or on a bit-depth of the video frame. The scaling factor can be determined by an encoder using a rate distortion analysis and encoded into the compressed bitstream. The at least one parameter can include a quantization step and a plurality of offsets. The encoded scaling factor can be a 1-bit value indicating that the at least one parameter is to be multiplied by four.

Description

Atty. Doc. No. GOGL-2265-A-WO PATENTSCALING IN CROSS-COMPONENT SAMPLE OFFSETCROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the benefit of U.S. Provisional Patent Application Serial No. 63 / 692,129, filed September 8, 2024, the entire disclosure of which is incorporated herein by reference.BACKGROUND

[0002] Digital video streams may represent video using a sequence of frames or still images. Digital video can be used for various applications including, for example, video conferencing, high-definition video entertainment, video advertisements, or sharing of usergenerated videos. A digital video stream can contain a large amount of data and consume a significant amount of computing or communication resources of a computing device for processing, transmission, or storage of the video data. Various approaches have been proposed to reduce the amount of data in video streams, including encoding or decoding techniques.SUMMARY

[0003] An aspect of the disclosed implementations is a method that includes scaling at least one parameter of a cross-component sample offset filter to produce at least one scaled parameter and applying the cross-component sample offset filter to reconstructed pixels of a video frame to produce filtered pixels. The cross-component sample offset filter is applied using the at least one scaled parameter.

[0004] An aspect of the disclosed implementations is a non-transitory computer-readable storage medium that stores instructions that when executed by a processor performs steps of the foregoing method.

[0005] An aspect of the disclosed implementations is a non-transitory computer-readable storage medium that stores a compressed bitstream that is encodable or decodable using steps of the foregoing method.

[0006] These and other aspects of the present disclosure are disclosed in the following detailed description of the embodiments, the appended claims and the accompanying figures.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The description herein makes reference to the accompanying drawings described below, wherein like reference numerals refer to like parts throughout the several views.

[0008] FIG. 1 is a schematic of a video encoding and decoding system.

[0009] FIG. 2 is a block diagram of an example of a computing device that can implement a transmitting station or a receiving station.

[0010] FIG. 3 is a diagram of a typical video stream to be encoded and subsequently decoded.

[0011] FIG. 4 is a block diagram of an encoder according to implementations of this disclosure.

[0012] FIG. 5 is a block diagram of a decoder according to implementations of this disclosure.

[0013] FIG. 6 is an illustration of examples of portions of a video frame.

[0014] FIG. 7 is a block diagram of an example of a video frame filtering stage.

[0015] FIG. 8 is a block diagram that illustrates the operations of a Cross -ComponentSample Offset (CCSO) filter.

[0016] FIG. 9 illustrates syntax elements that may be used to signal aspects of the CCSO filter.

[0017] FIG. 10 illustrates a flowchart of a technique for scaling a CCSO filter.DETAIEED DESCRIPTION

[0018] Video compression schemes may include breaking respective images, or frames, of a video stream into smaller portions, such as blocks, or coding tree units (CTUs), and generating an encoded bitstream using techniques to limit the information included for respective CTUs thereof. The bitstream can be decoded to re-create the source frames from the limited information. Encoding CTUs to or decoding CTUs from a bitstream can include predicting the values of pixels or CTUs based on similarities with other pixels or CTUs in the same frame which have already been coded. Those similarities can be determined using intra prediction, which attempts to predict the pixel values of a coding unit (CU) of a CTU using pixels peripheral to the CU (e.g., pixels that are in the same frame as the CU, but which are outside the CU). During encoding, the result of an intra-prediction mode performed against a CU is a prediction unit (PU). A prediction residual can be determined based on a difference between the pixel values of the CU and the pixel values of the PU. The prediction residual and the intra prediction mode used to ultimately obtain that prediction residual can then beencoded to a bitstream. During decoding, the prediction residual is reconstructed into a CU using a PU produced based on the intra prediction mode and is thereafter included in an output video stream.

[0019] A CU includes a luminance, also referred to as luma, component and two chrominance, also referred to as chroma, components. These luma and chroma components may in some cases be referred to as a luma block and chroma blocks. The luma component of a CU may, for example, be expressed within a Y component (also referred as “plane”) of the CU and the chroma components may be expressed either within U and V components or Cr and Cb components of the CU. The luma component is understood to include some number of luma samples and each chroma component is understood to include some number of chroma samples. Generally, the luma samples provide measures of brightness throughout a subject CU and thus represent the structural qualities of the video content of the subject CU, whereas the chroma samples provide measures of color throughout the subject CU. Because of this, conventional video compression schemes often use finer prediction approaches for predicting luma components of CUs than chroma components thereof. Such schemes may also use approaches directed to predicting those chroma components from the predicted luma components.

[0020] The process of video compression and decompression can introduce artifacts and distortions in the reconstructed video. To mitigate these issues, various filtering techniques are often employed during the decoding process. These filters aim to improve the visual quality of the reconstructed video, such as by smoothing out blocky artifacts, reducing noise, and enhancing details. Depending on the specific codec and configuration, zero or more filters may be applied to a reconstructed block. One such filter, known as CCSO, leverages the correlation between different color components (luma and chroma) to enhance the visual quality of the reconstructed video.

[0021] Briefly, the CCSO filter operates by adjusting at least one of luma or chroma samples based on the characteristics of corresponding and neighboring samples. CCSO utilizes a lookup table (LUT) that maps quantized differences between neighboring samples to offset values applied to current samples. The offset values are determined by the encoder and transmitted to the decoder. The encoder determines these offset values based on the source video data.

[0022] The behavior of the CCSO filter can be controlled through various syntax elements signaled by an encoder to a decoder in a compressed video bitstream. Among these elements are block-level control flags, which direct the decoder on whether to apply the CCSO filter toa specific color component (i.e., color plane) of a block. A block-level control flag associated with a color component is used by the decoder to determine whether the CCSO filter should be applied to that color component of the particular block. The block-level control flags are applied at the CCSO unit level. A reconstructed frame is partitioned into CCSO units (i.e., filter processing units), and the application of the CCSO filter to a CCSO unit is determined by the corresponding block-level control flag. The size of a CCSO unit can be that of the largest coding block, which may also be referred to as a macroblock or superblock. For some codecs, the largest coding block size can be 256x256 pixels for a luma block or 128x128 pixels for a corresponding chroma block; however, other sizes for the CCSO unit are possible. A CCSO unit may contain multiple variable-size coding blocks used for prediction and transform coding and may operate independently of the coding block partitioning (e.g., as used for prediction and transformation). The CCSO filter is further described with respect to FIG. 8.

[0023] The syntax elements utilized to control the behavior of the CCSO filter provide a limited range of adaptability for the filter (e.g., based on the number of bits used for the syntax elements). For example, an implementation of a CCSO filter may provide for up to 4 quantization steps (2-bit syntax element) and 9 3-bit offset values (each providing up to 8 possible values).

[0024] Implementations according to this disclosure utilize a scaling factor to scale syntax elements (or resulting parameters) used to control the behavior of the CCSO filter. For example, a scale syntax element may be included in an encoded bitstream by an encoder and decoded at the decoder to scale the quantization step, one or more of the offset values, or combinations thereof. For example, the scale syntax element may allow for the selection of scaling factors from { 1, 4}. For example, scaling may be performed by multiplying the scaling factor and the parameters to be scaled. In some implementations, the scaling factor may be dependent on a bit depth or a resolution of a frame. For example, the scaling factors may be different for an 8-bit depth as compared to a 12-bit depth (e.g., the scaling factor for the 8-bit depth may be less than the 12-bit depth). In some implementations, the scaling factor may be determined at the decoder based on bit depth or resolution instead of being determined by the encoder and transmitted in the encoded bitstream.

[0025] By permitting the scaling of CCSO parameters, the range of operation of the CCSO filter is increased and thus may be able to more accurately filter across a wider range of conditions, such as in the case of frames with higher bit depth (e.g., having a larger range of potential pixel values) or in frames with higher resolution (e.g., which may have smaller changes in pixel values between pixels).

[0026] Further details of techniques for scaling in cross-component sample offset filtering are described herein with initial reference to a system in which they can be implemented.

[0027] FIG. 1 is a schematic of a video encoding and decoding system 100. A transmitting station 102 can be, for example, a computer having an internal configuration of hardware such as that described in FIG. 2. However, other implementations of the transmitting station 102 are possible. For example, the processing of the transmitting station 102 can be distributed among multiple devices.

[0028] A network 104 can connect the transmitting station 102 and a receiving station 106 for encoding and decoding of the video stream. Specifically, the video stream can be encoded in the transmitting station 102, and the encoded video stream can be decoded in the receiving station 106. The network 104 can be, for example, the Internet. The network 104 can also be a local area network (LAN), wide area network (WAN), virtual private network (VPN), cellular telephone network, or any other means of transferring the video stream from the transmitting station 102 to, in this example, the receiving station 106.

[0029] The receiving station 106, in one example, can be a computer having an internal configuration of hardware such as that described in FIG. 2. However, other suitable implementations of the receiving station 106 are possible. For example, the processing of the receiving station 106 can be distributed among multiple devices.

[0030] Other implementations of the video encoding and decoding system 100 are possible. For example, an implementation can omit the network 104. In another implementation, a video stream can be encoded and then stored for transmission at a later time to the receiving station 106 or any other device having memory. In one implementation, the receiving station 106 receives (e.g., via the network 104, a computer bus, and / or some communication pathway) the encoded video stream and stores the video stream for later decoding. In an example implementation, a real-time transport protocol (RTP) is used for transmission of the encoded video over the network 104. In another implementation, a transport protocol other than RTP may be used, e.g., a video streaming protocol based on the Hypertext Transfer Protocol (HTTP).

[0031] When used in a video conferencing system, for example, the transmitting station 102 and / or the receiving station 106 may include the ability to both encode and decode a video stream as described below. For example, the receiving station 106 could be a video conference participant who receives an encoded video bitstream from a video conference server (e.g., the transmitting station 102) to decode and view and further encodes andtransmits his or her own video bitstream to the video conference server for decoding and viewing by other participants.

[0032] FIG. 2 is a block diagram of an example of a computing device 200 that can implement a transmitting station or a receiving station. For example, the computing device 200 can implement one or both of the transmitting station 102 and the receiving station 106 of FIG. 1. The computing device 200 can be in the form of a computing system including multiple computing devices, or in the form of one computing device, for example, a mobile phone, a tablet computer, a laptop computer, a notebook computer, a desktop computer, and the like.

[0033] A processor 202 in the computing device 200 can be a conventional central processing unit. Alternatively, the processor 202 can be another type of device, or multiple devices, capable of manipulating or processing information now existing or hereafter developed. For example, although the disclosed implementations can be practiced with one processor as shown (e.g., the processor 202), advantages in speed and efficiency can be achieved by using more than one processor.

[0034] A memory 204 in computing device 200 can be a read-only memory (ROM) device or a random-access memory (RAM) device in an implementation. However, other suitable types of storage device can be used as the memory 204. The memory 204 can include code and data 206 that is accessed by the processor 202 using a bus 212. The memory 204 can further include an operating system 208 and application programs 210, the application programs 210 including at least one program that permits the processor 202 to perform the techniques described herein. For example, the application programs 210 can include applications 1 through N, which further include a video coding application that performs the techniques described herein. The computing device 200 can also include a secondary storage 214, which can, for example, be a memory card used with a mobile computing device.Because the video communication sessions may contain a significant amount of information, they can be stored in whole or in part in the secondary storage 214 and loaded into the memory 204 as needed for processing.

[0035] The computing device 200 can also include one or more output devices, such as a display 218. The display 218 may be, in one example, a touch sensitive display that combines a display with a touch sensitive element that is operable to sense touch inputs. The display 218 can be coupled to the processor 202 via the bus 212. Other output devices that permit a user to program or otherwise use the computing device 200 can be provided in addition to or as an alternative to the display 218. When the output device is or includes a display, thedisplay can be implemented in various ways, including by a liquid crystal display (LCD), a cathode-ray tube (CRT) display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0036] The computing device 200 can also include or be in communication with an image-sensing device 220, for example, a camera, or any other image-sensing device 220 now existing or hereafter developed that can sense an image such as the image of a user operating the computing device 200. The image-sensing device 220 can be positioned such that it is directed toward the user operating the computing device 200. In an example, the position and optical axis of the image-sensing device 220 can be configured such that the field of vision includes an area that is directly adjacent to the display 218 and from which the display 218 is visible.

[0037] The computing device 200 can also include or be in communication with a soundsensing device 222, for example, a microphone, or any other sound-sensing device now existing or hereafter developed that can sense sounds near the computing device 200. The sound-sensing device 222 can be positioned such that it is directed toward the user operating the computing device 200 and can be configured to receive sounds, for example, speech or other utterances, made by the user while the user operates the computing device 200.

[0038] Although FIG. 2 depicts the processor 202 and the memory 204 of the computing device 200 as being integrated into one unit, other configurations can be utilized. The operations of the processor 202 can be distributed across multiple machines (wherein individual machines can have one or more processors) that can be coupled directly or across a local area or other network. The memory 204 can be distributed across multiple machines such as a network-based memory or memory in multiple machines performing the operations of the computing device 200. Although depicted here as one bus, the bus 212 of the computing device 200 can be composed of multiple buses. Further, the secondary storage 214 can be directly coupled to the other components of the computing device 200 or can be accessed via a network and can comprise an integrated unit such as a memory card or multiple units such as multiple memory cards. The computing device 200 can thus be implemented in a wide variety of configurations.

[0039] FIG. 3 is a diagram of an example of a video stream 300 to be encoded and subsequently decoded. The video stream 300 includes a video sequence 302. At the next level, the video sequence 302 includes a number of adjacent frames 304. While three frames are depicted as the adjacent frames 304, the video sequence 302 can include any number of adjacent frames 304. The adjacent frames 304 can then be further subdivided into individualframes, for example, a frame 306. At the next level, the frame 306 can be divided into a series of planes or segments 308. The segments 308 can be subsets of frames that permit parallel processing, for example. The segments 308 can also be subsets of frames that can separate the video data into separate colors. For example, a frame 306 of color video data can include a luminance component and two chrominance components. The segments 308 may be sampled at different resolutions.

[0040] Whether or not the frame 306 is divided into segments 308, the frame 306 may be further subdivided into blocks 310, which can contain data corresponding to, for example, 16x16 pixels in the frame 306. The blocks 310 can also be arranged to include data from one or more segments 308 of pixel data. The blocks 310 can also be of any other suitable size such as 4x4 pixels, 8x8 pixels, 16x8 pixels, 8x16 pixels, 16x16 pixels, or larger. Unless otherwise noted, the terms block and macroblock are used interchangeably herein.

[0041] FIG. 4 is a block diagram of an encoder 400 according to implementations of this disclosure. The encoder 400 can be implemented, as described above, in the transmitting station 102, such as by providing a computer software program stored in memory, for example, the memory 204. The computer software program can include machine instructions that, when executed by a processor such as the processor 202, cause the transmitting station 102 to encode video data in the manner described in FIG. 4. The encoder 400 can also be implemented as specialized hardware included in, for example, the transmitting station 102. In one particularly desirable implementation, the encoder 400 is a hardware encoder.

[0042] The encoder 400 has the following stages to perform the various functions in a forward path (shown by the solid connection lines) to produce an encoded or compressed bitstream 420 using the video stream 300 as input: an intra / inter prediction stage 402, a transform stage 404, a quantization stage 406, and an entropy encoding stage 408. The encoder 400 may also include a reconstruction path (shown by the dotted connection lines) to reconstruct a frame for encoding of future blocks. In FIG. 4, the encoder 400 has the following stages to perform the various functions in the reconstruction path: a dequantization stage 410, an inverse transform stage 412, a reconstruction stage 414, and a loop filtering stage 416. Other structural variations of the encoder 400 can be used to encode the video stream 300.

[0043] When the video stream 300 is presented for encoding, respective adjacent frames 304, such as the frame 306, can be processed in units of blocks. At the intra / inter prediction stage 402, respective blocks can be encoded using intra-frame prediction (also called intraprediction) or inter- frame prediction (also called inter-prediction). In any case, a predictionblock can be formed. In the case of intra-prediction, a prediction block may be formed from samples in the current frame that have been previously encoded and reconstructed. In the case of inter-prediction, a prediction block may be formed from samples in one or more previously constructed reference frames.

[0044] Next, the prediction block can be subtracted from the current block at the intra / inter prediction stage 402 to produce a residual block (also called a residual). The transform stage 404 transforms the residual into transform coefficients in, for example, the frequency domain using block-based transforms. The quantization stage 406 converts the transform coefficients into discrete quantum values, which are referred to as quantized transform coefficients, using a quantizer value or a quantization level. For example, the transform coefficients may be divided by the quantizer value and truncated.

[0045] The quantized transform coefficients are then entropy encoded by the entropy encoding stage 408. The entropy-encoded coefficients, together with other information used to decode the block (which may include, for example, syntax elements such as used to indicate the type of prediction used, transform type, motion vectors, a quantizer value, or the like), are then output to the compressed bitstream 420. The compressed bitstream 420 can be formatted using various techniques, such as variable length coding (VLC) or arithmetic coding. The compressed bitstream 420 can also be referred to as an encoded video stream or encoded video bitstream, and the terms will be used interchangeably herein.

[0046] The reconstruction path (shown by the dotted connection lines) can be used to ensure that the encoder 400 and a decoder 500 (described below with respect to FIG. 5) use the same reference frames to decode the compressed bitstream 420. The reconstruction path performs functions that are similar to functions that take place during the decoding process (described below with respect to FIG. 5), including dequantizing the quantized transform coefficients at the dequantization stage 410 and inverse transforming the dequantized transform coefficients at the inverse transform stage 412 to produce a derivative residual block (also called a derivative residual). At the reconstruction stage 414, the prediction block that was predicted at the intra / inter prediction stage 402 can be added to the derivative residual to create a reconstructed block. The loop filtering stage 416 can be applied to the reconstructed block to reduce distortion such as blocking artifacts. The CCSO filter, as described herein, would be applied within the loop filtering stage 416. However, in some implementations, the CCSO filter could instead be applied as a part of a rate distortion analysis in order to evaluate distortion of a decoded output (e.g., in the case where the CCSO filter is utilized as a post filter in the decoder) or not be applied by the encoder.

[0047] Other variations of the encoder 400 can be used to encode the compressed bitstream 420. In some implementations, a non-transform based encoder can quantize the residual signal directly without the transform stage 404 for certain blocks or frames. In some implementations, an encoder can have the quantization stage 406 and the dequantization stage 410 combined in a common stage.

[0048] FIG. 5 is a block diagram of a decoder 500 according to implementations of this disclosure. The decoder 500 can be implemented in the receiving station 106, for example, by providing a computer software program stored in the memory 204. The computer software program can include machine instructions that, when executed by a processor such as the processor 202, cause the receiving station 106 to decode video data in the manner described in FIG. 5. The decoder 500 can also be implemented in hardware included in, for example, the transmitting station 102 or the receiving station 106.

[0049] The decoder 500, similar to the reconstruction path of the encoder 400 discussed above, includes in one example the following stages to perform various functions to produce an output video stream 516 from the compressed bitstream 420: an entropy decoding stage 502, a dequantization stage 504, an inverse transform stage 506, an intra / inter prediction stage 508, a reconstruction stage 510, a loop filtering stage 512, and a post filtering stage 514. Other structural variations of the decoder 500 can be used to decode the compressed bitstream 420.

[0050] When the compressed bitstream 420 is presented for decoding, the data elements within the compressed bitstream 420 can be decoded by the entropy decoding stage 502 to produce a set of quantized transform coefficients. The dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by the quantizer value), and the inverse transform stage 506 inverse transforms the dequantized transform coefficients to produce a derivative residual that can be identical to that created by the inverse transform stage 412 in the encoder 400. Using header information decoded from the compressed bitstream 420, the decoder 500 can use the intra / inter prediction stage 508 to create the same prediction block as was created in the encoder 400 (e.g., at the intra / inter prediction stage 402).

[0051] At the reconstruction stage 510, the prediction block can be added to the derivative residual to create a reconstructed block. The loop filtering stage 512 can be applied to the reconstructed block to reduce blocking artifacts. Other filtering can be applied to the reconstructed block. In this example, the post filtering stage 514 is applied to the reconstructed block to reduce blocking distortion, and the result is output as the output videostream 516. The CCSO filter, as described herein, would be applied within the loop filtering stage 512. However, in some implementations, the CCSO filter could be applied elsewhere, such as in post filtering stage 514. The output video stream 516 can also be referred to as a decoded video stream, and the terms will be used interchangeably herein. Other variations of the decoder 500 can be used to decode the compressed bitstream 420. In some implementations, the decoder 500 can produce the output video stream 516 without the post filtering stage 514.

[0052] FIG. 6 is an illustration of examples of portions of a video frame 600, which may, for example, be the frame 306 shown in FIG. 3. The video frame 600 includes a number of 64x64 CTUs, such as four 64x64 CTUs 610 in two rows and two columns in a matrix or Cartesian component, as shown. Each 64x64 CTU 610 may include up to four 32x32 CUs 620. Each 32x32 CU 620 may include up to four 16x16 CUs 630. Each 16x16 CU 630 may include up to four 8x8 CUs 640. Each 8x8 CU 640 may include up to four 4x4 CUs 650. Each 4x4 CU 650 may include 16 pixels, which may be represented in four rows and four columns in each respective CU in the Cartesian plane or matrix.

[0053] In some implementations, the video frame 600 may include CTUs larger than 64x64 and / or CUs smaller than 4x4. Subject to features within the video frame 600 and / or other criteria, the video frame 600 may be partitioned into various arrangements. Although one arrangement of CUs is shown, any arrangement may be used. Although FIG. 6 shows NxN CTUs and CUs, in some implementations, NxM CTUs and / or CUs may be used, wherein N and M are different numbers. For example, 32x64 CTUs, 64x32 CTUs, 16x32 CUs, 32x16 CUs, or any other size may be used. In some implementations, Nx2N CTUs or CUs, 2NxN CTUs or CUs, or a combination thereof, may be used.

[0054] The pixels may include information representing an image captured in the video frame 600, such as luminance information, color information, and location information. In some implementations, a block, such as a 16x16 pixel block as shown, may include a luminance block 660, which may include luminance pixels 662; and two chrominance blocks 670, 680, such as a U or Cb chrominance block 670, and a V or Cr chrominance block 680. The chrominance blocks 670, 680 may include chrominance pixels 690. For example, the luminance block 660 may include 16x16 luminance pixels 662 and each chrominance block 670, 680 may include 8x8 chrominance pixels 690 as shown.

[0055] In some implementations, coding the video frame 600 may include ordered blocklevel coding. Ordered block-level coding may include coding CUs of the video frame 600 in an order, such as raster-scan order, wherein CUs may be identified and processed startingwith a CTU in the upper left comer of the video frame 600, or portion of the video frame 600, and proceeding along rows from left to right and from the top row to the bottom row, identifying each CU in turn for processing. For example, the 64x64 CTU in the top row and left column of the video frame 600 may be the first CTU coded and the 64x64 CTU immediately to the right of the first CTU may be the second CTU coded. The second row from the top may be the second row coded, such that the 64x64 CTU in the left column of the second row may be coded after the 64x64 CTU in the rightmost column of the first row.

[0056] In some implementations, coding a CTU of the video frame 600 may include using quad-tree coding, which may include coding smaller CUs within a CTU in raster-scan order. For example, the 64x64 CTU shown in the bottom left corner of the portion of the video frame 600 may be coded using quad-tree coding wherein the top left 32x32 CU may be coded, then the top right 32x32 CU may be coded, then the bottom left 32x32 CU may be coded, and then the bottom right 32x32 CU may be coded. Each 32x32 CU may be coded using quad-tree coding wherein the top left 16x16 CU may be coded, then the top right 16x16 CU may be coded, then the bottom left 16x16 CU may be coded, and then the bottom right 16x16 CU may be coded. Each 16x16 CU may be coded using quad-tree coding wherein the top left 8x8 CU may be coded, then the top right 8x8 CU may be coded, then the bottom left 8x8 CU may be coded, and then the bottom right 8x8 CU may be coded. Each 8x8 CU may be coded using quad-tree coding wherein the top left 4x4 CU may be coded, then the top right 4x4 CU may be coded, then the bottom left 4x4 CU may be coded, and then the bottom right 4x4 CU may be coded. In some implementations, 8x8 CUs may be omitted for a 16x16 CU, and the 16x16 CU may be coded using quad-tree coding wherein the top left 4x4 CU may be coded, then the other 4x4 CUs in the 16x16 CU may be coded in raster- scan order.

[0057] In some implementations, coding the video frame 600 may include encoding the information included in the original version of the image or video frame by, for example, omitting some of the information from that original version of the image or video frame from a corresponding encoded image or encoded video frame. For example, the coding may include reducing spectral redundancy, reducing spatial redundancy, or a combination thereof. Reducing spectral redundancy may include using a color model based on a luminance component (Y) and two chrominance components (U and V or Cb and Cr), which may be referred to as the YUV or YCbCr color model, or color space. Using the YUV color model may include using a relatively large amount of information to represent the luminance component of a portion of the video frame 600, and using a relatively small amount ofinformation to represent each corresponding chrominance component for the portion of the video frame 600. For example, a portion of the video frame 600 may be represented by a high-resolution luminance component, which may include a 16x16 block of luma samples, and by two lower resolution chrominance components, each of which represents the portion of the image as an 8x8 block of chroma samples. A sample may indicate a value, for example, a value in the range from 0 to 255, and may be stored or transmitted using, for example, eight bits. Although this disclosure is described in reference to the YUV color model, another color model may be used. Reducing spatial redundancy may include transforming a CU into the frequency domain using, for example, a discrete cosine transform. For example, a unit of an encoder may perform a discrete cosine transform using transform coefficient values based on spatial frequency.

[0058] Although described herein with reference to matrix or Cartesian representation of the video frame 600 for clarity, the video frame 600 may be stored, transmitted, processed, or a combination thereof, in a data structure such that pixel values and / or luma and chroma samples may be efficiently represented for the video frame 600. For example, the video frame 600 may be stored, transmitted, processed, or any combination thereof, in a two-dimensional data structure such as a matrix as shown, or in a one-dimensional data structure, such as a vector array. Furthermore, although described herein as showing a chrominance subsampled image where U and V have half the resolution of Y, the video frame 600 may have different configurations for the color channels thereof. For example, referring still to the YUV color space, full resolution may be used for all color channels of the video frame 600. In another example, a color space other than the YUV color space may be used to represent the resolution of color channels of the video frame 600.

[0059] FIG. 7 is a block diagram of an example of a video frame filtering stage 700. The video frame filtering stage 700 performs filtering on a reconstructed frame 702 to obtain an enhanced frame 704 to prepare the reconstructed frame 702 for display or storage. During encoding, the video frame filtering stage 700 may be the loop filtering stage 416 of the encoder 400 shown in FIG. 4 or a stage that performs some, but not all, of the operations performed by the loop filtering stage 416. During decoding, the video frame filtering stage 700 may be the loop filtering stage 512 of the decoder 500 shown in FIG. 5 or a stage that performs some, but not all, of the operations performed by the loop filtering stage 512.

[0060] The reconstructed frame 702 is a video frame output from a reconstruction stage, which may be the reconstruction stage 414 of the encoder 400 or the reconstruction stage 510 of the decoder 500. After the filtering performed by the video frame filtering stage 700, theenhanced frame 704 is sent as output for display or storage 706. The display or storage 706 may represent or include operations for storing the enhanced frame 704 in a reference frame buffer of the encoder 400 or of the decoder 500. Alternatively, the display or storage 706 may represent or include operations for outputting the enhanced frame 704 within an output video stream for display at a device that receives the output video stream.

[0061] The video frame filtering stage 700 receives the reconstructed frame 702 after the video frame is output from the reconstruction stage and prepares the reconstructed frame 702 for sending as output for the display or storage 706. The video frame filtering stage 700 may apply one or more filters to the reconstructed frame 702 to obtain the enhanced frame 704. The encoder selects and signals in the compressed bitstream which of the available filters are to be applied to the reconstructed frame 702. The encoder can encode (e.g., signal) parameters (e.g., configuration characteristics) for the filters. In an example, a deblocking filter 708 is always applied and, as such, the encoder does not signal whether the deblocking filter 708 is to be applied.

[0062] The video frame filtering stage 700 uses processing units to process individual regions of the reconstructed frame 702 one at a time. A processing unit is a coding structure of size MxN, where M and N may be the same or different numbers. The size of the processing units may be based on the size of the largest block within the reconstructed frame 702. For example, if the largest block within the reconstructed frame 702 is 128x128 pixels, the processing units used by the video frame filtering stage 700 may be of size 128x128 pixels or larger. In an example, each filter of the video frame filtering stage 700 may use different processing unit sizes.

[0063] The processing units are typically square in shape. However, as the processing units may be of size MxN, the processing units may be square (M=N) or rectangular (M^N) in shape. The processing units typically have the same size and shape. However, in some cases, the processing units may be variably sized and / or variably shaped. For example, a variable size and / or variable shape processing unit partitioning scheme can be used to divide the reconstructed frame 702 into a plurality of processing units.

[0064] Each of the processing units includes pixel values from a region of the reconstructed frame 702. Each of the pixel values of the video frame is included in a single processing unit. As such, the video frame filtering stage 700 filters each of the pixel values of the reconstructed frame 702 by processing each of the processing units. The video frame filtering stage 700 sequentially processes the processing units one at a time. The order for processing the processing units at the video frame filtering stage 700 may depend upon a scanorder or other order for the encoding or decoding of the reconstructed frame 702. For example, where a raster scan order is used, the video frame filtering stage 700 first processes a processing unit that includes top-left-most pixel values of the reconstructed frame 702.

[0065] The video frame filtering stage 700 is shown as including the deblocking filter 708, a constrained directional enhancement filter (CDEF) 710, a loop restoration (LR) filter 712, and a CCSO filter 714. The video frame filtering stage 700 may include more or fewer filters. The CCSO filter 714 can be performed in parallel with the CDEF 710. That is, the input to the CCSO filter 714 is the same as that provided to the CDEF 710, and the output is applied io the CDEF-filtered samples. The CCSO filter 714 is further described with respect to FIG. 8.

[0066] The deblocking filter 708 can be applied across transform block boundaries to remove block artifacts caused by quantization errors. The CDEF 710 performs edge direction searching at an 8x8 block-level. In CDEF, eight edge directions can be identified within blocks according to edge templates. A primary filter processes reconstruction samples along the edge direction while a secondary filter processes reconstruction samples along a direction 45-degrees from the edge direction.

[0067] The LR filter 712 is applied to units of either 64x64-pixel, 128xl28-pixel, or 256x256-pixel blocks, named loop restoration units (LRU). Bypass filtering, a Wiener filter, or a self-guided filter can be independently selected for each LRU. The self-guided filter scheme applies simple filters to reconstructed pixels, X, to generate two denoised versions, Xi and X2. Their differences from the reconstructed pixels, (Xi-X) and (X2- X), are used to span a sub-space, upon which the differences between the reconstructed pixels and the original pixels (Xs- X) are projected.

[0068] The Wiener filter can be a 7x7 separable filter that includes a 7-tap vertical filter and a 7-tap horizontal filter. Filtering of the reconstruction samples of a block can be performed by applying the vertical and horizontal filters sequentially. After applying the vertical and horizontal filters, the final filtered reconstruction samples are generated. The decoded frame pixel values (at (p, q) and a k X k neighbourhood of the pixel) are used to filter to obtain a filtered frame value at corresponding pixels. The process can be formulated as shown in equation (1)

[0069] In equation (1), (p, q) indicates a location of a pixel of an image or video frame, and f(p,qj(m, r) are the filter coefficients for its k X k neighbourhood. The filter coefficientscan be derived by an encoder and signaled into a compressed or encoded bitstream. Alternatively, a set of filters may be pre-defined and stored at both an encoder and a decoder, and a predefined logic can be used be select the filter for a pixel or block at both the encoder and the decoder. Some other shape of the neighborhood, such as a diamond shape, may be used instead of the rectangle or square (i.e., k X fc) shape.

[0070] In some implementations, symmetric Wiener filters can be used to reduce the bit overhead associated with filter coefficient signaling as well as to reduce the computational complexity of the filtering process. As such, only three coefficients need to be signaled for a 7-tap filter, with the three mirrored coefficients derived as the same values. That is, in the symmetric filter, the value of f(p,qj(m, ri) is equal to fp,q—m, — r).

[0071] Different implementations of video frame filtering stage 700 are possible including those that arrange the components of video frame filtering stage 700 in a different ordering, implement components differently, or add or remove components.

[0072] FIG. 8 is a block diagram 800 that illustrates the operations of the CCSO filter. The CCSO filter may adjust reconstructed samples of a video frame to enhance the visual quality of the reconstructed video frame, such as the reconstructed frame 702 of FIG. 7. The CCSO filter corrects luma and / or co-located chroma reconstruction samples with offsets. The CCSO filter operates on CCSO units (also referred to herein as “block”) of the reconstructed video frame. As mentioned above, a CCSO unit can have the size of the largest possible coding block size.

[0073] A filter 802 illustrates that the CCSO filter uses a 3-tap filter applied to luma pixels. The CCSO filter may be applied to either or both of the luma or chroma pixels of a current block. In CCSO, a set of 3-tap filters are used. The input luma reconstructed samples located at the three filter taps include a current luma pixel 804 (i.e., denoted r / ) in the center, and two symmetrically neighboring luma samples 808 and 810, denoted po and pi, respectively. In this context, the current luma pixel 804 has a co-located chroma pixel 806, denoted rd. When the CCSO filter is applied to a luma sample, the current luma pixel 804 is an actual luma sample of reconstructed video frame; and when the CCSO filter is applied to a chroma sample, the current luma pixel 804 is a co-located (or collocated) luma pixel to a chroma sample. The co-located luma pixel may be obtained as described herein and may be, in some implementations, an actual luma sample of reconstructed video frame.

[0074] To illustrate the concept of a luma pixel being co-located with a chroma pixel, a 4:2:0 chroma subsampling scheme is assumed. In this scheme, each chroma pixelcorresponds to four luma pixels. For example, a luma block 812 corresponds to a chroma block 814. In this scheme, each group of four luma pixels in the luma block 812 (e.g., luma pixels 818 numbered 0, 1, 4, and 5) corresponds to one chroma pixel (e.g., chroma pixel 816 numbered 0) in the chroma block 814. The luma co-located pixel for the chroma pixel 816 can be derived from the corresponding luma pixels 818. The co-located luma pixel may be the average, the median, or some other function of the luma pixels 818. The co-located luma pixel may be one of the luma pixels 818, such as the top-left luma pixel (e.g., the luma pixel numbered 0). Other ways of obtaining the co-located luma pixel are also possible.

[0075] The differences between these luma samples used in the filtering are computed, and these differences are quantized into discrete levels denoted as do and di. The quantized values are then used to determine a combination index from a combination lookup table (LUT) 820.

[0076] That is, given pi and r / , where i = 0, 1, the following steps are applied to process the input samples:1) the delta values, denoted rm, between pi and n are computed (i.e., nn=ri-pi), then2) the delta values, rm, are quantized as di based on a quantization step size Qccso, using the following quantization process: a) di is set equal to -1, if mi is less than -Qccso b) di is set equal to 0, if mi is in the range [-Qccso, Qccso], and c) di is set equal to 1, if mi is greater than Qccso

[0077] The quantization step size, Qccso, can be 8, 16, 32, or 64. Qccso may also be referred to as an EdgeThreshold. After do and di are calculated, an offset value (denoted ,s) is derived using the LUT 820. Each combination of do and di is used to identify a row in the LUT 820 to retrieve the offset value, namely a gradient offset. The offset values can be integers including 0, 1, -1, 3, -3, 7, -7, and -10. Finally, the derived offset 5 of CCSO is applied on chroma color component using equation (2): rc' = clip(rc+ s) (2)

[0078] In equation (2), rcis the reconstructed sample to be filtered by CCSO, and .s' is the derived offset value retrieved from the LUT 820. The filtered sample value rc' is further clipped into the range specified by the bit depth.

[0079] The same CCSO process can be applied to the luma components with the exception that the output is applied on the luma reconstruction samples themselves, and a colocated luma pixel need not be used (e.g., calculated, selected, etc.).

[0080] In CCSO, there are six optional filter shapes, denoted as / ], i = 1...6, as shown in filters 822. These six filter shapes are switchable (e.g., selectable or set) at video frame level, and the selection can be signaled by a syntax element, exlJiller_suppor using a 3-bit fixed length code.

[0081] With respect to the LUT 820, the offset values so through ss may not be fixed and may vary from video frame to video frame. As such, the offset values so through ss can be calculated by an encoder and transmitted to the decoder in the compressed bitstream. The offset values so through ss can be transmitted as nine 3-bit offset values.

[0082] Different implementations of a CCSO filter are possible. For example, implementations may categorize pixels into groups differently than the 9 groups shown in LUT 820, such as a different number of groups or using different techniques to categorize the groups.

[0083] FIG. 9 illustrates syntax elements that may be used to signal aspects of the CCSO filter. The signaling of CCSO can be categorized into frame-level and block-level syntax elements. A table 900 illustrates the frame-level syntax elements ;=, and a table 910 illustrates the block-level (i.e., CCSO unit-level) syntax elements. In this context, a "block" refers to a CCSO unit (e.g., the largest coding block size). For example, with respect to a luma block, the block (e.g., the CCSO unit) may be 256x256 pixels, and the corresponding chroma blocks (CCSO units) may each be 128x128 pixels in a 4:2:0 subsampling format. The block- level flags enable CCSO to be selectively applied within these blocks, depending on the content and the desired level of filtering.

[0084] The table 900 is shown as including syntax elements 902 through 908. The syntax elements of the table 900 can be included in a frame header of a compressed or encoded bitstream for each color component of a video frame. For purposes of this description, the syntax elements of the table 900 are described with respect to the luma component. The syntax element 902 (e.g., FLAG) is a one-bit flag indicating whether CCSO is applied (e.g., enabled) for the luma component of the frame. That is, the syntax element 902 indicates whether the CCSO filter is applied to at least one luma block to the frame. If CCSO is not enabled for the luma component of the frame, then the other syntax elements (e.g., the syntax elements 902 through 908) are not included in the compressed bitstream.

[0085] The syntax element 904 (e.g., EXT_FILTER_SUPPORT) is a 3-bit field indicating the filter shape to be used during the CCSO process. The filter shapes can be as described with respect to FIG. 8. The syntax element 906 (e.g., Q_STEP) is a 2-bit field indicating the selection of the quantization step (e.g., Qccso described above). This fielddefines the level of quantization applied to the calculated differences, mi, which in turn affects the offsets used in the CCSO process. Syntax elements 908 (e.g., OFFSETS) includes nine 3-bit values that are used in the LUT (e.g., the LUT 820 of FIG. 8) for determining the offset values applied during the CCSO process.

[0086] In some implementations, a different number of OFFSETS (e.g., 3-bit values) may be utilized. For example, a number up to 72 or 128 offset values may be possible in certain implementations. For example, in some implementations, LUT 820 may have 8 entries providing for 8 groupings of pixels each associated with an offset value.

[0087] Syntax element 909 (e.g., SCALE) includes a single-bit scaling factor that indicates whether or how much syntax element 906 and / or some or all of syntax elements 908 will be scaled. For example, syntax element 909 may indicate whether one of { 1,4} scaling factors will be utilized. In some implementations, the scaling factors may vary depending on, e.g., a bit-depth or resolution of a frame. For example, one or more of the scaling factors may be greater for a larger bit-depth or for a larger resolution.

[0088] In some implementations, syntax element 909 may be a two-bit scaling factor.For example, syntax element 909 may be referred to as “scale_index” with a possible range of values from 0 to 3. In such an implementation, the value of scale_index may be utilized to select a value from an array referred to as ccso_scale which may be defined as ccso_scale[4] = { 1, 2, 3, 4}. The value of ccso_scale obtained using scale_index may then be applied to scale values of the LUT. For example, the LUT may be referred to as ccso_offset which may be defined in the encoder and decoder as ccso_offset[8] = { -10, -7, -3, -1, 0, 1, 3, 7 }. A value at position x in ccso_offset may be scaled using the following formula: ccso_offset[x] * ccso_scale[scale_index]. For example, an OFFSET value encoded in syntax element 908 may be utilized to obtain a value from ccso_offset which is then scaled before use (e.g., the scaled offset value = ccso_offset[x] * ccso_scale[scale_index]).

[0089] In such an implementation, the selection of quantization step (which may also be referred to as EdgeThreshold) may be based both on syntax element 906 and syntax element 909. For example, syntax element 906 may be referred to as quant_index and a two dimensional array quant_sz may be defined in the encoder and decoder as: const uintl6_t quant_sz[4][4] = { { 0, 8, 16, 32}, { 16, 32, 64, 128 }, { 24, 48, 96, 192 }, { 32, 64, 128, 256 } } . In the foregoing example, quant_index and scale_index may be utilized to identify the quantization step (e.g., quant_sz[scale_index][quant_index]).

[0090] In other implementations, the selection of quantization step may be based only on syntax element 906. For example, in such an implementation, syntax element 906 may bereferred to as quant_index and a one dimensional array quant_sz may be defined in the encoder and decoder as: const uint8_t quant_sz[4] = { 16, 8, 32, 64 }. In the foregoing example, quant_index may be utilized to identify the quantization step (e.g., quant_sz[quant_index]. In such an implementation, the quantization step may be scaled to obtain a scaled quantization step by multiplying the quantization step by a value obtained using scale index (e.g., ccso_scale[scale_index]).

[0091] Again, the frame-level syntax elements 902 through 909 may be included in the frame header for each of the Y (luma), U (chroma U), and V (chroma V) color components. In some implementations, the U and V syntax elements may be assumed to be the same, allowing for a more efficient bitstream. That is, the frame header would not include separate syntax elements for the chroma U and chroma V color components.

[0092] The table 910 is shown as including flags 912 through 916. The flag 912 (e.g., LUMA_FLAG) is a 1 -bit flag indicating whether CCSO is applied to the current luma block (the current CCSO unit). This flag provides fine control at the block level, enabling or disabling CCSO for specific luma blocks within the frame. The flag 914 (e.g., CHROMA_U_FLAG) is a 1 -bit flag indicating whether CCSO is applied to the current chroma U block. Similar to the luma flag, this flag 914 allows selective application of CCSO to chroma U blocks. The flag 916 (e.g., CHROMA_V_FLAG) is a 1 -bit flag indicating whether CCSO is applied to the current chroma V block. This flag enables or disables CCSO for specific chroma V blocks, allowing for precise control over the application of CCSO across different color components.

[0093] In some implementations one or two additional 1 -bit reuse syntax elements may be utilized. For example, a reuse syntax element may be provided for frame-level CCSO syntax elements (e.g., table 900), block-level CCSO syntax elements (e.g., table 910), and / or a combination thereof. If a 1 -bit reuse syntax element is set to enabled, then a 3-bit reference frame index may then be transmitted to identify reference frame to use to obtain the corresponding syntax elements to be reused instead of encoding those syntax elements in the current frame or block. For example, in such a case, where the reuse syntax element is set to enabled for a frame, syntax elements 904, 906, 908, and 909 may be omitted from the current frame and corresponding syntax elements from the identified reference frame may be utilized instead for the current frame. Likewise, where the reuse syntax element is set to enabled for blocks, flags 912, 914, and 916 may be omitted from the current frame and corresponding flags from the identified reference frame may be utilized for corresponding blocks in the current frame.

[0094] FIG. 10 illustrates a flowchart of a technique 1000 for encoding and decoding using scaling in a cross component sample offset filter. Technique 1000 includes techniques 1010 used by an encoder and technique 1030 used by a decoder and / or a reconstruction loop in an encoder. One or more steps of technique 1010 may be performed by an encoder, such as in the loop filtering stage 416 of the encoder 400. One or more steps of technique 1030 may be performed by a decoder or a reconstruction loop of an encoder, such as in the loop filtering stage 512 of the decoder 500, the loop filtering stage 416 of the encoder 400, and / or CCSO filter 714 of the video frame filtering stage 700. Technique 1000 may utilize a CCSO filter using operations such as those described with respect to FIG. 8 and syntax elements such as those described with respect to FIG. 9. More generally, the CCSO filter includes mapping quantized differences between neighboring samples to offset values applied to current samples.

[0095] Technique 1010 includes steps 1020 and 1022. Step 1020 includes determining a scaling factor using a rate distortion analysis. For example, an encoder (e.g., encoder 400) can encode some of (or all of) a frame multiple times using different values for syntax elements such as the quantization step, offsets, and scale (e.g., scaling factor) of a CCSO filter (e.g., syntax elements 906, 908, 909). The resulting encoding can then be analyzed to determine a rate (e.g., number of bits taken to encode the frame or portion thereof) and a distortion (e.g., a difference or error between a reconstruction of the encoded frame or portion thereof and the original frame or portion thereof). The encoder can then select one of the combinations of parameters based on a tradeoff between minimizing rate and minimizing error.

[0096] Step 1022 includes encoding the scaling factor into a compressed bitstream. For example, an encoder (e.g., encoder 400) can encode the scaling factor (e.g., syntax element 909) into the compressed bitstream (e.g., compressed bitstream 420) using entropy encoding techniques (e.g., using entropy encoding stage 408).

[0097] Step 1040 includes decoding CCSO syntax elements from the compressed bitstream. For example, this may include decoding the scaling factor encoded in the compressed bitstream. Additional syntax elements, such as those described with respect to FIG. 9, may also be decoded from the compressed bitstream. In some implementations, syntax elements from a prior reference frame may be reutilized responsive to a reuse syntax element. A decoder (e.g., such as decoder 500) may decode the syntax elements from the compressed bitstream (e.g., compressed bitstream 420).

[0098] Step 1042 includes scaling at least one parameter of a cross-component sample offset (CCSO) filter to produce at least one scaled parameter. For example, a decoder (e.g., decoder 500) may scale parameters (e.g., such as values obtained using syntax elements described in FIG. 9) to produce scaled parameters to be used by the CCSO filter instead of the parameters that are scaled. For example, the scaling can be based on a scaling factor encoded in the compressed bitstream (e.g., as encoded in step 1022 or decoded in step 1040). For example, the encoded scaling factor may be 1 bit that indicates whether the at least one parameter is to be multiplied by four. In other words, the scaling factor may select between scaling factors of { 1,4}. For example, the encoded scaling factor may be an index utilized to look up a corresponding value in an array to identify the scaled parameter.

[0099] Depending on the implementation, other (or a different number of) scaling factors may be used. For example, scaling factors of { 1,2} or { 1,2, 4, 8} could be utilized. For example, one or more of the scaling factors may be adjusted based on one or more characteristics of a video frame, such as bit-depth or resolution. For example, one or more scaling factors may be increased at a higher bit-depth or lower resolution. In some implementations, scaling factors may be determined based on one or more characteristics of a video frame and may not be encoded in the compressed bitstream. For example, the scaling may be based on a bit-depth of the video frame and the scaling factor may be selected based on bit-depth (e.g., 1 for an 8 bit-depth and 4 for a 12 bit-depth).

[0100] In some implementations, the scaling factor is two bits. For example, the decoded or identified scaling factor may be referred to as a scaling index scale_index with a possible range of values from 0 to 3. In such an implementation, the value of scale_index may be utilized to select a value from an array referred to as ccso_scale which may be defined as ccso_scale[4] = { 1, 2, 3, 4}. The value of ccso_scale obtained using scale_index may then be applied to scale values of the LUT. For example, the LUT may be referred to as ccso_offset which may be defined in the encoder and decoder or identified in the offset syntax elements as ccso_offset[8] = { -10, -7, -3, -1, 0, 1, 3, 7 }. A value at position x in ccso_offset may be scaled using the following formula: ccso_offset[x] * ccso_scale[scale_index]. For example, an OFFSET value encoded in syntax element 908 may be utilized to update a value in ccso_offset which is then scaled before use (e.g., the scaled offset value = ccso_offset[x] * ccso_scale[scale_index]) where x is determined based on a classification or grouping of a pixel to be filtered (e.g., as described above with respect to FIG. 8).

[0101] In such an implementation, the selection of quantization step may be based both on a scaling index and a quantization step index quant_index. A two-dimensionalquantization step array quant_sz may be defined in the encoder and decoder as: const uintl6_t quant_sz[4][4] = { { 0, 8, 16, 32}, { 16, 32, 64, 128 }, { 24, 48, 96, 192 }, { 32, 64, 128, 256 } }. In the foregoing example, quant_index and scale_index may be utilized to identify the quantization step (e.g., quant_sz[scale_index][quant_index]).

[0102] Step 1044 includes applying the cross-component sample offset (CCSO) filter to reconstructed pixels of a video frame to produce filtered pixels wherein the cross-component sample offset (CCSO) filter is applied using the at least one scaled parameter. For example, the scaled parameters may be obtained by scaling of a quantization step (e.g., determined based on syntax element 906), one or more offsets (e.g., determined based on syntax element 908), or combinations thereof. For example, the CCSO filter may be implemented using CCSO filter 714, the reconstructed pixels of the video frame may be obtained from deblocking filter 708, and the filtered pixels may be output to LR filter 712. In some implementations, the reconstructed pixels may be obtained from a different source (e.g., directly following the reconstruction of pixels using a prediction and residual, or following a different filtering operation or combination of filtering operations).

[0103] In some implementations, the at least one parameter of the CCSO filter includes the quantized differences and the offset values and the at least one scaled parameter of the CCSO filter includes scaled quantized differences and scaled offset values. Applying the cross-component sample offset filter includes utilizing the scaled quantized differences and scaled offset values instead of the quantized differences and offset values.

[0104] For simplicity of explanation, the techniques of FIG. 10 are depicted and described as a series of steps or operations. However, the steps or operations in accordance with this disclosure can occur in various orders and / or concurrently. Additionally, other steps or operations not presented and described herein may be used. Furthermore, not all illustrated steps or operations may be required to implement a technique in accordance with the disclosed subject matter.

[0105] The aspects of encoding and decoding described above illustrate some examples of encoding and decoding techniques. However, it is to be understood that encoding and decoding, as those terms are used in the claims, could mean compression, decompression, transformation, or any other processing or change of data.

[0106] The word “example” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “example” is not necessarily to be construed as being preferred or advantageous over other aspects or designs. Rather, use of the word “example” is intended to present concepts in a concrete fashion. As used in thisapplication, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless specified otherwise or clearly indicated otherwise by the context, the statement “X includes A or B” is intended to mean any of the natural inclusive permutations thereof. That is, if X includes A; X includes B; or X includes both A and B, then “X includes A or B” is satisfied under any of the foregoing instances. In addition, the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more,” unless specified otherwise or clearly indicated by the context to be directed to a singular form. Moreover, use of the term “an implementation” or the term “one implementation” throughout this disclosure is not intended to mean the same embodiment or implementation unless described as such.

[0107] Implementations of the transmitting station 102 and / or the receiving station 106 (and the algorithms, methods, instructions, etc., stored thereon and / or executed thereby, including by the encoder 400 and the decoder 500) can be realized in hardware, software, or any combination thereof. The hardware can include, for example, computers, intellectual property (IP) cores, application- specific integrated circuits (ASICs), programmable logic arrays, optical processors, programmable logic controllers, microcode, microcontrollers, servers, microprocessors, digital signal processors, or any other suitable circuit. In the claims, the term “processor” should be understood as encompassing any of the foregoing hardware, either singly or in combination. The terms “signal” and “data” are used interchangeably. Further, portions of the transmitting station 102 and the receiving station 106 do not necessarily have to be implemented in the same manner.

[0108] Further, in one aspect, for example, the transmitting station 102 or the receiving station 106 can be implemented using a general-purpose computer or general-purpose processor with a computer program that, when executed, carries out any of the respective methods, algorithms, and / or instructions described herein. In addition, or alternatively, for example, a special purpose computer / processor can be utilized which can contain other hardware for carrying out any of the methods, algorithms, or instructions described herein.

[0109] The transmitting station 102 and the receiving station 106 can, for example, be implemented on computers in a video conferencing system. Alternatively, the transmitting station 102 can be implemented on a server, and the receiving station 106 can be implemented on a device separate from the server, such as a handheld communications device. In this instance, the transmitting station 102, using an encoder 400, can encode content into an encoded video signal and transmit the encoded video signal to the communications device. In turn, the communications device can then decode the encodedvideo signal using a decoder 500. Alternatively, the communications device can decode content stored locally on the communications device, for example, content that was not transmitted by the transmitting station 102. Other suitable transmitting and receiving implementation schemes are available. For example, the receiving station 106 can be a generally stationary personal computer rather than a portable communications device, and / or a device including an encoder 400 may also include a decoder 500.

[0110] Further, all or a portion of implementations of the present disclosure can take the form of a computer program product accessible from, for example, a non-transitory computer-usable or computer-readable medium. A computer-usable or computer-readable medium can be any device that can, for example, tangibly contain, store, communicate, or transport a program including instructions for use by or in connection with any processor. For example, a processor may be configured to perform executed instructions stored in the memory (e.g., computer readable medium) to perform techniques embodied in the instructions. For example, a non-transitory computer-readable storage medium may include executable instructions that, when executed by a processor, facilitate performance of operations corresponding to techniques described in this disclosure. For example, a non- transitory computer-readable storage medium may store an encoded bitstream that is encodable or decodable using techniques described in this disclosure. The medium can be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable mediums are also available.

[0111] The above-described embodiments, implementations, and aspects have been described in order to facilitate easy understanding of this disclosure and do not limit this disclosure. On the contrary, this disclosure is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope is to be accorded the broadest interpretation as is permitted under the law so as to encompass all such modifications and equivalent arrangements.

Claims

What is claimed is:

1. A method, comprising: scaling at least one parameter of a cross-component sample offset filter to produce at least one scaled parameter; and applying the cross-component sample offset filter to reconstructed pixels of a video frame to produce filtered pixels, wherein the cross-component sample offset filter is applied using the at least one scaled parameter.

2. The method of claim 1, wherein scaling the at least one parameter of the crosscomponent sample offset filter includes scaling based on a scaling factor encoded in a compressed bitstream.

3. The method of claim 2, wherein the encoded scaling factor is 1 bit indicating whether the at least one parameter is to be multiplied by four.

4. The method of claim 2, wherein a decoder determines how to scale the at least one parameter depending on the encoded scaling factor and another characteristic of the video frame.

5. The method of claim 2, wherein the at least one parameter includes a quantization step and a plurality of offsets.

6. The method of claim 2, wherein the scaling factor is a scaling index that is used to select a value in a scaling array that is multiplied by an offset to obtain a scaled offset that is utilized by the cross -component sample offset filter.

7. The method of claim 2, wherein the scaling factor is a scaling index that is used, along with a quantization step index to obtain a scaled quantization step from a two-dimensional quantization step array.

8. The method of claim 2, further comprising: decoding the scaling factor encoded in the compressed bitstream.

9. The method of claim 2, further comprising: determining the scaling factor using a rate distortion analysis; and encoding the scaling factor into the compressed bitstream.

10. The method of claim 2, wherein the scaling factor is reused from a reference frame.

11. The method of claim 1, wherein scaling the at least one parameter of the crosscomponent sample offset filter includes scaling based on a bit-depth of the video frame.

12. The method of claim 1, wherein the cross-component sample offset filter includes mapping quantized differences between neighboring samples to offset values applied to current samples, wherein the at least one parameter includes the quantized differences and the offset values and the at least one scaled parameter includes scaled quantized differences and scaled offset values, and wherein applying the cross-component sample offset filter includes utilizing the scaled quantized differences and scaled offset values instead of the quantized differences and offset values.

13. A device, comprising: a processor that is configured to perform the method of any one of claims 1 to 12.

14. A device, comprising: a memory; and a processor, the processor configured to execute instructions stored in the memory to perform the method of any one of claims 1 to 12.

15. A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising operations that perform the method of any one of claims 1 to 12.

16. A non-transitory computer-readable storage medium having stored thereon an encoded bitstream, wherein the encoded bitstream is decodable using the method of any one of claims 1 to 8 or 10 to 12.- l-17. A non-transitory computer-readable storage medium having stored thereon an encoded bitstream, wherein the encoded bitstream is generated by an encoder performing the method of any one of claims 1 to 7 or 9 to 12.

Images