Coding enhancement in cross-component sample adaptive offset

Abstract

An electronic device performs a method for decoding video data, and the method comprises: receiving a picture frame including a first component and a second component in a first color space from a video signal; determining a classifier for the second component in the first color space from a set of one or more samples of the first component associated with each sample of the second component in the first color space, wherein the set of one or more samples is in the second color space; determining a sample offset for each sample of the second component in the first color space according to the classifier; and modifying the value of each sample of the second component in the first color space based on the determined sample offset.

Description

Technology Field

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 200,626, filed March 18, 2021, with the title "Cross-component Sample Adaptive Offset," and U.S. Provisional Application No. 63 / 164,459, filed March 22, 2021, with the title "Cross-component Sample Adaptive Offset," the entirety of which is incorporated herein by reference.

[0003] Technology field

[0004] The present application generally relates to video coding and compression, and more specifically, to methods and apparatus for improving both luma and chroma coding efficiency. Background Technology

[0005] Digital video is supported by various electronic devices such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video teleconferencing devices, and video streaming devices. Electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression standards. Some well-known video coding standards include Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC) (also known as H.265 or MPEG-H Part 2), and Advanced Video Coding (AVC) (also known as H.264 or MPEG-4 Part 10), which were jointly developed by ISO / IEC MPEG and ITU-T VCEG. AOMedia Video 1 (AV1) was developed by the Alliance for Open Media (AOM) as a successor to its predecessor standard, VP9. AVS (Audio Video Coding), which refers to digital audio and digital video compression standards, is another series of video compression standards developed by the Audio and Video Coding Standards Working Group.

[0006] Video compression typically involves performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or eliminate redundancy inherent in video data. In the case of block-based video coding, a video frame is partitioned into one or more slices, and each slice has multiple video blocks, which may also be referred to as coding tree units (CTUs). Each CTU may contain one coding unit (CU) or may be recursively divided into smaller CUs until a predefined minimum CU size is reached. Each CU (also referred to as a leaf CU) contains one or more transform units (TUs), and each CU also contains one or more prediction units (PUs). Each CU may be coded in intra, inter, or IBC modes. Video blocks in an intra-coded (I) slice of a video frame are encoded using spatial prediction for reference samples in neighboring blocks within the same video frame. Video blocks in an inter-coded (P or B) slice of a video frame may use spatial prediction for reference samples in neighboring blocks within the same video frame or temporal prediction for reference samples in other previous and / or future reference video frames.

[0007] Spatial or temporal predictions based on previously encoded reference blocks, e.g., neighboring blocks, yield a prediction block for the current video block to be coded. The process of finding the reference block can be achieved by a block matching algorithm. Residual data representing pixel differences between the current block to be coded and the prediction block is referred to as residual blocks or prediction errors. Intercoded blocks are encoded according to the reference block in the reference frame forming the prediction block, and the motion vector pointing to the residual block. The process of determining the motion vector is typically referred to as motion estimation. Intracoded blocks are encoded according to the intra prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to a transform domain, e.g., the frequency domain, resulting in residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, are scanned to generate a one-dimensional vector of transform coefficients and can subsequently be entropy-encoded into the video bitstream to achieve even greater compression.

[0008] The encoded video bitstream is then stored in a computer-readable storage medium (e.g., flash memory) so that it may be accessed by another electronic device having digital video capabilities or transmitted directly to the electronic device via wired or wireless means. Subsequently, the electronic device performs video decompression (which is the opposite process of the video compression described above) by, for example, parsing the encoded video bitstream to obtain syntax elements from the bitstream, reconstructing digital video data from the encoded video bitstream into its original format based at least partially on the syntax elements obtained from the bitstream, and rendering the reconstructed digital video data on the display of the electronic device.

[0009] As digital video quality progresses from high definition to 4Kx2K or even 8Kx4K, the amount of video data to be encoded / decoded increases exponentially. There is an ongoing challenge regarding how video data can be encoded / decoded more efficiently while maintaining the image quality of the decoded video data.

[0010] The present application describes implementations related to video data encoding and decoding, and more specifically, relates to methods and apparatus for improving the coding efficiency of both luminance and chroma components, including improving coding efficiency by exploring cross-component relationships between luminance components and chroma components.

[0011] According to a first aspect of the present application, a method for decoding a video signal comprises: receiving a picture frame from a video signal comprising a first component and a second component in a first color space; determining a classifier for the second component in the first color space from a first set of one or more samples of the first component associated with each sample of the second component in the first color space, wherein the first set of one or more samples is converted to the second color space; determining a sample offset for each sample of the second component in the first color space according to the classifier; and modifying the value of each sample of the second component in the first color space based on the determined sample offset.

[0012] In some embodiments, the first component is a component selected from the group consisting of a lumina component, a first chroma component, and a second chroma component, and the second component is a component selected from the group consisting of a lumina component, a first chroma component, and a second chroma component.

[0013] In some embodiments, a first set of one or more samples is converted into a second color space by a linear transformation.

[0014] According to a second aspect of the present application, an electronic device comprises one or more processing units, a memory, and a plurality of programs stored in the memory. When executed by one or more processing units, the programs cause the electronic device to perform a method of coding a video signal as described above.

[0015] According to a third aspect of the present application, a non-transient computer-readable storage medium stores a plurality of programs to be executed by an electronic device having one or more processing units. When executed by one or more processing units, the programs cause the electronic device to perform the method of coding the aforementioned video signal.

[0016] According to a fourth aspect of the present application, a computer-readable storage medium stores a bitstream containing video information generated by a method for video coding as described above.

[0017] It should be understood that both the foregoing general description and the following detailed description are merely examples and do not limit the scope of the disclosure. Brief explanation of the drawing

[0018] The accompanying drawings, which are included to provide a further understanding of the implementations, are incorporated into this specification, and constitute part of this specification. They serve to illustrate the described implementations and explain the basic principles together with the description. Similar reference numbers refer to corresponding parts. FIG. 1 is a block diagram illustrating an exemplary video encoding and decoding system according to some implementations of the present disclosure. FIG. 2 is a block diagram illustrating an exemplary video encoder according to some implementations of the present disclosure. FIG. 3 is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure. FIGS. 4a through 4e are block diagrams illustrating how a frame is recursively partitioned into a plurality of video blocks of different sizes and shapes according to some embodiments of the present disclosure. FIG. 5 is a block diagram illustrating four gradient patterns used in a Sample Adaptive Offset (SAO) according to some implementations of the present disclosure. FIG. 6a is a block diagram illustrating a system and process of a CCSAO that is applied to chroma samples and uses DBF Y as input, according to some implementations of the present disclosure. FIG. 6b is a block diagram illustrating a system and process of CCSAO applied to luminance and chroma samples and using DBF Y / Cb / Cr as input, according to some implementations of the present disclosure. FIG. 6c is a block diagram illustrating a system and process of a CCSAO that can operate independently according to some implementations of the present disclosure. FIG. 6d is a block diagram illustrating a system and process of CCSAO that can be applied recursively (twice or N times) with the same or different offsets according to some implementations of the present disclosure. FIG. 6e is a block diagram illustrating a system and process of CCSAO applied in parallel with the Enhanced Sample Adaptive Offset (ESAO) in the AVS standard according to some implementations of the present disclosure. FIG. 6f is a block diagram illustrating a system and process of CCSAO applied after SAO according to some implementations of the present disclosure. FIG. 6g is a block diagram illustrating that, according to some implementations of the present disclosure, the system and process of CCSAO can operate independently without CCALF. FIG. 6h is a block diagram illustrating a system and process of a CCSAO applied in parallel with a Cross-Component Adaptive Loop Filter (CCALF) according to some implementations of the present disclosure. FIG. 7 is a block diagram illustrating a sample process using CCSAO according to some implementations of the present disclosure. FIG. 8 is a block diagram illustrating a CCSAO process interleaved with vertical and horizontal deblocking filters (DBF) according to some implementations of the present disclosure. FIG. 9 is a flowchart illustrating an exemplary process for decoding a video signal using cross-component correlation according to some implementations of the present disclosure. FIG. 10a is a block diagram illustrating a classifier using different luminance (or chroma) sample locations for C0 classification according to some embodiments of the present disclosure. FIG. 10b illustrates some examples of different shapes for Luma candidates according to some embodiments of the present disclosure. FIG. 11 is a block diagram of a sample process illustrating that all collocated and neighboring luminar / chroma samples can be fed to a CCSAO classification according to some embodiments of the present disclosure. FIG. 12 illustrates exemplary classifiers according to some embodiments of the present disclosure, by replacing a juxtaposed luma sample value with a value obtained by weighting the juxtaposed and neighboring luma samples. FIG. 13a is a block diagram illustrating that, according to some embodiments of the present disclosure, CCSAO is not applied to the current chroma (luma) sample when any of the juxtaposed and adjacent lumina (chroma) samples used for classification are out of the current picture. FIG. 13b is a block diagram illustrating that, according to some embodiments of the present disclosure, CCSAO is applied to the current luminar or chroma sample when any of the juxtaposed and adjacent luminar or chroma samples used for classification are out of the current picture. FIG. 14 is a block diagram illustrating that, according to some embodiments of the present disclosure, CCSAO is not applied to the current chroma sample when the corresponding selected juxtaposed or neighboring chroma sample used for classification is outside the virtual space defined by the virtual boundary (VB). FIG. 15 illustrates that repetitive or mirror padding is applied to luminance samples outside a virtual boundary according to some embodiments of the present disclosure. FIG. 16 illustrates that, according to some embodiments of the present disclosure, an additional 1 luma line buffer is required when all 9 parallel neighboring luma samples are used for classification. FIG. 17 illustrates an example in AVS in which nine luminance candidate CCSAOs intersecting with VB can increase two additional luminance line buffers, according to some implementations of the present disclosure. FIG. 18a illustrates an example in VVC in which nine luminance candidate CCSAOs intersecting with VB, according to some implementations of the present disclosure, can increase one additional luminance line buffer. FIG. 18b illustrates an example in which, according to some embodiments of the present disclosure, when juxtaposed or adjacent chroma samples are used to classify current chroma samples, a selected chroma candidate may span across VB and an additional chroma line buffer may be required. FIGS. 19a through 19c illustrate that, in AVS and VVC according to some embodiments of the present disclosure, CCSAO is disabled for a chroma sample when any of the luminance candidates of the chroma sample are across the VB (currently outside the chroma sample VB). FIGS. 20a through 20c illustrate that, according to some embodiments of the present disclosure, in AVS and VVC, CCSAO is enabled by iterative padding of the chroma sample when any of the luminance candidates of the chroma sample spans the VB (currently outside the chroma sample VB). FIGS. 21a through 21c illustrate that, according to some embodiments of the present disclosure, in AVS and VVC, CCSAO is enabled using mirror padding for a chroma sample when any of the luminance candidates of the chroma sample spans the VB (currently outside the chroma sample VB). FIGS. 22a and 22b illustrate that a CCSAO is enabled using double-sided symmetric padding for different CCSAO sample shapes according to some embodiments of the present disclosure. FIG. 23 illustrates limitations of using a limited number of luma candidates for classification according to some embodiments of the present disclosure. FIG. 24 illustrates that, according to some implementations of the present disclosure, the CCSAO application area is not aligned with the coding tree block (CTB) / coding tree unit (CTU) boundary. FIG. 25 illustrates that, according to some implementations of the present disclosure, a CCSAO applied area frame partition can be fixed with CCSAO parameters. FIG. 26 illustrates that, according to some implementations of the present disclosure, the CCSAO application area may be a binary-tree (BT), quad-tree (QT), or ternary-tree (TT) partitioned from the frame / slice / CTB level. FIG. 27 is a block diagram illustrating a plurality of classifiers used and switched at different levels within a picture frame according to some embodiments of the present disclosure. FIG. 28 is a block diagram illustrating that, according to some implementations of the present disclosure, a CCSAO application area partition can be dynamic and switched at the picture level. FIG. 29 is a diagram illustrating that CCSAO classifiers can consider current or cross-component coding information according to some implementations of the present disclosure. FIG. 30 is a block diagram illustrating that the SAO classification methods disclosed in the present disclosure serve as post-prediction filters according to some implementations of the present disclosure. FIG. 31 is a block diagram illustrating that, for a post-prediction SAO filter according to some embodiments of the present disclosure, each component can use current and neighboring samples for classification. FIG. 32 is a flowchart illustrating an exemplary process for decoding a video signal using cross-component correlation according to some embodiments of the present disclosure. FIG. 33 is a drawing illustrating a computing environment combined with a user interface according to some implementations of the present disclosure. Specific details for implementing the invention

[0019] References to specific implementations will now be made in detail, and examples thereof are illustrated in the accompanying drawings. In the following detailed description, a number of non-limiting specific details are disclosed to aid in understanding the subject matter of the invention presented herein. However, it will be apparent to those skilled in the art that various alternatives may be used without departing from the scope of the claims and that the subject matter of the invention may be practiced without these specific details. For example, it will be apparent to those skilled in the art that the subject matter of the invention presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0020] The first generation AVS standard includes the Chinese national standards "Information Technology, Advanced Audio Video Coding, Part 2: Video" (known as AVS1) and "Information Technology, Advanced Audio Video Coding Part 16: Radio Television Video" (known as AVS+). This can provide approximately 50% bitrate savings compared to the MPEG-2 standard at the same perceptual quality. The second generation AVS standard includes a series of Chinese national standards "Information Technology, Efficient Multimedia Coding" (known as AVS2), which are primarily targeted for the transmission of auxiliary HD TV programs. The coding efficiency of AVS2 is twice that of AVS+. Meanwhile, the video portion of the AVS2 standard was submitted by the IEEE (Institute of Electrical and Electronics Engineers) as an international standard for applications. The AVS3 standard is a new generation video coding standard for UHD video applications that aims to surpass the coding efficiency of the latest international standard HEVC, providing approximately 30% bitrate savings compared to the HEVC standard. In March 2019, at the 68th AVS meeting, the AVS3-P2 baseline was finalized, providing approximately 30% bitrate savings compared to the HEVC standard. Currently, a single reference software called the High Performance Model (HPM) is maintained by the AVS Group to describe the reference implementation of the AVS3 standard. Like HEVC, the AVS3 standard is built on a block-based hybrid video coding framework.

[0021] FIG. 1 is a block diagram illustrating an exemplary system (10) for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. As illustrated in FIG. 1, the system (10) includes a source device (12) that generates and encodes video data to be later decoded by a destination device (14). The source device (12) and the destination device (14) may include any of a wide variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device (12) and the destination device (14) have wireless communication capabilities.

[0022] In some implementations, the destination device (14) may receive encoded video data to be decoded via a link (16). The link (16) may include any type of communication medium or device capable of moving the encoded video data from the source device (12) to the destination device (14). In one example, the link (16) may include a communication medium that enables the source device (12) to transmit the encoded video data directly to the destination device (14) in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device (14). The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for facilitating communication from the source device (12) to the destination device (14).

[0023] In some other implementations, encoded video data may be transmitted from the output interface (22) to the storage device (32). Subsequently, the encoded video data in the storage device (32) may be accessed by the destination device (14) through the input interface (28). The storage device (32) may include any of various distributed or locally accessed data storage media, such as a hard drive, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded video data. In additional examples, the storage device (32) may correspond to a file server or other intermediate storage device capable of holding the encoded video data generated by the source device (12). The destination device (14) may access the stored video data from the storage device (32) via streaming or downloading. The file server may be any type of computer capable of storing the encoded video data and transmitting the encoded video data to the destination device (14). Exemplary file servers include web servers (e.g., for websites), FTP servers, NAS (network attached storage) devices, or local disk drives. The destination device (14) can access the encoded video data via any standard data connection suitable for accessing the encoded video data stored on the file server, including a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both. Transmission of the encoded video data from the storage device (32) may be a streaming transmission, a download transmission, or a combination of both.

[0024] As illustrated in FIG. 1, the source device (12) includes a video source (18), a video encoder (20), and an output interface (22). The video source (18) may include a source such as a video capture device, e.g., a video camera, a video archive containing previously captured video, a video feed interface receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as a source video, or a combination of such sources. As an example, if the video source (18) is a video camera of a security surveillance system, the source device (12) and the destination device (14) may form camera phones or video phones. However, the implementations described in this application may generally be applicable to video coding and may be applicable to wireless and / or wired applications.

[0025] Captured, pre-captured, or computer-generated video may be encoded by a video encoder (20). The encoded video data may be transmitted directly to a destination device (14) through an output interface (22) of a source device (12). The encoded video data may also (or alternatively) be stored on a storage device (32) for later access by the destination device (14) or other devices for decoding and / or playback. The output interface (22) may further include a modem and / or transmitter.

[0026] The destination device (14) includes an input interface (28), a video decoder (30), and a display device (34). The input interface (28) includes a receiver and / or a modem and can receive encoded video data via the link (16). The encoded video data communicated via the link (16) or provided on the storage device (32) may include various syntax elements generated by the video encoder (20) for use by the video decoder (30) when decoding the video data. Such syntax elements may be included in the encoded video data transmitted over a communication medium, stored on a storage medium, or stored on a file server.

[0027] In some implementations, the destination device (14) may include a display device (34) which may be an integrated display device and an external display device configured to communicate with the destination device (14). The display device (34) displays decoded video data to a user and may include any of various display devices such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0028] The video encoder (20) and video decoder (30) may operate according to proprietary or industry standards such as VVC, HEVC, MPEG-4, Part 10, AVC (Advanced Video Coding), AVS, or extensions of these standards. It should be understood that the present application is not limited to a specific video coding / decoding standard and may be applicable to other video coding / decoding standards. It is generally considered that the video encoder (20) of the source device (12) may be configured to encode video data according to any of these current or future standards. Similarly, it is also generally considered that the video decoder (30) of the destination device (14) may be configured to decode video data according to any of these current or future standards.

[0029] Each of the video encoder (20) and video decoder (30) may be implemented as any of various suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software on a suitable non-transient computer-readable medium and execute the instructions in hardware using one or more processors to perform the video coding / decoding operations disclosed in this disclosure. Each of the video encoder (20) and video decoder (30) may be included in one or more encoders or decoders, and any one of these may be integrated as part of a combined encoder / decoder (CODEC) in each device.

[0030] FIG. 2 is a block diagram illustrating an exemplary video encoder (20) according to some of the embodiments described in the present application. The video encoder (20) can perform intra and inter prediction coding of video blocks within video frames. Intra prediction coding relies on spatial prediction to reduce or eliminate spatial redundancy in video data within a given video frame or picture. Inter prediction coding relies on temporal prediction to reduce or eliminate temporal redundancy in video data within adjacent video frames or pictures of a video sequence.

[0031] As illustrated in FIG. 2, the video encoder (20) includes a video data memory (40), a prediction processing unit (41), a decoded picture buffer (DPB) (64), an summer (50), a transformation processing unit (52), a quantization unit (54), and an entropy encoding unit (56). The prediction processing unit (41) further includes a motion estimation unit (42), a motion compensation unit (44), a partition unit (45), an intra prediction processing unit (46), and an intra block copy (BC) unit (48). In some implementations, the video encoder (20) also includes an inverse quantization unit (58), an inverse transformation processing unit (60), and a summer (62) for video block reconstruction. An in-loop filter (63), such as a deblocking filter, may be positioned between the summer (62) and the DPB (64) to filter block boundaries and remove blocky artifacts from the reconstructed video. In addition to the deblocking filter, another in-loop filter (63) may also be used to filter the output of the summer (62). Additional in-loop filtering (63), such as a sample adaptive offset (SAO) and an adaptive in-loop filter (ALF), may be applied to the reconstructed CU before it is placed in a reference picture store and used as a reference for coding future video blocks. The video encoder (20) may take the form of a fixed or programmable hardware unit, or may be divided among one or more of the exemplified fixed or programmable hardware units.

[0032] The video data memory (40) can store video data to be encoded by the components of the video encoder (20). The video data in the video data memory (40) can be obtained, for example, from a video source (18). The DPB (64) is a buffer that stores reference video data to be used to encode video data by the video encoder (20) (for example, in intra or inter predictive coding modes). The video data memory (40) and the DPB (64) can be formed by any of various memory devices. In various examples, the video data memory (40) may be on-chip with other components of the video encoder (20) or off-chip with respect to such components.

[0033] As illustrated in FIG. 2, after receiving video data, a partition unit (45) within a prediction processing unit (41) partitions the video data into video blocks. This partitioning may also include partitioning the video frame into slices, tiles, or other larger coding units (CUs) according to predefined partitioning structures, such as a quad-tree structure associated with the video data. A video frame may be divided into a plurality of video blocks (or sets of video blocks referred to as tiles). The prediction processing unit (41) may select one of a plurality of possible prediction coding modes, such as one of a plurality of intra prediction coding modes or one of a plurality of inter prediction coding modes, for the current video block based on error results (e.g., coding rate and distortion level). The prediction processing unit (41) may provide the resulting intra or inter prediction coded block to a summer (50) to generate a residual block and provide it to a summer (62) to reconstruct the encoded block for subsequent use as part of a reference frame. The prediction processing unit (41) also provides syntax elements such as motion vectors, intra-mode indicators, partition information, and other such syntax information to the entropy encoding unit (56).

[0034] To select an appropriate intra-predictive coding mode for the current video block, the intra-predictive processing unit (46) within the prediction processing unit (41) may provide spatial prediction by performing intra-predictive coding of the current video block for one or more neighboring blocks in the same frame as the current block to be coded. The motion estimation unit (42) and the motion compensation unit (44) within the prediction processing unit (41) provide temporal prediction by performing inter-predictive coding of the current video block for one or more prediction blocks in one or more reference frames. The video encoder (20) may perform a number of coding passes, for example, to select an appropriate coding mode for each block of video data.

[0035] In some implementations, the motion estimation unit (42) determines an inter-prediction mode for the current video frame by generating a motion vector that indicates the displacement of the prediction unit (PU) of the video block in the current video frame relative to the prediction block in the reference video frame, according to a predetermined pattern in a sequence of video frames. The motion estimation performed by the motion estimation unit (42) is a process of generating motion vectors that estimate motion for video blocks. The motion vector may indicate, for example, the displacement of the PU of the video block in the current video frame or picture relative to the prediction block in the reference frame (or other coded unit) relative to the current block being coded in the current frame (or other coded unit). The predetermined pattern may designate the video frames in the sequence as P frames or B frames. The intra-BC unit (48) may determine vectors for intra-BC coding, for example, block vectors, in a manner similar to the determination of motion vectors by the motion estimation unit (42) for inter-prediction, or may determine block vectors using the motion estimation unit (42).

[0036] The prediction block is a block of reference frames considered to closely match the PU of the video block to be coded in terms of pixel difference, which can be determined by the sum of absolute difference (SAD), the sum of square difference (SSD), or other difference metrics. In some implementations, the video encoder (20) can calculate values ​​for sub-integer pixel positions of the reference frames stored in the DPB (64). For example, the video encoder (20) can interpolate the values ​​of quarter pixel positions, eighth pixel positions, or other fractional pixel positions of the reference frames. Thus, the motion estimation unit (42) can perform motion search for whole pixel positions and fractional pixel positions and output a motion vector with fractional pixel precision.

[0037] The motion estimation unit (42) calculates a motion vector for a PU of a video block in an inter-predicted coded frame by comparing the position of a PU with the position of a predicted block of a reference frame selected from a first reference frame list (List 0) or a second reference frame list (List 1), each identifying one or more reference frames stored in the DPB (64). The motion estimation unit (42) transmits the calculated motion vector to the motion compensation unit (44) and then to the entropy encoding unit (56).

[0038] Motion compensation performed by the motion compensation unit (44) may involve fetching or generating a prediction block based on a motion vector determined by the motion estimation unit (42). Upon receiving a motion vector for the PU of the current video block, the motion compensation unit (44) may find the prediction block pointed to by the motion vector in one of the reference frame lists, retrieve the prediction block from the DPB (64), and forward the prediction block to the summer (50). Then, the summer (50) forms a residual video block of pixel difference values ​​by subtracting the pixel values ​​of the prediction block provided by the motion compensation unit (44) from the pixel values ​​of the current video block being coded. The pixel difference values ​​forming the residual video block may include luminance or chroma difference components, or both. The motion compensation unit (44) may also generate syntax elements associated with the video blocks of the video frame for use by the video decoder (30) when decoding the video blocks of the video frame. Syntax elements may include, for example, syntax elements defining motion vectors used to identify prediction blocks, any flags indicating a prediction mode, or any other syntax information described herein. Note that while the motion estimation unit (42) and the motion compensation unit (44) may be highly integrated, they are illustrated separately for conceptual purposes.

[0039] In some implementations, the intra BC unit (48) may generate vectors and fetch prediction blocks in a manner similar to that described above in relation to the motion estimation unit (42) and the motion compensation unit (44), but the prediction blocks are in the same frame as the current block being coded, and the vectors are referred to as block vectors, unlike motion vectors. In particular, the intra BC unit (48) may determine the intra prediction mode to use to encode the current block. In some examples, the intra BC unit (48) may, for example, encode the current block using various intra prediction modes during individual encoding passes and test their performance through rate-distortion analysis. Next, the intra BC unit (48) may select and use an appropriate intra prediction mode from among the various tested intra prediction modes and generate an intra mode indicator accordingly. For example, the intra BC unit (48) can calculate rate-distortion values ​​using rate-distortion analysis for various tested intra prediction modes and select the intra prediction mode having the best rate-distortion characteristics among the tested modes as the appropriate intra prediction mode to use. Rate-distortion analysis generally determines the amount of distortion (or error) between the encoded block and the original unencoded block encoded to generate the encoded block, as well as the bit rate (i.e., the number of bits) used to generate the encoded block. The intra BC unit (48) can calculate ratios from the distortions and rates for various encoded blocks to determine which intra prediction mode exhibits the best rate-distortion value for the block.

[0040] In other examples, the intra-BC unit (48) may use the motion estimation unit (42) and the motion compensation unit (44), wholly or partially, to perform such functions for intra-BC prediction according to the implementations described herein. In either case, for an intra-block copy, the prediction block may be a block considered to be closely matched with the block to be coded in terms of pixel difference, which may be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics, and the identification of the prediction block may include the calculation of values ​​for sub-integer pixel locations.

[0041] Regardless of whether the prediction block comes from the same frame according to intra prediction or from a different frame according to inter prediction, the video encoder (20) can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being coded to form pixel difference values. The pixel difference values ​​forming the residual video block may include both luminance and chroma component differences.

[0042] The intra prediction processing unit (46) can intra-predict the current video block as an alternative to the intra-prediction performed by the motion estimation unit (42) and the motion compensation unit (44) as described above, or the intra-block copy prediction performed by the intra-BC unit (48). In particular, the intra-predict processing unit (46) can determine the intra-predict mode to be used to encode the current block. To do so, the intra-predict processing unit (46) can encode the current block using various intra-predict modes, for example, during individual encoding passes, and the intra-predict processing unit (46) (or, in some examples, the mode selection unit) can select an appropriate intra-predict mode to be used from the tested intra-predict modes. The intra-predict processing unit (46) can provide information indicating the selected intra-predict mode for the block to the entropy encoding unit (56). The entropy encoding unit (56) can encode the information indicating the selected intra-predict mode into a bitstream.

[0043] After the prediction processing unit (41) determines a prediction block for the current video block through inter-prediction or intra-prediction, the summer (50) forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more transformation units (TU) and provided to the transformation processing unit (52). The transformation processing unit (52) transforms the residual video data into residual transformation coefficients using a transformation such as the Discrete Cosine Transform (DCT) or a conceptually similar transformation.

[0044] The conversion processing unit (52) can transmit the resulting conversion coefficients to the quantization unit (54). The quantization unit (54) quantizes the conversion coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit (54) can then perform a scan of the matrix containing the quantized conversion coefficients. Alternatively, the entropy encoding unit (56) can perform the scan.

[0045] Following quantization, the entropy encoding unit (56) entropies the quantized transform coefficients into a video bitstream using, for example, CAVLC (context adaptive variable length coding), CABAC (context adaptive binary arithmetic coding), SBAC (syntax-based context-adaptive binary arithmetic coding), PIPE (probability interval partitioning entropy) coding, or other entropy encoding methods or techniques. The encoded bitstream may then be transmitted to a video decoder (30) or stored in a storage device (32) for later transmission to the video decoder (30) or retrieval therefrom. The entropy encoding unit (56) may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0046] The inverse quantization unit (58) and the inverse transform processing unit (60) each apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain to generate a reference block for predicting other video blocks. As previously described, the motion compensation unit (44) can generate a motion-compensated prediction block from one or more reference blocks of frames stored in the DPB (64). The motion compensation unit (44) can also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0047] The summer (62) adds the reconstructed residual block to the motion-compensated prediction block generated by the motion compensation unit (44) to generate a reference block to be stored in the DPB (64). Then, the reference block can be used as a prediction block for inter-predicting other video blocks in subsequent video frames by the intra-BC unit (48), the motion estimation unit (42), and the motion compensation unit (44).

[0048] FIG. 3 is a block diagram illustrating an exemplary video decoder (30) according to some embodiments of the present application. The video decoder (30) includes a video data memory (79), an entropy decoding unit (80), a prediction processing unit (81), an inverse quantization unit (86), an inverse transform processing unit (88), a summer (90), and a DPB (92). The prediction processing unit (81) further includes a motion compensation unit (82), an intra prediction processing unit (84), and an intra BC unit (85). The video decoder (30) can perform a decoding process that is generally the inverse of the encoding process described above for the video encoder (20) in relation to FIG. 2. For example, the motion compensation unit (82) can generate prediction data based on motion vectors received from the entropy decoding unit (80), while the intra prediction unit (84) can generate prediction data based on intra prediction mode indicators received from the entropy decoding unit (80).

[0049] In some examples, a unit of the video decoder (30) may be tasked with performing the implementations of the present application. Additionally, in some examples, the implementations of the present disclosure may be divided among one or more of the units of the video decoder (30). For example, the intra-BC unit (85) may perform the implementations of the present application alone or in combination with other units of the video decoder (30), such as the motion compensation unit (82), the intra-prediction processing unit (84), and the entropy decoding unit (80). In some examples, the video decoder (30) may not include the intra-BC unit (85), and the function of the intra-BC unit (85) may be performed by other components of the prediction processing unit (81), such as the motion compensation unit (82).

[0050] The video data memory (79) can store video data, such as an encoded video bitstream to be decoded by other components of the video decoder (30). The video data stored in the video data memory (79) can be obtained, for example, from a storage device (32), from a local video source such as a camera, via wired or wireless network communication of the video data, or by accessing physical data storage media (e.g., a flash drive or a hard disk). The video data memory (79) may include a coded picture buffer (CPB) that stores encoded video data from the encoded video bitstream. The decoded picture buffer (DPB) (92) of the video decoder (30) stores reference video data (e.g., in intra or inter predictive coding modes) to be used when the video decoder (30) decodes the video data. The video data memory (79) and DPB (92) may be formed by any of the various memory devices, such as synchronous DRAM (SDRAM), magneto-resistive RAM (MRAM), resistive RAM (RRAM), or dynamic random access memory (DRAM), including other types of memory devices. For the purpose of illustration, the video data memory (79) and DPB (92) are shown in FIG. 3 as two separate components of the video decoder (30). However, it will be apparent to those skilled in the art that the video data memory (79) and DPB (92) may be provided by the same memory device or by separate memory devices. In some examples, the video data memory (79) may be on-chip with respect to other components of the video decoder (30) or off-chip with respect to such components.

[0051] During the decoding process, the video decoder (30) receives an encoded video bitstream representing video blocks of encoded video frames and associated syntax elements. The video decoder (30) may receive syntax elements at the video frame level and / or video block level. The entropy decoding unit (80) of the video decoder (30) entropy decodes the bitstream to generate quantized coefficients, motion vectors or intra-prediction mode indicators and other syntax elements. Subsequently, the entropy decoding unit (80) forwards the motion vectors and other syntax elements to the prediction processing unit (81).

[0052] When a video frame is coded as an intra-predicted coded (I) frame or for intra-coded prediction blocks in frames of other types, the intra-predicted processing unit (84) of the prediction processing unit (81) can generate prediction data for a video block of the current video frame and reference data from previously decoded blocks of the current frame based on the signaled intra-predicted mode.

[0053] When a video frame is coded as an inter-predicted coded (i.e., B or P) frame, the motion compensation unit (82) of the prediction processing unit (81) generates one or more prediction blocks for a video block of the current video frame based on motion vectors and other syntax elements received from the entropy decoding unit (80). Each prediction block may be generated from a reference frame within one of the reference frame lists. The video decoder (30) may construct reference frame lists List 0 and List 1 using default configuration techniques based on the reference frames stored in the DPB (92).

[0054] In some examples, when a video block is coded according to the intra-BC mode described herein, the intra-BC unit (85) of the prediction processing unit (81) generates prediction blocks for the current video block based on block vectors and other syntax elements received from the entropy decoding unit (80). The prediction blocks may be within a reconstructed area of ​​the same picture as the current video block defined by the video encoder (20).

[0055] The motion compensation unit (82) and / or the intra-BC unit (85) determine prediction information for video blocks of the current video frame by parsing motion vectors and other syntax elements, and then generate prediction blocks for the current video block to be decoded using the prediction information. For example, the motion compensation unit (82) determines, using some of the received syntax elements, a prediction mode (e.g., intra or inter prediction) used to code video blocks of the video frame, an inter prediction frame type (e.g., B or P), configuration information for one or more of the reference frame lists for the frame, motion vectors for each inter prediction encoded video block of the frame, an inter prediction state for each inter prediction encoded video block of the frame, and other information for decoding video blocks in the current video frame.

[0056] Similarly, the intra-BC unit (85) can determine, using some of the received syntax elements, for example, flags, that the current video block is predicted using the intra-BC mode, configuration information of which video blocks of the frame are within the reconstructed area and should be stored in the DPB (92), block vectors for each intra-BC predicted video block of the frame, the intra-BC predicted state for each intra-BC predicted video block of the frame, and other information for decoding the video blocks in the current video frame.

[0057] The motion compensation unit (82) may also perform interpolation using interpolation filters such as those used by the video encoder (20) during the encoding of video blocks to calculate interpolated values ​​for sub-integer pixels of reference blocks. In this case, the motion compensation unit (82) may determine the interpolation filters used by the video encoder (20) from the received syntax elements and generate prediction blocks using the interpolation filters.

[0058] The inverse quantization unit (86) determines the degree of quantization by inversely quantizing the quantized transform coefficients provided to the bitstream and entropy decoded by the entropy decoding unit (80) using the same quantization parameters calculated by the video encoder (20) for each video block in the video frame. The inverse transform processing unit (88) reconstructs residual blocks in the pixel domain by applying an inverse transform, e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients.

[0059] After the motion compensation unit (82) or the intra-BC unit (85) generates a prediction block for the current video block based on vectors and other syntax elements, the summer (90) reconstructs the decoded video block for the current video block by summing the residual block from the inverse transformation processing unit (88) and the corresponding prediction block generated by the motion compensation unit (82) and the intra-BC unit (85). An in-loop filter (91) may be placed between the summer (90) and the DPB (92) to further process the decoded video block. In-loop filtering (91), such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive in-loop filter (ALF), may be applied to the reconstructed CU before it is placed in the reference picture storage. Subsequently, the decoded video blocks in a given frame are stored in the DPB (92), which stores reference frames used for subsequent motion compensation of the next video blocks. The DPB (92), or a memory device separated from the DPB (92), can also store the decoded video for later presentation on a display device such as the display device (34) of FIG. 1.

[0060] In a typical video coding process, a video sequence typically contains an ordered set of frames or pictures. Each frame may contain three sample arrays denoted as SL, SCb, and SCr. SL is a two-dimensional array of luminance samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other cases, the frame may be monochromatic and thus contains only one two-dimensional array of luminance samples.

[0061] Like HEVC, the AVS3 standard is built on a block-based hybrid video coding framework. Input video signals are processed in blocks (referred to as Coding Units (CUs)). Unlike HEVC, which partitions blocks based solely on quad-trees, AVS3 divides a single Coding Tree Unit (CTU) into CUs to adapt to changing local characteristics based on quad, binary, and extended-quad-trees. Additionally, the concept of multiple partition unit types found in HEVC is eliminated; that is, the separation of CUs, Predict Units (PUs), and Transform Units (TUs) does not exist in AVS3. Instead, each CU is always utilized as the primary unit for both prediction and transformation without additional partitions. In the tree partition structure of AVS3, a single CTU is first partitioned based on a quad-tree structure. Subsequently, each quad-tree leaf node can be further partitioned based on binary and extended-quad-tree structures.

[0062] As illustrated in FIG. 4a, a video encoder (20) (or more specifically, a partition unit (45)) first generates an encoded representation of a frame by partitioning the frame into a set of coding tree units (CTUs). A video frame may contain an integer number of CTUs ordered sequentially in raster scan order from left to right and from top to bottom. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder (20) in a set of sequence parameters so that all CTUs in the video sequence have the same size, which is one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that the present application is not necessarily limited to a specific size. As illustrated in FIG. 4b, each CTU may include one coding tree block (CTB) of luminance samples, two corresponding coding tree blocks of chroma samples, and syntax elements used to code the samples of the coding tree blocks. Syntax elements, including inter or intra prediction, intra prediction mode, motion vectors, and other parameters, describe the attributes of units of different types of blocks of coded pixels and how a video sequence can be reconstructed in a video decoder (30). In monochrome pictures or pictures having three distinct color planes, the CTU may include a single coding tree block and syntax elements used to code samples of the coding tree block. The coding tree block may be an NxN block of samples.

[0063] To achieve better performance, the video encoder (20) may recursively perform tree partitioning, such as binary-tree partitioning, ternary-tree partitioning, quad-tree partitioning, or a combination of both, on the coding tree blocks of the CTU and divide the CTU into smaller coding units (CUs). As illustrated in FIG. 4c, a 64x64 CTU (400) is first divided into four smaller CUs, each having a block size of 32x32. Among the four smaller CUs, CU (410) and CU (420) are each divided into four CUs of 16x16 by block size. Two 16x16 CUs (430 and 440) are each further divided into four CUs of 8x8 by block size. FIG. 4d illustrates a quad-tree data structure illustrating the final result of the partitioning process of a CTU (400) as illustrated in FIG. 4c, where each leaf node of the quad-tree corresponds to a single CU of its own size ranging from 32x32 to 8x8. As with the CTU illustrated in FIG. 4b, each CU may include a coding block (CB) of luminance samples, two corresponding coding blocks of chroma samples of the same size frame, and syntax elements used to code the samples of the coding blocks. In monochrome pictures or pictures having three distinct color planes, the CU may include a single coding block and syntax structures used to code the samples of the coding block. The quad-tree partitioning illustrated in FIG. 4c and FIG. 4d is for illustrative purposes only, and a single CTU may be divided into CUs to adapt to changing local characteristics based on quad / ternary / binary-tree partitions. In a multi-type tree structure, one CTU is partitioned by a quad-tree structure, and each quad-tree leaf CU can be further partitioned by binary and ternary tree structures.As shown in Fig. 4e, AVS3 has five partitioning types: quaternary partitioning, horizontal binary partitioning, vertical binary partitioning, horizontally extended quad-tree partitioning, and vertically extended quad-tree partitioning.

[0064] In some implementations, the video encoder (20) may further partition the coding block of the CU into one or more MxN prediction blocks (PB). A prediction block is a rectangular (square or non-square) block of samples to which the same prediction, inter, or intra is applied. The prediction unit (PU) of the CU may include a prediction block of luminance samples, two corresponding prediction blocks of chroma samples, and syntax elements used to predict the prediction blocks. In monochrome pictures or pictures having three distinct color planes, the PU may include a single prediction block and syntax structures used to predict the prediction block. The video encoder (20) may generate prediction luminance, Cb, and Cr blocks for the luminance, Cb, and Cr prediction blocks of each PU of the CU.

[0065] The video encoder (20) can generate prediction blocks for a PU using intra prediction or inter prediction. When the video encoder (20) generates prediction blocks for a PU using intra prediction, the video encoder (20) can generate prediction blocks for a PU based on decoded samples of a frame associated with a PU. When the video encoder (20) generates prediction blocks for a PU using inter prediction, the video encoder (20) can generate prediction blocks for a PU based on decoded samples of one or more frames other than the frame associated with a PU.

[0066] After the video encoder (20) has generated predicted luminance, Cb, and Cr blocks for one or more PUs of the CU, the video encoder (20) may generate a luminance residual block for the CU by subtracting the predicted luminance blocks of the CU from its original luminance coding block, such that each sample in the luminance residual block of the CU indicates the difference between a luminance sample in one of the predicted luminance blocks of the CU and a corresponding sample in the original luminance coding block of the CU. Similarly, the video encoder (20) may generate a Cb residual block and a Cr residual block for the CU, respectively, such that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU indicates the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0067] Furthermore, as illustrated in FIG. 4c, the video encoder (20) may use quad-tree partitioning to decompose the luminance, Cb, and Cr residual blocks of the CU into one or more luminance, Cb, and Cr transform blocks. A transform block is a rectangular (square or non-square) block of samples to which the same transform is applied. A transform unit (TU) of the CU may include a transform block of luminance samples, two corresponding transform blocks of chroma samples, and syntax elements used to transform the transform block samples. Thus, each TU of the CU may be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with the TU may be a subblock of the luminance residual block of the CU. The Cb transform block may be a subblock of the Cb residual block of the CU. The Cr transform block may be a subblock of the Cr residual block of the CU. In monochrome pictures or pictures having three distinct color planes, the TU may include a single transform block and syntax structures used to transform samples of the transform block.

[0068] The video encoder (20) may apply one or more transformations to the luminance transformation block of the TU to generate a luminance coefficient block for the TU. The coefficient block may be a two-dimensional array of transformation coefficients. The transformation coefficients may be scalar quantities. The video encoder (20) may apply one or more transformations to the Cb transformation block of the TU to generate a Cb coefficient block for the TU. The video encoder (20) may apply one or more transformations to the Cr transformation block of the TU to generate a Cr coefficient block for the TU.

[0069] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the video encoder (20) can quantize the coefficient block. Quantization generally refers to a process that provides additional compression by quantizing the transform coefficients to reduce the amount of data used to represent the transform coefficients. After the video encoder (20) quantizes the coefficient block, the video encoder (20) can entropy-encode the syntax elements representing the quantized transform coefficients. For example, the video encoder (20) can perform Context-Adaptive Binary Arithmetic Coding (CABAC) on the syntax elements representing the quantized transform coefficients. Finally, the video encoder (20) can output a bitstream containing a sequence of bits forming a representation of the coded frames and associated data, which is stored in the storage device (32) or transmitted to the destination device (14).

[0070] After receiving a bitstream generated by a video encoder (20), a video decoder (30) can parse the bitstream to obtain syntax elements from the bitstream. The video decoder (30) can reconstruct frames of video data based at least partially on the syntax elements obtained from the bitstream. The process of reconstructing video data is generally the inverse of the encoding process performed by the video encoder (20). For example, the video decoder (30) can reconstruct residual blocks associated with the TUs of the current CU by performing an inverse transform on the coefficient blocks associated with the TUs of the current CU. The video decoder (30) also reconstructs the coding blocks of the current CU by adding samples of prediction blocks for the PUs of the current CU to the corresponding samples of transform blocks for the TUs of the current CU. After reconstructing the coding blocks for each CU of the frame, the video decoder (30) can reconstruct the frame.

[0071] SAO is a process of modifying decoded samples by conditionally adding an offset value to each sample after the application of a deblocking filter, based on values ​​in lookup tables transmitted by the encoder. SAO filtering is performed on a region basis, based on the filtering type selected per CTB by the syntax element sao-type-idx. A value of 0 for sao-type-idx indicates that the SAO filter is not applied to the CTB, while values ​​1 and 2 signal the use of band offset and edge offset filtering types, respectively. In the band offset mode specified by sao-type-idx equal to 1, the selected offset value depends directly on the sample amplitude. In this mode, the entire sample amplitude range is uniformly divided into 32 segments called bands, and sample values ​​belonging to four of these bands (consecutive within the 32 bands) are modified by adding transmitted values ​​denoted as band offsets, which can be positive or negative. The primary reason for using four consecutive bands is that in smoothing regions where banding artifacts may appear, the sample amplitudes of the CTB tend to be concentrated in only a few bands. In addition, the design choice of using four offsets is integrated into an edge offset operation mode that also uses four offset values. In the edge offset mode specified by the sao-type-idx identical to 2, the syntax element sao-eo-class, having values ​​from 0 to 3, signals whether one of the horizontal, vertical, or two diagonal gradient directions is used for edge offset classification in the CTB.

[0072] FIG. 5 is a block diagram illustrating four gradient patterns used in SAO according to some implementations of the present disclosure. The four gradient patterns (502, 504, 506, and 508) are for each sao-eo-class in edge offset mode. A sample labeled "p" indicates a center sample to be considered. Two samples labeled "n0" and "n1" designate two neighboring samples along gradient patterns of (a) horizontal (sao-eo-class = 0), (b) vertical (sao-eo-class = 1), (c) 135° diagonal (sao-eo-class = 2), and (d) 45° (sao-eo-class = 3). Each sample within the CTB is classified into one of five EdgeIdx categories by comparing a sample value p located at a certain location with the values ​​n0 and n1 of two samples located at neighboring locations, as illustrated in FIG. 5. Since this classification is performed for each sample based on the decoded sample values, no additional signaling is required for EdgeIdx classification. Depending on the EdgeIdx category at the sample location, for EdgeIdx categories 1 through 4, an offset value from the transmitted lookup table is added to the sample value. The offset values ​​are always positive for categories 1 and 2 and negative for categories 3 and 4. Thus, the filter generally has a smoothing effect in edge offset mode. Table 1 below illustrates the sample EdgeIdx categories in the SAO edge classes.

[0073]

[0074] Table 1: Sample EdgeIdx Categories in SAO Edge Classes

[0075] For SAO types 1 and 2, a total of four amplitude offset values ​​are transmitted to the decoder for each CTB. For type 1, the sign is also encoded. Offset values ​​such as sao-type-idx and sao-eo-class, and associated syntax elements, are determined by the encoder—typically using criteria that optimize rate-distortion performance. SAO parameters can be indicated to be inherited from the left or upper CTB using a merge flag to facilitate efficient signaling. In summary, SAO is a non-linear filtering operation that allows for further refinement of the reconstructed signal, and it can improve signal representation in both smooth regions and near edges.

[0076] In some embodiments, methods and systems for improving coding efficiency or reducing the complexity of Sample Adaptive Offset (SAO) by introducing cross-component information are disclosed herein. SAO is used in HEVC, VVC, AVS2, and AVS3 standards. While existing SAO designs in HEVC, VVC, AVS2, and AVS3 standards are used as the basic SAO methods in the following descriptions, to a person skilled in the art of video coding, the cross-component methods described herein may also be applied to other loop filter designs or other coding tools having similar design concepts. For example, in the AVS3 standard, SAO is replaced by a coding tool called Enhanced Sample Adaptive Offset (ESAO). However, the CCSAO disclosed herein may also be applied in parallel with ESAO. In another example, CCSAO may be applied in parallel with Constrained Directional Enhancement Filter (CDEF) in the AV1 standard.

[0077] In the case of existing SAO designs in HEVC, VVC, AVS2, and AVS3 standards, the luminance Y, chroma Cb, and chroma Cr sample offset values ​​are determined independently. That is, for example, the current chroma sample offset is determined solely by the current and neighboring chroma sample values, without considering juxtaposed or neighboring luminance samples. However, luminance samples preserve more original picture detail information than chroma samples, which can be beneficial in determining the current chroma sample offset. Additionally, since chroma samples typically lose high-frequency details after color conversion from RGB to YCbCr, or after quantization and deblocking filters, introducing luminance samples with preserved high-frequency details for chroma offset determination can be beneficial for chroma sample reconstruction. Therefore, additional gains can be expected by exploring cross-component correlations, for example, by utilizing methods and systems of Cross-Component Sample Adaptive Offset (CCSAO). In some embodiments, the correlation includes not only cross-component sample values ​​but also picture / coding information such as prediction / residual coding modes, transformation types, and quantization / deblocking / SAO / ALF parameters from the cross-components.

[0078] Another example concerns SAO, where luminance sample offsets are determined solely by the luminance samples. However, for example, a luminance sample with the same band offset (BO) classification can be further classified by its juxtaposed and neighboring chroma samples, which can lead to more effective classification. SAO classification can be taken as a shortcut to compensate for sample differences between the original picture and the reconstructed picture. Therefore, effective classification is required.

[0079] FIG. 6a is a block diagram illustrating a system and process of CCSAO applied to chroma samples and using DBF Y as input, according to some implementations of the present disclosure. The luminance samples (DBF Y) after the luminance deblocking filter are used to determine additional offsets for the chroma Cb and Cr after SAO Cb and SAO Cr. For example, the current chroma sample (602) is first classified using the juxtaposed (604) and neighboring (white) luminance samples (606), and the corresponding CCSAO offset value of the corresponding class is added to the current chroma sample value. FIG. 6b is a block diagram illustrating a system and process of CCSAO applied to luminance and chroma samples and using DBF Y / Cb / Cr as input, according to some implementations of the present disclosure. FIG. 6c is a block diagram illustrating a system and process of CCSAO that can operate independently, according to some implementations of the present disclosure. FIG. 6d is a block diagram illustrating a system and process of CCSAO that may be applied recursively (twice or N times) with the same or different offsets at the same codec stage or repeated at different stages according to some embodiments of the present disclosure. In summary, in some embodiments, information of current and neighboring luminance samples and information of juxtaposed and neighboring chroma samples (Cb and Cr) may be used to classify the current luminance sample. In some embodiments, juxtaposed and neighboring luminance samples, juxtaposed and neighboring cross-chroma samples, and current and neighboring chroma samples may be used to classify the current chroma sample (Cb or Cr). In some embodiments, CCSAO may be cascaded after (1) DBF Y / Cb / Cr, (2) the reconstructed image Y / Cb / Cr prior to DBF, or (3) SAO Y / Cb / Cr, or (4) ALF Y / Cb / Cr.

[0080] In some embodiments, CCSAO may also be applied in parallel with other coding tools, e.g., ESAO in AVS standards or CDEF in AV1 standards. FIG. 6e is a block diagram illustrating a system and process of CCSAO applied in parallel with ESAO in AVS standards according to some implementations of the present disclosure.

[0081] FIG. 6f is a block diagram illustrating the system and process of a CCSAO applied after an SAO according to some embodiments of the present disclosure. In some embodiments, FIG. 6f illustrates that the location of the CCSAO may be after the SAO, i.e., the location of the Cross-Component Adaptive Loop Filter (CCALF) in the VVC standard. FIG. 6g is a block diagram illustrating that the system and process of the CCSAO may operate independently without a CCALF according to some embodiments of the present disclosure. In some embodiments, the SAO Y / Cb / Cr may be replaced by an ESAO, for example, in the AVS3 standard.

[0082] FIG. 6h is a block diagram illustrating a system and process of a CCSAO applied in parallel with a CCALF according to some embodiments of the present disclosure. In some embodiments, FIG. 6h illustrates that the CCSAO can be applied in parallel with the CCALF. In some embodiments, the positions of the CCALF and the CCSAO in FIG. 6h may be switched. In some embodiments, in FIG. 6a through 6h, or throughout the present disclosure, the SAO Y / Cb / Cr blocks may be replaced by ESAO Y / Cb / Cr (in AVS3) or CDEF (in AV1). Note that Y / Cb / Cr may also be denoted as Y / U / V in the video coding area.

[0083] In some embodiments, the current chroma sample classification reuses the SAO type (edge ​​offset (EO) or BO), class, and category of the juxtaposed luminance sample. The corresponding CCSAO offset may be signaled or derived from the decoder itself. For example, let h_Y be the juxtaposed luminance SAO offset, and let h_Cb and h_Cr be the CCSAO Cb and Cr offsets, respectively. h_Cb (or h_Cr) = w * h_Y, where w can be selected from a limited table. For example, ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4... etc., where |w| includes only powers of 2.

[0084] In some embodiments, the comparison score [-8, 8] of the juxtaposed luminance samples (Y0) and the neighboring 8 luminance samples is used, which yields a total of 17 classes.

[0085] Initial Class = 0

[0086] Loop for 8 neighboring luminance samples (Yi, i=1 to 8)

[0087] if Y0 > Yi Class += 1

[0088] else if Y0 < Yi Class -= 1

[0089] In some embodiments, the aforementioned classification methods may be combined. For example, diversity is increased using a comparison score combined with SAO BO (32-band classification), which yields a total of 17*32 classes. In some embodiments, Cb and Cr may use the same class to reduce complexity or reduce bits.

[0090] FIG. 7 is a block diagram illustrating a sample process using CCSAO according to some implementations of the present disclosure. Specifically, FIG. 7 shows that the inputs of CCSAO may introduce vertical and horizontal DBF inputs to simplify class determination or increase flexibility. For example, let Y0_DBF_V, Y0_DBF_H, and Y0 be parallel luminance samples in the inputs of DBF_V, DBF_H, and SAO, respectively. Yi_DBF_V, Yi_DBF_H, and Yi are eight neighboring luminance samples in the inputs of DBF_V, DBF_H, and SAO, respectively, where i = 1 to 8.

[0091]

[0092] And supply max Y0 and max Yi to the CCSAO classification.

[0093] FIG. 8 is a block diagram illustrating a CCSAO process interleaved in vertical and horizontal DBFs according to some embodiments of the present disclosure. In some embodiments, the CCSAO blocks of FIG. 6, FIG. 7 and FIG. 8 may be optional. For example, Y0_DBF_V and Yi_DBF_V are used for a first CCSAO_V applying the same sample processing as in FIG. 6, while the input of DBF_V luminance samples is used as the CCSAO input.

[0094] In some embodiments, the implemented CCSAO syntax is shown in Table 2 below.

[0095]

[0096] Table 2: Examples of CCSAO Syntax

[0097] In some embodiments, when an additional chroma offset is signaled to signal the CCSAO Cb and Cr offset values, other chroma component offsets may be derived by a plus or minus sign, or by weighting to save bit overhead. For example, let h_Cb and h_Cr be the offsets of CCSAO Cb and Cr, respectively. By an explicit signaling w—where w = +- |w| and is a limited set of |w| candidates—h_Cr can be derived from h_Cb without the explicit signaling h_Cr itself.

[0098]

[0099] FIG. 9 is a flowchart illustrating an exemplary process (900) for decoding a video signal using cross-component correlation according to some implementations of the present disclosure.

[0100] A video decoder (30) receives a video signal including a first component and a second component (910). In some embodiments, the first component is a luminance component of the video signal, and the second component is a chroma component.

[0101] The video decoder (30) also receives multiple offsets associated with the second component (920).

[0102] Next, the video decoder (30) obtains a classification category associated with the second component using a characteristic measurement of the first component (930). For example, in FIG. 6, the current chroma sample (602) is first classified using the juxtaposed (604) and neighboring (white) luminance samples (606), and the corresponding CCSAO offset value is added to the current chroma sample.

[0103] The video decoder (30) further selects a first offset from a plurality of offsets for a second component according to a classification category (940).

[0104] The video decoder (30) further modifies the second component based on the selected first offset (950).

[0105] In some embodiments, obtaining a classification category associated with a second component using a characteristic measurement of a first component (930) includes obtaining a classification category of each respective sample of a second component using each sample of a first component, wherein each sample of a first component is a juxtaposed sample of a first component for each respective sample of a second component. For example, current chroma sample classification reuses the SAO type (EO or BO), class, and category of the juxtaposed chroma sample.

[0106] In some embodiments, obtaining a classification category associated with a second component using a characteristic measurement of a first component (930) includes obtaining a classification category of each respective sample of the second component using each respective sample of the first component, and each sample of the first component is reconstructed before or after deblocking. In some embodiments, the first component is deblocked in a deblocking filter (DBF). In some embodiments, the first component is deblocked in a luma deblocking filter (DBF Y). For example, as an alternative to FIG. 6 or FIG. 7, the CCSAO input may also be prior to DBF Y.

[0107] In some embodiments, the characteristic measurement is derived by dividing the range of sample values ​​of the first component into several bands and selecting a band based on the intensity value of the sample in the first component. In some embodiments, the characteristic measurement is derived from the band offset (BO).

[0108] In some embodiments, the characteristic measurement is derived based on the direction and intensity of the edge information of the sample in the first component. In some embodiments, the characteristic measurement is derived from the edge offset (EO).

[0109] In some embodiments, modifying the second component (950) involves directly adding a selected first offset to the second component. For example, a corresponding CCSAO offset value is added to the current chroma component sample.

[0110] In some embodiments, modifying the second component (950) includes mapping a selected first offset to a second offset and adding the mapped second offset to the second component. For example, when one additional chroma offset is signaled to signal CCSAO Cb and Cr offset values, another chroma component offset may be derived by using a plus or minus sign, or by using weights to save bit overhead.

[0111] In some embodiments, receiving a video signal (910) includes receiving a syntax element indicating whether a method of decoding the video signal using CCSAO is enabled for the video signal in the Sequence Parameter Set (SPS). In some embodiments, cc_sao_enabled_flag indicates whether CCSAO is enabled at the sequence level.

[0112] In some embodiments, receiving a video signal (910) includes receiving a syntax element indicating whether a method of decoding the video signal using CCSAO is enabled for a second component on the slice level. In some embodiments, slice_cc_sao_cb_flag or slice_cc_sao_cr_flag indicates whether CCSAO is enabled on the respective slice for Cb or Cr.

[0113] In some embodiments, receiving a plurality of offsets associated with the second component (920) includes receiving different offsets for different coding tree units (CTUs). In some embodiments, for a CTU, cc_sao_offset_sign_flag indicates the sign for the offset, and cc_sao_offset_abs indicates the CCSAO Cb and Cr offset values ​​of the current CTU.

[0114] In some embodiments, receiving a plurality of offsets associated with the second component (920) includes receiving a syntax element indicating whether the received offsets of the CTU are identical to the offsets of one of the neighboring CTUs of the CTU, and the neighboring CTU is either the left or the top neighboring CTU. For example, cc_sao_merge_up_flag indicates whether the CCSAO offset is merged from the left or top CTU.

[0115] In some embodiments, the video signal further comprises a third component, and a method for decoding the video signal using CCSAO further comprises receiving a second plurality of offsets associated with the third component; obtaining a second classification category associated with the third component using a characteristic measurement of the first component; selecting a third offset from the second plurality of offsets for the third component according to the second classification category; and modifying the third component based on the selected third offset.

[0116] FIG. 11 is a block diagram of a sample process illustrating that all juxtaposed and adjacent (white) luminance / chroma samples can be fed to a CCSAO classification according to some embodiments of the present disclosure. FIG. 6a, FIG. 6b and FIG. 11 illustrate the input to the CCSAO classification. In FIG. 11, the current chroma sample is 1104, the cross-component juxtaposed chroma sample is 1102, and the juxtaposed luminance sample is 1106.

[0117] In some embodiments, the classifier example (C0) uses the juxtaposed luminance or chroma sample values ​​(Y0) of FIG. 12 below (Y4 / U4 / V4 in FIG. 6b and 6c) for classification. Let band_num be the number of equally divided bands of the luminance or chroma dynamic range, and bit_depth be the sequence bit depth. An example of a class index for the current chroma sample is as follows:

[0118]

[0119] In some embodiments, classification is rounded, for example, by taking into account the following:

[0120]

[0121] Some band_num and bit_depth examples are listed in Table 3 below. Table 3 shows three classification examples where the number of bands differs for each classification example.

[0122]

[0123] Table 3: Exemplary band_num and bit_depth for each class index

[0124] In some embodiments, the classifier uses different luminance sample locations for the C0 classification. FIG. 10a is a block diagram illustrating a classifier according to some embodiments of the present disclosure that uses different luminance (or chroma) sample locations for the C0 classification, for example, using neighboring Y7 rather than Y0 for the C0 classification.

[0125] In some embodiments, different classifiers may be switched in the sequence parameter set (SPS) / adaptation parameter set (APS) / picture parameter set (PPS) / picture header (PH) / slice header (SH) / region / coding tree unit (CTU) / coding unit (CU) / subblock / sample level. For example, in FIG. 10, as shown in Table 4 below, Y0 is used for POC0 but Y7 is used for POC1.

[0126]

[0127] Table 4: Different classifiers applied to different pictures

[0128] In some embodiments, FIG. 10b illustrates some examples of different shapes for luminance candidates according to some embodiments of the present disclosure. For example, constraints may be applied to the shapes. In some cases, as illustrated in FIG. 10b (b), (c), and (d), the total number of luminance candidates must be a power of 2. In some examples, the number of luminance candidates must be horizontally and vertically symmetric with respect to the chroma sample (at the center), as illustrated in FIG. 10b (a), (c), (d), and (e). In some embodiments, power of 2 constraints and symmetry constraints may also be applied to the chroma candidates. The U / V portions of FIG. 6b and 6c illustrate examples of symmetry constraints. In some embodiments, different color formats may have different classifier "constraints". For example, the 420 color format uses a luminance / chroma candidate selection (one candidate selected from a 3x3 shape) as shown in FIGS. 6b and 6c, but the 444 color format uses (f) of FIG. 10b for luminance and chroma candidate selection, and the 422 color format uses (g) of FIG. 10b for luminance (two chroma samples share four luminance candidates) and (f) of FIG. 10b for chroma candidates.

[0129] In some embodiments, the C0 position and C0 band_num may be combined and switched at SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels. Different combinations may be different classifiers, as shown in Table 5 below.

[0130]

[0131] Table 5: Different classifier and band number combinations applied to different pictures

[0132] In some embodiments, a juxtaposed luminance sample value (Y0) is replaced by a value (Yp) obtained by weighting the juxtaposed and neighboring luminance samples. FIG. 12 illustrates exemplary classifiers according to some embodiments of the present disclosure by replacing a juxtaposed luminance sample value with a value obtained by weighting the juxtaposed and neighboring luminance samples. A juxtaposed luminance sample value (Y0) may be replaced by a phase-corrected value (Yp) obtained by weighting the neighboring luminance samples. Different Yp may be different classifiers.

[0133] In some embodiments, different Yp are applied to different chroma formats. For example, in FIG. 12, Yp of (a) is used for the 420 chroma format, Yp of (b) is used for the 422 chroma format, and Y0 is used for the 444 chroma format.

[0134] In some embodiments, another classifier (C1) is the comparison score [-8, 8] of the juxtaposed luminance samples (Y0) and the neighboring 8 luminance samples, which yields a total of 17 classes as shown below.

[0135] Initial Class (C1) = 0, loop for 8 neighboring luminance samples (Yi, i=1 to 8)

[0136] if Y0 > Yi Class += 1

[0137] else if Y0 < Yi Class -= 1

[0138] In some embodiments, the C1 example is equivalent to the following function where the threshold th is 0.

[0139]

[0140] If xy > th, then f(x, y) = 1; if xy = th, then f(x, y) = 0; and if xy < th, then f(x, y) = -1

[0141] Here, Index2ClassTable is a lookup table (LUT), C is the current or parallel sample, and P1 to P8 are neighboring samples.

[0142] In some embodiments, similar to a C4 classifier, one or more thresholds may be signaled or predefined at SPS / APS / PPS / PH / SH / regions / CTU / CU / subblocks / samples levels to help classify (quantize) differences (e.g., may be maintained in a LUT).

[0143] In some embodiments, the variant (C1') counts only comparison scores [0, 8], which yields 8 classes. (C1, C1') is a classifier group, and the PH / SH level flag can be signaled to switch between C1 and C1'.

[0144] Initial Class (C1') = 0, loop for 8 neighboring luminance samples (Yi, i=1 to 8)

[0145] if Y0 > Yi Class += 1

[0146] In some embodiments, the variant (C1s) optionally uses N neighboring samples out of M neighboring samples to count the comparison score. An M-bit bitmask may be signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels to indicate which neighboring samples are selected to count the comparison score. Using FIG. 6b as an example for a luminance classifier: 8 neighboring luminance samples are candidates, and an 8-bit bitmask (01111110) is signaled at PH to indicate that 6 samples Y1 through Y6 are selected, and thus the comparison score is at [-6, 6], which yields 13 offsets. The optional classifier C1s provide the encoder with more choices to trade off between offset signaling overhead and classification granularity.

[0147] Similar to C1s, the variation (C1's) counts only the comparison score [0, +N], and the previous bitmask 01111110 example provides the comparison score at [0, 6], which yields 7 offsets.

[0148] In some embodiments, different classifiers are combined to produce a general classifier. For example, for different pictures (different POC values), different classifiers are applied as shown in Table 6-1 below.

[0149]

[0150] Table 6-1: Different general classifiers applied to different pictures

[0151] In some embodiments, another classifier example (C3) uses a bitmask for classification as illustrated in Table 6-2. A 10-bit bitmask is signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels to represent the classifier. For example, the bitmask 11 1100 0000 means that for a given 10-bit luma sample value, only the most significant bit (MSB): 4 bits are used for classification, yielding a total of 16 classes. Another exemplary bitmask 10 0100 0001 means that only 3 bits are used for classification, yielding a total of 8 classes.

[0152] In some embodiments, the bitmask length (N) may be fixed or switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels. For example, in the case of a 10-bit sequence, a 4-bit bitmask 1110 is signaled at the PH in the picture, and the MSB 3 bits b9, b8, b7 are used for classification. Another example is a 4-bit bitmask 0011 on the LSB, and b0, b1 are used for classification. The bitmask classifier may be applied to luminance or chroma classification. Whether to use the MSB or LSB for the bitmask N may be fixed or switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels.

[0153] In some embodiments, the luma position and C3 bitmask may be combined and switched at SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels. Different combinations may be different classifiers.

[0154] In some embodiments, a "max number of 1s" of the bitmask limit may be applied to limit the number of corresponding offsets. For example, the "max number of 1s" of the bitmask is limited to 4 in the SPS, which yields a maximum number of offsets in the sequence of 16. Bitmasks in different POCs may differ, but the "max number of 1s" must not exceed 4 (the entire class must not exceed 16). The "max number of 1s" value may be signaled and switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels.

[0155]

[0156] Table 6-2: Classifier examples use bitmasks for classification (bitmask locations are underlined)

[0157] In some embodiments, as illustrated in FIG. 11, other cross-component chroma samples, e.g., chroma sample 1102 and its neighboring samples, may also be fed to the CCSAO classification for the current chroma sample 1104, e.g. Cr chroma samples may be fed to the CCSAO Cb classification. Cb chroma samples may be fed to the CCSAO Cr classification. The classifier of the cross-component chroma samples may be the same as the lumens cross-component classifier, or may have its own classifier as described in this disclosure. Two classifiers may be combined to form a co-classifier for classifying the current chroma sample. For example, a co-classifier combining cross-component lumens and chroma samples yields a total of 16 classes, as illustrated in Table 6-3 below.

[0158]

[0159] Table 6-3: Example of a classifier using a co-classifier combining cross-component luminar and chroma samples (bit mask positions are underlined)

[0160] All of the aforementioned classifications (C0, C1, C1', C2, C3) can be combined. For example, refer to Table 6-4 below.

[0161]

[0162] Table 6-4: Different classifiers combined

[0163] In some embodiments, the classifier example (C2) uses the difference (Yn) between juxtaposed and neighboring luma samples. FIG. 12(c) illustrates an example of Yn having a dynamic range of [-1024, 1023] when the bit depth is 10. Let C2 band_num be the number of equally divided bands of the Yn dynamic range.

[0164]

[0165] In some embodiments, C0 and C2 are combined to yield a general classifier. For example, for different pictures (different POCs), different classifiers are applied as shown in Table 7 below.

[0166]

[0167] Table 7: Different general classifiers applied to different pictures

[0168] In some embodiments, all of the classifiers mentioned above (C0, C1, C1', C2) are combined. For example, for different pictures (different POCs), different classifiers are applied as shown in Table 8-1 below.

[0169]

[0170] Table 8-1: Different general classifiers applied to different pictures

[0171] In some embodiments, classifier example (C4) uses the difference between the CCSAO input values ​​and the sample values ​​to be compensated for classification, as shown in Table 8-2 below. For example, when CCSAO is applied at the ALF stage, the difference between the current component ALF-pre- and ALF-post-sample values ​​is used for classification. To help classify (quantize) the difference, one or more threshold values ​​may be predefined (e.g., maintained in a lookup table (LUT)) or signaled at SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels. The C4 classifier may be combined with the C0 Y / U / V bandNum to form a co-classifier (e.g., POC1 example as shown in Table 8-2).

[0172]

[0173] Table 8-2: Classifier example uses the difference between CCSAO input values ​​and compensated sample values ​​for classification

[0174] In some embodiments, the classifier example (C5) uses "coding information" to aid in subblock classification, as different coding modes can introduce different distortion statistics in the reconstructed image. The CCSAO sample is classified by the coding information prior to the sample, and a combination of coding information can form a classifier, for example, as shown in Table 8-3 below. FIG. 30 below illustrates another example of different stages of coding information for C5.

[0175]

[0176] Table 8-3: CCSAO samples are classified by their prior coding information, and combinations of coding information can form classifiers

[0177] In some embodiments, the classifier example (C6) uses YUV color-converted values ​​for classification. For example, to classify the current Y component, 1 / 1 / 1 juxtaposed or neighboring Y / U / V samples are selected to be color-converted to RGB, and the R value is quantized using C3 bandNum to become the current Y component classifier.

[0178] In some embodiments, other classifier examples that use only current component information for current component classification may be used as cross-component classifiers. For example, as illustrated in FIG. 5 and Table 1, luminance sample information and eo-class are used to derive EdgeIdx and classify current chroma samples. Other "non-cross-component" classifiers that may also be used as cross-component classifiers include edge direction, pixel intensity, pixel variation, pixel variance, pixel sum-of-Laplacian, sobel operator, compass operator, high-pass filtered value, low-pass filtered value, etc.

[0179] In some embodiments, multiple classifiers are used in the same POC. The current frame is divided by several regions, each region using the same classifier. For example, three different classifiers are used in POC0, and which classifier (0, 1, or 2) is used is signaled at the CTU level as shown in Table 9 below.

[0180]

[0181] Table 9: Different general classifiers applied to different regions in the same picture

[0182] In some embodiments, the maximum number of multiple classifiers (multiple classifiers may also be referred to as alternative offset sets) may be fixed or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels. In one example, the fixed (predefined) maximum number of multiple classifiers is 4. In that case, 4 different classifiers are used in POC0, and which classifier (0, 1, or 2) is used is signaled at the CTU level. A truncated-unary (TU) code may be used to indicate the classifier used for each luminary or chroma CTB. For example, as shown in Table 10 below, when the TU code is 0: CCSAO is not applied; when the TU code is 10: Set 0 is applied; when the TU code is 110: Set 1 is applied; when the TU code is 1110: Set 2 is applied; When the TU code is 1111: Set 3 applies. Fixed-length codes, golomb-rice codes, and exponential-golomb codes can also be used to denote classifiers (offset set indices) for CTBs. Three different classifiers are used in POC1.

[0183]

[0184] Table 10: TU (truncated-unary) codes are used to indicate the classifier used for each chroma CTB.

[0185] Examples of Cb and Cr CTB offset set indices are given for the 1280x720 sequence POC0 (if the CTU size is 128x128, the number of CTUs in the frame is 10x6). POC0 Cb uses 4 offset sets, and Cr uses 1 offset set. As shown in Table 11-1 below, when the offset set index is 0: CCSAO is not applied; when the offset set index is 1: Set 0 is applied; when the offset set index is 2: Set 1 is applied; when the offset set index is 3: Set 2 is applied; when the offset set index is 4: Set 3 is applied. Type refers to the position of the selected juxtaposed luminance sample (Yi). Different offset sets may have different types, band_num, and corresponding offsets.

[0186]

[0187] Table 11-1: Examples of Cb and Cr CTB offset set indices are given for the 1280x720 sequence POC0 (when the CTU size is 128x128, the number of CTUs in the frame is 10x6).

[0188] In some embodiments, examples of jointly using juxtaposed / current and neighboring Y / U / V samples for classification are listed in Table 11-2 below (3-component joint bandNum classification for each Y / U / V component). In POC0, the {2,4,1} offset sets are used for {Y, U, V}, respectively. Each offset set can be adaptively switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels. Different offset sets may have different classifiers. For example, to classify the current Y4 luma sample as a candidate position (candPos) shown in FIGS. 6b and 6c, Y set0 selects {current Y4, juxtaposed U4, juxtaposed V4} as candidates, each having different bandNum {Y, U, V} = {16,1,2}. If {candY, candU, candV} are used as sample values ​​for the selected {Y, U, V} candidates, the total number of classes is 32, and the derivation of class indices can be illustrated as follows:

[0189] Another example is in the POC1 component V set 1 classification. In that example, candPos = {neighboring Y8, neighboring U3, neighboring V0} with bandNum = {4,1,2} is used to create 8 classes.

[0190]

[0191] Table 11-2: Example of jointly using juxtaposed / current and neighboring Y / U / V samples for classification

[0192] In some embodiments, for example, as illustrated in Table 11-3 below, examples are listed in which juxtaposed and neighboring Y / U / V samples are jointly utilized for the current Y / U / V sample classification (three-component joint edgeNum (C1s) and bandNum classification for each Y / U / V component). Edge CandPos is the central position used for the C1s classifier, edge bitMask is the activation indicator for the C1s neighboring samples, and edgeNum is the corresponding number of C1s classes. In this example, C1s is applied only to the Y classifier (so edgeNum is the same as edgeNumY), and edge candPos is always Y4 (current / juxtaposed sample position). However, C1s can be applied to Y / U / V classifiers having edge candPos as neighboring sample positions.

[0193] If diff represents the comparison scores of Y C1s, the derivation of classIdx is as follows.

[0194]

[0195]

[0196] Table 11-3 (Part 1): Examples of jointly using juxtaposed / current and neighboring Y / U / V samples for classification

[0197]

[0198] Table 11-3 (Part 2): Examples of jointly using juxtaposed / current and neighboring Y / U / V samples for classification

[0199]

[0200] Table 11-3 (Part 3): Examples of jointly using juxtaposed / current and neighboring Y / U / V samples for classification

[0201] In some embodiments, the maximum band_num (bandNumY, bandNumU, or bandNumV) may be fixed or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels. For example, the maximum band_num is fixed to 16 in the decoder, and for each frame, 4 bits are signaled to indicate the C0 band_num in the frame. Some other examples of maximum band_num are listed in Table 12 below.

[0202]

[0203] Table 12: Maximum band_num and band_num bit examples

[0204] In some embodiments, the maximum number of classes or offsets for each set (or all added sets) (combinations using multiple classifiers together, e.g., C1s edgeNum * C1 bandNumY * bandNumU * bandNumV) may be fixed or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels. For example, the maximum is fixed for all added sets, such that class_num=256*4, and an encoder conformance check or decoder normative check may be used to check the constraints.

[0205] In some embodiments, restrictions may be applied to the C0 classification, for example, by restricting band_num (bandNumY, bandNumU, or bandNumV) to only powers of 2. Instead of explicitly signaling band_num, the syntax band_num_shift is signaled. The decoder may use a shift operation to avoid multiplication. Different band_num_shifts may be used for different components.

[0206]

[0207] Another example of operation is considering rounding to reduce errors.

[0208]

[0209] For example, if band_num_max(Y, U, or V) is 16, the possible band_num_shift candidates are 0, 1, 2, 3, and 4, corresponding to band_num = 1, 2, 4, 8, and 16, as shown in Table 13.

[0210]

[0211] Table 13: Band_num and corresponding band_num_shift candidates

[0212] In some embodiments, the classifiers applied to Cb and Cr are different. The Cb and Cr offsets for all classes can be signaled separately. For example, different signaled offsets are applied to different chroma components as shown in Table 14 below.

[0213]

[0214] Table 14: Cb and Cr offsets for all classes can be signaled separately

[0215] In some embodiments, the maximum offset value is fixed or signaled in the sequence parameter set (SPS) / adaptation parameter set (APS) / picture parameter set (PPS) / picture header (PH) / slice header (SH) / region / CTU / CU / subblock / sample level. For example, the maximum offset is between [-15, 15]. Different components may have different maximum offset values.

[0216] In some embodiments, offset signaling may utilize Differential pulse-code modulation (DPCM). For example, offsets {3, 3, 2, 1, -1} may be signaled as {3, 0, -1, -1, -2}.

[0217] In some embodiments, offsets may be stored in an APS or memory buffer for reuse of the next picture / slice. An index may be signaled to indicate which stored previous frame offsets are used for the current picture.

[0218] In some embodiments, the classifiers for Cb and Cr are the same. The Cb and Cr offsets for all classes can be signaled together, for example, as shown in Table 15 below.

[0219]

[0220] Table 15: Cb and Cr offsets for all classes can be jointly signaled

[0221] In some embodiments, the classifiers for Cb and Cr may be the same. The Cb and Cr offsets for all classes may be signaled together with the sign flag difference, for example, as shown in Table 16 below. According to Table 16, when the Cb offsets are (3, 3, 2, -1), the derived Cr offsets are (-3, -3, -2, 1).

[0222]

[0223] Table 16: Cb and Cr offsets for all classes can be signaled together with the sign flag difference

[0224] In some embodiments, the sign flag may be signaled for each class, for example, as shown in Table 17 below. According to Table 17, when the Cb offsets are (3, 3, 2, -1), the derived Cr offsets are (-3, 3, 2, 1) according to their respective sign flags.

[0225]

[0226] Table 17: Cb and Cr offsets for all classes can be signaled jointly with the sign flag signaled for each class

[0227] In some embodiments, the classifiers for Cb and Cr may be identical. The Cb and Cr offsets for all classes may be signaled together with weight differences, for example, as shown in Table 18 below. The weight (w) may be selected from a limited table, e.g., ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4 ... etc., where |w| includes only powers of 2. According to Table 18, when the Cb offsets are (3, 3, 2, -1), the derived Cr offsets are (-6, -6, -4, 2) according to their respective sign flags.

[0228]

[0229] Table 18: Cb and Cr offsets for all classes can be co-signaled with weight differences

[0230] In some embodiments, weights may be signaled for each class, for example, as shown in Table 19 below. According to Table 19, when the Cb offsets are (3, 3, 2, -1), the derived Cr offsets are (-6, 12, 0, -1) according to their respective sign flags.

[0231]

[0232] Table 19: Cb and Cr offsets for all classes can be co-signaled with the weights signaled for each class

[0233] In some embodiments, when multiple classifiers are used in the same POC, different offset sets are signaled individually or jointly.

[0234] In some embodiments, previously decoded offsets may be stored for use in future frames. To reduce offset signaling overhead, an index may be signaled to indicate which previously decoded offset set is used for the current frame. For example, POC0 offsets may be reused by POC2 with a signaling offset set idx=0 as shown in Table 20 below.

[0235]

[0236] Table 20: An index may be signaled to indicate which previously decoded offset set is used for the current frame

[0237] In some embodiments, the reuse offset set idx for Cb and Cr may differ, for example, as shown in Table 21 below.

[0238]

[0239] Table 21: An index may be signaled to indicate which previously decoded offset set is used for the current frame, and the index may differ for the Cb and Cr components.

[0240] In some embodiments, to reduce signaling overhead, offset signaling may utilize additional syntax including a start and a length. For example, when band_num=256, only offsets of band_idx=37–44 are signaled. In the example in Table 22-1 below, both the start and length syntaxes are 8-bit fixed-length coded to match the band_num bits.

[0241]

[0242] Table 22-1: Offset signaling uses additional syntax including start and length

[0243] In some embodiments, where CCSAO is applied to all YUV 3 components, juxtaposed and neighboring YUV samples may be used jointly for classification, and all aforementioned offset signaling methods for Cb / Cr may be extended to Y / Cb / Cr. In some embodiments, different component offset sets may be stored and used separately (each component has its own stored sets) or jointly (each component shares / reuses the same stored ones). An example of a separate set is shown in Table 22-2 below.

[0244]

[0245] Table 22-2: An example illustrating that different component offset sets can be stored and used separately (each component has its own stored sets) or jointly (each component shares / reuses the same stored set).

[0246] In some embodiments, if the sequence bit depth is higher than 10 (or a specific bit depth), the offset may be quantized before signaling. At the decoder side, the decoded offset is de-quantized before application, as shown in Table 23 below. For example, for a 12-bit sequence, the decoded offsets are shifted left by 2 (de-quantized).

[0247]

[0248] Table 23: Decoded offset is inversely quantized before application

[0249] In some embodiments, the offset can be calculated as CcSaoOffsetVal=(1 - 2 * ccsao_offset_sign_flag) * (ccsao_offset_abs << (BitDepth - Min(10, BitDepth))).

[0250] In some embodiments, sample processing is described below. Let R(x, y) be the input luminance or chroma sample value prior to CCSAO, and R'(x, y) be the output luminance or chroma sample value after CCSAO:

[0251] offset = ccsao_offset[R(x, y)'s class_index]

[0252] R'(x, y) = Clip3( 0, (1 << bit_depth) - 1, R(x, y) + offset )

[0253] According to the above mathematical formulas, each luminance or chroma sample value R(x, y) is classified using the indicated classifier of the current picture and / or current offset set idx. The corresponding offset of the derived class index is added to each luminance or chroma sample value R(x, y). The clip function Clip 3 is applied to (R(x, y) + offset) to produce an output luminance or chroma sample value R'(x, y) within a bit depth dynamic range, e.g., range 0 to (1 << bit_depth) - 1.

[0254] In some embodiments, boundary processing is described below. If any of the juxtaposed and adjacent luminar (chroma) samples used for classification are outside the current picture, CCSAO is not applied to the current chroma (luma) sample. FIG. 13a is a block diagram illustrating that, according to some embodiments of the present disclosure, CCSAO is not applied to the current chroma (luma) sample when any of the juxtaposed and adjacent luminar (chroma) samples used for classification are outside the current picture. For example, in FIG. 13a(a), if a classifier is used, CCSAO is not applied to the chroma components in the left column 1 of the current picture. For example, if C1' is used, CCSAO is not applied to the chroma components in the left column 1 and top row 1 of the current picture, as illustrated in FIG. 13a(b).

[0255] FIG. 13b is a block diagram illustrating that, according to some embodiments of the present disclosure, a CCSAO is applied to a current luminary or chroma sample when any of the juxtaposed and adjacent luminary or chroma samples used for classification are out of the current picture. In some embodiments, a variation is that when any of the juxtaposed and adjacent luminary or chroma samples used for classification are out of the current picture, the missing samples are repeatedly used as shown in FIG. 13b(a), or the missing samples are mirror-padded to generate samples for classification as shown in FIG. 13b(b), and a CCSAO may be applied to the current luminary or chroma samples. In some embodiments, the disabled / repetitive / mirror picture boundary processing method disclosed herein may also be applied to a subpicture / slice / tile / CTU / 360 virtual boundary if any of the juxtaposed and neighboring luminance (chroma) samples used for classification are currently outside the subpicture / slice / tile / patch / CTU / 360 virtual boundary.

[0256] For example, a picture is divided into one or more tile rows and one or more tile columns. A tile is a sequence of CTUs that covers a rectangular area of ​​the picture.

[0257] A slice consists of an integer number of complete tiles within a picture tile or an integer number of consecutive complete CTU rows.

[0258] A subpicture includes one or more slices that collectively cover a rectangular area of ​​the picture.

[0259] In some embodiments, 360-degree video is captured on a sphere and essentially has no "boundaries," and reference samples outside the boundaries of the reference picture in the projected domain can always be acquired from neighboring samples in the sphere domain. For projection formats composed of multiple faces, discontinuities appear between two or more adjacent faces in the frame-packed picture, regardless of what kind of compact frame-packing array is used. In VVC, vertical and / or horizontal virtual boundaries are introduced where in-loop filtering operations are disabled, and the locations of such boundaries are signaled in the SPS or picture header. Compared to using two tiles, one for each set of consecutive faces, the use of 360 virtual boundaries is more flexible because it does not require the face size to be a multiple of the CTU size. In some embodiments, the maximum number of vertical 360 virtual boundaries is 3, and the maximum number of horizontal 360 virtual boundaries is also 3. In some embodiments, the distance between two virtual boundaries is greater than the CTU size, and the virtual boundary particle size is 8 luminar samples, for example, an 8x8 sample grid.

[0260] FIG. 14 is a block diagram illustrating that, according to some embodiments of the present disclosure, CCSAO is not applied to the current chroma sample when a corresponding selected juxtaposed or neighboring luminance sample used for classification is outside the virtual space defined by a virtual boundary. In some embodiments, the virtual boundary (VB) is a virtual line separating the space within a picture frame. In some embodiments, when the virtual boundary (VB) is applied to the current frame, CCSAO is not applied to chroma samples having a selected corresponding luminance location outside the virtual space defined by the virtual boundary. FIG. 14 illustrates an example having a virtual boundary for a C0 classifier having nine luminance location candidates. For each CTU, CCSAO is not applied to chroma samples where the corresponding selected luminance location is outside the virtual space enclosed by the virtual boundary. For example, in FIG. 14 (a), CCSAO is not applied to the chroma sample (1402) when the selected Y7 luminance sample location is on the other side of the horizontal virtual boundary (1406) located 4 pixel lines from the bottom side of the frame. For example, in FIG. 14 (b), CCSAO is not applied to the chroma sample (1404) when the selected Y5 luminance sample location is on the other side of the vertical virtual boundary (1408) located y pixel lines from the right side of the frame.

[0261] FIG. 15 illustrates that iterative or mirror padding may be applied to luminance samples outside the virtual boundary according to some embodiments of the present disclosure. FIG. 15(a) illustrates an example of iterative padding. When the original Y7 is selected as the classifier located at the bottom side of VB (1502), the Y4 luminance sample value is used for classification instead of the original Y7 luminance sample value (copied to the Y7 position). FIG. 15(b) illustrates an example of mirror padding. When Y7 is selected as the classifier located at the bottom side of VB (1504), the Y1 luminance sample value, which is symmetric to the Y7 value for the Y0 luminance sample, is used for classification instead of the original Y7 luminance sample value. Since padding methods provide more chroma sample possibilities for applying CCSAO, more coding gain can be achieved.

[0262] In some embodiments, limitations may be applied to reduce the CCSAO required line buffer and simplify boundary processing condition checks. FIG. 16 illustrates that an additional 1.00 line buffer, namely, all line luminance samples of line -5 above the current VB (1602), may be required when all 9 parallel neighboring luminance samples are used for classification according to some implementations of the present disclosure. FIG. 10b(a) illustrates an example using only 6 luminance candidates for classification, which reduces the line buffer and does not require any additional boundary checks as in FIG. 13a and FIG. 13b.

[0263] In some embodiments, using luminance samples for CCSAO classification may increase the luminance line buffer and thus increase the cost of decoder hardware implementation. FIG. 17 illustrates an example in AVS according to some implementations of the present disclosure in which nine luminance candidate CCSAOs intersecting VB (1702) may increase two additional luminance line buffers. For luminance and chroma samples above the virtual boundary (VB) (1702), DBF / SAO / ALF is processed in the current CTU row. For luminance and chroma samples below VB (1702), DBF / SAO / ALF is processed in the next CTU row. In AVS decoder hardware design, the pre-DBF samples for luminance lines -4 to -1, the pre-SAO samples for line -5, and the pre-DBF samples for chroma lines -3 to -1, and the pre-SAO samples for line -4 are stored as line buffers for processing the next CTU row DBF / SAO / ALF. When processing the next CTU row, luminance and chroma samples that are not in the line buffers are not available. However, for example, at the chroma line -3(b) position, the chroma sample is processed in the next CTU row, but the CCSAO requires pre-SAO luminance sample lines -7, -6, and -5 for classification. Since pre-SAO luminance sample lines -7 and -6 are not in the line buffers, they are not available. Furthermore, adding pre-SAO luminance sample lines -7 and -6 to the line buffers would increase the cost of decoder hardware implementation. In some examples, the lumina VB (line -4) and chroma VB (line -3) may differ (not aligned).

[0264] Similar to FIG. 17, FIG. 18a illustrates an example in VVC according to some implementations of the present disclosure in which nine luminance candidate CCSAOs intersecting with VB (1802) can increase one additional luminance line buffer. VB may differ in different standards. In VVC, since the luminance VB is line -4 and the chroma VB is line -2, nine candidate CCSAOs can increase one luminance line buffer.

[0265] In some embodiments, in the first solution, CCSAO is disabled for the chroma sample if any of the luminance candidates of the chroma sample span across VB (which is currently outside the chroma sample VB). FIGS. 19a through 19c illustrate that in AVS and VVC, according to some embodiments of the present disclosure, CCSAO is disabled for the chroma sample if any of the luminance candidates of the chroma sample span across VB (1902) (which is currently outside the chroma sample (VB)). FIG. 14 also illustrates some examples of this embodiment.

[0266] In some embodiments, in the second solution, repetitive padding is used for "cross VB" luminance candidates, for a CCSAO from a luminance line on the other side of VB and close thereto, e.g., luminance line -4. In some embodiments, repetitive padding from the nearest neighbor luminance below VB is implemented for "cross VB" chroma candidates. FIGS. 20a through 20c illustrate that in AVS and VVC, according to some embodiments of the present disclosure, a CCSAO is enabled using repetitive padding for a chroma sample when any of the luminance candidates of the chroma sample spans VB (2002) (which is currently outside the chroma sample (VB)). FIG. 14(a) also illustrates some examples of this implementation.

[0267] In some embodiments, in the third solution, mirror padding is used for the CCSAO from below the luminance VB for "cross VB" luminance candidates. FIGS. 21a through 21c illustrate that in AVS and VVC, according to some embodiments of the present disclosure, the CCSAO is enabled using mirror padding for the chroma sample when any of the luminance candidates of the chroma sample spans the VB (2102) (which is currently outside the chroma sample (VB)). FIG. 14(b) and FIG. 13b(b) also illustrate some examples of this implementation. In some embodiments, in the fourth solution, "two-sided symmetric padding" is used to apply the CCSAO. FIGS. 22a and 22b illustrate that a CCSAO is enabled using bilateral symmetric padding for some examples of different CCSAO shapes (e.g., 9 luminance candidates (Fig. 22a) and 8 luminance candidates (Fig. 22b)) according to some embodiments of the present disclosure. For a set of luminance samples having a juxtaposed central luminance sample of chroma samples, if one side of the set of luminance samples is outside the VB (2202), bilateral symmetric padding is applied to both sides of the set of luminance samples. For example, in FIG. 22a, luminance samples Y0, Y1, and Y2 are outside the VB (2202), and thus both Y0, Y1, Y2 and Y6, Y7, Y8 are padded using Y3, Y4, Y5. For example, in FIG. 22b, the luma sample Y0 is outside VB (2202), and thus Y0 is padded using Y2 and Y7 is padded using Y5.

[0268] FIG. 18b illustrates an example according to some embodiments of the present disclosure where juxtaposed or adjacent chroma samples are used to classify current lumina samples, and a selected chroma candidate may span across VB and an additional chroma line buffer is required. Solutions 1 through 4 similar to those described above may be applied to address the problem.

[0269] Solution 1 is to disable CCSAO for luminance samples when any of the chroma candidates may span across VB.

[0270] Solution 2 is to use iterative padding from the nearest neighbor of the chroma under VB for "cross VB" chroma candidates.

[0271] Solution 3 is to use mirror padding from below the chroma VB for "cross VB" chroma candidates.

[0272] Solution 4 is to use "two-sided symmetric padding." For a candidate set centered on a CCSAO juxtaposed chroma sample, if one side of the candidate set is outside the VB, two-sided symmetric padding is applied to both sides.

[0273] Padding methods provide the possibility of more luminance or chroma samples for applying CCSAO, so greater coding gain can be achieved.

[0274] In some embodiments, at the bottom picture (or slice, tile, brick) boundary CTU row, samples under VB are processed in the current CTU row, so the special processing (solutions 1, 2, 3, 4) above is not applied at the bottom picture (or slice, tile, brick) boundary CTU row. For example, a 1920x1080 frame is divided by 128x128 CTUs. The frame contains 15x9 CTUs (rounded). The bottom CTU row is the 15th CTU row. The decoding process is for each CTU row, and for each CTU row, for each CTU. Deblocking needs to be applied along the horizontal CTU boundaries between the current and next CTU rows. CTB VB is applied to each CTU row because, within a single CTU, at the bottom 4 / 2 luminance / chroma line, DBF samples (in the case of VVC) are processed in the next CTU row and are not available for CCSAO in the current CTU row. However, at the bottom CTU row of the picture frame, at the bottom 4 / 2 luminance / chroma line DBF samples are available in the current CTU row because there is no next CTU row to the left and they are processed as DBFs in the current CTU row.

[0275] In some embodiments, the VB displayed in FIGS. 13 through 22 may be replaced by the boundary of the subpicture / slice / tile / patch / CTU / 360 virtual boundary. In some embodiments, the positions of the chroma and luminance samples in FIGS. 13 through 22 may be switched. In some embodiments, the positions of the chroma and luminance samples in FIGS. 13 through 22 may be replaced by the positions of the first chroma sample and the second chroma sample. In some embodiments, the ALF VB inside the CTU may be horizontal in common. In some embodiments, the boundary of the subpicture / slice / tile / patch / CTU / 360 virtual boundary may be horizontal or vertical.

[0276] In some embodiments, limitations may be applied to reduce the CCSAO required line buffer and to simplify boundary processing condition checks as described in FIG. 16. FIG. 23 illustrates limitations on using a limited number of luma candidates for classification according to some implementations of the present disclosure. FIG. 23(a) illustrates a limitation of using only 6 luma candidates for classification. FIG. 23(b) illustrates a limitation of using only 4 luma candidates for classification.

[0277] In some embodiments, the applied area is implemented. The CCSAO applied area unit may be CTB-based. That is, in a single CTB, the on / off control and CCSAO parameters (offsets used for classification, luminance candidate locations, band_num, bitmask, etc., offset set index) are identical.

[0278] In some embodiments, the applied area may not be aligned with the CTB boundary. For example, the applied area is not aligned with the chroma CTB boundary but is shifted. Syntaxes (on / off controls, CCSAO parameters) are still signaled for each CTB, but the truly applied area is not aligned with the CTB boundary. FIG. 24 illustrates that, according to some implementations of the present disclosure, the CCSAO applied area is not aligned with the CTB / CTU boundary (2406). For example, the applied area is not aligned with the chroma CTB / CTU boundary (2406), but the (4, 4) samples shifted to the top left are aligned with VB (2408). This unaligned CTB boundary design is advantageous for the deblocking process because the same deblocking parameters are used for each 8x8 deblocking process area.

[0279] In some embodiments, the CCSAO application area unit (mask size) may be varied (larger or smaller than the CTB size) as shown in Table 24. The mask size may differ for different components. The mask size may be switched at the SPS / APS / PPS / PH / SH / area / CTU / CU / subblock / sample levels. For example, at PH, a series of mask on / off flags and offset set indices are signaled to indicate the respective CCSAO area information.

[0280]

[0281] Table 24: CCSAO application area unit (mask size) may vary

[0282] In some embodiments, the CCSAO applied area frame partitions may be fixed. For example, the frame is partitioned into N areas. FIG. 25 illustrates that, according to some implementations of the present disclosure, the CCSAO applied area frame partitions may be fixed to CCSAO parameters.

[0283] In some embodiments, each region may have its own region on / off control flag and CCSAO parameters. Additionally, if the region size is larger than the CTB size, it may have both CTB on / off control flags and region on / off control flags. FIGS. 25 (a) and (b) illustrate some examples of partitioning a frame into N regions. FIGS. 25 (a) illustrates vertical partitioning of four regions. FIGS. 25 (b) illustrates square partitioning of four regions. In some embodiments, similar to the picture-level CTB all on control flag (ph_cc_sao_cb_ctb_control_flag / ph_cc_sao_cr_ctb_control_flag), if the region on / off control flag is off, the CTB on / off flags may be further signaled. Otherwise, the CCSAO is applied to all CTBs in this region without further signaling the CTB flags.

[0284] In some embodiments, different CCSAO application areas may share the same area on / off control and CCSAO parameters. For example, in FIG. 25(c), areas 0 through 2 share the same parameters, and areas 3 through 15 share the same parameters. FIG. 25(c) also illustrates area on / off control flags, and CCSAO parameters may be signaled in Hilbert scan order.

[0285] In some embodiments, the CCSAO coverage area unit may be partitioned from the picture / slice / CTB level into a quad-tree / binary-tree / ternary-tree. Similar to CTB partitioning, a series of partition flags are signaled to indicate the CCSAO coverage area partition. FIG. 26 illustrates that, according to some implementations of the present disclosure, the CCSAO coverage area may be a binary-tree (BT) / quad-tree (QT) / ternary-tree (TT) partitioned from the frame / slice / CTB level.

[0286] FIG. 27 is a block diagram illustrating a plurality of classifiers used and switched at different levels within a picture frame according to some embodiments of the present disclosure. In some embodiments, when a plurality of classifiers are used in a single frame, the method of how to apply the classifier set index may be switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels. For example, four sets of classifiers are used in a frame and are switched at PH as shown in Table 25 below. FIG. 27 (a) and (c) illustrate a default fixed region classifier. FIG. 27 (b) illustrates the classifier set index being signaled at the mask / CTB level, where 0 means CCSAO off for this CTB, and 1 through 4 mean set index.

[0287]

[0288] Table 25: Four sets of classifiers are used in the frame and switched in the PH

[0289] In some embodiments, for the default region case, the region-level flag may be signaled when CTBs in this region do not use the default set index (e.g., the region-level flag is 0), but use a different classifier set in this frame. For example, when the default set index is used, the region-level flag is 1. For example, in the square partition 4 regions, the following classifier sets are used as shown in Table 26 below.

[0290]

[0291] Table 26: Zone-level flags can be signaled to indicate whether CTBs in this zone are not using default set indexes

[0292] FIG. 28 is a block diagram illustrating that, according to some implementations of the present disclosure, CCSAO applied area partitioning can be dynamic and switched at the picture level. For example, FIG. 28(a) illustrates that three sets of CCSAO offsets are used in this POC (set_num = 3), and the picture frame is vertically partitioned into three areas. FIG. 28(b) illustrates that four sets of CCSAO offsets are used in this POC (set_num = 4), and the picture frame is horizontally partitioned into four areas. FIG. 28(c) illustrates that three sets of CCSAO offsets are used in this POC (set_num = 3), and the picture frame is raster-partitioned into three areas. Each area may have its own area that is an all-on flag to store on / off control bits per CTB. The number of areas depends on the signaled picture set_num.

[0293] A CCSAO application area may be a specific region based on coding information within a block (sample position, sample coded modes, loop filter parameters, etc.). For example, 1) a CCSAO application area may be applied only when samples are coded in skip mode, 2) a CCSAO application area may include only N samples along CTU boundaries, 3) a CCSAO application area may include only samples on an 8x8 grid within a frame, 4) a CCSAO application area may include only DBF-filtered samples, or 5) a CCSAO application area may include only the topmost M and the leftmost N rows in a CU. Different application areas may use different classifiers. For example, in a CTU, skip mode uses C1, the 8x8 grid uses C2, and both skip mode and the 8x8 grid use C3. For example, in a CTU, skip-mode coded samples use C1, samples at the CU center use C2, and skip-mode coded samples at the CU center use C3. FIG. 29 illustrates that CCSAO classifiers according to some implementations of the present disclosure may consider current or cross-component coding information. For example, different coding modes / parameters / sample locations may form different classifiers. Different coding information may be combined to form a co-classifier. Different regions may use different classifiers. FIG. 29 also illustrates another example of an applied region.

[0294] In some embodiments, the implemented CCSAO syntax is illustrated in Table 27 below. In some examples, the binarization of each syntax element may be changed. In AVS3, the term patch is similar to slice, and the patch header is similar to the slice header. FLC represents a fixed-length code. TU represents a truncated unary code. EGk represents an exponential Golom code of order k, where k can be fixed. SVLC represents a signed EG0. UVLC represents an unsigned EG0.

[0295]

[0296]

[0297]

[0298]

[0299] Table 27: Exemplary CCSAO Syntax

[0300] If the upper level flag is off, the lower level flags can be inferred from the flag's off state and do not need to be signaled. For example, if ph_cc_sao_cb_flag is false in this picture, ph_cc_sao_cb_band_num_minus1, ph_cc_sao_cb_luma_type, cc_sao_cb_offset_sign_flag, cc_sao_cb_offset_abs, ctb_cc_sao_cb_flag, cc_sao_cb_merge_left_flag, and cc_sao_cb_merge_up_flag are inferred to be non-existent and false.

[0301] In some embodiments, SPS ccsao_enabled_flag is conditioned on the SPS SAO enabled flag as shown in Table 28 below.

[0302]

[0303] Table 28: SPS ccsao_enabled_flag is conditioned on the SPS SAO enabled flag

[0304] In some embodiments, ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_control_flag indicate whether to enable Cb / Cr CTB on / off control particle size. If ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_control_flag are enabled, ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag may be further signaled. Otherwise, whether CCSAO is applied in the current picture depends on ph_cc_sao_cb_flag and ph_cc_sao_cr_flag without further signaling ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag at the CTB level.

[0305] In some embodiments, for ph_cc_sao_cb_type and ph_cc_sao_cr_type, to reduce bit overhead, a flag may be further signaled to distinguish whether a center-juxtended luminal position (position Y0 in FIG. 10) is used for classification of chroma samples. Similarly, when cc_sao_cb_type and cc_sao_cr_type are signaled at the CTB level, a flag may be further signaled by the same mechanism. For example, if the number of C0 luminal position candidates is 9, cc_sao_cb_type0_flag is further signaled to distinguish whether a center-juxtended luminal position is used, as shown in Table 29 below. If a center-juxtended luminal position is not used, cc_sao_cb_type_idc is used to indicate which of the remaining 8 neighboring luminal positions is used.

[0306]

[0307] Table 29: cc_sao_cb_type0_flag is signaled to distinguish whether a center-juxtaposed luma position is used

[0308] Table 30 below illustrates examples in AVS where a single (set_num = 1) or multiple (set_num > 1) classifiers are used in the frame. Note that the syntax notation can be mapped to the notation used above.

[0309]

[0310]

[0311]

[0312] Table 30: Examples in AVS where a single (set_num = 1) or multiple (set_num > 1) classifier is used in a picture frame

[0313] If each region is combined with FIG. 25 or FIG. 27 having its own set, the syntax example is the region on / off control flag as shown in Table 31 below. It may include.

[0314]

[0315] Table 31: Each zone has its own set, and the syntax example is a zone on / off control flag. May include

[0316] In some embodiments, extensions to intra and inter posterior prediction SAO filters are further illustrated below. In some embodiments, the SAO classification methods disclosed in this disclosure may serve as posterior prediction filters, and the prediction may be intra, inter, or other prediction tools, e.g., intra block copy. FIG. 30 is a block diagram illustrating the SAO classification methods disclosed in this disclosure serving as posterior prediction filters according to some implementations of this disclosure.

[0317] In some embodiments, for each Y, U, and V component, a corresponding classifier is selected. Then, for each component prediction sample, it is first classified, and a corresponding offset is added. For example, each component may use current and neighboring samples for classification. Y uses current Y and neighboring Y samples, and U / V uses current U / V samples for classification as shown in Table 32 below. FIG. 31 is a block diagram illustrating that for a post-prediction SAO filter, each component may use current and neighboring samples for classification according to some implementations of the present disclosure.

[0318]

[0319] Table 32: Corresponding classifiers selected for each Y, U, and V component

[0320] In some embodiments, refined prediction samples (Ypred', Upred', Vpred') are updated by adding a corresponding class offset and then used for intra, inter, or other predictions.

[0321] Ypred' = clip3(0, (1 << bit_depth)-1, Ypred + h_Y[i])

[0322] Upred' = clip3(0, (1 << bit_depth)-1, Upred + h_U[i])

[0323] Vpred' = clip3(0, (1 << bit_depth)-1, Vpred + h_V[i])

[0324] In some embodiments, for the chroma U and V components, a cross-component (Y) may be used for additional offset classification in addition to the current chroma component. The additional cross-component offset (h'_U, h'_V) may be added to the current component offset (h_U, h_V), for example, as shown in Table 33 below.

[0325]

[0326] Table 33: For chroma U and V components, in addition to the current chroma components, the cross-component (Y) can be used for additional offset classification.

[0327] In some embodiments, refined prediction samples (Upred'', Vpred'') are updated by adding a corresponding class offset and then used for intra, inter, or other predictions.

[0328] Upred" = clip3(0, (1 << bit_depth)-1, Upred' + h'_U[i])

[0329] Vpred" = clip3(0, (1 << bit_depth)-1, Vpred' + h'_V[i])

[0330] In some embodiments, intra and inter predictions may use different SAO filter offsets.

[0331] FIG. 32 is a flowchart illustrating an exemplary process (3200) for decoding a video signal using cross-component correlation according to some implementations of the present disclosure.

[0332] A video decoder (30) (as illustrated in FIG. 3) receives a picture frame (3210) containing a first component and a second component in a first color space from a video signal.

[0333] The video decoder (30) determines a classifier for the second component in the first color space from a first set of one or more samples of the first component in the second color space associated with each sample of the second component in the first color space (3220).

[0334] The video decoder (30) determines the sample offset for each sample of the second component in the first color space according to the classifier (3230).

[0335] The video decoder (30) modifies the values ​​of each sample of the second component in the first color space based on the determined sample offset (3240).

[0336] In some embodiments, the first component is a component selected from the group consisting of a lumina component, a first chroma component, and a second chroma component, and the second component is a component selected from the group consisting of a lumina component, a first chroma component, and a second chroma component.

[0337] In some embodiments, a first set of one or more samples is obtained by using a linear transformation to convert one or more samples of the first component associated with each sample of the second component from the first color space to the second color space.

[0338] In some embodiments, the linear transformation is expressed by the following.

[0339]

[0340] Here, a, b, c, d, e, f, g, h, and i are predetermined constants, I1, I2, and I3 are the values ​​of samples of the first component, second component, and third component in the first color space, respectively, and O1, O2, and O3 are the values ​​of samples of the first component, second component, and third component in the second color space, respectively.

[0341] In some embodiments, the first color space is a YUV color space and the second color space is an RGB color space.

[0342] In some embodiments, the first color space is an RGB color space, and the second color space is a YUV color space.

[0343] In some embodiments, a first set of one or more samples is obtained by using a non-linear transformation to convert one or more samples of the first component associated with each sample of the second component from the first color space to the second color space.

[0344] In some embodiments, determining a classifier (3220) further comprises determining a classifier for a second component in a first color space from a second set of one or more samples of a second component associated with each sample of the second component, and the second set of one or more samples is in the first color space or the second color space.

[0345] In some embodiments, the picture frame further comprises a third component, and determining the classifier (3220) further comprises determining a classifier for the second component in the first color space from a third set of one or more samples of the third component associated with each sample of the second component, the third component is a second chroma component, and the third set of one or more samples is in the first color space or the second color space.

[0346] In some embodiments, the classifier is determined by at least one of a first sub-classifier and / or a second sub-classifier, the first sub-classifier is determined by dividing a first dynamic range of values ​​of a first set of one or more samples of a first component into a first number of bands and selecting a band based on the intensity value of a first set of one or more samples of a first component, and the second sub-classifier is determined based on the direction and intensity of edge information of a first sub-group of a second set of one or more samples of a second component.

[0347] In some embodiments, the first component is a first chroma component and the second component is a lumina component, and a first set of one or more samples of the first component associated with each sample of the second component is selected from one or more of the juxtaposed and neighboring samples of the first component for each sample of the second component.

[0348] In some embodiments, a second set of one or more samples of the second component associated with each sample of the second component is selected from one or more of the current and neighboring samples of the second component for each sample of the second component.

[0349] In some embodiments, determining a classifier (3220) further comprises determining a classifier for a second component in a first color space from a fourth set of one or more samples of one or more of the first, second, and third components associated with each sample of the second component, and the fourth set of one or more samples is in the first color space or the second color space.

[0350] In some embodiments, determining the classifier (3220) further includes determining the classifier for the second component in the first color space from the prior coding information of the fourth set of one or more samples.

[0351] In some embodiments, determining the classifier (3220) further includes determining the class index of the classifier as follows:

[0352]

[0353] Here, C is the value of the current or paralleled sample, P1, P2, ... P8 are the values ​​of neighboring samples, Index2ClassTable is a predefined table of index classes, th is a threshold, and if xy > th, f(x, y) = 1; if xy = th, f(x, y) = 0; and if xy < th, f(x, y) = -1.

[0354] FIG. 33 illustrates a computing environment (3310) combined with a user interface (3350). The computing environment (3310) may be part of a data processing server. The computing environment (3310) includes a processor (3320), memory (3330), and an input / output (I / O) interface (3340).

[0355] The processor (3320) typically controls the overall operations of the computing environment (3310), such as operations associated with display, data acquisition, data communication, and image processing. The processor (3320) may include one or more processors for executing instructions to perform all or part of the steps in the aforementioned methods. Additionally, the processor (3320) may include one or more modules that facilitate interaction between the processor (3320) and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip machine, a graphics processing unit (GPU), etc.

[0356] Memory (3330) is configured to store various types of data to support the operation of the computing environment (3310). Memory (3330) may include predetermined software (3332). Examples of such data include instructions for any applications or methods operating on the computing environment (3310), video datasets, image data, etc. Memory (3330) may be implemented using any type of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EEEPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic memory, flash memory, magnetic or optical disk, or a combination thereof.

[0357] The I / O interface (3340) provides an interface between the processor (3320) and peripheral interface modules such as a keyboard, a click wheel, and buttons. The buttons may include, but are not limited to, a home button, a scan start button, and a scan stop button. The I / O interface (3340) may be combined with an encoder and a decoder.

[0358] In an embodiment, a non-transient computer-readable storage medium is also provided, comprising a plurality of programs executable by a processor (3320) in a computing environment (3310), for example, in memory (3330), to perform the methods described above. Alternatively, the non-transient computer-readable storage medium may store a bitstream or data stream comprising encoded video information (e.g., video information including one or more syntax elements) generated by an encoder (e.g., video encoder (20) in FIG. 2) using the encoding method described above for use by a decoder (e.g., video decoder (30) in FIG. 3) when decoding video data, for example. The non-transient computer-readable storage medium may be, for example, ROM, RAM (Random Access Memory), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0359] In an embodiment, a computing device is also provided comprising one or more processors (e.g., processor (3320)); and a non-transient computer-readable storage medium or memory (3330) storing a plurality of programs executable by one or more processors, wherein one or more processors are configured to perform the methods described above when executing the plurality of programs.

[0360] In an embodiment, to perform the methods described above, a computer program product is also provided that includes a plurality of programs, for example, in memory (3330), executable by a processor (3320) in a computing environment (3310). For example, the computer program product may include a non-transient computer-readable storage medium.

[0361] In an embodiment, the computing environment (3310) may be implemented with one or more ASICs, DSPs, DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0362] Additional embodiments also include various subsets of the above embodiments combined in various other embodiments or rearranged in other ways.

[0363] In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted through a computer-readable medium as one or more instructions or codes, and may be executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media corresponding to media of the same type as data storage media, or communication media including any medium that facilitates the transmission of a computer program from one place to another according to a communication protocol, for example. In this way, computer-readable media may generally correspond to (1) a non-transient type of computer-readable storage media or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes and / or data structures for the implementation of the embodiments described in this application. A computer program product may include a computer-readable medium.

[0364] The terms used in the description of the implementations in this specification are intended to describe specific implementations only and are not intended to limit the scope of the claims. As used in the description of the implementations and the appended claims, singular forms (“a,” “an,” and “the”) are intended to include plural forms unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used in this specification refers to and includes any and all possible combinations of one or more of the associated enumerated items. It will also be understood that the terms “comprises” and / or “comprising,” as used in this specification, specify the presence of the stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or groups thereof.

[0365] It will be understood that while terms such as first, second, etc. may be used to describe various elements in this specification, these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, without departing from the scope of the embodiments, the first electrode may be referred to as the second electrode, and similarly, the second electrode may be referred to as the first electrode. The first electrode and the second electrode are both electrodes, but they are not the same electrode.

[0366] Throughout this specification, references to “one example,” “an example,” “an exemplary example,” etc., in the singular or plural form, mean that one or more specific features, structures, or characteristics described in relation to the examples are included in at least one example of this disclosure. Accordingly, the appearance of phrases such as “in one example,” “in an example,” or “in an exemplary example,” appearing in the singular or plural in various places throughout this specification, do not necessarily refer to the same example. Furthermore, specific features, structures, or characteristics in one or more examples may include combinations in any suitable manner.

[0367] The description of this application is provided for illustrative and illustrative purposes only and is not intended to limit the invention in its entirety or as disclosed. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art who have the benefit of the teachings set forth in the foregoing description and the associated drawings. The embodiments have been selected and described to best illustrate the principles and practical applications of the invention, and to enable those skilled in the art to understand the invention in various embodiments and to best utilize various embodiments having various modifications suitable for the basic principles and the specific use being considered. Accordingly, it should be understood that the scope of the claims is not limited to specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of the appended claims.

Claims

Claim 1 A method for decoding a video signal, comprising: receiving a picture frame from the video signal comprising a first component and a second component in a first color space; determining a classifier for the second component in the first color space from a first set of one or more samples of the first component associated with each sample of the second component in the first color space, wherein the first set of one or more samples is in a second color space; determining a sample offset for each sample of the second component in the first color space according to the classifier; and modifying the value of each sample of the second component in the first color space based on the determined sample offset. Claim 2 A method according to claim 1, wherein the first component is a component selected from the group consisting of a lumina component, a first chroma component, and a second chroma component, and the second component is a component selected from the group consisting of the lumina component, the first chroma component, and the second chroma component. Claim 3 A method according to claim 1, wherein the first set of one or more samples is obtained by converting one or more samples of the first component associated with each sample of the second component from the first color space to the second color space using a linear transformation. Claim 4 In paragraph 3, the above linear transformation is expressed by the following, and Here, a, b, c, d, e, f, g, h, i are predetermined constants, I1, I2, I3 are values ​​of samples of the first component, the second component, and the third component in the first color space, respectively, and O1, O2, O3 are values ​​of samples of the first component, the second component, and the third component in the second color space, respectively, method. Claim 5 A method according to claim 1, wherein the first color space is a YUV color space and the second color space is an RGB color space. Claim 6 A method according to claim 1, wherein the first color space is an RGB color space and the second color space is a YUV color space. Claim 7 A method according to claim 1, wherein the first set of one or more samples is obtained by converting one or more samples of the first component associated with each sample of the second component from the first color space to the second color space using a non-linear transformation. Claim 8 The method according to claim 1, wherein the step of determining the classifier further comprises the step of determining a classifier for the second component in the first color space from a second set of one or more samples of the second component associated with each sample of the second component, wherein the second set of one or more samples is in the first color space or the second color space. Claim 9 In claim 8, the picture frame further comprises a third component, and the step of determining the classifier further comprises the step of determining a classifier for the second component in the first color space from a third set of one or more samples of the third component associated with each sample of the second component, wherein the third component is a second chroma component, and the third set of one or more samples is in the first color space or the second color space. Claim 10 A method according to claim 8, wherein the classifier is determined by a first sub-classifier, a second sub-classifier, or at least one of the first sub-classifier and the second sub-classifier, wherein the first sub-classifier is determined by dividing a first dynamic range of values ​​of a first set of one or more samples of the first component into a first number of bands and selecting a band based on the intensity value of the first set of one or more samples of the first component, and the second sub-classifier is determined based on the direction and intensity of edge information of a first sub-group of a second set of one or more samples of the second component. Claim 11 A method according to claim 1, wherein the first component is a first chroma component and the second component is a lumina component, and a first set of one or more samples of the first component associated with each sample of the second component is selected from one or more of the juxtaposed and neighboring samples of the first component for each sample of the second component. Claim 12 In claim 8, a second set of one or more samples of the second component associated with each sample of the second component is selected from one or more of the current and neighboring samples of the second component for each sample of the second component. Claim 13 The method according to claim 1, wherein the step of determining the classifier further comprises the step of determining a classifier for the second component in the first color space from a fourth set of one or more samples of one or more components among the first, second, and third components associated with each sample of the second component, wherein the fourth set of one or more samples is in the first color space or the second color space. Claim 14 In claim 13, the step of determining the classifier further comprises the step of determining the classifier for the second component in the first color space from the prior coding information of the fourth set of the one or more samples. Claim 15 In paragraph 13, the step of determining the classifier further includes the step of determining the class index of the classifier as follows, and Here, C is the value of the current or paralleled sample, P1, P2, ... P8 are the values ​​of neighboring samples, Index2ClassTable is a predefined table of index classes, th is a threshold, and if xy > th, f(x, y) = 1; if xy = th, f(x, y) = 0; and if xy < th, f(x, y) = -1, method. Claim 16 A method according to claim 1, wherein the chroma format of the video signal is 4:2:2 or 4:4:

4. Claim 17 An electronic device comprising: one or more processing units; a memory coupled to the one or more processing units; and a plurality of programs stored in the memory, wherein the plurality of programs, when executed by the one or more processing units, cause the electronic device to perform the method of any one of claims 1 to 16. Claim 18 A non-transient computer-readable storage medium, wherein the non-transient computer-readable storage medium stores a bitstream comprising video information decoded by a method for decoding a video signal according to any one of claims 1 to 16. Claim 19 A non-transient computer-readable storage medium storing a plurality of programs to be executed by an electronic device having one or more processing units, wherein the plurality of programs, when executed by the one or more processing units, cause the electronic device to perform the method of any one of claims 1 to 16. Claim 20 A computer program stored on a computer-readable storage medium, wherein the computer program comprises instructions executed by an electronic device having one or more processing units, and the instructions, when executed by the one or more processing units, cause the electronic device to perform the method of any one of claims 1 to 16. Claim 21 A method for storing a bitstream, comprising the step of storing the bitstream in a digital storage medium, wherein the bitstream comprises data decoded by the method of any one of claims 1 to 15. Claim 22 In claim 21, the method wherein the chroma format of the video signal in the bitstream is 4:2:2 or 4:4:4.

Images