Coding extension of cross-component sample adaptive offsets.

Abstract

An electronic device performs a method for decoding video data, the method including receiving from a hierarchical video bitstream a first syntax element associated with a first level of the hierarchy, and in accordance with determining that the first syntax element indicates that cross-component sample adaptive offset (CCSAO) filter information is present at the first level, reconstructing from the video bitstream one or more regions below the first level jointly in accordance with the CCSAO filter information, and in accordance with determining that the first syntax element indicates that CCSAO filter information is not present at the first level, reconstructing from the video bitstream one or more regions individually in accordance with CCSAO filter information present at a second level of the hierarchy.

Description

[Technical Field]

[0001] Related Applications This application claims priority to U.S. Provisional Patent Application No. 63 / 174,920, filed April 14, 2021, and entitled "Cross-component Sample Adaptive Offset," which is incorporated by reference in its entirety.

[0002] This application relates generally to video encoding and compression, and more particularly to methods and apparatus for improving both luma and chroma encoding efficiency. [Background technology]

[0003] Digital video is supported by a variety of electronic devices, including digital televisions, laptop and desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, videoconferencing devices, and video streaming devices. Electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression standards. Some well-known video coding standards include Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC, also known as H.265 or MPEG-H Part 2), and Advanced Video Coding (AVC, also known as H.264 or MPEG-4 Part 10), which are jointly developed by ISO / IEC MPEG and ITU-T VCEG. AOMedia Video 1 (AV1) was developed by the Alliance for Open Media (AOM) as the successor to the previous standard, VP9. Audio Video Coding (AVS), which refers to a compression standard for digital audio and digital video, is another family of video compression standards developed by the Audio and Video Coding Standard Workgroup.

[0004] Video compression typically involves performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or eliminate redundancy inherent in video data. In block-based video coding, a video frame is divided into one or more slices, each of which contains multiple video blocks, sometimes referred to as coding tree units (CTUs). Each CTU may contain one coding unit (CU) or may be recursively divided into smaller CUs until a predetermined minimum CU size is reached. Each CU (also known as a leaf CU) contains one or more transform units (TUs), which in turn contain one or more prediction units (PUs). Each CU can be coded in either intra-mode, inter-mode, or IBC mode. Video blocks in an intra-coded (I) slice of a video frame are coded using spatial prediction with respect to reference samples in neighboring blocks within the same video frame. For video blocks in inter-coded (P or B) slices of a video frame, spatial prediction may be used with respect to reference samples in neighboring blocks within the same video frame, and temporal prediction may be used with respect to reference samples in other previous and / or future reference video frames.

[0005] Spatial or temporal prediction based on previously coded reference blocks, e.g., neighboring blocks, results in a predicted block for the current video block being coded. The process of searching for the reference block may be achieved by a block matching algorithm. Residual data representing pixel differences between the current block being coded and the predicted block is called a residual block or prediction error. Inter-coded blocks are coded according to the residual block and a motion vector pointing to a reference block in a reference frame that forms the predicted block. The process of determining the motion vector is generally called motion estimation. Intra-coded blocks are coded according to an intra-prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to a transform domain, e.g., the frequency domain, resulting in residual transform coefficients, which may then be quantized. The quantized transform coefficients are initially arranged in a two-dimensional array, which may be scanned to generate a one-dimensional vector of transform coefficients, which are then entropy coded into a video bitstream for stronger compression.

[0006] The encoded video bitstream is then stored in a computer-readable storage medium (e.g., flash memory) that can be accessed by another electronic device with digital video capabilities or transmitted directly to the electronic device via wired or wireless connection, and video decompression (which is the reverse process of the video compression described above) is then performed on the electronic device, for example, by parsing the encoded video bitstream to obtain syntax elements from the bitstream, reconstructing digital video data from the encoded video bitstream into its original format based at least in part on the syntax elements obtained from the bitstream, and rendering the reconstructed digital video data on a display of the electronic device.

[0007] As digital video quality progresses from high definition to 4Kx2K and even 8Kx4K, the amount of video data to be encoded / decoded increases exponentially, which poses a constant challenge: how to encode / decode video data more efficiently while maintaining the image quality of the decoded video data. Summary of the Invention

[0008] This application describes implementations related to video data encoding and decoding, and in particular, methods and apparatus for improving the coding efficiency of both luma and chroma components, including improving the coding efficiency by examining cross-component relationships between the luma and chroma components.

[0009] According to a first aspect of the present application, a method for decoding video data includes receiving, from a video bitstream having a hierarchical structure, a first syntax element associated with a first level of the hierarchical structure; and, in accordance with determining that the first syntax element indicates that cross-component sample adaptive offset (CCSAO) filter information is present at the first level, reconstructing from the video bitstream one or more regions below the first level jointly in accordance with the CCSAO filter information; and, in accordance with determining that the first syntax element indicates that CCSAO filter information is not present at the first level, reconstructing from the video bitstream one or more regions individually in accordance with CCSAO filter information present at a second level of the hierarchical structure.

[0010] In some embodiments, the video bitstream comprises a first component and a second component, and reconstructing one or more regions from the video bitstream according to the CCSAO filter information includes: determining a classifier for the second component from a set of one or more samples of the first component associated with each sample of the second component according to the CCSAO filter information, depending on the CCSAO filter being applied; determining whether to modify values ​​of each sample of the second component within a region of the one or more regions of the video bitstream according to the classifier; and, in response to determining to modify values ​​of each sample of the second component within the region according to the classifier, determining a sample offset for each sample of the second component according to the classifier; and modifying the values ​​of each sample of the second component based on the determined sample offset.

[0011] According to a second aspect of the present application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory, the programs, when executed by the one or more processing units, causing the electronic device to perform the method for encoding a video signal, as described above.

[0012] According to a third aspect of the present application, a non-transitory computer-readable storage medium stores a plurality of programs for execution by an electronic device having one or more processing units, the programs, when executed by the one or more processing units, causing the electronic device to perform the method for encoding a video signal, as described above.

[0013] According to a fourth aspect of the present application, a computer-readable storage medium stores a bitstream comprising video information generated by the video encoding method described above.

[0014] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

[0015] The accompanying drawings, which are included to provide a further understanding of the implementations, and which are incorporated in and constitute a part of this specification, illustrate the described implementations and, together with the description, serve to explain the underlying principles, with like reference numerals referring to corresponding parts. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a block diagram illustrating an exemplary video encoding system and a video decoding system according to some implementations of the present disclosure.

[0017] [Figure 2] FIG. 1 is a block diagram illustrating an exemplary video encoder according to some implementations of the present disclosure.

[0018] [Figure 3] FIG. 2 is a block diagram illustrating an exemplary video decoder according to some implementations of the present disclosure.

[0019] [Figure 4A] 1 is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes, according to some implementations of the present disclosure. [Figure 4B] 1 is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes, according to some implementations of the present disclosure. [Figure 4C] 1 is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes, according to some implementations of the present disclosure. [Figure 4D] 1 is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes, according to some implementations of the present disclosure. [Figure 4E] 1 is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes, according to some implementations of the present disclosure.

[0020] [Figure 5] FIG. 1 is a block diagram illustrating four gradient patterns used in pixel adaptive offset (SAO) according to some implementations of the present disclosure.

[0021] [Figure 6A] FIG. 1 is a block diagram illustrating a system and process of CCSAO applied to chroma samples and using DBF Y as input, according to some implementations of the present disclosure.

[0022] [Figure 6B] FIG. 1 is a block diagram illustrating a system and process for CCSAO applied to luma samples and chroma samples and using DBF Y / Cb / Cr as input, according to some implementations of the present disclosure.

[0023] [Figure 6C] FIG. 1 is a block diagram illustrating systems and processes of a CCSAO that can operate independently, according to some implementations of the present disclosure.

[0024] [Figure 6D] FIG. 1 is a block diagram illustrating a system and process for a CCSAO that can be applied recursively (2 or N times) with the same or different offsets, according to some implementations of the present disclosure.

[0025] [Figure 6E] FIG. 1 is a block diagram illustrating a system and process of CCSAO applied in parallel with the Enhanced Pixel Adaptive Offset (ESAO) of the AVS standard according to some implementations of the present disclosure.

[0026] [Figure 6F] FIG. 1 is a block diagram illustrating a system and process for a CCSAO applied after an SAO, according to some implementations of the present disclosure.

[0027] [Figure 6G]FIG. 10 is a block diagram illustrating that, according to some implementations of the present disclosure, the systems and processes of the CCSAO can operate independently without the use of CCALF.

[0028] [Figure 6H] FIG. 1 is a block diagram illustrating a system and process of a CCSAO applied in parallel with a cross-component adaptive loop filter (CCALF) according to some implementations of the present disclosure.

[0029] [Figure 7] FIG. 10 is a block diagram illustrating a sample process using a CCSAO according to some implementations of the present disclosure.

[0030] [Figure 8] FIG. 10 is a block diagram illustrating the CCSAO process being interleaved with vertical and horizontal deblocking filters (DBFs), according to some implementations of the present disclosure.

[0031] [Figure 9] 1 is a flowchart illustrating an exemplary process for decoding a video signal using cross-component correlation, according to some implementations of the present disclosure.

[0032] [Figure 10A] FIG. 10 is a block diagram illustrating a classifier that uses different luma (or chroma) sample positions for C0 classification, according to some implementations of the present disclosure.

[0033] [Figure 10B] 10A-10C illustrate some examples of different shapes of luma candidates, according to some implementations of the present disclosure.

[0034] [Figure 11] FIG. 10 is a block diagram of a sample process illustrating that all of the co-located luma / chroma samples and neighboring luma / chroma samples can be placed into a CCSAO classification, according to some implementations of the present disclosure.

[0035] [Figure 12] 10 illustrates an exemplary classifier that uses replacing co-located luma sample values ​​with values ​​obtained by weighting the co-located luma sample and neighboring luma samples, according to some implementations of the present disclosure.

[0036] [Figure 13A] FIG. 10 is a block diagram illustrating that CCSAO is not applied to the current chroma (luma) sample if any of the co-located luma (chroma) sample and neighboring luma (chroma) samples used for classification are outside the current picture, according to some implementations of the present disclosure.

[0037] [Figure 13B] FIG. 10 is a block diagram showing that CCSAO is applied to a current luma sample or a current chroma sample when either the co-located luma sample or chroma sample used for classification and the neighboring luma sample or chroma sample are outside the current picture, according to some implementations of the present disclosure.

[0038] [Figure 14] FIG. 10 is a block diagram illustrating that, according to some implementations of the present disclosure, if a selected co-located luma sample or a neighboring luma sample corresponding to the current chroma sample used for classification is outside a virtual space defined by a virtual boundary (VB), the CCSAO is not applied to the current chroma sample.

[0039] [Figure 15] 10 illustrates the application of repeat or mirror padding to luma samples outside the virtual boundary, according to some implementations of the present disclosure.

[0040] [Figure 16] According to some implementations of the present disclosure, if all nine co-located luma samples and neighboring luma samples are used for classification, one more luma line buffer is required.

[0041] [Figure 17] 10 illustrates an AVS diagram in which, according to some implementations of the present disclosure, a CCSAO of nine luma candidates may cross a VB, resulting in two more luma line buffers.

[0042] [Figure 18A] 10 illustrates a VVC diagram in which a CCSAO of nine luma candidates may cross a VB to add one more luma line buffer, according to some implementations of the present disclosure.

[0043] [Figure 18B] FIG. 10 illustrates that, according to some implementations of the present disclosure, when co-located or neighboring chroma samples are used to classify the current luma sample, the selected chroma candidate may exceed VB and an additional chroma line buffer may be required.

[0044] [Figure 19A] According to some implementations of the present disclosure, AVS and VVC indicate that if any of the luma candidates for a chroma sample are beyond VB (outside the current chroma sample VB), CCSAO is disabled for the chroma sample. [Figure 19B] According to some implementations of the present disclosure, AVS and VVC indicate that if any of the luma candidates for a chroma sample are beyond VB (outside the current chroma sample VB), CCSAO is disabled for the chroma sample. [Figure 19C] According to some implementations of the present disclosure, AVS and VVC indicate that if any of the luma candidates for a chroma sample are beyond VB (outside the current chroma sample VB), CCSAO is disabled for the chroma sample.

[0045] [Figure 20A]According to some implementations of the present disclosure, in AVS and VVC, if any of the luma candidates of a chroma sample exceeds VB (outside the current chroma sample VB), it indicates that CCSAO is enabled for the chroma sample using repetitive padding. [Figure 20B] According to some implementations of the present disclosure, in AVS and VVC, if any of the luma candidates of a chroma sample exceeds VB (outside the current chroma sample VB), it indicates that CCSAO is enabled for the chroma sample using repetitive padding. [Figure 20C] According to some implementations of the present disclosure, in AVS and VVC, if any of the luma candidates of a chroma sample exceeds VB (outside the current chroma sample VB), it indicates that CCSAO is enabled for the chroma sample using repetitive padding.

[0046] [Figure 21A] According to some implementations of the present disclosure, in AVS and VVC, if any of the luma candidates of a chroma sample exceeds VB (outside the current chroma sample VB), mirror padding is used to indicate that CCSAO is enabled for the chroma sample. [Figure 21B] According to some implementations of the present disclosure, in AVS and VVC, if any of the luma candidates of a chroma sample exceeds VB (outside the current chroma sample VB), mirror padding is used to indicate that CCSAO is enabled for the chroma sample. [Figure 21C] According to some implementations of the present disclosure, in AVS and VVC, if any of the luma candidates of a chroma sample exceeds VB (outside the current chroma sample VB), mirror padding is used to indicate that CCSAO is enabled for the chroma sample.

[0047] [Figure 22A]We show that according to some implementations of the present disclosure, CCSAO is enabled using double-sided symmetric padding for different CCSAO sample shapes. [Figure 22B] We show that according to some implementations of the present disclosure, CCSAO is enabled using double-sided symmetric padding for different CCSAO sample shapes.

[0048] [Figure 23] 1 illustrates the limitation of using a limited number of luma candidates for classification according to some implementations of the present disclosure.

[0049] [Figure 24] This indicates that, according to some implementations of the present disclosure, the CCSAO application region does not align with a coding tree block (CTB) / coding tree unit (CTU) boundary.

[0050] [Figure 25] It is shown that, according to some implementations of the present disclosure, the division of frames in the CCSAO application area can be left unchanged using CCSAO parameters.

[0051] [Figure 26] According to some implementation examples of the present disclosure, it is shown that the CCSAO application domain can be divided into binary tree (BT) / quad tree (QT) / ternary tree (TT) from the frame / slice / CTB level.

[0052] [Figure 27] FIG. 10 is a block diagram illustrating multiple classifiers being used and modified at different levels within a picture frame, according to some implementations of the present disclosure.

[0053] [Figure 28] FIG. 10 is a block diagram illustrating that, according to some implementations of the present disclosure, the division of the CCSAO application area is a dynamic division and can be changed at the picture level.

[0054] [Figure 29] 10A and 10B illustrate that, according to some implementations of the present disclosure, the CCSAO classifier can take into account current encoding information or cross-component encoding information.

[0055] [Figure 30] FIG. 10 is a block diagram illustrating the SAO classification method disclosed in this disclosure used as a post-prediction filter, according to some implementations of the present disclosure.

[0056] [Figure 31] FIG. 10 is a block diagram illustrating that the current sample and neighboring samples for each component of a predicted SAO filter can be used for classification, according to some implementations of the present disclosure.

[0057] [Figure 32] 1 is a flowchart illustrating an exemplary process for decoding a video signal using cross-component correlation according to some implementations of the present disclosure.

[0058] [Figure 33] FIG. 1 illustrates a computing environment connected to a user interface according to some implementations of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0059] Reference will now be made in detail to specific implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to aid in understanding the subject matter disclosed herein. However, it will be apparent to one skilled in the art that various modifications may be employed without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to one skilled in the art that the subject matter disclosed herein may be practiced in many types of electronic devices with digital imaging capabilities.

[0060] The first-generation AVS standards include the Chinese national standards "Information Technology, Advanced Audio Video Coding, Part 2: Video" (known as AVS1) and "Information Technology, Advanced Audio Video Coding Part 16: Radio Television Video" (known as AVS+). These standards achieve approximately 50% bitrate savings compared to the MPEG-2 standard with the same perceptual quality. The second-generation AVS standards include the Chinese national standard "Information Technology, Efficient Multimedia Coding" (known as AVS2), primarily aimed at transmitting extra HD television programs. The coding efficiency of AVS2 is twice that of AVS+. Meanwhile, the video portion of the AVS2 standard has been proposed by the Institute of Electrical and Electronics Engineers (IEEE) as one of the international standards for such applications. The AVS3 standard is a new-generation video coding standard for UHD video applications that aims to surpass the coding efficiency of the latest international standard, HEVC, thereby achieving approximately 30% bitrate savings over HEVC. The AVS3-P2 baseline was completed at the 68th AVS Conference in March 2019. This achieves approximately 30% bitrate savings over the HEVC standard. A reference software called the High Performance Model (HPM) is now maintained by the AVS group and provides a reference implementation of the AVS3 standard. Like HEVC, the AVS3 standard is built on a block-based composite video coding framework.

[0061] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some implementations of the present disclosure. As shown in Figure 1, system 10 includes a source device 12 that generates and encodes video data that is subsequently decoded by a destination device 14. Source device 12 and destination device 14 may comprise any of a wide variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0062] In some implementations, the destination device 14 may receive the encoded video data to be decoded via link 16. Link 16 may comprise any type of communication medium or device capable of moving the encoded video data from source device 12 to destination device 14. In one example, link 16 may comprise a communication medium that enables source device 12 to transmit the encoded video data directly to destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to destination device 14. The communication medium may comprise any wireless or wired communication medium, such as the radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may be part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from source device 12 to destination device 14.

[0063] In another implementation, the encoded video data may be transmitted from output interface 22 to storage device 32. The encoded video data in storage device 32 may then be accessed by destination device 14 via input interface 28. Storage device 32 may include any of a variety of distributed or locally accessible data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In yet another example, storage device 32 may correspond to a file server or another intermediate storage device capable of holding the encoded video data generated by source device 12. The stored video data from storage device 32 may be accessed by destination device 14 via streaming or download. A file server may be any type of computer capable of storing encoded video data and transmitting the encoded video data to destination device 14. Typical file servers include web servers (e.g., for websites), FTP servers, network-attached storage (NAS) devices, and local disk drives. The encoded video data may be accessed by destination device 14 through any standard data connection, including a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both wireless and wired channels suitable for accessing encoded video data stored on a file server. The transmission of the encoded video data from storage device 32 may be a streaming transmission, a download transmission, or a combination of both.

[0064] 1, source device 12 includes video source 18, video encoder 20, and output interface 22. Video source 18 may include a video source such as a video capture device (e.g., a video camera), a video archive storing previously captured video, a video feed interface receiving video from a video content provider, and / or a computer graphics system generating computer graphics data as source video, or a combination of such video sources. As an example, if video source 18 is a video camera in a security surveillance system, source device 12 and destination device 14 may form a camera phone or video phone. However, the implementations described herein are applicable to most video encoding and may be applicable to wireless and / or wired applications.

[0065] Live, pre-recorded, or computer-generated video may be encoded by video encoder 20. The encoded video data may be transmitted directly to destination device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored in storage device 32 for subsequent access by destination device 14 or other devices for decoding and / or playback. Output interface 22 may further include a modem and / or transmitter.

[0066] Destination device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or modem and may receive encoded video data using link 16. The encoded video data communicated using link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included in the encoded video data transmitted over a communications medium, stored on a storage medium, or stored on a file server.

[0067] In some implementations, destination device 14 may include a display device 34, which may be an integrated display device or an external display device configured to communicate with destination device 14. Display device 34 displays the decoded video data to a user and may comprise any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0068] Video encoder 20 and video decoder 30 may operate in accordance with proprietary or industry standards, such as VVC, HEVC, MPEG-4 Part 10, Advanced Video Coding (AVC), AVS, or extensions to such standards. It should be understood that the present application is not limited to any particular video encoding / decoding standard and may be applicable to other video encoding / decoding standards. It should be understood that in many cases, video encoder 20 of source device 12 may be configured to encode video data in accordance with any of these current or future standards. Similarly, it should be understood that in many cases, video decoder 30 of destination device 14 may be configured to decode video data in accordance with any of these current or future standards.

[0069] Each of video encoder 20 and video decoder 30 may be implemented as any of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. If implemented partially in software, an electronic device may store instructions for the software on a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the video encoding / decoding operations disclosed in this disclosure. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, any of which may be incorporated into the respective devices as part of a combined encoder / decoder (CODEC).

[0070] 2 is a block diagram illustrating an exemplary video encoder 20 according to some implementations described herein. Video encoder 20 may perform intra-predictive and inter-predictive coding of video blocks within a video frame. Intra-predictive coding relies on spatial prediction to reduce or eliminate spatial redundancy in video data within a given video frame or picture. Inter-predictive coding relies on temporal prediction to reduce or eliminate temporal redundancy in video data within adjacent video frames or pictures of a video sequence.

[0071] As shown in FIG. 2, the video encoder 20 includes a video data memory 40, a prediction processor 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processor 52, a quantizer 54, and an entropy encoder 56. The prediction processor 41 further includes a motion estimator 42, a motion compensation processor 44, a segmenter 45, an intra-predictor 46, and an intra-block copy (BC) processor 48. In some implementations, the video encoder 20 also includes an inverse quantizer 58, an inverse transform processor 60, and an adder 62 used for video block reconstruction. An in-loop filter 63, such as a deblocking filter, may be disposed between the adder 62 and the DPB 64 to perform filtering on block boundaries and remove blocky artifacts from the reconstructed image. Another in-loop filter 63, in addition to the deblocking filter, may also be used to filter the output of the adder 62. Further in-loop filter locations 63, such as a pixel adaptive offset (SAO) or an adaptive in-loop filter (ALF), may be used on the reconstructed CU before it is placed in a reference picture store and used as a reference for encoding future encoded video blocks. Video encoder 20 may take the form of programmable or fixed hardware locations, or video encoder 20 may be split into one or more of the programmable or fixed hardware locations shown.

[0072] Video data memory 40 may store video data to be encoded by components of video encoder 20. The video data for video data memory 40 may be obtained, for example, from video source 18. DPB 64 is a buffer that stores reference video data used in encoding video data by video encoder 20 (e.g., an encoder in an intra-predictive coding mode or an inter-predictive coding mode). Video data memory 40 and DPB 64 may be formed by any of a variety of memory devices. In various examples, video data memory 40 may be on-chip with other components of video encoder 20 or off-chip relative to those components.

[0073] As shown in FIG. 2, after receiving video data, a partitioner 45 in the prediction processor 41 partitions the video data into video blocks. This partitioning may include dividing the video frame into slices, tiles, or other larger coding units (CUs) according to a predetermined partitioning structure, such as a quadtree structure, associated with the video data. The video frame may be partitioned into multiple video blocks (or collections of video blocks called tiles). The prediction processor 41 may select one of multiple possible predictive coding modes, such as one of multiple intra-predictive coding modes or one of multiple inter-predictive coding modes, for the current video block based on an error result (e.g., code rate or distortion level). The prediction processor 41 may provide the resulting intra- or inter-predictively coded block to adder 50 to generate a residual block, and may provide the resulting intra- or inter-predictively coded block to adder 62 to reconstruct a coded block that is then used as part of a reference frame. Prediction unit 41 also provides syntax elements such as motion vectors, intra-mode indicators, partition information, and other such syntax information to entropy encoder 56 .

[0074] To select an appropriate intra-prediction coding mode for a current video block, intra-prediction unit 46 in prediction unit 41 may intra-predictively code the current video block relative to one or more neighboring blocks in the same frame as the current block to be coded, thereby achieving spatial prediction. Motion estimation unit 42 and motion compensation unit 44 in prediction unit 41 may inter-predictively code the current video block relative to one or more predictive blocks in one or more reference frames, thereby achieving temporal prediction. Video encoder 20 may implement multiple coding schemes, for example, selecting an appropriate coding mode for each block of video data.

[0075] In some implementations, motion estimation unit 42 determines the inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern in the sequence of video frames. A motion vector indicates the displacement of a prediction unit (PU) of a video block in the current video frame relative to a predictive block in a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. A motion vector may indicate, for example, the displacement of a PU of a video block in a current video frame or picture relative to a predictive block in a reference frame (or other coded unit) relative to a current block being coded in the current frame (or other coded unit). A video frame in the sequence may be designated as a P frame or a B frame according to the predetermined pattern. Intra BC unit 48 may determine vectors, e.g., block vectors, used for intra BC coding in a manner similar to the determination of motion vectors by motion estimation unit 42 used for inter prediction, or may utilize motion estimation unit 42 to determine the block vectors.

[0076] The prediction block is a block of the reference frame that is expected to closely match the PU of the video block being encoded in terms of pixel differences, which may be determined by sum of absolute difference (SAD), sum of square difference (SSD), or other difference measures. In some implementations, video encoder 20 may calculate values ​​at sub-integer pixel positions of the reference frame stored in DPB 64. For example, video encoder 20 may interpolate values ​​at quarter-pixel positions, eighth-pixel positions, or other sub-pixel positions of the reference frame. Thus, motion estimation unit 42 can perform motion search at whole-pixel positions and sub-pixel positions and output motion vectors with sub-pixel accuracy.

[0077] The motion estimation unit 42 calculates a motion vector of a PU of a video block in a frame coded using inter prediction by comparing the position of the PU with the position of a prediction block of a reference frame selected from the first reference frame list (List 0) or the second reference frame list (List 1). List 0 and List 1 each identify one or more reference frames stored in the DPB 64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44 and then to the entropy coding unit 56.

[0078] The motion compensation performed by motion compensation unit 44 may involve obtaining or generating a prediction block based on the motion vector determined by motion estimation unit 42. Upon receiving the motion vector of the PU of the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in one of the reference frame lists, obtain the prediction block from DPB 64, and forward the prediction block to summer 50. Summer 50 then forms a residual video block of pixel difference values ​​by subtracting pixel values ​​of the prediction block provided by motion compensation unit 44 from pixel values ​​of the current video block being coded. The pixel difference values ​​forming the residual video block may include a luma difference component, a chroma difference component, or both. Motion compensation unit 44 may also generate syntax elements associated with the video block of the video frame for use by video decoder 30 when decoding the video block of the video frame. The syntax elements may include, for example, syntax elements defining the motion vector used to identify the prediction block, any flags indicating a prediction mode, and any other syntax information described herein. It is noted that the motion estimator 42 and the motion compensator 44 may be integrated to provide a greater degree of integration, although the motion estimator 42 and the motion compensator 44 are shown separately for conceptual purposes.

[0079] In some implementations, the intra BC unit 48 may generate a vector and obtain a prediction block in a manner similar to that described above in connection with the motion estimation unit 42 and the motion compensation unit 44. However, the prediction block is within the same frame as the current block being coded, and the vector is referred to as a block vector in contrast to a motion vector. In particular, the intra BC unit 48 may determine an intra prediction mode to be used to code the current block. In some examples, the intra BC unit 48 may code the current block using various intra prediction modes, e.g., various intra prediction modes may be used in different coding schemes, and the performance of these may be verified through rate-distortion analysis. The intra BC unit 48 may then select an appropriate intra prediction mode to use from the verified various intra prediction modes and generate an intra mode indicator accordingly. For example, the intra BC unit 48 may calculate rate-distortion values ​​for the verified various intra prediction modes using rate-distortion analysis, and select the intra prediction mode with the best rate-distortion characteristics from the verified modes as the appropriate intra prediction mode to use. In many cases, the rate-distortion analysis determines the amount of distortion (or error) between a coded block and the original uncoded block that was previously coded to generate the coded block, as well as the bitrate (i.e., number of bits) used to generate the coded block. The intra BC unit 48 may calculate a ratio between the distortion and rate of various coded blocks to determine which intra prediction mode exhibits the best rate-distortion value for the block.

[0080] Alternatively, the intra BC unit 48 may use all or part of the motion estimation unit 42 and motion compensation unit 44 to perform the above-described functions of intra BC prediction according to the implementations described herein. In either case, for intra block copying, the predictive block may be a block that is considered to be a close match to the block being coded in terms of pixel differences. This may be determined by sum of absolute difference (SAD), sum of squared difference (SSD), or other difference measures. Identifying the predictive block may involve calculating values ​​at a fractional number of pixels.

[0081] Regardless of whether the predictive block is derived from the same frame according to intra prediction or from a different frame according to inter prediction, video encoder 20 may form a residual video block by subtracting pixel values ​​of the predictive block from pixel values ​​of the current video block being encoded to form pixel difference values. The pixel difference values ​​forming the residual video block may include both luma and chroma component differences.

[0082] The intra-prediction unit 46 may intra-predict the current video block as an alternative to the inter-prediction performed by the motion estimation unit 42 and motion compensation unit 44, or as an alternative to the intra-block copy prediction performed by the intra-BC unit 48 as described above. In particular, the intra-prediction unit 46 may determine the intra-prediction mode to use to encode the current block. To do this, the intra-prediction unit 46 may encode the current block using various intra-prediction modes, e.g., various intra-prediction modes may be used in different encoding schemes, and the intra-prediction unit 46 (or, in some examples, a mode selection unit) may select an appropriate intra-prediction mode to use from the examined intra-prediction modes. The intra-prediction unit 46 may provide information indicating the selected intra-prediction mode for the block to the entropy encoder 56. The entropy encoder 56 may encode the information indicating the selected intra-prediction mode in the bitstream.

[0083] After prediction processor 41 determines a prediction block for the current video block, either through inter-prediction or intra-prediction, adder 50 subtracts the prediction block from the current video block to form a residual video block. Residual video data in the residual block may be included in one or more transform units (TUs), which are provided to transform processor 52. Transform processor 52 converts the residual video data into residual transform coefficients using a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform.

[0084] The transform processor 52 may send the resulting transform coefficients to a quantizer 54, which quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the quantizer 54 may then perform a scan of the matrix containing the quantized transform coefficients. Alternatively, the scan may be performed by an entropy encoder 56.

[0085] Following quantization, entropy encoder 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) encoding, or another entropy encoding method or technique. The encoded bitstream may then be transmitted to video decoder 30 or archived in storage device 32 for subsequent transmission to or retrieval by video decoder 30. Entropy encoder 56 may also entropy encode motion vectors and other syntax elements of the current video frame being encoded.

[0086] Inverse quantization unit 58 and inverse transform unit 60 perform inverse quantization and inverse transformation, respectively, to reconstruct the residual video block in the pixel domain to generate reference blocks used to predict other video blocks. As described above, motion compensation unit 44 may generate a motion-compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate a non-integer number of pixel values ​​used in motion estimation.

[0087] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block that is stored in DPB 64. The reference block may then be used by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 as a prediction block for inter predicting another video block in a subsequent video frame.

[0088] 3 is a block diagram illustrating an exemplary video decoder 30 according to some implementations of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. Furthermore, the prediction processing unit 81 includes a motion compensation unit 82, an intra-prediction processing unit 84, and an intra-BC unit 85. The video decoder 30 may perform a decoding process that is substantially the reverse of the encoding process described above for the video encoder 20 in conjunction with FIG. 2. For example, the motion compensation unit 82 may generate prediction data based on motion vectors received from the entropy decoding unit 80, while the intra-prediction unit 84 may generate prediction data based on an intra-prediction mode indicator received from the entropy decoding unit 80.

[0089] In some examples, the tasks of performing the implementation of the present disclosure may be assigned to the components of the video decoder 30. In addition, in some examples, the implementation of the present disclosure may be distributed among one or more components of the video decoder 30. For example, the implementation of the present disclosure may be performed by the intra BC unit 85 alone or in combination with other components of the video decoder 30, such as the motion compensation unit 82, the intra prediction processor 84, or the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra BC unit 85, and the functionality of the intra BC unit 85 may be performed by other components of the prediction processor 81, such as the motion compensation unit 82.

[0090] Video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of video decoder 30. The video data stored in video data memory 79 may be obtained, for example, from storage device 32, from an attached video source such as a camera, via wired or wireless network communication of the video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include a coded picture buffer (CPB) that stores coded video data derived from the coded video bitstream. A decoded picture buffer (DPB) 92 of video decoder 30 stores reference video data used by video decoder 30 when decoding video data (e.g., in intra-prediction or inter-prediction coding modes). Video data memory 79 and DPB 92 may be formed using any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous dynamic random access memory (SDRAM), magnetoresistive random access memory (MRAM), resistive random access memory (RRAM), or other types of memory devices. For purposes of illustration, video data memory 79 and DPB 92 are shown in Figure 3 as two separate components of video decoder 30. However, it will be apparent to those skilled in the art that video data memory 79 and DPB 92 may be implemented in the same memory device or in separate memory devices. In some examples, video data memory 79 may be on-chip with other components of video decoder 30 or off-chip with respect to those components.

[0091] During the decoding process, video decoder 30 receives an encoded video bitstream representing video blocks of encoded video frames and associated syntax elements. Video decoder 30 may receive the syntax elements at the video frame level and / or the video block level. An entropy decoding unit 80 of video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors, or intra-prediction mode indicators, and other syntax elements. Entropy decoding unit 80 then forwards the motion vectors and other syntax elements to a prediction processor 81.

[0092] If the video frame is coded as an intra-prediction coded (I) frame or coded for an intra-coded predictive block in another type of frame, the intra-prediction processing unit 84 of the prediction processing unit 81 may generate predictive data for the video block of the current video frame based on the signaled intra-prediction mode and reference data obtained from previously decoded blocks of the current frame.

[0093] If a video frame is coded as an inter-prediction coded (i.e., B or P) frame, motion compensation unit 82 of prediction processor 81 generates one or more prediction blocks for the video blocks of the current video frame based on the motion vectors and other syntax elements received from entropy decoder 80. Each of the prediction blocks may be generated from a reference frame in one of the reference frame lists. Video decoder 30 may construct the reference frame lists, i.e., List 0 and List 1, using a default construction technique based on the reference frames stored in DPB 92.

[0094] In some examples, when a video block is coded according to an intra BC mode described in this application, intra BC unit 85 of prediction processor 81 generates a prediction block of the current video block based on the block vectors and other syntax elements received from entropy decoder 80. The prediction block may be within the same reconstructed region of the picture as the current video block as determined by video encoder 20.

[0095] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for video blocks of the current video frame by parsing the motion vectors and other syntax elements, and use the prediction information to generate a prediction block for the current video block being decoded. For example, motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode (e.g., inter-prediction or intra-prediction) used to encode the video blocks of the video frame, the type of inter-predicted frame (e.g., B or P), one or more configuration information of a reference frame list for the frame, the motion vector of each inter-predictively coded video block of the frame, the inter-prediction status of each inter-predictively coded video block of the frame, and other information for decoding video blocks in the current video frame.

[0096] Similarly, the intra BC unit 85 may use some of the received syntax elements, such as a flag for determining that the current video block was predicted using intra BC mode, configuration information about which video blocks of the frame are within the reconstructed region and should be stored in the DPB 92, block vectors for each intra BC predicted video block of the frame, the intra BC prediction status for each intra BC predicted video block of the frame, and other information to decode video blocks in the current video frame.

[0097] Motion compensation unit 82 may also perform interpolation using an interpolation filter, such as was used by video encoder 20 when encoding the video block, to calculate interpolated values ​​for a non-integer number of pixels of the reference block. In this case, motion compensation unit 82 may determine the interpolation filter used by video encoder 20 from the received syntax element and use the interpolation filter to generate the prediction block.

[0098] An inverse quantizer 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoder 80 using the same quantization parameters calculated by video encoder 20 for each video block in a video frame to determine the degree of quantization. An inverse transform processor 88 applies an inverse transform, such as an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients to reconstruct residual blocks in the pixel domain.

[0099] After motion compensation unit 82 or intra BC unit 85 generates a prediction block for the current video block based on the vectors and other syntax elements, summer 90 reconstructs a decoded video block for the current video block by summing the residual block from inverse transform processor 88 with the corresponding prediction block generated by motion compensation unit 82 and intra BC unit 85. An in-loop filter 91 may be disposed between summer 90 and DPB 92 for further processing the decoded video block. An in-loop filter unit 91, such as a deblocking filter, pixel adaptive offset (SAO), or adaptive in-loop filter (ALF), may be used on the reconstructed CU before it is placed in reference picture storage. Decoded video blocks in a given frame are then stored in DPB 92, which stores reference frames used for subsequent motion compensation of the next video block. DPB 92, or a memory device separate from DPB 92, may also store decoded video for subsequent display on a display device, such as display device 34 of FIG. 1 .

[0100] In a typical video encoding process, a video sequence typically includes an ordered set of frames or pictures. Each frame may include three sample arrays (noted SL, SCb, and SCr): SL is a two-dimensional array of luma samples; SCb is a two-dimensional array of Cb chroma samples; and SCr is a two-dimensional array of Cr chroma samples. In other examples, a frame may be monochrome and therefore include only one two-dimensional array of luma samples.

[0101] Like HEVC, the AVS3 standard is built on a block-based hybrid video coding framework. The input video signal is processed block by block (called a coding unit (CU)). Unlike HEVC, which divides blocks solely based on a quadtree, AVS3 divides a single coding tree unit (CTU) into CUs based on a quadtree, binary tree, or extended quadtree to adapt to various local characteristics. Additionally, the concept of multiple partitioning units used in HEVC is eliminated. That is, AVS3 does not distinguish between CUs, prediction units (PUs), and transform units (TUs). Instead, each CU is always used as the basic unit for both prediction and transformation and is not further divided. In the AVS3 tree partitioning structure, a CTU is first partitioned based on a quadtree structure. Then, each quadtree leaf node can be further partitioned based on a binary tree structure and an extended quadtree structure.

[0102] As shown in FIG. 4A, video encoder 20 (or, in particular, divider 45) generates a coded representation of a frame by first dividing the frame into a set of coding tree units (CTUs). A video frame may include an integer number of CTUs arranged consecutively in raster scan order from left to right and top to bottom. Each CTU is a logically largest coding unit, and the width and height of the CTU are signaled by video encoder 20 in a continuous parameter set, such that all CTUs in a video sequence have the same size, and the size is one of 128x128, 64x64, 32x32, and 16x16. However, it should be noted that the present application is not necessarily limited to a particular size. As shown in FIG. 4B, each CTU may comprise one coding tree block (CTB) for luma samples, two corresponding coding tree blocks for chroma samples, and syntax elements used to encode the samples in the coding tree blocks. The syntax elements describe the characteristics of different types of coded blocks of pixels and how the video sequence can be reconstructed by video decoder 30, including inter- or intra-prediction, intra-prediction mode, motion vectors, and other parameters. For monochrome pictures or pictures with three separate color planes, a CTU may comprise one coding tree block and syntax elements used to encode the samples of the coding tree block. A coding tree block may be an NxN block of samples.

[0103] To achieve superior performance, video encoder 20 may recursively perform tree partitioning, such as binary tree partitioning, ternary tree partitioning, quad tree partitioning, or a combination of both, on the coding tree blocks of a CTU to partition the CTU into smaller coding units (CUs). As shown in FIG. 4C , 64×64 CTU 400 is first partitioned into four smaller CUs, each with a block size of 32×32. Of the four smaller CUs, CU 410 and CU 420 are each partitioned into four CUs with a block size of 16×16. Two 16×16 CUs, 430 and 440, are each further partitioned into four CUs with a block size of 8×8. FIG. 4D shows a quad tree data structure illustrating the final result of the partitioning process for CTU 400 shown in FIG. 4C , where each leaf node of the quad tree corresponds to a CU, with sizes ranging from 32×32 to 8×8. As shown in FIG. 4B, each CU may include a coded block (CB) of luma samples, two corresponding coded blocks of chroma samples for the same frame size, and syntax elements used to encode the samples of the coded block. In a monochrome picture or a picture with three separate color planes, a CU may include one coded block and syntax elements used to encode the samples of the coded block. It should be noted that the quadtree partitioning shown in FIGS. 4C and 4D is for illustrative purposes only, and a CTU can be divided into CUs based on quadtree, ternary tree, or binary tree partitioning to accommodate various local characteristics. In various tree structures, a CTU is divided by a quadtree structure, and each quadtree leaf CU can be further divided by binary tree and ternary tree structures. As shown in FIG. 4E, there are five partitioning types in AVS3: quadtree, horizontal bisection, vertical bisection, horizontally extended quadtree, and vertically extended quadtree.

[0104] In some implementations, video encoder 20 may further divide a coding block of a CU into one or more M×N prediction blocks (PBs). A prediction block is a rectangular (square or non-square) block of samples to which the same prediction (inter or intra) is applied. A prediction unit (PU) of a CU may comprise a prediction block of luma samples, two corresponding prediction blocks of chroma samples, and syntax elements used to predict the prediction block. For monochrome pictures or pictures with three separate color planes, a PU may comprise one prediction block and syntax structures used to predict the samples in the prediction block. Video encoder 20 may generate a predicted luma block, a predicted Cb block, and a predicted Cr block for the luma prediction block, the Cb prediction block, and the Cr prediction block of each PU of a CU.

[0105] Video encoder 20 may use intra prediction or inter prediction to generate the predictive blocks of a PU. When video encoder 20 uses intra prediction to generate the predictive blocks of a PU, video encoder 20 may generate the predictive blocks of the PU based on decoded samples of a frame associated with the PU. When video encoder 20 uses inter prediction to generate the predictive blocks of a PU, video encoder 20 may generate the predictive blocks of the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0106] After video encoder 20 generates the predictive luma block, predictive Cb block, and predictive Cr block of one or more PUs of a CU, video encoder 20 may generate the luma residual block of the CU by subtracting the predictive luma block of the CU from its original luma coding block, such that each sample in the luma residual block of the CU indicates a difference between a luma sample in one of the predictive luma blocks of the CU and a corresponding sample in the original luma coding block of the CU. Similarly, video encoder 20 may generate the Cb residual block and the Cr residual block of the CU, such that each sample in the Cb residual block of the CU indicates a difference between a Cb sample in one of the predictive Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and such that each sample in the Cr residual block of the CU indicates a difference between a Cr sample in one of the predictive Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0107] Further, as shown in FIG. 4C , video encoder 20 may use quadtree partitioning to decompose the luma, Cb, and Cr residual blocks of a CU into one or more luma, Cb, and Cr transform blocks. A transform block is a rectangular (square or non-square) block of samples to which the same transform is applied. A transform unit (TU) of a CU may comprise a transform block of luma samples, two corresponding transform blocks of chroma samples, and syntax elements used to transform the transform block samples. Thus, each TU of a CU may be associated with a luma transform block, a Cb transform block, and a Cr transform block. In some examples, the luma transform block associated with a TU may be a sub-block of the luma residual block of the CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU. In monochrome pictures or pictures with three separate color planes, a TU may comprise one transform block and syntax structures used to transform the samples of the transform block.

[0108] Video encoder 20 may apply one or more transforms to a luma transform block of a TU to generate a luma coefficient block of the TU. The coefficient block may be a two-dimensional array of transform coefficients. The transform coefficients may be scalar quantities. Video encoder 20 may apply one or more transforms to a Cb transform block of the TU to generate a Cb coefficient block of the TU. Video encoder 20 may apply one or more transforms to a Cr transform block of the TU to generate a Cr coefficient block of the TU.

[0109] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), video encoder 20 may quantize the coefficient block. Broadly speaking, quantization refers to a process of quantifying transform coefficients, possibly reducing the amount of data used to represent the transform coefficients, thereby achieving further compression. After video encoder 20 quantizes a coefficient block, video encoder 20 may entropy encode syntax elements indicating the quantized transform coefficients. For example, video encoder 20 may perform Context-Adaptive Binary Arithmetic Coding (CABAC) on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 may output a bitstream including a series of bits forming a representation of the coded frame and associated data, which may be saved to storage device 32 or transmitted to destination device 14.

[0110] After receiving the bitstream generated by video encoder 20, video decoder 30 may parse the bitstream to obtain syntax elements from the bitstream. Video decoder 30 may reconstruct frames of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing video data is generally the reverse of the encoding process performed by video encoder 20. For example, video decoder 30 may perform an inverse transform on coefficient blocks associated with TUs of a current CU to reconstruct residual blocks associated with the TUs of the current CU. Video decoder 30 may also reconstruct coding blocks of the current CU by adding samples of predictive blocks of PUs of the current CU to corresponding samples of transform blocks of TUs of the current CU. After reconstructing the coding blocks of each CU of the frame, video decoder 30 may reconstruct the frame.

[0111] SAO is a process of modifying decoded samples by conditionally adding an offset value to each sample after applying a deblocking filter, based on the lookup table values ​​transmitted by the encoder. SAO filtering is performed region-wise based on the filtering type selected for each CTB using the syntax element sao-type-idx. A value of 0 for sao-type-idx indicates that no SAO filter is applied to the CTB, while values ​​1 and 2 indicate the use of band-offset and edge-offset filtering types, respectively. In band-offset mode, specified by sao-type-idx equal to 1, the selected offset value depends directly on the sample amplitude. In this mode, the entire sample amplitude range is evenly divided into 32 segments called bands, and sample values ​​belonging to four of these bands (consecutive out of the 32 bands) are modified by adding a transmitted value, expressed as a band offset, which can be positive or negative. The main reason for using four consecutive bands is that in smooth areas where banding artifacts may appear, the sample amplitudes in the CTB tend to be concentrated in only a few bands. In addition to this, the design chosen to use four offsets is integrated with an edge offset mode of operation that also uses four offset values. In the edge offset mode specified by sao-type-idx equal to 2, the syntax element sao-eo-class with a value between 0 and 3 indicates whether the horizontal, vertical, or one of the two diagonal slope directions is used for the edge offset classification of the CTB.

[0112] Figure 5 is a block diagram illustrating four gradient patterns used in SAO according to some implementations of the present disclosure. Four gradient patterns 502, 504, 506, and 508 are gradient patterns for each sao-eo-class in edge offset mode. The sample labeled "p" indicates that the central sample is under consideration. Two samples labeled "n0" and "n1" represent two neighboring samples along the (a) horizontal (sao-eo-class=0) gradient pattern, (b) vertical (sao-eo-class=1) gradient pattern, (c) 135° diagonal (sao-eo-class=2) gradient pattern, and (d) 45° (sao-eo-class=3) gradient pattern. Each sample in the CTB is classified into one of five EdgeIdx categories by comparing the sample value p at a certain location with the values ​​n0 and n1 of the two samples at neighboring locations shown in Figure 5. This classification is performed sample-by-sample based on the decoded sample value, so no additional signaling is required for EdgeIdx classification. Depending on the EdgeIdx category at the sample location, for EdgeIdx categories 1 through 4, the sample value is added with an offset value obtained from the transmitted lookup table. The offset value is always positive for categories 1 and 2, and negative for categories 3 and 4. Thus, the filter typically has a smoothing effect in edge offset mode. Table 1 below shows sample EdgeIdx categories for SAO edge classes. [Table 1]

[0113] In SAO types 1 and 2, a total of four amplitude offset values ​​are transmitted to the decoder per CTB. In type 1, the sign is also coded. The offset values ​​and associated syntax elements such as sao-type-idx and sao-eo-class are determined by the encoder, typically using criteria that optimize rate-distortion performance. A merge flag can be used to indicate whether SAO parameters are inherited from the left or top CTB, improving signaling efficiency. In summary, SAO is a nonlinear filtering operation that allows for further refinement of the reconstructed signal, expanding the signal representation both in smooth regions and around edges.

[0114] In some embodiments, the present application discloses a method and system for improving coding efficiency and reducing the complexity of pixel adaptive offset (SAO) by introducing cross-component information. SAO is used in the HEVC, VVC, AVS2, and AVS3 standards. In the following description, the existing SAO design of the HEVC, VVC, AVS2, and AVS3 standards is used as the basic SAO method. However, those skilled in the art of video coding can apply the cross-component method described in this disclosure to other loop filter designs and other coding tools that share the same design spirit. For example, in the AVS3 standard, SAO is replaced by a coding tool called enhanced pixel adaptive offset (ESAO). Alternatively, the CCSAO disclosed in the present application can be applied in parallel with the ESAO. In another example, the CCSAO can be applied in parallel with the Constrained Directional Enhancement Filter (CDEF) of the AV1 standard.

[0115] In the existing SAO designs of the HEVC, VVC, AVS2, and AVS3 standards, the luma Y sample offset value, the chroma Cb sample offset value, and the chroma Cr sample offset value are determined separately. That is, for example, the current chroma sample offset is determined only by the current chroma sample value and the neighboring chroma sample values ​​without considering the co-located luma sample or the neighboring luma sample. Meanwhile, the luma sample maintains more detailed information of the original picture than the chroma sample, which can better determine the current chroma sample offset. Furthermore, since the chroma sample usually loses high-frequency details after the RGB-to-YCbCr color conversion or the quantization and deblocking filter, introducing the luma sample with the high-frequency details that are maintained for determining the chroma offset can better reconstruct the chroma sample. Therefore, further gains can be expected by examining the cross-component correlation, for example, using the Cross-Component Sample Adaptive Offset (CCSAO) method and system. In some embodiments, the correlations described herein include not only cross-component sample values, but also picture / coding information such as prediction / residual coding mode, transform type, and quantization / deblocking / SAO / ALF parameters derived from the cross-components.

[0116] Another example is SAO, where the luma sample offset is determined by the luma sample only. However, for example, luma samples with the same band offset (BO) classification can be further classified by the co-located chroma sample and its neighboring chroma samples, thereby achieving more effective classification. SAO classification can be seen as a shortcut to compensate for sample differences between the original picture and the reconstructed picture. Therefore, effective classification is desirable.

[0117] FIG. 6A is a block diagram illustrating a system and process for a CCSAO applied to chroma samples and using DBF Y as input, according to some implementations of the present disclosure. The luma sample (DBF Y) after the luma deblocking filter is used to determine separate offsets for chroma Cb and chroma Cr after SAO Cb and SAO Cr. For example, a current chroma sample 602 is first classified using a co-located luma sample 604 and a neighboring luma sample 606 (white), and the corresponding CCSAO offset value of the corresponding class is added to the current chroma sample value. FIG. 6B is a block diagram illustrating a system and process for a CCSAO applied to luma samples and chroma samples and using DBF Y / Cb / Cr as input, according to some implementations of the present disclosure. FIG. 6C is a block diagram illustrating a system and process for a CCSAO that can operate independently, according to some implementations of the present disclosure. 6D is a block diagram illustrating a system and process for CCSAO, which can be applied recursively (twice or N times) with the same or different offsets in the same codec stage or repeated at different stages, according to some implementations of the present disclosure. In summary, in some embodiments, information about the current luma sample and neighboring luma samples, and information about the co-located chroma sample and neighboring chroma samples (Cb and Cr) can be used to classify the current luma sample. In some embodiments, information about the co-located luma sample and neighboring luma samples, the co-located cross-chroma sample and neighboring cross-chroma samples, and the current chroma sample and neighboring chroma samples can be used to classify the current chroma sample (Cb or Cr). In some embodiments, CCSAO can branch in a cascade fashion (1) after DBF Y / Cb / Cr, (2) after the reconstructed image Y / Cb / Cr before DBF, (3) after SAO Y / Cb / Cr, or (4) after ALF Y / Cb / Cr.

[0118] In some embodiments, the CCSAO can be applied in parallel with other encoding tools, such as the ESAO of the AVS standard or the CDEF of the AV1 standard. Figure 6E is a block diagram illustrating a system and process of the CCSAO applied in parallel with the ESAO of the AVS standard according to some implementations of the present disclosure.

[0119] FIG. 6F is a block diagram illustrating a system and process of a CCSAO applied after an SAO according to some implementations of the present disclosure. In some embodiments, FIG. 6F illustrates that the location of the CCSAO can be after the SAO, i.e., the location of a Cross-Component Adaptive Loop Filter (CCALF) in the VVC standard. FIG. 6G is a block diagram illustrating that the system and process of a CCSAO can operate independently without a CCALF according to some implementations of the present disclosure. In some embodiments, the SAO Y / Cb / Cr can be replaced with an ESAO, such as the ESAO in the AVS3 standard.

[0120] FIG. 6H is a block diagram illustrating a system and process for CCSAO applied in parallel with CCALF according to some implementations of the present disclosure. In some embodiments, FIG. 6H illustrates that CCSAO can be applied in parallel with CCALF. In some embodiments, the positions of CCALF and CCSAO can be changed in FIG. 6H. In some embodiments, the SAO Y / Cb / Cr blocks can be replaced with ESAO Y / Cb / Cr (ESAO Y / Cb / Cr in AVS3) or CDEF (CDEF in AV1) in FIGS. 6A through 6H or throughout this disclosure. Note that Y / Cb / Cr can also be represented as Y / U / V in the video coding domain. In some embodiments, even if the video is in RGB format, the present disclosure allows CCSAO to be applied simply by mapping the YUV representation to GBR, respectively.

[0121] In some embodiments, the current chroma sample classification uses again the SAO type (Edge Offset (EO) or BO), class, and category of the luma sample at the same position. The corresponding CCSAO offset can be signaled or derived from the decoder itself. For example, let h_Y be the luma SAO offset at the same position, and let h_Cb and h_Cr be the CCSAO Cb offset and CCSAO Cr offset respectively. Then h_Cb (or h_Cr) = w * h_Y, and w can be selected from a limited table. For example, it can be ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4, etc., and |w| only includes values that are powers of 2.

[0122] In some embodiments, the comparison score [-8, 8] between the luma sample (Y0) at the same position and the eight neighboring luma samples is used, resulting in a total of 17 classes. Initial Class = 0 Apply loop processing to the eight neighboring luma samples (Yi, i = 1~8) If Y0 > Yi, then Class += 1 In cases other than the above, if Y0 < Yi, then Class -= 1

[0123] In some embodiments, the above classification methods can be combined. For example, a comparison score combined with SAO BO (32-band classification) is used to increase diversity, resulting in a total of 17 * 32 classes. In some embodiments, the same class can be used for Cb and Cr to reduce complexity or save bits.

[0124] 7 is a block diagram illustrating a sample process using the CCSAO according to some implementations of the present disclosure. In particular, FIG. 7 illustrates that the input of the CCSAO can introduce vertical and horizontal DBF inputs to simplify class determination and increase flexibility. For example, let Y0_DBF_V, Y0_DBF_H, and Y0 be the co-located luma samples at the inputs of DBF_V, DBF_H, and SAO, respectively. Yi_DBF_V, Yi_DBF_H, and Yi are the eight neighboring luma samples at the inputs of DBF_V, DBF_H, and SAO, respectively, where i=1 to 8. Max Y0=max(Y0_DBF_V,Y0_DBF_H,Y0_DBF) Max Yi=max(Yi_DBF_V,Yi_DBF_H,Yi_DBF) Also, max Y0 and max Yi are given to the CCSAO classification.

[0125] 8 is a block diagram illustrating that the CCSAO process is interleaved with vertical and horizontal DBFs, according to some implementations of the present disclosure. In some embodiments, the CCSAO blocks in FIGS. 6, 7, and 8 can be appropriately selected. For example, the initial CCSAO_V uses Y0_DBF_V and Yi_DBF_V (applying the same sample processing as in FIG. 6), while using the input of DBF_V luma samples as the CCSAO input.

[0126] In some embodiments, the CCSAO syntax implemented is shown in Table 2 below. [Table 2]

[0127] In some embodiments, for signaling the CCSAO Cb offset value and the CCSAO Cr offset value, if one additional chroma offset is signaled, the other chroma component offset can be derived using a plus or minus sign or weighting to save bit overhead. For example, let h_Cb and h_Cr be the offsets of the CCSAO Cb and CCSAO Cr, respectively. With w explicitly stated, h_Cr can be derived from h_Cb without explicitly indicating h_Cr itself, although in this case limited |w| candidates are used and w=±|w|. h_Cr=w*h_Cb

[0128] FIG. 9 is a flowchart illustrating an exemplary process 900 for decoding a video signal using cross-component correlation, according to some implementations of the present disclosure.

[0129] Video decoder 30 receives 910 a video signal including a first component and a second component. In some embodiments, the first component is a luma component of the video signal and the second component is a chroma component of the video signal.

[0130] Video decoder 30 also receives a plurality of offsets associated with the second component (920).

[0131] Video decoder 30 then utilizes the characteristic measurements of the first component to obtain a classification category associated with the second component (930). For example, in Figure 6, the current chroma sample 602 is first classified with the co-located luma sample 604 and the neighboring luma sample 606 (white), and the corresponding CCSAO offset value is added to the current chroma sample.

[0132] Video decoder 30 further selects (940) a first offset from the plurality of offsets of the second component according to the classification category.

[0133] Video decoder 30 further modifies the second component based on the selected first offset (950).

[0134] In some embodiments, obtaining 930 a classification category associated with the second component using the characteristic measurements of the first component includes obtaining a corresponding classification category for a corresponding sample of the second component using each sample of the first component, where each sample of the first component is co-located with the corresponding sample of the second component. For example, the current chroma sample classification reuses the SAO type (EO or BO), class, and category of the co-located luma sample.

[0135] In some embodiments, obtaining a classification category associated with the second component using the characteristic measurements of the first component (930) includes obtaining a corresponding classification category for a corresponding sample of the second component using each sample of the first component, where each sample of the first component is reconstructed before deblocking or after deblocking. In some embodiments, the first component has been deblocked with a deblocking filter (DBF). In some embodiments, the first component has been deblocked with a luma deblocking filter (DBF Y). For example, instead of FIG. 6 or FIG. 7, the CCSAO input can be before DBF Y.

[0136] In some embodiments, the characteristic measure is derived by dividing the range of sample values ​​of the first component into bands and selecting the bands based on the intensity values ​​of the samples of the first component, hi some embodiments, the characteristic measure is derived from a band offset (BO).

[0137] In some embodiments, the characteristic measure is derived based on the direction and strength of edge information of the samples of the first component, hi some embodiments, the characteristic measure is derived from edge offset (EO).

[0138] In some embodiments, modifying (950) the second component includes adding the selected first offset directly to the second component, e.g., a corresponding CCSAO offset value is added to the current chroma component sample.

[0139] In some embodiments, modifying (950) the second component includes matching the selected first offset to a second offset and adding the matched second offset to the second component. For example, for signaling a CCSAO Cb offset value and a CCSAO Cr offset value, if one additional chroma offset is signaled, the other chroma component offsets can be derived using a plus or minus sign or weighting to save bit overhead.

[0140] In some embodiments, receiving 910 the video signal includes receiving a syntax element indicating whether decoding the video signal using CCSAO is enabled for the video signal in the sequential parameter set (SPS). In some embodiments, cc_sao_enabled_flag indicates whether CCSAO is enabled at the sequence level.

[0141] In some embodiments, receiving 910 the video signal includes receiving a syntax element indicating whether a method for decoding the video signal using CCSAO is enabled for the second component at the slice level. In some embodiments, slice_cc_sao_cb_flag or slice_cc_sao_cr_flag indicates whether CCSAO is enabled for Cb or Cr in the respective slice.

[0142] In some embodiments, receiving 920 multiple offsets associated with the second component includes receiving different offsets for different coding tree units (CTUs), where for a CTU, cc_sao_offset_sign_flag indicates the sign of the offset and cc_sao_offset_abs indicates the CCSAO Cb offset value and the CCSAO Cr offset value for the current CTU.

[0143] In some embodiments, receiving 920 multiple offsets associated with the second component includes receiving a syntax element indicating whether the received offset of the CTU is the same as the offset of one of the CTU's neighboring CTUs, where the neighboring CTU is either a left neighboring CTU or an up neighboring CTU. For example, cc_sao_merge_up_flag indicates whether the CCSAO offset is merged from a left CTU or an up CTU.

[0144] In some embodiments, the video signal further includes a third component, and the method of decoding the video signal using the CCSAO further includes receiving a second plurality of offsets associated with the third component, obtaining a second classification category associated with the third component using the characteristic measurement of the first component, selecting a third offset from the second plurality of offsets for the third component according to the second classification category, and modifying the third component based on the selected third offset.

[0145] 11 is a block diagram of a sample process illustrating that all of the co-located luma / chroma samples and neighboring luma / chroma samples (white) can be entered into the CCSAO classification, according to some implementations of the present disclosure. Figures 6A, 6B, and 11 show the input of the CCSAO classification. In Figure 11, the current chroma sample is 1104, the cross-component co-located chroma sample is 1102, and the co-located luma sample is 1106.

[0146] In some embodiments, an example classifier (C0) uses the luma or chroma sample value (Y0) at the same position in Figure 12 (Y4 / U4 / V4 in Figure 6B and Figure 6C) for classification. If band_num is the number of evenly divided bands in the luma or chroma dynamic range, and bit_depth is the bit depth of the sequence, an example class index for the current chroma sample is as follows: Class(C0)=(Y0*band_num)>>bit_depth

[0147] In some embodiments, the classification takes rounding into account, for example: Class(C0)=((Y0*band_num)+(1<<bit_depth))> >bit_depth

[0148] Some examples of band_num and bit_depth are listed below in Table 3. Table 3 shows three classification examples where the number of bands varies for each classification example. [Table 3]

[0149] In some embodiments, the classifier uses a different luma sample location for the C0 classification. Figure 10A is a block diagram illustrating a classifier that uses a different luma (or chroma) sample location for the C0 classification, for example, using neighboring Y7 instead of Y0 for the C0 classification, according to some implementations of this disclosure.

[0150] In some embodiments, different classifiers can be switched at continuous parameter set (SPS) / adaptive parameter set (APS) / picture parameter set (PPS) / picture header (PH) / slice header (SH) / region / coding tree unit (CTU) / coding unit (CU) / sub-block / sample level. For example, in Figure 10, Y0 is used for POC0 while Y7 is used for POC1, as shown in Table 4 below. [Table 4]

[0151] In some embodiments, FIG. 10B shows some examples of different shapes of luma candidates according to some implementations of the present disclosure. For example, shape constraints can be imposed. In some examples, the total number of luma candidates must be a power of two, as shown in FIG. 10B(b), (c), and (d). In some examples, the number of luma candidates must be horizontally and vertically symmetric with respect to the chroma sample (which is in the center), as shown in FIG. 10B(a), (c), (d), and (e). In some embodiments, the power of two constraint and symmetry constraint can also be applied to the chroma candidates. The U / V portions of FIG. 6B and FIG. 6C show examples of symmetry constraints. In some embodiments, different color formats can have different "constraints" on the classifier. For example, in the 420 color format, luma / chroma candidates as shown in Figures 6B and 6C are selected and used (one candidate is selected from a 3x3 shape), while in the 444 color format, Figure 10B(f) is used for the selected luma and chroma candidates, and in the 422 color format, Figure 10B(g) is used for the luma candidates (two chroma samples share four luma candidates) and Figure 10B(f) is used for the chroma candidates.

[0152] In some embodiments, C0 position and C0 band_num can be combined and switched at SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Different combinations can be different classifiers as shown in Table 5 below. [Table 5]

[0153] In some embodiments, the luma sample value (Y0) at the same position is replaced with a value (Yp) obtained by weighting the luma sample at the same position and neighboring luma samples. FIG. 12 shows a typical classifier that uses replacing the luma sample value at the same position with a value obtained by weighting the luma sample at the same position and neighboring luma samples according to some implementations of the present disclosure. The luma sample value (Y0) at the same position can be replaced with a phase correction value (Yp) obtained by weighting neighboring luma samples. Different Yps can be different classifiers.

[0154] In some embodiments, different Yps are applied to different chroma formats. For example, in FIG. 12, the Yp in (a) is used for the 420 chroma format, the Yp in (b) is used for the 422 chroma format, and Y0 is used for the 444 chroma format.

[0155] In some embodiments, another classifier (C1) is the comparison score [-8, 8] between the luma sample (Y0) at the same position and eight neighboring luma samples, whereby a total of 17 classes are obtained as shown below. Initial Class(C1)=0, apply loop processing to eight neighboring luma samples (Yi, i = 1~8) If Y0>Yi, Class += 1 In other cases than the above, if Y0<Yi, Class -= 1

[0156] In some embodiments, an example of C1 is equal to the following function when the threshold th is 0. ClassIdx = Index2ClassTable(f(C, P1)+f(C, P2)+…+f(C, P8)) If x - y>th, then f(x, y)=1; if x - y = th, then f(x, y)=0; if x - y<th, then f(x, y)=-1 Index2ClassTable is a look-up table (LUT), C is the current sample or a co-located sample, and P1 to P8 are neighboring samples.

[0157] In some embodiments, similar to the C4 classifier, one or more thresholds may be predefined (e.g., stored in a LUT) or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level to facilitate classifying (quantizing) the differences.

[0158] In some embodiments, variant (C1') simply counts the comparison scores [0,8], resulting in 8 classes. (C1,C1') is a classifier group, and PH / SH level flags can be signaled to switch between C1 and C1'.

[0159] Initial Class(C1')=0, loop processing is applied to the 8 nearest luma samples (Yi,i=1~8) If Y0>Yi, Class+=1

[0160] In some embodiments, variant (C1s) selectively uses neighboring N of M neighboring samples to count the comparison score. An M-bit bit mask can be signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level to indicate which neighboring samples are selected to count the comparison score. Using FIG. 6B as an example of a luma classifier, i.e., eight neighboring luma samples are candidates, and an 8-bit bit mask (01111110) is signaled at PH to indicate that six samples Y1 to Y6 are selected, and therefore the comparison score is within [-6, 6], resulting in 13 offsets. An appropriate choice for classifier C1s provides the encoder with more options for trading off the overhead of offset signaling and classification granularity.

[0161] Similar to C1s, variant (C1') only counts comparison scores [0,+N], so the bitmask 01111110 example above would result in comparison scores within [0,6], which would give us an offset of 7.

[0162] In some embodiments, different classifiers are combined to obtain a generic classifier, for example, different classifiers are applied to different pictures (different POC values) as shown in Table 6-1 below. [Table 6]

[0163] In some embodiments, another example classifier (C3) uses a bit mask for classification, as shown in Table 6-2. A 10-bit bit mask is signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level to inform the classifier. For example, a bit mask of 11 1100 0000 means that for a given 10-bit luma sample value, only the most significant bits (MSBs), or 4 bits, are used for classification, resulting in a total of 16 classes. Another example bit mask of 10 0100 0001 means that only 3 bits are used for classification, resulting in a total of 8 classes.

[0164] In some embodiments, the bit mask length (N) can be fixed or variable at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, for a 10-bit sequence, a 4-bit bit mask 1110 is signaled in the PH in the picture, and the three most significant bits, b9, b8, and b7, are used for classification. Another example is a 4-bit bit mask 0011 on the least significant bits, with b0 and b1 used for classification. The bit mask classifier can be applied to luma classification or chroma classification. Whether the most significant bit or least significant bit is used for the bit mask N can be fixed or variable at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0165] In some embodiments, the luma position and C3 bitmask can be combined or changed at SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Different combinations can result in different classifiers.

[0166] In some embodiments, a bitmask restriction "max # of 1s" can be applied to limit the corresponding number of offsets. For example, if an SPS restricts the bitmask "max # of 1s" to 4, then the maximum offset in a sequence is 16. The bitmasks for different POCs can be different, but the "max # of 1s" will not exceed 4 (total number of classes will not exceed 16). The value of "max # of 1s" can be signaled or changed at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. [Table 7]

[0167] In some embodiments, as shown in FIG. 11 , other cross-component chroma samples, such as chroma sample 1102 or its neighboring samples, can also be placed into the CCSAO classification (e.g., as is done for the current chroma sample 1104). For example, the Cr chroma sample can be placed into the CCSAO Cb classification. The Cb chroma sample can be placed into the CCSAO Cr classification. The classifier for the cross-component chroma sample can be the same as the luma cross-component classifier or can have its own classifier as described in this disclosure. The two classifiers can be combined to form a joint classifier that classifies the current chroma sample. For example, a joint classifier that combines the cross-component luma sample and the cross-component chroma sample can result in a total of 16 classes, as shown in Table 6-3 below. [Table 8]

[0168] All of the above classifications (C0, C1, C1', C2, C3) can be combined. For example, see Table 6-4 below. [Table 9]

[0169] In some embodiments, an example classifier (C2) uses the difference (Yn) between the co-located luma sample and a neighboring luma sample. Figure 12(c) shows an example of Yn with a dynamic range of [-1024, 1023] when the bit depth is 10. Let C2 band_num be the number of evenly divided bands in the Yn dynamic range. Class(C2)=(Yn+(1<<bit_depth)*band_num)> >(bit_depth+1).

[0170] In some embodiments, C0 and C2 are combined to obtain a generic classifier, for example, different classifiers are applied to different pictures (different POCs) as shown in Table 7 below. [Table 10]

[0171] In some embodiments, all the above classifiers (C0, C1, C1', C2) are combined, for example, different classifiers are applied to different pictures (different POCs) as shown in Table 8-1 below. [Table 11]

[0172] In some embodiments, as shown in Table 8-2 below, an example classifier (C4) uses the difference between the CCSAO input value and the compensated sample value for classification. For example, if CCSAO is applied in the ALF stage, the difference between the pre-ALF sample value and the post-ALF sample value of the current component is used for classification. One or more thresholds can be predefined (e.g., stored in a look-up table (LUT)) or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level to facilitate classifying (quantizing) the difference. The C4 classifier can be combined with C0 Y / U / V bandNum to form a joint classifier (e.g., the example POC1 shown in Table 8-2). [Table 12]

[0173] In some embodiments, because different encoding modes may introduce different distortion statistics into the reconstructed image, an example classifier (C5) uses "encoding information" to facilitate sub-block classification. A CCSAO sample is classified using its previous encoding information, and a classifier can be formed by combining the encoding information, for example, as shown in Table 8-3 below. Figure 30, described below, shows another example of different stages of encoding information for C5. [Table 13]

[0174] In some embodiments, the example classifier (C6) uses the YUV color transform value for classification. For example, to classify the current Y component, select the 1 / 1 / 1 co-located Y / U / V sample or a nearby Y / U / V sample to be color converted to RGB, and use C3 bandNum to quantize the R value to be the current Y component classifier.

[0175] In some embodiments, other classifier examples that use only the current component information for the classification of the current component can be used as the cross-component classifier. For example, as shown in Figure 5 and Table 1, luma sample information and eo-class are used to derive EdgeIdx and classify the current chroma sample. Other "non-cross-component" classifiers that can also be used as cross-component classifiers include edge direction, pixel intensity, pixel change, pixel disparity, pixel sum-of-Laplacian, Sobel operator, compass operator, high-pass filter value, low-pass filter value, etc.

[0176] In some embodiments, multiple classifiers are used in the same POC. The current frame is divided into several regions, and the same classifier is used in each region. For example, three different classifiers are used in POC 0, and which classifier is used (0, 1, or 2) is signaled at the CTU level, as shown in Table 9 below. [Table 14]

[0177] In some embodiments, the maximum number of classifiers (which may also be referred to as alternate offset sets) can be fixed or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. In one example, the fixed (predetermined) maximum number of classifiers is 4. In this case, four different classifiers are used in POC0, and the classifier to be used (0, 1, or 2) is signaled at the CTU level. A truncated-unary (TU) code can be used to indicate the classifier to be used for each luma CTB or each chroma CTB. For example, as shown in Table 10 below, if the TU code is 0, CCSAO is not applied; if the TU code is 10, set 0 is applied; if the TU code is 110, set 1 is applied; if the TU code is 1110, set 2 is applied; and if the TU code is 1111, set 3 is applied. Fixed-length codes, Golomb-Rice codes, and exponential-Golomb codes can also be used to indicate the classifiers (offset set indices) of CTBs. Three different classifiers are used in POC1. [Table 15]

[0178] Examples of Cb CTB offset set indexes and Cr CTB offset set indexes for a 1280x720 sequence POC0 are described below (when the CTU size is 128x128, the number of CTUs in a frame is 10x6). POC0 Cb uses four offset sets, and POC0 Cr uses one offset set. As shown in Table 11-1 below, if the offset set index is 0, CCSAO is not applied; if the offset set index is 1, set 0 is applied; if the offset set index is 2, set 1 is applied; if the offset set index is 3, set 2 is applied; and if the offset set index is 4, set 3 is applied. The type indicates the position of the selected co-located luma sample (Yi). Different offset sets can have different types, band_nums, and corresponding offsets. [Table 16]

[0179] In some embodiments, an example of using the co-located Y / U / V sample / current Y / U / V sample and neighboring Y / U / V samples together for classification is shown in Table 11-2 below (three-component collaborative bandNum classification for each Y / U / V component). In POC0, an offset set of {2, 4, 1} is used for {Y, U, V}, respectively. Each offset set can be changed adaptively at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Different offset sets can have different classifiers. For example, as indicated by the candidate position (candPos) in Figures 6B and 6C, to classify the current Y4 luma sample, Y set 0 selects {current Y4, co-located U4, co-located V4} as candidates using different bandNum{Y,U,V}={16, 1, 2}, respectively. Using {candY,candU,candV} as the sample values ​​of the selected {Y,U,V} candidates, the total number of classes is 32, and the derivation of the class index can be shown as follows: bandY=(candY*bandNumY)>>BitDepth, bandU=(candU*bandNumU)>>BitDepth, bandV=(candV*bandNumV)>>BitDepth, classIdx=bandY*bandNumU*bandNumV +bandU*bandNumV +bandV

[0180] In some embodiments, the derivation of classIdx for the collaborative classifiers can be expressed as an "or-shift" format to simplify the derivation process. For example, max bandNum={16,4,4} classIdx=(bandY<<4)|(bandU<<2)|bandV

[0181] Another example is in POC1 component V set 1 classification. In this example, we use candPos={neighbor Y8, neighbor U3, neighbor V0} with bandNum={4,1,2}, which gives us 8 classes. [Table 17]

[0182] In some embodiments, an example of using the co-located Y / U / V sample and the neighboring Y / U / V sample together for the current Y / U / V sample classification is described, for example, as shown in Table 11-3 below (three-component joint edgeNum(C1s) and bandNum classification for each Y / U / V component). Edge CandPos is the center position used for the C1s classifier, the edge bit mask is the C1s neighboring sample activation indicator, and edgeNum is the corresponding number of C1s classes. In this example, C1s is only applied to the Y classifier (hence edgeNum is equal to edgeNumY), and edge candPos is always Y4 (current sample position / co-located sample position). On the other hand, when edge candPos is used as the neighboring sample position, C1s can be applied to the Y / U / V classifier.

[0183] If we denote the comparison score of Y C1s by diff, the derivation of classIdx can be as follows: bandY=(candY*bandNumY)>>BitDepth, bandU=(candU*bandNumU)>>BitDepth, bandV=(candV*bandNumV)>>BitDepth, edgeIdx=diff+(edgeNum>>1), bandIdx=bandY*bandNumU*bandNumV +bandU*bandNumV +bandV classIdx=bandIdx*edgeNum+edgeIdx [Table 18] [Table 19] [Table 20]

[0184] In some embodiments, max band_num (bandNumY, bandNumU, or bandNumV) can be constant or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, if max band_num=16 is constant per frame at the decoder, then 4 bits are signaled to indicate C0 band_num in the frame. Other max band_num examples are listed below in Table 12. [Table 21]

[0185] In some embodiments, the maximum number of classes or offsets (combination of multiple classifiers used together, e.g., C1s edgeNum*C1bandNumY*bandNumU*bandNumV) for each set (or all added sets) can be made constant or signaled at SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, the maximum is made constant (class_num=256*4) for all added sets, and encoder conformance check or decoder criteria conformance check can be used to check the constraint.

[0186] In some embodiments, restrictions can be applied to the C0 classification, for example restricting band_num (bandNumY, bandNumU or bandNumV) to only be values ​​that are a power of 2. Instead of signaling band_num explicitly, the syntax band_num_shift is signaled. The decoder can use shift operations to avoid multiplications. Different band_num_shift can be used for different components. Class(C0)=(Y0>>band_num_shift)>>bit_depth

[0187] Another example of an operation is one that takes into account rounding to reduce error. Class(C0)=((Y0+(1<<(band_num_shift-1)))>>band_num_shift)>>bit_depth

[0188] For example, if band_num_max (Y, U or V) is 16, then the possible band_num_shift candidates are 0, 1, 2, 3, 4 corresponding to band_num=1, 2, 4, 8, 16, as shown in Table 13. [Table 22]

[0189] In some embodiments, the classifier applied to Cb is different from the classifier applied to Cr. The Cb and Cr offsets for all classes can be signaled separately. For example, different signaled offsets are applied to different chroma components, as shown in Table 14 below. [Table 23]

[0190] In some embodiments, the maximum offset value is fixed or signaled at the continuous parameter set (SPS) / adaptive parameter set (APS) / picture parameter set (PPS) / picture header (PH) / slice header (SH) / region / CTU / CU / sub-block / sample level. For example, the maximum offset is between [-15, 15]. Different components can have different maximum offset values.

[0191] In some embodiments, differential pulse code modulation (DPCM) can be used to signal the offset, for example, the offset {3,3,2,1,-1} can be signaled as {3,0,-1,-1,-2}.

[0192] In some embodiments, the offsets can be stored in an APS or memory buffer for reuse in the next picture / slice. An index can be signaled to indicate which of the stored previous frame offsets is to be used for the current picture.

[0193] In some embodiments, the classifiers for Cb and Cr are the same. The Cb and Cr offsets for all classes can be signaled together, for example, as shown in Table 15 below. [Table 24]

[0194] In some embodiments, the classifiers for Cb and Cr can be the same. Different sign flags can be used to signal the Cb and Cr offsets of all classes together, for example, as shown in Table 16 below. According to Table 16, if the Cb offset is (3,3,2,-1), the derived Cr offset is (-3,-3,-2,1). [Table 25]

[0195] In some embodiments, the sign flag can be signaled for each class, for example, as shown in Table 17 below: According to Table 17, if the Cb offset is (3,3,2,-1), the derived Cr offset according to the flag containing the respective sign is (-3,3,2,1). [Table 26]

[0196] In some embodiments, the classifiers for Cb and Cr can be the same. Different weights can be used to signal the Cb and Cr offsets of all classes together, for example, as shown in Table 18 below. The weights (w) can be selected from a limited table, for example, ±¼, ±½, 0, ±1, ±2, ±4..., and |w| contains only values ​​that are powers of 2. According to Table 18, if the Cb offset is (3, 3, 2, -1), the derived Cr offset is (-6, -6, -4, 2) according to the flags containing the respective signs. [Table 27]

[0197] In some embodiments, weights can be signaled for each class, for example as shown in Table 19 below: According to Table 19, if the Cb offset is (3,3,2,-1), the Cr offset derived according to the flags with their respective signs is (-6,12,0,-1). [Table 28]

[0198] In some embodiments, when multiple classifiers are used in the same POC, different sets of offsets are signaled individually or together.

[0199] In some embodiments, previously decoded offsets can be stored for use in future frames. An index can be signaled to indicate which of the previously decoded offset sets is used for the current frame, reducing the overhead of offset signaling. For example, the offsets of POC0 can be reused by POC2 by signaling offset set idx=0 as shown in Table 20. [Table 29]

[0200] In some embodiments, the reuse offset set idx for Cb and the reuse offset set idx for Cr can be different, for example as shown in Table 21 below. [Table 30]

[0201] In some embodiments, signaling the offset can use additional syntax including start and length to reduce signaling overhead. For example, if band_num=256, only offsets for band_idx=37-44 are signaled. In the following example in Table 22-1, both the start and length syntaxes are coded with a fixed length of 8 bits, which naturally matches the band_num bits. [Table 31]

[0202] In some embodiments, if CCSAO is applied to all YUV triplets, co-located and neighboring YUV samples can be used together for classification, and all of the Cb / Cr offset signaling methods described above can be extended to Y / Cb / Cr. In some embodiments, different component offset sets can be stored and used separately (each component has its own stored set) or together (each component shares / reuses the same memory). Examples of separate sets are shown in Table 22-2 below. [Table 32]

[0203] In some embodiments, if the sequence bit depth is greater than 10 (or a specified bit depth), the offset may be quantized before signaling. At the decoder side, the decoded offset is dequantized before being applied, as shown in Table 23 below. For example, for a 12-bit sequence, the decoded offset is shifted left (dequantized) by 2. [Table 33]

[0204] In some embodiments, the offset can be calculated as CcSaoOffsetVal=(1-2*ccsao_offset_sign_flag)*(ccsao_offset_abs<<(BitDepth-Min(10,BitDepth))).

[0205] In some embodiments, the description further introduces the concept of filter strength. For example, further weighting can be performed before applying the classifier offset to the samples. The weight (w) can be selected from a table of power-of-two values, such as ±¼, ±½, 0, ±1, ±2, ±4..., where |w| includes only power-of-two values. The weight index can be signaled at the SPS / APS / PPS / PH / SH / Region (Set) / CTU / CU / Sub-block / Sample level. The signaling of the quantized offset can be considered as a subset of such weight application. When a recursive CCSAO is applied as shown in FIG. 6D, a similar weight index mechanism can be applied between the first and second stages.

[0206] In some instances, weightings for different classifiers, i.e., offsets for multiple classifiers, can be combined and applied to the same sample. A similar weight index mechanism can be signaled as above. For example, offset_final=w*offset_1+(1-w)*offset_2 or offset_final=w1*offset_1+w2*offset_2+…

[0207] In some embodiments, the sample processing is as described below: Let R(x,y) be the input luma or chroma sample value before CCSAO, and let R'(x,y) be the output luma or chroma sample value after CCSAO. offset=ccsao_offset[class_indexofR(x,y)] R'(x,y)=Clip3(0,(1< <bit_depth)-1,R(x,y)+offset)

[0208] According to the above formula, each luma or chroma sample value R(x,y) is classified using the notified classifier of the current picture and / or the current offset set idx. The corresponding offset of the derived class index is added to each luma or chroma sample value R(x,y). The clip function Clip3 is applied to (R(x,y)+offset) to put the output luma or chroma sample value R’(x,y) into the bit-depth dynamic range (e.g., range 0~(1<<bit_depth)-1).

[0209] In some embodiments, the boundary processing is as described below. If either the luma (chroma) sample at the same position used for classification or the neighboring luma (chroma) sample is outside the current picture, CCSAO is not applied to the current chroma (luma) sample. FIG. 13A is a block diagram showing that CCSAO is not applied to the current chroma (luma) sample when either the luma (chroma) sample at the same position used for classification or the neighboring luma (chroma) sample is outside the current picture according to some implementation examples of the present disclosure. For example, in FIG. 13A(a), when a classifier is used, CCSAO is not applied to the chroma components in the leftmost column of the current picture. For example, when C1’ is used, as shown in FIG. 13A(b), CCSAO is not applied to the chroma components in the leftmost column and the topmost row of the current picture.

[0210] 13B is a block diagram illustrating the application of CCSAO to a current luma or chroma sample when either the co-located luma or chroma sample used for classification and a neighboring luma or chroma sample are outside the current picture, according to some implementations of the present disclosure. In some embodiments, when either the co-located luma or chroma sample used for classification and a neighboring luma or chroma sample are outside the current picture, a variation is to repeat the processing leakage sample as shown in FIG. 13B(a) or perform mirror padding on the processing leakage sample to create a sample for classification as shown in FIG. 13B(b), and then apply CCSAO to the current luma or chroma sample. In some embodiments, if any of the co-located luma (chroma) samples used for classification and neighboring luma (chroma) samples are outside the current subpicture / slice / tile / patch / CTU / 360 virtual boundary, the invalidation / repetition / mirror picture boundary processing methods disclosed in this application can also be applied to the subpicture / slice / tile / CTU / 360 virtual boundary.

[0211] For example, a picture is divided into one or more tile rows and one or more tile columns, where a tile is a set of CTUs that cover a rectangular area of ​​the picture.

[0212] A slice consists of an integer number of complete tiles, or an integer number of consecutive complete CTU rows within a tile of a picture.

[0213] A subpicture contains one or more slices that together cover a rectangular area of ​​the picture.

[0214] In some embodiments, 360-degree video is captured on a sphere and is inherently "boundary-free," meaning that reference samples outside the boundaries of a reference picture in the projection domain can always be obtained from neighboring samples in the spherical domain. In a multi-face projection format, discontinuities will appear between two or more adjacent faces in a frame-packed picture, regardless of the type of compact frame-packing configuration used. VVC introduces vertical and / or horizontal virtual boundaries where in-loop filtering operations are disabled, and the location of these boundaries is signaled in either the SPS or the picture header. Compared to using two tiles, one for each set of contiguous faces, the use of 360 virtual boundaries is more flexible because the face size does not need to be a multiple of the CTU size. In some embodiments, the maximum number of vertical 360 virtual boundaries is three, and the maximum number of horizontal 360 virtual boundaries is also three. In some embodiments, the distance between two virtual boundaries is equal to or greater than the CTU size, and the granularity of the virtual boundaries is 8 luma samples (e.g., an 8 × 8 sample grid).

[0215] FIG. 14 is a block diagram illustrating that, according to some implementations of the present disclosure, if a selected co-located luma sample or a neighboring luma sample corresponding to a current chroma sample used for classification falls outside the virtual space defined by the virtual boundary, the CCSAO is not applied to the current chroma sample. In some embodiments, a virtual boundary (VB) is an imaginary line separating spaces within a picture frame. In some embodiments, when a virtual boundary (VB) is applied to the current frame, the CCSAO is not applied to chroma samples at selected corresponding luma positions that fall outside the virtual space defined by the virtual boundary. FIG. 14 illustrates an example in which a virtual boundary is used in a C0 classifier with nine candidate luma positions. For each CTU, the CCSAO is not applied to chroma samples whose corresponding selected luma positions fall outside the virtual space enclosed by the virtual boundary. For example, in Figure 14(a), if the Y7 luma sample position selected for chroma sample 1402 is on the other side of horizontal virtual boundary 1406 that is four pixel lines from the bottom of the frame, then no CCSAO is applied to chroma sample 1402. For example, in Figure 14(b), if the Y5 luma sample position selected for chroma sample 1404 is on the other side of vertical virtual boundary 1408 that is y pixel lines from the right side of the frame, then no CCSAO is applied to chroma sample 1404.

[0216] Figure 15 illustrates that repetition padding or mirror padding can be applied to luma samples outside the virtual boundary, according to some implementations of the present disclosure. Figure 15(a) illustrates an example of repetition padding. When the original Y7 is selected to be the classifier located at the bottom of VB1502, the Y4 luma sample value is used for classification (copied to the Y7 position) instead of the original Y7 luma sample value. Figure 15(b) illustrates an example of mirror padding. When Y7 is selected to be the classifier located at the bottom of VB1504, the Y1 luma sample value, which is symmetrical to the Y7 value with respect to the Y0 luma sample, is used for classification instead of the original Y7 luma sample value. This padding method allows more chroma samples to have the possibility of applying CCSAO, thereby achieving a high coding gain.

[0217] In some embodiments, restrictions can be applied to reduce the line buffer required for CCSAO and simplify the boundary processing state check. Figure 16 shows that according to some implementations of the present disclosure, if all nine co-located luma samples and neighboring luma samples are used for classification, one more luma line buffer (i.e., all line luma samples in line -5 above the current VB1602) may be required. Figure 10B(a) shows an example where only six luma candidates are used for classification, in which case the line buffer is reduced and none of the additional boundary checks of Figures 13A and 13B are required.

[0218] In some embodiments, using luma samples for CCSAO classification may increase the luma line buffer, thus increasing the implementation cost of the decoder hardware. Figure 17 shows an AVS diagram in which, according to some implementations of the present disclosure, the CCSAO of nine luma candidates crosses VB 1702, which may increase two additional luma line buffers. For luma and chroma samples above the virtual boundary (VB) 1702, DBF / SAO / ALF is processed in the current CTU row. For luma and chroma samples below VB 1702, DBF / SAO / ALF is processed in the next CTU row. In the decoder hardware design of AVS, the pre-DBF samples of luma lines -1 to -4, the pre-SAO sample of line -5, the pre-DBF samples of chroma lines -3 to -1, and the pre-SAO sample of line -4 are stored as line buffers in preparation for DBF / SAO / ALF processing of the next CTU row. When processing the next CTU row, luma samples and chroma samples that are not in the line buffer cannot be used. However, for example, at chroma line -3(b) position, chroma samples are processed in the next CTU row, while SAO pre-luma sample lines -7, -6, and -5 are required for classification in CCSAO. SAO pre-luma sample lines -7 and -6 are not in the line buffer and therefore cannot be used. Also, adding SAO pre-luma sample lines -7 and -6 to the line buffer increases the implementation cost of decoder hardware. In some examples, luma VB (line -4) and chroma VB (line -3) may differ (not be aligned).

[0219] 17, FIG. 18A shows a VVC diagram in which the CCSAO of nine luma candidates across VB 1802 may add one more luma line buffer, according to some implementations of this disclosure. Different standards may have different VBs. In VVC, the CCSAO of nine candidates may add one more luma line buffer, since the luma VB is line -4 and the chroma VB is line -2.

[0220] In some embodiments, in a first solution, if any of the luma candidates for a chroma sample exceeds VB (outside the current chroma sample VB), the CCSAO is disabled for that chroma sample. Figures 19A-19C show that, according to some implementations of the present disclosure, in AVS and VVC, if any of the luma candidates for that chroma sample exceeds VB1902 (outside the current chroma sample VB), the CCSAO is disabled for that chroma sample. Figure 14 also shows some examples of this implementation.

[0221] In some embodiments, in a second solution, for "cross VB" luma candidates, repeat padding is used from a luma line that is near but on the other side of VB, e.g., luma line -4, to the CCSAO. In some embodiments, repeat padding from the luma closest to the neighbor below VB is performed on the "cross VB" chroma candidate. Figures 20A-20C show that, according to some implementations of the present disclosure, in AVS and VVC, CCSAO is enabled for a chroma sample using repeat padding if any of the luma candidates for the chroma sample is beyond VB2002 (outside the current chroma sample VB). Figure 14(a) also shows some examples of this implementation.

[0222] In some embodiments, the third solution uses mirror padding for CCSAO below luma VB for "cross-VB" luma candidates. Figures 21A-21C show that, in AVS and VVC, CCSAO is enabled for a chroma sample using mirror padding if any of the luma candidates for the chroma sample exceeds VB2 102 (outside the current chroma sample VB), according to some implementations of the present disclosure. Figures 14(b) and 13B(b) also show some examples of this implementation. In some embodiments, the fourth solution uses "double-sided symmetric padding" to apply the CCSAO. Figures 22A-22B show some examples of CCSAO enabled using double-sided symmetric padding for different CCSAO shapes (e.g., 9 luma candidates (Figure 22A) and 8 luma candidates (Figure 22B)) according to some implementations of the present disclosure. For a luma sample set that includes a central luma sample at the same position as the chroma samples, if one side of the luma sample set is outside VB2 202, bilateral symmetric padding is applied to both sides of the luma sample set. For example, in Figure 22A, luma samples Y0, Y1, and Y2 are outside VB2 202, so both Y0, Y1, Y2 and Y6, Y7, Y8 are padded with Y3, Y4, and Y5. For example, in Figure 22B, luma sample Y0 is outside VB2 202, so Y0 is padded with Y2 and Y7 is padded with Y5.

[0223] 18B illustrates that, according to some implementations of the present disclosure, when co-located or neighboring chroma samples are used to classify the current luma sample, the selected chroma candidate may exceed VB, and an additional chroma line buffer may be required. Similar solutions 1-4 described above can be used to address the problem.

[0224] Solution 1 is to disable CCSAO for a luma sample if any of the chroma candidates for the luma sample can be on the other side beyond VB.

[0225] Solution 2 is to use repeated padding for "cross-VB" chroma candidates from the chroma closest to the neighbor below VB.

[0226] Solution 3 uses mirror padding below chroma VB for "cross-VB" chroma candidates.

[0227] Solution 4 uses "double-sided symmetric padding." For a candidate set in the center of co-located chroma samples in CCSAO, if one side of the candidate set is outside the VB, double-sided symmetric padding is applied to both sides.

[0228] The padding method allows more luma or chroma samples to have the possibility of applying CCSAO, thereby achieving a high coding gain.

[0229] In some embodiments, the above special processing (solutions 1, 2, 3, and 4) is not applied to the CTU row at the bottom picture (or slice, tile, or brick) boundary because the samples below VB are processed in the current CTU row. For example, a 1920x1080 frame is divided into 128x128 CTUs. The frame contains 15x9 CTUs (rounded up). The bottom CTU row is the 15th CTU row. The decoding process is a CTU-row-by-CTU process for each CTU row. Deblocking needs to be applied along the horizontal CTU boundary between the current CTU row and the next CTU row. In the bottom 4 / 2 luma / chroma lines within a CTU, the DBF samples (in the case of VVC) are processed in the next CTU row and are not available for CCSAO in the current CTU row, so CTB VB is applied to each CTU row. Meanwhile, in the bottom CTU row of the picture frame, the DBF samples of the bottom 4 / 2 luma / chroma lines are available in the current CTU row because there are no remaining next CTU rows and they are DBF processed in the current CTU row.

[0230] In some embodiments, the VBs shown in Figures 13-22 may be replaced with subpicture / slice / tile / patch / CTU / 360 virtual boundary boundaries. In some embodiments, the positions of the chroma and luma samples in Figures 13-22 may be changed. In some embodiments, the positions of the chroma and luma samples in Figures 13-22 may be replaced with first and second chroma sample positions. In some embodiments, the ALF VBs inside the CTU may be generally horizontal. In some embodiments, the subpicture / slice / tile / patch / CTU / 360 virtual boundary boundaries may be horizontal or vertical.

[0231] In some embodiments, restrictions can be applied to reduce the line buffer required for CCSAO and simplify the boundary processing state check, as described in Figure 16. Figure 23 illustrates restrictions for using a limited number of luma candidates for classification according to some implementations of the present disclosure. Figure 23(a) illustrates a restriction for using only six luma candidates for classification. Figure 23(b) illustrates a restriction for using only four luma candidates for classification.

[0232] In some embodiments, the applied region is implemented. The unit of the CCSAO applied region can be with CTB, i.e., the on / off control, CCSAO parameters (classification, offset used for offset set index, luma candidate position, band_num, bitmask, etc.) are the same within one CTB.

[0233] In some embodiments, the applied region may not be aligned with the CTB boundary. For example, the applied region may not be aligned with the chroma CTB boundary, but may be offset. Although syntax (on / off control, CCSAO parameters) is also signaled per CTB, the applied region may not be aligned with the CTB boundary. Figure 24 illustrates that, according to some implementations of the present disclosure, the CCSAO applied region may not be aligned with the CTB / CTU boundary 2406. For example, the applied region may not be aligned with the chroma CTB / CTU boundary 2406, but the (4,4) sample offset to the upper left aligns with the VB 2408. This misaligned CTB boundary design is useful for the deblocking process because the same deblocking parameters are used for each 8x8 deblocking process region.

[0234] In some embodiments, the unit of the CCSAO application region (mask size) can be in various units (larger or smaller than the CTB size), as shown in Table 24. The mask size can be different for different components. The mask size can be changed at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, a series of mask on / off flags and offset set indexes are signaled in the PH to indicate each CCSAO region information. [Table 34]

[0235] In some embodiments, the division of a frame in a CCSAO application region can be left unchanged, for example, by dividing the frame into N regions. Figure 25 illustrates that the division of a frame in a CCSAO application region can be left unchanged using CCSAO parameters, according to some implementations of the present disclosure.

[0236] In some embodiments, each region may have its own region on / off control flag and CCSAO parameters. Also, if the region size is larger than the CTB size, there may be a CTB on / off control flag and a region on / off control flag. Figures 25(a) and 25(b) show some examples of dividing a frame into N regions. Figure 25(a) shows a vertical division of four regions. Figure 25(b) shows a square division of four regions. In some embodiments, if the region on / off control flag is off, the CTB on / off flag can be further signaled, similar to the picture-level CTBs (ph_cc_sao_cb_ctb_control_flag / ph_cc_sao_cr_ctb_control_flag) all of which have their control flags on. Otherwise, the CCSAO is applied to all CTBs in this region without further signaling of the CTB flag.

[0237] In some embodiments, different regions with CCSAO applied can share the same region on / off control and CCSAO parameters. For example, in Figure 25(c), regions 0-2 share the same parameters, and regions 3-15 share the same parameters. Figure 25(c) also shows that region on / off control flags and CCSAO parameters can be signaled in Hilbert scan order.

[0238] In some embodiments, the units of the CCSAO application area can be quadtree / binary / ternary tree partitioned from the picture / slice / CTB level. Similar to CTB partitioning, a series of partition flags are signaled to indicate the partitioning of the CCSAO application area. Figure 26 shows that according to some implementations of the present disclosure, the CCSAO application area can be binary tree (BT) partitioned / quadtree (QT) partitioned / ternary tree (TT) partitioned from the frame / slice / CTB level.

[0239] Figure 27 is a block diagram illustrating multiple classifiers used within a picture frame and modified at different levels, according to some implementations of the present disclosure. In some embodiments, when multiple classifiers are used in a frame, the method of applying the classifier set index can be modified at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, as shown in Table 25 below, four sets of classifiers are used in the frame and modified at the PH. Figures 27(a) and (c) show the region classifiers with no default modifications. Figure 27(b) shows the classifier set index signaled at the mask / CTB level, where 0 indicates that the CCSAO is off for this CTB and 1-4 indicate the set index. [Table 35]

[0240] In some embodiments, in the case of a default region, the region level flag can be signaled when the CTB for this region does not use the default set index (e.g., the region level flag is 0) and another classifier set is used in this frame. For example, if the default set index is used, the region level flag is 1. For example, for a square division of four regions, the following classifier sets are used as shown in Table 26 below: [Table 36]

[0241] Figure 28 is a block diagram illustrating that, according to some implementations of the present disclosure, the division of the CCSAO application region is dynamic and can be changed at the picture level. For example, Figure 28(a) shows that three CCSAO offset sets are used in this POC (set_num=3), so the picture frame is divided vertically into three regions. Figure 28(b) shows that four CCSAO offset sets are used in this POC (set_num=4), so the picture frame is divided horizontally into four regions. Figure 28(c) shows that three CCSAO offset sets are used in this POC (set_num=3), so the picture frame is divided raster-wise into three regions. Each region has its own region-specific flag that is on for all regions, saving on / off control bits in each CTB. The number of regions depends on the signaled picture set_num.

[0242] The CCSAO application area can be a specific area according to the coding information inside the block (sample position, sample coding mode, loop filter parameters, etc.). For example, 1) the CCSAO application area can be applied only if the sample is coded in skip mode, or 2) the CCSAO application area includes only N samples along the CTU boundary, or 3) the CCSAO application area includes only samples on an 8x8 grid within the frame, or 4) the CCSAO application area includes only samples filtered by DBF, or 5) the CCSAO application area includes only M rows above and N rows to the left of the CU. Different classifiers can be used in different application areas. For example, in a CTU, C1 is used in skip mode, C2 is used on the 8x8 grid, and C3 is used in skip mode and the 8x8 grid. For example, in a CTU, C1 is used for samples coded in skip mode, C2 is used for samples in the CU center, and C3 is used for samples coded in skip mode in the CU center. Figure 29 illustrates that, according to some implementations of the present disclosure, the CCSAO classifier can take into account current coding information or cross-component coding information. For example, different coding modes / parameters / sample positions can form different classifiers. Different coding information can be combined to form a joint classifier. Different classifiers can be used in different regions. Figure 29 also illustrates another example of applied regions.

[0243] In some embodiments, the implemented CCSAO syntax is shown in Table 27 below. In some instances, the binarization of each syntax element can be changed. In AVS3, the terms patch is similar to slice, and patch header is similar to slice header. FLC stands for fixed-length code. TU stands for truncated-unary code. EGk stands for exponential-golomb code with order k, where k can be constant. SVLC stands for signed EG0. UVLC stands for unsigned EG0. [Table 37] JPEG0007789083000038.jpg236170

[0244] If the higher-level flags are off, the lower-level flags can be inferred from their off states and there is no need to signal the lower-level flags. For example, if ph_cc_sao_cb_flag is false for this picture, then ph_cc_sao_cb_band_num_minus1, ph_cc_sao_cb_luma_type, cc_sao_cb_offset_sign_flag, cc_sao_cb_offset_abs, ctb_cc_sao_cb_flag, cc_sao_cb_merge_left_flag, and cc_sao_cb_merge_up_flag are not present and are inferred to be false.

[0245] In some embodiments, the SPS ccsao_enabled_flag is conditioned on the SPS SAO enabled flag, as shown in Table 28. [Table 38]

[0246] In some embodiments, ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_control_flag indicate whether to enable Cb / Cr CTB on / off control granularity. If ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_control_flag are enabled, ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag can be further signaled. Otherwise, whether CCSAO is applied to the current picture depends on ph_cc_sao_cb_flag and ph_cc_sao_cr_flag, and ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag are not further signaled at the CTB level.

[0247] In some embodiments, for ph_cc_sao_cb_type and ph_cc_sao_cr_type, a flag can be further signaled to identify whether the central co-located luma position is used for chroma sample classification (position Y0 in FIG. 10) or not to save bit overhead. Similarly, when cc_sao_cb_type and cc_sao_cr_type are signaled at the CTB level, a flag can be further signaled using the same mechanism. For example, if the number of C0 luma position candidates is 9, cc_sao_cb_type0_flag is further signaled to identify whether the central co-located luma position is used or not, as shown in Table 29 below. If the central co-located luma position is not used, cc_sao_cb_type_idc is used to indicate which of the remaining eight neighboring luma positions is used. [Table 39]

[0248] Table 30 below shows examples of AVS where one classifier (set_num=1) or multiple classifiers (set_num>1) are used in a frame. Note that the syntax notation can correspond to the notation used above. [Table 40]

[0249] When combined with FIG. 25 or FIG. 27, where each region has its own set, an example syntax can include a region on / off control flag (picture_ccsao_lcu_control_flag[compIdx][setIdx]), as shown in Table 31 below. [Table 41]

[0250] In some embodiments, for high level syntax, pps_ccsao_info_in_ph_flag and gci_no_sao_constraint_flag may be added.

[0251] In some embodiments, pps_ccsao_info_in_ph_flag equal to 1 indicates that CCSAO filter information may be present in the PH syntax structure, but may not be present in slice headers pointing to PPSs that do not contain PH syntax structures. pps_ccsao_info_in_ph_flag equal to 0 indicates that CCSAO filter information may not be present in the PH syntax structure, but may be present in slice headers pointing to PPSs. If not present, the value of pps_ccsao_info_in_ph_flag is inferred to be equal to 0.

[0252] In some embodiments, gci_no_ccsao_constraint_flag equal to 1 indicates that sps_ccsao_enabled_flag is equal to 0 for all pictures in OlsInScope. gci_no_ccsao_constraint_flag equal to 0 does not impose such a constraint. In some embodiments, a video bitstream comprises one or more output layer sets (OLSs) that comply with the rules. In the examples of this description, OlsInScope points to one or more OLSs that are in scope. In some examples, the profile_tier_level() syntax structure provides level information, as appropriate, providing profile, layer, subprofile, and / or general constraint information that OlsInScope follows. If the profile_tier_level() syntax structure is included in a VPS, OlsInScope is one or more OLSs specified by the VPS. If the profile_tier_level() syntax structure is included in an SPS, OlsInScope is an OLS that includes only the lowest layer of the layers that point to the SPS, where this lowest layer is an independent layer.

[0253] In some embodiments, extensions to intra- and inter-prediction post-SAO filters are further illustrated below. In some embodiments, the SAO classification methods disclosed in this disclosure can be used as post-prediction filters, where the prediction can be an intra-prediction tool such as intra-block copy, an inter-prediction tool, or other prediction tool. Figure 30 is a block diagram illustrating the SAO classification methods disclosed in this disclosure used as post-prediction filters, according to some implementations of the present disclosure.

[0254] In some embodiments, a corresponding classifier is selected for each of the Y, U, and V components. Furthermore, each component prediction sample is first classified and then a corresponding offset is added. For example, for each component, the current sample and neighboring samples can be used for classification. As shown in Table 32 below, for Y, the current Y sample and neighboring Y samples are used for classification, and for U / V, the current U / V sample is used for classification. Figure 31 is a block diagram illustrating that for a post-prediction SAO filter, the current sample and neighboring samples can be used for classification for each component, according to some implementations of the present disclosure. [Table 42]

[0255] In some embodiments, the refined prediction samples (Ypred', Upred', Vpred') are updated by adding the corresponding class offsets and used for subsequent intra-, inter-, or other predictions.

[0256] Ypred'=clip3(0,(1< <bit_depth)-1,Ypred+h_Y[i])

[0257] Upred'=clip3(0,(1< <bit_depth)-1,Upred+h_U[i])

[0258] Vpred'=clip3(0,(1< <bit_depth)-1,Vpred+h_V[i])

[0259] In some embodiments, for the chroma U and V components, in addition to the current chroma component, the cross component (Y) can be used for further offset classification. An additional cross component offset (h'_U, h'_V) can be added to the current component offset (h_U, h_V), for example, as shown in Table 33 below. [Table 43]

[0260] In some embodiments, the refined prediction samples (Upred'', Vpred'') are updated by adding the corresponding class offsets and used for subsequent intra-, inter- or other-prediction.

[0261] Upred''=clip3(0,(1< <bit_depth)-1,Upred’+h’_U[i])

[0262] Vpred''=clip3(0,(1< <bit_depth)-1,Vpred’+h’_V[i])

[0263] In some embodiments, different SAO filter offsets can be used for intra-prediction and inter-prediction.

[0264] FIG. 32 is a flowchart illustrating an exemplary process 3200 for decoding a video signal using cross-component correlation according to some implementations of the present disclosure.

[0265] Video decoder 30 (such as the video decoder shown in FIG. 3) receives (3210) a first syntax element associated with a first level of the hierarchy from a hierarchical video bitstream.

[0266] In accordance with determining that the first syntax element indicates that cross-component sample adaptive offset (CCSAO) filter information is present at the first level, video decoder 30 jointly reconstructs (3220) one or more regions below the first level from the video bitstream in accordance with the CCSAO filter information.

[0267] In accordance with determining that the first syntax element indicates that CCSAO filter information is not present at the first level, video decoder 30 reconstructs (3230) one or more regions from the video bitstream individually in accordance with CCSAO filter information present at a second level of the hierarchy.

[0268] In some embodiments, the video bitstream comprises a first component and a second component. In some embodiments, reconstructing one or more regions from the video bitstream according to the CCSAO filter information includes, in response to an applied CCSAO filter, video decoder 30 determining a classifier for the second component from a set of one or more samples of the first component associated with each sample of the second component according to the CCSAO filter information, video decoder 30 determining whether to modify values ​​of each sample of the second component in regions of the one or more regions of the video bitstream according to the classifier, video decoder 30 determining sample offsets for each sample of the second component according to the classifier in response to determining to modify values ​​of each sample of the second component in the region according to the classifier, and video decoder 30 modifying the values ​​of each sample of the second component based on the determined sample offsets.

[0269] For example, pps_ccsao_info_in_ph_flag equal to 1 indicates that the CCSAO filter information may be present in the PH syntax structure but may not be present in slice headers pointing to PPSs that do not contain a PH syntax structure. pps_ccsao_info_in_ph_flag equal to 0 indicates that the CCSAO filter information may not be present in the PH syntax structure but may be present in slice headers pointing to PPSs.

[0270] In some embodiments, reconstructing (3230) one or more regions from the video bitstream individually according to the CCSAO filter information present at the second level of the hierarchy includes receiving from the video bitstream a second syntax element associated with a second level of the hierarchy below the first level, and, in accordance with determining that the second syntax element indicates that CCSAO filter information is present at the second level, reconstructing one or more regions from the video bitstream individually according to each CCSAO filter information present at the second level of the hierarchy of regions for each of the one or more regions. For example, a pps_ccsao_info_in_ph_flag equal to 0 indicates that CCSAO filter information is not present in the PH syntax structure but may be present in a slice header pointing to a PPS.

[0271] In some embodiments, the first level of the hierarchical structure is a picture header (PH) syntax structure that points to a picture parameter set (PPS).

[0272] In some embodiments, the second level of the hierarchy is a slice header (SH) syntax structure that points to a PPS.

[0273] In some embodiments, the CCSAO filter information includes one or more selected from the group consisting of a syntax element indicating whether CCSAO is enabled, a syntax element indicating which of the previously decoded offset sets is used, a syntax element indicating whether component on / off control is enabled at the coding tree block (CTB) level, a syntax element indicating the number of bands of the component of the corresponding class, a syntax element indicating the number of offset sets of the component of the corresponding class, a syntax element indicating the edge direction of the component of the corresponding class, a syntax element indicating whether the current component can use other components for classification, a syntax element indicating the component offset value of the corresponding class, a syntax element indicating the number of alternative sets used for the picture / slice, and a syntax element indicating the classifier candidate position.

[0274] In some embodiments, the CCSAO filter information further includes a syntax element indicating a component offset sign value of the corresponding class and a syntax element indicating a component offset absolute value of the corresponding class, and the syntax element indicating the component offset sign value of the corresponding class is decoded from the bitstream in response to determining that the syntax element indicating the component offset absolute value of the corresponding class is not zero, for example, when offset_sign_flag is (offset_abs != 0).

[0275] In some embodiments, in process 3200 for decoding a video signal, video decoder 30 receives a third syntax element from the video bitstream, and, in accordance with a determination that the third syntax element indicates that the constraint includes that a CCSAO filter is not enabled for pictures in a given Output Layer Set (OLS) in the video bitstream, video decoder 30 reconstructs one or more regions of the video bitstream with the constraint, and in accordance with a determination that the third syntax element indicates that the constraint does not apply, video decoder 30 reconstructs one or more regions of the video bitstream without the constraint. For example, gci_no_ccsao_constraint_flag equal to 1 indicates that sps_ccsao_enabled_flag of all pictures in OlsInScope is equal to 0. For example, gci_no_ccsao_constraint_flag equal to 0 does not impose such a constraint.

[0276] In some embodiments, the video bitstream further includes a third component, and each of the first, second, and third components is selected from one of the YUV components or one of the GBR components. For example, even if the video is in RGB format, the CCSAO can be applied simply by corresponding the YUV representation to GBR, respectively.

[0277] In some embodiments, determining a sample offset for each sample of the second component according to the classifier includes determining a weighted sample offset for each sample of the second component according to the classifier, for example, the classifier offsets may be weighted before being applied to the samples.

[0278] In some embodiments, determining the sample offsets of each sample of the second component according to the classifiers includes determining the sample offsets of each sample of the second component according to weighted sample offsets of the one or more classifiers.

[0279] 33 shows a computing environment 3310 connected to a user interface 3350. The computing environment 3310 may be part of a data processing server. The computing environment 3310 includes a processor 3320, a memory 3330, and an input / output (I / O) interface 3340.

[0280] Typically, the processor 3320 controls the overall operation of the computing environment 3310, such as operations related to display, data acquisition, data communication, and image processing. The processor 3320 may include one or more processors that execute instructions to perform all or a portion of the steps of the methods described above. Additionally, the processor 3320 may include one or more modules that facilitate interaction between the processor 3320 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip machine, a graphics processing unit (GPU), etc.

[0281] The memory 3330 is configured to store various types of data to support the operation of the computing environment 3310. The memory 3330 may include certain software 3332. Examples of such data include instructions for any applications or methods operated by the computing environment 3310, video data sets, image data, etc. The memory 3330 may be implemented using any type of volatile or non-volatile memory device, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk, or any combination thereof.

[0282] The I / O interface 3340 provides an interface between the processor 3320 and peripheral interface modules such as a keyboard, click wheel, buttons, etc. The buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. The I / O interface 3340 may be connected to an encoder and a decoder.

[0283] In one embodiment, a non-transitory computer-readable storage medium is also provided that comprises a plurality of programs, e.g., in memory 3330, executable by processor 3320 of computing environment 3310, for use in performing the above-described methods. Alternatively, the non-transitory computer-readable storage medium may store a bitstream or datastream comprising encoded video information (e.g., video information comprising one or more syntax elements) generated by an encoder (e.g., video encoder 20 of FIG. 2) using, e.g., the encoding method described above, for use by a decoder (e.g., video decoder 30 of FIG. 3) in decoding video data. The non-transitory computer-readable storage medium may be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0284] In one embodiment, a computing device is also provided that includes one or more processors (e.g., processor 3320) and a non-transitory computer-readable storage medium, i.e., memory 3330, storing a plurality of programs executable by the one or more processors, wherein the one or more processors are configured to perform the above-described method upon execution of the plurality of programs.

[0285] In one embodiment, a computer program product is also provided that is used to perform the above-described methods and that comprises a plurality of programs, e.g., in memory 3330, executable by processor 3320 of computing environment 3310. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0286] In one embodiment, the computing environment 3310 may be implemented using one or more ASICs, DSPs, digital signal processing devices (DSPs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0287] Further embodiments include various subsets of the above embodiments combined or otherwise rearranged in various other embodiments.

[0288] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted using a computer-readable medium as one or more instructions or code for execution by a hardware-based processor. Computer-readable media may include computer-readable storage media, which correspond to tangible media such as data storage media, or communication media, including any medium that facilitates transfer of a computer program from one place to another, for example, according to a communication protocol. In this manner, computer-readable media can generally correspond to (1) tangible computer-readable storage media that is non-transitory, or (2) communication media such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the implementations described herein. A computer program product may include computer-readable media.

[0289] The terms used in the description of implementations herein are for the purpose of describing particular implementations only and are not intended to limit the scope of the claims. When used in the description of implementations and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. The term "and / or," as used in this application, is also understood to refer to and include any and all possible combinations of one or more of the associated listed items. It is further understood that the terms "comprises" and / or "comprising," when used in this application, indicate the presence of stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or groups thereof.

[0290] In this application, terms such as "first," "second," etc. may be used to describe various elements, but it should be understood that the elements are not limited by these terms. These terms are merely used to distinguish one element from another. For example, a first electrode can be referred to as a second electrode, and a second electrode can be referred to as a first electrode, without departing from the scope of the implementation. Although a first electrode and a second electrode are both electrodes, they are not the same electrode.

[0291] References throughout this specification to the singular or plural "in one example," "an example," "an exemplary example," etc., mean that one or more particular features, structures, or characteristics described in connection with an example are included in at least one example of the disclosure. Thus, the appearance of phrases such as "in one example," "in an example," or "an exemplary example" in various places throughout this specification do not necessarily all refer to the same example. Furthermore, particular features, structures, or characteristics in one or more examples may be included in any suitable combination.

[0292] The description in this application has been given for purposes of illustration and description and is not intended to be exhaustive or limited to the precise form disclosed. Numerous modifications, variations, and alternative implementations will be apparent to those skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. The embodiments have been chosen and described to best explain the principles and practical uses of the invention and to enable others skilled in the art to understand the invention in various implementations, making full use of the underlying principles and various implementations with various modifications as suited to the particular uses contemplated. It is understood, therefore, that the scope of the claims is not limited to the particular implementations disclosed, and that modifications and other implementations are intended to be included within the scope of the appended claims.

Claims

1. receiving, from a hierarchical video bitstream, a first syntax element associated with a first level of the hierarchical structure; reconstructing one or more regions from the video bitstream individually according to the CCSAO filter information present at a second level of the hierarchical structure in accordance with determining that the first syntax element indicates that no CCSAO filter information is present at the first level; and 12. A method for decoding video data, comprising:

2. reconstructing the one or more regions below the first level from the video bitstream together in accordance with the CCSAO filter information in accordance with determining that the first syntax element indicates that the CCSAO filter information is present at the first level. The method of claim 1 further comprising:

3. the video bitstream comprises a first component and a second component, and reconstructing the one or more regions from the video bitstream according to the CCSAO filter information includes: Depending on the CCSAO filter applied, determining a classifier for the second component from a set of one or more samples of the first component associated with each sample of the second component according to the CCSAO filter information; determining whether to modify values ​​of the respective samples of the second component within regions of the one or more regions of the video bitstream according to the classifier; in response to determining to modify the values ​​of the respective samples of the second component within the region according to the classifier, determining a sample offset for the respective samples of the second component according to the classifier; modifying the value of the respective sample of the second component based on the determined sample offset; The method of claim 1 , comprising:

4. reconstructing the one or more regions from the video bitstream individually according to the CCSAO filter information present at the second level of the hierarchical structure, receiving from the video bitstream a second syntax element associated with the second level of the hierarchical structure below the first level; reconstructing the one or more regions from the video bitstream individually according to respective CCSAO filter information present at the second level of the hierarchical structure for each of the one or more regions in accordance with determining that the second syntax element indicates that the CCSAO filter information exists at the second level. The method of claim 1 , comprising:

5. 2. The method of claim 1, wherein the first level of the hierarchical structure is a picture header (PH) syntax structure that points to a picture parameter set (PPS), or the second level of the hierarchical structure is a slice header (SH) syntax structure that points to a picture parameter set (PPS).

6. 2. The method of claim 1 , wherein the CCSAO filter information includes one or more selected from the group consisting of: a syntax element indicating whether CCSAO is enabled; a syntax element indicating which of previously decoded offset sets is used; a syntax element indicating whether component on / off control is enabled at a coding tree block (CTB) level; a syntax element indicating the number of bands of a component of a corresponding class; a syntax element indicating the number of offset sets of a component of the corresponding class; a syntax element indicating an edge direction of a component of the corresponding class; a syntax element indicating whether other components can be used for classification in the current component; a syntax element indicating a component offset value of the corresponding class; a syntax element indicating the number of alternative sets used for the picture / slice; and a syntax element indicating a classifier candidate position.

7. The CCSAO filter information is a syntax element indicating a component offset sign value of a corresponding class; and a syntax element indicating a component offset absolute value of the corresponding class. Including, the syntax element indicating the component offset code value of the corresponding class is decoded from the video bitstream in response to determining that the syntax element indicating the component offset absolute value of the corresponding class is not zero. The method of claim 1.

8. receiving a third syntax element from the video bitstream; reconstructing the one or more regions from the video bitstream with the constraints in accordance with determining that the third syntax element indicates that the constraints include that a CCSAO filter is not enabled for pictures in a given Output Layer Set (OLS) in the video bitstream; and reconstructing the one or more regions from the video bitstream without the constraint in accordance with determining that the third syntax element indicates that the constraint does not apply. The method of claim 1 further comprising:

9. 4. The method of claim 3, wherein the video bitstream further includes a third component, and each of the first component, the second component, and the third component is selected from one of a YUV component or one of a GBR component.

10. 4. The method of claim 3, wherein determining the sample offsets of the respective samples of the second component according to the classifier comprises determining weighted sample offsets of the respective samples of the second component according to the classifier.

11. 4. The method of claim 3, wherein determining the sample offsets of the respective samples of the second component according to the classifiers comprises determining sample offsets of the respective samples of the second component according to weighted sample offsets of one or more classifiers.

12. 1. An electronic device, comprising: one or more processing units; a memory coupled to the one or more processing units; a plurality of programs stored in the memory that, when executed by the one or more processing units, cause the electronic device to perform the method of any one of claims 1 to 11; An electronic device comprising:

13. A non-transitory computer-readable storage medium, comprising: Storing a plurality of programs for execution by an electronic device having one or more processing units, said plurality of programs, when executed by said one or more processing units, causing the electronic device to perform the method of any one of claims 1 to 11. A non-transitory computer-readable storage medium.

14. A method for storing a bitstream, comprising: performing an encoding method to generate said bitstream; storing the generated bitstream; The encoding method includes: acquiring video frames having a hierarchical structure; configuring a first syntax element associated with a first level of the hierarchy, the first syntax element indicating whether cross-component sample adaptive offset (CCSAO) filter information is present at the first level of the hierarchy; If the CCSAO filter information is not present at a first level of the hierarchical structure, encoding one or more regions of the video frame individually according to the CCSAO filter information present at a second level of the hierarchical structure; and 1. A method for storing a bitstream, comprising:

Images