Enhanced coding in intercomponent sample-adaptive offsets.

Abstract

An electronic device performs a method for decoding video data, the method including receiving a picture frame from a video signal, the picture frame including a first component and a second component, determining a classifier for the first component based on one or more samples of a first set of the second component associated with each sample of the first component and one or more samples of the second set of the first component associated with each sample of the first component, determining sample offsets for each sample of the first component according to the classifier, and modifying values ​​of the respective samples of the first component based on the determined sample offsets. In some embodiments, the first component is a luma component and the second component is a chroma component, and the classifier is determined by a combination of a band offset classifier and an edge offset classifier.

Description

[Technical field]

[0001] Related Applications This application claims priority to U.S. Provisional Patent Application No. 63 / 152,324, entitled “Cross-component Sample Adaptive Offset,” filed February 22, 2021, which is incorporated by reference in its entirety.

[0002] This application relates generally to video coding and compression, and more particularly to methods and apparatus for improving both luma and chroma coding efficiency. [Background technology]

[0003] Digital video is supported by a variety of electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video gaming consoles, smartphones, video teleconferencing devices, and video streaming devices. Electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression standards. Some well-known video coding standards include Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC) (also known as H.265 or MPEG-H Part 2), and Advanced Video Coding (AVC) (also known as H.264 or MPEG-4 Part 10), which were jointly developed by ISO / IEC MPEG and ITU-T VCEG. AOMedia Video 1 (AV1) was developed by the Alliance for Open Media (AOM) as a successor to the predecessor standard VP9. Audio Video Coding (AVS) refers to digital audio and digital video compression standards, another series of video compression standards developed by the China Audio and Video Coding Standard Workgroup.

[0004] Video compression typically involves performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in video data. For block-based video coding, a video frame is divided into one or more slices, each having multiple video blocks, sometimes referred to as coding tree units (CTUs). Each CTU may contain one coding unit (CU) or may be recursively divided into smaller CUs until a predefined minimum CU size is reached. Each CU (also referred to as a leaf CU) contains one or more transform units (TUs), and each CU also contains one or more prediction units (PUs). Each CU may be coded in intra, inter, or IBC mode. Video blocks in an intra-coded (I) slice of a video frame are coded using spatial prediction relative to reference samples in neighboring blocks in the same video frame. Video blocks in an inter-coded (P or B) slice of a video frame may use spatial prediction relative to reference samples in neighboring blocks in the same video frame, or temporal prediction relative to reference samples in other previous and / or future reference video frames.

[0005] Spatial or temporal prediction based on previously coded reference blocks, e.g., neighboring blocks, results in a prediction block for the current video block to be coded. The process of finding the reference block may be realized by a block matching algorithm. Residual data representing pixel differences between the current block to be coded and the prediction block is called the residual block or prediction error. Inter-coded blocks are coded according to motion vectors that point to reference blocks and residual blocks in a reference frame that form the prediction block. The process of determining the motion vector is typically called motion estimation. Intra-coded blocks are coded according to an intra-prediction mode and a residual block. For further compression, the residual block is transformed from the pixel domain to a transform domain, e.g., the frequency domain, to result in residual transform coefficients, which may then be quantized. The quantized transform coefficients are initially arranged in a two-dimensional array and may be scanned to result in a one-dimensional vector of transform coefficients, which may then be entropy coded into a video bitstream to achieve further compression.

[0006] The encoded video bitstream may then be stored in a computer-readable storage medium (e.g., flash memory) for access by another electronic device having digital video capabilities, or transmitted directly to the electronic device via wired or wireless connection. The electronic device may then perform video decompression (which is the reverse process of video compression described above), e.g., by parsing the encoded video bitstream to obtain syntax elements from the bitstream, reconstructing the digital video data from the encoded video bitstream to its original form based at least in part on the syntax elements obtained from the bitstream, and rendering the reconstructed digital video data on a display of the electronic device. Summary of the Invention [Problem to be solved by the invention]

[0007] As the quality of digital video goes from high definition to 4K×2K or even 8K×4K, the amount of video data to be encoded / decoded increases exponentially. How the video data can be encoded / decoded more efficiently while maintaining the image quality of the decoded video data is a constant challenge. [Means for solving the problem]

[0008] This application describes implementation examples relating to the encoding and decoding of video data, and more particularly, to methods and apparatus for improving the coding efficiency of both luma and chroma components, including improving the coding efficiency by exploring inter-component relationships between the luma and chroma components.

[0009] According to a first aspect of the present application, a method for decoding a video signal includes receiving from the video signal a picture frame including a first component and a second component, determining a classifier for the first component based on one or more samples of a first set of the second component associated with each sample of the first component and one or more samples of the second set of the first component associated with each sample of the first component, determining a sample offset for each sample of the first component in accordance with the classifier, and modifying a value of each sample of the first component based on the determined sample offset.

[0010] According to a second aspect of the present application, an electronic device includes one or more processing units, a memory, and a number of programs stored in the memory, the programs, when executed by the one or more processing units, causing the electronic device to perform the method for encoding a video signal as described above.

[0011] According to a third aspect of the present application, a non-transitory computer-readable storage medium stores a number of programs for execution by an electronic device having one or more processing units, the programs, when executed by the one or more processing units, causing the electronic device to perform the method for encoding a video signal as described above.

[0012] According to a fourth aspect of the present application, a computer-readable storage medium stores a bitstream including video information generated by the video decoding method described above.

[0013] It is to be understood that both the foregoing general description and the following detailed description are merely exemplary and are not restrictive of the present disclosure.

[0014] The accompanying drawings, which are included to provide a further understanding of implementations, are incorporated in and constitute a part of this specification, illustrate implementations described and, together with the description, serve to explain the underlying principles. Like reference numerals refer to corresponding parts. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram illustrating an example video encoding and decoding system according to some implementations of the present disclosure. [Diagram 2] FIG. 2 is a block diagram illustrating an example video encoder according to some implementations of the present disclosure. [Diagram 3] FIG. 2 is a block diagram illustrating an example video decoder according to some implementations of the present disclosure. [Figure 4A] A block diagram showing how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some implementations of this disclosure. [Figure 4B] A block diagram showing how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some implementations of this disclosure. [Figure 4C]A block diagram showing how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some implementations of this disclosure. [Figure 4D] A block diagram showing how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some implementations of this disclosure. [Figure 4E] A block diagram showing how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some implementations of this disclosure. [Diagram 5] FIG. 1 is a block diagram illustrating four gradient patterns used in sample adaptive offset (SAO) according to some implementations of the present disclosure. [Figure 6A] FIG. 1 is a block diagram illustrating a CCSAO system and process applied to chroma samples by some implementations of the present disclosure and using DBF Y as an input. [Figure 6B] FIG. 1 is a block diagram illustrating a CCSAO system and process applied to luma samples and chroma samples by some implementations of the present disclosure and using DBF Y / Cb / Cr as input. [Figure 6C] FIG. 1 is a block diagram illustrating a CCSAO system and process that can function independently according to some implementations of the present disclosure. [Figure 6D] FIG. 1 is a block diagram illustrating a CCSAO system and process that may be applied recursively (2 or N times) with the same or different offsets by some implementations of the present disclosure. [Figure 6E] FIG. 1 is a block diagram illustrating a CCSAO system and process applied in parallel to Enhanced Sample Adaptive Offset (ESAO) in the AVS standard according to some implementations of the present disclosure. [Figure 6F] FIG. 1 is a block diagram illustrating a CCSAO system and process applied after SAO according to some implementations of the present disclosure. [Figure 6G]A block diagram illustrating that some implementations of the present disclosure allow the CCSAO system and process to function independently without CCALF. [Figure 6H] FIG. 1 is a block diagram illustrating a CCSAO system and process applied in parallel to a cross-component adaptive loop filter (CCALF) according to some implementations of the present disclosure. [Figure 7] FIG. 13 is a block diagram illustrating a sample process using a CCSAO in accordance with some implementations of the present disclosure. [Figure 8] A block diagram showing how the CCSAO process is interleaved into vertical and horizontal deblocking filters (DBFs) according to some implementations of the present disclosure. [Figure 9] 1 is a flow diagram illustrating an example process for decoding a video signal using inter-component correlation according to some implementations of the present disclosure. [Figure 10A] FIG. 1 is a block diagram illustrating a classifier that uses different luma (or chroma) sample positions for C0 classification in accordance with some implementations of the present disclosure. [Figure 10B] 1A and 1B are diagrams illustrating several examples of different shapes for luma candidates according to some implementations of the present disclosure. [Figure 11] FIG. 13 is a block diagram of a sample process illustrating that all co-located and adjacent luma / chroma samples can be provided to CCSAO classification according to some implementations of the present disclosure. [Figure 12] FIG. 13 illustrates an example classifier by replacing a co-located luma sample value with a value obtained by weighting the co-located and neighboring luma samples, according to some implementations of the present disclosure. [Figure 13A] A block diagram showing that, according to some implementation examples of the present disclosure, CCSAO is not applied to the current chroma (luma) sample if any of the co-located and adjacent luma (chroma) samples used for classification are outside the current picture. [Figure 13B]FIG. 13 is a block diagram illustrating that, in accordance with some implementations of the present disclosure, CCSAO is applied to a current luma or chroma sample when any of the co-located and neighboring luma or chroma samples used for classification are outside the current picture. [Figure 14] A block diagram showing that, according to some implementation examples of the present disclosure, CCSAO is not applied to a current chroma sample if the corresponding selected co-located or adjacent luma sample used for classification is outside the virtual space defined by a virtual boundary (VB). [Figure 15] 13 illustrates how repetition or mirror padding is applied to luma samples that are outside the virtual boundary in accordance with some implementations of the present disclosure. [Figure 16] FIG. 13 illustrates that, according to some implementations of the present disclosure, an additional 1 luma line buffer is required when all 9 co-located and adjacent luma samples are used for classification. [Figure 17] FIG. 13 illustrates that in some implementations of the present disclosure, in an AVS, a nine luma candidate CCSAO intersecting a VB can be increased by two additional luma line buffers. [Figure 18A] FIG. 13 illustrates that in VVC, nine luma candidate CCSAOs intersecting a VB may be increased by one additional luma line buffer according to some implementation examples of the present disclosure. [Figure 18B] FIG. 13 illustrates that, according to some implementations of the present disclosure, when co-located or adjacent chroma samples are used to classify the current luma sample, the selected chroma candidate may cross the VB and require an additional chroma line buffer. [Figure 19A] A diagram showing that in AVS and VVC, CCSAO is disabled for a chroma sample if any of the luma candidates for the chroma sample crosses the VB (is outside the current chroma sample VB) according to some implementation examples of the present disclosure. [Figure 19B]A diagram showing that in AVS and VVC, CCSAO is disabled for a chroma sample if any of the luma candidates for the chroma sample crosses the VB (is outside the current chroma sample VB) according to some implementation examples of the present disclosure. [Figure 19C] A diagram showing that in AVS and VVC, CCSAO is disabled for a chroma sample if any of the luma candidates for the chroma sample crosses the VB (is outside the current chroma sample VB) according to some implementation examples of the present disclosure. [Figure 20A] A diagram showing that, in some implementation examples of the present disclosure, in AVS and VVC, CCSAO is enabled using repeat padding on chroma samples when any of the luma candidates for a chroma sample crosses the VB (is outside the current chroma sample VB). [Figure 20B] A diagram showing that, in some implementation examples of the present disclosure, in AVS and VVC, CCSAO is enabled using repeat padding on chroma samples when any of the luma candidates for a chroma sample crosses the VB (is outside the current chroma sample VB). [Figure 20C] A diagram showing that, in some implementation examples of the present disclosure, in AVS and VVC, CCSAO is enabled using repeat padding on chroma samples when any of the luma candidates for a chroma sample crosses the VB (is outside the current chroma sample VB). [Figure 21A] A diagram illustrating that, in some implementation examples of the present disclosure, in AVS and VVC, CCSAO is enabled using mirror padding on a chroma sample when any of the luma candidates for the chroma sample crosses the VB (is outside the current chroma sample VB). [Figure 21B] A diagram illustrating that, in some implementation examples of the present disclosure, in AVS and VVC, CCSAO is enabled using mirror padding on a chroma sample when any of the luma candidates for the chroma sample crosses the VB (is outside the current chroma sample VB). [Figure 21C] A diagram illustrating that, in some implementation examples of the present disclosure, in AVS and VVC, CCSAO is enabled using mirror padding on a chroma sample when any of the luma candidates for the chroma sample crosses the VB (is outside the current chroma sample VB). [Figure 22A] A diagram illustrating that CCSAO is enabled using two-sided symmetric padding for different CCSAO sample shapes according to some implementations of the present disclosure. [Figure 22B] A diagram illustrating that CCSAO is enabled using two-sided symmetric padding for different CCSAO sample shapes according to some implementations of the present disclosure. [Figure 23] FIG. 1 illustrates a limitation of using a limited number of luma candidates for classification according to some implementations of the present disclosure. [Figure 24] A diagram showing that, according to some implementation examples of the present disclosure, the CCSAO application region is not aligned to the coding tree block (CTB) / coding tree unit (CTU) boundary. [Diagram 25] A diagram showing that CCSAO application area frame partitioning can be fixed by CCSAO parameters according to some implementation examples of the present disclosure. [Figure 26] A diagram showing that according to some implementation examples of the present disclosure, the CCSAO application area may be a binary tree (BT) / quadtree (QT) / tertile tree (TT) split from the frame / slice / CTB level. [Figure 27] A block diagram illustrating multiple classifiers that are used and switched at different levels within a picture frame in accordance with some implementations of the present disclosure. [Figure 28] A block diagram showing that CCSAO application region partitioning is dynamic and can be switched at the picture level according to some implementation examples of the present disclosure. [Figure 29] FIG. 13 is a block diagram illustrating the SAO classification method disclosed in this disclosure acting as a post-prediction filter, according to some implementations of the present disclosure. [Diagram 30] FIG. 13 is a block diagram illustrating that, for a post-prediction SAO filter, each component can use the current sample and neighboring samples for classification, according to some implementations of the present disclosure. [Diagram 31] 1 is a flow diagram illustrating an example process for decoding a video signal using inter-component correlation according to some implementations of the present disclosure. [Diagram 32] FIG. 1 illustrates a computing environment coupled to a user interface in accordance with some implementations of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Reference will now be made in detail to specific implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to one skilled in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to one skilled in the art that the subject matter presented herein may be implemented in many types of electronic devices having digital video capabilities.

[0017] The first generation of AVS standards includes the Republic of China national standard "Information Technology, Advanced Audio Video Coding, Part 2: Video" (known as AVS1), and "Information Technology, Advanced Audio Video Coding Part 16: Radio Television Video" (known as AVS+), which can provide about 50% bit rate savings with the same perceptual quality compared to the MPEG-2 standard. The second generation of AVS standards includes a series of Republic of China national standards "Information Technology, Efficient Multimedia Coding" (known as AVS2), which mainly targets the transmission of additional HD TV programs. The coding efficiency of AVS2 is twice that of AVS+. Meanwhile, the video part of the AVS2 standard has been submitted by the Institute of Electrical and Electronics Engineers (IEEE) as one international standard for application. The AVS3 standard is a new generation video coding standard for UHD video that aims to surpass the coding efficiency of the latest international standard HEVC, providing approximately 30% bit rate savings over the HEVC standard. In March 2019, at the 68th AVS Conference, the AVS3-P2 baseline was completed, which provides approximately 30% bit rate savings over the HEVC standard. Currently, a reference software called High Performance Model (HPM) is maintained by the AVS group to demonstrate the reference implementation of the AVS3 standard. Like HEVC, the AVS3 standard is built on a block-based hybrid video coding framework.

[0018] 1 is a block diagram illustrating an example system 10 for encoding and decoding video blocks in parallel according to some implementations of the present disclosure. As shown in FIG. 1, system 10 includes a source device 12 that generates and encodes video data to be subsequently decoded by a destination device 14. Source device 12 and destination device 14 may comprise any of a wide variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, and the like. In some implementations, source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0019] In some implementations, the destination device 14 may receive the encoded video data to be decoded via a link 16. The link 16 may comprise any type of communication medium or device capable of moving the encoded video data from the source device 12 to the destination device 14. In one example, the link 16 may comprise a communication medium to enable the source device 12 to transmit the encoded video data directly to the destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device 14. The communication medium may include any wireless or wired communication medium, such as the radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network, e.g., the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from the source device 12 to the destination device 14.

[0020] In some other implementations, the encoded video data may be transmitted from the output interface 22 to the storage device 32. The encoded video data in the storage device 32 may then be accessed by the destination device 14 via the input interface 28. The storage device 32 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing the encoded video data. In a further example, the storage device 32 may correspond to a file server or another intermediate storage device that may hold the encoded video data generated by the source device 12. The destination device 14 may access the stored video data via streaming or download from the storage device 32. The file server may be any type of computer capable of storing the encoded video data and transmitting the encoded video data to the destination device 14. Exemplary file servers include a web server (e.g., for websites), an FTP server, a network attached storage (NAS) device, or a local disk drive. The destination device 14 may access the encoded video data over any standard data connection, including a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both, suitable for accessing the encoded video data stored on a file server. The transmission of the encoded video data from the storage device 32 may be a streaming transmission, a download transmission, or a combination of both.

[0021] As shown in FIG. 1, source device 12 includes video source 18, video encoder 20, and output interface 22. Video source 18 may include sources such as a video capture device, e.g., a video camera, a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video, or a combination of such sources. As an example, if video source 18 is a video camera of a security surveillance system, source device 12 and destination device 14 may form a camera phone or video phone. However, the implementations described herein may be applicable to video coding in general and may be applied to wireless and / or wired applications.

[0022] The captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be transmitted directly to destination device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by destination device 14 or other devices for decoding and / or playback. Output interface 22 may further include a modem and / or a transmitter.

[0023] Destination device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem and may receive encoded video data over link 16. The encoded video data communicated over link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data that is transmitted over a communication medium, stored on a storage medium, or stored on a file server.

[0024] In some implementations, destination device 14 may include a display device 34, which may be an integrated display device or an external display device configured to communicate with destination device 14. Display device 34 displays the decoded video data to a user and may comprise any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0025] Video encoder 20 and video decoder 30 may operate according to a proprietary or industry standard, such as VVC, HEVC, MPEG-4, Part 10, Advanced Video Coding (AVC), AVS, or an extension of such a standard. It should be understood that the present application is not limited to a particular video coding / decoding standard and may apply to other video coding / decoding standards. It is generally contemplated that video encoder 20 of source device 12 may be configured to encode video data according to any of these current or future standards. Similarly, it is generally contemplated that video decoder 30 of destination device 14 may be configured to decode video data according to any of these current or future standards.

[0026] Video encoder 20 and video decoder 30 may each be implemented as any of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When implemented partially in software, an electronic device may store instructions for the software on a suitable non-transitory computer-readable medium and execute those instructions in hardware using one or more processors to perform the video coding / decoding operations disclosed in this disclosure. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0027] 2 is a block diagram illustrating an example video encoder 20 according to some implementations described herein. Video encoder 20 can perform intra- and inter-predictive coding of video blocks within a video frame. Intra-predictive coding relies on spatial prediction to reduce or remove spatial redundancy of video data within a given video frame or picture. Inter-predictive coding relies on temporal prediction to reduce or remove temporal redundancy of video data within adjacent video frames or pictures of a video sequence.

[0028] As shown in FIG. 2, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a partition unit 45, an intra-prediction processing unit 46, and an intra-block copy (BC) unit 48. In some implementations, the video encoder 20 also includes an inverse quantization unit 58 for reconstruction of video blocks, an inverse transform processing unit 60, and an adder 62. An in-loop filter 63, such as a deblocking filter, may be placed between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another in-loop filter 63 may also be used to filter the output of the adder 62. Further in-loop filters 63, such as a sample adaptive offset (SAO) and an adaptive in-loop filter (ALF), may be applied on the reconstructed CU before being placed into a reference picture store and used as a reference for coding future video blocks. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be split among one or more of the fixed or programmable hardware units shown.

[0029] Video data memory 40 may store video data to be encoded by components of video encoder 20. The video data in video data memory 40 may be obtained, for example, from video source 18. DPB 64 is a buffer that stores reference video data for use by video encoder 20 in encoding the video data (e.g., intra- or inter-predictive coding modes). Video data memory 40 and DPB 64 may be formed by any of a variety of memory devices. In various examples, video data memory 40 may be on-chip with other components of video encoder 20 or off-chip with respect to those components.

[0030] As shown in FIG. 2, after receiving the video data, partition unit 45 in prediction processing unit 41 partitions the video data into video blocks. This partitioning may also include partitioning the video frame into slices, tiles, or other larger coding units (CUs) according to a predefined partitioning structure, such as a quadtree structure, associated with the video data. The video frame may be partitioned into a number of video blocks (or sets of video blocks, called tiles). Prediction processing unit 41 may select one of a number of possible predictive coding modes, such as one of a number of intra-predictive coding modes or one of a number of inter-predictive coding modes, for the current video block based on the error result (e.g., coding rate and distortion level). Prediction processing unit 41 may provide the resulting intra- or inter-predictive coded block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coded block for later use as part of the reference frame. Prediction processing unit 41 also provides syntax elements, such as motion vectors, intra-mode indicators, partition information, and other such syntax information, to entropy coding unit 56.

[0031] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-predictive coding of the current video block relative to one or more neighboring blocks in the same frame as the current block to be coded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 perform inter-predictive coding of the current video block relative to one or more predictive blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple coding passes, e.g., to select an appropriate coding mode for each block of video data.

[0032] In some implementations, motion estimation unit 42 determines the inter prediction mode for the current video frame by generating a motion vector indicating the displacement of a prediction unit (PU) of a video block in the current video frame relative to a predictive block in a reference video frame according to a predetermined pattern in a sequence of video frames. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors to estimate motion for video blocks. The motion vector may indicate, for example, the displacement of a PU of a video block in a current video frame or picture relative to a predictive block (or other coding unit) in a reference frame relative to a current block (or other coding unit) being coded in the current frame. The predetermined pattern may specify a video frame in the sequence as a P frame or a B frame. Intra BC unit 48 may determine a vector, e.g., a block vector, for intra BC coding, similar to the determination of a motion vector by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine a block vector.

[0033] The prediction block is a block of a reference frame that is deemed to closely match the PU of the video block to be coded in terms of pixel difference, which may be determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metric. In some implementations, video encoder 20 may calculate values ​​for sub-integer pixel locations of the reference frame stored in DPB 64. For example, video encoder 20 may interpolate values ​​for quarter, eighth, or other fractional pixel locations of the reference frame. Thus, motion estimation unit 42 may perform motion search for full and fractional pixel locations and output motion vectors with fractional pixel accuracy.

[0034] Motion estimation unit 42 calculates a motion vector for a PU of a video block in an inter-predictive coded frame by comparing the position of the PU with the position of a predictive block of a reference frame selected from a first reference frame list (List0) or a second reference frame list (List1), each of which identifies one or more reference frames stored in DPB 64. Motion estimation unit 42 sends the calculated motion vector to motion compensation unit 44 and then to entropy encoding unit 56.

[0035] The motion compensation performed by motion compensation unit 44 may involve fetching or generating a predictive block based on the motion vector determined by motion estimation unit 42. Upon receiving the motion vector for the PU of the current video block, motion compensation unit 44 may locate the predictive block to which the motion vector points in one of the reference frame lists, retrieve the predictive block from DPB 64, and forward the predictive block to summer 50. Summer 50 then forms a residual video block of pixel difference values ​​by subtracting pixel values ​​of the predictive block provided by motion compensation unit 44 from pixel values ​​of the current video block being coded. The pixel difference values ​​forming the residual video block may include luma or chroma difference components or both. Motion compensation unit 44 may also generate syntax elements associated with the video blocks of the video frames for use by video decoder 30 in decoding the video blocks of the video frames. The syntax elements may include, for example, syntax elements defining the motion vector used to identify the predictive block, any flags indicating a prediction mode, or any other syntax information described herein. It should be noted that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are shown separately for conceptual purposes.

[0036] In some implementations, the intra BC unit 48 may generate a vector and fetch a predictive block, similar to that described above in connection with the motion estimation unit 42 and the motion compensation unit 44, except that the predictive block is in the same frame as the current block being coded, and the vector is referred to as a block vector rather than a motion vector. In particular, the intra BC unit 48 may determine an intra prediction mode to use to code the current block. In some implementations, the intra BC unit 48 may code the current block using various intra prediction modes, e.g., during separate coding passes, and test their performance by rate-distortion analysis. The intra BC unit 48 may then select a suitable intra prediction mode from among the various tested intra prediction modes, and use and generate an intra mode indicator accordingly. For example, the intra BC unit 48 may calculate a rate-distortion value using a rate-distortion analysis for the various tested intra prediction modes, and select and use the intra prediction mode with the best rate-distortion characteristics from among the tested modes as the suitable intra prediction mode. Rate-distortion analysis generally determines the amount of distortion (or error) between an encoded block and the original uncoded block that was coded to result in the encoded block, as well as the bit rate (i.e., number of bits) used to result in the encoded block. Intra BC unit 48 may calculate ratios from the distortion and rate for various coded blocks to determine which intra prediction mode exhibits the best rate-distortion value for that block.

[0037] In other examples, intra BC unit 48 may use, in whole or in part, motion estimation unit 42 and motion compensation unit 44, depending on the implementation described herein, to perform such functions for intra BC prediction. In either case, for intra block copying, the predictive block may be a block deemed to closely match the block to be coded in terms of pixel differences, which may be determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metric, and identification of the predictive block may include calculation of values ​​for sub-integer pixel locations.

[0038] Regardless of whether the predictive block is from the same frame according to intra prediction or from a different frame according to inter prediction, video encoder 20 may form a residual video block by subtracting pixel values ​​of the predictive block from pixel values ​​of the current video block being coded to form pixel difference values. The pixel difference values ​​that form the residual video block may include both luma and chroma component differences.

[0039] Intra-prediction processing unit 46 may intra-predict the current video block as an alternative to inter-prediction performed by motion estimation unit 42 and motion compensation unit 44, or intra-block-copy prediction performed by intra BC unit 48, as described above. In particular, intra-prediction processing unit 46 may determine an intra-prediction mode to use to encode the current block. In doing so, intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, e.g., during separate encoding passes, and intra-prediction processing unit 46 (or, in some examples, a mode selection unit) may select and use an appropriate intra-prediction mode from the tested intra-prediction modes. Intra-prediction processing unit 46 may provide information indicative of the selected intra-prediction mode for the block to entropy coding unit 56. Entropy coding unit 56 may encode the information indicative of the selected intra-prediction mode in the bitstream.

[0040] After prediction processing unit 41 determines a predictive block for a current video block, via inter- or intra-prediction, summer 50 forms a residual video block by subtracting the predictive block from the current video block. The residual video data in the residual block may be included in one or more transform units (TUs) and provided to transform processing unit 52. Transform processing unit 52 converts the residual video data into residual transform coefficients using a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform.

[0041] Transform processing unit 52 may send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, quantization unit 54 may then perform a scan of a matrix including the quantized transform coefficients. Alternatively, entropy coding unit 56 may perform the scan.

[0042] Following quantization, entropy encoding unit 56 entropy encodes the quantized transform coefficients into a video bitstream, e.g., using context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy encoding method or technique. The encoded bitstream may then be transmitted to video decoder 30 or stored in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being coded.

[0043] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain and to generate reference blocks for prediction of other video blocks. As described above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0044] Summer 62 adds the reconstructed residual block to the motion compensated prediction block provided by motion compensation unit 44 to provide a reference block for storage in DPB 64. The reference block may then be used by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 as a prediction block for inter predicting another video block in a subsequent video frame.

[0045] 3 is a block diagram illustrating an example video decoder 30 according to some implementations of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-prediction processing unit 84, and an intra-BC unit 85. The video decoder 30 may perform a decoding process, which is generally reverse to the encoding process described above for the video encoder 20 in connection with FIG. 2. For example, the motion compensation unit 82 may generate prediction data based on a motion vector received from the entropy decoding unit 80, and the intra-prediction unit 84 may generate prediction data based on an intra-prediction mode indicator received from the entropy decoding unit 80.

[0046] In some examples, the units of the video decoder 30 may be tasked with performing implementations of the present application. Also, in some examples, implementations of the present disclosure may be divided among one or more of the units of the video decoder 30. For example, the intra BC unit 85 may perform implementations of the present application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction processing unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0047] The video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of the video decoder 30. The video data stored in the video data memory 79 may be obtained, for example, from the storage device 32, from a local video source, such as a camera, via wired or wireless network communication of the video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). The video data memory 79 may include a coded picture buffer (CPB) that stores coded video data from the coded video bitstream. A decoded picture buffer (DPB) 92 of the video decoder 30 stores reference video data for use in decoding the video data by the video decoder 30 (e.g., for intra- or inter-predictive coding modes). The video data memory 79 and the DPB 92 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For purposes of illustration, video data memory 79 and DPB 92 are depicted in FIGURE 3 as two separate components of video decoder 30. However, it will be apparent to one skilled in the art that video data memory 79 and DPB 92 may be provided by the same memory device or by separate memory devices. In some examples, video data memory 79 may be on-chip with other components of video decoder 30 or off-chip with respect to those components.

[0048] During the decoding process, video decoder 30 receives an encoded video bitstream representing video blocks and associated syntax elements of encoded video frames. Video decoder 30 may receive the syntax elements at the video frame level and / or the video block level. Entropy decoding unit 80 of video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors, or intra-prediction mode indicators, and other syntax elements. Entropy decoding unit 80 then forwards the motion vectors and other syntax elements to prediction processing unit 81.

[0049] When a video frame is coded as an intra-predictive coded (I) frame, or coded for intra-coded predictive blocks in other types of frames, intra-prediction processing unit 84 of prediction processing unit 81 may generate predictive data for video blocks of the current video frame based on the signaled intra-prediction mode and reference data from previously decoded blocks of the current frame.

[0050] When a video frame is coded as an inter-predictive coded (i.e., B or P) frame, motion compensation unit 82 of prediction processing unit 81 produces one or more prediction blocks for a video block of the current video frame based on the motion vectors and other syntax elements received from entropy decoding unit 80. Each of the prediction blocks may be produced from one reference frame of the reference frame lists. Video decoder 30 may construct reference frame lists List0 and List1 using a default construction technique based on reference frames stored in DPB 92.

[0051] In some examples, when a video block is coded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 produces a predictive block for the current video block based on the block vectors and other syntax elements received from entropy decoding unit 80. The predictive block may be located within the same reconstruction region of the picture as the current video block as defined by video encoder 20.

[0052] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for video blocks of the current video frame by analyzing the motion vectors and other syntax elements, and then use the prediction information to result in a predictive block for the current video block being decoded. For example, motion compensation unit 82 uses some of the received syntax elements to determine a prediction mode (e.g., intra or inter prediction) used to code video blocks of the video frame, an inter prediction frame type (e.g., B or P), construction information for one or more of the reference frame lists for the frame, a motion vector for each inter predictive coded video block of the frame, an inter prediction state for each inter predictive coded video block of the frame, and other information for decoding video blocks in the current video frame.

[0053] Similarly, intra BC unit 85 may use some of the received syntax elements, such as flags, to determine that the current video block was predicted using intra BC mode, construction information for the video blocks of the frame that are located within the reconstruction domain and should be stored in DPB 92, block vectors for each intra BC predicted video block of the frame, intra BC prediction states for each intra BC predicted video block of the frame, and other information for decoding video blocks in the current video frame.

[0054] Motion compensation unit 82 may also perform the interpolation using the interpolation filters used by video encoder 20 during encoding of the video block to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 from the received syntax elements and may use those interpolation filters to result in the predictive block.

[0055] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients entropy decoded by entropy decoding unit 80 provided in the bitstream using the same quantization parameters calculated by video encoder 20 for each video block in a video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform, e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients to reconstruct the residual block in the pixel domain.

[0056] After the motion compensation unit 82 or the intra BC unit 85 generates a predictive block for the current video block based on the vectors and other syntax elements, the adder 90 reconstructs a decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 and the corresponding predictive block generated by the motion compensation unit 82 and the intra BC unit 85. To further process the decoded video block, an in-loop filter 91 may be placed between the adder 90 and the DPB 92. Before being put into the reference picture store, an in-loop filter 91, such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive in-loop filter (ALF), may be applied on the reconstructed CU. The decoded video blocks in a given frame are then stored in the DPB 92, which stores reference frames used for motion compensation after the next video block. The DPB 92, or a memory device separate from the DPB 92, may also store the decoded video for later presentation on a display device, such as the display device 34 of FIG. 1.

[0057] In a typical video coding process, a video sequence typically includes an ordered set of frames or pictures. Each frame may include three sample arrays, denoted SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other cases, a frame may be monochromatic and thus include only one two-dimensional array of luma samples.

[0058] Similar to HEVC, the AVS3 standard is built on a block-based hybrid video coding framework. The input video signal is processed by blocks (called coding units (CUs)). Unlike HEVC, which divides blocks based only on quadtrees, in AVS3, one coding tree unit (CTU) is divided into CUs based on quadtrees / binary trees / extended quadtrees to adapt to varying local characteristics. In addition, the concept of multiple partitioning unit types in HEVC is removed, i.e., there is no separation of CUs, prediction units (PUs), and transform units (TUs) in AVS3. Instead, each CU is always used as a basic unit for both prediction and transformation without further partitioning. In the tree partitioning structure of AVS3, first, one CTU is partitioned based on a quadtree structure. Then, each quadtree leaf node can be further partitioned based on binary tree and extended quadtree structures.

[0059] As shown in FIG. 4A, video encoder 20 (or, more specifically, partitioning unit 45) generates a coded representation of a frame by first partitioning the frame into a set of coding tree units (CTUs). A video frame may contain an integer number of CTUs sequentially ordered from left to right and top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by video encoder 20 in a sequence parameter set, and all CTUs in a video sequence have the same size, one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that the present application is not necessarily limited to a particular size. As shown in FIG. 4B, each CTU may comprise one coding tree block (CTB) of luma samples, two corresponding coding tree blocks of chroma samples, and syntax elements used to code the samples in these coding tree blocks. The syntax elements describe the properties of different types of units of coding blocks of pixels and how a video sequence may be reconstructed at video decoder 30, including inter or intra prediction, intra prediction modes, motion vectors, and other parameters. For a monochrome picture, or a picture with three separate color planes, a CTU may comprise a single coding tree block and syntax elements used to code samples of this coding tree block. A coding tree block may be an N×N block of samples.

[0060] To achieve better performance, video encoder 20 may recursively perform tree partitioning, such as binary tree partitioning, ternary tree partitioning, quad tree partitioning, or a combination thereof, on the coding tree blocks of the CTU to partition the CTU into smaller coding units (CUs). As illustrated in FIG. 4C, 64×64 CTU 400 is first partitioned into four smaller CUs, each with a block size of 32×32. Among the four smaller CUs, CU 410 and CU 420 are each partitioned into four CUs with a block size of 16×16. The two 16×16 CUs 430 and 440 are each further partitioned into four CUs with a block size of 8×8. FIG. 4D illustrates a quad tree data structure illustrating the end result of the partitioning process of CTU 400 illustrated in FIG. 4C, where each leaf node of the quad tree corresponds to one CU with a respective size ranging from 32×32 to 8×8. Similar to the CTU depicted in FIG. 4B, each CU may comprise a coded block (CB) of luma samples, two corresponding coded blocks of chroma samples of the same size frame, and syntax elements used to code the samples of these coded blocks. In a monochromatic picture, or a picture with three separate color planes, a CU may comprise a single coded block and syntax structures used to code the samples of this coded block. It should be noted that the quadtree partition depicted in FIG. 4C and FIG. 4D is for illustrative purposes only, and one CTU may be partitioned into multiple CUs to accommodate varying local characteristics based on quadtree / ternary tree / binary tree partitioning. In multiple types of tree structures, one CTU may be partitioned by a quadtree structure, and each quadtree leaf CU may be further partitioned by binary and ternary tree structures. As shown in FIG. 4E, there are five split / partition types in AVS3: 4-way split, horizontal 2-way split, vertical 2-way split, horizontal extended quadtree split, and vertical extended quadtree split.

[0061] In some implementations, video encoder 20 may further divide a coding block of a CU into one or more M×N predictive blocks (PBs). A predictive block is a rectangular (square or non-square) block of samples to which the same inter or intra prediction is applied. A prediction unit (PU) of a CU may comprise a predictive block of luma samples and two corresponding predictive blocks of chroma samples and syntax elements used to predict these predictive blocks. In a monochromatic picture or a picture with three separate color planes, a PU may comprise a single predictive block and syntax structures used to predict this predictive block. Video encoder 20 may generate predictive luma, Cb, and Cr blocks for the luma, Cb, and Cr predictive blocks of each PU of a CU.

[0062] Video encoder 20 may generate the predictive blocks for a PU using intra prediction or inter prediction. If video encoder 20 uses intra prediction to generate the predictive blocks for a PU, video encoder 20 may generate the predictive blocks for the PU based on decoded samples of a frame associated with the PU. If video encoder 20 uses inter prediction to generate the predictive blocks for a PU, video encoder 20 may generate the predictive blocks for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0063] After video encoder 20 generates the predictive luma, Cb, and Cr blocks for one or more PUs of a CU, video encoder 20 may generate a luma residual block for the CU by subtracting the predictive luma block of the CU from its original luma coded block, where each sample in the luma residual block of the CU indicates a difference between a luma sample in one of the predictive luma blocks of the CU and a corresponding sample in the original luma coded block of the CU. Similarly, video encoder 20 may generate a Cb residual block and a Cr residual block for the CU, respectively, where each sample in the Cb residual block of the CU indicates a difference between a Cb sample in one of the predictive Cb blocks of the CU and a corresponding sample in the original Cb coded block of the CU, and each sample in the Cr residual block of the CU indicates a difference between a Cr sample in one of the predictive Cr blocks of the CU and a corresponding sample in the original Cr coded block of the CU.

[0064] Further, as shown in FIG. 4C, video encoder 20 may use quadtree partitioning to decompose the luma, Cb, and Cr residual blocks of a CU into one or more luma, Cb, and Cr transform blocks. A transform block is a rectangular (square or non-square) block of samples to which the same transform is applied. A transform unit (TU) of a CU may comprise a transform block of luma samples and two corresponding transform blocks of chroma samples, and syntax elements used to transform the transform block samples. Thus, each TU of a CU may be associated with a luma transform block, a Cb transform block, and a Cr transform block. In some examples, the luma transform block associated with a TU may be a sub-block of the luma residual block of the CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU. In a monochromatic picture or a picture with three separate color planes, a TU may comprise a single transform block and syntax structures used to transform the samples of the transform block.

[0065] Video encoder 20 may apply one or more transforms to a luma transform block of a TU to generate a luma coefficient block for the TU. The coefficient block may be a two-dimensional array of transform coefficients. The transform coefficients may be scalar quantities. Video encoder 20 may apply one or more transforms to a Cb transform block of the TU to generate a Cb coefficient block for the TU. Video encoder 20 may apply one or more transforms to a Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0066] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the video encoder 20 may quantize the coefficient block. Quantization generally refers to a process in which transform coefficients are quantized to possibly reduce the amount of data used to represent the transform coefficients to provide further compression. After the video encoder 20 quantizes the coefficient block, the video encoder 20 may entropy encode syntax elements indicating the quantized transform coefficients. For example, the video encoder 20 may perform context-adaptive binary arithmetic coding (CABAC) on the syntax elements indicating the quantized transform coefficients. Finally, the video encoder 20 may output a bitstream including a sequence of bits and associated data forming a representation of the coded frame, which may be stored in the storage device 32 or transmitted to the destination device 14.

[0067] After receiving the bitstream generated by video encoder 20, video decoder 30 may parse the bitstream to obtain syntax elements from the bitstream. Video decoder 30 may reconstruct a frame of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally reverse to the encoding process performed by video encoder 20. For example, video decoder 30 may perform an inverse transform on coefficient blocks associated with the TUs of the current CU to reconstruct residual blocks associated with the TUs of the current CU. Video decoder 30 also reconstructs the coded blocks of the current CU by adding samples of the predictive blocks for the PUs of the current CU to corresponding samples of the transform blocks of the TUs of the current CU. After reconstructing the coded blocks for each CU of the frame, video decoder 30 may reconstruct the frame.

[0068] SAO is a process of modifying decoded samples, after application of a deblocking filter, by conditionally adding an offset value to each sample based on the value of a look-up table transmitted by the encoder. SAO filtering is performed region-by-region based on the filtering type selected per CTB by the syntax element sao-type-idx. A value of 0 for sao-type-idx indicates that no SAO filter is applied to the CTB, while values ​​1 and 2 signal the use of band-offset and edge-offset filtering types, respectively. In band-offset mode, specified by sao-type-idx equal to 1, the offset value selected depends directly on the sample amplitude. In this mode, the complete sample amplitude range is uniformly divided into segments called 32 bands, and sample values ​​belonging to four of these bands (that are consecutive within the 32 bands) are modified by adding a transmitted value, denoted as band-offset, which can be positive or negative. The main reason for using four consecutive bands is that in smooth areas where banding artifacts may appear, the sample amplitudes in the CTB tend to be concentrated in only a few of these bands. In addition, the design choice of using four offsets unifies with the edge-offset operation mode, which also uses four offset values. In the edge-offset mode specified by sao-type-idx equal to 2, the syntax element sao-eo-class with a value between 0 and 3 signals whether horizontal, vertical, or one of the two diagonal gradient directions is used for edge-offset classification in the CTB.

[0069] FIG. 5 is a block diagram illustrating four gradient patterns used in SAO according to some implementations of the present disclosure. The four gradient patterns 502, 504, 506, and 508 are for each sao-eo-class in edge-offset mode. The sample labeled "p" indicates the center sample to be considered. The two samples labeled "n0" and "n1" designate two adjacent samples along the following gradient patterns: (a) horizontal (sao-eo-class=0), (b) vertical (sao-eo-class=1), (c) diagonal 135° (sao-eo-class=2), and (d) 45° (sao-eo-class=3). Each sample in the CTB is classified into one of five EdgeIdx categories by comparing the sample value p located at a certain position with the values ​​n0 and n1 of two samples located at adjacent positions, as shown in FIG. 5. This classification is performed for each sample based on the decoded sample value, and therefore no additional signaling for EdgeIdx classification is required. For EdgeIdx categories 1 through 4, an offset value from the transmitted lookup table is added to the sample value depending on the EdgeIdx category at the sample location. The offset value is always positive for categories 1 and 2, and negative for categories 3 and 4. Thus, the filter generally has a smoothing effect in edge offset mode. Table 1 below shows sample EdgeIdx categories for SAO edge classes.

[0070] [Table 1] Table 1: Sample EdgeIdx categories in SAO edge classes For SAO types 1 and 2, a total of four amplitude offset values ​​are transmitted to the decoder for each CTB. For type 1, the sign is also coded. The offset values ​​and associated syntax elements such as sao-type-idx and sao-eo-class are determined by the encoder, typically using a criterion that optimizes rate-distortion performance. SAO parameters can be indicated as inherited from the left or top CTB using a merge flag, for efficient signaling. In summary, SAO is a nonlinear filtering operation that allows additional refinement of the reconstructed signal, enhancing the signal representation, both in smooth areas and around edges.

[0071] In some embodiments, methods and systems are disclosed herein for improving coding efficiency or reducing the complexity of sample adaptive offset (SAO) by introducing inter-component information. SAO is used in the HEVC, VVC, AVS2, and AVS3 standards. In the following description, the existing SAO design in the HEVC, VVC, AVS2, and AVS3 standards is used as a basic SAO method for those skilled in the art of video coding, but the inter-component method described in this disclosure may also be applied to other loop filter designs or other coding tools with similar design spirit. For example, in the AVS3 standard, SAO is replaced by a coding tool called Enhanced Sample Adaptive Offset (ESAO). However, the CCSAO disclosed herein may be applied in parallel to the ESAO. In another example, the CCSAO may be applied in parallel to the Constrained Directional Enhancement Filter (CDEF) in the AV1 standard.

[0072] For existing SAO designs in HEVC, VVC, AVS2, and AVS3 standards, luma Y, chroma Cb, and chroma Cr sample offset values ​​are determined independently. That is, for example, the current chroma sample offset is determined only by the current chroma sample value and the adjacent chroma sample values, without considering the co-located or adjacent luma samples. However, luma samples maintain more original picture detail information than chroma samples and can benefit from the determination of the current chroma sample offset. Furthermore, since chroma samples usually lose high frequency details after RGB to YCbCr color conversion or after quantization and deblocking filters, introducing luma samples with high frequency details maintained for chroma offset determination can benefit from the reconstruction of chroma samples. Therefore, further gains can be expected by exploring inter-component correlations, for example, by using inter-component sample adaptive offset (CCSAO) methods and systems. Another example is that in the case of SAO, luma sample offsets are determined only by luma samples. But for example, luma samples with the same band offset (BO) classification may have different frequencies depending on their respective luma samples. same position and may be further classified by adjacent chroma samples, thereby obtaining a more effective classification. The SAO classification may be obtained as a shortcut to compensate for the sample difference between the original picture and the reconstructed picture. Therefore, an effective classification is desired.

[0073] 6A is a block diagram illustrating a CCSAO system and process applied on chroma samples by some implementations of the present disclosure, using DBF Y as an input. The luma sample after the luma deblocking filter (DBF Y) is used to determine additional offsets for chroma Cb and Cr after SAO Cb and SAO Cr. For example, the current chroma sample 602 is first same positionA luma sample 604 and an adjacent (white) luma sample 606 are used to classify the current luma sample, and the corresponding CCSAO offset value of the corresponding class is added to the current chroma sample value. FIG. 6B is a block diagram illustrating a CCSAO system and process that is applied on the luma sample and chroma sample by some implementations of the present disclosure and uses DBF Y / Cb / Cr as input. FIG. 6C is a block diagram illustrating a CCSAO system and process that can function independently by some implementations of the present disclosure. FIG. 6D is a block diagram illustrating a CCSAO system and process that may be applied recursively (2 or N times) with the same or different offsets by some implementations of the present disclosure. In summary, in some embodiments, to classify the current luma sample, the current luma sample and the adjacent luma samples, same position and adjacent chroma samples (Cb and Cr) may be used. In some embodiments, to classify the current chroma sample (Cb or Cr), same position and adjacent luma samples, same position and adjacent cross chroma samples, as well as the current chroma sample and the adjacent chroma sample may be used. In some embodiments, the CCSAO may be cascaded (1) after DBF Y / Cb / Cr, (2) after the reconstructed image Y / Cb / Cr before DBF, or (3) after the SAO Y / Cb / Cr, or (4) after the ALF Y / Cb / Cr.

[0074] In some embodiments, the CCSAO may also be applied in parallel to other coding tools, such as the ESAO in the AVS standard, or the CDEF in the AV1 standard. Figure 6E is a block diagram illustrating a CCSAO system and process that may be applied in parallel to the ESAO in the AVS standard by some implementations of the present disclosure.

[0075] FIG. 6F is a block diagram illustrating a CCSAO system and process applied after the SAO according to some implementations of the present disclosure. In some embodiments, FIG. 6F illustrates that the location of the CCSAO may be after the SAO, i.e., the location of the Inter-Component Adaptive Loop Filter (CCALF) in the VVC standard. FIG. 6G is a block diagram illustrating that the CCSAO system and process may function independently without the CCALF according to some implementations of the present disclosure. In some embodiments, the SAO Y / Cb / Cr may be replaced with ESAO, for example in the AVS3 standard.

[0076] FIG. 6H is a block diagram illustrating a CCSAO system and process applied in parallel to CCALF according to some implementations of the present disclosure. In some embodiments, FIG. 6H illustrates that CCSAO may be applied in parallel to CCALF. In some embodiments, in FIG. 6H, the locations of CCALF and CCSAO may be switched. In some embodiments, in FIG. 6A-FIG. 6H or throughout the present disclosure, the SAO Y / Cb / Cr block may be swapped for ESAO Y / Cb / Cr (AVS3) or CDEF (AV1). Note that Y / Cb / Cr may also be denoted as Y / U / V in the video coding domain.

[0077] In some embodiments, the current chroma sample classification is: same position We reuse the SAO type (edge ​​offset (EO) or BO), class, and category of the luma sample. The corresponding CCSAO offset can be signaled or derived from the decoder itself. For example, h_Y can be same position Let h_Y be the luma SAO offset, and h_Cb and h_Cr be the CCSAO Cb and Cr offsets, respectively. h_Cb (or h_Cr) = w*h_Y, where w can be chosen in a restricted table, e.g., ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4..., etc., where |w| includes only values ​​that are powers of 2.

[0078] In some embodiments, same positionThe comparison scores of a luma sample (Y0) and its eight adjacent luma samples [-8,8] are used, resulting in a total of 17 classes.

[0079] Initial Class=0 Loop through 8 adjacent luma samples (Yi, i=1~8) if Y0>Yi Class+=1 else if Y0 <Yi Class-=1 In some embodiments, the classification methods described above may be combined. For example, to increase diversity, the comparison scores combined with SAO BO (32 band classification) are used, resulting in a total of 17*32 classes. In some embodiments, Cb and Cr may use the same class to reduce complexity or save bits.

[0080] FIG. 7 is a block diagram illustrating a sample process using a CCSAO according to some implementations of the present disclosure. Specifically, FIG. 7 illustrates that the input of the CCSAO can introduce vertical and horizontal DBF inputs to simplify class determination or increase flexibility. For example, Y0_DBF_V, Y0_DBF_H, and Y0 can be respectively inputs to DBF_V, DBF_H, and SAO. same position Let Yi_DBF_V, Yi_DBF_H, and Yi be the 8 adjacent luma samples at the input of DBF_V, DBF_H, and SAO, respectively, where i=1 to 8.

[0081] Max Y0=max(Y0_DBF_V,Y0_DBF_H,Y0_DBF) Max Yi=max(Yi_DBF_V,Yi_DBF_H,Yi_DBF) max Y0 and max Yi are given as CCSAO classifications.

[0082] Figure 8 is a block diagram illustrating that the CCSAO process is interleaved into vertical and horizontal DBFs according to some implementations of the present disclosure. In some embodiments, the CCSAO blocks of Figures 6, 7, and 8 may be optional. For example, use the DBF_V luma sample input as the CCSAO input while using Y0_DBF_V and Yi_DBF_V for the first CCSAO_V applying the same sample processing as in Figure 6.

[0083] In some embodiments, the CCSAO syntax implemented is shown in Table 2 below.

[0084] [Table 2] Table 2: An example of CCSAO syntax In some embodiments, when one additional chroma offset is signaled to signal the CCSAO Cb and Cr offset values, the other chroma component offset can be derived by positive or negative sign, or by weighting to save bit overhead. For example, let h_Cb and h_Cr be the offsets of CCSAO Cb and Cr, respectively. By explicitly signaling w, when w=±|w| and the |w| candidates are limited, h_Cr can be derived from h_Cb without explicitly signaling h_Cr itself.

[0085] h_Cr=w*h_Cb FIG. 9 is a flow diagram illustrating an example process 900 for decoding a video signal using inter-component correlation according to some implementations of the present disclosure.

[0086] Video decoder 30 receives a video signal including a first component and a second component (910). In some embodiments, the first component is a luma component and the second component is a chroma component of the video signal.

[0087] Video decoder 30 also receives a number of offsets associated with the second component (920).

[0088] Video decoder 30 then utilizes the characteristic measure of the first component to obtain a classification category associated with the second component (930). For example, in FIG. 6, the current chroma sample 602 is first same position A luma sample 604 and an adjacent (white) luma sample 606 are used to sort and add the corresponding CCSAO offset value to the current chroma sample.

[0089] Video decoder 30 further selects a first offset from the plurality of offsets for the second component according to the classification category (940).

[0090] Video decoder 30 additionally modifies the second component based on the selected first offset (950).

[0091] In some embodiments, obtaining a classification category associated with the second component using the characteristic measurement of the first component (930) includes obtaining a classification category for each sample of the second component using a respective sample of the first component, the respective sample of the first component being a classification category for each sample of the second component. same position For example, the current chroma sample classification is same position It reuses the SAO types (EO or BO), classes, and categories from the Luma sample.

[0092] In some embodiments, utilizing the characteristic measure of the first component to obtain a classification category associated with the second component (930) includes utilizing a respective sample of the first component to obtain a respective classification category for each respective sample of the second component, where each sample of the first component is reconstructed before being deblocked or reconstructed after being deblocked. In some embodiments, the first component is deblocked with a deblocking filter (DBF). In some embodiments, the first component is deblocked with a luma deblocking filter (DBF Y). For example, as an alternative to FIG. 6 or FIG. 7, the CCSAO input may be before DBF Y.

[0093] In some embodiments, the characteristic measure is derived by dividing a range of sample values ​​of the first component into bands and selecting bands based on intensity values ​​of samples in the first component, hi some embodiments, the characteristic measure is derived from a band offset (BO).

[0094] In some embodiments, the characteristic measure is derived based on a direction and intensity of edge information of the samples in the first component. In some embodiments, the characteristic measure is derived from an edge offset (EO).

[0095] In some embodiments, modifying the second component (950) includes adding the selected first offset directly to the second component, for example, a corresponding CCSAO offset value is added to the current chroma component sample.

[0096] In some embodiments, modifying the second component (950) includes mapping the selected first offset to a second offset and adding the mapped second offset to the second component. For example, if one additional chroma offset is signaled to signal the CCSAO Cb and Cr offset values, the other chroma component offset can be derived by using a positive or negative sign or by weighting to save bit overhead.

[0097] In some embodiments, receiving (910) the video signal includes receiving a syntax element indicating whether a method for decoding the video signal using CCSAO is enabled for the video signal in a sequence parameter set (SPS). In some embodiments, cc_sao_enabled_flag indicates whether CCSAO is enabled at the sequence level.

[0098] In some embodiments, receiving (910) the video signal includes receiving a syntax element indicating whether decoding the video signal using CCSAO is enabled for the second component at a slice level. In some embodiments, slice_cc_sao_cb_flag or slice_cc_sao_cr_flag indicates whether CCSAO is enabled in the slice for Cb or Cr, respectively.

[0099] In some embodiments, receiving (920) multiple offsets associated with the second component includes receiving different offsets for different coding tree units (CTUs), where for a CTU, cc_sao_offset_sign_flag indicates a sign for the offset and cc_sao_offset_abs indicates CCSAO Cb and Cr offset values ​​for the current CTU.

[0100] In some embodiments, receiving (920) a plurality of offsets associated with the second component includes receiving a syntax element indicating whether the received offset of the CTU is the same as one of the CTU's neighboring CTUs, where the neighboring CTU is a left or upper neighboring CTU. For example, cc_sao_merge_up_flag indicates whether the CCSAO offset is merged from a left or upper CTU.

[0101] In some embodiments, the video signal further includes a third component, and the method of decoding the video signal using the CCSAO further includes receiving a second plurality of offsets associated with the third component, utilizing a characteristic measurement of the first component to obtain a second classification category associated with the third component, selecting a third offset from the second plurality of offsets for the third component according to the second classification category, and modifying the third component based on the selected third offset.

[0102] FIG. 11 illustrates all of the exemplary implementations of the present disclosure. same position 6A, 6B, and 11 show the inputs of the CCSAO classification. In FIG. 11, the current chroma sample is 1104, and the inter-component same position The chroma sample is 1102, same position The luma sample is 1106.

[0103] In some embodiments, the exemplary classifier (C0) is same position The luma or chroma sample value (Y0) (Y4 / U4 / V4 in Figure 6B and Figure 6C) is used for classification. Let band_num be the number of equally divided bands of the luma or chroma dynamic range and bit_depth be the sequence bit depth. An example class index for the current chroma sample is:

[0104] class(C0)=(Y0*band_num)>>bit_depth In some embodiments, this classification takes into account rounding, for example:

[0105] class(C0)=((Y0*band_num)+(1<<bit_depth))> >bit_depth Some band_num and bit_depth examples are given below in Table 3. Table 3 shows three classification examples when the number of bands for each of the classification examples is different.

[0106] [Table 3] Table 3: Example band_num and bit_depth for each class index In some embodiments, the classifier uses a different luma sample location for the C0 classification. Figure 10A is a block diagram illustrating a classifier that uses a different luma (or chroma) sample location for the C0 classification, for example, using adjacent Y7 instead of Y0 for the C0 classification, according to some implementations of the present disclosure.

[0107] In some embodiments, different classifiers can be switched at sequence parameter set (SPS) / adaptation parameter set (APS) / picture parameter set (PPS) / picture header (PH) / slice header (SH) / coding tree unit (CTU) / coding unit (CU) level. For example, in Figure 10, we use Y0 for POC0 but Y7 for POC1, as shown in Table 4 below.

[0108] [Table 4] Table 4: Different classifiers are applied to different pictures In some embodiments, FIG. 10B shows some examples of different shapes for luma candidates according to some implementations of the present disclosure. For example, constraints may be applied to the shapes. In some cases, the total number of luma candidates must be a power of two, as shown in FIG. 10B(b)(c)(d). In some cases, the number of luma candidates must be horizontally and vertically symmetric about the chroma sample (center), as shown in FIG. 10B(a)(c)(d)(e). In some embodiments, the power of two constraint and the symmetry constraint may also be applied to the chroma candidates. The U / V portions of FIG. 6B and FIG. 6C show an example for the symmetry constraint. In some embodiments, different color formats may have different classifier "constraints." For example, the 420 color format uses the luma / chroma candidate selection shown in Figures 6B and 6C (one candidate is selected from a 3x3 shape), while the 444 color format uses Figure 10B(f) for luma and chroma candidate selection, and the 422 color format uses Figure 10B(g) for luma (two chroma samples share four luma candidates) and Figure 10B(f) for chroma candidates.

[0109] In some embodiments, C0 position and C0 band_num may be combined and switched at SPS / APS / PPS / PH / SH / CTU / CU levels. Different combinations may result in different classifiers, as shown in Table 5 below.

[0110] [Table 5] Table 5: Different combinations of classifiers and number of bands are applied to different pictures In some embodiments, same position The luma sample value (Y0) is same position and a value (Yp) obtained by weighting adjacent luma samples. same position The luma sample value is same position and replacing it with a value obtained by weighting adjacent luma samples. same position The luma sample value (Y0) may be replaced with a phase correction value (Yp) obtained by weighting adjacent luma samples. Different Yp may be different classifiers.

[0111] In some embodiments, different Yp are applied for different chroma formats, for example, in Figure 12, Yp in (a) is used for 420 chroma format, Yp in (b) is used for 422 chroma format, and Y0 is used for 444 chroma format.

[0112] In some embodiments, the other classifier (C1) is same position The comparison scores of a luma sample (Y0) and its eight adjacent luma samples [-8,8], resulting in a total of 17 classes, as shown below.

[0113] Initial class (C1) = 0, loop through 8 adjacent luma samples (Yi,i = 1 to 8) if Y0>Yi Class+=1 else if Y0 <Yi Class-=1 In some embodiments, the variation (C1') counts only the comparison score [0,8], which results in 8 classes. (C1,C1') are classifier groups, and a PH / SH level flag can be signaled to switch between C1 and C1'.

[0114] Initial class (C1') = 0, loop through 8 adjacent luma samples (Yi,i = 1 to 8) if Y0>Yi Class+=1 In some embodiments, the variation (C1) selectively uses adjacent N out of M adjacent samples to count the comparison score. An M-bit bitmask may be signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock level to indicate which adjacent samples were selected to count the comparison score. Using FIG. 6B as an example for the luma classifier, 8 adjacent luma samples are candidates and an 8-bit bitmask (01111110) is signaled at PH, which indicates that 6 samples Y1-Y6 are selected, and thus the comparison score is [-6,6], resulting in an offset of 13. The selective classifier C1 gives the encoder more options to trade off between offset signaling overhead and classification granularity.

[0115] Similar to C1, the perturbation (C1') only counts the comparison score [0,+N], so the previous example bitmask 01111110 gives a comparison score of [0,6], resulting in an offset of 7.

[0116] In some embodiments, different classifiers are combined to result in a generic classifier, for example, different classifiers are applied to different pictures (different POC values), as shown in Table 6-1 below.

[0117] [Table 6] Table 6-1: Different generic classifiers apply to different pictures In some embodiments, another exemplary classifier (C3) uses a bit mask for classification, as shown in Table 6-2. To indicate this classifier, a 10-bit bit mask is signaled at the SPS / APS / PPS / PH / SH / Region / CTU / CU / Sub-block level. For example, a bit mask of 11 1100 0000 means that for a given 10-bit luma sample value, only the most significant bits (MSBs): 4 bits are used for classification, resulting in a total of 16 classes. Another exemplary bit mask of 10 0100 0001 means that only 3 bits are used for classification, resulting in a total of 8 classes.

[0118] In some embodiments, the bitmask length (N) may be fixed or switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock level. For example, for a 10-bit sequence, a 4-bit bitmask 1110 is signaled in the picture with PH, and the MSB 3 bits b9, b8, b7 are used for classification. Another example is a 4-bit bitmask 0011 on the LSBs, and b0, b1 are used for classification. The bitmask classifier can correspond to luma or chroma classification. Whether to use the MSB or LSB for the bitmask N may be fixed or switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock level.

[0119] In some embodiments, the luma position and C3 bitmask may be combined and can be switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock level. Different combinations may result in different classifiers.

[0120] In some embodiments, a bitmask restriction "max # of 1s" may be applied to limit the number of corresponding offsets. For example, limiting the bitmask "max # of 1s" to 4 in an SPS results in a maximum offset of 16 for a sequence. Bitmasks for different POCs may be different, but the "max # of 1s" shall not exceed 4 (and shall not exceed 16 for all classes). The "max # of 1s" value may be signaled and can be switched at the SPS / APS / PPS / PH / SH / Region / CTU / CU / Sub-block level.

[0121] [Table 7] Table 6-2: An example classifier uses bitmasks for classification (bitmask positions are underlined) In some embodiments, as shown in FIG. 11, other inter-component chroma samples, e.g., chroma sample 1102 and its neighbors, may also be provided to the CCSAO classification, e.g., for the current chroma sample 1104. For example, the Cr chroma sample may be provided to the CCSAO Cb classification. The Cb chroma sample may be provided to the CCSAO Cr classification. The classifier for the inter-component chroma sample may be the same as the luma inter-component classifier, or may have its own classifier, as described in this disclosure. The two classifiers may be combined to form a joint classifier for classifying the current chroma sample. For example, the joint classifier combining the inter-component luma sample and the chroma sample results in a total of 16 classes, as shown in Table 6-3 below.

[0122] [Table 8] Table 6-3: Exemplary classifier using joint classifier combining inter-component luma and chroma samples (bitmask positions are underlined) All the above mentioned classifiers (C0, C1, C1', C2, C3) can be combined. See for example Table 6-4 below.

[0123] [Table 9] Table 6-4: Different classifiers are combined In some embodiments, an exemplary classifier (C2) is: same position and the difference of adjacent luma samples (Yn). Figure 12(c) shows an example of Yn, which has a dynamic range of [-1024, 1023] when the bit depth is 10. Let band_num in C2 be the number of equally divided bands of the Yn dynamic range.

[0124] Class(C2)=(Yn+(1<<bit_depth)*band_num)> > (bit_depth+1).

[0125] In some embodiments, C0 and C2 are combined to result in a generic classifier. For example, different classifiers are applied to different pictures (different POCs), as shown in Table 7 below.

[0126] [Table 10] Table 7: Different generic classifiers are applied to different pictures In some embodiments, all the above mentioned classifiers (C0, C1, C1', C2) are combined. For example, different classifiers are applied to different pictures (different POCs), as shown in Table 8 below.

[0127] [Table 11] Table 8: Different generic classifiers are applied to different pictures In some embodiments, multiple classifiers are used in the same POC. The current frame is divided by several regions, and each region uses the same classifier. For example, three different classifiers are used in POC0, and which classifier (0, 1, or 2) is used is signaled at the CTU level, as shown in Table 9 below.

[0128] [Table 12] Table 9: Different generic classifiers are applied to different regions in the same picture In some embodiments, the maximum number of multiple classifiers (multiple classifiers can also be called alternative offset sets) may be fixed or signaled at the SPS / APS / PPS / PH / SH / CTU / CU level. In one example, the fixed (predefined) maximum number of multiple classifiers is 4. In that case, 4 different classifiers are used at POC0, and which classifier (0, 1, or 2) is used is signaled at the CTU level. Abbreviated unary (TU) codes may be used to indicate the classifiers used for each luma or chroma CTB. For example, as shown in Table 10 below, when the TU code is 0, CCSAO is not applied, when the TU code is 10, set 0 is applied, when the TU code is 110, set 1 is applied, when the TU code is 1110, set 2 is applied, and when the TU code is 1111, set 3 is applied. Fixed length codes, Golomb-Rice codes, and exponential-Golomb codes may be used to indicate the classifiers for the CTB (offset set index). Three different classifiers are used in POC1.

[0129] [Table 13] Table 10: Abbreviated unary (TU) codes are used to indicate the classifier used for each chroma CTB An example of CTB offset set index for Cb and Cr is given in 1280x720 sequence POC0 (if CTU size is 128x128, the number of CTUs in a frame is 10x6). Cb in POC0 uses 4 offset sets and Cr uses 1 offset set. As shown in Table 11-1 below, when the offset set index is 0, CCSAO is not applied, when the offset set index is 1, set 0 is applied, when the offset set index is 2, set 1 is applied, when the offset set index is 3, set 2 is applied, and when the offset set index is 4, set 3 is applied. The type is determined by the selected same position Refers to the position of the luma sample (Yi). Different offset sets can have different types, band_num, and corresponding offsets.

[0130] [Table 14] Table 11-1: An example of CTB offset set index for Cb and Cr is given for a 1280x720 sequence POC0 (if the CTU size is 128x128, the number of CTUs in the frame is 10x6) In some embodiments, same position An example of jointly using current and adjacent Y / U / V samples for classification is given in Table 11-2 below (three-component joint bandNum classification for each Y / U / V component). At POC0, {2,4,1} offset set is used for {Y,U,V} respectively. Each offset set can be adaptively switched at SPS / APS / PPS / PH / SH / CTU / CU / sub-block level. Different offset sets can have different classifiers. For example, to classify the current Y4 luma sample as the candidate position (candPos) shown in Figure 6B and Figure 6C, Y set0 is used for {current Y4, same position U4, same positionV4} are selected as candidates, with different bandNum{Y,U,V}={16,1,2}. When {candY,candU,candV} are used as the sample values ​​of the selected {Y,U,V} candidates, the total number of classes is 32, and the class index derivation can be shown as follows:

[0131] bandY=(candY*bandNumY)>>BitDepth; bandU=(candU*bandNumU)>>BitDepth; bandV=(candV*bandNumV)>>BitDepth; classIdx=bandY*bandNumU*bandNumV +bandU*bandNumV +bandV; Another example is the component V set1 classification in POC1, where candPos={neighbor Y8, neighbor U3, neighbor V0} is used with bandNum={4,1,2}, resulting in 8 classes.

[0132] [Table 15] Table 11-2: same position An example of using current and adjacent Y / U / V samples jointly for classification In some embodiments, for example as shown in Table 11-3 below: same position Here is an example of jointly using Y4 and adjacent Y / U / V samples for the current Y / U / V sample classification (three-component joint edgeNum(C1) and bandNum classification for each Y / U / V component). Edge CandPos is the center position used for the C1 classifier, edge bitMask is the C1 adjacent sample activation indicator, and edgeNum is the number of the corresponding C1 class. In this example, C1 is only applied to the Y classifier (hence edgeNum is equal to edgeNumY), and edge candPos is always Y4 (current / same positionHowever, C1 may be applied to a Y / U / V classifier that has the edge candPos as an adjacent sample position.

[0133] If diff denotes the C1 comparison score of Y, then the classIdx derivation could be:

[0134] bandY=(candY*bandNumY)>>BitDepth; bandU=(candU*bandNumU)>>BitDepth; bandV=(candV*bandNumV)>>BitDepth; edgeIdx=diff+(edgeNum>>1); bandIdx=bandY*bandNumU*bandNumV +bandU*bandNumV +bandV; classIdx=bandIdx*edgeNum+edgeIdx;

[0135] [Table 16] Table 11-3 (1 copy): same position An example of using current and adjacent Y / U / V samples jointly for classification

[0136] [Table 17] Table 11-3 (2 parts): same position An example of using current and adjacent Y / U / V samples jointly for classification

[0137] [Table 18] Table 11-3 (3 parts): same position An example of using current and adjacent Y / U / V samples jointly for classification In some embodiments, the max band_num (bandNumY, bandNumU, or bandNumV) may be fixed or signaled at the SPS / APS / PPS / PH / SH / CTU / CU level. For example, if the decoder fixes max band_num=16 for each frame, then 4 bits are signaled to indicate the band_num of C0 in the frame. Some other max band_num examples are listed in Table 12 below.

[0138] [Table 19] Table 12: Examples of maximum band_num and band_num bits In some embodiments, the maximum number of classes or offsets (combinations using multiple classifiers jointly, e.g., C1 edgeNum*C1 bandNumY*bandNumU*bandNumV) for each set (or all added sets) may be fixed or signaled at SPS / APS / PPS / PH / SH / Region / CTU / CU level. For example, max may be fixed at class_num=256*4 for all added sets and an encoder conformance check or a decoder norm check may be used to verify the constraint.

[0139] In some embodiments, restrictions may be applied to the C0 classification, for example restricting band_num (bandNumY, bandNumU, or bandNumV) to only power-of-two values. Instead of explicitly signaling band_num, the syntax band_num_shift is signaled. The decoder can use a shift operation to avoid multiplications. Different band_num_shifts can be used for different components.

[0140] class(C0)=(Y0>>band_num_shift)>>bit_depth Another example operation considers rounding to reduce error.

[0141] class(C0)=((Y0+(1<<(band_num_shift-1)))>>band_num_shift)>>bit_depth For example, if band_num_max (Y, U, or V) is 16, then the possible band_num_shift candidates are 0, 1, 2, 3, 4, corresponding to band_num=1, 2, 4, 8, 16, as shown in Table 13.

[0142] [Table 20]

[0143] [Table 21] Table 13: band_num and corresponding band_num_shift candidates In some embodiments, the classifiers applied to Cb and Cr are different. The Cb and Cr offsets for all classes can be signaled separately. For example, different offsets are signaled and applied to different chroma components, as shown in Table 14 below.

[0144] [Table 22] Table 14: Cb and Cr offsets for all classes can be signaled separately In some embodiments, the maximum offset value is fixed or signaled in the sequence parameter set (SPS) / adaptation parameter set (APS) / picture parameter set (PPS) / picture header (PH) / slice header (SH). For example, the maximum offset is [-15,15]. Different components can have different maximum offset values.

[0145] In some embodiments, the signaling of the offset may use differential pulse code modulation (DPCM), for example, the offset {3, 3, 2, 1, -1} may be signaled as {3, 0, -1, -1, -2}.

[0146] In some embodiments, the offsets may be stored in an APS or memory buffer for reuse for the next picture / slice, and an index may be signaled to indicate which stored previous frame offset is to be used for the current picture.

[0147] In some embodiments, the classifiers for Cb and Cr are the same. The Cb and Cr offsets for all classes may be signaled jointly, for example as shown in Table 15 below.

[0148] [Table 23] Table 15: Cb and Cr offsets for all classes can be jointly signaled In some embodiments, the classifiers for Cb and Cr may be the same. For example, the Cb and Cr offsets for all classes may be jointly signaled by a sign flag difference, as shown in Table 16 below. According to Table 16, when the Cb offset is (3,3,2,-1), the derived Cr offset is (-3,-3,-2,1).

[0149] [Table 24] Table 16: Cb and Cr offsets for all classes may be jointly signaled by the sign flag difference In some embodiments, a sign flag may be signaled for each class, for example as shown in Table 17 below: According to Table 17, when the Cb offset is (3,3,2,-1), the derived Cr offset is (-3,3,2,1) according to the respective sign flags.

[0150] [Table 25] Table 17: Cb and Cr offsets for all classes may be jointly signaled by the sign flag signaled for each class In some embodiments, the classifiers for Cb and Cr may be the same. The Cb and Cr offsets for all classes may be jointly signaled by weight differences, for example as shown in Table 18 below. The weights (w) may be selected in a restricted table, for example ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4..., etc., where |w| includes only power-of-two values. According to Table 18, when the Cb offset is (3,3,2,-1), the derived Cr offset is (-6,-6,-4,2) according to the respective sign flags.

[0151] [Table 26] Table 18: Cb and Cr offsets for all classes can be jointly signaled by weight differences In some embodiments, weights may be signaled for each class, for example as shown in Table 19 below: According to Table 19, when the Cb offset is (3, 3, 2, -1), the derived Cr offset is (-6, 12, 0, -1) according to the respective sign flags.

[0152] [Table 27] Table 19: Cb and Cr offsets for all classes can be jointly signaled with weights signaled for each class In some embodiments, when multiple classifiers are used in the same POC, different offset sets are signaled separately or jointly.

[0153] In some embodiments, previously decoded offsets may be stored for use in future frames. To reduce the overhead of offset signaling, an index may be signaled to indicate which previously decoded offset set is used for the current frame. For example, as shown in Table 20 below, POC0 offsets may be reused by POC2 by signaling offset set idx=0.

[0154] [Table 28] Table 20: An index can be signaled to indicate which previously decoded offset set is used for the current frame In some embodiments, the reuse offsets set idx for Cb and Cr may be different, for example as shown in Table 21 below.

[0155] [Table 29] Table 21: An index may be signaled to indicate which previously decoded offset set is used for the current frame, the index may be different for the Cb and Cr components In some embodiments, offset signaling can use additional syntax including start and length to reduce signaling overhead. For example, when band_num=256, only offsets from band_idx=37 to 44 are signaled. In the example in Table 22-1 below, the syntax for start and length are both 8-bit fixed length codes and should be aligned to the band_num bits.

[0156] [Table 30] Table 22-1: Signalling offsets uses additional syntax including start and length In some embodiments, when CCSAO is applied to all YUV triplets: same position and adjacent YUV samples may be used jointly for classification, and all of the above described offset signaling methods for Cb / Cr may be extended to Y / Cb / Cr. In some embodiments, different component offset sets may be stored and used separately (each component has its own stored set) or jointly (each component shares / reuses the same stored set). Examples of separate sets are shown in Table 22-2 below.

[0157] [Table 31] JPEG0007654811000032.jpg21167 Table 22-2: An example showing how different component offset sets may be stored and used separately (each component has its own stored set) or jointly (each component shares / reuses the same stored set) In some embodiments, if the sequence bit depth is greater than 10 (or a specific bit depth), the offset may be quantized before signaling. At the decoder side, the decoded offset is dequantized before application, as shown in Table 23 below. For example, for a 12-bit sequence, the decoded offset is left shifted (dequantized) by 2.

[0158] [Table 32] Table 23: The decoded offset is dequantized before application In some embodiments, the offset may be calculated as CcSaoOffsetVal=(1-2*ccsao_offset_sign_flag)*(ccsao_offset_abs<<(BitDepth-Min(10,BitDepth))).

[0159] In some embodiments, the sample processing is described below. Let R(x,y) be the input luma or chroma sample value before CCSAO, and R’(x,y) be the output luma or chroma sample value after CCSAO.

[0160] offset = ccsao_offset[class_index of R(x,y)] R’(x,y) = Clip3(0, (1<<bit_depth)-1, R(x,y) + offset) According to the above equation, each luma or chroma sample value R(x,y) is classified using the classifier indicated by the current picture and / or the current offset set idx. The corresponding offset of the derived class index is added to each luma or chroma sample value R(x,y). The clip function Clip3 is applied to (R(x,y) + offset) to create an output luma or chroma sample value R’(x,y) within the bit depth dynamic range, e.g., in the range from 0 to (1<<bit_depth)-1.

[0161] In some embodiments, the boundary processing is described below. If any of the same position and adjacent luma (chroma) samples used for classification is located outside the current picture, CCSAO is not applied to the current chroma (luma) sample. FIG. 13A shows, according to some implementation examples of the present disclosure, that same position if any of the and adjacent luma (chroma) samples used for classification is located outside the current picture, CCSAO is not applied to the current chroma (luma) sample. For example, in FIG. 13A(a), when the classifier is used, CCSAO is not applied to the chroma components in the leftmost column of the current picture. For example, when C1’ is used, as shown in FIG. 13A(b), CCSAO is not applied to the chroma components in the leftmost column and the topmost row of the current picture.

[0162] FIG. 13B shows, according to some implementation examples of the present disclosure, that the same position13B(b) is a block diagram showing that the CCSAO is applied to the current luma or chroma sample if any of the neighboring luma or chroma samples are located outside the current picture. In some embodiments, the variation is used for classification, as shown in FIG. 13B(b). same position And if any of the adjacent luma or chroma samples are located outside the current picture, the missing samples may be used iteratively or mirror-padded to create a sample for classification, as shown in FIG. 13B(a), and CCSAO may be applied to the current luma or chroma sample.

[0163] FIG. 14 illustrates a corresponding selected image used for classification according to some implementations of the present disclosure. same position 14 is a block diagram showing that the CCSAO is not applied to the current chroma sample if the adjacent luma sample is outside the virtual space defined by the virtual boundary. In some embodiments, the virtual boundary (VB) is a virtual line separating spaces in a picture frame. In some embodiments, when the virtual boundary (VB) is applied in the current frame, the CCSAO is not applied to the chroma samples that selected corresponding luma positions that are outside the virtual space defined by the virtual boundary. FIG. 14 shows an example of virtual boundaries for a C0 classifier with 9 luma position candidates. For each CTU, the CCSAO is not applied to the chroma samples whose corresponding selected luma positions are outside the virtual space enclosed by the virtual boundary. For example, in FIG. 14(a), the CCSAO is not applied to the chroma sample 1402 when the selected Y7 luma sample position is on the opposite side of the horizontal virtual boundary 1406 that is located on a line of 4 pixels from the bottom of the frame. For example, in FIG. 14(b), when the selected Y5 luma sample position is located on the opposite side of the vertical virtual boundary 1408 that is located y pixels in a line from the right edge of the frame, CCSAO is not applied to the chroma sample 1404.

[0164] FIG. 15 illustrates that, according to some implementations of the present disclosure, repetition or mirror padding may be applied to luma samples outside the virtual boundary. FIG. 15(a) illustrates an example of repetition padding. If the original Y7 is selected to be the classifier located at the bottom of VB1502, the Y4 luma sample value is used for classification (copied to the Y7 position) instead of the original Y7 luma sample value. FIG. 15(b) illustrates an example of mirror padding. If Y7 is selected to be the classifier located at the bottom of VB1504, the Y1 luma sample value, which is symmetric to the Y7 value with respect to the Y0 luma sample, is used for classification instead of the original Y7 luma sample value. The padding method gives the possibility to apply CCSAO to more chroma samples, and therefore more coding gain may be realized.

[0165] In some embodiments, restrictions may be applied to reduce the line buffer required by the CCSAO and simplify boundary processing condition checks. FIG. 16 illustrates the use of all nine same position and shows that if adjacent luma samples are used for classification, an additional 1 luma line buffer may be required, i.e., full line luma samples of line -5 above the current VB1602. Figure 10B(a) shows an example where only 6 luma candidates are used for classification, which reduces the line buffer and does not require any additional boundary checks in Figures 13A and 13B.

[0166] In some embodiments, using luma samples for CCSAO classification may increase the luma line buffer and therefore increase the decoder hardware implementation cost. FIG. 17 illustrates that in some implementations of the present disclosure, in AVS, nine luma candidate CCSAOs crossing VB 1702 may increase two additional luma line buffers. For luma and chroma samples above the virtual boundary (VB) 1702, DBF / SAO / ALF are processed in the current CTU row. For luma and chroma samples below VB 1702, DBF / SAO / ALF are processed in the next CTU row. In the AVS decoder hardware design, the pre-DBF samples of luma lines -4 to -1, pre-SAO samples of line -5, and pre-DBF samples of chroma lines -3 to -1, pre-SAO samples of line -4 are stored as line buffers for DBF / SAO / ALF processing of the next CTU row. When processing the next CTU row, luma and chroma samples that are not in the line buffer cannot be used. However, for example, in the chroma line-3(b) position, a chroma sample is processed in the next CTU row, but CCSAO requires SAO pre-luma sample lines-7, -6, and -5 for classification. SAO pre-luma sample lines-7, -6 are not in the line buffer and therefore cannot be used. Adding SAO pre-luma sample lines-7 and -6 to the line buffer increases the decoder hardware implementation cost. In some examples, luma VB (line-4) and chroma VB (line-3) may be different (not aligned).

[0167] Similar to FIG. 17, FIG. 18A illustrates that in VVC, nine luma candidate CCSAOs intersecting VB 1802 may increase one additional luma line buffer according to some implementations of the present disclosure. VB may be different in different standards. In VVC, luma VB is line-4 and chroma VB is line-2, so nine candidate CCSAOs may increase one luma line buffer.

[0168] In some embodiments, in the first solution, if any of the luma candidates of a chroma sample crosses VB (outside the current chroma sample VB), the CCSAO is disabled for the chroma sample. Figures 19A-19C show that in some implementations of the present disclosure, if any of the luma candidates of a chroma sample crosses VB1902 (outside the current chroma sample VB), the CCSAO is disabled for the chroma sample in AVS and VVC. Figure 14 also shows some examples of this implementation.

[0169] In some embodiments, in the second solution, for the "crossed VB" luma candidates, repetition padding is used for CCSAO from a luma line close to VB and on the opposite side of VB, e.g., luma line -4. In some embodiments, repetition padding from the luma closest to the neighboring luma below VB is implemented for the "crossed VB" chroma candidates. Figures 20A-20C show that, according to some implementation examples of the present disclosure, in AVS and VVC, CCSAO is enabled using repetition padding for a chroma sample if any of the luma candidates of the chroma sample crosses VB2002 (outside the current chroma sample VB). Figure 14(a) also shows some examples of this implementation example.

[0170] In some embodiments, in the third solution, mirror padding is used for CCSAO from below the luma VB for the "crossed VB" luma candidates. Figures 21A-21C show that some implementation examples of the present disclosure enable CCSAO using mirror padding for a chroma sample in AVS and VVC if any of the luma candidates of the chroma sample crosses VB2102 (outside the current chroma sample VB). Figures 14(b) and 13B(b) also show some examples of this implementation. In some embodiments, in the fourth solution, "two-sided symmetric padding" is used to apply the CCSAO. Figures 22A-22B show that some implementation examples of the present disclosure enable CCSAO using two-sided symmetric padding for some examples of different CCSAO shapes (e.g., 9 luma candidates (Figure 22A) and 8 luma candidates (Figure 22B)). Same position For a luma sample set with a central luma sample, if one side of the luma sample set is outside of VB2 202, then double-sided symmetric padding is applied to both sides of the luma sample set. For example, in Figure 22A, luma samples Y0, Y1, and Y2 are outside of VB2 202, and therefore Y0, Y1, Y2 and Y6, Y7, Y8 are both padded using Y3, Y4, Y5. For example, in Figure 22B, luma sample Y0 is outside of VB2 202, and therefore Y0 is padded using Y2 and Y7 is padded using Y5.

[0171] FIG. 18B illustrates a method for classifying a current luma sample according to some implementations of the present disclosure. same position Alternatively, adjacent chroma samples may be used and the selected chroma candidate may cross VB, requiring additional chroma line buffers. Solutions 1-4 similar to those described above may be applied to address this issue.

[0172] Solution 1 is to disable CCSAO for a luma sample when any of its chroma candidates may cross VB.

[0173] Solution 2 is to use repeated padding from the chroma closest to the adjacent chroma below VB for the "cross VB" chroma candidate.

[0174] Solution 3 is to use mirror padding from below chroma VB for the "intersect VB" chroma candidate.

[0175] Solution 4 is to use "double-sided symmetric padding". same position For a candidate set located at the center of the chroma sample, if one side of the candidate set is outside the VB, double-sided symmetric padding is applied on both sides.

[0176] The padding method offers the possibility to apply CCSAO to more luma or chroma samples, and therefore more coding gain can be realized.

[0177] In some embodiments, in a bottom picture (or slice, tile, brick) boundary CTU row, samples below VB are processed in the current CTU row, so the above special handling (solutions 1, 2, 3, 4) is not applied to this bottom picture (or slice, tile, brick) boundary CTU row. For example, a 1920x1080 frame is divided into 128x128 CTUs. The frame contains 15x9 CTUs (rounded up). The bottom CTU row is the 15th CTU row. The decoding process is done CTU row by CTU, and for each CTU row, deblocking needs to be applied along the horizontal CTU boundary between the current CTU row and the next CTU row. Inside one CTU, in the bottom 4 / 2 luma / chroma line, the VB of the CTB is applied to each CTU row, because the DBF samples (for VVC) are processed in the next CTU row and are not available for the CCSAO of the current CTU row. However, for the bottom CTU row of the picture frame, since there is no next CTU row remaining, the bottom 4 / 2 luma / chroma line DBF samples are available in the current CTU row and are DBF processed in the current CTU row.

[0178] In some embodiments, restrictions may be applied to reduce the line buffer required by the CCSAO and simplify boundary processing condition checks, as described in Figure 16. Figure 23 illustrates restrictions to use a limited number of luma candidates for classification, according to some implementations of the present disclosure. Figure 23(a) illustrates restrictions to use only six luma candidates for classification. Figure 23(b) illustrates restrictions to use only four luma candidates for classification.

[0179] In some embodiments, an application region is implemented. The CCSAO application region unit can be based on a CTB, i.e., on / off control, CCSAO parameters (offset used for classification offset set index, luma candidate position, band_num, bitmask... etc.) are the same within one CTB.

[0180] In some embodiments, the application region may not be aligned to the CTB boundary. For example, the application region is not aligned to the chroma CTB boundary, but is shifted. Although the syntax (on / off control, CCSAO parameters) is still shown for each CTB, the true application region is not aligned to the CTB boundary. FIG. 24 shows that, according to some implementations of the present disclosure, the CCSAO application region is not aligned to the CTB / CTU boundary 2406. For example, the application region is not aligned to the chroma CTB / CTU boundary 2406, but is sample shifted (4,4) up-left relative to the VB 2408. This unaligned CTB boundary design benefits the deblocking process because the same deblocking parameters are used for each 8×8 deblocking process region.

[0181] In some embodiments, the CCSAO application region unit (mask size) may be variable (larger or smaller than the CTB size), as shown in Table 24. The mask size may be different for different components. The mask size may be switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block level. For example, in the PH, a series of mask on / off flags and offset set indexes are shown to indicate each CCSAO region information.

[0182] [Table 33] Table 24: CCSAO application area unit (mask size) may be variable In some embodiments, the CCSAO applicability region frame partition may be fixed, for example, by dividing the frame into N regions. Figure 25 illustrates that the CCSAO applicability region frame partition may be fixed by a CCSAO parameter according to some implementations of the present disclosure.

[0183] In some embodiments, each region may have its own region on / off control flag and CCSAO parameters. Also, if the region size is larger than the CTB size, it may have both a CTB on / off control flag and a region on / off control flag. Figures 25(a) and (b) show some examples of dividing a frame into N regions. Figure 25(a) shows a vertical division consisting of four regions. Figure 25(b) shows a square division consisting of four regions. In some embodiments, similar to the picture level CTB on all-on control flag (ph_cc_sao_cb_ctb_control_flag / ph_cc_sao_cr_ctb_control_flag), if the region on / off control flag is off, the CTB on / off flag may be further signaled. Otherwise, the CCSAO applies to all CTBs in this region without further signaling the CTB flag.

[0184] In some embodiments, different CCSAO application regions can share the same region on / off control and CCSAO parameters. For example, in Figure 25(c), regions 0-2 share the same parameters, and regions 3-15 share the same parameters. Figure 25(c) also shows that the region on / off control flags and CCSAO parameters can be signaled in Hilbert scan order.

[0185] In some embodiments, the CCSAO application domain unit can be a quadtree / binarytree / tertile split from the picture / slice / CTB level. Similar to the CTB split, a series of split flags are signaled to indicate the CCSAO application domain partition. Figure 26 shows that according to some implementation examples of the present disclosure, the CCSAO application domain can be a binary tree (BT) / quadtree (QT) / tertile (TT) split from the frame / slice / CTB level.

[0186] FIG. 27 is a block diagram illustrating multiple classifiers used and switched at different levels within a picture frame according to some implementations of the present disclosure. In some embodiments, when multiple classifiers are used in a frame, the method of applying the classifier set index may be switched at SPS / APS / PPS / PH / SH / region / CTU / CU / subblock level. For example, four classifier sets are used in a frame and switched at PH, as shown in Table 25 below. FIG. 27(a) and FIG. 27(c) show the default fixed region classifier. FIG. 27(b) shows the classifier set index is signaled at the mask / CTB level, where 0 means CCSAO off for this CTB, and 1-4 means set index.

[0187] [Table 34] Table 25: Four classifier sets are used within a frame and switched with PH In some embodiments, for a default region, the region level flag may be signaled if the CTB in this region does not use the default set index (e.g., the region level flag is 0) but uses other classifier sets in this frame. For example, if the default set index is used, the region level flag will be 1. For example, in the four regions of the square partition, the following classifier sets are used, as shown in Table 26 below:

[0188] [Table 35] Table 26: A region level flag may be signaled to indicate whether CTBs in this region do not use the default set index. FIG. 28 is a block diagram illustrating that, according to some implementations of the present disclosure, CCSAO application region partitioning may be dynamic and may be switched at the picture level. For example, FIG. 28(a) shows that three CCSAO offset sets are used in this POC (set_num=3), and thus the picture frame is divided vertically into three regions. FIG. 28(b) shows that four CCSAO offset sets are used in this POC (set_num=4), and thus the picture frame is divided horizontally into four regions. FIG. 28(c) shows that three CCSAO offset sets are used in this POC (set_num=3), and thus the picture frame is raster-divided into three regions. Each region may have its own region all-on flag to save CTB on / off control bits. The number of regions depends on the picture set_num being signaled.

[0189] In some embodiments, the CCSAO syntax implemented is shown in Table 27 below. In AVS3, the term patch is similar to slice and patch header is similar to slice header. FLC stands for Fixed Length Code. TU stands for Shortened Unary Code. EGk stands for Exponential-Golomb Code with degree k, where k may be fixed.

[0190] [Table 36] JPEG0007654811000038.jpg250149 JPEG0007654811000039.jpg22159 Table 27: Example CCSAO syntax If the higher-order flags are off, the lower-order flags may be inferred from the off state of the flags and do not need to be signaled. For example, if ph_cc_sao_cb_flag is false in this picture, then ph_cc_sao_cb_band_num_minus1, ph_cc_sao_cb_luma_type, cc_sao_cb_offset_sign_flag, cc_sao_cb_offset_abs, ctb_cc_sao_cb_flag, cc_sao_cb_merge_left_flag, and cc_sao_cb_merge_up_flag are inferred to be not present and false.

[0191] In some embodiments, the ccsao_enabled_flag of the SPS is adjusted with the SAO enabled flag of the SPS, as shown in Table 28 below.

[0192] [Table 37] Table 28: SPS ccsao_enabled_flag is adjusted by SPS SAO enable flag In some embodiments, ph_cc_sao_cb_ctb_control_flag, ph_cc_sao_cr_ctb_control_flag indicate whether to enable the granularity of CTB on / off control of Cb / Cr. If ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_control_flag are enabled, ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag may be further signaled. Otherwise, whether CCSAO applies to the current picture depends on ph_cc_sao_cb_flag, ph_cc_sao_cr_flag, and ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag are not further signaled at the CTB level.

[0193] In some embodiments, for ph_cc_sao_cb_type and ph_cc_sao_cr_type, a central In the same position A flag may be further signaled to distinguish whether the luma position (Y0 position in FIG. 10) is used for classification for chroma samples. Similarly, if cc_sao_cb_type and cc_sao_cr_type are signaled at the CTB level, a flag may be further signaled by the same mechanism. For example, if the number of C0 luma position candidates is 9, the center In the same position cc_sao_cb_type0_flag is further signaled to distinguish whether the luma position is used or not. In the same position If the luma position is not used, cc_sao_cb_type_idc is used to indicate which of the remaining eight adjacent luma positions is used.

[0194] [Table 38] Table 29: Center In the same position cc_sao_cb_type0_flag is signaled to distinguish whether luma position is used or not Table 30 below shows an example where a single (set_num=1) or multiple (set_num>1) classifiers are used within a frame in AVS. The syntax notation can be mapped to the notation used above.

[0195] [Table 39]

[0196] [Table 40] Table 30: An example of how single (set_num=1) or multiple (set_num>1) classifiers are used in a picture frame in AVS When combined with FIG. 25 or FIG. 27, where each region has its own set, an example syntax may include region on / off control flags (picture_ccsao_lcu_control_flag[compIdx][setIdx]), as shown in Table 31 below.

[0197] [Table 41] Table 31: Each region has its own set, and an example syntax can include region on / off control flags (picture_ccsao_lcu_control_flag[compIdx][setIdx]). In some embodiments, extensions to intra and inter post-prediction SAO filters are further illustrated below. In some embodiments, the SAO classification method disclosed in this disclosure can act as a post-prediction filter, and the prediction can be intra, inter, or other prediction tools, such as intra block copy. Figure 29 is a block diagram showing the SAO classification method disclosed in this disclosure acting as a post-prediction filter according to some implementations of the present disclosure.

[0198] In some embodiments, for each Y, U, and V component, a corresponding classifier is selected. For each component prediction sample, the corresponding classifier is first classified and a corresponding offset is added. For example, each component can use the current sample and adjacent samples for classification. As shown below in Table 32, Y uses the current Y sample and adjacent Y sample, and U / V uses the current U / V sample for classification. Figure 30 is a block diagram showing that for a post-prediction SAO filter, each component can use the current sample and adjacent samples for classification according to some implementations of the present disclosure.

[0199] [Table 42] Table 32: For each Y, U, and V component a corresponding classifier is selected In some embodiments, the refined prediction samples (Ypred', Upred', Vpred') are updated by adding the corresponding class offsets and are then used for intra, inter, or other prediction.

[0200] Ypred'=clip3(0,(1< <bit_depth)-1,Ypred+h_Y[i]) Upred'=clip3(0,(1< <bit_depth)-1,Upred+h_U[i]) Vpred'=clip3(0,(1< <bit_depth)-1,Vpred+h_V[i]) In some embodiments, in addition to the current chroma component, for the chroma U and V components, an inter-component offset (Y) may be used for further offset classification. For example, as shown below in Table 33, an additional inter-component offset (h'_U, h'_V) may be added with the current component offset (h_U, h_V).

[0201] [Table 43] Table 33: In addition to the current chroma components, for the chroma U and V components, the intercomponent (Y) can be used for further offset classification In some embodiments, the improved prediction samples (Upred”, Vpred”) are updated by adding the corresponding class offset and are then used for intra, inter, or other prediction.

[0202] Upred”=clip3(0,(1< <bit_depth)-1,Upred’+h’_U[i]) Vpred”=clip3(0,(1< <bit_depth)-1,Vpred’+h’_V[i]) In some embodiments, intra and inter prediction may use different SAO filter offsets.

[0203] FIG. 31 is a flow diagram illustrating an example process 3100 for decoding a video signal using inter-component correlation according to some implementations of the present disclosure.

[0204] Video decoder 30 (shown in FIG. 3) receives a picture frame from a video signal, the picture frame including a first component and a second component. In some embodiments, the first component is a luma component and the second component is a first chroma component (3110).

[0205] The video decoder 30 determines a classifier for the first component based on one or more samples of a first set of the second component associated with each sample of the first component and one or more samples of a second set of the first component associated with each sample of the first component (3120). In some embodiments, the classifier is determined by a first sub-classifier and a second sub-classifier, where the first sub-classifier is determined by dividing a first dynamic range of values ​​of one or more samples of the first set of the second component into a first number of bands and selecting the bands based on intensity values ​​of one or more samples of the first set of the second component, and the second sub-classifier is determined based on a direction and intensity of edge information of a first subgroup of one or more samples of the second set of the first component (3120-1). For example, a combined luma edge offset (EO) and chroma band offset (BO) classifier is used to classify the luma samples.

[0206] Video decoder 30 determines (3130) sample offsets for each sample of the first component according to the classifier.

[0207] Video decoder 30 modifies the value of each sample of the first component based on the determined sample offset (3140).

[0208] In some embodiments, the picture frame further includes a third component, and the classifier for the first component is further based on one or more samples of a third set of the third component associated with each sample of the first component, and the third component is a second chroma component.

[0209] In some embodiments, determining (3130) a sample offset for each sample of the first component according to the classifier includes selecting a sample offset from a plurality of offsets for the first component according to the selected band and edge information.

[0210] In some embodiments, prior to determining 3120 a classifier for the first component, each sample of the first component is reconstructed by an in-loop filter, and one or more samples of a first set of the second component are reconstructed by an in-loop filter, where the in-loop filter is a deblocking filter (DBF) or a sample adaptive offset (SAO).

[0211] In some embodiments, the one or more samples of the first set of second components associated with each sample of the first component include a ratio of the number of samples of the second component to each sample of the first component. same position and one or more of the adjacent samples.

[0212] In some embodiments, the one or more samples of the second set of the first component associated with the respective sample of the first component are selected from one or more of the current sample and adjacent samples of the first component for the respective sample of the first component.

[0213] In some embodiments, the classifier for each sample of the first component is further determined by determining a third sub-classifier of the classifier by dividing a second dynamic range of values ​​of a second subgroup of one or more samples of the second set of first components into a second number of bands, determining a fourth sub-classifier of the classifier by dividing a third dynamic range of values ​​of one or more samples of the third set of third components into a third number of bands, and combining the first sub-classifier, the second sub-classifier, the third sub-classifier, and the fourth sub-classifier.

[0214] In some embodiments, modifying (3140) the value of each sample of the first component based on the determined sample offset is applied recursively.

[0215] In some embodiments, the total number of classes of the classifier is based on a combination of the first sub-classifier and the second sub-classifier, and the total number of classes is within a maximum range, the maximum range being: Fixed or At one or more of the following levels: sequence parameter set (SPS), adaptation parameter set (APS), picture parameter set (PPS), picture header (PH), and slice header (SH) Trust For example, subclassifier 1 has 4 classes, subclassifier 2 has 16 classes, and the combined classifier has 4×16 classes, which is within the maximum range.

[0216] In some embodiments, if one or more samples of the first set of the second component associated with the respective samples of the first component are located on the same side of a virtual boundary of a coding tree unit (CTU) with respect to the respective samples of the first component, the value of the respective samples of the first component is modified based on the determined sample offset (3140). For example, chroma candidates are used to classify the luma samples. If any of the chroma candidates of a luma sample are not on the same side of the VB of the luma sample, the CCSAO is disabled for this luma sample.

[0217] In some embodiments, if a first subset of one or more samples of a first set of a second component associated with a respective sample of a first component is located on a different side of a virtual boundary with respect to the respective sample of the first component and a remaining subset of one or more samples of a first set of a second component associated with a respective sample of a first component is located on the same side of a virtual boundary with respect to the respective sample of the first component, the first subset of one or more samples of a first set of a second component are replaced by copying a second subset from the remaining subset of one or more samples of a first set of a second component, and if one or more samples of a first set of a second component associated with a respective sample of a first component are located on a different side of a virtual boundary with respect to the respective sample of the first component, the one or more samples of a first set of a second component are replaced by copying one or more samples of a fourth set of a second component that are on the same side of a virtual boundary with respect to the respective sample of the first component.

[0218] For example, chroma candidates are used to classify a luma sample. If any of the chroma candidates for a luma sample are not on the same side of the VB of the luma sample, then iteration and / or mirror padding are used to derive chroma candidates on the other side of the VB of the luma sample based on the chroma candidates on the same side of the VB of the luma sample. Then, CCSAO is applied to the luma sample based on the repeated and / or mirror padded chroma candidates.

[0219] In some embodiments, the second subset from the remaining subset of the one or more samples of the first set of the second component is from the closest row to the first subset, or the second subset is located at a virtual boundary or symmetrical location of each sample of the first component to the first subset.

[0220] In some embodiments, the one or more samples of the fourth set that are on the same side of the virtual boundary as the respective samples of the first component are from the nearest row to the one or more samples of the first set of the second component, or the one or more samples of the fourth set are located at a symmetrical location of the virtual boundary or the respective samples of the first component to the one or more samples of the first set of the second component.

[0221] In some embodiments, if a first subset of one or more samples of the first set of a second component associated with a respective sample of the first component is located on a different side of a virtual boundary with respect to the respective sample of the first component, and a remaining subset of one or more samples of the first set of a second component associated with a respective sample of the first component is located on the same side of the virtual boundary with respect to the respective sample of the first component, the first subset at the first boundary location is replaced by copying a second subset of the one or more central subsets from the remaining subset of one or more samples of the first set of the second component, and a third subset within the remaining subsets at the second boundary location is replaced by copying the second subset or a fourth subset of the one or more central subsets from the remaining subset of one or more samples of the first set of the second component.

[0222] For example, chroma candidates are used to classify a luma sample. If any of the chroma candidates of a luma sample are not on the same side of the VB of the luma sample, then double-sided symmetric padding is used to derive chroma candidates on the other side of the VB of the luma sample based on the chroma candidates on the same side of the VB of the luma sample. Then, CCSAO is applied to the luma sample based on the double-sided symmetric padded chroma candidates.

[0223] In some embodiments, the third subset and the first subset are symmetrically located within one or more samples of the first set of second components.

[0224] 32 shows a computing environment 3210 coupled to a user interface 3250. The computing environment 3210 may be part of a data processing server. The computing environment 3210 includes a processor 3220, a memory 3230, and an input / output (I / O) interface 3240.

[0225] The processor 3220 typically controls the overall operation of the computing environment 3210, such as operations associated with display, data acquisition, data communication, and image processing. The processor 3220 may include one or more processors to execute instructions for performing all or some of the steps in the methods described above. Additionally, the processor 3220 may include one or more modules that facilitate interaction between the processor 3220 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single chip machine, a graphical processing unit (GPU), or the like.

[0226] The memory 3230 is configured to store various types of data to support the operation of the computing environment 3210. The memory 3230 may include predefined software 3232. Examples of such data include instructions for any application or method operated on the computing environment 3210, video data sets, image data, etc. The memory 3230 may be implemented by using any type of volatile or non-volatile memory device, or combinations thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

[0227] The I / O interface 3240 provides an interface between the processor 3220 and a peripheral interface module, such as a keyboard, a click wheel, buttons, etc. The buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. The I / O interface 3240 may be coupled to an encoder and a decoder.

[0228] In one embodiment, a non-transitory computer readable storage medium is also provided that comprises, for example, in memory 3230, a number of programs executable by processor 3220 in computing environment 3210 to perform the aforementioned methods. Alternatively, the non-transitory computer readable storage medium may store a bitstream or datastream that includes encoded video information (e.g., video information including one or more syntax elements), generated by an encoder (e.g., video encoder 20 of FIG. 2) using, for example, the encoding method described above for use by a decoder (e.g., video decoder 30 of FIG. 3) in decoding the video data. The non-transitory computer readable storage medium may be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0229] In one embodiment, a computing device is also provided that includes one or more processors (e.g., processor 3220) and a non-transitory computer-readable storage medium or memory 3230 having stored thereon a plurality of programs executable by the one or more processors, the one or more processors being configured to, upon execution of the plurality of programs, perform the aforementioned method.

[0230] In one embodiment, a computer program product is also provided that includes a number of programs executable by the processor 3220 in the computing environment 3210 to perform the aforementioned methods, e.g., in the memory 3230. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0231] In one embodiment, the computing environment 3210 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0232] Further embodiments also include various subsets of the above embodiments combined or otherwise rearranged in various other embodiments.

[0233] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium, such as a data storage medium, or a communication medium including any medium that facilitates the transfer of a computer program from one place to another, for example according to a communication protocol. In this manner, a computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that may be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementation of the implementations described in this application. A computer program product may include a computer-readable medium.

[0234] The terminology used in the description of implementations herein is intended only to describe the particular implementation and is not intended to limit the scope of the claims. When used in the description of implementations and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that, as used herein, the term "and / or" refers to and includes any possible combination of one or more of the associated listed items. It will be further understood that the terms "comprises" and / or "comprising," as used herein, specify the presence of the stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or groups thereof.

[0235] In this specification, terms such as first, second, etc. may be used to describe various elements, but it will be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another element. For example, a first electrode may be called a second electrode, and similarly, a second electrode may be called a first electrode, without departing from the scope of the implementation. The first electrode and the second electrode are both electrodes, but are not the same electrode.

[0236] Throughout this specification, references to "in one example," "example," "illustrative," and the like mean that one or more particular features, structures, or characteristics described in connection with an example are included in at least one example of the disclosure. Thus, the appearances of the phrases "in one example," "in an example," "illustratively," and the like in various places throughout this specification do not necessarily all refer to the same example. Furthermore, particular features, structures, or characteristics in one or more examples may include being combined in any suitable manner.

[0237] The description in this application has been presented for purposes of illustration and explanation and is not intended to be exhaustive or to be limited to the invention in the form disclosed. Many modifications, variations, and alternative implementations will be apparent to those skilled in the art having the benefit of the teachings presented in the foregoing description and the associated drawings. The embodiments have been chosen and described in order to best explain the principles of the invention, its practical application, and to enable those skilled in the art to understand the invention in terms of its various implementations and to best utilize the underlying principles and various implementations with various modifications suited to the particular use contemplated. It is therefore to be understood that the scope of the claims is not limited to the specific examples of implementations disclosed, but that modifications and other implementations are intended to be included within the scope of the appended claims.

Claims

1. 1. A method for decoding a video signal, comprising the steps of: receiving a picture frame from the video signal, the picture frame including a first component and a second component; determining a classifier for the first component based on one or more samples of a first set of the second component associated with each sample of the first component and one or more samples of a second set of the first component associated with the each sample of the first component; determining a sample offset for the respective samples of the first component according to the classifier; and modifying a value of the respective sample of the first component based on the determined sample offset; the first component is a luma component and the second component is a first chroma component; 11. The method of claim 10, wherein the classifier is determined by a first sub-classifier and a second sub-classifier, the first sub-classifier being determined by dividing a first dynamic range of values ​​of one or more samples of the first set of the second component into a first number of bands and selecting bands based on intensity values ​​of the one or more samples of the first set of the second component, and the second sub-classifier being determined based on direction and intensity of edge information of a first subset of one or more samples of the second set of the first component.

2. 2. The method of claim 1 , wherein the picture frame further includes a third component, and the classifier for the first component is further based on one or more samples of a third set of the third component associated with the respective samples of the first component, the third component being a second chroma component.

3. determining the sample offsets for the respective samples of the first component according to the classifier; The method of claim 1 , comprising selecting the sample offset from a plurality of offsets for the first component according to the selected band and the edge information.

4. 2. The method of claim 1 , wherein prior to determining the classifier for the first component, the respective samples of the first component are reconstructed by an in-loop filter, and one or more samples of the first set of the second component are reconstructed by an in-loop filter, the in-loop filter being a deblocking filter (DBF) or a sample adaptive offset (SAO).

5. 2. The method of claim 1 , wherein the one or more samples of the first set of the second component associated with the respective sample of the first component are selected from one or more of co-located and adjacent samples of the second component to the respective sample of the first component.

6. 2. The method of claim 1 , wherein one or more samples of the second set of the first component associated with the respective sample of the first component are selected from one or more of a current sample and adjacent samples of the first component to the respective sample of the first component.

7. The classifier for each sample of the first component comprises: determining a third sub-classifier of the classifier by dividing a second dynamic range of values ​​of a second subgroup of one or more samples of the second set of the first components into a second number of bands; determining a fourth sub-classifier of the classifier by dividing a third dynamic range of values ​​of one or more samples of a third set of the third component into a third number of bands; The method of claim 2 , further determined by combining the first sub-classifier, the second sub-classifier, the third sub-classifier, and the fourth sub-classifier.

8. The method of claim 1 , wherein modifying the values ​​of the respective samples of the first component based on the determined sample offsets is applied recursively.

9. 2. The method of claim 1, wherein a total number of classes of the classifier is based on a combination of the first sub-classifier and the second sub-classifier, the total number of classes being within a maximum range, the maximum range being fixed or signaled at one or more of a sequence parameter set (SPS), an adaptation parameter set (APS), a picture parameter set (PPS), a picture header (PH), and a slice header (SH) level.

10. 2. The method of claim 1, further comprising: modifying the value of the respective sample of the first component based on the determined sample offset if one or more samples of the first set of the second component associated with the respective sample of the first component are located on the same side of a virtual boundary of a coding tree unit as the respective sample of the first component.

11. if a first subset of one or more samples of the first set of the second component associated with the respective sample of the first component is located on a different side of a virtual boundary with respect to the respective sample of the first component, and a remaining subset of one or more samples of the first set of the second component associated with the respective sample of the first component is located on a same side of the virtual boundary with respect to the respective sample of the first component, the first subset of one or more samples of the first set of the second component is replaced by copying a second subset from the remaining subset of one or more samples of the first set of the second component; if one or more samples of the first set of the second component associated with the respective sample of the first component are located on different sides of the virtual boundary relative to the respective sample of the first component, one or more samples of the first set of the second component are exchanged by copying one or more samples of a fourth set of the second component that are on the same side of the virtual boundary as the respective sample of the first component to exchange the one or more samples of the first set of the second component. The method of claim 1.

12. 12. The method of claim 11 , wherein the second subset from the remaining subset of one or more samples of the first set of the second component is from a nearest row to the first subset, or the second subset is located at a symmetric location of the virtual boundary or the respective sample of the first component with respect to the first subset.

13. 12. The method of claim 11 , wherein one or more samples of the fourth set that are on the same side of the virtual boundary to the respective sample of the first component are from a nearest row to one or more samples of the first set of the second component, or one or more samples of the fourth set are located at a symmetrical location of the virtual boundary or the respective sample of the first component to one or more samples of the first set of the second component.

14. if a first subset of one or more samples of the first set of the second component associated with the respective sample of the first component is located on a different side of a virtual boundary with respect to the respective sample of the first component, and a remaining subset of one or more samples of the first set of the second component associated with the respective sample of the first component is located on a same side of the virtual boundary with respect to the respective sample of the first component, the first subset at a first boundary location is replaced by copying a second subset of one or more central subsets from the remaining subset of one or more samples of the first set of the second component; a third subset within the remaining subset at a second boundary position is replaced by copying the second subset or a fourth subset of the one or more central subsets from the remaining subset of one or more samples of the first set of the second component; The method of claim 1.

15. The method of claim 14 , wherein the third subset and the first subset are symmetrically located within one or more samples of the first set of the second components.

16. one or more processing units; a memory coupled to the one or more processing units; and a plurality of programs stored in said memory, said plurality of programs, when executed by said one or more processing units, causing said electronic device to perform a method according to any one of claims 1 to 15. electronic equipment.

17. A non-transitory computer readable storage medium storing a plurality of programs for execution by an electronic device having one or more processing units, the plurality of programs, when executed by the one or more processing units, causing the electronic device to perform a method according to any one of claims 1 to 15.

18. 16. A computer program comprising instructions for execution by an electronic device having one or more processing units, the instructions, when executed by the one or more processing units, causing the electronic device to perform the method of any one of claims 1 to 15.

19. A method for transmitting a bitstream, comprising the steps of: performing an encoding method to generate a bitstream; transmitting the bitstream to a decoding device; The encoding method comprises: determining a classifier for a first component of a picture frame based on a first set of one or more samples of a second component of the picture frame associated with respective samples of the first component and a second set of one or more samples of the first component associated with the respective samples of the first component; determining a sample offset for the respective samples of the first component according to the classifier; and modifying a value of the respective sample of the first component based on the determined sample offset; the first component is a luma component and the second component is a first chroma component; the classifier is determined by a first sub-classifier and a second sub-classifier, the first sub-classifier being determined by dividing a first dynamic range of values ​​of one or more samples of the first set of the second component into a first number of bands and selecting bands based on intensity values ​​of one or more samples of the first set of the second component, and the second sub-classifier being determined based on direction and intensity of edge information of a first subgroup of one or more samples of the second set of the first component. method.

Images