Chroma coding enhancement in cross-component sample adaptive offset

Abstract

An electronic device performs a method of decoding video data. The method includes receiving, from a video signal, a picture frame including a first component and a second component, receiving, from the video signal, a first syntax element indicating whether cross-component sample adaptive offset (CCSAO) is enabled for the second component at a picture level, receiving, from the video signal, a second syntax element indicating whether CCSAO is controlled for the second component at a coding tree block (CTB) level when the first syntax element indicates that CCSAO is enabled for the second component at the picture level, and receiving, from the video signal, a third syntax element indicating whether CCSAO is enabled for the second component at the CTB level when the second syntax element indicates that CCSAO is controlled for the second component at the CTB level.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 068,279, filed August 20, 2020, entitled “Cross-component Sample Adaptive Offset,” and U.S. Provisional Patent Application No. 63 / 068,509, filed August 21, 2020, entitled “Cross-component Sample Adaptive Offset,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application generally relates to video encoding and decoding and compression, and more specifically, to methods and apparatus for improving the efficiency of chroma encoding and decoding. Background Technology

[0004] Digital video is supported by a variety of electronic devices, such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, and video streaming devices. Electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression standards. Some well-known video codec standards include Universal Video Codec (VVC), High Efficiency Video Codec (HEVC, also known as H.265 or MPEG-H Part 2), and Advanced Video Codec (AVC, also known as H.264 or MPEG-4 Part 10), which were jointly developed by ISO / IEC MPEG and ITU-T VCEG. AOMedia Video 1 (AV1) was developed by the Alliance for Open Media (AOM) as the successor to its predecessor, VP9. Audio Video Codec (AVS) refers to digital audio and digital video compression standards, another series of video compression standards developed by the China Audio and Video Codec Standards Working Group.

[0005] Video compression typically involves performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in video data. For block-based video coding and decoding, video frames are divided into one or more stripes, each strip containing multiple video blocks, also known as coding tree units (CTUs). Each CTU may contain a coding unit (CU) or be recursively split into smaller CUs until a predefined minimum CU size is reached. Each CU (also called a leaf CU) contains one or more transform units (TUs), and each CU also contains one or more prediction units (PUs). Each CU can be encoded in intra-frame, inter-frame, or IBC modes. Video blocks in an intra-frame coded (I) strip of a video frame are encoded using spatial prediction with respect to reference samples in adjacent blocks within the same video frame. Video blocks in an inter-frame coded (P or B) strip of a video frame can be encoded using spatial prediction with respect to reference samples in adjacent blocks within the same video frame or temporal prediction with respect to reference samples in other previous and / or future reference video frames.

[0006] A prediction block for the current video block to be encoded is generated based on spatial or temporal predictions of previously encoded reference blocks (e.g., neighboring blocks). The process of finding the reference block can be accomplished using a block-matching algorithm. The residual data representing the pixel difference between the current block to be encoded and the prediction block is called the residual block or prediction error. Inter-frame coded blocks are encoded based on the motion vector pointing to the reference block in the reference frame that forms the prediction block and the residual block. The process of determining the motion vector is often called motion estimation. Intra-frame coded blocks are encoded based on the intra-frame prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to the transform domain, such as the frequency domain, to obtain residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to produce a one-dimensional vector of transform coefficients, which are then entropy-encoded into the video bitstream for even more compression.

[0007] The encoded video bitstream is then stored in a computer-readable storage medium (e.g., flash memory) for access by another electronic device with digital video capabilities or directly transmitted to the electronic device via wired or wireless means. The electronic device then performs video decompression (the reverse of the video compression process described above) by, for example, parsing the encoded video bitstream to obtain syntax elements from the bitstream and reconstructing the digital video data from the encoded video bitstream to its original format based at least in part on the syntax elements obtained from the bitstream, and renders the reconstructed digital video data on the display of the electronic device.

[0008] As digital video quality progresses from high definition to 4Kx2K and even 8Kx4K, the amount of video data to be encoded / decoded grows exponentially. Maintaining image quality while encoding / decoding video data more efficiently remains a challenge. Summary of the Invention

[0009] This application describes implementations related to video data encoding and decoding, and more specifically, implementations related to methods and apparatus for improving the encoding and decoding efficiency of chroma encoding and decoding, including improving encoding and decoding efficiency by exploring cross-component relationships between luminance and chroma components.

[0010] According to a first aspect of this application, a method for decoding a video signal includes: receiving from the video signal a picture frame comprising a first component and a second component; determining whether Cross-Component Sample Adaptive Offset (CCSAO) is enabled for the second component; in response to determining that the CCSAO is enabled for the second component, determining a classifier for the second component based on a set of one or more samples of the first component relative to each sample of the second component; selecting a sample offset from a mapping table of the second component according to the classifier, the mapping table comprising a plurality of entries, each entry comprising a class index determined by the classifier and a corresponding sample offset associated with the second component; and obtaining a cross-component offset sample value of the second component by modifying the second component based on the sample offset.

[0011] In some exemplary embodiments or any combination of the foregoing exemplary embodiments, a mapping table is received from the video signal.

[0012] In some exemplary embodiments or any combination of the foregoing exemplary embodiments, the mapping table is predefined based on the bit depth of the video signal.

[0013] In some exemplary embodiments or any combination of the foregoing exemplary embodiments, determining whether to enable CCSAO for the second component includes: receiving a first syntax element from the video signal, the first syntax element indicating whether to enable the CCSAO for the second component at the picture level; in response to the first syntax element indicating that the CCSAO is enabled for the second component at the picture level, receiving a second syntax element from the video signal, the second syntax element indicating whether to control the CCSAO for the second component at the coding tree block (CTB) level; in response to the second syntax element indicating that the CCSAO is controlled for the second component at the CTB level, receiving a third syntax element from the video signal, the third syntax element indicating whether to enable the CCSAO for the second component at the CTB level; and determining whether to enable the CCSAO for the second component at the CTB level based on the third syntax element.

[0014] In some exemplary embodiments or any combination of the foregoing exemplary embodiments, the method of decoding a video signal further includes: inferring that the second syntax element is false in response to the first syntax element indicating that the CCSAO is not enabled for the second component at the picture level.

[0015] In some exemplary embodiments or any combination of the foregoing exemplary embodiments, the method of decoding a video signal further includes: in response to the second syntax element indicating that the CCSAO is not controlled for the second component at the CTB level, determining, according to the first syntax element, whether to enable the CCSAO for the second component at the picture level.

[0016] In some exemplary embodiments or any combination of the foregoing exemplary embodiments, determining whether to enable CCSAO for the second component includes: receiving a fourth syntax element indicating whether to enable Sample Adaptive Offset (SAO) at the Sequence Parameter Set (SPS) level; in response to the fourth syntax element indicating whether to enable the SAO at the SPS level, receiving a fifth syntax element indicating whether to enable the CCSAO at the SPS level; and determining whether to enable the CCSAO at the SPS level based on the fifth syntax element.

[0017] In some exemplary embodiments or any combination of the foregoing exemplary embodiments, the method of decoding a video signal further includes receiving a sixth syntax element, the sixth syntax element indicating whether the center co-location of the first component relative to each of the second components is used to determine the classifier.

[0018] In some exemplary embodiments or any combination of the foregoing exemplary embodiments, the method of decoding a video signal further includes: in response to the sixth syntax element indicating that the center co-location of the first component relative to each sample of the second component is not used to determine the classifier, receiving a seventh syntax element indicating the location of a neighboring sample of the first component relative to each sample of the second component, the sample location being used to determine the classifier.

[0019] In some exemplary embodiments or any combination of the foregoing exemplary embodiments, the method for decoding a video signal further includes: receiving from the video signal a second mapping table comprising a plurality of entries, each entry including a class index determined by the classifier and a corresponding quantized sample offset associated with the second component; obtaining a dequantized sample offset associated with the second component by dequantizing each of the quantized sample offsets in the second mapping table for each of the quantized sample offsets in the second mapping table; updating the second mapping table by replacing each quantized sample offset in the second mapping table with a corresponding dequantized sample offset; and determining the updated second mapping table as the mapping table.

[0020] In some exemplary embodiments or any combination of the foregoing exemplary embodiments, the quantization level of the quantization sample offset in the second mapping table is related to the bit depth of the video signal.

[0021] In some exemplary embodiments or any combination of the foregoing exemplary embodiments, when the bit depth of the video signal is greater than 10, the corresponding quantization sample offset in the second mapping table is dequantized.

[0022] In some exemplary embodiments or any combination of the foregoing exemplary embodiments, the set of one or more samples of the first component relative to each sample of the second component lies within a virtual space defined by the virtual boundary of the image frame.

[0023] In some exemplary embodiments or any combination of the foregoing exemplary embodiments, the first component is a luminance component and the second component is a chrominance component.

[0024] According to a second aspect of this application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory. When executed by the one or more processing units, the programs cause the electronic device to perform the method of encoding and decoding video data as described above.

[0025] According to a third aspect of this application, a non-transitory computer-readable storage medium stores a plurality of programs for execution by an electronic device having one or more processing units. When executed by the one or more processing units, the programs cause the electronic device to perform the method of encoding and decoding video data as described above. Attached Figure Description

[0026] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated herein by reference and form part of the specification. The drawings illustrate the described embodiments and, together with the description, serve to explain the basic principles. Similar reference numerals indicate corresponding parts.

[0027] Figure 1 This is a block diagram illustrating an exemplary video encoding and decoding system according to some embodiments of the present disclosure.

[0028] Figure 2 This is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0029] Figure 3 This is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0030] Figures 4A to 4E This is a block diagram illustrating how, according to some embodiments of the present disclosure, a frame is recursively divided into multiple video blocks of different sizes and shapes.

[0031] Figure 5 This is a block diagram depicting four gradient modes used in Sample Adaptive Offset (SAO) according to some embodiments of the present disclosure.

[0032] Figure 6A This is a block diagram illustrating a system and process for cross-component sample adaptive offset (CCSAO) according to some embodiments of the present disclosure.

[0033] Figure 6B This is a block diagram illustrating a system and process for CCSAO applied in parallel with Enhanced Sample Adaptive Offset (ESAO) in the AVS standard, according to some embodiments of this disclosure.

[0034] Figure 7 This is a block diagram illustrating the sampling process using CCSAO according to some embodiments of the present disclosure.

[0035] Figure 8 This is a block diagram illustrating the CCSAO process according to some embodiments of the present disclosure being interleaved with vertical and horizontal deblocking filters (DBF).

[0036] Figure 9 This is a flowchart illustrating an exemplary process for decoding a video signal using cross-component correlation according to some embodiments of the present disclosure.

[0037] Figure 10 This is a block diagram illustrating a classifier that uses different brightness sample point locations for classification according to some embodiments of the present disclosure.

[0038] Figure 11 This is a block diagram of a sampling process according to some embodiments of the present disclosure, showing that in addition to luminance, other cross-component co-location and adjacent chromaticity samples are also fed into the CCSAO classification.

[0039] Figure 12An exemplary classifier according to some embodiments of the present disclosure is shown, which replaces the co-position luminance sample values ​​with values ​​obtained by weighting co-position and adjacent luminance samples.

[0040] Figure 13A This is a block diagram illustrating, according to some embodiments of the present disclosure, that CCSAO is not applied to the current chromaticity sample if any of the co-occurring and adjacent luminance samples used for classification are outside the current image.

[0041] Figure 13B This is a block diagram illustrating, according to some embodiments of the present disclosure, the application of CCSAO to the current chromaticity sample if either of the co-occurring and adjacent luminance samples used for classification is outside the current image.

[0042] Figure 14 This is a block diagram illustrating, according to some embodiments of the present disclosure, that CCSAO is not applied to the current chromaticity sample if the corresponding selected co-occurring or adjacent luminance samples used for classification are outside the virtual space defined by the virtual boundary.

[0043] Figure 15 This is a flowchart illustrating an exemplary process for decoding a video signal using cross-component correlation according to some embodiments of the present disclosure. Detailed Implementation

[0044] Reference will now be made in detail to specific embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives may be used without departing from the scope of the claims and that the subject matter may be practiced without these specific details. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0045] The first generation of AVS standards included the Chinese national standards "Information Technology, Advanced Audio Video Coding, Part 2: Video" (referred to as AVS1) and "Information Technology, Advanced Audio Video Coding, Part 16: Radio Television Video" (referred to as AVS+). Compared to the MPEG-2 standard, it could save approximately 50% of the bitrate while maintaining the same perceptual quality. The second generation of AVS standards included the Chinese national standard "Information Technology, Efficient Multimedia Coding". The AVS2 series of encoding and decoding standards is primarily designed for the transmission of ultra-high-definition television programs. AVS2's encoding and decoding efficiency is twice that of AVS+. The video portion of the AVS2 standard was submitted by the Institute of Electrical and Electronics Engineers (IEEE) as an international standard application. The AVS3 standard is a next-generation video coding standard for UHD video applications, aiming to surpass the encoding and decoding efficiency of the latest international standard, HEVC, offering approximately 30% bitrate savings. In March 2019, at the 68th AVS meeting, the AVS3-P2 baseline was completed, offering approximately 30% bitrate savings compared to the HEVC standard. Currently, a reference software called the High Performance Model (HPM) is maintained by the AVS group to demonstrate a reference implementation of the AVS3 standard. Similar to HEVC, the AVS3 standard is built on a block-based hybrid video encoding and decoding framework.

[0046] Figure 1 This is a block diagram illustrating an exemplary system 10 for parallel encoding and decoding of video blocks according to some embodiments of the present disclosure. Figure 1 As shown, system 10 includes a source device 12 that generates and encodes video data, which will then be decoded by a target device 14. The source device 12 and target device 14 can include any of a variety of electronic devices, including desktop or laptop computers, tablets, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and target device 14 are equipped with wireless communication capabilities.

[0047] In some implementations, target device 14 may receive encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving encoded video data from source device 12 to target device 14. In one example, link 16 may include a communication medium enabling source device 12 to transmit encoded video data directly to target device 14 in real time. The encoded video data may be modulated according to a communication standard such as a wireless communication protocol and transmitted to target device 14. The communication medium may include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, switch, base station, or any other device that can be used to facilitate communication from source device 12 to target device 14.

[0048] In some other implementations, encoded video data can be transferred from output interface 22 to storage device 32. The encoded video data in storage device 32 can then be accessed by target device 14 via input interface 28. Storage device 32 can include any of a variety of distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In a further example, storage device 32 can correspond to a file server or another intermediate storage device capable of storing encoded video data generated by source device 12. Target device 14 can access the stored video data from storage device 32 via streaming or downloading. The file server can be any type of computer capable of storing and transferring encoded video data to target device 14. Exemplary file servers include web servers (e.g., for websites), FTP servers, network attached storage (NAS) devices, or local disk drives. Target device 14 can access encoded video data via any standard data connection, including wireless channels (e.g., Wi-Fi connections), wired connections (e.g., DSL, cable modems, etc.), or combinations thereof, suitable for accessing encoded video data stored on a file server. Transmission of encoded video data from storage device 32 can be streaming, downloading, or a combination of both.

[0049] like Figure 1As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources such as video capture devices, such as video cameras, video archives containing previously captured video, video feed interfaces for receiving video from video content providers, and / or computer graphics systems for generating computer graphics data as source video, or combinations of these sources. As an example, if video source 18 is a video camera in a security surveillance system, source device 12 and target device 14 may form a camera phone or video phone. However, the embodiments described in this application are applicable to general video encoding and decoding and can be applied to wireless and / or wired applications.

[0050] Captured, pre-captured, or computer-generated video can be encoded by video encoder 20. The encoded video data can be directly transmitted to target device 14 via output interface 22 of source device 12. The encoded video data can also (or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback. Output interface 22 may also include a modem and / or transmitter.

[0051] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such syntax elements may be included within the encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0052] In some embodiments, target device 14 may include display device 34, which may be an integrated display device or an external display device configured to communicate with target device 14. Display device 34 displays decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0053] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards, such as VVC, HEVC, MPEG-4, Part 10, Advanced Video Codec (AVC), AVS, or extensions of such standards. It should be understood that this application is not limited to any particular video encoding / decoding standard and may be applicable to other video encoding / decoding standards. It is generally anticipated that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is generally anticipated that the video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0054] The video encoder 20 and video decoder 30 can each be implemented as any of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store software instructions in a suitable, non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0055] Figure 2 This is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in this application. The video encoder 20 can perform intra-frame and inter-frame predictive coding of video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence.

[0056] like Figure 2As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a partitioning unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copying (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter, such as a deblocking filter (not shown), can be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter (not shown) can be used to filter the output of the adder 62. Before the reconstructed CU is placed into the reference image storage and used as a reference for encoding future video blocks, further loop filtering, such as Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF), can be applied to the reconstructed CU. The video encoder 20 can be in the form of a fixed or programmable hardware unit, or can be partitioned among one or more of the fixed or programmable hardware units shown.

[0057] Video data memory 40 can store video data to be encoded by components of video encoder 20. The video data in video data memory 40 can be obtained, for example, from video source 18. DPB 64 is a buffer that stores reference video data for encoding video data by video encoder 20 (e.g., in intra-frame or inter-frame predictive coding modes). Video data memory 40 and DPB 64 can be formed from any of a variety of memory devices. In various examples, video data memory 40 can be on-chip along with other components of video encoder 20, or off-chip relative to those components.

[0058] like Figure 2As shown, after receiving video data, the partitioning unit 45 within the prediction processing unit 41 segments the video data into video blocks. This partitioning may also include segmenting the video frame into stripes, tiles, or other larger coding units (CUs) according to a predefined splitting structure associated with the video data, such as a quadtree structure. The video frame can be divided into multiple video blocks (or a set of video blocks referred to as tiles). The prediction processing unit 41 may select one of several possible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one of several intra-frame predictive coding modes or one of several inter-frame predictive coding modes. The prediction processing unit 41 may provide the resulting intra-frame or inter-frame predictive coded block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coded block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements, such as motion vectors, intra-frame mode indicators, partitioning information, and other such syntax information, to entropy coding unit 56.

[0059] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block relative to one or more adjacent blocks in the same frame to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block relative to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can execute multiple coding channels, for example, to select an appropriate coding mode for each video data block.

[0060] In some implementations, the motion estimation unit 42 determines the inter-frame prediction mode of the current video frame by generating motion vectors that indicate the displacement of the prediction unit (PU) of a video block within the current video frame relative to a prediction block within a reference video frame, according to a predetermined mode within the video frame sequence. Motion estimation performed by the motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. For example, the motion vectors may indicate the displacement of the PU of a video block within the current video frame or picture relative to the prediction block within a reference frame (or other coding unit) relative to the currently encoded block within the current frame (or other coding unit). The predetermined mode may designate video frames in the sequence as P-frames or B-frames. The intra-frame BC unit 48 may determine vectors for intra-frame BC coding, such as block vectors, in a manner similar to how the motion estimation unit 42 determines motion vectors for inter-frame prediction, or the motion estimation unit 42 may be used to determine block vectors.

[0061] A prediction block is a block of reference frames that is considered to closely match the PU of the video block to be encoded in terms of pixel differences, which can be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 may compute values ​​for sub-integer pixel positions of the reference frames stored in the DPB 64. For example, the video encoder 20 may interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frames. Thus, the motion estimation unit 42 may perform motion search relative to full pixel positions and fractional pixel positions and output motion vectors with fractional pixel precision.

[0062] The motion estimation unit 42 calculates the motion vector of the PU for the video block of the inter-frame predictive coding frame by comparing the position of the PU with the position of the prediction block of a reference frame selected from either the first reference frame list (list 0) or the second reference frame list (list 1). Each of the first reference frame list (list 0) or the second reference frame list (list 1) identifies one or more reference frames stored in the DPB 64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44 and then to the entropy coding unit 56.

[0063] Motion compensation performed by motion compensation unit 44 may involve acquiring or generating prediction blocks based on motion vectors determined by motion estimation unit 42. Upon receiving the motion vector of the PU for the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in one of the reference frame lists, retrieve the prediction block from DPB 64, and forward the prediction block to adder 50. Adder 50 then forms a residual video block of pixel differences by subtracting the pixel values ​​of the prediction block provided by motion compensation unit 44 from the pixel values ​​of the current video block being encoded. The pixel differences forming the residual video block may include luma or chroma difference components, or both. Motion compensation unit 44 may also generate syntax elements associated with video blocks of a video frame for use by video decoder 30 when decoding video blocks of a video frame. Syntax elements may include, for example, syntax elements defining motion vectors for identifying prediction blocks, any flags indicating prediction modes, or any other syntax information described herein. Note that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes.

[0064] In some implementations, the intra-BC unit 48 can generate vectors and acquire prediction blocks in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44; however, the prediction blocks are in the same frame as the current block being encoded, and the vectors are referred to as block vectors rather than motion vectors. Specifically, the intra-BC unit 48 can determine an intra-prediction mode for encoding the current block. In some examples, the intra-BC unit 48 can encode the current block using various intra-prediction modes, for example, in separate coding channels, and test their performance through rate-distortion analysis. Next, the intra-BC unit 48 can select an appropriate intra-prediction mode from the various tested intra-prediction modes to use and generate intra-mode indicators accordingly. For example, the intra-BC unit 48 can use rate-distortion analysis for various tested intra-prediction modes to calculate rate-distortion values ​​and select the intra-prediction mode with the best rate-distortion characteristics from the tested modes as the appropriate intra-prediction mode for use. Rate-distortion analysis typically determines the amount of distortion (or error) between the coded block and the original uncoded block that was encoded to produce the coded block, as well as the bit rate (i.e., the number of bits) used to produce the coded block. Intra-frame BC unit 48 can calculate the ratio based on the distortion and rate of various coded blocks to determine which intra-frame prediction mode displays the optimal rate-distortion value for the block.

[0065] In other examples, the intra-frame BC unit 48 may use the motion estimation unit 42 and the motion compensation unit 44, either wholly or partially, to perform such functions for intra-frame BC prediction according to the embodiments described herein. In any case, for intra-frame block copying, the predicted block may be a block that is considered to closely match the block to be encoded in terms of pixel differences, which may be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics, and the identification of the predicted block may include the calculation of values ​​for sub-integer pixel positions.

[0066] Regardless of whether the prediction block originates from the same frame based on intra-frame prediction or from different frames based on inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current encoded video block to generate a pixel difference. The pixel difference that forms the residual video block may include differences in luminance and chrominance components.

[0067] Intra-prediction processing unit 46 can intra-predict the current video block as an alternative to inter-frame prediction performed by motion estimation unit 42 and motion compensation unit 44, or intra-block copy prediction performed by intra-BC unit 48, as described above. Specifically, intra-prediction processing unit 46 can determine an intra-prediction mode for encoding the current block. To this end, intra-prediction processing unit 46 can encode the current block using various intra-prediction modes, for example, in a separate encoding channel, and intra-prediction processing unit 46 (or mode selection unit, in some examples) can select an appropriate intra-prediction mode from tested intra-prediction modes for use. Intra-prediction processing unit 46 can provide information indicating the intra-prediction mode selected for the block to entropy coding unit 56. Entropy coding unit 56 can encode the information indicating the selected intra-prediction mode into the bitstream.

[0068] After prediction processing unit 41 determines the prediction block of the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more transform units (TUs) and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform such as discrete cosine transform (DCT) or a conceptually similar transform.

[0069] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some instances, the quantization unit 54 can then perform a scan of a matrix containing the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0070] After quantization, entropy coding unit 56 entropy-encodes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length codec (CAVLC), context-adaptive binary arithmetic codec (CABAC), syntax-based context-adaptive binary arithmetic codec (SBAC), probabilistic interval partitioned entropy (PIPE) codec, or another entropy coding method or technique. The encoded bitstream can then be transmitted to video decoder 30 or archived in storage device 32 for later transmission to or retrieval from video decoder 30. Entropy coding unit 56 can also entropy-encode the motion vectors and other syntax elements of the current video frame being encoded.

[0071] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for generating a reference block for predicting other video blocks. As described above, the motion compensation unit 44 can generate a motion-compensated prediction block from one or more reference blocks of a frame stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction block to compute sub-integer pixel values ​​for motion estimation.

[0072] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to produce a reference block for storage in DPB 64. The reference block can then be used by intra-frame BC unit 48, motion estimation unit 42, and motion compensation unit 44 as a prediction block for inter-frame prediction of another video block in subsequent video frames.

[0073] Figure 3 This is a block diagram illustrating an exemplary video decoder 30 according to some embodiments of this application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction processing unit 84, and an intra-frame prediction (BC) unit 85. The video decoder 30 can perform a decoding process, which is generally combined with the above description. Figure 2 The encoding process described for video encoder 20 is the reverse. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, while intra-frame prediction unit 84 may generate prediction data based on intra-frame prediction mode indicators received from entropy decoding unit 80.

[0074] In some examples, units of the video decoder 30 may be responsible for performing the embodiments of this application. Furthermore, in some examples, the embodiments of this disclosure may be divided among one or more units of the video decoder 30. For example, the intra-frame BC unit 85 may perform the embodiments of this application alone or in combination with other units of the video decoder 30 (e.g., motion compensation unit 82, intra-frame prediction processing unit 84, and entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra-frame BC unit 85, and the function of the intra-frame BC unit 85 may be performed by other components of the prediction processing unit 81 (e.g., motion compensation unit 82).

[0075] Video data memory 79 can store video data to be decoded by other components of video decoder 30, such as encoded video bitstreams. The video data stored in video data memory 79 can be obtained, for example, from storage device 32, a local video source (e.g., a camera), wired or wireless network communication via video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include an encoded picture buffer (CPB) that stores encoded video data from the encoded video bitstream. Decoded picture buffer (DPB) 92 of video decoder 30 stores reference video data for use by video decoder 30 in decoding video data (e.g., in intra-frame or inter-frame predictive coding modes). Video data memory 79 and DPB 92 can be formed of any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, video data memory 79 and DPB 92 are depicted as... Figure 3 The video decoder 30 consists of two distinct components. However, it will be apparent to those skilled in the art that the video data memory 79 and the DPB 92 can be provided by the same memory device or separate memory devices. In some examples, the video data memory 79 may be on-chip along with other components of the video decoder 30, or off-chip relative to those components.

[0076] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks and associated syntax elements of encoded video frames. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantized coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors and other syntax elements to the prediction processing unit 81.

[0077] When a video frame is encoded as an intra-predictive coded (I) frame or used as an intra-coded prediction block in other types of frames, the intra-predictive processing unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-predictive mode notified by a signal and reference data from the previous decoded block of the current frame.

[0078] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks of the video block of the current video frame based on motion vectors and other syntax elements received from the entropy decoding unit 80. Each prediction block can be generated based on reference frames within one of the reference frame lists. The video decoder 30 can construct the reference frame lists, list 0 and list 1, using a default construction technique based on reference frames stored in the DPB 92.

[0079] In some examples, when a video block is encoded according to the intra-BC mode described herein, the intra-BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block can be located within the reconstructed region of the same picture as the current video block defined by the video encoder 20.

[0080] Motion compensation unit 82 and / or intra-frame BC unit 85 determine the prediction information of video blocks in the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded. For example, motion compensation unit 82 uses some received syntax elements to determine the prediction mode (e.g., intra-frame or inter-frame prediction) for video blocks used to encode the video frame, the inter-frame prediction frame type (e.g., B or P), the construction information of one or more reference frame lists of the frame, the motion vectors of each inter-frame prediction encoded video block of the frame, the inter-frame prediction state of each inter-frame prediction encoded video block of the frame, and other information for decoding video blocks in the current video frame.

[0081] Similarly, the intra-BC unit 85 can use some received syntax elements (e.g., flags) to determine which video block is predicted using the intra-BC mode, which video blocks of the frame have construction information in the reconstruction region and should be stored in the DPB 92, the block vector of each intra-BC predicted video block of the frame, the intra-BC prediction state of each intra-BC predicted video block of the frame, and other information for decoding video blocks in the current video frame.

[0082] The motion compensation unit 82 can also perform interpolation using the interpolation filter used by the video encoder 20 during the encoding of the video block to calculate the interpolated values ​​of the sub-integer pixels of the reference block. In this case, the motion compensation unit 82 can determine the interpolation filter used by the video encoder 20 based on the received syntax elements and use the interpolation filter to generate the prediction block.

[0083] The inverse quantization unit 86 uses the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to inverse quantize the quantization transform coefficients provided in the bitstream and the entropy decoded by the entropy decoding unit 80 to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform, such as inverse DCT, inverse integer transform, or a conceptually similar inverse transform process, to reconstruct the residual block in the pixel domain.

[0084] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by summing the residual block from the inverse transform processing unit 88 and the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter (not shown) may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. Loop filters, such as deblocking filters, sample adaptive offset (SAO), and adaptive loop filters (ALF), may be applied to the reconstructed CU before it is placed into the reference picture repository. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store the decoded video for later display on a display device (e.g., Figure 1 It is displayed on the display device 34).

[0085] In a typical video coding process, a video sequence usually consists of a set of ordered frames or pictures. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other cases, a frame may be monochrome and therefore include only a two-dimensional array of luma samples.

[0086] Like HEVC, the AVS3 standard is built on a block-based hybrid video codec framework. The input video signal is processed block by block (called a coding unit (CU)). Unlike HEVC, which is based solely on quadtree-based block partitioning, in AVS3, a coding tree unit (CTU) is split into CUs to accommodate different local characteristics based on quadtrees / binary trees / extended quadtrees. Furthermore, the concept of multiple partitioned unit types in HEVC is removed; that is, there is no separation of CUs, prediction units (PUs), and transform units (TUs) in AVS3. Instead, each CU is always used as the basic unit for prediction and transform, without further partitioning. In AVS3's tree partitioning structure, a CTU is first partitioned based on a quadtree structure. Then, each quadtree leaf node can be further partitioned based on binary tree and extended quadtree structures.

[0087] like Figure 4A As shown, the video encoder 20 (or more specifically, partitioning unit 45) generates a coded representation of a frame by first dividing the frame into a set of coding tree units (CTUs). A video frame may include an integer number of CTUs, which are arranged consecutively from left to right and from top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in the sequence parameter set, so that all CTUs in the video sequence have the same size, namely one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not necessarily limited to a specific size. Figure 4B As shown, each CTU may include a coding tree block (CTB) for luma samples, two corresponding coding tree blocks for chroma samples, and syntax elements for encoding the samples of the coding tree blocks. The syntax elements describe the attributes of different types of units within the coded pixel blocks and how the video sequence can be reconstructed at the video decoder 30, including inter-frame or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In monochrome images or images with three independent color planes, the CTU may include a single coding tree block and syntax elements for encoding the samples of the coding tree block. The coding tree block may be an NxN sample block.

[0088] To achieve better performance, the video encoder 20 can recursively perform tree splitting on the coding tree blocks of the CTU, such as binary tree splitting, ternary tree splitting, quadtree splitting, or a combination of both, and divide the CTU into smaller coding units (CUs). Figure 4C The depicted 64x64 CTU 400 is first divided into four smaller CUs, each with a block size of 32x32. Of these four smaller CUs, CU 410 and CU 420 are each further divided into four 16x16 CUs. The two 16x16 CUs, CU 430 and CU 440, are each further divided into four 8x8 CUs. Figure 4D The quadtree data structure is described, and it is illustrated as follows: Figure 4C The final result of the CTU 400 partitioning process depicted in the diagram is that each leaf node of the quadtree corresponds to a CU of a corresponding size ranging from 32x32 to 8x8. (As follows...) Figure 4B The depicted CTU (Color Unit) may include, for each CU (Color Unit), a coded block (CB) of the same size for the luminance samples of a frame and two corresponding coded blocks for the chrominance samples, as well as syntax elements for encoding the samples of the coded blocks. In monochrome images or images with three independent color planes, a CU may include a single coded block and a syntax structure for encoding the samples of the coded block. It should be noted that... Figure 4C and 4DThe quadtree partitioning depicted is for illustrative purposes only. A CTU can be split into CUs to accommodate different local characteristics based on quadtree / ternary / binary tree partitions. In multi-type tree structures, a CTU is partitioned by a quadtree structure, and each quadtree leaf CU can be further partitioned by binary and ternary tree structures. Figure 4E As shown, AVS3 has five split / segmentation types: quadruple split, horizontal binary split, vertical binary split, horizontal extended quadtree split, and vertical extended quadtree split.

[0089] In some implementations, the video encoder 20 may further segment the coded blocks of the CU into one or more MxN prediction blocks (PBs). A prediction block is a rectangular (square or non-square) sample block on which the same inter-frame or intra-frame prediction is applied. A prediction unit (PU) of the CU may include a luma sample prediction block, two corresponding chroma sample prediction blocks, and syntax elements for predicting the prediction blocks. In a monochrome image or an image with three independent color planes, a PU may include a single prediction block and a syntax structure for predicting the prediction blocks. The video encoder 20 may generate predictive luma, Cb, and Cr blocks for each PU of the CU.

[0090] Video encoder 20 can generate prediction blocks for the PU using intra-frame prediction or inter-frame prediction. If video encoder 20 uses intra-frame prediction to generate prediction blocks for the PU, then video encoder 20 can generate prediction blocks for the PU based on decoded samples of the frame associated with the PU. If video encoder 20 uses inter-frame prediction to generate prediction blocks for the PU, then video encoder 20 can generate prediction blocks for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0091] After the video encoder 20 generates predicted luminance, Cb, and Cr blocks for one or more PUs of the CU, the video encoder 20 can generate a luminance residual block for the CU by subtracting the predicted luminance block of the CU from its original luminance coding block, such that each sample in the luminance residual block of the CU indicates the difference between a luminance sample in one of the predicted luminance blocks of the CU and a corresponding sample in the original luminance coding block of the CU. Similarly, the video encoder 20 can generate Cb residual blocks and Cr residual blocks for the CU respectively, such that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predictive Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and the difference between each sample in the Cr residual block of the CU indicates the difference between a Cr sample in one of the predictive Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0092] In addition, such as Figure 4CAs shown, the video encoder 20 can use quadtree partitioning to decompose the luminance, Cb, and Cr residual blocks of the CU into one or more luminance, Cb, and Cr transform blocks. A transform block is a rectangular (square or non-square) sample block on which the same transform is applied. A transform unit (TU) of the CU may include a transform block for luminance samples, two corresponding transform blocks for chrominance samples, and syntax elements for transforming the transform block samples. Therefore, each TU of the CU may be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with a TU may be a sub-block of the CU's luminance residual block. A Cb transform block may be a sub-block of the CU's Cb residual block. A Cr transform block may be a sub-block of the CU's Cr residual block. In a monochrome image or an image with three independent color planes, a TU may include a single transform block and a syntax structure for transforming the samples of the transform block.

[0093] The video encoder 20 can apply one or more transforms to the luminance transform block of the TU to generate a luminance coefficient block of the TU. The coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be scalars. The video encoder 20 can apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block of the TU. The video encoder 20 can apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block of the TU.

[0094] After generating coefficient blocks (e.g., luminance coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks), video encoder 20 may quantize the coefficient blocks. Quantization generally refers to the process of quantizing transform coefficients to potentially reduce the amount of data used to represent the transform coefficients, providing further compression. After quantizing the coefficient blocks, video encoder 20 may entropy encode the syntax elements indicating the quantized transform coefficients. For example, video encoder 20 may perform context-adaptive binary arithmetic encoding / decoding (CABAC) on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 may output a bitstream comprising bit sequences that form representations of coded frames and associated data, which may be stored in storage device 32 or transmitted to target device 14.

[0095] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain syntax elements. The video decoder 30 can reconstruct frames of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally the reverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on the coefficient block associated with the TU of the current CU to reconstruct the residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coded blocks of the current CU by adding samples of the prediction block of the PU of the current CU to the corresponding samples of the transform block of the TU of the current CU. After reconstructing the coded blocks for each CU of the frame, the video decoder 30 can reconstruct the frame.

[0096] SAO is a process of modifying decoded samples by conditionally adding an offset value to each sample after applying a deblocking filter, based on values ​​in a lookup table transmitted by the encoder. SAO filtering is performed on a region-by-region basis, based on the filter type selected for each CTB by the syntax element `sao-type-idx`. A value of 0 for `sao-type-idx` indicates that the SAO filter is not applied to the CTB, while values ​​of 1 and 2 indicate the use of band offset and edge offset filtering types, respectively. In the band offset mode specified by `sao-type-idx` equal to 1, the selected offset value depends directly on the sample amplitude. In this mode, the entire sample amplitude range is uniformly divided into 32 segments called bands, and four sample values ​​belonging to these bands (which are consecutive within the 32 bands) are modified by adding a transmitted value represented as a band offset, which can be positive or negative. The main reason for using four consecutive bands is that in smooth regions where band artifacts may occur, the sample amplitude in the CTB tends to be concentrated in only a few bands. Furthermore, the design choice of using four offsets is consistent with the edge offset operation mode, which also uses four offset values. In the edge offset mode specified by sao-type-idx equal to 2, the syntax element sao-eo-class with a value from 0 to 3 indicates whether one of the horizontal, vertical, or two diagonal gradient directions is used for edge offset classification in CTB.

[0097] Figure 5This is a block diagram depicting four gradient patterns used in SAO according to some embodiments of the present disclosure. The four gradient patterns 502, 504, 506, and 508 are used for corresponding sao-eo-classes in the edge offset patterns. A sample labeled “p” indicates the center sample to be considered. Two samples labeled “n0” and “n1” specify two adjacent samples along (a) horizontal (sao-eo-class = 0), (b) vertical (sao-eo-class = 1), (c) 135° diagonal (sao-eo-class = 2), and (d) 45° (sao-eo-class = 3) gradient patterns. Each sample in the CTB is classified into one of five EdgeIdx categories by comparing the sample value p at a certain location with the values ​​n0 and n1 of the two adjacent samples, such as... Figure 5 As shown. This classification is based on the decoded sample values ​​for each sample, therefore EdgeIdx classification does not require additional signaling. Depending on the EdgeIdx category at the sample location, for EdgeIdx categories 1 to 4, an offset value from the transmitted lookup table is added to the sample value. The offset value is always positive for categories 1 and 2, and always negative for categories 3 and 4. Therefore, the filter typically has a smoothing effect in edge offset mode. Table 1 below illustrates the sample EdgeIdx categories in the SAO edge classes.

[0098]

[0099] Table 1: EdgeIdx categories of sample points in SAO edge classes.

[0100] For SAO types 1 and 2, a total of four amplitude offset values ​​are transmitted to the decoder of each CTB. For type 1, symbols are also encoded. The offset values ​​and associated syntax elements (e.g., sao-type-idx and sao-eo-class) are determined by the encoder—typically using a standard optimized for rate-distortion performance. A merge flag can be used to indicate that SAO parameters are inherited from the CTB to the left or above to make the signaling valid. In summary, SAO is a non-linear filtering operation that allows for additional refinement of the reconstructed signal and can enhance the signal representation around smooth regions and edges.

[0101] In some embodiments, the methods and systems disclosed herein improve encoding / decoding efficiency or reduce the complexity of Sample Adaptive Offset (SAO) by introducing cross-component information. SAO is used in the HEVC, VVC, AVS2, and AVS3 standards. Although existing SAO designs in the HEVC, VVC, AVS2, and AVS3 standards are used as the basic SAO approach in the following description, those skilled in the art of video coding will find that the cross-component approach described herein can also be applied to other loop filter designs or other encoding / decoding tools with similar design principles. For example, in the AVS3 standard, SAO is replaced by an encoding / decoding tool called Enhanced Sample Adaptive Offset (ESAO). However, the CCSAO disclosed herein can also be applied in parallel with ESAO. In another example, CCSAO can be applied in parallel with the Constrained Direction Enhancement Filter (CDEF) in the AV1 standard.

[0102] In existing SAO designs in HEVC, VVC, AVS2, and AVS3 standards, the luminance (Y), chrominance (Cb), and chrominance (Cr) sample offset values ​​are determined independently. That is, for example, the current chrominance sample offset is determined only by the current and adjacent chrominance sample values, without considering co-located or adjacent luminance samples. However, luminance samples retain more original image detail than chrominance samples, which can benefit the decision regarding the current chrominance sample offset. Furthermore, since chrominance samples typically lose high-frequency details after conversion from RGB color to YCbCr, or after quantization and deblocking filters, introducing luminance samples that preserve high-frequency details for chrominance offset decisions can benefit chrominance sample reconstruction. Therefore, further gains can be expected by exploring cross-component correlations, for example, through the use of cross-component sample adaptive offset (CCSAO) methods and systems.

[0103] Figure 6A This is a block diagram illustrating a system and process for CCSAO according to some embodiments of the present disclosure. Luminance samples after the luminance deblocking filter (DBF Y) are used to determine additional offsets for chrominance Cb and Cr after SAO Cb and SAO Cr. For example, the current chrominance sample 602 is first classified using co-occurrence 604 and adjacent (white) luminance samples 606, and the corresponding CCSAO offset value for the corresponding class is added to the current chrominance sample value.

[0104] In some embodiments, CCSAO can also be used in parallel with other codec tools, such as ESAO in the AVS standard or CDEF in the AV1 standard. Figure 6B This is a block diagram illustrating a system and process for CCSAO applied in parallel with ESAO in the AVS standard, according to some embodiments of this disclosure.

[0105] In some embodiments, the current chrominance sample classification re-uses the SAO type (EO or BO), class, and category of the co-located luma sample. The corresponding CCSAO offset can be signaled or derived from the decoder itself. For example, let h_Y be the co-located luma SAO offset, and h_Cb and h_Cr be the CCSAO Cb and Cr offsets respectively. h_Cb (or h_Cr) = w * h_Y, where w can be selected from a finite table. For example, +-1 / 4, +-1 / 2, 0, +-1, +-2, +-4... etc., where |w| only includes powers of 2.

[0106] In some embodiments, the comparison scores [-8, 8] of the co-located luma sample (Y0) and the adjacent 8 luma samples are used, which altogether results in 17 classes.

[0107] Initial class = 0

[0108] Loop through the adjacent 8 luma samples (Yi, i = 1 to 8)

[0109] If Y0 > Yi, class += 1

[0110] Otherwise if Y0 < Yi, class -= 1

[0111] In some embodiments, the above classification methods can be combined. For example, the comparison scores are combined with SAO BO (32-band classification) to increase diversity, resulting in a total of 17 * 32 classes. In some embodiments, Cb and Cr can use the same class to reduce complexity or save bits.

[0112] Figure 7 is a block diagram showing an example process of using CCSAO according to some embodiments of the present disclosure. Specifically, Figure 7 it shows that the input of CCSAO can introduce the inputs of vertical and horizontal DBF to simplify the determination of classes or increase flexibility. For example, let Y0_DBF_V, Y0_DBF_H, and Y0 be the co-located luma samples at the inputs of DBF_V, DBF_H, and SAO respectively. Yi_DBF_V, Yi_DBF_H, and Yi be the adjacent 8 luma samples at the inputs of DBF_V, DBF_H, and SAO respectively, where i = 1 to 8.

[0113] MaxY0 = max(Y0_DBF_V, Y0_DBF_H, Y0_DBF)

[0114] MaxYi = max(Yi_DBF_V, Yi_DBF_H, Yi_DBF)

[0115] And feed max Y0 and max Yi to the CCSAO classification.

[0116] Figure 8 This is a block diagram illustrating the CCSAO process according to some embodiments of the present disclosure being interwoven into vertical and horizontal DBF. In some embodiments, Figure 6, Figure 7 and Figure 8 The CCSAO block in the CCSAO can be selective. For example, Y0_DBF_V and Yi_DBF_V can be used for the first CCSAO_V, which applies the same sampling processing as in Figure 6, while using the input of the DBF_V luminance sample as the CCSAO input.

[0117] In some embodiments, the implemented CCSAO syntax is shown in Table 2 below.

[0118]

[0119] Table 2: CCSAO Syntax Examples

[0120] In some embodiments, to signal the CCSAO Cb and Cr offset values, if an additional chroma offset is signaled, the offsets of other chroma components can be derived by adding or subtracting a sign or by weighting to save bit overhead. For example, let h_Cb and h_Cr be the offsets of CCSAO Cb and Cr, respectively. Using explicit signaling w, where w = +-|w|, with a finite number of |w| candidates, h_Cr can be derived from h_Cb without explicitly signaling h_Cr itself.

[0121] h_Cr=w*h_Cb

[0122] Figure 9 This is a flowchart illustrating an exemplary process 900 of decoding a video signal using cross-component correlation according to some embodiments of the present disclosure.

[0123] The video decoder 30 receives a video signal (910) comprising a first component and a second component. In some embodiments, the first component is a luminance component, and the second component is a chrominance component of the video signal.

[0124] The video decoder 30 also receives multiple offsets (920) associated with the second component.

[0125] The video decoder 30 then uses the characteristic measurements of the first component to obtain the classification category (930) associated with the second component. For example, in Figure 6, the current chroma sample 602 is first classified using co-occurrence 604 and adjacent (white) luminance sample 606, and the corresponding CCSAO offset value is added to the current chroma sample.

[0126] The video decoder 30 further selects the first offset (940) from multiple offsets of the second component according to the classification category.

[0127] The video decoder 30 further modifies the second component (950) based on the selected first offset.

[0128] In some embodiments, using characteristic measurements of the first component to obtain a classification category associated with the second component (930) includes: using corresponding samples of the first component to obtain a corresponding classification category for each corresponding sample of the second component, wherein the corresponding sample of the first component is a corresponding isotope sample of the first component to each corresponding sample of the second component. For example, the current chromaticity sample classification is reusing the SAO type (EO or BO), class, and category of isotope luminance samples.

[0129] In some embodiments, obtaining a classification category (930) associated with a second component using characteristic measurements of the first component includes: obtaining a corresponding classification category for each corresponding sample of the second component using corresponding samples of the first component, wherein the corresponding samples of the first component are reconstructed before or after deblocking. In some embodiments, the first component is deblocked at a deblocking filter (DBF). In some embodiments, the first component is deblocked at a luminance deblocking filter (DBFY). For example, alternative to Figure 6 or Figure 7 The CCSAO input can also be placed before DBF Y.

[0130] In some embodiments, characteristic measurements are derived by dividing the range of sample values ​​of the first component into several segments and selecting segments based on the intensity values ​​of the sample points in the first component. In some embodiments, characteristic measurements are derived from segment offset (BO).

[0131] In some embodiments, characteristic measurements are derived based on the direction and intensity of edge information of samples in the first component. In some embodiments, the characteristic measurements are derived from edge offset (EO).

[0132] In some embodiments, modifying the second component (950) includes directly adding the selected first offset to the second component. For example, adding the corresponding CCSAO offset value to the current chromaticity component sample.

[0133] In some embodiments, modifying the second component (950) includes mapping the selected first offset to the second offset and adding the mapped second offset to the second component. For example, when signaling CCSAO Cb and Cr offset values, if an additional chroma offset is signaled, the other chroma component offset can be derived by using a plus or minus sign or weighting to save bit overhead.

[0134] In some embodiments, receiving a video signal (910) includes a receive syntax element indicating whether a method for decoding the video signal using CCSAO is enabled for the video signal in the Sequence Parameter Set (SPS). In some embodiments, cc_sao_enabled_flag indicates whether CCSAO is enabled at the sequence level.

[0135] In some embodiments, receiving a video signal (910) includes a receive syntax element indicating whether the method of decoding the video signal using CCSAO is enabled at the slice level for the second component. In some embodiments, slice_cc_sao_cb_flag or slice_cc_sao_cr_flag indicates whether CCSAO is enabled in the corresponding slice of Cb or Cr.

[0136] In some embodiments, receiving multiple offsets (920) associated with the second component includes receiving different offsets for different coding tree units (CTUs). In some embodiments, for a CTU, cc_sao_offset_sign_flag indicates the sign of the offset, and cc_sao_offset_abs indicates the CCSAO Cb and Cr offset values ​​of the current CTU.

[0137] In some embodiments, receiving multiple offsets (920) associated with the second component includes a receive syntax element indicating whether the received offset of the CTU is the same as the offset of one of the CTU's neighboring CTUs, where the neighboring CTU is either the left-adjacent CTU or the top-adjacent CTU. For example, cc_sao_merge_up_flag indicates whether the CCSAO offset is merged from the left CTU or the top CTU.

[0138] In some embodiments, the video signal further includes a third component, and the method of decoding the video signal using CCSAO further includes: receiving a second plurality of offsets associated with the third component; measuring a second classification category associated with the third component using characteristics of the first component; selecting a third offset for the third component from the second plurality of offsets according to the second classification category; and modifying the third component according to the selected third offset.

[0139] Figure 11 This is a block diagram of a sampling process according to some embodiments of the present disclosure, which shows that, in addition to luminance, other cross-component isotopes (1102) and adjacent (white) chromaticity samples are also fed into the CCSAO classification. Figure 6A , Figure 6B and Figure 11 The input for CCSAO classification is displayed. Figure 11In the current chromaticity sample point, the cross-component co-position chromaticity sample point is 1102, and the co-position luminance sample point is 1106.

[0140] In some embodiments, the classifier example (C0) uses the same luminance sample value (Y0) for classification. Let band_num be the number of equal segments in the luminance dynamic range, bit_depth be the sequence bit depth, and the class index of the current chrominance sample be:

[0141] Class(C0)=(Y0*band_num)>>bit_depth

[0142] In some embodiments, the classification takes into account rounding, for example:

[0143] Class(C0)=((Y0*band_num)+(1<<bit_depth))> >bit_depth

[0144] Table 3 below lists some examples of band_num and bit_depth. Table 3 shows three categories of examples, where the number of band segments is different for each category.

[0145]

[0146] Table 3: Example band_num and bit_depth for each class index.

[0147] In some embodiments, the classifier uses different brightness sample locations for C0 classification. Figure 10 This is a block diagram illustrating a classifier that performs C0 classification using different brightness sample point positions according to some embodiments of the present disclosure, for example, using adjacent Y7 instead of Y0 for C0 classification.

[0148] In some embodiments, different classifiers can be switched at the Sequence Parameter Set (SPS) / Adaptation Parameter Set (APS) / Image Parameter Set (PPS) / Image Header (PH) / Strip Header (SH) / Coding Tree Unit (CTU) / Coding Unit (CU) level.

[0149] For example, in Figure 10 In this table, POC0 uses Y0 and POC1 uses Y7, as shown in Table 4 below.

[0150] POC Classifier C0 band_num General Category 0 C0 uses position Y0 8 8 1 C0 uses position Y7 8 8

[0151] Table 4: Different classifiers applied to different images

[0152] In some embodiments, the C0 position and C0 band_num can be combined and switched at the SPS / APS / PPS / PH / SH / CTU / CU levels. Different combinations can be different classifiers, as shown in Table 5 below.

[0153] POC Classifier C0 band_num General Category 0 C0 uses position Y0 16 16 1 C0 uses position Y7 8 8

[0154] Table 5: Different classifier and band number combinations applied to different pictures

[0155] In some embodiments, the co-located luminance sample value (Y0) is replaced by a value (Yp) obtained by weighting the co-located and adjacent luminance samples. Figure 12 An exemplary classifier according to some embodiments of the present disclosure is shown, which replaces the co-located luminance sample value with a value obtained by weighting the co-located and adjacent luminance samples. The co-located luminance sample value (Y0) can be replaced by a phase correction value (Yp) obtained by weighting the adjacent luminance samples. Different YPs can be different classifiers.

[0156] In some embodiments, different Yps are applied to different chrominance formats. For example, in Figure 12 , the Yp of (a) is used for the 420 chrominance format, the Yp of (b) is used for the 422 chrominance format, and Y0 is used for the 444 chrominance format.

[0157] In some embodiments, another classifier (C1) is the comparison score [-8,8] of the co-located luminance sample (Y0) and the adjacent 8 luminance samples, which altogether produces 17 classes, as follows.

[0158] Initial Class (C1) = 0, loop on the adjacent 8 luminance samples (Yi, i = 1 to 8)

[0159] If Y0 > Yi class += 1

[0160] Otherwise if Y0 < Yi class -= 1

[0161] In some embodiments, the variant (C1') only calculates the comparison score [0,8], and this produces 8 classes. (C1, C1') is a classifier group, and the PH / SH level flag can be signaled to switch between C1 and C1'.

[0162] Initial Class (C1’) = 0, loop on the adjacent 8 luminance samples (Yi, i = 1 to 8)

[0163] If Y0 > Yi class += 1

[0164] In some embodiments, different classifiers are combined to produce a general classifier. For example, different classifiers are applied for different images (different POC values), as shown in Table 6 below.

[0165] POC Classifier C0 band_num General Category 0 Combining C0 and C1 16 16*17 1 Combinations C0 and C1' 16 16*9 2 Combining C0 and C1 7 7*17

[0166] Table 6: Different general classifiers applied to different images

[0167] In some embodiments, the classifier example (C2) uses the difference (Yn) between co-located and adjacent brightness samples. Figure 12 (c) shows an example of Yn with a dynamic range of [-1024, 1023] when the bit depth is 10. Let C2 band_num be the number of equal band segments in the dynamic range of Yn.

[0168] Class(C2) = (Yn + (1 <<bit_depth)*band_num)> >(bit_depth+1).

[0169] In some embodiments, C0 and C2 are combined to produce a general classifier. For example, different classifiers are applied for different images (different POCs), as shown in Table 7 below.

[0170] POC Classifier C0 band_num C2 band_num General Category 0 Combining C0 and C2 16 16 16*17 1 Combining C0 and C2 8 7 8*7

[0171] Table 7: Different general classifiers applied to different images

[0172] In some embodiments, all the above classifiers (C0, C1, C1', C2) are combined. For example, different classifiers are applied for different images (different POCs), as shown in Table 8 below.

[0173] POC Classifier C0 band_num C2 band_num General Category 0 Combinations C0, C1, and C2 4 4 4*17*4 1 Combinations C0, C1' and C2 6 4 6*9*4

[0174] Table 8: Different general classifiers applied to different images

[0175] In some embodiments, multiple classifiers are used in the same POC. The current frame is divided into several regions, each using the same classifier. For example, three different classifiers are used in POC0, and which classifier (0, 1, or 2) is used is signaled at the CTU level, as shown in Table 9 below.

[0176] POC Classifier C0 band_num area 0 C0 uses position Y0 16 0 0 C0 uses position Y0 8 1 0 C0 uses position Y1 8 2

[0177] Table 9: Different general classifiers applied to different regions of the same image

[0178] In some embodiments, the maximum band_num can be fixed or signaled at the SPS / APS / PPS / PH / SH / CTU / CU level. For example, the maximum band_num can be fixed at 16 in the decoder, and for each frame, 4 bits are signaled to indicate the C0 bandnum in the frame. Some other examples of maximum bandnum are listed in Table 10 below.

[0179] Band_num_min Band_num_max Band_num bit 1 1 0 1 2 1 1 4 2 1 8 3 1 16 4 1 32 5 1 64 6 1 128 7 1 256 8

[0180] Table 10: Examples of maximum band_num and band_num bits

[0181] In some embodiments, restrictions can be applied to the C0 classification, for example, restricting band_num to only powers of 2. Instead of explicitly signaling band_num, the syntax band_num_shift is signaled. The decoder can use shift operations to avoid multiplication.

[0182] Class(C0)=(Y0>>band_num_shift)>>bit_depth

[0183] Another example of this is considering rounding to reduce errors.

[0184] Class(C0)=((Y0+(1<<(band_num_shift-1)))>>band_num_shift)>>bit_depth

[0185] For example, if band_num_max is 16, then the possible band_num_shift candidates are 0, 1, 2, 3, 4, corresponding to bandnum = 1, 2, 4, 8, 16, as shown in Table 11.

[0186] POC Classifier C0 band_num_shift C0 band_num General Category 0 C0 uses position Y0 4 16 16 1 C0 uses position Y7 3 8 8

[0187] Band_num_max valid band_num Band_num_shift candidate 1 1 0 2 1,2 0,1 4 1,2,4 0,1,2 8 1,2,4,8 0,1,2,3 16 1,2,4,8,16 0,1,2,3,4 32 1,2,4,8,16,32 0,1,2,3,4,5 64 1,2,4,8,16,32,64 0,1,2,3,4,5,6 128 1,2,4,8,16,32,64,128 0,1,2,3,4,5,6,7 256 1,2,4,8,16,32,64,128,256 0,1,2,3,4,5,6,7,8

[0188] Table 11: Band_num and corresponding band_num_shift candidates

[0189] In some embodiments, the classifiers applied to Cb and Cr are different. The Cb and Cr offsets for all classes can be individually signaled. For example, different signaled offsets are applied to different chromaticity components, as shown in Table 12 below.

[0190] POC Quantity Classifier C0 General Category Offset notified by signal 0 Cb C0 16 16 16 0 Cr C0 5 5 5

[0191] Table 12: Cb and Cr offsets for all classes can be individually signaled.

[0192] In some embodiments, the maximum offset value is fixed or indicated by a signal in the Sequence Parameter Set (SPS) / Adaptation Parameter Set (APS) / Picture Parameter Set (PPS) / Picture Header (PH) / Strip Header (SH). For example, the maximum offset is between [-15, 15].

[0193] In some embodiments, offset signaling can use differential pulse code modulation (DPCM). For example, offset {3,3,2,1,-1} can be represented by a signal as {3,0,-1,-1,-2}.

[0194] In some embodiments, offsets may be stored in the APS or memory buffer for reuse in the next image / strip. The index can be signaled to indicate which stored previous frame offsets are used for the current image.

[0195] In some embodiments, the classifiers for Cb and Cr are the same. The Cb and Cr offsets for all classes can be jointly signaled, for example, as shown in Table 13 below.

[0196] POC Quantity Classifier C0 General Category Offset notified by signal 0 Cb and Cr C0 8 8 8

[0197] Table 13: Cb and Cr offsets for all classes can be signaled in conjunction.

[0198] In some embodiments, the classifiers for Cb and Cr can be the same. The Cb and Cr offsets for all classes can be jointly signaled, for example, with a sign difference, as shown in Table 14 below. According to Table 14, when the Cb offset is (3,3,2,-1), the derived Cr offset is (-3,-3,-2,1).

[0199]

[0200] Table 14: The Cb and Cr offsets for all classes can be signaled using a combination of sign flag differences.

[0201] In some embodiments, a symbolic flag can be signaled for each class. For example, as shown in Table 15 below. According to Table 15, when the Cb offset is (3,3,2,-1), the Cr offset derived from the corresponding symbolic flag is (-3,3,2,1).

[0202]

[0203] Table 15: Cb and Cr offsets for all classes can be jointly signaled using a symbolic flag for each class.

[0204] In some embodiments, the classifiers for Cb and Cr can be the same. The Cb and Cr offsets for all classes can be jointly signaled with weighted differences, for example, as shown in Table 16 below. The weights (w) can be selected from a limited table, such as ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4, etc., where |w| only includes powers of 2. According to Table 16, when the Cb offset is (3,3,2,-1), the Cr offset derived from the corresponding signed flag is (-6,-6,-4,2).

[0205]

[0206] Table 16: The Cb and Cr offsets for all classes can be signaled jointly using weight differences.

[0207] In some embodiments, the weight of each class can be signaled. For example, as shown in Table 17 below. According to Table 17, when the Cb offset is (3,3,2,-1), the Cr offset derived from the corresponding signed flag is (-6,12,0,-1).

[0208]

[0209]

[0210] Table 17: The Cb and Cr offsets for all classes can be jointly signaled using the weights signaled for each class.

[0211] In some embodiments, if multiple classifiers are used in the same POC, different offset sets are signaled individually or jointly.

[0212] In some embodiments, previously decoded offsets can be stored for use in future frames. The index can be signaled to indicate which set of previously decoded offsets is used for the current frame, reducing offset signaling overhead. For example, POC2 can reuse the POC0 offset and set the signaling offset to idx = 0, as shown in Table 18 below.

[0213]

[0214] Table 18: Signals can be used to notify the index which previously decoded offset set is used for the current frame.

[0215] In some embodiments, the reuse offset sets idx for Cb and Cr can be different, for example, as shown in Table 19 below.

[0216]

[0217]

[0218] Table 19: The index can be signaled to indicate which previously decoded offset set is used for the current frame, and the index can be different for the Cb and Cr components.

[0219] In some embodiments, the offset signaling can use additional syntax including start and length to reduce the signaling overhead. For example, when band_num = 256, only the offsets for band_idx = 37 to 44 are signaled. In the example of Table 20 below, both the start and length syntax are 8-bit fixed-length encodings and should match the band_num bits.

[0220]

[0221] Table 20: The offset signaling uses additional syntax including start and length

[0222] In some embodiments, if the sequence bit depth is higher than 10 (or a specific bit depth), the offset can be quantized before being signaled. On the decoder side, the decoded offset is de-quantized before being applied, as shown in Table 21 below. For example, for a 12-bit sequence, the decoded offset is left-shifted (de-quantized) by 2.

[0223]

[0224]

[0225] Table 21: The decoded offset is de-quantized before being applied

[0226] In some embodiments, the offset can be calculated as CcSaoOffsetVal = (1 - 2 * ccsao_offset_sign_flag) * (ccsao_offset_abs << (BitDepth - Min(10, BitDepth)))

[0227] In some embodiments, the sample processing is described as follows. Let R(x, y) be the input chroma sample value before CCSAO and R'(x, y) be the output chroma sample value after CCSAO:

[0228] offset = ccsao_offset[class_index of R(x, y)]

[0229] R'(x, y) = Clip3(0, (1 << bit_depth) - 1, R(x, y) + offset)

[0230] Based on the above equation, each chroma sample value R(x,y) is classified using a classifier indicated by the current image. The corresponding offset of the derived class index is added to each chroma sample value R(x,y). A clipping function Clip 3 is applied to (R(x,y) + offset) to make the output chroma sample value R'(x,y) within the bit depth dynamic range, for example, the range 0 to (1 < 0). <bit_depth)–1。

[0231] In some embodiments, boundary processing is described as follows. If any co-occurring and adjacent luminance samples used for classification are outside the current image, CCSAO will not be applied to the current chrominance sample. Figure 13A This is a block diagram illustrating, according to some embodiments of the present disclosure, that CCSAO is not applied to the current chromaticity sample if any of the co-occurring and adjacent luminance samples used for classification are outside the current image. For example, in Figure 13A In (a), if a classifier is used, CCSAO is not applied to the leftmost column of the chroma components of the current image. For example, if C1' is used, CCSAO is not applied to the leftmost column and the top row of the chroma components of the current image, such as... Figure 13A As shown in (b).

[0232] Figure 13B This is a block diagram illustrating, according to some embodiments of the present disclosure, the application of CCSAO to the current chromaticity sample if any of the co-occurring and adjacent luminance samples used for classification are outside the current image. In some embodiments, a variation is that if any of the co-occurring and adjacent luminance samples used for classification are outside the current image, the missing sample is reused, such as... Figure 13B As shown in (a), or by mirroring the missing samples to create samples for classification, as... Figure 13A As shown in (b), CCSAO can be applied to the current chromaticity sample point.

[0233] Figure 14 This is a block diagram illustrating, according to some embodiments of the present disclosure, that CCSAO is not applied to the current chroma sample if the corresponding selected co-position or adjacent luminance sample used for classification is outside the virtual space defined by the virtual boundary. In some embodiments, the virtual boundary (VB) is a virtual line separating spaces within an image frame. In some embodiments, if the virtual boundary (VB) is applied in the current frame, CCSAO is not applied to chroma samples at corresponding luminance positions that have already been selected outside the virtual space defined by the virtual boundary. Figure 14 An example of the virtual boundary of a C0 classifier with nine luminance location candidates is shown. For each CTU, CCSAO is not applied to chromaticity samples at the corresponding selected luminance location that are outside the virtual space enclosed by the virtual boundary. For example, in Figure 14In (a), when the selected Y7 luminance sample location is located on the other side of the horizontal virtual boundary, CCSAO is not applied to chrominance sample 1402, which is located 4 pixel rows from the bottom of the frame. For example, in Figure 14 In (b), when the selected Y5 luminance sample point is located on the other side of the vertical virtual boundary, CCSAO is not applied to chrominance sample point 1404, and the vertical virtual boundary is located y pixels from the right side of the frame.

[0234] In some embodiments, the implemented CCSAO syntax is shown in Table 22 below. In AVS3, the term patch is similar to stripe, and the patch header is similar to stripe header. FLC stands for fixed-length code. TU stands for truncated unary code. EGk stands for k-order exponential Golomb code, where k can be fixed.

[0235]

[0236]

[0237] Table 22: Exemplary CCSAO Syntax

[0238] If a higher-level flag is off, lower-level flags can be inferred from the off state of the flags without needing to be signaled. For example, if ph_cc_sao_cb_flag is false in this image, then ph_cc_sao_cb_band_num_minus1, ph_cc_sao_cb_luma_type, cc_sao_cb_offset_sign_flag, cc_sao_cb_offset_abs, ctb_cc_sao_cb_flag, cc_sao_cb_merge_left_flag, and cc_sao_cb_merge_up_flag do not exist and are inferred to be false.

[0239] In some embodiments, SPS ccsao_enabled_flag is conditional on the SPS SAO enable flag, as shown in Table 23 below.

[0240]

[0241] Table 23: SPS ccsao_enabled_flag is conditional on the SPS SAO enable flag.

[0242] In some embodiments, ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_control_flag indicate whether Cb / Cr CTB on / off control granularity is enabled. If ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_control_flag are enabled, ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag can be further signaled. Otherwise, whether to apply CCSAO in the current image depends on ph_cc_sao_cb_flag and ph_cc_sao_cr_flag, without further signaling to ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag at the CTB level.

[0243] In some embodiments, for ph_cc_sao_cb_type and ph_cc_sao_cr_type, a signal notification flag can be further used to distinguish whether a center-aligned brightness position is used. Figure 10 The Y0 position in the code is used for chromaticity sample classification to reduce bit overhead. Similarly, if cc_sao_cb_type and cc_sao_cr_type are signaled at the CTB level, the same mechanism can be used to further signal flags. For example, if the number of C0 luminance position candidates is 9, then cc_sao_cb_type0_flag is further signaled to distinguish whether a luminance position with the same center position is used, as shown in Table 24 below. If a luminance position with the same center position is not used, then ccsaocbtypeidc is used to indicate which of the remaining 8 adjacent luminance positions to use.

[0244]

[0245] Table 24: Signaling cc_sao_cb_type0_flag to distinguish whether to use the center-aligned brightness position.

[0246] Figure 15 This is a flowchart illustrating an exemplary process 1500 for decoding a video signal using cross-component correlation according to some embodiments of the present disclosure.

[0247] Video decoder 30 (e.g.) Figure 3 (As shown) Receives picture frames (1510) from the video signal, including the first component and the second component.

[0248] The video decoder 30 receives from the video signal a first syntax element (1520) indicating whether Cross-Component Sample Adaptive Offset (CCSAO) is enabled at the picture level for the second component. In some embodiments, the first syntax element is ph_cc_sao_cb_flag.

[0249] When the first syntax element indicates that CCSAO is enabled for the second component at the picture level, the video decoder 30 receives a second syntax element from the video signal, which indicates whether CCSAO is controlled for the second component at the coding tree block (CTB) level (1530). In some embodiments, the second syntax element is ph_cc_sao_cb_ctb_control_flag.

[0250] When the second syntax element indicates that CCSAO is controlled for the second component at the CTB level, the video decoder 30 receives a third syntax element from the video signal, which indicates whether CCSAO is enabled for the second component at the CTB level (1540). In some embodiments, the third syntax element is ctb_cc_sao_cb_flag.

[0251] When the third syntax element indicates that CCSAO is enabled for the second component at the CTB level, the video decoder 30 receives multiple sample offsets from the video signal in a mapping table associated with the second component. Each entry in the mapping table includes a class index and a corresponding sample offset (1550).

[0252] The video decoder 30 determines a classifier for the second component based on a set of one or more samples of the first component relative to each sample of the second component (1560).

[0253] The video decoder 30 selects the sample offset (1570) from the mapping table of the second component according to the classifier.

[0254] The video decoder 30 modifies the second component (1580) based on the sample offset.

[0255] In some embodiments, the mapping table is received from the video signal.

[0256] In some embodiments, the mapping table is predefined based on the bit depth of the video signal.

[0257] In some embodiments, when the first syntax element indicates that CCSAO is not enabled for the second component at the picture level, the video decoder 30 infers that the second syntax element is false.

[0258] In some embodiments, when the second syntax element indicates that CCSAO is not controlled for the second component at the CTB level, the video decoder 30 determines whether to enable CCSAO for the second component at the picture level based on the first syntax element.

[0259] In some embodiments, before receiving the first syntax element, the video decoder 30 receives a fourth syntax element, which indicates whether Sample Adaptive Offset (SAO) is enabled at the SPS level. In some embodiments, the fourth syntax element is sps_sao_enabled_flag.

[0260] In some embodiments, when the fourth syntax element indicates that SAO is enabled at the SPS level, the video decoder 30 receives a fifth syntax element indicating that CCSAO is enabled at the SPS level. In some embodiments, the fifth syntax element is sps_ccsao_enabled_flag.

[0261] In some embodiments, the video decoder 30 receives a sixth syntax element indicating whether the center co-location of the first component relative to each sample of the second component is used to determine the classifier. In some embodiments, the sixth syntax element is cc_sao_cb_type0_flag.

[0262] In some embodiments, when a sixth syntax element indicates that the center co-location of the first component relative to each sample of the second component is not used to determine the classifier, the video decoder 30 receives a seventh syntax element indicating that the adjacent sample locations of the first component relative to each sample of the second component are used to determine the classifier. In some embodiments, the seventh syntax element is cc_sao_cb_type_idc.

[0263] In some embodiments, receiving multiple sample offsets (1550) from a mapping table associated with the second component from a video signal includes: receiving multiple quantized sample offsets from a mapping table associated with the second component from a video signal; and dequantizing the multiple quantized sample offsets.

[0264] In some embodiments, the quantization level of a corresponding sample offset in a plurality of sample offsets is related to the bit depth of the video signal.

[0265] In some embodiments, when the bit depth of the video signal is greater than 10, the corresponding sample offsets in the multiple sample offsets are dequantized.

[0266] In some embodiments, a set of one or more samples of the first component relative to each sample of the second component lies within a virtual space defined by the virtual boundary of the picture frame.

[0267] In some embodiments, the first component is a luminance component and the second component is a chrominance component.

[0268] Further embodiments also include various subsets of the above embodiments that are combined or otherwise rearranged in various other embodiments.

[0269] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium such as a data storage medium, or a communication medium, including any medium that facilitates, for example, the transfer of a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures to implement the embodiments described in this application. A computer program product may include a computer-readable medium.

[0270] The terminology used in the description of the embodiments herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the claims. As used in the description of the embodiments and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It should also be understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of the stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or groups thereof.

[0271] It should also be understood that although the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first electrode may be referred to as a second electrode, and similarly, a second electrode may be referred to as a first electrode, without departing from the scope of the embodiments. Both the first electrode and the second electrode are electrodes, but they are not the same electrode.

[0272] Throughout this specification, references to the singular or plural forms of "an example," "example," "exemplary example," etc., mean that one or more specific features, structures, or characteristics described in connection with an example are included in at least one example of this disclosure. Therefore, phrases appearing throughout this specification in the singular or plural form such as "in an example," "in a sample," "in an exemplary example," etc., do not necessarily refer to the same example. Furthermore, specific features, structures, or characteristics in one or more examples may include combinations in any suitable manner.

[0273] The description in this application is presented for illustrative and descriptive purposes and is not intended to be exhaustive or limiting of the invention in the forms disclosed. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art who will benefit from the teachings presented in the foregoing description and the associated drawings. This embodiment was chosen and described in order to best explain the principles of the invention, its practical application, and to enable others skilled in the art to understand the various embodiments of the invention and to best utilize the basic principles and various embodiments with various modifications suitable for the particular intended use. Therefore, it should be understood that the scope of the claims is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of the appended claims.

Claims

1. A method for decoding a video signal, comprising: Receive image frames including a first component and a second component from the video signal; Determine whether to enable cross-component sample adaptive offset CCSAO for the second component; In response to determining that the CCSAO is enabled for the second component, a class index for the second component is determined based on a set of one or more samples of the first component relative to the samples of the second component, wherein the class index is an index corresponding to the band segment information and / or edge information of the component. Based on the class index, a sample offset for the second component is selected from multiple sample offsets; and The cross-component offset sample value of the second component is obtained based on the sample offset.

2. The method according to claim 1, wherein, Selecting a sample offset for the second component from multiple sample offsets based on the class index includes: The sample offset is selected from a mapping table for the second component according to the class index, wherein the mapping table includes an index and a corresponding sample offset associated with the second component.

3. The method according to claim 2, further comprising: Receive the corresponding sample offset from the mapping table from the video signal.

4. The method according to claim 2, wherein, The mapping table is predefined based on the bit depth of the video signal.

5. The method according to claim 1, wherein, Determining whether to enable CCSAO for the second component includes: A first syntax element is received from the video signal, the first syntax element indicating whether the CCSAO is enabled for the second component at the picture level; In response to the first syntax element indicating that the CCSAO is enabled for the second component at the picture level, a second syntax element is received from the video signal, the second syntax element indicating whether the CCSAO is controlled for the second component at the coding tree block (CTB) level; In response to the second syntax element indicating control of the CCSAO for the second component at the CTB level, a third syntax element is received from the video signal, the third syntax element indicating whether the CCSAO is enabled for the second component at the CTB level; and The third syntax element determines whether to enable the CCSAO for the second component at the CTB level.

6. The method according to claim 5, further comprising: In response to the first syntax element indicating that the CCSAO is not enabled for the second component at the image level, the second syntax element is inferred to be false.

7. The method according to claim 5, further comprising: In response to the second syntax element indicating that the CCSAO is not controlled for the second component at the CTB level, the first syntax element determines whether the CCSAO is enabled for the second component at the CTB level.

8. The method according to claim 1, wherein, Determining whether to enable CCSAO for the second component includes: Receive a fourth syntax element, which indicates whether to enable Sample Adaptive Offset (SAO) at the Sequence Parameter Set (SPS) level; Receive an indication of whether to enable the fifth syntax element of the CCSAO at the SPS level; and The fifth syntax element determines whether to enable the CCSAO at the SPS level.

9. The method according to claim 2, further comprising: A sixth syntax element is received, which indicates whether the center co-location of the first component relative to the sample point of the second component is used to determine the class index.

10. The method of claim 9, further comprising: In response to the sixth syntax element indicating that the center co-location of the first component relative to the sample point of the second component is not used to determine the class index, a seventh syntax element is received, the seventh syntax element indicating the adjacent sample point location of the first component relative to the sample point of the second component, the adjacent sample point location being used to determine the class index.

11. The method according to claim 2, further comprising: A second mapping table comprising multiple entries is received from the video signal, each entry including an index and a corresponding quantization sample offset associated with the second component; For each quantized sample offset in the second mapping table, the dequantized sample offset associated with the second component is obtained by dequantizing the quantized sample offset in the second mapping table. The second mapping table is updated by replacing each quantized sample offset in the quantized sample offset with the corresponding dequantized sample offset. as well as The updated second mapping table is determined as the mapping table.

12. The method according to claim 11, wherein, The quantization level of the quantization sample offset in the second mapping table is related to the bit depth of the video signal.

13. The method according to claim 11, wherein, When the bit depth of the video signal is greater than 10, the corresponding quantization sample offset in the second mapping table is dequantized.

14. The method according to claim 1, wherein, The set of one or more samples of the first component relative to the samples of the second component lies within a virtual space defined by the virtual boundary of the image frame.

15. The method according to claim 1, wherein, The first component is the luminance component and the second component is the chrominance component.

16. An electronic device comprising: One or more processing units; Memory coupled to the one or more processing units; as well as A plurality of programs stored in the memory, which, when executed by the one or more processing units, cause the electronic device to perform the method according to any one of claims 1-15.

17. A non-transitory computer-readable storage medium storing a plurality of programs for execution by an electronic device having one or more processing units, wherein, When the plurality of programs are executed by the one or more processing units, the electronic device performs the method according to any one of claims 1-15.

18. A computer program product comprising a plurality of instructions, which, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1-15.

Images